A prior knowledge-guided ultrasound imaging device for thyroid nodule detection

Abstract

This invention discloses a prior knowledge-guided ultrasound image thyroid nodule detection device. The method integrates prior knowledge and nodule distribution characteristics into a deep network, comprising two stages: The first stage, a multi-scale coarse detection module (ThyroidDetⅠ), designs a multi-scale region-based detection network to learn pyramid features to detect nodules at different feature scales. Region proposals are trained using prior knowledge about the actual nodule size and shape distribution for coarse nodule detection. The second stage, a multi-branch fine classification module (ThyroidDetⅡ), proposes a multi-branch fine classification network to integrate features for multi-view diagnosis. Each network branch captures and enhances a specific set of features commonly used by physicians for fine nodule classification. This invention effectively reduces the false negative rate for small nodules, lowers the false positive rate for challenging nodules, improves detection accuracy, and significantly reduces subjective judgment errors in the medical diagnostic process.

Description

Technical Field

[0001] This invention relates to the fields of deep learning and medicine, and more specifically, to a device for detecting thyroid nodules using ultrasound images guided by prior knowledge. Background Technology

[0002] Thyroid nodules are a common endocrine disorder, and they can be benign or malignant. If malignant nodules are not diagnosed and treated promptly, they may worsen and endanger the patient's life. Ultrasound examination is a crucial tool for assessing thyroid nodules, providing information on their boundaries, shape, and echogenicity, thus supporting further treatment. However, because physicians typically diagnose based on the ultrasound characteristics of the nodules, this process is relatively subjective and highly dependent on the physician's clinical experience, making misdiagnosis and missed diagnosis possible.

[0003] Deep learning models, especially convolutional neural networks (CNNs), have demonstrated superior performance compared to traditional learning methods in various visual recognition tasks. While existing deep learning methods are generally more powerful than traditional methods, they still have limitations in diagnostic performance. Similar to most traditional methods that rely on handcrafted features, existing deep learning methods tend to treat natural and medical images the same, directly using generic CNN models for thyroid nodule classification. This approach ignores important domain knowledge and expert experience relevant to specific medical diagnostic tasks, leading to operational problems.

[0004] To address this challenge, many computer-aided diagnostic (CAD) systems have been proposed for the automated objective classification of thyroid nodules. Typically, CAD systems comprise three basic components: nodule detection, feature extraction, and classifier construction. Traditional methods usually predefine the location of nodules for the physician, then extract handcrafted features (e.g., texture and morphological features) and construct a classifier. However, due to the variability in shape, size, and internal features of thyroid nodules, the low-level handcrafted features employed in traditional CAD methods have limited discriminative power due to their inherent simplicity and locality, leading to problems of missed and over-detected nodules. Summary of the Invention

[0005] To address the shortcomings of existing technologies, a thyroid nodule detection device based on prior knowledge using ultrasound imaging is proposed, comprising a multi-scale coarse detection module ThyroidDetⅠ and a multi-branch fine classification module ThyroidDetⅡ.

[0006] The ThyroidDetⅠ module, a multi-scale coarse nodule detection module, constructs a pyramid network to detect nodules at different scales. It incorporates a top-down module that integrates high-level semantic information with low-level feature maps for small nodule detection. Furthermore, to further improve detection performance, this invention references an important prior constraint—the true distribution of nodule size and shape—and designs reliable anchor points for each feature scale during proposal generation to better anchor the bounding boxes. Prior knowledge helps optimize proposal initialization and filtering, resulting in more accurate detection. Region proposals are trained using prior knowledge about the actual distribution of nodule size and shape for coarse nodule detection.

[0007] ThyroidDetⅡ, a multi-branch precision classification module for nodules, proposes a multi-branch precision classification network to integrate features for multi-view diagnosis. Each network branch captures and enhances a specific set of features commonly used by physicians to achieve precise classification of nodules. Specifically, in addition to a basic branch that extracts semantic features from the input patch, the multi-branch network integrates a contextual branch to extract enhanced contextual features closely related to malignant nodules. Furthermore, an aspect ratio preservation strategy is introduced to handle input image patches, which can preserve the aspect ratio of nodules during resizing for better classification performance.

[0008] The ultrasound image thyroid nodule detection device based on prior knowledge first performs coarse detection of nodules through the nodule multi-scale coarse detection module ThyroidDetⅠ, and then performs fine classification of nodules through the nodule multi-branch fine classification module ThyroidDetⅡ.

[0009] The specific implementation process of the nodule multi-scale coarse detection module ThyroidDetⅠ is as follows:

[0010] 1) ResNet-50 is used as the backbone of the detection network. It consists of one convolutional layer (i.e., Conv1 using 7×7 kernels) and four residual blocks (i.e., Conv2_x to Conv5_x), with 3, 4, 6, and 3 building blocks respectively. Each building block contains 3 convolutional layers, using 1×1, 3×3, and 1×1 kernels respectively. Each of these is followed by a max-pooling layer that downsamples the feature maps with a scaling factor of 2. Therefore, similar to FPN, the backbone here forms a bottom-up path that computes a pyramid feature hierarchy through progressively increasing abstraction, but with reduced spatial resolution. Conv2_x to Conv5_x have 3, 4, 6, and 3 building blocks respectively. Each building block contains 3 convolutional layers, using 1×1, 3×3, and 1×1 kernels respectively.

[0011] 2) To accurately detect targets of different sizes, this invention designs a top-down fusion module that hierarchically fuses high-level and low-level backbone features, and then uses the merged features at different scales to detect objects of different sizes. Specifically, from Conv2_x to Conv5_x, the top-down fusion module is injected between any two adjacent blocks. In each fusion module, a convolutional layer upsamples the high-level feature map by a factor of 2, and then concatenates it with the transferred feature map (via a 1×1 convolution) and the low-level feature map. The concatenated feature map is finally processed by a 3×3 convolutional layer to generate a fused feature map with reduced aliasing effects for detecting objects at that specific scale. In this way, the network proposed in this invention generates five region proposals of different scales (i.e., Conv2_x, Conv3_x, Conv4_x, Conv5_x, and Pool5), wherein the five feature map sizes are 1 / 4, 1 / 8, 1 / 16, 1 / 32, and 1 / 64 of the original input size, respectively, in width, height, and the corresponding feature map. These region proposals are finally fed into the R-CNN block to detect nodules of different sizes.

[0012] 3) Detection anchors limited by prior knowledge: The multi-scale detection network of this invention uses anchors with designed dimensions and aspect ratios (i.e., height divided by width) to predict region proposals for object detection. This invention pre-calculates the aspect ratio distribution of nodule sizes in the training dataset, as shown in the following results. Figure 4 As shown. From Figure 4 Two observations were made: (1) The aspect ratio range of all nodules in the training set was (0.2, 2.5), which means that the default aspect ratio range of the anchor in the experiment was also (0.2, 2.5); (2) The larger the nodule size, the smaller its corresponding aspect ratio, which indicates that the aspect ratio range defined at the bottom layer should be larger than that defined at the top layer. Therefore, in order to ensure that the size and shape of the proposed region are consistent with the actual nodule distribution, the anchor boxes of the lower feature maps are constrained to smaller sizes and the higher feature maps to have larger aspect ratios. In general, anchor boxes with one optional size (32, 64, 128, 256, 512) and three corresponding optional aspect ratios ((0.4, 1.0, 2.5), (0.4, 1.0, 2.5), (0.5, 1.0, 2.0), (0.7, 1.0, 1.5), (0.7, 1.0, 1.5)) are generated at each feature map location. Anchor box sizes and aspect ratios defined in different feature scale spaces can effectively cover different nodule sizes in the current image. Therefore, based on prior knowledge about aspect ratios and dimensions of thyroid nodules, region proposals can effectively improve the assistance of automatic nodule detection.

[0013] The specific implementation process of the nodule multi-branch fine classification module ThyroidDetⅡ is as follows:

[0014] The classification results produced by the ThyroidDetⅠ nodule multi-scale coarse detection module are relatively coarse, considering that proposal classification is only a supplementary task to aid the primary task of nodule detection in the first step. However, these classification results still provide valuable diagnostic suggestions. Samples misclassified by the ThyroidDetⅠ module may be challenging nodules that cannot be well distinguished from the relatively weak diagnostic features primarily learned for nodule detection. To further improve diagnostic performance, a more complex classification network is needed, which should incorporate the radiologist's perspective and attention (especially when diagnosing challenging nodules) to guide the learning of more discriminative features for automatic nodule identification. This invention proposes a multi-branch fine classification module, ThyroidDetⅡ, to learn features oriented towards multi-view classification for fine nodule classification.

[0015] Specifically, the multi-branch precision classification system ThyroidDet II employs two complementary branches to learn multi-view features of nodules from different regions based on the physician's attention. The structure of ThyroidDet II is as follows: Figure 5 As shown, the sub-network structures in all branches are the same, but the parameters differ. This invention refines the ZFnet architecture into a backbone of a multi-branch fine-tuning network. It retains 5 convolutional layers (i.e., Conv1 uses a 7×7 kernel, Conv2 uses a 5×5 kernel, and Conv3-Conv5 use 3×3 kernels), two fully connected layers with 4096 units each, and two additional fully connected layers with 1024 and 2 outputs respectively. After Conv1, Conv2, and Conv5, a max-pooling operation with a window size of 3×3 is applied.

[0016] The first branch takes the ROI of the original nodule as input. Since convolutional neural networks typically require standard-sized inputs, the region block containing the original ROI needs to be adjusted first. To preserve the original aspect ratio information of nodules with different shapes, an aspect ratio-preserving pruning strategy is used to adjust the original ROI. For each nodule of size h×w, it is first pruned and then filled with a square region block of size max(h,w). In this way, the nodule is not stretched in only one direction, and the aspect ratio information is preserved for nodule classification based on the convolutional neural network.

[0017] The second branch takes an image patch 1.5 times the size of the original nodule's ROI as input. Besides information within the nodule itself, features of surrounding tissue (such as diffuse sclerosis and external echo differences) also influence nodule classification. To account for this contextual information, an image patch centered on the original ROI and 1.5 times its size is cropped and used as input to the second branch; this is called the context branch.

[0018] Finally, the multi-view features from the two parallel branches (i.e., the output of FC3 in each refined ZFNet) are concatenated and fused by the FC layer (256 units) for automatic nodule classification by the softmax classifier.

[0019] The multi-task loss function for end-to-end region proposal and nodule detection is defined as follows:

[0020]

[0021] Among them, {p i} represents the classification score of the i-th anchor point, p i p is the probability that the current anchor point is classified as a foreground (target). i * It is a real tag; {t i} represents the predicted bounding box value at the i-th anchor point, t i These are the coordinates predicted by the network, t i * These are the coordinates of the actual target; L cls It is a classification loss, using the cross-entropy loss function; L reg It uses bounding box regression loss, specifically smooth L1 loss; N cls It is the regularization factor for classification loss; N reg λ is the regularization factor for the regression loss; λ is the weight that balances the two parts of the loss.

[0022] The beneficial effects of this invention are as follows:

[0023] This invention applies deep learning methods to combine the tasks of thyroid nodule detection and classification, while improving the network's nodule detection accuracy. It can detect thyroid nodules in ultrasound images in real time, obtain the benign and malignant classification results of thyroid nodules in patients, thereby providing a reliable basis for clinical diagnosis, reducing the misdiagnosis rate, improving diagnostic efficiency, and reducing unnecessary medical interventions for patients.

[0024] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Attached Figure Description

[0025] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the accompanying drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the accompanying drawings described below... Figure 1 This is one embodiment of the present invention. For those skilled in the art, other drawings can be obtained from these drawings without any creative effort.

[0026] Figure 1 This is a schematic diagram of a thyroid nodule detection device based on prior knowledge guided by ultrasound imaging, according to an embodiment of the present invention.

[0027] Figure 2 This is a schematic diagram of the network model structure of the present invention;

[0028] Figure 3 This is an example of the ThyroidDetⅠ multi-scale coarse detection module for nodules in this invention;

[0029] Figure 4 This is an example of the nodule size aspect ratio distribution in the dataset of this invention;

[0030] Figure 5 This is an example of the ThyroidDetⅡ nodule multi-branch fine classification module of the present invention;

[0031] Figure 6 This is a comparison of the qualitative detection results of the present invention with those of other methods. Detailed Implementation

[0032] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings. The described embodiments are some embodiments of the present invention, but not all embodiments.

[0033] Figure 1 This is a schematic diagram of a thyroid nodule detection device based on prior knowledge guided by ultrasound imaging, according to an embodiment of the present invention. Figure 1 As shown, the method includes the following two modules.

[0034] The ultrasound image thyroid nodule detection device based on prior knowledge first performs coarse detection of nodules through the nodule multi-scale coarse detection module ThyroidDetⅠ, and then performs fine classification of nodules through the nodule multi-branch fine classification module ThyroidDetⅡ.

[0035] The ThyroidDetⅠ module, a multi-scale coarse nodule detection module, constructs a pyramid network to detect nodules at different scales. It incorporates a top-down module that integrates high-level semantic information with low-level feature maps for small nodule detection. Furthermore, to improve detection performance, this invention references an important prior constraint—the true distribution of nodule size and shape—and designs reliable anchor points for each feature scale during proposal generation to better anchor the bounding boxes. Prior knowledge helps optimize proposal initialization and filtering, leading to more accurate detection. Region proposals are trained using prior knowledge about the actual distribution of nodule size and shape for coarse nodule detection.

[0036] The specific implementation process of the nodule multi-scale coarse detection module ThyroidDetⅠ is as follows:

[0037] 1) ResNet-50 is used as the backbone of the detection network. It consists of one convolutional layer (i.e., Conv1 using 7×7 kernels) and four residual blocks (i.e., Conv2_x to Conv5_x), with 3, 4, 6, and 3 building blocks respectively. Each building block contains 3 convolutional layers, using 1×1, 3×3, and 1×1 kernels respectively. Each of these is followed by a max-pooling layer that downsamples the feature maps with a scaling factor of 2. Therefore, similar to FPN, the backbone here forms a bottom-up path that computes a pyramid feature hierarchy through progressively increasing abstraction, but with reduced spatial resolution. Conv2_x to Conv5_x have 3, 4, 6, and 3 building blocks respectively. Each building block contains 3 convolutional layers, using 1×1, 3×3, and 1×1 kernels respectively.

[0038] 2) To accurately detect targets of different sizes, this invention designs a top-down fusion module that hierarchically fuses high-level and low-level backbone features, and then uses the merged features at different scales to detect objects of different sizes. Specifically, from Conv2_x to Conv5_x, the top-down fusion module is injected between any two adjacent blocks. In each fusion module, a convolutional layer upsamples the high-level feature map by a factor of 2, and then concatenates it with the transferred feature map (via a 1×1 convolution) and the low-level feature map. The concatenated feature map is finally processed by a 3×3 convolutional layer to generate a fused feature map with reduced aliasing effects for detecting objects at that specific scale. In this way, the network proposed in this invention generates five region proposals of different scales (i.e., Conv2_x, Conv3_x, Conv4_x, Conv5_x, and Pool5), wherein the five feature map sizes are 1 / 4, 1 / 8, 1 / 16, 1 / 32, and 1 / 64 of the original input size, respectively, in width, height, and the corresponding feature map. These region proposals are finally fed into the R-CNN block to detect nodules of different sizes.

[0039] 3) Detection anchors limited by prior knowledge: The multi-scale detection network of this invention uses anchors with designed dimensions and aspect ratios (i.e., height divided by width) to predict region proposals for object detection. This invention pre-calculates the aspect ratio distribution of nodule sizes in the training dataset, as shown in the following results. Figure 4 As shown. From Figure 4 Two observations were made: (1) The aspect ratio range of all nodules in the training set was (0.2, 2.5), which means that the default aspect ratio range of the anchor in the experiment was also (0.2, 2.5); (2) The larger the nodule size, the smaller its corresponding aspect ratio, which indicates that the aspect ratio range defined at the bottom layer should be larger than that defined at the top layer. Therefore, in order to ensure that the size and shape of the proposed region are consistent with the actual nodule distribution, the anchor boxes of the lower feature maps are constrained to smaller sizes and the higher feature maps to have larger aspect ratios. In general, anchor boxes with one optional size (32, 64, 128, 256, 512) and three corresponding optional aspect ratios ((0.4, 1.0, 2.5), (0.4, 1.0, 2.5), (0.5, 1.0, 2.0), (0.7, 1.0, 1.5), (0.7, 1.0, 1.5)) are generated at each feature map location. Anchor box sizes and aspect ratios defined in different feature scale spaces can effectively cover different nodule sizes in the current image. Therefore, based on prior knowledge about aspect ratios and dimensions of thyroid nodules, region proposals can effectively improve the assistance of automatic nodule detection.

[0040] ThyroidDetⅡ, a multi-branch precision classification module for nodules, proposes a multi-branch precision classification network to integrate features for multi-view diagnostics. Each network branch captures and enhances a specific set of features commonly used by physicians to achieve precise classification of nodules. Specifically, in addition to a basic branch that extracts semantic features from the input patch, the multi-branch network integrates a contextual branch to extract enhanced contextual features closely related to malignant nodules. Furthermore, an aspect ratio preservation strategy is introduced to handle input image patches, which preserves the aspect ratio of nodules during resizing for better classification performance.

[0041] The specific implementation process of the nodule multi-branch fine classification module ThyroidDetⅡ is as follows:

[0042] The classification results produced by the ThyroidDetⅠ nodule multi-scale coarse detection module are relatively coarse, considering that proposal classification is only a supplementary task to aid the primary task of nodule detection in the first step. However, these classification results still provide valuable diagnostic suggestions. Samples misclassified by the ThyroidDetⅠ module may be challenging nodules that cannot be well distinguished from the relatively weak diagnostic features primarily learned for nodule detection. To further improve diagnostic performance, a more complex classification network is needed, which should incorporate the radiologist's perspective and attention (especially when diagnosing challenging nodules) to guide the learning of more discriminative features for automatic nodule identification. This invention proposes a multi-branch fine classification module, ThyroidDetⅡ, to learn features oriented towards multi-view classification for fine nodule classification.

[0043] Specifically, the multi-branch precision classification system ThyroidDet II employs two complementary branches to learn multi-view features of nodules from different regions based on the physician's attention. The structure of ThyroidDet II is as follows: Figure 5 As shown, the sub-network structures in all branches are the same, but the parameters differ. This invention refines the ZFnet architecture into a backbone of a multi-branch fine-tuning network. It retains 5 convolutional layers (i.e., Conv1 uses a 7×7 kernel, Conv2 uses a 5×5 kernel, and Conv3-Conv5 use 3×3 kernels), two fully connected layers with 4096 units each, and two additional fully connected layers with 1024 and 2 outputs respectively. After Conv1, Conv2, and Conv5, a max-pooling operation with a window size of 3×3 is applied.

[0044] The first branch takes the ROI of the original nodule as input. Since convolutional neural networks typically require standard-sized inputs, the region block containing the original ROI needs to be adjusted first. To preserve the original aspect ratio information of nodules with different shapes, an aspect ratio-preserving pruning strategy is used to adjust the original ROI. For each nodule of size h×w, it is first pruned and then filled with a square region block of size max(h,w). In this way, the nodule is not stretched in only one direction, and the aspect ratio information is preserved for nodule classification based on the convolutional neural network.

[0045] The second branch takes an image patch 1.5 times the size of the original nodule's ROI as input. Besides information within the nodule itself, features of surrounding tissue (such as diffuse sclerosis and external echo differences) also influence nodule classification. To account for this contextual information, an image patch centered on the original ROI and 1.5 times its size is cropped and used as input to the second branch; this is called the context branch.

[0046] Finally, the multi-view features from the two parallel branches (i.e., the output of FC3 in each refined ZFNet) are concatenated and fused by the FC layer (256 units) for automatic nodule classification by the softmax classifier. Loss function: To train the detection network, the multi-task loss function for end-to-end region proposal and nodule detection is defined as follows:

[0047]

[0048] Among them, {p i} represents the classification score of the i-th anchor point, p i p is the probability that the current anchor point is classified as a foreground (target). i * It is a real tag; {t i} represents the predicted bounding box value at the i-th anchor point, t i These are the coordinates predicted by the network, t i * These are the coordinates of the actual target; L cls It is a classification loss, using the cross-entropy loss function; L reg It uses bounding box regression loss, specifically smooth L1 loss; N cls It is the regularization factor for classification loss; N reg λ is the regularization factor for the regression loss; λ is the weight that balances the two parts of the loss.

[0049] The quantitative detection results are shown in the table below.

[0050] Table 1. Quantitative comparison results of multi-scale detection networks for different nodule sizes (Note: The table headers represent AP for IoU = 0.50, 0.95, 0.50, 0.75, and area = small, medium, large, respectively).

[0051]

[0052] The ablation experimental results of the ThyroidDet II nodule multibranch classification module are shown in the table below.

[0053] Table 2 Ablation Experiment Results

[0054]

[0055] Finally, it should be noted that the above-described embodiments are merely specific implementations of the present invention, used to illustrate the technical solutions of the present invention, and not to limit it. The scope of protection of the present invention is not limited thereto. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that any person skilled in the art can still modify or easily conceive of changes to the technical solutions described in the foregoing embodiments within the technical scope disclosed in the present invention, or make equivalent substitutions for some of the technical features; and these modifications, changes, or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be covered within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A device for detecting thyroid nodules using ultrasound images guided by prior knowledge, characterized in that: It includes the ThyroidDetⅠ module for multi-scale coarse nodule detection and the ThyroidDetⅡ module for multi-branch fine nodule classification; wherein: The ThyroidDetⅠ nodule multi-scale coarse detection module uses ResNet-50 as the backbone of the detection network for coarse nodule detection. The ThyroidDetⅠ nodule multi-scale coarse detection module includes a first convolutional layer, a first residual block, a second residual block, a third residual block, a fourth residual block, a first fusion module, a second fusion module, a third fusion module, and a fourth fusion module. The first convolutional layer has a 7×7 kernel. The first, second, third, and fourth residual blocks contain 3, 4, 6, and 3 building blocks respectively. Each of these blocks is followed by a max-pooling layer with a scaling factor of 2. Each building block contains three second convolutional layers with kernels of 1×1, 3×3, and 1×1 respectively. Each of the first fusion module, the second fusion module, the third fusion module, and the fourth fusion module includes a first input terminal, a second input terminal, and an output terminal; the first input terminal is a 2x upsampling module, the second input terminal is a 1×1 convolutional layer, and the output terminal is a 3×3 convolutional layer; The first input terminal of the first fusion module is connected to the output terminal of the second fusion module, and the second input terminal of the first fusion module is connected to the first residual block; the first input terminal of the second fusion module is connected to the output terminal of the third fusion module, and the second input terminal of the second fusion module is connected to the second residual block; the first input terminal of the third fusion module is connected to the output terminal of the fourth fusion module, and the second input terminal of the third fusion module is connected to the third residual block; the first input terminal of the fourth fusion module is connected to the max pooling layer after the fourth residual block, and the second input terminal of the fourth fusion module is connected to the fourth residual block. The output of each of the first, second, third, and fourth fusion modules, as well as the max pooling layer after the fourth residual block, are all connected to the same R-CNN block; The first residual block, the second residual block, the third residual block, the fourth residual block, and the max pooling layer following the fourth residual block constitute five region proposals of different scales. The feature map sizes output by the region proposals are (32, 64, 128, 256, 512), and the aspect ratios of the anchor frames used in the feature maps are (0.4, 1.0, 2.5), (0.4, 1.0, 2.5), (0.5, 1.0, 2.0), (0.7, 1.0, 1.5), (0.7, 1.0, 1.5). The network backbone of the nodal multi-branch fine classification module ThyroidDetⅡ is refined from the ZFnet architecture, which includes a complementary first branch and a second branch. The first branch and the second branch have the same network structure, each containing 5 third convolutional layers, a first fully connected layer (FC1), a second fully connected layer (FC2), and a third fully connected layer (FC3). The first convolutional layer has a 7×7 kernel, the second convolutional layer has a 5×5 kernel, and the third to fifth convolutional layers have a 3×3 kernel. A max pooling layer with a window size of 3×3 is set after the first, second, and fifth convolutional layers. The first fully connected layer (FC1) and the second fully connected layer (FC2) each contain 4096 units, and the third fully connected layer (FC3) has 1024 outputs; The outputs of the third fully connected layer of the first branch and the third fully connected layer of the second branch are connected and fused in the fourth fully connected layer (FC4), which has 256 units and 2 outputs for use in a softmax classifier for automatic nodule classification. The input to the first branch is the ROI of the original nodule. The ROI of the original nodule is adjusted using an aspect ratio-preserving clipping strategy. For each nodule with a size of h×w, it is clipped and then filled with a square region block of size max(h,w). The input to the second branch is 1.5 times the size of the ROI of the original nodule.

2. The ultrasound image-based thyroid nodule detection device based on prior knowledge as described in claim 1, characterized in that, The multi-task loss function for region proposal and nodule detection is defined as follows: Among them, {p i } represents the classification score of the i-th anchor point, p i p is the probability that the current anchor point is classified as foreground. i * It is a real tag; {t i } represents the predicted bounding box value of the i-th anchor point, t i These are the coordinates predicted by the network, t i * These are the coordinates of the actual target; L cls It's a classification loss, using the cross-entropy loss function; L reg It uses bounding box regression loss, employing smoothed L1 loss; N cls It is the regularization factor for classification loss; N reg λ is the regularization factor for the regression loss; λ is the weight that balances the two parts of the loss.

Images