A thyroid nodule segmentation method based on prior attention and auxiliary guidance

Abstract

The application relates to a thyroid nodule segmentation method based on prior attention and auxiliary guidance, and belongs to the technical field of ultrasonic medical image processing. The method solves the problems of poor segmentation accuracy and low clinical application value of existing U-Net-based thyroid nodule ultrasonic image segmentation, and comprises the following steps: constructing a thyroid ultrasonic image dataset DS, dividing the thyroid ultrasonic image dataset DS into a training set, a verification set and a test set according to a proportion; using the thyroid nodule ultrasonic image I TN to perform feature extraction, performing minimum redundancy maximum relevance feature selection on the extracted features, and obtaining optimized benign and malignant prior features of the thyroid nodule; calculating an overall loss L, training a PADPU-Net network using the training set on the basis of the overall loss L, verifying the training set during the training process, and obtaining a PADPU-Net model M PADPU‑Net ; using the PADPU-Net model M PADPU‑Net to test the thyroid nodule ultrasonic image test set DS Test , optimizing the test result, and obtaining a final thyroid nodule segmentation result.

Description

TECHNICAL FIELD

[0001] The application relates to a thyroid nodule segmentation method based on prior attention and auxiliary guidance, and belongs to the technical field of ultrasonic medical image processing. BACKGROUND

[0002] The thyroid gland is the largest endocrine gland in the adult human body, and it plays an important role in regulating metabolism and growth and development. Among various thyroid abnormalities, thyroid nodules are a common disease with high incidence. Benign thyroid nodules can cause hormone-related diseases, while malignant nodules can seriously endanger health and even cause death. Although the detection rate of thyroid cancer has improved significantly with the advancement of medical technology, the mortality rate has remained stable or slightly increased. As a non-invasive, real-time imaging and cost-effective examination method, ultrasonic imaging has been widely used in the preliminary assessment of thyroid nodules. Thyroid nodule ultrasound images can provide information on thyroid size, shape, and echo characteristics, which is of great value to doctors in determining the nature of the nodules. Medical image analysis of ultrasound images has become one of the necessary procedures for clinical diagnosis of thyroid nodules, and the segmentation results can provide effective auxiliary information for detailed study of the lesion area, which can assist doctors in disease assessment and treatment plan development, etc. Therefore, an intelligent thyroid nodule segmentation method has great clinical application value.

[0003] With the development of deep learning technology, especially its wide application in the field of computer vision, deep learning has gradually replaced traditional manual segmentation methods, achieving more accurate target segmentation. As a classic convolutional neural network (CNN) structure, U-Net has been widely used in automatic segmentation tasks of CT, MRI and other medical images due to its outstanding performance in medical image segmentation. For example, U-Net has been successfully applied to automatic segmentation of organs in CT images, brain tumor segmentation in MRI images, etc. However, although U-Net has achieved remarkable results in the segmentation of other medical images, it still faces many challenges when applied to ultrasound images of thyroid nodules. Compared with X-ray, CT and MRI images, ultrasound images usually have lower resolution and poorer image quality, blurred boundaries and severe noise interference, which seriously affect the accuracy of intelligent segmentation.

[0004] Although the jump connections in U-Net fuse low-level and high-level features, improving the performance of the segmentation network, the single connection limits performance improvement, and the large differences in information hierarchy lead to unsatisfactory segmentation results. Furthermore, the excessively large span of channel dimensions in convolutional layers also has a negative impact. In the encoding pathway, a sudden increase in channel dimension may cause fine-grained information to be ignored by the initial layers. In the decoding pathway, a sudden decrease in channel dimension may also lead to the loss of important feature information. Moreover, in the decoding pathway, high-level features of different depths are not effectively utilized to assist in obtaining segmentation results. Finally, in implementing thyroid nodule segmentation methods, existing methods mainly focus on improving segmentation accuracy, neglecting the improvement of prior knowledge and the clinical application performance of the segmentation results. Summary of the Invention

[0005] To address the problems of poor accuracy and low clinical application value of existing U-Net-based thyroid nodule ultrasound image segmentation, this invention proposes a thyroid nodule segmentation method based on prior attention and assisted guidance.

[0006] The technical solution adopted by the present invention to solve the above problems is as follows: The steps of the present invention include:

[0007] Step 1: Construct the thyroid ultrasound image dataset DS, and divide the thyroid ultrasound image dataset DS into a training set DS according to the specified proportions. Train Validation set DS Val and test set DS Test ;

[0008] Step 2: Utilize thyroid nodule ultrasound images from the thyroid ultrasound image dataset DS TN Extract the gray-level co-occurrence matrix texture features F respectively GLCM Gray-level gradient co-occurrence matrix texture features F GLGCM and grayscale run-length matrix texture features F GLRLM The extracted features are subjected to minimum redundancy maximum relevance feature selection to obtain the optimized prior features F for benign and malignant thyroid nodules. Priori ;

[0009] Step 3: Calculate the main loss L main And auxiliary guidance loss L assist Based on the main loss L main And auxiliary guidance loss L assist Construct the overall loss L;

[0010] Step 4: Based on the overall loss L, use the training set DS Train Training the PADPU-Net network using DS during training. Val Validate and obtain the PADPU-Net model M. PADPU-Net ;

[0011] Step 5: using the PADPU-Net model M PADPU-Net , the thyroid nodule ultrasound image test set DS Test , the test results are optimized to obtain the final thyroid nodule segmentation results

[0012] Preferably, step 1 specifically includes:

[0013] Step 1.1: load all thyroid nodule ultrasound images I in grayscale mode TN = {I TN1 , I TN2 ,..., I TNn}, where is the nth thyroid nodule ultrasound image;

[0014] Step 1.2: for each thyroid nodule ultrasound image match the corresponding nodule segmentation label and benign and malignant classification label construct a thyroid nodule ultrasound image sample DS n ;

[0015] Step 1.3: use all thyroid nodule ultrasound image samples DS n to construct a thyroid nodule ultrasound image dataset DS, according to the sample division ratio of p Train : p Val : p Test , (p Train + p Val + p Test = 1) to divide the thyroid nodule ultrasound image dataset DS into training set DS Train , validation set DS Val and test set DS Test .

[0016] Preferably, step 2 specifically includes:

[0017] Step 2.1: based on the thyroid nodule ultrasound images I in the thyroid ultrasound image dataset DS TN extract the gray level co-occurrence matrix G1, based on the gray level co-occurrence matrix G1 of 4 different pixel pair directions, extract 14 kinds of texture features G1 1,θ -G1 14,θ , get the gray level co-occurrence matrix texture features wherein the 4 different pixel pair directions include 0°, 90°, 45°, 135°, and the 14 kinds of texture features G1 1,θ -G1 14,θincluding angular second moment, contrast, correlation, variance, inverse differential moment, and average, and variance, and entropy, entropy, difference variance, difference entropy, correlation measure 1, correlation measure 2 and homogeneity;

[0018] Step 2.2: based on the thyroid nodule ultrasound images I in the thyroid ultrasound image dataset DS TN extract the gray level gradient co-occurrence matrix G2, and extract 15 kinds of texture features G21-G2 based on the gray level gradient co-occurrence matrix G2 15 , to obtain the gray level gradient co-occurrence matrix texture features Among them, 15 kinds of texture features G21-G2 15 including small gradient dominance, large gradient dominance, gray level distribution non-uniformity, gradient distribution non-uniformity, energy, gray mean, gradient mean, gray mean square error, gradient mean square error, coherence, gray entropy, gradient entropy, mixed entropy, difference moment and inverse difference moment;

[0019] Step 2.3: based on the thyroid nodule ultrasound images I in the thyroid ultrasound image dataset DS TN extract the gray run-length matrix G3, and extract 6 kinds of texture features G3 based on the gray co-occurrence matrix G3 of 4 different angles 1,θ 6,θ , of the gray co-occurrence matrix texture features Among them, 6 kinds of texture features G3 1,θ 6,θ including short run-length factor, long run-length factor, gray level non-uniformity, length non-uniformity, run-length percentage and longest prominent run-length;

[0020] Step 2.4: based on the benign and malignant classification label O Class perform minimum redundancy maximum relevance feature selection on the extracted gray co-occurrence matrix texture features F GLCM , gray level gradient co-occurrence matrix texture features F GLGCM and gray run-length matrix texture features F GLRLM , to obtain the optimized thyroid nodule benign and malignant prior feature F Priori ;

[0021] The expression of the gray co-occurrence matrix G1 is:

[0022]

[0023] In formula (1), q is the gray level to which the thyroid nodule ultrasound image is mapped, d is the distance between pixel pairs, θ is the direction of the pixel pair, P(x1, x2, d) is the texture number in the thyroid nodule ultrasound image from the pixel with gray value x1 to the pixel with gray value x2 in the direction θ, and N is the total number of pixels in the thyroid ultrasound image;

[0024] ​​The expression for extracting the gray-level gradient co-occurrence matrix G2 is:

[0025]

[0026] In formula (2), P(x1,x2) is the number of pixels with a gray value of x1 and a gray gradient value of x2;

[0027] The expression for extracting the grayscale run-length matrix G3 is:

[0028]

[0029] In formula (3), r is the maximum number of times the gray value x1 appears consecutively in the θ direction, and G3(x1,x2,θ) is the number of times the gray value x1 appears consecutively x2 times in the θ direction;

[0030] The expression for minimum redundancy and maximum relevance feature selection is:

[0031]

[0032] In formula (4), G is the set of all texture features, g n and g m For any texture feature, For g n The benign or malignant classification labels are given, and p(·) is the probability density.

[0033] Preferably, in step 3, the main loss L main The calculation steps include:

[0034] Step 3.1.1: The PADPU-Net network processes the training set DS using the Multi-Residual Progressive Convolutional Module (MRP(m,C,c)). Train Extracting intermediate features F from each layer of the coding pathway L1 F L2 F L3 F L4 and F L5 Where m=0 is the encoding mode, m=1 is the encoding mode, c represents the number of channels to fill, and C represents the number of output channels after the double residual part;

[0035] Step 3.1.2: Based on intermediate features F L1 F L2 F L3 and F L4 Combining prior features F Priori Extracting prior features F P1 F P2 F P3 and F P4 ;

[0036] Step 3.1.3: Extracting self-optimized prior features by using channel-space attention mechanism and and sequentially extracting transposed convolution features F D6 , F D7 , F D8 and F D9 ;

[0037] Step 3.1.4: performing skip connection on each layer of the encoding path, and sequentially performing feature fusion on F L1 , F D9 and F L2 , F D8 and F L3 , F D7 and F L4 , F D6 and by using channel connection;

[0038] Step 3.1.5: after feature fusion, extracting intermediate features F L6 , F L7 , F L8 and F L9 in each layer of the decoding path of the training sample by using a multi-residual progressive convolution module MRP(m, C, c);

[0039] Step 3.1.6: based on F L9 , extracting thyroid nodule segmentation features F Seg , and calculating main loss L Seg by using thyroid nodule segmentation features F Seg and nodule segmentation label O main ;

[0040] The calculation formula of the intermediate features F L1 , F L2 , F L3 , F L4 and F L5 in each layer of the encoding path is as follows:

[0041]

[0042] In formula (5), MP(·) is a maximum pooling algorithm;

[0043] The calculation formula of the prior features F P1 , F P2 , F P3 and F P4 is as follows:

[0044]

[0045] In formula (6), Conv1(1) is a 1x1 convolution operation with an output channel of 1;

[0046] Intermediate features F L6 , F L7 , F L8 , and F L9 in each layer of the decoding path

[0047]

[0048] In formula (7), Concat(·) is channel concatenation.

[0049] The calculation formula of the main loss L main is as follows:

[0050] L main = -F Seg *log(Sig(O Seg ))-(1-F Seg )*log(1-Sig(O Seg ))(8).

[0051] In formula (8), Sig(·) is a Sigmoid operation.

[0052] Preferably, the calculation steps of the auxiliary guidance loss L assist and the overall loss L in step 3 include:

[0053] Step 3.2.1: Extract auxiliary guidance features F L5 , F L6 , F L7 , and F L8 using the auxiliary guidance path AG(H) based on the intermediate features F A1 , F A2 , F A3 , and F A4 , wherein H is the number of path input channels.

[0054] Step 3.2.2: Perform array sampling operations on the auxiliary guidance features F A1 , F A2 , F A3 , and F A4 . After the sampling operation, the auxiliary guidance features have the same size as the segmentation label O Seg , the mean of the auxiliary guidance features is calculated, and the nodule segmentation label O Seg is combined to calculate the auxiliary guidance loss L assist .

[0055] Step 3.2.3: Calculate the overall loss L based on the main loss L main and the auxiliary guidance loss L assistThe overall loss L is calculated;

[0056] The auxiliary guidance loss L assist The expression is:

[0057]

[0058] In formula (9), IP(·) represents an array sampling operation, and H represents an input channel number;

[0059] The calculation formula of the overall loss L is:

[0060] L=a*L main +b*L assist , a+b=1 (10) ;

[0061] In formula (10), a is a first constant, and b is a second constant.

[0062] Preferably, the step 5 specifically comprises:

[0063] Step 5.1: using the PADPU-Net model M PADPU-Net trained to test the thyroid nodule ultrasound image test set DS Test , to obtain a test result R Seg =M PADPU-Net (DS Test ) ;

[0064] Step 5.2: sequentially using an opening operation, a closing operation and a hole filling algorithm to optimize the test result R Seg =M PADPU-Net (DS Test ) to obtain a final thyroid nodule segmentation result R Seg =M PADPU-Net (DS Test ).

[0065] The present application has the advantages that:

[0066] 1. The present application is aimed at thyroid nodule ultrasound image segmentation, and a PADPU-Net network based on the U-shaped structure of the U-Net network is proposed, which obtains high-precision and high-clinical application value thyroid nodule segmentation results compared with the segmentation method based on the original U-Net. The PADPU-Net network introduces a prior feature extraction path on the U-Net jump connection, introduces the thyroid nodule texture features screened and containing the prior information of benign and malignant thyroid nodules into the network, and is used for extracting self-optimized prior features. In the present application, the deep prior features are self-optimized through the channel-space attention mechanism to improve the thyroid nodule segmentation performance and increase the explainability. At the same time, the PADPU-Net network replaces the continuous two convolution layers with a designed multi-residual progressive convolution module to avoid the influence of large dimension span between channels, and the multi-residual structure also improves the information correlation between features at high and low levels, realizing the overall improvement of the segmentation algorithm performance.

[0067] 2. In training the PADPU-Net network, an auxiliary guide path is proposed, which fully utilizes the effective intermediate layer features of each layer in the decoding path, extracts thyroid nodule auxiliary segmentation features, and improves the thyroid nodule segmentation performance. In the present application, the auxiliary guide path uses progressive convolution to process the intermediate layer features of each layer in the decoding path to obtain auxiliary guide features of each layer, and the auxiliary segmentation features are added after being sampled to the original size to obtain auxiliary segmentation features. The difference between the auxiliary segmentation features and the nodule segmentation label is calculated to obtain the auxiliary guide loss, and then the overall loss is formed with the traditional main loss, so that the obtained PADPU-Net model can provide high-performance thyroid nodule segmentation results. BRIEF DESCRIPTION OF DRAWINGS

[0068] Figure 1 A flowchart of a thyroid nodule segmentation method based on prior attention and auxiliary guide provided by the present application is provided.

[0069] Figure 2 The overall structure diagram of the PADPU-Net provided by the present application is provided.

[0070] Figure 3 The multi-residual progressive convolution structure diagram of the encoding and decoding path provided by the present application is provided.

[0071] Figure 4 The self-optimized prior attention jump connection structure diagram provided by the present application is provided.

[0072] Figure 5 The auxiliary guide path structure diagram provided by the present application is provided.

[0073] Figure 6 The thyroid nodule ultrasound image dataset sample diagram provided by the present application is provided.

[0074] Figure 7 A schematic diagram of the texture feature selection scoring results provided by the present invention;

[0075] Figure 8 This is a schematic diagram comparing the segmentation results of the present invention with those of the traditional U-Net. Detailed Implementation

[0076] Specific implementation method one: Combining Figures 1-5 This implementation method is described as follows: Figure 1 As shown, the specific steps of the thyroid nodule segmentation method based on prior attention and assisted guidance described in this embodiment include:

[0077] Step 1: Load the ultrasound image of the thyroid nodule I TN Match the corresponding nodule segmentation label O for the entire image. Seg and benign / malignant classification labels O Class A dataset of thyroid nodule ultrasound images, DS, was constructed. The dataset was divided into a training set DS and a training set DS. Train Validation set DS Val and test set DS Test The specific steps are as follows:

[0078] Step 11: Load all thyroid nodule ultrasound images in grayscale mode. TN n This represents the nth ultrasound image of a thyroid nodule;

[0079] Steps 1 and 2: For each ultrasound image of a thyroid nodule Match the corresponding nodule segmentation label Benign and malignant classification labels To construct an ultrasound image sample of a thyroid nodule (DS) n ;

[0080] Step 13: Construct a thyroid nodule ultrasound image dataset DS using all thyroid nodule ultrasound image samples, based on p Train :p Val :p Test ,(p Train +p Val +p Test The sample splitting ratio of 1) divides the dataset DS into the training set DS. Train Validation set DS Val and test set DS Test .

[0081] Step 2: Using ultrasound images of thyroid nodules I TN Extract the gray-level co-occurrence matrix texture features F respectively GLCM Gray-level gradient co-occurrence matrix texture features F GLGCMand gray level run length matrix texture feature F GLRLM , based on benign and malignant classification label O Class Perform minimum redundancy maximum relevance feature selection to obtain optimized thyroid nodule benign and malignant prior feature F Priori .

[0082] Step two: extracting gray level co-occurrence matrix G1 from the thyroid nodule ultrasound image I TN , based on four different pixel pair directions, 14 kinds of texture features G1 are extracted from the gray level co-occurrence matrix G1 1,θ -G1 14,θ , to obtain the gray level co-occurrence matrix texture feature Among them, the four different pixel pair directions include 0°, 90°, 45° and 135°, and the 14 kinds of texture features G1 1,θ -G1 14,θ include angular second moment, contrast, correlation, variance, inverse difference moment, and mean, and variance, and entropy, entropy, difference variance, difference entropy, correlation measure 1, correlation measure 2 and homogeneity;

[0083] The expression for extracting the gray level co-occurrence matrix G1 is:

[0084]

[0085] In formula (1), q is the gray level to which the thyroid nodule ultrasound image is mapped, d is the pixel pair distance, θ is the pixel pair direction, P(x1, x2, d) is the texture number in the thyroid nodule ultrasound image from the pixel with gray value x1 to the pixel with gray value x2 in the direction θ, and N is the total number of pixels in the thyroid ultrasound image;

[0086] Step two: extracting gray level co-occurrence matrix G1 from the thyroid nodule ultrasound image I TN , based on four different pixel pair directions, 14 kinds of texture features G1 are extracted from the gray level co-occurrence matrix G1 15 , to obtain the gray level co-occurrence matrix texture feature Among them, the four different pixel pair directions include 0°, 90°, 45° and 135°, and the 14 kinds of texture features G1 15 include small gradient dominance, large gradient dominance, gray level distribution non-uniformity, gradient distribution non-uniformity, energy, gray mean, gradient mean, gray mean square error, gradient mean square error, correlation, gray entropy, gradient entropy, mixed entropy, difference moment and inverse difference moment;

[0087] The expression for extracting the gray level co-occurrence matrix G1 is:

[0088]

[0089] In formula (2), P(x1, x2) is the number of pixels with a gray value of x1 and a gray gradient value of x2;

[0090] Step two three: based on the thyroid ultrasound image dataset DS in the thyroid nodule ultrasound image I TN Extract the gray run-length matrix G3, and extract six kinds of texture features G3 based on the gray co-occurrence matrix G3 of four different angles 1,θ -G3 6,θ , the gray co-occurrence matrix texture feature of Among them, six kinds of texture features G3 1,θ -G3 6,θ including short run factor, long run factor, gray unevenness, length unevenness, run percentage and longest prominent run;

[0091] The expression of extracting the gray run-length matrix G3 is:

[0092]

[0093] In formula (3), r is the maximum number of times that the gray value x1 appears continuously in the θ direction, and G3(x1, x2, θ) is the number of times that the gray value x1 appears x2 times continuously in the θ direction;

[0094] Step two four: based on benign and malignant classification label O Class Perform minimum redundancy maximum correlation feature selection on all extracted texture features F GLCM , F GLGCM and F GLRLM , and obtain the optimized benign and malignant prior features of thyroid nodules F Priori The selection rule is as follows:

[0095]

[0096] In formula (4), G is the set of all texture features, g n and g m are any kind of texture feature, O Classn is the benign and malignant classification label of g n , and p(·) is the probability density.

[0097] Step three: the PADPU-Net network first extracts the intermediate features F L1 , F L2 , F L3 , F L4 and F L5 in each layer of the encoding channel of the training sample, and then based on F L1 , F L2 , F L3 and F L4The prior feature F obtained according to step two Priori , each layer of self-optimized prior feature is extracted by using a channel-space attention mechanism and , and then transposed convolution features F D6 , F D7 , F D8 and F D9 are extracted in turn L6 , F L7 , F L8 and F L9 , and then intermediate features F L9 , F Seg , F Seg and F Seg are extracted in each layer of the decoding path after feature fusion main , and the main loss L L5 is calculated by using the thyroid nodule segmentation feature F L6 and the nodule segmentation label O L7 . Subsequently, auxiliary guide features F L8 , F A1 , F A2 and F A3 are extracted based on F A4 , F A1 , F A2 , F A3 and F A4 , and the auxiliary guide loss L Seg is calculated by using the guide segmentation feature F assist and the nodule segmentation label O main , and then the overall loss L is constituted by the main loss L Train to train the PADPU-Net network. The PADPU-Net network is trained by using the training set DS Val of the thyroid nodule ultrasound image, and DS PADPU-Net is used for verification during the training process, and finally the PADPU-Net model M L1 is obtained.

[0098] Step three one: the structure diagram of the PADPU-Net is shown in Figure 2 , which first extracts intermediate features F Figure 3 , F L2 , F L3 , F L4 and F L5 of each layer of the encoding path by using a multi-residual progressive convolution encoding mode as shown in L1 , and the multi-residual progressive convolution module is denoted as MRP(m, C, c), wherein m = 0 is the encoding mode, m = 1 is the encoding mode, c represents the number of channel filling, and C represents the output channel number of the double residual part, and then the intermediate features of each layer are calculated according to the following formula:

[0099]

[0100] In formula (5), MP(·) is a maximum pooling algorithm;

[0101] Step three two: based on F L1 , F L2 , F L3 and F L4 The prior feature F Priori obtained according to step two four is extracted, and the prior features F P1 , F P2 , F P3 and F P4 are calculated as follows:

[0102]

[0103] In formula (6), Conv1(1) is a 1x1 convolution operation with an output channel of 1;

[0104] Further, the prior feature channel-spatial attention module is realized to be free, and the self-optimized prior feature obtained is F and Then, after the skip connection, the feature fusion is performed, and the transposed convolution features F D6 , F D7 , F D8 and F D9 are taken, F L1 , F D9 and F L2 , F D8 and F L3 , F D7 and F L4 , F D6 and The channel connection is used for feature fusion, and the self-optimized prior attention skip connection structure of the PADPU-Net network involved in this step is shown in Figure 4 .

[0105] Step three: after the feature fusion, the multi-residual progressive convolution decoding mode shown in Figure 3 is used to extract the intermediate features F L6 , F L7 , F L8 and F L9 in each layer of the decoding path, and each layer of the intermediate feature is calculated as follows:

[0106]

[0107] In formula (7), Concat(·) is channel concatenation;

[0108] Further, based on F L9 extract the thyroid nodule segmentation feature F Seg , use the thyroid nodule segmentation feature F Seg and the nodule segmentation label O Seg to calculate the main loss L main , and the calculation formula is as follows:

[0109] L main = -F Seg *log(Sig(O Seg ))-(1-F Seg )*log(1-Sig(O Seg ))(8);

[0110] In formula (8), Sig(·) is Sigmoid operation.

[0111] Subsequently, based on F L5 , F L6 , F L7 and F L8 use the auxiliary guide path AG(H), H represents the number of input channels of the path to extract auxiliary guide features F A1 , F A2 , F A3 and F A4 , each auxiliary guide feature is subjected to array sampling operation, and after sampling, the auxiliary guide feature and the segmentation label O Seg have the same size, and then the mean value is calculated and compared with the nodule segmentation label O Seg to calculate the auxiliary guide loss L assist , and the calculation formula is as follows:

[0112]

[0113] In formula (9), IP(·) is array sampling operation, and H is the number of input channels;

[0114] The auxiliary guide path structure is shown in Figure 5 , and further, the overall loss L=a*L main +b*L assist , a+b=1,

[0115] Step three four: based on the overall loss, use the thyroid nodule ultrasound image training set DS Train to train the PADPU-Net network, and use DS Val for verification during the training process, and finally obtain the PADPU-Net model M PADPU-Net .

[0116] Step four: using the trained PADPU-Net model M PADPU-Net , a test set DS of thyroid nodule ultrasound images Test , and then optimizing the test result to obtain the final thyroid nodule segmentation result

[0117] Step four: using the trained PADPU-Net model M PADPU-Net , a test set DS of thyroid nodule ultrasound images Test , and then optimizing the test result to obtain the final thyroid nodule segmentation result R Seg = M PADPU-Net (DS Test )

[0118] Step four: using the trained PADPU-Net model M Seg , a test set DS of thyroid nodule ultrasound images PADPU-Net , and then optimizing the test result to obtain the final thyroid nodule segmentation result R Test .

[0119] Specific implementation method two: combined with Figures 6-8 In this implementation, the verification data is derived from thyroid medical screening, which is a real thyroid nodule section image. The thyroid segmentation label and the benign and malignant classification label are drawn by professional physicians.

[0120] Step one: load all 1154 thyroid nodule ultrasound images, match the corresponding nodule segmentation label and benign and malignant classification label for the thyroid nodule ultrasound images, construct 1154 thyroid nodule ultrasound image samples, and use all thyroid nodule ultrasound image samples to construct a thyroid nodule ultrasound image dataset. The sample of the thyroid nodule ultrasound image dataset is shown in Figure 6 According to the sample division ratio of about 0.6:0.2:0.2, the dataset is divided into a training set consisting of 714 samples, a verification set consisting of 220 samples, and a test set consisting of 220 samples.

[0121] Perform step two: for thyroid nodule ultrasound image with q = 16, d = 5 and θ = 0°, 45°, 90°, 135°, extract the gray level co-occurrence matrix angle second moment, contrast, coherence, variance, inverse difference moment, and average, and variance, and entropy, entropy, difference variance, difference entropy, correlation measure 1, correlation measure 2 and homogeneity texture features, then a total of 14 x 4 = 56 gray level co-occurrence matrix texture features can be extracted, and then for the thyroid nodule ultrasound image with q = 16, extract small gradient advantage, large gradient advantage, gray level distribution non-uniformity, gradient distribution non-uniformity, energy, gray mean, gradient mean, gray mean square error, gradient mean square error, coherence, gray entropy, gradient entropy, mixed entropy, difference moment and inverse difference moment texture features, then a total of 15 gray level gradient co-occurrence matrix texture features can be extracted, then for the thyroid nodule ultrasound image with q = 16 and θ = 0°, 45°, 90°, 135°, extract short run factor, long run factor, gray level non-uniformity, length non-uniformity, run percentage and longest prominent run gray level co-occurrence matrix texture features, then a total of 6 x 4 = 24 gray level co-occurrence matrix texture features can be extracted, then 56 + 15 + 24 = 95 thyroid nodule ultrasound image texture features are obtained.

[0122] Then based on the benign and malignant classification label of the thyroid nodule ultrasound image (benign is 0 and malignant is 1), the thyroid nodule ultrasound image texture features are screened by using the minimum redundancy maximum correlation feature selection, and 13 effective features containing benign and malignant classification prior information are reserved by selecting a score of 0.0005, and the texture feature selection score result is as shown in Figure 7 The feature 1-feature 13 reserved in the order of score from large to small are as follows: gray level gradient co-occurrence matrix gradient distribution non-uniformity, gray level gradient co-occurrence matrix inverse difference moment, gray run matrix 0°-short run factor, gray run matrix 45°-longest prominent run, gray level gradient co-occurrence matrix length distribution non-uniformity, gray co-occurrence matrix 45°-difference variance, gray run matrix 135°-longest prominent run, gray level gradient co-occurrence matrix small gradient advantage, gray run matrix 90°-short run factor, gray level gradient co-occurrence matrix gradient entropy, gray run matrix 90°-longest prominent run, gray co-occurrence matrix 90°-homogeneity, and gray level gradient co-occurrence matrix difference moment.

[0123] Perform step three: first, extract the intermediate features in each layer of the encoding path of the training sample by using the multi-residual progressive convolutional coding mode, and the multi-residual progressive modules used are MRP(0, 63, 1), MRP(0, 126, 2), MRP(0, 252, 4), MRP(0, 504, 8) and MRP(0, 1008, 16) in turn, and the intermediate feature channel numbers are 64, 128, 256, 512 and 1024 in turn.

[0124] Then the prior features are extracted based on the gradient distribution non-uniformity of the gray level gradient co-occurrence matrix, the gray level gradient co-occurrence matrix inverse difference moment, the 0°-short run length factor of the gray level run length matrix, the 45°-longest prominent run of the gray level run length matrix, the length distribution non-uniformity of the gray level gradient co-occurrence matrix, the 45°-difference variance of the gray level co-occurrence matrix, the 135°-longest prominent run of the gray level run length matrix, the small gradient advantage of the gray level gradient co-occurrence matrix, the 90°-short run length factor of the gray level run length matrix, the gradient entropy of the gray level gradient co-occurrence matrix, the 90°-longest prominent run of the gray level run length matrix, the 90°-homogeneity of the gray level co-occurrence matrix, and the difference moment of the gray level gradient co-occurrence matrix in the first four layers of the intermediate features based on the gray level gradient, and the deep prior features are sent to the prior feature channel-space attention module to obtain four layers of self-optimized prior features in turn, and the number of feature channels is 13, and then the feature fusion is performed after the skip connection, and the number of output feature channels after the feature fusion is 64+63+13=140, 128+126+13=267, 256+252+13=521, and 512+504+13=1029 in turn

[0125] Then the prior features are extracted based on the gradient distribution non-uniformity of the gray level gradient co-occurrence matrix, the gray level gradient co-occurrence matrix inverse difference moment, the 0°-short run length factor of the gray level run length matrix, the 45°-longest prominent run of the gray level run length matrix, the length distribution non-uniformity of the gray level gradient co-occurrence matrix, the 45°-difference variance of the gray level co-occurrence matrix, the 135°-longest prominent run of the gray level run length matrix, the small gradient advantage of the gray level gradient co-occurrence matrix, the 90°-short run length factor of the gray level run length matrix, the gradient entropy of the gray level gradient co-occurrence matrix, the 90°-longest prominent run of the gray level run length matrix, the 90°-homogeneity of the gray level co-occurrence matrix, and the difference moment of the gray level gradient co-occurrence matrix in the first four layers of the intermediate features based on the gray level gradient, and the deep prior features are sent to the prior feature channel-space attention module to obtain four layers of self-optimized prior features in turn, and the number of feature channels is 13, and then the feature fusion is performed after the skip connection, and the number of output feature channels after the feature fusion is 64+63+13=140, 128+126+13=267, 256+252+13=521, and 512+504+13=1029 in turn main +0.25*L assist to form the overall loss.

[0126] Then, the PADPU-Net network is trained by using a training set composed of 714 thyroid nodule ultrasound image samples, and in the training process, a validation set composed of 220 thyroid nodule ultrasound image samples is used for verification, and finally the PADPU-Net model M PADPU-Net is obtained.

[0127] Step four is performed: the PADPU-Net model M PADPU-Net is used to test a test set composed of 220 thyroid nodule ultrasound image samples, and a test result is obtained. Then the test result is optimized by using the open operation, the closed operation, and the hole filling algorithm in turn, and an improved thyroid nodule segmentation result is obtained. The comparison examples of the obtained thyroid nodule improved segmentation result are as follows Figure 8As shown, the comparison diagrams of the traditional segmentation result, the test segmentation result and the improved segmentation result and the thyroid nodule segmentation label are respectively given, and it can be seen from the diagrams that the segmentation result (improved) of the present application has more accurate segmentation result compared with the segmentation result (traditional) based on U-Net. Further, the precision improvement of the present application is compared by F1 score, intersection over union and Matthew correlation coefficient, and the clinical application value improvement of the present application is compared by aspect ratio, compactness, eccentricity and concave-convex degree, and the comparison evaluation results are shown in Table 1. The test results are slightly lower in aspect ratio, and other ratings are higher than the traditional algorithm, and the improved results of the present application not only greatly improve the thyroid nodule segmentation accuracy, but also greatly improve the clinical application value.

[0128] Table 1

[0129]

[0130] The above is only a preferred embodiment of the present application, and does not limit the present application in any form. Although the present application has been disclosed as above with a preferred embodiment, it is not intended to limit the present application. Any skilled person in the art can make some changes or modifications to the above disclosed technical content without departing from the technical solution of the present application, and make equivalent embodiments with equivalent changes, as long as they do not depart from the technical solution of the present application, and are within the spirit and principles of the present application. Any simple modification, equivalent replacement and improvement of the above embodiments, as long as they do not depart from the technical solution of the present application, are within the protection scope of the present application.

Claims

1. A method for thyroid nodule segmentation based on prior attention and auxiliary guidance, characterized in that, The steps of the thyroid nodule segmentation method based on prior attention and auxiliary guidance include: Step 1: Constructing a thyroid ultrasound image dataset A thyroid ultrasound image dataset is constructed , divided into training, validation, and test sets in a ratio of 6:2:2 ; Step 2: Utilizing the thyroid ultrasound image dataset of thyroid nodules in the thyroid ultrasound image dataset Extract the gray level co-occurrence matrix texture feature , the gray level gradient co-occurrence matrix texture feature and the gray run length matrix texture feature respectively, perform minimum redundancy maximum relevance feature selection on the extracted features to obtain optimized thyroid nodule benign and malignant prior features ; Step 3: Compute main loss and auxiliary guidance loss based on main loss and auxiliary guidance loss build overall loss L ; Step 3 main loss The calculation steps of the step 3 main loss include: Step 3.1.1: The PADPU-Net network passes through a multi-residual progressive convolution module For the training set Extracting the intermediate features of each layer in the encoding channel 、 、 、 and , where is the encoding mode, is the decoding mode, represents the number of channel padding, represents the number of output channels through the double residual part; Step 3.1.2: extracting prior features based on intermediate features , , and , in combination with the prior features , extracting prior features , , and ; Step 3.1.3: Extracting self-optimized prior features by using channel-space attention mechanism 、 、 and , and extracting transpose convolution features in turn 、 、 and ; Step 3.1.4: The encoding paths of each layer are connected in a skip connection, and after the skip connection, the , and ; , and ; , and ; , and feature fusion is performed using channel connection; Step 3.1.5: Feature fusion after multi-residual progressive convolution module Extracting intermediate features in each layer of the decoding path from the training sample 、 、 and ; Step 3.1.6: based on extracting thyroid nodule segmentation features , using the thyroid nodule segmentation features and the nodule segmentation label calculate the main loss ; Intermediate features of layers in an encoding pass , , , and The formula for calculating is: （5）； In equation (5), is the max pooling algorithm; prior features , , and the calculation formula is: （6）； In equation (6), For output channel 1 Convolution operation; Intermediate features in layers of a decoding pass , , and The formula for calculating the intermediate features in layers of a decoding pass is: （7）； In equation (7), is a channel connection; Main loss The formula for calculating the main loss is: （8）； In equation (8), is a sigmoid operation; The calculation steps of the auxiliary guidance loss in step 3 and the overall loss L include: Step 3.2.1: Intermediate feature based guidance , , and Utilizing auxiliary guidance pathways Extracting auxiliary guidance features , , and where, is the number of input channels for the pathway. Step 3.2.2: Array sampling operation is performed on the auxiliary guidance feature , , and the segmentation label , the mean value of the auxiliary guidance feature is calculated and combined with the nodule segmentation label to calculate the auxiliary guidance loss ; Step 3.2.3: computing the overall loss based on the main loss and the auxiliary guidance loss L ;​ auxiliary guidance loss The expression for the auxiliary guidance loss is: （9）； In equation (9), represents an array sampling operation, represents the number of input channels; Overall loss L The formula for calculating the overall loss is: （10）； In Equation (10), is a first constant, is a second constant; Step 4: On the basis of the overall loss L , the PADPU-Net network is trained using the training set , and the training process uses for verification to obtain the PADPU-Net model ; Step 5: using the PADPU-Net model the test set of the thyroid nodule ultrasound images Testing is performed, and the test results are optimized to obtain the final thyroid nodule segmentation results .

2. The thyroid nodule segmentation method based on prior attention and auxiliary guidance according to claim 1, characterized in that, Step 1 specifically includes: Step 1.1: Load all thyroid nodule ultrasound images in grayscale mode wherein is the th ultrasound image of the set of thyroid nodule ultrasound images; Step 1.2: For each ultrasound image of thyroid nodules Matching the corresponding nodules segmentation labels And benign and malignant classification labels , a sample of ultrasound images of thyroid nodules is constructed ; Step 1.3: Utilize all thyroid nodule ultrasound image samples Constructing a dataset of ultrasound images of thyroid nodules ,according to The sample division ratio of the thyroid nodule ultrasound image dataset Divided into training set Validation set and test set .

3. The thyroid nodule segmentation method based on prior attention and auxiliary guidance according to claim 1, characterized in that, Step 2 specifically includes: Step 2.1: Based on the thyroid ultrasound image dataset Ultrasound images of thyroid nodules Extracting the gray-level co-occurrence matrix Gray-level co-occurrence matrix based on 4 different pixel pair directions Extract 14 texture features respectively - The gray-level co-occurrence matrix texture features are obtained. Among them, the four different pixel pairs include the direction. 14 texture features - Including second moment of angle, contrast, coherence, variance, inverse partial moment, sum mean, sum variance, sum entropy, entropy, difference variance, difference entropy, correlation measure 1, correlation measure 2 and homogeneity; Step 2.2: thyroid nodule ultrasound images based on the thyroid ultrasound image dataset Step 2.3: extracting a gray level gradient co-occurrence matrix Step 2.4: extracting 15 kinds of texture features based on the gray level gradient co-occurrence matrix Step 2.5: obtaining gray level gradient co-occurrence matrix texture features Step 2.6: respectively extracting 15 kinds of texture features - Step 2.7: obtaining gray level gradient co-occurrence matrix texture features Step 2.8: the 15 kinds of texture features - include small gradient dominance, large gradient dominance, gray level distribution uniformity, gradient distribution uniformity, energy, gray mean, gradient mean, gray level mean square error, gradient mean square error, coherence, gray level entropy, gradient entropy, mixed entropy, difference matrix, and inverse difference matrix. Step 2.3: Thyroid nodule ultrasound images based on the thyroid ultrasound image dataset Step 2.3: Thyroid nodule ultrasound images based on the thyroid ultrasound image dataset Extracting gray run-length matrix , based on the gray run-length matrix of 4 different angles Extracting 6 kinds of texture features respectively - Gray run-length matrix texture features of , wherein the 4 different angles include 6 kinds of texture features - including short run length factor, long run length factor, gray unevenness, length unevenness, run percentage and longest prominent run. Step 2.4: benign and malignant classification label based on the extracted gray level co-occurrence matrix texture features , gray gradient co-occurrence matrix texture features and gray run length matrix texture features , to obtain the optimized benign and malignant prior features of thyroid nodules ; extracting a gray level co-occurrence matrix The expression is: （1）； In formula (1), This represents the grayscale level to which the ultrasound image of the thyroid nodule is mapped. The distance between pixels. For pixel pairs, In thyroid nodules on ultrasound images Direction from grayscale value Pixel to grayscale value The number of pixels in texture. This represents the total number of pixels in the thyroid ultrasound image. extracting a gray level gradient co-occurrence matrix The expression is: （2）； In formula (2), is the number of pixels having a gray value of and a gray gradient value of . Extracting a gray scale run matrix The expression is: （3）； In Equation (3), is a gray value In the direction of the maximum number of consecutive occurrences, is a gray value In the direction of consecutive occurrences times; The expression of the minimum redundancy maximum correlation feature selection is: （4）； In equation (4), is the set of all texture features, and is any one texture feature, is the benign / malignant classification label, is the probability density.

4. The thyroid nodule segmentation method based on prior attention and auxiliary guidance according to claim 1, characterized in that, Step 5 specifically includes: Step 5.1: Using the trained PADPU-Net model Test set of thyroid nodule ultrasound images Testing is performed to obtain test results ; Step 5.2: Test results The open operation, the close operation and the hole filling algorithm are sequentially used for optimization to obtain the final thyroid nodule segmentation result .

Images