Thyroid nodule segmentation method based on prior attention and auxiliary guidance

Abstract

The invention relates to a thyroid nodule segmentation method based on prior attention and auxiliary guidance, belongs to the technical field of ultrasonic medical image processing, and solves the problems that the existing thyroid nodule ultrasonic image segmentation based on U-Net is poor in accuracy and low in clinical application value. Dividing the thyroid ultrasound image data set DS into a training set, a verification set and a test set in proportion; feature extraction is carried out by using the thyroid nodule ultrasound image ITN, minimum redundancy and maximum correlation feature selection is carried out on the extracted features, and optimized thyroid nodule benign and malignant prior features are obtained; calculating the overall loss L, training the PADPU-Net network by using the training set on the basis of the overall loss L, and verifying by using the verification set in the training process to obtain a PADPU-Net model MPADPU-Net; and the PADPU-Net model MPADPU-Net is utilized to test the thyroid nodule ultrasonic image test set DSTest, and a test result is optimized to obtain a final thyroid nodule segmentation result.

Description

Technical Field

[0001] The invention relates to a thyroid nodule segmentation method based on prior attention and auxiliary guidance, and belongs to the technical field of ultrasonic medical image processing. Background Art

[0002] The thyroid gland is the largest endocrine gland in the adult body and plays an important role in regulating human metabolism and growth and development. Among various thyroid abnormalities, thyroid nodules are a common disease with a high incidence rate. Benign thyroid nodules can lead to hormone-related diseases, while malignant nodules seriously endanger health and may even lead to death. Although the detection rate of thyroid cancer has increased significantly with the advancement of medical technology, the mortality rate has stabilized or increased slightly. Ultrasound imaging, as a non-invasive, real-time imaging and cost-effective examination method, has been widely used in the preliminary evaluation of thyroid nodules. Ultrasound images of thyroid nodules can provide characteristic information such as thyroid size, shape and echo, which is of great value for doctors to judge the nature of nodules. Medical image analysis of ultrasound images has become one of the necessary procedures for clinical diagnosis of thyroid nodules, and the segmentation results can provide effective auxiliary information for detailed study of the lesion area, which can assist doctors in disease assessment and treatment plan formulation, etc. Therefore, the invention of an intelligent segmentation method for thyroid nodules is of great clinical application value.

[0003] With the development of deep learning technology, especially its wide application in the field of computer vision, deep learning has gradually replaced traditional manual segmentation methods and achieved more accurate target segmentation. U-Net, as a classic convolutional neural network (CNN) structure, has been widely used in automatic segmentation tasks of medical images such as CT and MRI due to its outstanding performance in medical image segmentation. For example, U-Net has been successfully applied to automatic organ segmentation in CT images and brain tumor segmentation in MRI images. However, although U-Net has a significant segmentation effect in other medical images, it still faces many challenges for ultrasound images of thyroid nodules. Compared with X-ray, CT and MRI images, ultrasound images usually have lower resolution and poorer image quality, and blurred boundaries and noise interference seriously affect the accuracy of intelligent segmentation.

[0004] Although the skip connection in U-Net integrates low-level and high-level features and improves the performance of the segmentation network, the single connection leads to limited performance improvement, and the information level difference is large, resulting in unsatisfactory segmentation results. At the same time, the large span of the channel dimension in the convolutional layer will also have a negative impact. In the encoding pathway, the sudden increase in the channel dimension may cause fine-grained information to be ignored by the initial layer. In the decoding pathway, the sudden decrease in the channel dimension may also lead to the loss of important feature information. In addition, in the decoding pathway, high-level features of different depths are not effectively utilized to assist in obtaining segmentation results. Finally, when implementing thyroid nodule segmentation methods, existing methods mainly focus on improving segmentation accuracy, ignoring the improvement of prior knowledge and clinical application performance of segmentation results. Summary of the invention

[0005] In order to solve the problems of poor accuracy and low clinical application value of existing U-Net-based thyroid nodule ultrasound image segmentation, the present invention proposes a thyroid nodule segmentation method based on prior attention and auxiliary guidance.

[0006] The technical solution adopted by the present invention to solve the above-mentioned problem is: the steps of the present invention include:

[0007] Step 1: Construct a thyroid ultrasound image dataset DS and divide the thyroid ultrasound image dataset DS into a training set DS in proportion. Train , validation set DS Val and the test set DS Test ;

[0008] Step 2: Use the thyroid nodule ultrasound image I in the thyroid ultrasound image dataset DS TN Extract the gray level co-occurrence matrix texture features F GLCM , gray gradient co-occurrence matrix texture feature F GLGCM and grayscale run-length matrix texture feature F GLRLM , perform minimum redundant and maximum correlation feature selection on the extracted features to obtain the optimized benign and malignant prior features F of thyroid nodules Priori ;

[0009] Step 3: Calculate the main loss L main And the auxiliary guidance loss L assist , based on the main loss L main And the auxiliary guidance loss L assist Construct the overall loss L;

[0010] Step 4: Based on the overall loss L, use the training set DS Train Train the PADPU-Net network and use DS during training Val Verify and obtain the PADPU-Net model M PADPU-Net ;

[0011] Step 5: Utilize PADPU-Net model M PADPU-Net , for the thyroid nodule ultrasound image test set DS Test Perform tests and optimize the test results to obtain the final thyroid nodule segmentation results

[0012] Preferably, step 1 specifically includes:

[0013] Step 1.1: Load all thyroid nodule ultrasound images in grayscale mode I TN = {I TN1 ,I TN2 ,...,I TNn},in is the nth ultrasound image of thyroid nodule;

[0014] Step 1.2: For each thyroid nodule ultrasound image Match the corresponding nodule segmentation label and benign and malignant classification labels Construct a thyroid nodule ultrasound image sample DS n ;

[0015] Step 1.3: Using all thyroid nodule ultrasound image samples DS n Construct a thyroid nodule ultrasound image dataset DS, based on p Train :p Val :p Test ,(p Train +p Val +p Test =1) divides the thyroid nodule ultrasound image dataset DS into a training set DS Train , validation set DS Val and the test set DS Test .

[0016] Preferably, step 2 specifically includes:

[0017] Step 2.1: Based on the thyroid nodule ultrasound image I in the thyroid ultrasound image dataset DS TN Extract the gray-level co-occurrence matrix G1, and extract 14 texture features G1 based on the gray-level co-occurrence matrix G1 of 4 different pixel pairs. 1,θ -G1 14,θ , get the gray level co-occurrence matrix texture features Among them, 4 different pixel pairs include 0°, 90°, 45°, 135°, and 14 texture features G1 1,θ -G1 14,θIncludes angular second moment, contrast, coherence, variance, inverse difference moment, sum mean, sum variance, sum entropy, entropy, difference variance, difference entropy, correlation measure 1, correlation measure 2, and homogeneity;

[0018] Step 2.2: Based on the thyroid nodule ultrasound image I in the thyroid ultrasound image dataset DS TN Extract the grayscale gradient co-occurrence matrix G2, and extract 15 texture features G21-G2 based on the grayscale gradient co-occurrence matrix G2. 15 , get the gray gradient co-occurrence matrix texture features Among them, 15 texture features G21-G2 15 Including small gradient advantage, large gradient advantage, grayscale distribution unevenness, gradient distribution unevenness, energy, grayscale mean, gradient mean, grayscale mean square error, gradient mean square error, coherence, grayscale entropy, gradient entropy, mixed entropy, difference moment and inverse difference moment;

[0019] Step 2.3: Based on the thyroid nodule ultrasound image I in the thyroid ultrasound image dataset DS TN Extract the grayscale run matrix G3, and extract 6 texture features G3 based on the grayscale co-occurrence matrix G3 at 4 different angles 1,θ -G3 6,θ , with gray-level co-occurrence matrix texture features Among them, 6 texture features G3 1,θ -G3 6,θ Including short run factor, long run factor, grayscale non-uniformity, length non-uniformity, run percentage and longest prominent run;

[0020] Step 2.4: Based on the benign and malignant classification label O Class The extracted gray level co-occurrence matrix texture feature F GLCM , gray gradient co-occurrence matrix texture feature F GLGCM and grayscale run-length matrix texture feature F GLRLM Perform minimum redundancy and maximum correlation feature selection to obtain the optimized benign and malignant prior features F of thyroid nodules Priori ;

[0021] The expression for extracting the gray level co-occurrence matrix G1 is:

[0022]

[0023] In formula (1), q is the gray level to which the thyroid nodule ultrasound image is mapped, d is the pixel pair distance, θ is the pixel pair direction, P(x1,x2,d) is the number of textures in the thyroid nodule ultrasound image from the gray value x1 pixel to the gray value x2 pixel in the θ direction, and N is the total number of pixels in the thyroid ultrasound image;

[0024] The expression for extracting the gray gradient co-occurrence matrix G2 is:

[0025]

[0026] In formula (2), P(x1,x2) is the number of pixels with gray value x1 and gray gradient value x2;

[0027] The expression for extracting the grayscale run matrix G3 is:

[0028]

[0029] In formula (3), r is the maximum number of times the gray value x1 appears consecutively in the θ direction, and G3(x1,x2,θ) is the number of times the gray value x1 appears consecutively x2 times in the θ direction;

[0030] The expression for minimum redundancy and maximum relevance feature selection is:

[0031]

[0032] In formula (4), G is the set of all texture features, g n and g m is any texture feature, g n The benign and malignant classification labels are p(·) and p(·) is the probability density.

[0033] Preferably, the main loss L in step 3 is main The calculation steps include:

[0034] Step 3.1.1: The PADPU-Net network uses the multi-residual progressive convolution module MRP(m,C,c) to train the training set DS Train Extract the intermediate features F of each layer in the encoding path L1 、F L2 、F L3 、F L4 and F L5 , where m = 0 is the coding mode, m = 1 is the coding mode, c represents the number of channel filling, and C represents the number of output channels after the double residual part;

[0035] Step 3.1.2: Based on the intermediate feature F L1 、F L2 、F L3 and F L4 , combined with the prior feature F Priori , extract the prior features F P1 、F P2 、F P3 and F P4 ;

[0036] Step 3.1.3: Extract self-optimizing prior features using channel-spatial attention mechanism and And extract the transposed convolution features F in turn D6 、F D7 、F D8 and F D9 ;

[0037] Step 3.1.4: Perform skip connections on each layer of the encoding path, and then perform skip connections on F L1 、F D9 and F L2 、F D8 and F L3 、F D7 and F L4 、F D6 and Use channel connections for feature fusion;

[0038] Step 3.1.5: After feature fusion, the multi-residual progressive convolution module MRP(m,C,c) is used to extract the intermediate features F of each layer of the decoding path for the training samples L6 、F L7 、F L8 and F L9 ;

[0039] Step 3.1.6: Based on F L9 Extract thyroid nodule segmentation feature F Seg , using the thyroid nodule segmentation feature F Seg and nodule segmentation label O Seg Calculate the main loss L main ;

[0040] The intermediate features F of each layer in the encoding path L1 、F L2 、F L3 、F L4 and F L5 The calculation formula is:

[0041]

[0042] In formula (5), MP(·) is the maximum pooling algorithm;

[0043] Prior feature F P1 、F P2 、F P3 and F P4 The calculation formula is:

[0044]

[0045] In formula (6), Conv1(1) is a 1×1 convolution operation with an output channel of 1;

[0046] Intermediate features F at each layer of the decoding path L6 、F L7 、F L8 and F L9 The calculation formula is:

[0047]

[0048] In formula (7), Concat(·) is channel connection;

[0049] Main loss L main The calculation formula is:

[0050] L main =-F Seg *log(Sig(O Seg ))-(1-F Seg )*log(1-Sig(O Seg ))(8);

[0051] In formula (8), Sig(·) is the Sigmoid operation.

[0052] Preferably, the auxiliary guidance loss L in step 3 is assist The calculation steps of the overall loss L include:

[0053] Step 3.2.1: Based on the intermediate feature F L5 、F L6 、F L7 and F L8 Extracting auxiliary guidance feature F using auxiliary guidance path AG(H) A1 、F A2 、F A3 and F A4 , where H is the number of input channels of the path;

[0054] Step 3.2.2: Auxiliary guide feature F A1 、F A2 、F A3 and F A4 Perform array sampling operation, and after sampling operation, assist in guiding features and segmentation labels O Seg The size is the same, calculate the mean of the auxiliary guide features and combine them with the nodule segmentation label O Seg Calculate the auxiliary guidance loss L assist ;

[0055] Step 3.2.3: Based on the main loss L main and auxiliary guidance loss L assistCalculate the overall loss L;

[0056] Auxiliary guidance loss L assist The expression is:

[0057]

[0058] In formula (9), IP(·) represents array sampling operation, and H represents the number of input channels;

[0059] The calculation formula of the overall loss L is:

[0060] L=a*L main +b*L assist ,a+b=1(10);

[0061] In formula (10), a is the first constant and b is the second constant.

[0062] Preferably, step 5 specifically includes:

[0063] Step 5.1: Use the trained PADPU-Net model M PADPU-Net DS test set of thyroid nodules ultrasound images Test Perform the test and get the test result R Seg =M PADPU-Net (DS Test );

[0064] Step 5.2: Test the result R Seg =M PADPU-Net (DS Test ) The opening operation, closing operation and hole filling algorithm are used in turn to optimize and obtain the final thyroid nodule segmentation result R Seg =M PADPU-Net (DS Test ).

[0065] The beneficial effects of the present invention are:

[0066] 1. The present invention proposes a PADPU-Net network based on the U-shaped structure of the U-Net network for the segmentation of thyroid nodules ultrasound images, and obtains thyroid nodule segmentation results with high precision and high clinical application value compared with the segmentation method based on the original U-Net. The PADPU-Net network introduces a prior feature extraction pathway on the U-Net jump connection, and introduces the texture features of thyroid nodules that have been screened and contain prior information on the benign and malignant nature of thyroid nodules into the network for extracting self-optimizing prior features. In the present invention, the deep prior features are self-optimized through the channel-space attention mechanism to improve the thyroid nodule segmentation performance and increase the interpretability. At the same time, the PADPU-Net network replaces two consecutive convolutional layers with a designed multi-residual progressive convolution module to avoid the influence of large dimensional spans between channels. At the same time, the multi-residual structure also improves the information correlation between features at high and low levels, achieving an overall improvement in the performance of the segmentation algorithm.

[0067] 2. When training the PADPU-Net network, the present invention proposes an auxiliary guidance pathway, which makes full use of the effective intermediate layer features of each layer of the decoding pathway, extracts the auxiliary segmentation features of the thyroid nodules, and improves the thyroid nodule segmentation performance. In the present invention, the auxiliary guidance pathway uses progressive convolution to process the intermediate layer features of each layer of the decoding pathway to obtain the auxiliary guidance features of each layer. The auxiliary segmentation features are numerically added after being sampled to the original image size to obtain the auxiliary segmentation features. The difference between the auxiliary segmentation features and the nodule segmentation labels is calculated to obtain the auxiliary guidance loss, which is then combined with the traditional main loss to form an overall loss, so that the obtained PADPU-Net model can provide high-performance thyroid nodule segmentation results. BRIEF DESCRIPTION OF THE DRAWINGS

[0068] Figure 1 A flowchart of a thyroid nodule segmentation method based on prior attention and auxiliary guidance provided by the present invention;

[0069] Figure 2 A schematic diagram of the overall structure of PADPU-Net provided by the present invention;

[0070] Figure 3 A schematic diagram of the multi-residual progressive convolution structure of the encoding and decoding paths provided by the present invention;

[0071] Figure 4 A schematic diagram of the self-optimizing prior attention jump connection structure provided by the present invention;

[0072] Figure 5 A schematic diagram of the auxiliary guide path structure provided by the present invention;

[0073] Figure 6 A schematic diagram of a sample of a thyroid nodule ultrasound image dataset provided by the present invention;

[0074] Figure 7 A schematic diagram of the texture feature selection and scoring results provided by the present invention;

[0075] Figure 8 Schematic diagram comparing the segmentation results of the present invention and traditional U-Net. DETAILED DESCRIPTION

[0076] Specific implementation method 1: Combination Figure 1-5 To illustrate this embodiment, Figure 1 As shown, the specific steps of the thyroid nodule segmentation method based on prior attention and auxiliary guidance described in this embodiment include:

[0077] Step 1: Load the ultrasound image of thyroid nodules TN , is the nodule segmentation label O corresponding to the entire image matching Seg and benign and malignant classification label O Class , a thyroid nodule ultrasound image dataset DS was constructed, and the dataset was divided into a training set DS Train , validation set DS Val and the test set DS Test The specific steps are as follows:

[0078] Step 1: Load all thyroid nodule ultrasound images in grayscale mode TN n represents the nth ultrasound image of thyroid nodule;

[0079] Step 12: For each thyroid nodule ultrasound image Match the corresponding nodule segmentation label and benign and malignant classification labels To form a sample DS of a thyroid nodule ultrasound image n ;

[0080] Step 13: Use all thyroid nodule ultrasound image samples to construct the thyroid nodule ultrasound image dataset DS. Train :p Val :p Test ,(p Train +p Val +p Test =1) divides the data set DS into a training set DS Train , validation set DS Val and the test set DS Test .

[0081] Step 2: Using ultrasound images of thyroid nodules TN Extract the gray level co-occurrence matrix texture features F GLCM , gray gradient co-occurrence matrix texture feature F GLGCMand grayscale run-length matrix texture feature F GLRLM , based on the benign and malignant classification label O Class Perform minimum redundancy and maximum correlation feature selection to obtain the optimized benign and malignant prior features F of thyroid nodules Priori .

[0082] Step 21: Ultrasound image of thyroid nodules TN Extract the gray-level co-occurrence matrix G1, and extract 14 texture features G1 based on the gray-level co-occurrence matrix G1 of 4 different pixel pairs. 1,θ -G1 14,θ , get the gray level co-occurrence matrix texture features Among them, 4 different pixel pairs include 0°, 90°, 45°, 135°, and 14 texture features G1 1,θ -G1 14,θ Includes angular second moment, contrast, coherence, variance, inverse difference moment, sum mean, sum variance, sum entropy, entropy, difference variance, difference entropy, correlation measure 1, correlation measure 2, and homogeneity;

[0083] The expression for extracting the gray level co-occurrence matrix G1 is:

[0084]

[0085] In formula (1), q is the gray level to which the thyroid nodule ultrasound image is mapped, d is the pixel pair distance, θ is the pixel pair direction, P(x1,x2,d) is the number of textures in the thyroid nodule ultrasound image from the gray value x1 pixel to the gray value x2 pixel in the θ direction, and N is the total number of pixels in the thyroid ultrasound image;

[0086] Step 22: Based on the thyroid nodule ultrasound image I in the thyroid ultrasound image dataset DS TN Extract the grayscale gradient co-occurrence matrix G2, and extract 15 texture features G21-G2 based on the grayscale gradient co-occurrence matrix G2. 15 , get the gray gradient co-occurrence matrix texture features Among them, 15 texture features G21-G2 15 Including small gradient advantage, large gradient advantage, grayscale distribution unevenness, gradient distribution unevenness, energy, grayscale mean, gradient mean, grayscale mean square error, gradient mean square error, coherence, grayscale entropy, gradient entropy, mixed entropy, difference moment and inverse difference moment;

[0087] The expression for extracting the gray gradient co-occurrence matrix G2 is:

[0088]

[0089] In formula (2), P(x1,x2) is the number of pixels with gray value x1 and gray gradient value x2;

[0090] Step 23: Based on the thyroid nodule ultrasound image I in the thyroid ultrasound image dataset DS TN Extract the grayscale run matrix G3, and extract 6 texture features G3 based on the grayscale co-occurrence matrix G3 at 4 different angles 1,θ -G3 6,θ , with gray-level co-occurrence matrix texture features Among them, 6 texture features G3 1,θ -G3 6,θ Including short run factor, long run factor, grayscale non-uniformity, length non-uniformity, run percentage and longest prominent run;

[0091] The expression for extracting the grayscale run matrix G3 is:

[0092]

[0093] In formula (3), r is the maximum number of times the gray value x1 appears consecutively in the θ direction, and G3(x1,x2,θ) is the number of times the gray value x1 appears consecutively x2 times in the θ direction;

[0094] Step 24: Based on the benign and malignant classification label O Class Perform minimum redundancy and maximum correlation feature selection to extract all the texture features F GLCM 、F GLGCM and F GLRLM Screening is performed to obtain the optimized prior features F of benign and malignant thyroid nodules Priori , the selection rules are as follows:

[0095]

[0096] In formula (4), G is the set of all texture features, g n and g m is any texture feature, O Classn g n The benign and malignant classification labels are p(·) and p(·) is the probability density.

[0097] Step 3: The PADPU-Net network first extracts the intermediate features F of each layer of the encoding path for the training samples L1 、F L2 、F L3 、F L4 and F L5 , and then based on F L1 、F L2 、F L3 and F L4According to the prior feature F obtained in step 2 Priori , using the channel-spatial attention mechanism to extract self-optimized prior features of each layer and And extract the transposed convolution features F in turn D6 、F D7 、F D8 and F D9 , and then extract the intermediate features F of each layer of the decoding path after feature fusion L6 、F L7 、F L8 and F L9 , and then based on F L9 Extract thyroid nodule segmentation feature F Seg , using the thyroid nodule segmentation feature F Seg and nodule segmentation label O Seg Calculate the main loss L main Then, based on F L5 、F L6 、F L7 and F L8 Extract auxiliary guiding features F A1 、F A2 、F A3 and F A4 , using the guided segmentation feature F A1 、F A2 、F A3 and F A4 and nodule segmentation label O Seg Calculate the auxiliary guidance loss L assist , and then with the main loss L main The overall loss L is constructed to train the PADPU-Net network. Using the thyroid nodule ultrasound image training set DS Train Train the PADPU-Net network and use DS during training Val Verify and finally obtain the PADPU-Net model M PADPU-Net .

[0098] Step 31: PADPU-Net structure diagram as shown Figure 2 As shown, the network first passes through Figure 3 The multi-residual progressive convolutional coding mode shown in the figure extracts the intermediate features F of each layer of the coding path for the training samples. L1 、F L2 、F L3 、F L4 and F L5 , the multi-residual progressive convolution module is recorded as MRP(m,C,c), where m=0 is the encoding mode, m=1 is the encoding mode, c represents the number of channel padding, and C represents the number of output channels after the double residual part. The intermediate features of each layer are calculated as follows:

[0099]

[0100] In formula (5), MP(·) is the maximum pooling algorithm;

[0101] Step 32: Based on F L1 、F L2 、F L3 and F L4 According to the prior feature F obtained in step 24 Priori , extract the prior features F P1 、F P2 、F P3 and F P4 , the specific calculation is as follows:

[0102]

[0103] In formula (6), Conv1(1) is a 1×1 convolution operation with an output channel of 1;

[0104] Then, the prior feature channel-spatial attention module is liberalized, and the self-optimized prior feature is obtained as and Then, feature fusion is performed after the jump connection, and the transposed convolution feature F is taken. D6 、F D7 、F D8 and F D9 , and then L1 、F D9 and F L2 、F D8 and F L3 、F D7 and F L4 、F D6 and Using channel connection for feature fusion, the self-optimized prior attention jump connection structure of the PADPU-Net network involved in this step is as follows Figure 4 shown.

[0105] Step 3: After feature fusion, use Figure 3 The multi-residual progressive convolutional decoding mode shown in the figure extracts the intermediate features F of each layer of the decoding path L6 、F L7 、F L8 and F L9 , the intermediate features of each layer are calculated as follows:

[0106]

[0107] In formula (7), Concat(·) is channel connection;

[0108] Furthermore, based on F L9 Extract thyroid nodule segmentation feature F Seg , using the thyroid nodule segmentation feature F Seg and nodule segmentation label O Seg Calculate the main loss L main , the calculation formula is as follows:

[0109] L main =-F Seg *log(Sig(O Seg ))-(1-F Seg )*log(1-Sig(O Seg ))(8);

[0110] In formula (8), Sig(·) is the Sigmoid operation.

[0111] Then, based on F L5 、F L6 、F L7 and F L8 Using the auxiliary guidance path AG(H), H represents the number of input channels of the path to extract the auxiliary guidance feature F A1 、F A2 、F A3 and F A4 Each auxiliary guide feature is sampled by an array, and after sampling, the auxiliary guide feature and the segmentation label O Seg have the same size, and then calculate the mean and compare it with the nodule segmentation label O Seg Calculate the auxiliary guidance loss L assist , the calculation formula is as follows:

[0112]

[0113] In formula (9), IP(·) is the array sampling operation, and H is the number of input channels;

[0114] Auxiliary guide pathway structure Figure 5 As shown, the overall loss L = a*L main +b*L assist ,a+b=1,

[0115] Step 3 and 4: Based on the overall loss, use the thyroid nodule ultrasound image training set DS Train Train the PADPU-Net network and use DS during training Val Verify and finally obtain the PADPU-Net model M PADPU-Net .

[0116] Step 4: Use the trained PADPU-Net model M PADPU-Net , for the thyroid nodule ultrasound image test set DS Test Conduct tests and then optimize the test results to obtain the final thyroid nodule segmentation results

[0117] Step 41: Use the trained PADPU-Net model M PADPU-Net , for the thyroid nodule ultrasound image test set DS Test Perform the test and obtain the test result R Seg =M PADPU-Net (DS Test )

[0118] Step 42: Optimize the test results by using the opening operation, closing operation and hole filling algorithm in turn to obtain the final thyroid nodule segmentation result R Seg =M PADPU-Net (DS Test ).

[0119] Specific implementation method 2: Combination Figure 6-8 This embodiment is described. In this embodiment, the verification data is derived from a thyroid medical survey, which is a real collected thyroid nodule slice image, and the thyroid segmentation label and benign and malignant classification label are drawn by professional physicians.

[0120] Execute step 1: load all 1154 thyroid nodule ultrasound images, match the corresponding nodule segmentation labels and benign and malignant classification labels for the thyroid nodule ultrasound images, and form 1154 thyroid nodule ultrasound image samples. Use all thyroid nodule ultrasound image samples to construct a thyroid nodule ultrasound image dataset. The sample of the thyroid nodule ultrasound image dataset is as follows: Figure 6 As shown, the dataset is divided into a training set consisting of 714 samples, a validation set consisting of 220 samples, and a test set consisting of 220 samples according to a sample division ratio of approximately 0.6:0.2:0.2.

[0121] Execute step 2: for the thyroid nodule ultrasound image, q = 16, d = 5 and θ = 0°, 45°, 90°, 135° are used to extract the gray level co-occurrence matrix angular second moment, contrast, coherence, variance, inverse difference moment, sum average, sum variance, sum entropy, entropy, difference variance, difference entropy, correlation measure 1, correlation measure 2 and homogeneity texture features. A total of 14×4=56 gray level co-occurrence matrix texture features can be extracted, and then for the thyroid nodule ultrasound image, q = 16 is used to extract small gradient advantage, large gradient advantage, gray distribution inhomogeneity, gradient distribution inhomogeneity, energy, gray mean, gradient mean, Grayscale mean square error, gradient mean square error, coherence, grayscale entropy, gradient entropy, mixed entropy, differential moment and inverse differential moment texture features are used to extract a total of 15 grayscale gradient co-occurrence matrix texture features. Subsequently, short run factor, long run factor, grayscale unevenness, length unevenness, run percentage and longest prominent run grayscale co-occurrence matrix texture features are extracted for thyroid nodule ultrasound images with q=16 and θ=0°, 45°, 90°, 135°. A total of 6×4=24 grayscale co-occurrence matrix texture features can be extracted, and 56+15+24=95 thyroid nodule ultrasound image texture features are obtained.

[0122] Then, based on the benign and malignant classification labels of thyroid nodules ultrasound images (benign is 0, malignant is 1), the texture features of thyroid nodules ultrasound images were screened using the minimum redundancy maximum correlation feature selection. A score of 0.0005 was selected to retain 13 valid features containing prior information on benign and malignant classification. The schematic diagram of the texture feature selection score result is shown in the figure. Figure 7 As shown in the figure, the features retained from feature 1 to feature 13 in descending order of scores are: grayscale gradient symbiosis matrix gradient distribution unevenness, grayscale gradient symbiosis matrix inverse difference moment, grayscale run matrix 0°-short run factor, grayscale run matrix 45°-longest prominent run, grayscale gradient symbiosis matrix length distribution unevenness, grayscale symbiosis matrix 45°-difference variance, grayscale run matrix 135°-longest prominent run, grayscale gradient symbiosis matrix small gradient advantage, grayscale run matrix 90°-short run factor, grayscale gradient symbiosis matrix gradient entropy, grayscale run matrix 90°-longest prominent run, grayscale symbiosis matrix 90°-homogeneity, grayscale gradient symbiosis matrix difference moment.

[0123] Execute step three: First, extract the intermediate features of each layer of the coding path of the training samples through the multi-residual progressive convolutional coding mode. The multi-residual progressive modules used are MRP (0,63,1), MRP (0,126,2), MRP (0,252,4), MRP (0,504,8) and MRP (0,1008,16), and the number of intermediate feature channels is 64, 128, 256, 512 and 1024 respectively.

[0124] Then, based on the first four layers of intermediate features, the grayscale gradient symbiosis matrix gradient distribution unevenness, grayscale gradient symbiosis matrix inverse difference moment, grayscale run matrix 0°-short run factor, grayscale run matrix 45°-longest prominent run, grayscale gradient symbiosis matrix length distribution unevenness, grayscale symbiosis matrix 45°-difference variance, grayscale run matrix 135°-longest prominent run, grayscale gradient symbiosis matrix small gradient advantage, grayscale run matrix 90°-short run factor, grayscale gradient symbiosis matrix gradient entropy, grayscale run matrix 90°- The longest prominent run, gray level co-occurrence matrix 90°-homogeneity, and gray level gradient co-occurrence matrix differential moment are used to extract prior features, and the deep prior features are sent to the prior feature channel-spatial attention module, and 4 layers of self-optimized prior features with 13 feature channels are obtained in turn. Feature fusion is then performed after jump connection, and the number of output feature channels after feature fusion is 64+63+13=140, 128+126+13=267, 256+252+13=521, and 512+504+13=1029, respectively.

[0125] Then, based on the 5th layer encoding intermediate features and fusion features, the intermediate features of each layer of the decoding path are extracted. The multi-residual progressive modules used are MRP (1,504,8), MRP (1,252,4), MRP (1,126,2) and MRP (0,504,8) respectively. Then, the thyroid nodule segmentation features are extracted, and the main loss is calculated using the thyroid nodule segmentation features and the nodule segmentation labels. Subsequently, the auxiliary guidance features are extracted using the auxiliary guidance pathways AG (1008), AG (504), AG (252) and AG (126) in turn. The auxiliary guidance loss is calculated by averaging the sum of the auxiliary guidance features after array sampling and the nodule segmentation label, and L = 0.75*L main +0.25*L assist constitutes an overall loss.

[0126] Subsequently, the PADPU-Net network was trained using a training set consisting of 714 thyroid nodule ultrasound image samples. During the training process, a validation set consisting of 220 thyroid nodule ultrasound image samples was used for verification. Finally, the PADPU-Net model M was obtained. PADPU-Net .

[0127] Execute step 4: Use PADPU-Net model M PADPU-Net The test set consisting of 220 thyroid nodule ultrasound image samples was tested to obtain the test results. The test results were then optimized using the open operation, close operation, and hole filling algorithm in turn to obtain the improved thyroid nodule segmentation results. The comparison example of the obtained thyroid nodule improved segmentation results is as follows: Figure 8As shown, comparison diagrams of traditional segmentation results, test segmentation results, improved segmentation results and thyroid nodule segmentation labels are given respectively. It can be seen from the figure that the segmentation result (improved) of the present invention has a more accurate segmentation result than the segmentation result (traditional) based on U-Net. Furthermore, the accuracy improvement of the present invention is compared by F1 score, intersection ratio and Matthews correlation coefficient, and the improvement of the clinical application value of the present invention is compared by aspect ratio, compactness, eccentricity and concavity. The comparative evaluation results are shown in Table 1. Except for a slight loss in aspect ratio, the other ratings of the test results are higher than those of the traditional algorithm, and the improved results of the present invention not only greatly improve the accuracy of thyroid nodule segmentation, but also greatly improve the clinical application value.

[0128] Table 1

[0129]

[0130] The above is only a preferred embodiment of the present invention and does not limit the present invention in any form. Although the present invention has been disclosed as a preferred embodiment as above, it is not used to limit the present invention. Any technician familiar with this profession can make some changes or modify the technical contents disclosed above into equivalent embodiments without departing from the scope of the technical solution of the present invention. However, any simple modification, equivalent replacement and improvement made to the above embodiments without departing from the content of the technical solution of the present invention, based on the technical essence of the present invention, within the spirit and principles of the present invention, still fall within the protection scope of the technical solution of the present invention.

Claims

1. A thyroid nodule segmentation method based on prior attention and auxiliary guidance, characterized in that: The steps of the thyroid nodule segmentation method based on prior attention and auxiliary guidance include: Step 1: Construct a thyroid ultrasound image dataset DS and divide the thyroid ultrasound image dataset DS into a training set DS in proportion. Train , validation set DS Val and the test set DS Test ; Step 2: Use the thyroid nodule ultrasound image I in the thyroid ultrasound image dataset DS TN Extract the gray level co-occurrence matrix texture features F GLCM , gray gradient co-occurrence matrix texture feature F GLGCM and grayscale run-length matrix texture feature F GLRLM , perform minimum redundant and maximum correlation feature selection on the extracted features to obtain the optimized benign and malignant prior features F of thyroid nodules Priori ; Step 3: Calculate the main loss L main And the auxiliary guidance loss L assist , based on the main loss L main And the auxiliary guidance loss L assist Construct the overall loss L; Step 4: Based on the overall loss L, use the training set DS Train Train the PADPU-Net network and use DS during training Val Verify and obtain the PADPU-Net model M PADPU-Net ; Step 5: Utilize PADPU-Net model M PADPU-Net , for the thyroid nodule ultrasound image test set DS Test Perform tests and optimize the test results to obtain the final thyroid nodule segmentation results 2. The thyroid nodule segmentation method based on prior attention and auxiliary guidance according to claim 1, characterized in that: Step 1 specifically includes: Step 1.1: Load all thyroid nodule ultrasound images in grayscale mode in is the nth ultrasound image of thyroid nodule; Step 1.2: For each thyroid nodule ultrasound image Match the corresponding nodule segmentation label and benign and malignant classification labels Construct a thyroid nodule ultrasound image sample DS n ; Step 1.3: Using all thyroid nodule ultrasound image samples DS n Construct a thyroid nodule ultrasound image dataset DS, based on p Train :p Val :p Test ,(p Train +p Val +p Test =1) divides the thyroid nodule ultrasound image dataset DS into a training set DS Train , validation set DS Val and the test set DS Test .

3. The thyroid nodule segmentation method based on prior attention and auxiliary guidance according to claim 1, characterized in that: Step 2 specifically includes: Step 2.1: Based on the thyroid nodule ultrasound image I in the thyroid ultrasound image dataset DS TN Extract the gray-level co-occurrence matrix G1, and extract 14 texture features G1 based on the gray-level co-occurrence matrix G1 of 4 different pixel pairs. 1,θ -G1 14,θ , get the gray level co-occurrence matrix texture features Among them, 4 different pixel pairs include 0°, 90°, 45°, 135°, and 14 texture features G1 1,θ -G1 14,θ Includes angular second moment, contrast, coherence, variance, inverse difference moment, sum mean, sum variance, sum entropy, entropy, difference variance, difference entropy, correlation measure 1, correlation measure 2, and homogeneity; Step 2.2: Based on the thyroid nodule ultrasound image I in the thyroid ultrasound image dataset DS TN Extract the grayscale gradient co-occurrence matrix G2, and extract 15 texture features G21-G2 based on the grayscale gradient co-occurrence matrix G2. 15 , get the gray gradient co-occurrence matrix texture features Among them, 15 texture features G21-G2 15 Including small gradient advantage, large gradient advantage, grayscale distribution unevenness, gradient distribution unevenness, energy, grayscale mean, gradient mean, grayscale mean square error, gradient mean square error, coherence, grayscale entropy, gradient entropy, mixed entropy, difference moment and inverse difference moment; Step 2.3: Based on the thyroid nodule ultrasound image I in the thyroid ultrasound image dataset DS TN Extract the grayscale run matrix G3, and extract 6 texture features G3 based on the grayscale co-occurrence matrix G3 at 4 different angles 1,θ -G3 6,θ Texture features with gray-level co-occurrence matrix Among them, 4 different angles include 0°, 90°, 45°, 135°, and 6 texture features G3 1,θ -G3 6,θ Including short run factor, long run factor, grayscale non-uniformity, length non-uniformity, run percentage and longest prominent run; Step 2.4: Based on the benign and malignant classification label O Class The extracted gray level co-occurrence matrix texture feature F GLCM , gray gradient co-occurrence matrix texture feature F GLGCM and grayscale run-length matrix texture feature F GLRLM Perform minimum redundancy and maximum correlation feature selection to obtain the optimized benign and malignant prior features F of thyroid nodules Priori ; The expression for extracting the gray level co-occurrence matrix G1 is: In formula (1), q is the gray level to which the thyroid nodule ultrasound image is mapped, d is the pixel pair distance, θ is the pixel pair direction, P(x1,x2,d) is the number of textures in the thyroid nodule ultrasound image from the gray value x1 pixel to the gray value x2 pixel in the θ direction, and N is the total number of pixels in the thyroid ultrasound image; The expression for extracting the gray gradient co-occurrence matrix G2 is: In formula (2), P(x1,x2) is the number of pixels with gray value x1 and gray gradient value x2; The expression for extracting the grayscale run matrix G3 is: In formula (3), r is the maximum number of times the gray value x1 appears consecutively in the θ direction, and G3(x1,x2,θ) is the number of times the gray value x1 appears consecutively x2 times in the θ direction; The expression for minimum redundancy and maximum relevance feature selection is: In formula (4), G is the set of all texture features, g n and g m is any texture feature, g n The benign and malignant classification labels are p(·) and p(·) is the probability density.

4. The thyroid nodule segmentation method based on prior attention and auxiliary guidance according to claim 1, characterized in that: The main loss L in step 3 main The calculation steps include: Step 3.1.1: The PADPU-Net network uses the multi-residual progressive convolution module MRP(m,C,c) to train the training set DS Train Extract the intermediate features F of each layer in the encoding path L1 、F L2 、F L3 、F L4 and F L5 , where m = 0 is the coding mode, m = 1 is the coding mode, c represents the number of channel filling, and C represents the number of output channels after the double residual part; Step 3.1.2: Based on the intermediate feature F L1 、F L2 、F L3 and F L4 , combined with the prior feature F Priori , extract the prior features F P1 、F P2 、F P3 and F P4 ; Step 3.1.3: Extract self-optimizing prior features using channel-spatial attention mechanism and And extract the transposed convolution features F in turn D6 、F D7 、F D8 and F D9 ; Step 3.1.4: Perform skip connections on each layer of the encoding path, and then perform skip connections on each pair of F L1 、F D9 and F L2 、F D8 and F L3 、F D7 and F L4 、F D6 and Use channel connections for feature fusion; Step 3.1.5: After feature fusion, the multi-residual progressive convolution module MRP(m,C,c) is used to extract the intermediate features F of each layer of the decoding path for the training samples L6 、F L7 、F L8 and F L9 ; Step 3.1.6: Based on F L9 Extract thyroid nodule segmentation feature F Seg , using the thyroid nodule segmentation feature F Seg and nodule segmentation label O Seg Calculate the main loss L main ; The intermediate features F of each layer in the encoding path L1 、F L2 、F L3 、F L4 and F L5 The calculation formula is: In formula (5), MP(·) is the maximum pooling algorithm; Prior feature F P1 、F P2 、F P3 and F P4 The calculation formula is: In formula (6), Conv1(1) is a 1×1 convolution operation with an output channel of 1; Intermediate features F at each layer of the decoding path L6 、F L7 、F L8 and F L9 The calculation formula is: In formula (7), Concat(·) is channel connection; Main loss L main The calculation formula is: THE main =-F Seg *log(Sig(O Seg ))-(1-F Seg )*log(1-Sig(O Seg )) (8); In formula (8), Sig(·) is the Sigmoid operation.

5. The thyroid nodule segmentation method based on prior attention and auxiliary guidance according to claim 1, characterized in that: The auxiliary guidance loss L in step 3 assist The calculation steps of the overall loss L include: Step 3.2.1: Based on the intermediate feature F L5 、F L6 、F L7 and F L8 Extracting auxiliary guidance feature F using auxiliary guidance path AG(H) A1 、F A2 、F A3 and F A4 , where H is the number of input channels of the path; Step 3.2.2: Auxiliary guide feature F A1 、F A2 、F A3 and F A4 Perform array sampling operation, and after sampling operation, assist in guiding features and segmentation labels O Seg The size is the same, calculate the mean of the auxiliary guide features and combine them with the nodule segmentation label O Seg Calculate the auxiliary guidance loss L assist ; Step 3.2.3: Based on the main loss L main and auxiliary guidance loss L assist Calculate the overall loss L; Auxiliary guidance loss L assist The expression is: In formula (9), IP(·) represents array sampling operation, and H represents the number of input channels; The calculation formula of the overall loss L is: L=a*L main +b*L assist ,a+b=1 (10); In formula (10), a is the first constant and b is the second constant.

6. The thyroid nodule segmentation method based on prior attention and auxiliary guidance according to claim 1, characterized in that: Step 5 specifically includes: Step 5.1: Use the trained PADPU-Net model M PADPU-Net DS test set of thyroid nodules ultrasound images Test Perform the test and get the test result R Seg =M PADPU-Net (DS Test ); Step 5.2: Test the result R Seg =M PADPU-Net (DS Test ) The opening operation, closing operation and hole filling algorithm are used in turn to optimize and obtain the final thyroid nodule segmentation result R Seg =M PADPU-Net (DS Test ).

Images