Thyroid gland and nodule segmentation method based on improved U-Net network

Abstract

This invention relates to a thyroid gland and nodule segmentation method based on an improved U-Net network. Ultrasound is the preferred method for thyroid nodule examination; however, ultrasound images are noisy, and thyroid nodules vary in shape, posing a significant challenge to doctors' diagnosis. Current automatic thyroid nodule segmentation methods perform poorly on irregularly edged and small nodules. To overcome the shortcomings of inaccurate segmentation of thyroid gland edges and small nodules due to insufficient feature extraction from high-resolution data, unsatisfactory segmentation results for large and irregular targets, and the tedious and time-consuming nature of manual segmentation, this invention proposes a thyroid gland and nodule segmentation method based on an improved U-Net network. By introducing ResNeSt modules, Atrous Spatial Pyramid Pooling (ASPP), and Deformable Convolution (DC) v3 into the encoder and decoder of the U-Net network for feature extraction, a Deformable-Pyramid Split Attention Residual U-Net (DSRU-Net) is established. Experimental and methodological analyses show that using the DSRU-Net network can significantly improve the accuracy of thyroid gland and nodule segmentation.

Description

Technical Field

[0001] This invention belongs to the field of image processing and artificial intelligence-assisted disease diagnosis, specifically a method for segmenting thyroid glands and nodules based on an improved U-Net network. Background Technology

[0002] It is well known that early diagnosis and treatment of malignant thyroid nodules can significantly improve or effectively prevent the incidence of thyroid cancer. Ultrasound has become the preferred method for thyroid examination due to its advantages such as no radiation, non-invasiveness, and good real-time performance. However, ultrasound images contain speckles and noise, and the size, location, and shape of thyroid nodules vary greatly among different populations, posing a significant challenge to doctors in diagnosis.

[0003] Delineating the boundaries of thyroid glands and nodules is crucial for accurate clinical assessment. Currently, this segmentation is done manually by physicians. This process is extremely time-consuming and labor-intensive, and its effectiveness is heavily influenced by the physician's diagnostic skill. With the rapid development of artificial intelligence technology, various deep learning-based methods for segmenting thyroid glands and nodules have been proposed. While these methods have solved some problems, their effectiveness remains unsatisfactory for segmenting nodules with complex backgrounds, high similarity, and varying shapes and sizes. Summary of the Invention

[0004] To overcome the shortcomings of inaccurate segmentation of thyroid gland margins and small nodules due to insufficient feature extraction in high-resolution data, unsatisfactory segmentation results for large and irregular targets, and the tedious and time-consuming nature of manual segmentation, a thyroid gland and nodule segmentation method based on an improved U-Net network is proposed. The specific implementation steps are described below:

[0005] Step 1: Dataset Setup: A professional doctor selects a hospital... a The total number of patients b A series of thyroid ultrasound images were processed and filtered to obtain a final image. d ( d ≤ b A dataset consisting of 100 thyroid ultrasound images S ; S In r ( r<d () as training dataset S 1, S The remaining ones d - r Amount as test dataset S 2; S It contains both benign and malignant samples. Benign samples include inflammatory nodules, cystic nodules, and tumor nodules, while malignant samples include papillary carcinoma and follicular carcinoma.

[0006] Step 2, Model Building: Improve the U-Net network by introducing ResNeSt module, Atrous Spatial Pyramid Pooling (ASPP), and Deformable Convolution (DC) v3 into the encoder and decoder of the U-Net network for feature extraction, and build a Deformable-Pyramid Split Attention Residual U-Net (DSRU-Net).

[0007] Step 3, Model Training: Using S 1 r Four thyroid ultrasound images were used to train the DSRU-Net network model using a loss function and optimizer. During training, data augmentation strategies were employed to enhance the samples in real time, including increasing / decreasing brightness, increasing / decreasing shrinkage, horizontal mirroring, random angle rotation, random cropping, and random stretching. To evaluate the model output and parameters, a weighted cross-entropy loss function was used. L wce and dice loss function L dice Define a new loss function L total The optimized model parameters are then updated to obtain the trained DSRU-Net network. L wce Defined as:

[0008] (1)

[0009] in, n For any sample in any batch, N For batch size, c For any category, C For the number of categories, nc For the sample n Category ntc For the sample n The correct category, w ntc For the sample n The correct class weights, x nc For the sample n The predicted value corresponding to the category, x ntc For the sample n The predicted value corresponding to the correct category. t For single-use non-redundant categories, T This represents the total number of non-redundant categories. wt For non-redundant class weights in batch processing, exp (·) is an exponential function; L dice Defined as:

[0010] (2)

[0011] in, e It is a relatively small constant, usually taken as 1. y nc Indicates sample n The true value of the category; L total Defined as:

[0012] L total = L wce + L dice (3);

[0013] Step 4, Thyroid gland and nodule segmentation: Using the trained DSRU-Net network, segmentation is performed on the thyroid gland and nodules. S 2 d - r The thyroid gland and nodules in a thyroid ultrasound image were segmented to obtain the segmentation results. R ;

[0014] Step 5: Segmentation Performance Evaluation: Specificity (SP), Sensitivity (SE), Precision (PR), Accuracy (ACC), Intersection over Union (IoU), and Dice Similarity Coefficient (DSC) are used to evaluate the segmentation performance. R Evaluation and analysis are conducted; they are defined as follows:

[0015] (4)

[0016] (5)

[0017] (6)

[0018] (7)

[0019] (8)

[0020] (9)

[0021] in, TP It is the number of true positives. TN It is the number of true negatives. FP It is the number of false positives. FN It is the number of false negatives.

[0022] Furthermore, in step 2, the DSRU-Net network is defined as follows: First, to enhance feature extraction capabilities, a ResNeSt module with good feature extraction tendency is introduced into the encoder and decoder. The ResNeSt module includes a split-transform-merge structure inherited from GoogleNet and ResNeXt, and a channel attention structure inspired by SENet and SKNet. Second, to improve the segmentation capability of targets at different scales, ASPP is introduced between the encoder and decoder to extract multi-scale features. Finally, to improve the model's adaptability to targets with special shapes, DCv3 is introduced into ASPP.

[0023] The beneficial effects of this invention are as follows: (1) The U-Net network has shortcomings in thyroid segmentation: firstly, edge segmentation errors and false negatives and false negatives of small nodules are caused by insufficient high-resolution data feature extraction; secondly, the segmentation of large and irregular targets is slightly insufficient. Therefore, ResNeSt module, ASPP and DCv3 are introduced into the encoder and decoder of the U-Net network to establish a DSRU-Net network; (2) In order to overcome the shortcomings of inaccurate segmentation of thyroid edge and small nodules caused by insufficient high-resolution data feature extraction, unsatisfactory segmentation effect of large and irregular targets, and tedious and time-consuming manual segmentation, a thyroid gland and nodule segmentation method based on the improved U-Net network is proposed. Attached Figure Description

[0024] Figure 1 DSRU-Net network overall architecture;

[0025] Figure 2 : ResNeSt module structure;

[0026] Figure 3 SRU-Net overall architecture;

[0027] Figure 4 The structure of ASPP and DASPP;

[0028] Figure 5 Malignant sample from thyroid ultrasound image;

[0029] Figure 6 Experimental segmentation results of malignant samples from thyroid ultrasound images;

[0030] Figure 7: Correct segmentation results of malignant samples from thyroid ultrasound images;

[0031] Figure 8 : Benign sample of thyroid ultrasound image;

[0032] Figure 9 Results of experimental segmentation of benign thyroid ultrasound images;

[0033] Figure 10 : Correct segmentation results of benign thyroid ultrasound images; Detailed Implementation

[0034] The present invention will be further described below with reference to the accompanying drawings and specific embodiments, but the scope of protection of the present invention is not limited thereto.

[0035] Step 1: Dataset Construction: Professional physicians processed and filtered 76,496 ultrasound images from 5,021 patients at Xuzhou Medical University Affiliated Hospital and Nanjing First Hospital between 2012 and 2018, resulting in a dataset of 5,822 thyroid ultrasound images. S ; S 4658 images were used as the training dataset S 1, S The remaining 1164 images were used as the test dataset. S 2; S It contains both benign and malignant samples. Benign samples include inflammatory nodules, cystic nodules, and tumor nodules, while malignant samples include papillary carcinoma and follicular carcinoma.

[0036] Step 2, Model Building: The U-Net network is improved by introducing ResNeSt, ASPP, and DC v3 modules into the encoder and decoder of the U-Net network for feature extraction, thus establishing a DSRU-Net network. The overall architecture of this network is as follows: Figure 1 As shown. To enhance feature extraction capabilities, ResNeSt modules with good feature extraction tendencies are introduced into both the encoder and decoder. The ResNeSt module structure is as follows. Figure 2 As shown; a split attention mechanism is introduced into this model, which improves the model's performance and interpretability to some extent; the Split-Attention Residual U-Net (SRU-Net) in the ResNeSt module, as shown... Figure 3As shown, both the encoder and decoder of U-Net use ResNeSt modules instead of convolutional modules to process feature maps. Simultaneously, as indicated by the gray shaded area, an additional layer is added on top of the encoder to improve the model's performance in extracting edges and small objects. This approach improves model efficiency while retaining many of the advantages of U-Net, simplifies backpropagation and parameter updates, and enhances model stability. To improve the segmentation ability of objects at different scales, ASPP is introduced between the encoder and decoder to extract multi-scale features. However, the simple dilated convolution of ASPP is prone to losing important features while expanding the receptive field. To improve the model's adaptability to objects with special shapes, DC v3 is introduced into ASPP, and a Deformable Atrous Spatial Pyramid Pooling (DASPP) structure is designed, as shown... Figure 4 As shown. Among them, d The expansion rate is denoted as .

[0037] Step 3, Model Training: Using S 4658 thyroid ultrasound images were used in dataset 1. A loss function and optimizer were employed to train the DSRU-Net network model. During training, data augmentation strategies were used in real-time to enhance the samples, including increasing / decreasing brightness, decreasing / decreasing shrinkage, horizontal mirroring, random angle rotation, random cropping, and random stretching. A new loss function was defined to evaluate the model output and parameters. L total The optimized model parameters are then updated to obtain the trained DSRU-Net network.

[0038] Step 4, Thyroid gland and nodule segmentation: Using the trained DSRU-Net network, segmentation is performed on the thyroid gland and nodules. S 1164 thyroid ultrasound images from 2 sources were segmented to obtain the thyroid gland and nodules. The segmentation results are shown below for a malignant thyroid ultrasound image, its corresponding experimental segmentation result, and the correct segmentation result. Figure 5 , 6 As shown in Figure 7, a benign thyroid ultrasound image, the corresponding experimental segmentation result, and the correct segmentation result are shown in Figure 7. Figure 8 , 9 As shown in Figure 10. Figure 6 , 7 The outer contours in 9 and 10 represent the thyroid gland contours, while the inner contours represent the nodule contours.

[0039] Step 5: Segmentation Performance Evaluation: The segmentation results were evaluated and analyzed using evaluation indicators such as SP, SE, PR, ACC, IoU, and DSC. The analysis results were: SP=97.9%, SE=93.8%, PR=90.8%, ACC=97.2%, IoU=85.8%, and DSC=92.5%.

Claims

1. A method for segmenting thyroid glands and nodules based on an improved U-Net network, characterized in that, Includes the following steps: Step 1: Dataset Setup: A professional doctor selects a hospital... a The total number of patients b A series of thyroid ultrasound images were processed and filtered to obtain a final image. d ( d ≤ b A dataset consisting of 100 thyroid ultrasound images S ; S In r ( r<d () as training dataset S 1, S The remaining ones d - r Amount as test dataset S 2; S It contains both benign and malignant samples. Benign samples include inflammatory nodules, cystic nodules, and tumor nodules, while malignant samples include papillary carcinoma and follicular carcinoma. Step 2, Model Building: Improve the U-Net network by introducing ResNeSt module, Atrous Spatial Pyramid Pooling (ASPP), and Deformable Convolution (DC) v3 into the encoder and decoder of the U-Net network for feature extraction, and build a Deformable-Pyramid Split Attention Residual U-Net (DSRU-Net). Step 3, Model Training: Using S 1 r A number of thyroid ultrasound images were used to train the DSRU-Net network model using a loss function and optimizer; During training, data augmentation strategies are employed to enhance samples in real time. These strategies include increasing / decreasing brightness, increasing / decreasing shrinkage, horizontal mirroring, random angle rotation, random cropping, and random stretching. To evaluate the model output and parameters, a weighted cross-entropy loss function is used. L wce and dice loss function L dice Define a new loss function L total The optimized model parameters are then updated to obtain the trained DSRU-Net network. L wce Defined as: （1） in, n For any sample in any batch, N For batch size, c For any category, C For the number of categories, nc For the sample n Category ntc For the sample n The correct category, w ntc For the sample n The correct class weights, x nc For the sample n The predicted value corresponding to the category, x ntc For the sample n The predicted value corresponding to the correct category. t For single-use non-redundant categories, T This represents the total number of non-redundant categories. w t For non-redundant class weights in batch processing, exp (·) is an exponential function; L dice Defined as: （2） in, e It is a relatively small constant, usually taken as 1. y nc Indicates sample n The true value of the category; L total Defined as: L total = L wce + L dice （3）； Step 4, Thyroid gland and nodule segmentation: Using the trained DSRU-Net network, segmentation is performed on the thyroid gland and nodules. S 2 d - r The thyroid gland and nodules in a thyroid ultrasound image were segmented to obtain the segmentation results. R ; Step 5: Segmentation Performance Evaluation: Specificity (SP), Sensitivity (SE), Precision (PR), Accuracy (ACC), Intersection over Union (IoU), and Dice Similarity Coefficient (DSC) are used to evaluate the segmentation performance. R Evaluation and analysis are conducted; they are defined as follows: （4） （5） （6） （7） （8） （9） in, TP It is the number of true positives. TN It is the number of true negatives. FP It is the number of false positives. FN It is the number of false negatives.

2. The method according to claim 1, characterized in that: In step 2, the DSRU-Net network is defined as follows: First, to enhance feature extraction capabilities, a ResNeSt module with good feature extraction tendencies is introduced into the encoder and decoder. The ResNeSt module includes a split-transform-merge structure inherited from GoogleNet and ResNeXt, as well as a channel attention structure inspired by SENet and SKNet. Second, to improve the segmentation capability of targets at different scales, ASPP is introduced between the encoder and decoder to extract multi-scale features. Finally, to improve the model's adaptability to targets with special shapes, DC v3 is introduced into ASPP.

Images