A lung nodule segmentation method based on a U-Net model

Abstract

The application discloses a lung nodule segmentation method based on a U-Net model, and the lung nodule region of a lung CT image can be effectively segmented through the improved model; the improved lung nodule segmentation model is based on a U-Net model and comprises an encoder, a decoder, a residual block and an improved ASPP module; the BN layer, the Dropout layer and the residual block are added to the improved lung nodule segmentation model, thereby effectively solving problems such as overfitting, gradient explosion and gradient disappearance; the ASPP module is improved by adding the convolution layer, the CBAM convolution attention module and increasing the number of parallel hollow convolution layers, so that the receptive field is enlarged, multi-scale context information can be effectively captured, the resolution can be ensured while the receptive field is enlarged, the weight of the improved network channel is modified, the model can better pay attention to the information of important channels, and important features can be strengthened to learn; experiment results show that, compared with the U-Net original segmentation model, the application effectively improves the accuracy of lung nodule segmentation.

Description

Technical Field

[0001] This invention relates to the field of medical image processing technology, and in particular to an improved U-Net lung nodule segmentation method. Background Technology

[0002] In recent years, with the continuous development of deep learning technology, it has achieved remarkable results in medical image processing. Traditional segmentation methods can no longer meet the increasingly large and complex medical image segmentation tasks, while deep learning can be applied to the field of medical image segmentation and recognition. Compared with doctors' experience-based judgment, it has higher recognition accuracy and has become a hot research topic in the current medical field.

[0003] Lung cancer is one of the cancers with high incidence and mortality rates, and its incidence has been steadily increasing in recent years. Because early-stage lung cancer often presents with no obvious symptoms, it is frequently overlooked by patients. By the time noticeable symptoms appear, the cancer cells have already spread and reached the late stage of lung cancer. Early-stage lung cancer primarily manifests as pulmonary nodules. Screening for these nodules is the most effective way to prevent and treat lung cancer, significantly reducing mortality and treatment costs.

[0004] Computed tomography (CT) is a high-precision method for screening and analyzing lung nodules. Compared to the entire lung in a CT image, lung nodules occupy only a small area and vary in shape, making accurate segmentation of lung nodules quite challenging. Furthermore, radiologists are often influenced by subjective judgment and experience when segmenting lung nodules, especially under conditions of visual fatigue, which can easily lead to visual misjudgments. Utilizing segmentation techniques in image processing to assist doctors in diagnosing lung nodules can reduce their workload, improve segmentation efficiency, provide a basis for early lung cancer screening, and ultimately improve public health.

[0005] Medical image segmentation is the process of dividing a medical image into several non-overlapping "connected" regions based on certain similarity features (such as brightness, color, texture, area, shape, location, local statistical features, or spectral features). The relevant features show consistency or similarity in the same region, but show obvious differences in different regions. In other words, there is a certain discontinuity of pixels at the region boundaries.

[0006] U-Net (UnityNetworking) was originally designed to solve problems in biomedical images, and has since been widely applied in various areas of semantic segmentation. U-Net is an excellent semantic segmentation model, and its main execution process is similar to other semantic segmentation models. However, unlike Convolutional Neural Networks (CNNs), which perform image-level classification, U-Net performs pixel-level classification, outputting the category of each pixel.

[0007] However, the U-Net model suffers from problems during segmentation, such as vanishing gradients, loss of spatial information, and low feature utilization. Furthermore, as the network depth increases, it becomes prone to overfitting. In addition, the U-Net model does not fully utilize feature information between different stages, leading to incomplete feature utilization. These issues negatively impact the accuracy of U-Net segmentation. Therefore, improving the accuracy of U-Net model in segmenting lung nodules is crucial.

[0008] To address the aforementioned issues, this invention proposes a lung nodule segmentation method based on the U-Net model, thereby improving the accuracy of lung nodule segmentation. Summary of the Invention

[0009] The purpose of this invention is to provide a lung nodule segmentation method based on U-Net network to solve the problems of gradient vanishing, gradient exploding, easy loss of spatial information, low feature utilization and overfitting in the U-Net network mentioned above.

[0010] According to the present invention, a lung nodule segmentation method based on U-Net network is proposed. By constructing an improved U-Net network model, the lung nodule region in lung CT images can be effectively segmented. The improved lung nodule segmentation model is based on the U-Net model and includes an encoder, a decoder, a residual module and an improved ASPP module.

[0011] This invention provides a lung nodule segmentation method based on U-Net network, comprising the following steps:

[0012] S1: Read lung CT images and preprocess the CT images;

[0013] S2: Cut out the lung nodule image and generate the corresponding mask;

[0014] S3: Divide the dataset according to the acquired lung nodule images and corresponding masks to obtain training and test sets;

[0015] S4: Construct the improved U-Net network structure;

[0016] S5: Input the obtained training set into the improved U-Net network for training;

[0017] S6: Input the CT image of the lung nodule to be segmented into the trained U-Net improved network to obtain the segmentation result of the lung nodule.

[0018] Preferably, in step S1, lung CT images of a specified format are read from the dataset, and the original lung CT images are preprocessed using methods such as binarization, image edge filling, histogram correction, grayscale transformation, image smoothing, Gaussian filtering, cropping, image translation, thresholding, morphology, and data augmentation.

[0019] Preferably, in step S2, the portion of the lung CT image containing lung nodules is cut out, and a mask image corresponding to the lung nodules is generated.

[0020] Preferably, in step S3, the dataset is divided according to the acquired lung nodule images and corresponding masks to obtain a training set and a test set, wherein the training set accounts for 80% and the test set accounts for 20%.

[0021] Preferably, in step S4, to facilitate model design and the fusion of shallow and deep features, this invention replaces the "padding=valid" convolution in the original U-Net network with "padding=same" convolution. Using this setting, the feature map obtained after convolution processing will maintain the same dimension as the original input. Furthermore, to accelerate model convergence and avoid gradient vanishing and exploding problems, this invention connects a BN (Batch Normalization) layer after each convolution operation. Finally, to prevent overfitting and improve network performance, this invention designs a Dropout layer with a 30% deactivation rate, improving upon the original U-Net network's 3x3 convolutional layer and ReLU layer: Dropout The Dropout layer follows the 3x3 convolution of the original U-Net network, followed by a BN layer, and finally a ReLU layer. Using a residual structure, the input information is directly passed to the output, reducing information loss and mitigating the gradient vanishing problem, thus allowing the network model to converge faster. The BN layer accelerates network training and convergence, controls gradient explosion, prevents gradient vanishing, and prevents overfitting. The Dropout layer works by randomly deactivating some feature detection units with a certain probability during model training, effectively training multiple models. This results in a more generalizable and robust model, effectively avoiding overfitting. These improvements effectively address overfitting, gradient explosion, and gradient vanishing issues.

[0022] Preferably, in step S4, adding a BN layer, a Dropout layer, and a residual block method to the original U-Net network, and constructing an improved U-Net network structure by improving the ASPP module, specifically:

[0023] Input image → First residual double convolutional module → First 2x2 max pooling layer → Second residual double convolutional module → Second 2x2 max pooling layer → Third residual double convolutional module → Third 2x2 max pooling layer → Fourth residual double convolutional module → Fourth 2x2 max pooling layer → Improved dilated convolutional pyramid module (improved ASPP module) → 3x3 convolutional combination → First 2x2 upsampling layer → First residual double convolutional stitching module → Second 2x2 upsampling → Second residual double convolutional stitching module → Third 2x2 upsampling → Third residual double convolutional stitching module → Fourth 2x2 upsampling → Fourth residual double convolutional stitching module → 1x1 convolution → Segmentation map.

[0024] Preferably, in step S4, the residual double convolution module is specifically configured as follows:

[0025] First, a 3x3 convolution combination is used to obtain a feature map. Then, the obtained feature map is used to obtain a new feature map through another 3x3 convolution combination. The two 3x3 convolution combinations are then combined with residual blocks to form a whole. The above 3x3 convolution combination is composed of a 3×3 convolution, a BN layer, a Dropout layer and a ReLU layer connected in sequence.

[0026] Preferably, in step S4, the residual double convolutional stitching module is specifically configured as follows:

[0027] First, a skip connection is used to merge the left encoding path and the right decoding path together through a superposition operation. Then, a 3x3 convolution combination is used to obtain a feature map. Then, the obtained feature map is used to obtain a new feature map through another 3x3 convolution combination. The two 3x3 convolution combinations are then combined with residual blocks to form a whole. The above 3x3 convolution combination is composed of a 3×3 convolution, a BN layer, a Dropout layer and a ReLU layer connected in sequence. The 3x3 convolution combination in the residual double convolution module and the residual double convolution concatenation module has the same composition.

[0028] The skip connection merges the left encoding path and the right decoding path by superposition, that is, it merges the low-level feature information obtained from the encoding path with the high-level feature information obtained from the decoding path. The feature maps at both ends of the skip connection have the same number of channels and resolution. Therefore, the fused feature map has both shallow feature information and deep semantic feature information, which is conducive to obtaining more accurate segmentation results.

[0029] Preferably, in step S4, the improved ASPP module is specifically configured as follows:

[0030] The original ASPP had 5 branches: 1×1 convolution, 3×3 convolution with 6, 12, and 18 dilatations, and 1x1 global pooling. Here, we increase the number of parallel dilated convolution layers from 3 to 6. This allows us to obtain different receptive fields and convolutional blocks through convolution, with each layer essentially operating on different feature blocks from the same origin, which is more beneficial for subsequent semantic fusion and supplementation. Adding 1×1 convolutions and 1x1 global pooling further reduces the total to 8 branches. Furthermore, convolutional blocks are added before 7 of these branches (except for the 1x1 global pooling layer). In the process, after a 1x1 global pooling layer, the information is then fused with the information from the other 7 branches through a 1x1 convolution and upsampling operation. In addition, CBAM (Convolutional Block Attention Module) is introduced to improve the weights of the network channels, enhance the learning of important features, improve information utilization, and strengthen feature fusion. After extracting the features from the 8 branches, a 1×1 convolution is used to adjust the number of channels, resulting in a feature block with very rich semantic information.

[0031] Preferably, in step S5, when training the network, DiceLoss is used as the loss function, Stochastic Gradient Descent (SGD) is used as the optimization function, and the Crossover Union Ratio (CUPR) is used as the evaluation metric for the model.

[0032] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0033] 1. This invention adds Batch Normalization (BN) layers, Dropout layers, and residual blocks to each block of the original U-Net network. The residual structure directly transmits input information to the output, reducing information loss and mitigating the gradient vanishing problem, thus enabling the constructed network model to converge faster. The BN layer can accelerate the training and convergence speed of the network, control gradient explosion, prevent gradient vanishing, and prevent overfitting. The Dropout layer randomly deactivates some feature detection units with a certain probability, resulting in a more generalizable and robust model that effectively avoids overfitting. Through the above improvements, the problems of overfitting, gradient explosion, and gradient vanishing in the original U-Net network model can be effectively solved.

[0034] 2. While traditional downsampling can increase the receptive field, it reduces spatial resolution. In the original U-Net network, by adding an improved ASPP module, not only is the receptive field expanded, but multi-scale contextual information can also be effectively captured. The improved ASPP module also introduces a CBAM convolutional attention module, which can expand the receptive field while maintaining resolution. In addition, it can improve the weights of network channels, enabling the model to better focus on the information of important channels, strengthen the learning of important features, improve information utilization, and fully fuse the extracted information, thereby improving segmentation accuracy. Attached Figure Description

[0035] Figure 1 This is a schematic diagram of the method flow described in this invention;

[0036] Figure 2 The overall framework diagram for this invention, based on the improved U-Net network;

[0037] Figure 3 This is a schematic diagram of the improved ASPP module structure of the present invention;

[0038] Figure 4 This is a structural diagram of the residual unit used in this invention;

[0039] Figure 5 This is a structural diagram of the CBAM convolutional attention module used in this invention. Detailed Implementation

[0040] The technical solutions of the present invention will be clearly and thoroughly described below with reference to the accompanying drawings of the embodiments of the present invention. However, these embodiments do not limit the present invention. It should be noted that the following embodiments are only used to illustrate specific embodiments of the present invention and are not all embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the protection scope of the present invention.

[0041] like Figure 1 As shown, this invention provides an improved U-Net lung nodule segmentation method, comprising the following steps:

[0042] Step 1: Obtain a dataset of labeled lung CT scans from professional hospitals and doctors;

[0043] Step 2: Read the lung CT dataset and the doctor's annotation file, cut out the part with lung nodules in the CT image through the annotation information, and generate the mask image corresponding to the lung nodules through the edge segmentation information;

[0044] Step 3: Divide the images captured in Step 2 into a dataset, with the training set accounting for 80% and the test set accounting for 20%.

[0045] Step 4: Build the improved U-Net network model;

[0046] Step 5: Input the training set divided in Step 3 into the network constructed in Step 4 for model training;

[0047] Step 6: Input the test set into the network trained in Step 5 to obtain the lung nodule segmentation result image.

[0048] In step one, the dataset contains 800 patients, with each case having approximately 200-400 CT images, which have been annotated by professional doctors. Each CT image is 512x512 pixels in size.

[0049] In step two, the part containing lung nodules is cropped into a 64x64 image centered on the nodule based on the location information marked in the dataset, and then a corresponding mask image is generated based on the nodule contour information.

[0050] In step three, the dataset is divided according to the acquired lung nodule images and corresponding masks to obtain a training set and a test set, with the training set accounting for 80% and the test set accounting for 20%.

[0051] In step four, a BN layer, a Dropout layer, and a residual block method are added to the original U-Net network, and an improved U-Net network structure is constructed by improving the ASPP module. Figure 2 As shown, specifically:

[0052] Block 1: Input a 512x512 lung CT image to be tested. After passing through a 3x3 convolution combination, a 64-channel feature map is obtained. Then, the obtained feature map is passed through another 3x3 convolution combination to obtain a new 512x512x64 (width, height, number of channels) feature map. Subsequently, the new 64-channel feature map passes through the first 2x2 max pooling layer. In addition, a residual block is introduced in the second 3x3 convolution combination. The residual block starts at the second 3x3 convolution combination and ends at the first 2x2 max pooling layer. The above 3x3 convolution combination is composed of a 3×3 convolution, a BN layer, a Dropout layer and a ReLU layer connected in sequence. The structure of the following 3x3 convolution combinations is the same as the above 3x3 convolution combination and will not be repeated.

[0053] Block 2: After the first 2x2 max pooling layer, a 256x256x64 feature map is obtained. Then, after two 3x3 convolutional combinations, a 256x256x128 feature map is obtained. Subsequently, the new 128-channel feature map goes through a second 2x2 max pooling layer. In addition, there is a branch in the first 3x3 convolutional combination that introduces a residual block. The residual block starts at the first 3x3 convolutional combination and ends at the second 2x2 max pooling layer.

[0054] Block 3: After the second 2x2 max pooling layer, a 128x128x128 feature map is obtained. Then, after two 3x3 convolutional combinations, a 128x128x256 feature map is obtained. Subsequently, the new 256-channel feature map goes through a third 2x2 max pooling layer. In addition, there is a branch in the first 3x3 convolutional combination that introduces a residual block. The residual block starts from the first 3x3 convolutional combination and ends at the third 2x2 max pooling layer.

[0055] Block 4: After the third 2x2 max pooling layer, a 64x64x256 feature map is obtained. Then, after two 3x3 convolutional combinations, a 64x64x512 feature map is obtained. Subsequently, the new 512-channel feature map goes through a fourth 2x2 max pooling layer. In addition, there is a branch in the first 3x3 convolutional combination that introduces a residual block. The residual block starts from the first 3x3 convolutional combination and ends at the fourth 2x2 max pooling layer.

[0056] Block 5: After the fourth 2x2 max pooling layer, the map passes through an improved ASPP module, followed by a 3x3 convolution combination to obtain a 32x32x1024 feature map. Then, it passes through the first 2x2 upsampling layer. The original ASPP had five branches: 1x1 convolution, 3x3 convolution with 6, 12, and 18 dilatations, and 1x1 global pooling. The improved ASPP module increases the number of parallel dilated convolution layers from 3 to 6. This allows for the acquisition of different receptive fields and convolutional blocks through convolution, with each layer essentially operating on different feature blocks from the same origin, which is more beneficial for subsequent processing. The system incorporates semantic fusion and supplementation, along with 1×1 convolutions and 1×1 global pooling, resulting in a total of 8 branches. Convolutional blocks are added before 7 of these branches (except for the 1×1 global pooling layer). After the 1×1 global pooling layer, a 1×1 convolution and upsampling operation are performed, and the resulting information is then fused with the information from the other 7 branches. Furthermore, a CBAM convolutional attention module is introduced to improve the weights of the network channels, enhance the learning of important features, improve information utilization, and strengthen feature fusion. After extracting features from the 8 branches, a 1×1 convolution is used to adjust the number of channels, resulting in a feature block with very rich semantic information.

[0057] Block 6: After the first 2x2 upsampling layer, the resulting feature map is skipped to the feature map obtained from the second 3x3 convolution combination in Block 4, resulting in a 64x64x1024 feature map. After two more 3x3 convolution combinations, a 64x64x512 feature map is obtained. Subsequently, the new 512-channel feature map passes through the second 2x2 upsampling layer. In addition, there is a branch in the first 3x3 convolution combination that introduces a residual block. The residual block starts at the first 3x3 convolution combination and ends at the second 2x2 upsampling layer.

[0058] Block 7: After the second 2x2 upsampling layer, the resulting feature map is skipped and connected to the feature map obtained from the second 3x3 convolution combination in Block 3, resulting in a 128x128x512 feature map. After two more 3x3 convolution combinations, a 128x128x256 feature map is obtained. Subsequently, the new 256-channel feature map passes through a third 2x2 upsampling layer. In addition, there is a branch in the first 3x3 convolution combination that introduces a residual block. The residual block starts at the first 3x3 convolution combination and ends at the third 2x2 upsampling layer.

[0059] Block 8: After the third 2x2 upsampling layer, the resulting feature map is skipped to the feature map obtained from the second 3x3 convolution combination in Block 2, resulting in a 256x256x256 feature map. After two more 3x3 convolution combinations, a 256x256x128 feature map is obtained. Subsequently, the new 128-channel feature map goes through a fourth 2x2 upsampling layer. In addition, there is a branch in the first 3x3 convolution combination that introduces a residual block. The residual block starts at the first 3x3 convolution combination and ends at the fourth 2x2 upsampling layer.

[0060] Block 9: After the fourth 2x2 upsampling layer, two 3x3 convolutions are combined to obtain a 512x512x64 feature map. Then, a 1x1 convolution is performed to obtain a 512x512x1 feature map.

[0061] In step 5, when training the network, DiceLoss is used as the loss function, Stochastic Gradient Descent (SGD) is used as the optimization function, and the Crossover Union Ratio (CUPR) is used as the evaluation metric for the model.

[0062] In step 6, the test set is input into the trained network model to view the segmentation results.

[0063] To facilitate understanding of this invention patent, a brief introduction to residual blocks and CBAM is provided here, such as... Figure 4 As shown, the residual block is divided into two parts: the direct mapping part and the residual part. Figure 4In convolutional networks, "weight" refers to the convolution operation. CBAM (Convolutional Block Attention Module) is a lightweight convolutional attention module that combines channel and spatial attention mechanisms. Figure 5 As can be seen, CBAM contains two sub-modules: CAM (Channel Attention Module) and SAM (Spartial Attention Module), which perform channel and spatial attention respectively. This not only saves parameters and computing power, but also ensures that it can be integrated into existing network architectures as a plug-and-play module.

[0064] After training, the segmentation performance of the traditional U-Net network model and the improved U-Net network model of this invention were evaluated. The cross-over ratio (COR) of the traditional U-Net network model was 0.88, while that of the improved U-Net network model of this invention was 0.95. Compared with the traditional U-Net network model, the segmentation accuracy of the improved U-Net network model of this invention was improved by 8%.

[0065] The above description is merely a specific embodiment of the invention, but the scope of protection of the invention is not limited thereto. Any changes or substitutions conceived without inventive effort should be included within the scope of protection of the invention. Therefore, the scope of protection of the invention should be determined by the scope defined in the claims.

Claims

1. An improved U-Net method for lung nodule segmentation, characterized in that, Includes the following steps: Step 1: Read lung CT images and preprocess the CT images; Step 2: Cut out the lung nodule image and generate the corresponding mask; Step 3: Divide the dataset according to the acquired lung nodule images and corresponding masks to obtain training and test sets; Step 4: Construct the improved U-Net network structure; Step 5: Input the obtained training set into the improved U-Net network for training; Step 6: Input the CT image of the lung nodule to be segmented into the trained U-Net improved network to obtain the segmentation result of the lung nodule; In step four, the constructed improved U-Net network model structure is as follows: Input image → First residual double convolutional module → First 2x2 max pooling layer → Second residual double convolutional module → Second 2x2 max pooling layer → Third residual double convolutional module → Third 2x2 max pooling layer → Fourth residual double convolutional module → Fourth 2x2 max pooling layer → Improved dilated convolutional pyramid module → 3x3 convolutional combination → First 2x2 upsampling layer → First residual double convolutional concatenation module → Second 2x2 upsampling → Second residual double convolutional concatenation module → Third 2x2 upsampling → Third residual double convolutional concatenation module → Fourth 2x2 upsampling → Fourth residual double convolutional concatenation module → 1x1 convolution → Segmentation map. The original dilated convolutional pyramid module had 5 branches: 1×1 convolution, 3×3 convolution with 6, 12, and 18 dilatations, and 1x1 global pooling. The improved dilated convolutional pyramid module increases the number of parallel dilated convolutional layers from 3 to 6, and adds 1×1 convolution and 1x1 global pooling, resulting in a total of 8 branches. Convolutional blocks are added before the 7 branches except for the 1x1 global pooling layer. After the 1x1 global pooling layer, a 1x1 convolution and upsampling operation are performed, and the information obtained is then fused with the information from the other 7 branches. In addition, CBAM is introduced to improve the weights of the network channels. After extracting the features of the 8 branches, the number of channels is adjusted using 1×1 convolution. CBAM is a convolutional attention module.

2. The improved U-Net lung nodule segmentation method according to claim 1, characterized in that, In step one, lung CT images in a specified format are read from the dataset, and the original lung CT images are preprocessed using binarization, image edge filling, histogram correction, grayscale transformation, thresholding, morphology, and data augmentation methods.

3. The improved U-Net lung nodule segmentation method according to claim 1, characterized in that, In step two, the portion of the lung CT image containing lung nodules is cut out, and a mask image corresponding to the lung nodules is generated.

4. The improved U-Net lung nodule segmentation method according to claim 1, characterized in that, In step three, the dataset is divided based on the acquired lung nodule images and corresponding masks to obtain a training set and a test set, with the training set accounting for 80% and the test set accounting for 20%.

5. The improved U-Net lung nodule segmentation method according to claim 1, characterized in that, In step five, during network training, Dice Loss is used as the loss function, stochastic gradient descent (SGD) is used as the optimization function, and crossover ratio (CRO) is used as the model evaluation metric.

Images