Lung tumor image segmentation method and device based on improved U-Net model, and medium
The improved U-Net model with a coordinate attention module and partial convolution addresses the limitations of existing lung tumor segmentation methods by enhancing spatial feature extraction and attention mechanisms, resulting in more accurate and efficient tumor segmentation.
Patent Information
- Application Number
- CN202411608117.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-11-12
- Publication Date
- 2025-03-14
AI Technical Summary
Current methods for segmenting lung tumors in radiation therapy, such as the U-Net model, face challenges in accurately capturing spatial information due to reduced feature extraction capabilities and limited receptive fields, leading to suboptimal segmentation results, especially with noisy CT images and imbalanced training data.
An improved U-Net model incorporating a coordinate attention module with a partial convolution and skip connections, which generates and embeds attention weights to refine feature extraction, enhancing the model's ability to precisely locate tumors by integrating spatial information.
The proposed method achieves higher segmentation accuracy and robustness by effectively utilizing spatial attention mechanisms, improving the precision of tumor delineation and reducing computational complexity.
Smart Images

Figure CN119624984A_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the technical field of medical image processing, and in particular relates to a lung tumor image segmentation method, device and medium based on an improved U-Net model. Background Art
[0002] Lung cancer is a cancer with a high incidence and mortality rate. Radiotherapy is currently one of the most effective means of treating lung cancer. Accurately outlining the tumor target area during radiotherapy is a key step in achieving precision radiotherapy. In clinical practice, the gross tumor volume (GTV) is the target tumor area for radiotherapy, specifically referring to the range of malignant lesions with a certain shape and size that can be diagnosed using conventional methods, visible, and other tumor target areas, including the clinical target volume (CTV), internal target volume (ITV), and planned target volume (PTV), are delineated on the basis of GTV, taking into account the influence of factors such as potential invaded tissues, organ movement, and positioning errors. Therefore, if the GTV segmentation is inaccurate, it will lead to excessive exposure of the endangered organs and unnecessary damage, or under-irradiation of the tumor area and fail to achieve the ideal treatment effect.
[0003] Currently, during radiotherapy, GTV is mainly manually outlined by doctors after they review the patient's CT images layer by layer. With the development of computer-aided diagnosis technology, the use of image segmentation algorithms to assist doctors in completing the segmentation of tumor areas is of great significance for improving doctors' work efficiency and enabling patients to receive treatment earlier. However, the shape, size, location and relationship of the tumor with surrounding tissues vary from patient to patient, and they behave differently under different scanning conditions. There may be more interference factors such as noise and artifacts in the image, which affect the accuracy of the segmentation results. The Chinese patent application with application number 2024108373672 proposes a method for segmenting lung cancer targets in radiotherapy plans, which equips the U-Net model with parallel skip connections, integrated attention and SE-Res modules, improving the U-Net multi-scale feature processing capabilities and segmentation accuracy. However, in U-Net, the spatial information will be reduced after each convolution, which affects the model's ability to understand the details of the image and thus the final segmentation effect. At the same time, the convolution operation can usually only focus on the information of the local area, and the receptive field is limited. Although the receptive field can be gradually expanded by stacking multiple convolutional layers, this will also increase the complexity and computational complexity of the model. In addition, when using Unet for image segmentation, it is necessary to prepare sufficient and diverse training data. In lung cancer CT image data, the data of different pathological types, stages and patient characteristics are unbalanced, making it difficult for the model to learn a comprehensive feature representation and prone to overfitting on a few categories. In summary, it is urgent to design a new lung tumor image segmentation method to further improve the accuracy of GTV delineation. Summary of the invention
[0004] The purpose of the present invention is to overcome the defects of the above-mentioned prior art and to provide a lung tumor image segmentation method, device and medium based on an improved U-Net model to efficiently and accurately outline the tumor target area.
[0005] The purpose of the present invention can be achieved by the following technical solutions:
[0006] The present invention provides a lung tumor image segmentation method based on an improved U-Net model, comprising the following steps:
[0007] Obtain a lung tumor image to be segmented, input a trained lung tumor image segmentation model based on an improved U-Net model, and obtain a lung tumor image segmentation result;
[0008] Among them, the lung tumor image segmentation model based on the improved U-Net model includes an encoder and a decoder, the encoder is used to downsample the input image step by step through partial convolution to extract features; the decoder is used to upsample the features extracted by the encoder step by step to obtain a segmentation result; the encoder and the decoder are connected through a jump connection, and a coordinate attention module is introduced in the jump connection. The coordinate attention module includes a coordinate attention generation unit and a coordinate information embedding unit. The coordinate attention generation unit is used to generate attention weights in the horizontal and vertical directions according to the features extracted by the partial convolution, and the coordinate information embedding unit is used to correct the features extracted by the partial convolution based on the attention weights in the horizontal and vertical directions.
[0009] Furthermore, the process of generating attention weights in the horizontal and vertical directions according to the features extracted by partial convolution by the coordinate attention generation unit is as follows:
[0010] Get the feature X of size C×H×W extracted by partial convolution, where C, H, and W are the number of channels, the height and width of feature X, respectively. Then, use pooling kernels of size (H, 1) and (1, W) to encode each channel of feature X in the horizontal and vertical directions, respectively, and obtain the vertical and horizontal output features of each channel.
[0011] After the vertical and horizontal output features of each channel are aggregated in the same dimension, a 1×1 convolution F1 is input to change the number of output channels, and then a nonlinear transformation function is passed to obtain a feature f of size C / r×1×(W+H), where r is the partial ratio of the partial convolution;
[0012] Split feature f into vertical tensors f h and the horizontal tensor f w , the number of channels of the two is adjusted to C using the convolution function, and then processed by the Sigmoid activation function to finally obtain the attention weight in the vertical direction and the attention weights in the horizontal direction
[0013] Furthermore, the process of obtaining the vertical output features and the horizontal output features of each channel is as follows:
[0014]
[0015] Among them, x c (h,i) and x c (j,w) represents the input of the cth channel at height h and the input of the cth channel at width w, respectively. and They represent the output of the c-th channel at height h and the output of the c-th channel at width w respectively.
[0016] Furthermore, the partial ratio r of the partial convolution is:
[0017] r=c p / C
[0018] Among them, c p is the channel for spatial feature extraction, and C is the total input channel.
[0019] Furthermore, the process of obtaining feature f is as follows:
[0020]
[0021] Among them, δ is a nonlinear transformation function, and They represent the output of the c-th channel at height h and the output of the c-th channel at width w respectively.
[0022] Furthermore, the vertical attention weight and the attention weights in the horizontal direction The specific expression is as follows:
[0023]
[0024] Among them, σ is the Sigmoid activation function, F h and F w They act on the vertical tensor f h and the horizontal tensor f w The convolution function.
[0025] Furthermore, the coordinate information embedding unit modifies the features extracted by the partial convolution based on the attention weights in the horizontal and vertical directions as follows:
[0026]
[0027] Among them, x c (i,j) and y c (i, j) are the features before and after correction respectively. and are the attention weights in the horizontal and vertical directions, respectively.
[0028] The lung tumor image segmentation model based on the improved U-Net model is further trained by the BCEWithLogitsLoss loss function, and the expression of the loss function is specifically:
[0029] Loss={l1,…,l n ,…,l N}
[0030] l n =-[y n ·log(σ(x n ))+(1-y n )·log(1-σ(x n ))]
[0031] Among them, Loss is the loss function, l n is the loss value of the nth sample, x n and n are the predicted label and true label of the nth sample respectively, and σ is the Sigmoid function.
[0032] The present invention also provides an electronic device, comprising a memory, a processor, and a program stored in the memory, wherein the processor implements the above method when executing the program.
[0033] The present invention also provides a computer-readable storage medium on which a computer program is stored, and the program implements the above method when executed by a processor.
[0034] Compared with the prior art, the present invention has the following beneficial effects:
[0035] The present invention proposes a lung tumor image segmentation method based on an improved U-Net model. The lung tumor image to be segmented is input into a trained lung tumor image segmentation model based on the improved U-Net model to obtain a lung tumor image segmentation result. The lung tumor image segmentation model based on the improved U-Net model includes an encoder and a decoder. The encoder is used to downsample the input image step by step through partial convolution to extract features. By replacing the ordinary convolution of the U-Net encoder part with partial convolution, the feature extraction ability of the model can be improved while reducing the number of parameters. The present invention introduces a coordinate attention module in the jump connection between the encoder and the decoder, including a coordinate attention generation unit and a coordinate information embedding unit. The coordinate attention generation unit is used to generate attention weights in the horizontal and vertical directions according to the features extracted by the partial convolution. The coordinate information embedding unit is used to correct the features extracted by the partial convolution based on the attention weights in the horizontal and vertical directions. Through the above-mentioned coordinate attention module, the model can more accurately obtain the location information of the tumor, thereby improving the segmentation accuracy. BRIEF DESCRIPTION OF THE DRAWINGS
[0036] Figure 1 This is a schematic diagram of the structure of the lung tumor image segmentation model based on the improved U-Net model.
[0037] Among them, the downward arrow represents downsampling, the upward arrow represents upsampling, the darker grayscale rectangle and the lighter grayscale rectangle represent the upsampled image and the image obtained by information enhancement of the image of the corresponding step in the left half, respectively. The numbers on each rectangle represent the number of output channels, the horizontal dotted arrow represents the jump connection, and CA represents the coordinate attention module;
[0038] Figure 2 Schematic diagram of the structure of the coordinate attention module. DETAILED DESCRIPTION
[0039] The present invention is described in detail below in conjunction with the accompanying drawings and specific embodiments. This embodiment is implemented based on the technical solution of the present invention, and provides a detailed implementation method and specific operation process, but the protection scope of the present invention is not limited to the following embodiments.
[0040] Example:
[0041] This embodiment provides a lung tumor image segmentation method based on an improved U-Net model, comprising the following steps:
[0042] Obtain a lung tumor image to be segmented, input the trained lung tumor image segmentation model based on the improved U-Net model, and obtain the lung tumor image segmentation result;
[0043] like Figure 1As shown, the lung tumor image segmentation model based on the improved U-Net model includes an encoder and a decoder, and the encoder and the decoder are connected by a jump connection. Among them, the encoder is used to downsample the input image step by step through partial convolution to extract features, and the decoder is used to upsample the features extracted by the encoder step by step to obtain the segmentation result. In this embodiment, the encoder part of the U-Net network replaces the ordinary convolution with partial convolution (PConv), which can improve the feature extraction ability of the model while reducing the number of model parameters. Partial convolution applies conventional convolution to a part of the input channel for spatial feature extraction, and keeps the remaining channels unchanged. For continuous or regular memory access, the first or last continuous channel is regarded as a representative of the entire feature map for calculation. Without loss of generality, the input and output feature maps have the same number of channels. Therefore, the FLOPs of partial convolution (PConv) are only:
[0044]
[0045] Among them, h represents the height of the feature map, w is the width of the feature map, k is the size of the convolution kernel, and c is the p Indicates the channel for spatial feature extraction. In a typical part ratio r = c p When / c=1 / 2, the FLOPs convolution of PConv is only 1 / 4 of that of ordinary convolution. In addition, PConv has smaller memory access. When r=1 / 2, the memory access is 1 / 2 of that of ordinary convolution. That is:
[0046]
[0047] This embodiment introduces a coordinate attention module in the jump connection, so that the network can more accurately locate the position of the lung tumor while obtaining channel information, thereby improving the segmentation accuracy of the network. The coordinate attention module can embed the position information into the channel attention, not only considering the relationship between channels, but also obtaining the position information of the X-axis and Y-axis, thereby helping the model to more accurately locate the position of the lung tumor.
[0048] The coordinate attention module includes a coordinate attention generation unit and a coordinate information embedding unit. The coordinate attention generation unit is used to generate attention weights in the horizontal and vertical directions according to the features extracted by partial convolution. The specific process is as follows: Figure 2 As shown, the following steps are included:
[0049] (1) Obtain a feature X of size C×H×W extracted by partial convolution, where C, H, and W are the number of channels, the height, and the width of feature X, respectively. Then, use pooling kernels of size (H, 1) and (1, W) to encode each channel of feature X in the horizontal and vertical directions, respectively, and obtain the vertical and horizontal output features of each channel:
[0050]
[0051] Among them, x c (h,i) and x c (j,w) represents the input of the cth channel at height h and the input of the cth channel at width w, respectively. and They represent the output of the c-th channel at height h and the output of the c-th channel at width w respectively.
[0052] (2) After the vertical and horizontal output features of each channel are aggregated in the same dimension, a 1×1 convolution F1 is input to change the number of output channels, and then a nonlinear transformation function is passed to obtain a feature f of size C / r×1×(W+H):
[0053]
[0054] Among them, δ is a nonlinear transformation function, and They represent the output of the c-th channel at height h and the output of the c-th channel at width w respectively.
[0055] r is the partial ratio of the partial convolution, expressed as:
[0056] r=c p / C
[0057] Among them, c p is the channel for spatial feature extraction, and C is the total input channel.
[0058] (3) Split feature f into vertical tensor f h and the horizontal tensor f w , the number of channels of the two is adjusted to C using the convolution function, and then processed by the Sigmoid activation function to finally obtain the attention weight in the vertical direction and the attention weights in the horizontal direction
[0059]
[0060] Among them, σ is the Sigmoid activation function, F h and F w They act on the vertical tensor f h and the horizontal tensor f w The convolution function.
[0061] The coordinate information embedding unit is used to modify some of the features extracted by convolution based on the attention weights in the horizontal and vertical directions:
[0062]
[0063] Among them, x c (i,j) and y c (i, j) are the features before and after correction respectively. and are the attention weights in the horizontal and vertical directions, respectively.
[0064] This embodiment uses the BCEWithLogitsLoss loss function to train a lung tumor image segmentation model based on the improved U-Net model. The expression of the loss function Loss is specifically:
[0065] Loss={l1,…,l n ,…,l N}
[0066] l n =-[y n ·log(σ(x n ))+(1-y n )·log(1-σ(x n ))]
[0067] Among them, l n is the loss value of the nth sample, x n and n are the predicted label and true label of the nth sample respectively, and σ is the Sigmoid function.
[0068] To verify the effectiveness of the above method, this embodiment conducted experiments on a public data set, and analyzed and compared some mainstream methods currently used in medical image segmentation. The experiments were trained and tested in accordance with the experimental regulations of the corresponding data set. The data sources are as follows: The RIDER lung CT data set from the Cancer Imaging Archive website was obtained. The data set provides repeated CT scan images of 32 non-small cell lung cancer (NSCLC) patients on the same day, and the lesion contours annotated by radiologists are used as the reference standard, totaling 15,419 CT images. 100 images were randomly selected as experimental data. The data set was marked in DICOM format and converted into .png format first. The image pixel size was uniformly adjusted to 512×512, and the data was divided into training set, validation set, and test set according to 6:2:2. The experimental results are shown in Table 1.
[0069] Table 1 Experimental results of RIDER lung CT dataset
[0070] Dice IoU Recall U-Net 64.25 53.71 65.73 Attention U-Net 65.31 54.14 68.51 Mobile U-Net 65.05 54.19 65.82 FCN 66.68 54.00 64.65 Method of the present invention 70.04 56.63 71.23
[0071] As the most widely used network in the field of medical image segmentation, U-Net has achieved good results in multiple segmentation tasks. Subsequent segmentation networks are improved on the basis of U-Net, so it is used as the benchmark network. Attention U-Net improves the sensitivity of the model to the target area by adding the attention mechanism to U-Net. Mobile U-Net uses the lightweight MobileNetV2 as the feature extraction network, which contains separable convolutions of multiple sizes and depths, which reduces the number of network parameters while improving the network feature extraction capability. FCN is the earliest semantic segmentation network proposed. By replacing the last fully connected layer of CNN with a convolutional layer, pixel-level classification is achieved. By comparing the data in the table, it can be seen that the method of the present invention is superior to other algorithms in all three indicators. It can be seen that the coordinate attention module and partial convolution introduced by the method of the present invention can ensure a higher segmentation integrity and improve the segmentation accuracy.
[0072] If the above method is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program codes.
[0073] The above description of the embodiments is to facilitate the understanding and use of the invention by those skilled in the art. It is obvious that those skilled in the art can easily make various modifications to these embodiments and apply the general principles described herein to other embodiments without creative work. Therefore, the present invention is not limited to the above embodiments, and improvements and modifications made by those skilled in the art based on the disclosure of the present invention without departing from the scope of the present invention should be within the scope of protection of the present invention.
Claims
1. A lung tumor image segmentation method based on an improved U-Net model, characterized in that: The following steps are involved: Obtain a lung tumor image to be segmented, input a trained lung tumor image segmentation model based on an improved U-Net model, and obtain a lung tumor image segmentation result; The lung tumor image segmentation model based on the improved U-Net model includes an encoder and a decoder, wherein the encoder is used to downsample the input image step by step through partial convolution to perform feature extraction; The decoder is used to upsample the features extracted by the encoder step by step to obtain a segmentation result; the encoder and the decoder are connected via a jump connection, a coordinate attention module is introduced into the jump connection, the coordinate attention module includes a coordinate attention generation unit and a coordinate information embedding unit, the coordinate attention generation unit is used to generate attention weights in the horizontal and vertical directions according to the features extracted by partial convolution, and the coordinate information embedding unit is used to correct the features extracted by partial convolution based on the attention weights in the horizontal and vertical directions.
2. The lung tumor image segmentation method based on the improved U-Net model according to claim 1, characterized in that: The process of generating attention weights in the horizontal and vertical directions according to the features extracted by the partial convolution by the coordinate attention generation unit is as follows: Get the feature X of size C×H×W extracted by partial convolution, where C, H, and W are the number of channels, the height and width of feature X, respectively. Then, use pooling kernels of size (H, 1) and (1, W) to encode each channel of feature X in the horizontal and vertical directions, respectively, and obtain the vertical and horizontal output features of each channel. After the vertical and horizontal output features of each channel are aggregated in the same dimension, a 1×1 convolution F1 is input to change the number of output channels, and then a nonlinear transformation function is passed to obtain a feature f of size C / r×1×(W+H), where r is the partial ratio of the partial convolution; Split feature f into vertical tensors f h and the horizontal tensor f w , the number of channels of the two is adjusted to C using the convolution function, and then processed by the Sigmoid activation function to finally obtain the attention weight in the vertical direction and the attention weights in the horizontal direction 3. The lung tumor image segmentation method based on the improved U-Net model according to claim 2, characterized in that: The process of obtaining the vertical output features and horizontal output features of each channel is as follows: Among them, x c (h,i) and x c (j,w) represents the input of the cth channel at height h and the input of the cth channel at width w, respectively. and They represent the output of the c-th channel at height h and the output of the c-th channel at width w respectively.
4. The lung tumor image segmentation method based on the improved U-Net model according to claim 2, characterized in that: The partial ratio r of the partial convolution is: r=c p / C Among them, c p is the channel for spatial feature extraction, and C is the total input channel.
5. The lung tumor image segmentation method based on the improved U-Net model according to claim 2, characterized in that: The process of obtaining feature f is as follows: Among them, δ is a nonlinear transformation function, and They represent the output of the c-th channel at height h and the output of the c-th channel at width w respectively.
6. The lung tumor image segmentation method based on the improved U-Net model according to claim 2, characterized in that: Vertical attention weight and the attention weights in the horizontal direction The specific expression is as follows: Among them, σ is the Sigmoid activation function, F h and F w They act on the vertical tensor f h and the horizontal tensor f w The convolution function.
7. The lung tumor image segmentation method based on the improved U-Net model according to claim 1, characterized in that: The coordinate information embedding unit modifies the features extracted by some convolutions based on the attention weights in the horizontal and vertical directions as follows: Among them, x c (i,j) and y c (i, j) are the features before and after correction respectively, and are the attention weights in the horizontal and vertical directions, respectively.
8. The lung tumor image segmentation method based on the improved U-Net model according to claim 1, characterized in that: The lung tumor image segmentation model based on the improved U-Net model is trained by the BCEWithLogitsLoss loss function, and the expression of the loss function is specifically: Loss={l1,…,l n ,…,l N } l n =-[y n ·log(σ(x n ))+(1-y n )·log(1-σ(x n ))] Among them, Loss is the loss function, l n is the loss value of the nth sample, x n and n are the predicted label and true label of the nth sample respectively, and σ is the Sigmoid function.
9. An electronic device comprising a memory, a processor, and a program stored in the memory, characterized in that: When the processor executes the program, the method according to any one of claims 1 to 8 is implemented.
10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the method according to any one of claims 1 to 8 is implemented.
