A COVID-19 lesion segmentation method based on nested U-shaped network

Abstract

This paper presents a COVID-19 lesion segmentation method based on a nested U-shaped network. This method constructs a COVID-19 segmentation network using RSUs as basic units and is trained on a small number of labeled samples. The network introduces edge and semantic information to enhance its ability to characterize COVID-19. This network can accurately and quickly segment lesions. Furthermore, when labeled images are extremely limited, a semi-supervised learning strategy is used to fully utilize a large amount of unlabeled image data, further improving the model's performance.

Description

Technical Field

[0001] The present invention relates to a COVID-19 lesion segmentation method based on a nested U-shaped network, belonging to the technical field of lesion segmentation. Background Art

[0002] Computed tomography (CT) can help medical staff identify COVID-19 early, accurately, and promptly, and is of great significance for improving the efficiency of COVID-19 case treatment. However, manual diagnosis of medical images relies too much on professional knowledge, takes a long time to analyze, has difficulty detecting hidden lesions in the early stages, and is difficult to distinguish from other viral pneumonias and bacterial pneumonias. In addition, with the continuous development of COVID-19, the number of images to be analyzed has increased dramatically, making manual segmentation impractical in clinical practice. Fortunately, the use of artificial intelligence technology can convert visual image information into deep feature information, which helps reduce manual operations and improve the efficiency of precise quantitative analysis. Therefore, the COVID-19 lesion segmentation model can effectively assist medical experts in performing efficient and accurate analysis of COVID-19.

[0003] Currently, the segmentation of COVID-19 lesions has become a research focus both domestically and internationally. Existing deep learning methods for segmenting CT images of COVID-19 patients have limitations: first, insufficient labeled image samples make the model prone to overfitting; second, COVID-19-related imaging features exhibit large variations in texture, size, and position, resulting in poor performance of existing lesion segmentation models. Therefore, to overcome these shortcomings, an improved COVID-19 lesion segmentation model based on a nested U-shaped network is proposed. This model improves lesion segmentation accuracy and more effectively assists experts in diagnosis. Summary of the Invention

[0004] The purpose of the present invention is to address the shortcomings of existing methods and provide a COVID-19 lesion segmentation method based on a nested U-shaped network. The model can learn the visual features of COVID-19 related lesions from a small number of labeled computed tomography (CT) images to achieve automatic segmentation of infected areas. In addition, in order to alleviate the shortcomings of labeled images, a semi-supervised segmentation framework based on a random selection propagation strategy is proposed. The framework can effectively utilize more unlabeled images, thereby improving the learning ability of the model and obtaining higher segmentation performance. In order to quantitatively evaluate the segmentation effect of the technical solution, six indicators are used, including precision (Prec), recall (Recall), dice similarity coefficient (DSC), structural metric (S m ), enhanced alignment metric And the mean absolute error (MAE), which is calculated as follows:

[0005]

[0006]

[0007]

[0008] S m =α·S o (P,G)+(1-α)·S r (P,G) (4)

[0009]

[0010]

[0011] Among them, TP, FP, and FN are true positive, false positive, and false negative, respectively; P is the predicted image, and G is the true value image; Sr represents the region-aware structural similarity measurement, So represents the target-aware structural similarity measurement, and α is empirically taken as 0.5; h and w represent the height and width of the input image, respectively, and (x, y) represents the corresponding coordinate positions of the predicted image and the true value image.

[0012] To achieve the above object, the technical solution of the present invention is:

[0013] A COVID-19 lesion segmentation method based on a nested U-shaped network includes the following steps:

[0014] Step 1: Using RSU as the basic unit, construct a U-shaped network structure to construct a backbone network with a nested U-shaped network;

[0015] Step 2: Introduce the edge supervision module to capture rich spatial detail information;

[0016] Step 3: Introduce a high-level semantic enhancement module to learn the location information of the lesion area;

[0017] Step 4: In the decoder part, the multi-scale features of different levels in the decoder stage are fused through the attention fusion module;

[0018] Step 5: Segment the input COVID-19 image based on the fusion results of the multi-scale feature maps;

[0019] Step 6: Further introduce semi-supervised strategy learning into the model M obtained by the above process;

[0020] Step 7: Add unlabeled data and randomly select them according to the ratio of label 7 to unlabel 3;

[0021] Step 8: Use model M to predict the amplified data in step 7 and generate pseudo labels;

[0022] Step 9: Merge the pseudo-labeled and initial labeled data into a new labeled dataset. At the same time, remove the randomly selected data from the unlabeled dataset.

[0023] Step 10: Retrain model M on the new labeled dataset to obtain the iterated model. Step 11: Repeat steps 7-10 until the unlabeled dataset is empty.

[0024] Step 12: Get the final model M'.

[0025] The RSU in step 1 specifically includes three parts:

[0026] An input convolution layer that converts the input feature map into an intermediate layer feature map with the number of channels Cout;

[0027] A U-shaped symmetric codec structure, where L is the number of codec layers;

[0028] Residual connection that fuses local features and multi-scale features through addition operation;

[0029] The present invention proposes a model design based on the residual U-block (RSU).

[0030] The existing residual U-block (RSU) is briefly introduced as follows:

[0031] Inspired by U-Net, a backbone network with RSU as the basic unit is constructed to obtain more contextual information. The RSU is used to capture internal multi-scale receptive field features. The structure (Cin, M, Cout) mainly consists of three parts: 1) an input convolutional layer that converts the input feature map (H×W×Cin) into an intermediate layer feature map with the number of channels Cout; 2) a U-shaped symmetric encoder-decoder structure, where L is the number of encoder-decoder layers; and 3) a residual connection that fuses local features and multi-scale features through addition operations. This design can mix receptive fields of different sizes in the RSU, thereby capturing more contextual information. In addition, a larger number of layers L can form deeper RSUs and more pooling operations, resulting in richer local and global features while also avoiding an increase in computational cost.

[0032] The overall network structure is described as follows. First, by introducing edge and semantic information in the encoding stage, the network's ability to represent COVID-19 is jointly enhanced. Here, considering that the low-level features have high resolution, they can capture rich spatial detail information. Therefore, the "edge supervision module" is designed to integrate the corresponding features into the S1~S2 stages of the figure to highlight the low-level edge features. In addition, high-level semantic features contain position information, so the "high-level semantic enhancement module" is designed to merge the corresponding features into the S3~S5 stages of the figure. Subsequently, the "attention fusion module" is designed to fuse the multi-scale features of different levels in the decoding stage. Finally, the input COVID-19 image is segmented based on the fusion result of the obtained multi-scale feature map.

[0033] This paper constructs a COVID-19 segmentation network using RSUs as basic units and training them on a small number of labeled samples. The network introduces edge and semantic information to enhance its ability to characterize COVID-19. This network can accurately and quickly segment lesion areas. Furthermore, in the case of extremely limited labeled images, a semi-supervised learning strategy fully utilizes a large amount of unlabeled image data, further improving the model's performance. Therefore, this model can effectively assist medical experts in performing efficient and accurate analysis of COVID-19. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0035] Figure 1 It is the RSU structure of the COVID-19 lesion segmentation method based on the nested U-shaped network of the present invention;

[0036] Figure 2 This is the overall block diagram of the nested U-shaped network of the COVID-19 lesion segmentation method based on the nested U-shaped network of the present invention.

[0037] Figure 3 Comparison of visual effects corresponding to different modules configured in the COVID-19 lesion segmentation method based on nested U-shaped network of the present invention, where (a) is the image input, (b) is the true value map, (c) is the complete structure of the technical solution, (d) is not configured with attention fusion, (e) is not configured with high-level semantic enhancement, and (f) is not configured with edge supervision. DETAILED DESCRIPTION

[0038] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0039] A COVID-19 lesion segmentation method based on a nested U-shaped network includes the following steps:

[0040] The present invention provides a COVID-19 lesion segmentation network with a nested U-shaped network, comprising the following steps:

[0041] Step 1: Take RSU as the basic unit and construct it with a U-shaped network structure, that is, construct a backbone network with a nested U-shaped network.

[0042] Step 2: In stages S1-S2, an edge supervision module is introduced to capture rich spatial detail information.

[0043] Step 3: In stages S3 to S5, a high-level semantic enhancement module is introduced to learn the location information of the lesion area.

[0044] Step 4: In the decoder part, the multi-scale features of different levels in the decoder stage are fused through the attention fusion module.

[0045] Step 5: Segment the input COVID-19 image based on the fusion structure of multi-scale feature maps.

[0046] Step 6: Further introduce semi-supervised strategy learning into the model M obtained by the above process.

[0047] Step 7: Add unlabeled data and randomly select them in a ratio of 7 (label): 3 (unlabel).

[0048] Step 8: Use model M to predict the amplified data in step 7 and generate pseudo labels.

[0049] Step 9: Merge the pseudo-label and initial label data into a new labeled dataset. At the same time, remove the randomly selected data from the unlabeled dataset.

[0050] Step 10: Train the model M again with the new labeled dataset.

[0051] Step 11: Repeat steps 7-10 until the unlabeled dataset is empty.

[0052] Step 12: Get the final model M'.

[0053] In order to quantitatively evaluate the segmentation effect of the technical solution, six indicators are used, including precision (Prec), recall (Recall), dice similarity coefficient (DSC), structural metric (S m ), enhanced alignment metric And the mean absolute error (MAE), which is calculated as follows:

[0054]

[0055]

[0056]

[0057] S m =α·S o (P,G)+(1-α)·S r (P,G) (4)

[0058]

[0059]

[0060] Among them, TP, FP, and FN are true positive, false positive, and false negative, respectively; P is the predicted image, and G is the true value image; Sr represents the region-aware structural similarity measurement, So represents the target-aware structural similarity measurement, and α is empirically taken as 0.5; h and w represent the height and width of the input image, respectively, and (x, y) represents the corresponding coordinate positions of the predicted image and the true value image.

[0061] So far, quantitative and qualitative experiments have been conducted on steps 1 to 5 to verify the effects of each key component, as shown in Table 1 and Figure 3 shown.

[0062] Table 1 Quantitative evaluation of different modules

[0063]

[0064] Currently, labeled CT datasets with expert annotations are very limited, as manual segmentation of COVID-19 lesions is difficult and time-consuming. To address this issue, this invention further optimizes the network using a semi-supervised learning strategy that leverages a large number of unlabeled CT images to effectively expand the training dataset. This strategy, based on a random sampling strategy, can generate pseudo-labels for unlabeled CT images, as described in Algorithm 1.

[0065] Algorithm 1: Semi-supervised nested U-type network model

[0066] Input: A1 labeled training data D labeled and A2 unlabeled training data D unlabeled

[0067] Output: trained semi-supervised nested U-shaped network model

[0068] 1: All labeled data are used as training sets,

[0069] 2: Use the training set obtained in step 1 to train the model and obtain the pre-trained model M.

[0070] 3: Repeat steps 4 to 8.

[0071] 4: Randomly select images from the unlabeled data, keeping the ratio of 7:3 to the number of training in the previous step, and test them with the obtained pre-trained model M to generate pseudo-labeled data.

[0072] 5: Use the pseudo-label data obtained in step 4 to expand the initial training data set.

[0073] 6: Randomly select an image from D unlabeled Remove,

[0074] 7: Use the expanded dataset to fine-tune M,

[0075] 8: Until D unlabeled The data is empty,

[0076] 9: Return the final model M'.

[0077] The model framework using a semi-supervised learning strategy has two advantages. First, the training and image selection process is simple and easy to implement, and there is no need to evaluate the generated pseudo-labels. Second, this strategy can provide more robust performance than other semi-supervised learning methods, preventing overfitting.

[0078] The final improvement to the model from steps 7 to 11 is shown in Table 2. Step 7, as the key to the dynamic random selection strategy, can significantly reduce the training time of the semi-supervised learning strategy. Typically, random selection strategies use a fixed number of extractions. For example, from 1,600 unlabeled CT images, only K images are randomly selected in one iteration. When K = 5, the model needs to be iterated 320 times, taking several days to complete. This solution, however, uses a dynamic approach, extracting a fixed percentage each iteration, which accelerates the entire process and reduces the training time by several or even dozens of times.

[0079] Table 2 Quantitative evaluation before and after using semi-supervised learning

[0080]

[0081] The embodiments of the present invention are described in detail above with reference to the accompanying drawings, but the present invention is not limited to the described embodiments. It is apparent to those skilled in the art that various changes, modifications, substitutions, and variations to these embodiments may be made without departing from the principles and spirit of the present invention, and these changes and modifications still fall within the scope of protection of the present invention.

Claims

1. A COVID-19 lesion segmentation method based on a nested U-shaped network, characterized by: The following steps are involved: Step 1: Using RSU as the basic unit, construct a U-shaped network structure to construct a backbone network with a nested U-shaped network; Step 2: Introduce the edge supervision module to capture rich spatial detail information; Step 3: Introduce a high-level semantic enhancement module to learn the location information of the lesion area; Step 4: In the decoder part, the multi-scale features of different levels in the decoder stage are fused through the attention fusion module; Step 5: Segment the input COVID-19 image based on the fusion results of the multi-scale feature maps; Step 6: Further introduce semi-supervised strategy learning into the model M obtained by the above process; Step 7: Add unlabeled data and randomly select them according to the ratio of label 7 to unlabel 3; Step 8: Use model M to predict the amplified data in step 7 and generate pseudo labels; Step 9: Merge the pseudo-labeled and initial labeled data into a new labeled dataset. At the same time, remove the randomly selected data from the unlabeled dataset. Step 10: Retrain the model M on the new label dataset to obtain the iterated model; Step 11: Repeat steps 7-10 until the unlabeled dataset is empty. Step 12: Get the final model M'.

2. A COVID-19 lesion segmentation method based on a nested U-shaped network according to claim 1, characterized in that: The RSU in step 1 specifically includes three parts: An input convolution layer that converts the input feature map into an intermediate layer feature map with the number of channels Cout; A U-shaped symmetric codec structure, where L is the number of codec layers; Residual connections that fuse local features and multi-scale features through addition operations.

3. A COVID-19 lesion segmentation method based on a nested U-shaped network according to claim 1, characterized in that: The segmentation effect is quantitatively evaluated using precision (Prec), recall (Recall), dice similarity coefficient (DSC), structural metric (S m ), enhanced alignment metric And the mean absolute error (MAE) six indicators, the calculation formula is as follows: S m =α·S o (P, G)+(1-α)·S r (P, G) (4) Among them, TP, FP, and FN are true positive, false positive, and false negative, respectively; P is the predicted image, and G is the true value image; Sr represents the region-aware structural similarity measurement, So represents the target-aware structural similarity measurement, and α is 0.5; h and w represent the height and width of the input image, respectively, and (x, y) represents the corresponding coordinate positions of the predicted image and the true value image.

Images