A semi-supervised dual-network medical image segmentation method

Abstract

The present invention discloses a semi-supervised dual-network medical image segmentation method, comprising the following steps: processing a data set, randomly dividing the data set into a training set and a test set in a 4:1 ratio, and performing image enhancement; building a DNSS network model, using a geometry-aware network and an image transformation consistency-based network, and adding an uncertainty perception module at the end of the auxiliary network to effectively utilize unlabeled data; training the DNSS network on the training set, performing the segmentation task and generating a segmentation model; testing the model on the test set, and selecting the model with the best performance as the final model based on the test results and saving it. The present invention can generate efficient and accurate labeled data, reduce the reliance of medical image segmentation on manually recorded data, reduce the cost of manually labeling medical images, and assist doctors in diagnosis.

Description

Technical Field

[0001] The present invention belongs to the technical field of medical image analysis and artificial intelligence, and relates to a semi-supervised dual-network medical image segmentation method. Background Art

[0002] Deep learning-based methods have made great progress in the field of medical image segmentation. The U-Net and V-Net series networks, which are based on the encoder-decoder structure and adopt the technology of superimposed feature maps, as well as their corresponding improved models, have been widely used in medical image segmentation tasks and have achieved excellent results.

[0003] Training deep learning neural networks based on medical images typically relies on large, labeled datasets, which is very expensive. To address this issue, many deep learning methods have explored this area, such as weakly supervised methods that learn image segmentation from bounding boxes, scribbles, or image-level labels, and GAN methods that use generators and discriminators to compete to generate more reliable labels. However, these methods have some limitations. Weakly supervised methods require weakly labeled samples with pixel-level annotations and therefore do not use unlabeled data to extract additional training signals. Methods based on adversarial training utilize unlabeled data but can be more difficult to train. Summary of the Invention

[0004] In order to address the above-mentioned shortcomings, the present invention provides a semi-supervised dual-network medical image segmentation method that can generate highly accurate label data.

[0005] The object of the present invention is achieved by: a semi-supervised dual-network medical image segmentation method, characterized in that the method comprises the following steps:

[0006] Step 1: Collect samples, label them, and randomly divide the dataset into training and test sets in a ratio of 4:1. Perform data augmentation by random flipping and random cropping.

[0007] Step 2: Build the DNSS network model using a geometric perception network based on signed distance function and an image transformation consistency network, and add an uncertainty perception module at the end of the auxiliary network;

[0008] Step 3: Train the DNSS network on the training set, perform the segmentation task and generate training parameters;

[0009] Step 4: Test the model on the test set using the training parameters to generate segmentation results and save them.

[0010] Preferably, the data enhancement in step 1 includes the following steps:

[0011] Step 1-1: Use random cropping to directly crop the fixed-size image input network;

[0012] Step 1-2: Rotate the image by 90, 180, and 270 degrees, and flip the image along the axial plane.

[0013] Preferably, in step 2, the DNSS network model is constructed using V-Net as the backbone network;

[0014] The DNSS network model includes a geometry-aware network and an image transformation consistency network. The geometry-aware network adds a signed distance function at the end of the backbone network, and obtains a segmentation map similar to the true label by inverse transforming the signed distance function.

[0015] The image transformation consistency network retains the downsampling process of V-Net. In order to enhance the robustness of the network under different disturbances, the upsampling process expands the original single upsampling into two different upsampling processes.

[0016] Preferably, the auxiliary network in step 2 includes guided masking and guided clipping, and an uncertainty perception function is added to the auxiliary network to quantify uncertainty, and the uncertainty is used as a weight in unsupervised learning.

[0017] Preferably, the geometric perception network enhances the edge perception of the image by generating a signed distance map, the formula of which is as follows:

[0018]

[0019] where ||xy|| is the Euclidean distance between voxel x and voxel y, D in Represents the interior of the target object, and D out Represent the contour and exterior of the target object respectively; D(x) takes negative values ​​inside the target and positive values ​​outside the target, and its absolute value is defined by the distance to the nearest boundary point.

[0020] Preferably, the signed distance map is inversely transformed using a smooth approximation of the Heaviside function, which is defined as follows:

[0021]

[0022] Where z is the signed distance value under the corresponding voxel map, and k is a control factor as large as possible to approximate the transformation. In this paper, k is set to 1500 to make the transformed image more similar to the original segmentation image.

[0023] Preferably, the image transformation consistency network adds guided masking and guided cropping to the V-Net, wherein guided masking is to avoid the network from being overly dependent on contextual relationships; guided cropping is to avoid the network from being dependent on specific parts of the input samples.

[0024] Preferably, in step 2, an uncertainty perception module is added at the end of the auxiliary network, and the structure of the uncertainty perception module is:

[0025] The true prediction value is incorporated into the quantization function to enhance the stability of uncertainty, so that the main network can learn high-confidence predictions from the auxiliary network. The uncertainty perception function for the image transformation consistency network is defined as:

[0026]

[0027] Where i represents the index of the corresponding voxel, N is the total number of auxiliary classifiers, q is the predicted value of the main network, the predicted value of the auxiliary classifier is denoted as p, and n represents the index of the corresponding auxiliary classifier;

[0028] In order to better convert the uncertainty value into confidence probability, the above parameters are quantified into entropy. The higher the entropy value, the lower the prediction confidence of the auxiliary fee classifier, which is defined as follows:

[0029]

[0030] The uncertainty quantified by the above method is used to filter uncertain voxels, thereby guiding the model to learn from more reliable unlabeled data; i is represented as the index of the voxel corresponding to the prediction, and the gating function is set: e Used to filter unreliable auxiliary classifier voxels:

[0031]

[0032] Compared with the existing technology, the beneficial effects of the present invention are: 1. By adopting a geometric perception network, the model can extract boundary information more accurately and the segmentation effect becomes smoother; by adding an auxiliary network into the image transformation consistency network, a variety of pixel-level perturbations are generated, so that the image can be learned from different perturbation images, thereby enhancing the generalization ability of the segmentation model.

[0033] 2. By adding an uncertainty perception module under image transformation consistency and using uncertainty perception functions, unreliable information is filtered out and the stability of network learning is enhanced.

[0034] 3. Through DNSS medical image segmentation technology, medical images can be segmented quickly and accurately with a small amount of labeled data, which can help doctors make auxiliary judgments on medical images, improve doctors' work efficiency and diagnostic accuracy, and reduce missed diagnosis and misdiagnosis rates. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] Figure 1 Flow chart of the method of the present invention.

[0036] Figure 2 This is a network diagram of DNSS.

[0037] Figure 3 This is an illustration of each sub-module of DNSS.

[0038] Figure 4 This is the segmentation effect diagram. DETAILED DESCRIPTION

[0039] The present invention is further summarized below with reference to the accompanying drawings.

[0040] like Figure 1 As shown, a semi-supervised dual-network medical image segmentation method comprises the following steps:

[0041] Step 1: Collect samples, label them, and randomly divide the dataset into training and test sets in a ratio of 4:1. Perform data augmentation by random flipping and random cropping.

[0042] Step 1-1: Use random cropping to directly crop a fixed-size image and input it into the network. In this embodiment, the image is directly cropped to a size of 112*112*80;

[0043] Step 1-2: Rotate the image of size 112*112*80 by 90 degrees, 180 degrees, and 270 degrees, and flip the image along the axial plane.

[0044] Step 2: Use a geometry-aware network based on signed distance function and an image transformation consistency network to build a DNSS network model. Add an uncertainty perception module at the end of the image transformation consistency network to filter out unreliable information generated by the network.

[0045] Step 3: Perform consistency learning on the two models of the geometry perception network and the image transformation consistency network, that is, loss calculation, and generate optimized training parameters.

[0046] Step 4: Put the test picture into the test model, get the test results, and generate the segmentation effect diagram

[0047] like Figure 2 As shown, the geometric perception network enhances the image boundary perception by generating a signed distance map. The signed distance function formula is as follows:

[0048]

[0049] where ||xy|| is the Euclidean distance between voxel x and voxel y, D in Represents the interior of the target object, and D out They represent the outline and exterior of the target object, respectively. Generally speaking, D(x) takes negative values ​​inside the target and positive values ​​outside the target, and its absolute value is defined by the distance to the nearest boundary point.

[0050] In order to solve the problem of smoothness and continuity of the signed distance graph, the smooth approximate Heaviside function is used to inverse transform the signed distance graph, which is defined as follows:

[0051]

[0052] Where z is the signed distance value under the corresponding voxel map, and k is a control factor as large as possible to approximate the transformation. In this paper, k is set to 1500 to make the transformed image more similar to the original segmentation image.

[0053] like Figure 3 As shown, the image transformation consistency network retains the downsampling process of the V-Net. However, to enhance the network's robustness to various perturbations, the upsampling process expands the original single upsampling process into two different upsampling processes: guided masking and guided cropping. Guided masking and guided cropping serve as auxiliary networks. Furthermore, the perturbation labels generated by these two auxiliary classification networks must be consistent with the true label segmentation results. To strengthen the consistency of the predicted representations, this paper maps the outputs of different networks to the same preset space. This allows the dual network to learn multiple different segmentation representations from the objects segmented in different ways, thereby establishing multivariate regularization.

[0054] The purpose of guided masking is to prevent the network from being too dependent on context. obj To mask the detected foreground objects and generate a context mask M con =1-M obj , and then generate the perturbation where Z is the downsampling result of the main decoder.

[0055] Guided cropping To avoid the network's dependence on specific parts of the object, the bounding box of each detected object is found using the prediction q of the main decoder, and the spatial range of the downsampled feature map is detected based on the prediction of the main decoder. Then, the random shearing operation in the downsampled map is "zeroed" according to the spatial range, and the zeroed downsampled map is input into the auxiliary decoder for upsampling.

[0056] Incorporating the true prediction value into the quantization function enhances the stability of uncertainty, allowing the main network to learn high-confidence predictions from the auxiliary network. The uncertainty perception function for the image transformation consistency network is defined as:

[0057]

[0058] Where i represents the index of the corresponding voxel, N is the total number of auxiliary classifiers, q is the predicted value of the main network, the predicted value of the auxiliary classifier is denoted as p, and n represents the index of the corresponding auxiliary classifier.

[0059] In order to better convert the uncertainty value into confidence probability, the above value is quantified into entropy The higher the value of , the lower the prediction confidence of the auxiliary classifier, which is defined as follows

[0060]

[0061] The uncertainty quantified by the above method is used to filter uncertain voxels, thereby guiding the model to learn from more reliable unlabeled data. Let i be the index of the voxel corresponding to the prediction, and set the gating function: τ e Used to filter unreliable auxiliary classifier voxels:

[0062]

[0063] Each auxiliary decoder outputs a different version of the perturbed feature map, while the main decoder outputs the unperturbed feature map. The prediction from the main decoder is denoted as q, and the prediction from the nth auxiliary decoder is denoted as p n , and obtain a reliable prediction value p by screening the uncertainty perception module n , the consistency between the feature maps obtained by the main decoder and the auxiliary decoder is expressed using the mean square error function:

[0064]

[0065] Where N is the number of auxiliary decoders.

[0066] Step 3: Train the DNSS network on the training set, perform the segmentation task and generate training parameters;

[0067] The loss function consists of three parts: supervision loss Geometry-aware loss and the loss of the image transformation consistency network The joint loss function is:

[0068]

[0069] Take Dice and CE The average value of , α and β are two hyperparameters that balance different losses:

[0070]

[0071]

[0072] Represents the loss between the inverse transformation image and the predicted image, which is calculated using CE loss. It represents the loss between the inverse transformation map and the true label, and is calculated using the Dice loss, where λ is used to control the climbing weight coefficient of the trade-off between segmentation loss and consistency loss. In this paper, λ is set to 0.1×e -5×(1-t) / t max Where t is the number of iterations, t max Maximum number of iterations.

[0073] The formulas for Dice loss and CE loss are as follows:

[0074]

[0075]

[0076] Where C is the number of categories, g i is the true value, q i is the predicted value.

[0077] Step 4: Test the model on the test set using the training parameters to generate segmentation results and save them. The method of the present invention is compared with the classic image segmentation algorithm.

[0078] Experimental methods:

[0079] The experimental environment was an Ubuntu operating system, an Intel Xeon Gold 6240R CPU, and an A100-PCIE-40GB GPU utilizing CUDA 11.0 for GPU computing acceleration. The experimental framework used PyTorch, the compiler used PyCharm, and third-party data package support was installed. The LA2018 dataset was used, and V-Net was the experimental backbone. The model was trained using the stochastic gradient descent (SGD) optimizer for 6000 iterations. The initial learning rate (LR) was 0.01, decaying by 0.001 every 2500 iterations. The training batch size was 4, consisting of 2 labeled images and 2 unlabeled images. Randomly cropped 112×112×80 scanned images were used as the network input. A sliding window approach with a step size of 18×18×4 was used to obtain the final segmentation results. To fully utilize the limited training data, data augmentation was used to expand the training data. The augmentation operations include random rotation (±90°, ±180°, and ±270°) and dynamic flip operations.

[0080] Table 1 Comparison of semantic segmentation model experimental results

[0081] Model Labeled Unlabeled Dice (%) Jaccard (%) ASD 95HD V-Net 80 0 91.14 83.32 1.52 5.75 V-Net 16 0 85.93 74.23 2.61 12.02 CCT 16 64 88.83 80.06 2.49 8.44 UA-MT 16 64 88.88 80.21 2.26 7.32 SASS 16 64 89.27 80.82 3.13 8.83 DTC 16 64 89.42 80.89 2.10 7.32 Method of the present invention 16 64 90.13 81.17 1.91 2.07

[0082] Combine Figure 4 As shown in Table 1, the performance indicators of this method are significantly higher than those of CCT, UA-MT, SASS and DTC.

[0083] The foregoing is merely an embodiment of the present invention and is not intended to limit the present invention. It will be apparent to those skilled in the art that various modifications and variations of the present invention are possible. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention are intended to be included within the scope of the claims of the present invention.

Claims

1. A semi-supervised dual-network medical image segmentation method, characterized by: The method comprises the following steps: Step 1: Collect samples, label them, and randomly divide the dataset into training and test sets in a ratio of 4:

1. Perform data augmentation by random flipping and random cropping. Step 2: Build the DNSS network model. Use a geometric perception network based on signed distance function and an image transformation consistency network as auxiliary decoding networks to build the DNSS network model, and add an uncertainty perception module at the end of the auxiliary network. The DNSS network model is built using V-Net as the backbone network; The DNSS network model includes a geometry-aware network and an image perturbation consistency network. The geometry-aware network adds a signed distance function at the end of the backbone network, and obtains a segmentation map similar to the true label by performing an inverse transformation on the signed distance map generated by the signed distance function. The image perturbation consistency network retains the downsampling process of V-Net. In order to enhance the robustness of the network under different perturbations, the upsampling process expands the original single upsampling into two different upsampling processes. The image perturbation consistency network adds guided masking and guided cropping to V-Net. Guided masking is to prevent the network from being too dependent on context; guided cropping is to avoid the network's dependence on specific parts of the input sample. Add an uncertainty correction module at the end of the auxiliary network. The structure of the uncertainty correction module is: The true prediction value is incorporated into the quantization function to enhance the stability of uncertainty, so that the main network can learn high-confidence predictions from the auxiliary network. The uncertainty perception function for the image perturbation consistency network is defined as: Where i represents the index of the corresponding voxel, N is the total number of auxiliary classifiers, q is the predicted value of the main network, the predicted value of the auxiliary classifier is denoted as p, and n represents the index of the corresponding auxiliary classifier; In order to better convert the uncertainty value into confidence probability, the above value is quantified into entropy. The higher the entropy value, the lower the prediction confidence of the auxiliary fee classifier, which is defined as follows: The uncertainty quantified by the above method is used to filter uncertain voxels, thereby guiding the model to learn from more reliable unlabeled data; i is represented as the index of the voxel corresponding to the prediction, and the gating function is set: e Used to filter unreliable auxiliary classifier voxels: Step 3: Train the DNSS network on the training set, perform the segmentation task and generate training parameters; Step 4: Test the model on the test set using the training parameters to generate segmentation results and save them.

2. The dual-network medical image segmentation method according to claim 1, characterized in that: The data enhancement in step 1 includes the following steps: Step 1-1: Use random cropping to directly crop the fixed-size image input network; Step 1-2: Rotate the image by 90, 180, and 270 degrees, and flip the image along the axial plane.

3. The dual-network medical image segmentation method according to claim 1, wherein: In step 2, the auxiliary network includes guided masking and guided clipping. An uncertainty perception function is added to the auxiliary network to quantify uncertainty, and the uncertainty is used as a weight in unsupervised learning.

4. The dual-network medical image segmentation method according to claim 1, wherein: The geometric perception network enhances the image boundary perception by generating a signed distance map, and its formula is as follows: where ||xy|| is the Euclidean distance between voxel x and voxel y, and D in Represents the interior of the target object, and D out Represent the contour and exterior of the target object respectively; D(x) takes negative values ​​inside the target and positive values ​​outside the target, and its absolute value is defined by the distance to the nearest boundary point.

5. The dual-network medical image segmentation method according to claim 1, wherein: The signed distance map is inversely transformed using a smooth approximation of the Heaviside function, which is defined as follows: Among them, z is the signed distance value under the corresponding voxel map, k is a control factor as large as possible to approximate the transformation, and k is set to 1500 to make the transformed image more similar to the original segmentation image.

Images