Semi-supervised medical image segmentation method based on dual-view complementary consistency
Through the semi-supervised medical image segmentation method with dual-view complementary consistency, the dual-view data augmentation and teacher-student model architecture are used to solve the problems of high data acquisition cost and poor generalization ability in medical image segmentation, achieving more efficient Unlabeled data utilization and better model generalization capabilities improve segmentation accuracy and robustness.
Patent Information
- Application Number
- CN202411016166.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-27
- Publication Date
- 2024-11-15
- Estimated Expiration
- 2044-07-27
AI Technical Summary
The existing medical image segmentation method has the problems of high data acquisition cost, poor generalization ability of the model and insufficient utilization of unlabeled data. Especially in complex and variable medical image processing, the robustness and generalization ability of the model are limited. .
Using a semi-supervised medical image segmentation method based on dual-view complementary consistency, through dual-view data augmentation, teacher-student model architecture and feature space distillation mechanism, the unlabeled data is fully utilized to improve the accuracy of medical image segmentation and generalization of models. ability to reduce labeling costs. Specific steps include data augmentation, feature space distillation, dual-view fusion, uncertainty estimation, comprehensive loss calculation and model training.
Effectively reduce data acquisition costs, improve model generalization capabilities and segmentation performance, improve model robustness and reliability of segmentation results, and enhance the overall performance of the model by optimizing pseudo-label generation and uncertainty estimation.
Smart Images

Figure CN118968091A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of computer vision image processing, and in particular to a semi-supervised medical image segmentation method based on dual-view complementary consistency. Background Art
[0002] Medical image segmentation is a core task in medical image analysis and is widely used in clinical diagnosis, treatment planning, and postoperative evaluation. Accurate medical image segmentation can help doctors better understand and analyze key structures in medical images, such as organs and lesion areas, thereby improving the effectiveness of diagnosis and treatment.
[0003] Currently, existing medical image segmentation methods mainly include fully supervised learning, semi-supervised learning, and weakly supervised learning. Fully supervised learning methods can obtain higher segmentation accuracy by training a large amount of high-quality labeled data through a deep neural network model. However, this type of method relies heavily on a large amount of labeled data, which is not only time-consuming but also expensive to obtain. In addition, fully supervised learning models often show poor generalization and robustness when faced with scarce labeled data or new unseen data. Weakly supervised learning methods use incompletely labeled data for training. Although this type of method reduces the workload of labeling to a certain extent, the accuracy and reliability of the segmentation results may be affected due to inaccurate labeling information.
[0004] Semi-supervised learning methods combine a small amount of labeled data with a large amount of unlabeled data for training to alleviate the problem of scarce labeled data. These methods use unlabeled data to enhance the learning effect of the model, attempting to reduce the workload of labeling while maintaining high segmentation performance. However, when dealing with complex medical images, the existing semi-supervised learning methods have a low effective utilization rate of unlabeled data, resulting in the failure to significantly improve the segmentation accuracy and generalization ability of the model. When faced with complex and changeable medical images, the robustness and generalization ability of the model are limited. Therefore, there is an urgent need to design a more efficient semi-supervised learning algorithm to fully tap the potential of unlabeled data and improve the accuracy and reliability of medical image segmentation. Summary of the invention
[0005] In view of the shortcomings of the prior art, namely the high data acquisition cost, poor model generalization ability and insufficient utilization of unlabeled data in the existing medical image segmentation methods, the present invention proposes a semi-supervised medical image segmentation method based on dual-view complementary consistency. This method aims to make full use of unlabeled data through dual-view data enhancement strategy, teacher-student model architecture and feature space distillation mechanism, improve the accuracy of medical image segmentation and the generalization ability of the model, thereby reducing the annotation cost, improving the segmentation effect and the robustness of the model.
[0006] The above technical objectives of the present invention are achieved through the following technical solutions:
[0007] A semi-supervised medical image segmentation method based on dual-view complementary consistency includes the following steps: S1, data enhancement: performing dual-view data enhancement on the input image, including weak enhancement and strong enhancement, to generate diversified image data; S2, feature space distillation: adopting a teacher-student model architecture, extracting multi-scale semantic features through the teacher model, and transferring the multi-scale semantic features to the student model through a distillation mechanism; S3, dual-view fusion: predicting the weakly enhanced and strongly enhanced image data respectively, and calculating their average values to generate a smooth probability distribution; S4, uncertainty estimation: using a denoising autoencoder to learn to recover a clean signal from noise and generate an uncertainty map; S5, comprehensive loss calculation and model training: designing a comprehensive loss function to guide model training.
[0008] Furthermore, in step S1, the weak enhancement includes random rotation, random flipping, random cropping and scale change; the strong enhancement includes brightness adjustment, Gaussian noise addition, contrast adjustment and color disturbance.
[0009] Furthermore, in step S2, the teacher model uses three-dimensional convolution operations with kernel sizes of 1x1x1 and 3x3x3 to extract multi-scale features, uses batch normalization and rectified linear activation functions to improve the feature extraction effect, and then upsamples the features through trilinear upsampling operations to achieve matching of features of different scales, thereby enhancing the model's understanding of complex features.
[0010] Furthermore, in step S3, for each unlabeled sample, weak enhancement and strong enhancement are applied to generate two enhanced versions, the enhanced images are predicted respectively by the teacher model to generate two prediction distributions, the average of the two prediction distributions is calculated to obtain a smooth probability distribution, and the probability distribution is sharpened by adjusting the temperature parameter to generate a high-confidence pseudo-label to improve the quality of the pseudo-label, thereby optimizing the learning process of unlabeled data.
[0011] Further, in step S4, the specific steps include adding noise to the segmentation mask to generate a perturbed segmentation mask, then reconstructing the perturbed segmentation mask using a denoising autoencoder to generate a denoised segmentation mask, calculating the difference between the denoised segmentation mask and the prediction result of the teacher model, generating an uncertainty map, and improving the reliability of the segmentation result by focusing on the area where the model is uncertain.
[0012] Furthermore, in step S5, a comprehensive loss function is designed, including supervision loss, distillation loss, pseudo label loss and uncertainty estimation loss.
[0013] Furthermore, the supervision loss is calculated by processing labeled data to calculate the supervised learning loss of the model; the distillation loss is calculated based on the feature alignment of the teacher model and the student model; the pseudo-label loss is calculated based on the pseudo-label generated by dual-view fusion; the uncertainty estimation loss is based on the uncertainty map to calculate the loss of the model in the uncertain area.
[0014] Furthermore, in step S5, the model training step includes using an exponential sliding average to dynamically adjust the weight of the teacher model so that it is an exponential moving average of the student model weight; the student model is trained by a stochastic gradient descent algorithm, combining supervised loss and unsupervised loss to jointly optimize the model parameters, and continuously iterating and updating until the performance of the model on the verification data converges.
[0015] The present invention has the following beneficial effects:
[0016] 1. Reduce data acquisition costs: Through semi-supervised learning methods, unlabeled data can be effectively utilized and the dependence on high-quality labeled data can be reduced, thereby reducing the cost and time consumption of data acquisition.
[0017] 2. Improve model generalization ability: The teacher-student model architecture and feature space distillation mechanism enable the model to learn and generalize better, especially when labeled data is scarce or when processing new unseen data, showing higher robustness.
[0018] 3. Optimize pseudo-label generation: The dual-view fusion strategy and sharpening method improve the accuracy of pseudo-labels, thereby improving the utilization efficiency of unlabeled data and further enhancing the segmentation performance of the model.
[0019] 4. Enhanced uncertainty estimation: Using pre-trained autoencoders for uncertainty estimation helps the model better understand and handle uncertainty in predictions and improves the reliability of segmentation results.
[0020] 5. Comprehensive loss function to guide training: Design a comprehensive loss function for the model, including supervision loss, distillation loss, pseudo-label loss, and uncertainty estimation loss, to comprehensively guide the training of the student model and improve the overall performance of the model. BRIEF DESCRIPTION OF THE DRAWINGS
[0021] Figure 1 is a technical block diagram of an embodiment of the present invention;
[0022] Figure 2 is an execution flow chart of the algorithm model of an embodiment of the present invention;
[0023] Figure 3 A model diagram of the data enhancement and feature space distillation technology of an embodiment of the present invention;
[0024] Figure 4A model diagram of a dual-view fusion and uncertainty estimation technology according to an embodiment of the present invention;
[0025] Figure 5 A schematic diagram of visualizing the LA data set according to an embodiment of the present invention;
[0026] Figure 6 A schematic diagram of visualization of the LiTS data set according to an embodiment of the present invention. DETAILED DESCRIPTION
[0027] The technical solution of the present invention is further described below in conjunction with the accompanying drawings and embodiments.
[0028] This embodiment provides a semi-supervised medical image segmentation method based on dual-view complementary consistency. Figure 1 As shown, the following steps are included:
[0029] Step 1: First, input the labeled and unlabeled medical image data into the system, and perform dual-view data enhancement on the input images, including weak enhancement and strong enhancement. This enhancement method can generate multi-view image data and enrich the diversity of the data set.
[0030] Among them, the weak enhancement strategy includes random rotation, random flipping, random cropping and scale change of the image, which can increase the diversity of the data without changing the overall content of the image. The strong enhancement strategy includes brightness adjustment, Gaussian noise addition, contrast adjustment and color perturbation, which can change the image features more significantly and increase the robustness of the model.
[0031] Step 2: Use the teacher-student model to extract features. Figure 3 As shown in the figure, the teacher model extracts multi-scale semantic features from the augmented data through contrastive learning and transfers these features to the student model through the feature space distillation mechanism. The multi-scale learning method uses multi-scale convolution operations and upsampling techniques to enhance the model's understanding of complex features, thereby improving segmentation accuracy. The student model uses the knowledge transferred by the teacher model to learn, thereby improving the generalization ability of the model. Among them, the feature space distillation mechanism performs feature alignment by calculating the mean square error between the feature representations of the teacher model and the student model. The formula is as follows:
[0032] L distill =MSE(f teacher , f student )
[0033] MSE represents the mean square error loss function, that is:
[0034]
[0035] Among them, f teacherand f student are the feature representations of the teacher model and the student model, respectively.
[0036] Step 3: Generate and optimize pseudo labels through dual-view fusion strategy, and integrate the prediction results of the teacher model to adjust the learning process of unlabeled data. Figure 4 As shown in the figure, the specific steps include: applying weak enhancement and strong enhancement to each unlabeled sample to generate two enhanced versions; the teacher model processes these enhanced images to generate two prediction results; calculating the average of the two prediction results to form a smooth probability distribution; using the sharpening method to process the pseudo-label to improve its accuracy. Further, the specific formula of the sharpening method is as follows:
[0037]
[0038] Among them, T is the temperature parameter, p is the probability distribution, and p' is the sharpened pseudo label.
[0039] Step 4: Use the pre-trained autoencoder to estimate the uncertainty of the model and adjust the reliability and segmentation accuracy of the model. Figure 3 As shown in the figure, the specific steps include: reconstructing the noisy segmentation mask through the autoencoder to learn the global anatomical consistency of the decoder output; calculating the difference between the teacher model prediction and the decoder reconstruction result to form an uncertainty map. Among them, the uncertainty estimation step uses the difference between the teacher model prediction and the autoencoder reconstruction result to estimate the uncertainty. The formula is as follows:
[0040] U=||p teacher -p autoencoder || 2
[0041] Among them, p teacher represents the prediction of the teacher model, p autoencoder Represents the reconstruction result of the autoencoder.
[0042] Step 5: Define a comprehensive semi-supervised loss function, including supervision loss, distillation loss, pseudo-label loss, and uncertainty estimation loss, to guide the training of the student model. The comprehensive loss function is defined as follows:
[0043] L semi =L sup +L distill +L psu +L uncertainty
[0044] Among them, L sup represents the cross entropy loss based on labeled data, L distill represents the mean square error loss based on the feature space distillation mechanism, Lpsu represents the pseudo-label loss combining Dice and cross entropy, L uncertainty represents the uncertainty estimation loss based on the difference between the teacher model prediction and the autoencoder reconstruction result.
[0045] The student model is trained using the comprehensive loss function defined above, and the parameters of the teacher model are dynamically adjusted through exponential moving average. The specific implementation of exponential moving average is as follows:
[0046] θ t =m·θ t-1 +(1-m)·θ′
[0047] Among them, θ t represents the parameters of the current time step, θ (t-1) represents the parameters of the previous time step, θ' represents the new parameter update of the current time step, and m is the exponential sliding average update coefficient.
[0048] The execution process of the algorithm model in this embodiment is as follows: Figure 2 As shown in the figure, the flowchart describes the whole process from data input to the final model output. First, the labeled and unlabeled data are input into the system and data enhancement is performed. Then, knowledge distillation is performed through the teacher network to extract and pass multi-scale feature information to the student network. Next, the uncertainty graph guidance is obtained, and the network parameters are updated using the Adam optimizer. Subsequently, it is evaluated whether the current model is the optimal model. If so, the optimal model is saved and the iteration is checked to see if it is complete. After the iteration is completed, the final optimal model is obtained. Finally, the test data is input into the model to obtain the segmentation results, which are evaluated in combination with indicators such as Dice and Jaccard, and the results are finally output.
[0049] In the present invention, two main datasets were used for experiments and verification, as shown in Figure 1 below, namely the LA (left atrium) dataset and the LiTS (liver tumor segmentation challenge) dataset. The LA dataset is mainly used for the segmentation task of the left atrium, including the left atrium MRI image data of different patients. The dataset contains images of various resolutions acquired from different scanners and has high diversity. The images in the dataset have been annotated by experts and provide high-quality segmentation labels. The LiTS dataset is mainly used for the segmentation task of the liver and liver tumors, and contains multimodal CT scan images acquired from multiple hospitals. The dataset has also been annotated by experts and provides segmentation labels for the liver and liver tumors.
[0050] Table 1 Dataset information
[0051]
[0052] In order to evaluate the performance of the model, the dataset is split into the following proportions: 80% of the images are used for training, and 20% of the images are used for validation and testing to evaluate the generalization ability and actual performance of the model.
[0053] In the model training process of the present invention, each module and parameter are set in detail to ensure that the model can perform well on different data sets. The data enhancement parameters are set as follows: weak enhancement includes random rotation (angle range: -15° to 15°), random flipping (horizontal flipping and vertical flipping), random cropping (cropping ratio: 0.8 to 1.0), scale change (scale range: 0.9 to 1.1); strong enhancement includes brightness adjustment (adjustment range: 0.8 to 1.2), Gaussian noise addition (mean: 0, variance: 0.01), contrast adjustment (adjustment range: 0.8 to 1.2), color perturbation (random color channel offset). The model training parameters include: the initial learning rate is set to 0.001, and the Adam optimizer is used for optimization; the batch size during training is set to 16; the total number of training iterations is set to 100 epochs. In the momentum update parameters, the update coefficient m is set to 0.9 for exponential sliding average.
[0054] like Figure 5 and Figure 6 Shown are the experimental visualization results on the left atrium (LA) and liver tumor segmentation (LiTS) datasets.
[0055] Output the optimized and adjusted medical image segmentation results to evaluate the segmentation accuracy and model robustness. Through experiments, the accuracy of the segmentation results and the robustness of the model are evaluated, including the Dice Similarity Coefficient (DSC), Jaccard Index, 95% Hausdorff Distance (95% Hausdorff Distance, 95HD) and Average Surface Distance (Average Surface Distance, ASD).
[0056] In order to verify the effectiveness of the algorithm of this application, the method of this application is tested on the same test set with the mean-teacher model (MT), the pyramid consistency method based on uncertainty correction (URPC), and the method based on smoothness and class separation research (SS-Net). MT improves stability by averaging model weights, URPC uses uncertainty correction to enhance prediction accuracy, and SS-Net focuses on improving segmentation quality by optimizing feature expression. The algorithm of this application adopts a feature space distillation mechanism and optimizes using strategies such as dual view fusion. The performance indicators have been significantly improved. See Tables 2 and 3 for details.
[0057] Table 2 Test results of 10% and 20% labeled data on LA dataset
[0058]
[0059] Table 3 Test results of 10% and 20% labeled data on LiTS dataset
[0060]
[0061] According to the performance index comparison results shown in Table 2 and Table 3, it can be seen that the method of this application performs better than other comparison methods in the left atrium segmentation task and liver tumor segmentation task, whether on 10% or 20% labeled data sets. Specifically, the Dice coefficients of the method of this application on the LA data set reached 87.56% and 89.73%, respectively, and the Dice coefficients on the LiTS data set reached 78.35% and 82.87%, respectively, which are significantly higher than the MT, URPC and SS-Net methods. In addition, in other key indicators such as the Jaccard coefficient, 95% Hausdorff distance and average surface distance, the method of this application also shows superior performance.
[0062] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solution of the present invention can be modified or replaced by equivalents without departing from the purpose and scope of the technical solution of the present invention, which should be included in the scope of the claims of the present invention.
Claims
1. A semi-supervised medical image segmentation method based on dual-view complementary consistency, Features: The following steps are involved: S1, data enhancement: perform dual-view data enhancement on the input image, including weak enhancement and strong enhancement, to generate diversified image data; S2, feature space distillation: adopting the teacher-student model architecture, extracting multi-scale semantic features through the teacher model, and transferring the multi-scale semantic features to the student model through the distillation mechanism; S3, dual-view fusion: predict the weakly enhanced and strongly enhanced image data respectively, and calculate their average values to generate a smooth probability distribution; S4, uncertainty estimation: using denoising autoencoders to learn to recover clean signals from noise and generate uncertainty maps; S5, comprehensive loss calculation and model training: Design a comprehensive loss function to guide model training.
2. According to claim 1, a semi-supervised medical image segmentation method based on dual-view complementary consistency, Features: In step S1, the weak enhancement includes random rotation, random flipping, random cropping and scale change; the strong enhancement includes brightness adjustment, Gaussian noise addition, contrast adjustment and color disturbance.
3. The semi-supervised medical image segmentation method based on dual-view complementary consistency according to claim 2, Features: In step S2, the teacher model uses three-dimensional convolution operations with kernel sizes of 1x1x1 and 3x3x3 to extract multi-scale features, uses batch normalization and rectified linear activation functions to improve the feature extraction effect, and then upsamples the features through trilinear upsampling operations to achieve matching of features of different scales, thereby enhancing the model's understanding of complex features.
4. The semi-supervised medical image segmentation method based on dual-view complementary consistency according to claim 1, Features: In step S3, for each unlabeled sample, weak enhancement and strong enhancement are applied to generate two enhanced versions. The enhanced images are predicted by the teacher model respectively to generate two prediction distributions. The average of the two prediction distributions is calculated to obtain a smooth probability distribution. The probability distribution is sharpened by adjusting the temperature parameter to generate a high-confidence pseudo-label to improve the quality of the pseudo-label, thereby optimizing the learning process of unlabeled data.
5. The semi-supervised medical image segmentation method based on dual-view complementary consistency according to claim 1, Features: In step S4, the specific steps include adding noise to the segmentation mask to generate a perturbed segmentation mask, then reconstructing the perturbed segmentation mask using a denoising autoencoder to generate a denoised segmentation mask, calculating the difference between the denoised segmentation mask and the prediction result of the teacher model, and generating an uncertainty map to improve the reliability of the segmentation result by focusing on the area where the model is uncertain.
6. The semi-supervised medical image segmentation method based on dual-view complementary consistency according to claim 1, Features: In step S5, a comprehensive loss function is designed, including supervision loss, distillation loss, pseudo label loss and uncertainty estimation loss.
7. The semi-supervised medical image segmentation method based on dual-view complementary consistency according to claim 6, Features: The supervision loss calculates the supervised learning loss of the model by processing labeled data; the distillation loss is calculated based on the feature alignment of the teacher model and the student model; the pseudo-label loss is calculated based on the pseudo-label generated by dual-view fusion; the uncertainty estimation loss is based on the uncertainty map and calculates the loss of the model in the uncertain area.
8. The semi-supervised medical image segmentation method based on dual-view complementary consistency according to claim 1, Features: In step S5, the model training step includes using exponential sliding average to dynamically adjust the weight of the teacher model so that it is the exponential moving average of the student model weight; the student model is trained by the stochastic gradient descent algorithm, combining the supervised loss and the unsupervised loss to jointly optimize the model parameters, and continuously iteratively updating until the performance of the model on the verification data converges.
Citation Information
Patent Citations
Multi-modal medical image segmentation method and device based on graph flow distillation
CN115578353A
Semi-supervised medical image segmentation method, system, equipment and medium
CN117095014A
MR medical image colorectal cancer segmentation method and system based on semi-supervised learning
CN117409201A
Semi-supervised medical image segmentation method based on united feature consistency of difficult pixels and effective images
CN117893760A
Unsupervised domain adaptive image segmentation method based on uncertainty estimation
CN118334333A
Cited By
Contrast learning assisted semi-supervised image segmentation method and system for dual-region balance consistency learning
CN119478414A
Double-view 3D medical image segmentation method based on fuzzy clustering Mama driving
CN120689605A
Cross-center credible semi-supervised cardiac ultrasound image segmentation method
CN121280438A
