A semi-supervised medical image segmentation method based on dual-view complementary consistency

Abstract

The application relates to a semi-supervised medical image segmentation method based on double-view complementary consistency, which only uses a small amount of labeled medical image data to mine potential feature information in a large amount of unlabeled data to improve the segmentation model performance, and comprises the following steps: a feature space distillation mechanism is used to extract and transfer multi-scale semantic features through a teacher-student model; a double-view fusion strategy is used to optimize pseudo labels and optimize the learning effect of the model on unlabeled data; and an uncertainty estimation method assisted by a pre-trained autoencoder is used to optimize the reliability and segmentation precision of the model. Experimental results on a left atrium (LA) and liver tumor segmentation (LiTS) dataset show that the method achieves good effects in terms of segmentation precision and model robustness.

Description

Technical Field

[0001] This invention relates to the field of computer vision image processing technology, and in particular to a semi-supervised medical image segmentation method based on dual-view complementary consistency. Background Technology

[0002] Medical image segmentation is a core task in medical image analysis, widely used in clinical diagnosis, treatment planning, and postoperative evaluation. Accurate medical image segmentation helps doctors better understand and analyze key structures in medical images, such as organs and lesion areas, thereby improving diagnostic and treatment outcomes.

[0003] Currently, existing medical image segmentation methods mainly include fully supervised learning, semi-supervised learning, and weakly supervised learning. Fully supervised learning methods train deep neural network models on large amounts of high-quality labeled data, achieving high segmentation accuracy. However, these methods heavily rely on a large amount of labeled data, which is both time-consuming and expensive to acquire. Furthermore, fully supervised learning models often exhibit poor generalization ability and robustness when faced with scarce or novel labeled data. Weakly supervised learning methods utilize incomplete labeled data for training. While these methods reduce the amount of labeling work to some extent, the accuracy and reliability of the segmentation results may be affected due to insufficient labeling information.

[0004] Semi-supervised learning methods alleviate the problem of scarce labeled data by combining a small amount of labeled data with a large amount of unlabeled data for training. These methods utilize unlabeled data to enhance the model's learning performance, attempting to maintain high segmentation performance while reducing the workload of labeling. However, existing semi-supervised learning methods have low effective utilization of unlabeled data when dealing with complex medical images, resulting in limited improvement in the model's segmentation accuracy and generalization ability. The improvement in robustness and generalization ability is limited when facing complex and varied medical images. Therefore, there is an urgent need to design a more efficient semi-supervised learning algorithm that fully explores the potential of unlabeled data to improve the accuracy and reliability of medical image segmentation. Summary of the Invention

[0005] To address the shortcomings of existing technologies, namely high data acquisition costs, poor model generalization ability, and insufficient utilization of unlabeled data in current medical image segmentation methods, this invention proposes a semi-supervised medical image segmentation method based on dual-view complementary consistency. This method aims to fully utilize unlabeled data through dual-view data augmentation strategies, a teacher-student model architecture, and a feature space distillation mechanism, thereby improving the accuracy and generalization ability of medical image segmentation, reducing annotation costs, and enhancing segmentation performance and model robustness.

[0006] The above-mentioned technical objective of the present invention is achieved through the following technical solution:

[0007] A semi-supervised medical image segmentation method based on dual-view complementary consistency includes the following steps: S1, Data Augmentation: Perform dual-view data augmentation on the input image, including weak and strong augmentation, to generate diverse image data; S2, Feature Space Distillation: Adopt a teacher-student model architecture, extract multi-scale semantic features through the teacher model, and transfer the multi-scale semantic features to the student model through a distillation mechanism; S3, Dual-View Fusion: Predict the weakly and strongly augmented image data separately, and calculate their average values ​​to generate a smooth probability distribution; S4, Uncertainty Estimation: Use a denoising autoencoder to learn and recover clean signals from noise, generating an uncertainty map; S5, Comprehensive Loss Calculation and Model Training: Design a comprehensive loss function to guide model training.

[0008] Furthermore, in step S1, the weak enhancement includes random rotation, random flipping, random cropping, and scale variation; the strong enhancement includes brightness adjustment, Gaussian noise addition, contrast adjustment, and color perturbation.

[0009] Furthermore, in step S2, the teacher model uses three-dimensional convolution operations with kernel sizes of 1x1x1 and 3x3x3 to extract multi-scale features, utilizes batch normalization and modified linear activation functions to improve the feature extraction effect, and then performs trilinear upsampling to upsample the features to achieve matching of features at different scales, thereby enhancing the model's understanding of complex features.

[0010] Furthermore, in step S3, for each unlabeled sample, two enhanced versions are generated by applying weak enhancement and strong enhancement. The enhanced image is predicted by the teacher model to generate two prediction distributions. The average of the two prediction distributions is calculated to obtain a smooth probability distribution. The probability distribution is sharpened by adjusting the temperature parameter to generate a high-confidence pseudo-label to improve the quality of the pseudo-label and thus optimize the learning process of unlabeled data.

[0011] Furthermore, in step S4, the specific steps include adding noise to the segmentation mask to generate a perturbed segmentation mask, then using a denoising autoencoder to reconstruct the perturbed segmentation mask to generate a denoised segmentation mask, calculating the difference between the denoised segmentation mask and the teacher model prediction results, generating an uncertainty map, and improving the reliability of the segmentation results by focusing on the uncertain areas of the model.

[0012] Furthermore, in step S5, a comprehensive loss function is designed, including supervision loss, distillation loss, pseudo-label loss, and uncertainty estimation loss.

[0013] Furthermore, the supervised learning loss is calculated by processing labeled data to determine the model's supervised learning loss; the distillation loss is calculated based on the feature alignment of the teacher and student models; the pseudo-label loss is calculated based on pseudo-labels generated by dual-view fusion; and the uncertainty estimation loss is calculated based on an uncertainty map to determine the model's loss in uncertain regions.

[0014] Furthermore, in step S5, the model training step includes using an exponential moving average to dynamically adjust the weights of the teacher model so that they are an exponential moving average of the weights of the student model; the student model is trained using a stochastic gradient descent algorithm, and the model parameters are optimized by combining supervised loss and unsupervised loss, and continuously iterated and updated until the model's performance on the validation data converges.

[0015] The present invention has the following beneficial effects:

[0016] 1. Reduce data acquisition costs: By using semi-supervised learning methods, unlabeled data can be effectively utilized, reducing reliance on high-quality labeled data and thus reducing the cost and time consumption of data acquisition.

[0017] 2. Improve model generalization ability: The teacher-student model architecture and feature space distillation mechanism enable the model to learn and generalize better, especially when labeled data is scarce or when dealing with novel and unseen data, showing higher robustness.

[0018] 3. Optimize pseudo-label generation: The dual-view fusion strategy and sharpening method improve the accuracy of pseudo-labels, thereby improving the utilization efficiency of unlabeled data and further enhancing the segmentation performance of the model.

[0019] 4. Enhanced uncertainty estimation: Using a pre-trained autoencoder for uncertainty estimation helps the model better understand and handle uncertainties in the prediction, thus improving the reliability of the segmentation results.

[0020] 5. Comprehensive Loss Function-Guided Training: Design a comprehensive loss function for the model, including supervised loss, distillation loss, pseudo-label loss, and uncertainty estimation loss, to comprehensively guide students in training their models and improve the overall performance of the models. Attached Figure Description

[0021] Figure 1 This is a technical block diagram of an embodiment of the present invention;

[0022] Figure 2 This is a flowchart illustrating the execution of the algorithm model in an embodiment of the present invention.

[0023] Figure 3 This is a model diagram of the data augmentation and feature space distillation techniques according to an embodiment of the present invention;

[0024] Figure 4This is a model diagram of the dual-view fusion and uncertainty estimation technology according to an embodiment of the present invention;

[0025] Figure 5 This is a visualization diagram of the LA dataset according to an embodiment of the present invention;

[0026] Figure 6 This is a schematic diagram of the LiTS dataset in an embodiment of the present invention. Detailed Implementation

[0027] The technical solutions of the present invention will be further described below with reference to the accompanying drawings and embodiments.

[0028] This embodiment provides a semi-supervised medical image segmentation method based on dual-view complementary consistency, such as... Figure 1 As shown, it includes the following steps:

[0029] Step 1: First, labeled and unlabeled medical image data are input into the system. Dual-view data augmentation is then performed on the input images, including weak and strong augmentation. This augmentation method can generate multi-view image data, enriching the diversity of the dataset.

[0030] Weak enhancement strategies include operations such as random rotation, random flipping, random cropping, and scaling of the image. These operations can increase the diversity of the data without changing the overall content of the image. Strong enhancement strategies include operations such as brightness adjustment, Gaussian noise addition, contrast adjustment, and color perturbation. These operations can change image features more significantly and increase the robustness of the model.

[0031] Step 2: Use a teacher-student model for feature extraction. For example... Figure 3 As shown, the teacher model extracts multi-scale semantic features from augmented data through contrastive learning and then passes these features to the student model via a feature space distillation mechanism. The multi-scale learning method utilizes multi-scale convolution operations and upsampling techniques to enhance the model's understanding of complex features, thereby improving segmentation accuracy. The student model learns using the knowledge passed from the teacher model, thus improving its generalization ability. The feature space distillation mechanism aligns features by calculating the mean squared error between the feature representations of the teacher and student models. The formula is as follows:

[0032] L distill =MSE(f teacher f student )

[0033] MSE represents the mean squared error loss function, i.e.:

[0034]

[0035] Among them, f teacherand f student These are the feature representations for the teacher model and the student model, respectively.

[0036] Step 3: Generate and optimize pseudo-labels using a dual-view fusion strategy, integrating the teacher model's predictions to adjust the learning process for unlabeled data. For example... Figure 4 As shown, the specific steps include: applying weak and strong enhancement to each unlabeled sample to generate two enhanced versions; the teacher model processes these enhanced images to generate two prediction results; calculating the average of these two prediction results to form a smooth probability distribution; and using a sharpening method to process the pseudo-labels to improve their accuracy. Further, the specific formula for the sharpening method is as follows:

[0037]

[0038] Where T is the temperature parameter, p is the probability distribution, and p' is the sharpened pseudo-label.

[0039] Step 4: Use a pre-trained autoencoder to estimate the model's uncertainty and adjust the model's reliability and segmentation accuracy. For example... Figure 3 As shown, the specific steps include: reconstructing a noisy segmentation mask using an autoencoder, learning the global anatomical consistency of the decoder output; calculating the difference between the teacher model's prediction and the decoder's reconstruction result to form an uncertainty graph. The uncertainty estimation step uses the difference between the teacher model's prediction and the autoencoder's reconstruction result to estimate the uncertainty. The formula is as follows:

[0040] U = ||p teacher -p autoencoder || 2

[0041] Where, p teacher p represents the prediction of the teacher model. autoencoder This indicates the reconstruction result of the autoencoder.

[0042] Step 5: Define a comprehensive semi-supervised loss function, including supervised loss, distillation loss, pseudo-label loss, and uncertainty estimation loss, to guide the training of student models. The comprehensive loss function is defined as follows:

[0043] L semi =L sup +L distill +L psu +L uncertainty

[0044] Among them, L sup L represents the cross-entropy loss based on labeled data. distill L represents the mean square error loss based on the feature space distillation mechanism.psu L represents the pseudo-label loss combining Dice and cross-entropy. uncertainty This represents the uncertainty estimate loss based on the difference between the teacher model prediction and the autoencoder reconstruction results.

[0045] The student model is trained using the comprehensive loss function defined above, and the parameters of the teacher model are dynamically adjusted using an exponential moving average. The specific implementation of the exponential moving average is as follows:

[0046] θ t =m·θ t-1 +(1-m)·θ′

[0047] Where, θ t The parameter θ represents the current time step. (t-1) θ' represents the parameter at the previous time step, θ' represents the new parameter update at the current time step, and m is the exponential moving average update coefficient.

[0048] The execution flow of the algorithm model in this embodiment is as follows: Figure 2 As shown, this flowchart describes the entire process from data input to final model output. First, labeled and unlabeled data are input into the system and data augmentation is performed. Then, knowledge distillation is performed through the teacher network to extract and transmit multi-scale feature information to the student network. Next, uncertainty graph guidance is obtained, and the network parameters are updated using the Adam optimizer. Subsequently, the current model is evaluated to determine if it is optimal. If so, the optimal model is saved, and the iteration is checked to ensure completion. After iteration, the final optimal model is obtained. Finally, test data is input into the model to obtain segmentation results, which are then evaluated using metrics such as Dice and Jaccard, resulting in the final output.

[0049] In this invention, two main datasets were used for experiments and validation, as shown in Figure 1 below: the LA (Left Atrium) dataset and the LiTS (Liver Tumor Segmentation Challenge) dataset. The LA dataset is primarily used for left atrium segmentation tasks, including left atrium MRI images from different patients. The dataset contains images of various resolutions acquired from different scanners, exhibiting high diversity. The images in the dataset have been expert-annotated, providing high-quality segmentation labels. The LiTS dataset is primarily used for liver and liver tumor segmentation tasks, containing multimodal CT scan images acquired from multiple hospitals. This dataset has also been expert-annotated, providing segmentation labels for the liver and liver tumors.

[0050] Table 1 Dataset Information

[0051]

[0052] To evaluate the model's performance, the dataset was split as follows: 80% of the images were used for training, and 20% were used for validation and testing to assess the model's generalization ability and practical performance.

[0053] During the model training process of this invention, detailed settings were made for each module and parameter to ensure that the model could perform well on different datasets. Data augmentation parameters were set as follows: weak augmentations included random rotation (angle range: -15° to 15°), random flipping (horizontal and vertical flipping), random cropping (cropping ratio: 0.8 to 1.0), and scaling (scale range: 0.9 to 1.1); strong augmentations included brightness adjustment (adjustment range: 0.8 to 1.2), Gaussian noise addition (mean: 0, variance: 0.01), contrast adjustment (adjustment range: 0.8 to 1.2), and color perturbation (random color channel shift). Model training parameters included: an initial learning rate of 0.001, optimized using the Adam optimizer; a batch size of 16; and a total number of training iterations of 100 epochs. In the momentum update parameters, the update coefficient m was set to 0.9 for exponential moving average.

[0054] like Figure 5 and Figure 6 The image shows the experimental visualization results on the left atrium (LA) and liver tumor segmentation (LiTS) datasets.

[0055] The system outputs optimized and adjusted medical image segmentation results, evaluating segmentation accuracy and model robustness. Experiments are conducted to assess the accuracy of the segmentation results and the robustness of the model, including the Dice similarity coefficient (DSC), Jaccard index, 95% Hausdorff distance (95HD), and average surface distance (ASD).

[0056] To verify the effectiveness of the algorithm in this application, the method in this application was tested on the same test set with the mean-teacher model (MT), the uncertainty correction pyramid consistency method (URPC), and the method based on smoothness and class separation research (SS-Net). MT improves stability by averaging model weights, URPC enhances prediction accuracy by using uncertainty correction, and SS-Net focuses on improving segmentation quality by optimizing feature representation. The algorithm in this application, by adopting a feature space distillation mechanism and using strategies such as dual-view fusion, has significantly improved various performance indicators, as detailed in Tables 2 and 3 below.

[0057] Table 2 shows the test results for 10% and 20% labeled data on the LA dataset.

[0058]

[0059] Table 3 shows the test results on the LiTS dataset with 10% and 20% labeled data.

[0060]

[0061] Based on the performance comparison results shown in Tables 2 and 3, it can be seen that the proposed method outperforms other comparative methods in both the left atrial segmentation and liver tumor segmentation tasks on both 10% and 20% labeled datasets. Specifically, the proposed method achieves Dice coefficients of 87.56% and 89.73% on the LA dataset, and 78.35% and 82.87% on the LiTS dataset, significantly higher than the MT, URPC, and SS-Net methods. Furthermore, the proposed method also demonstrates superior performance on other key metrics such as Jaccard coefficient, 95% Hausdorff distance, and mean surface distance.

[0062] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. A semi-supervised medical image segmentation method based on dual-view complementary consistency, characterized in that: Includes the following steps: S1, Data Augmentation: Perform dual-view data augmentation on the input image, including weak and strong augmentation, to generate diverse image data; S2, Feature Space Distillation: A teacher-student model architecture is adopted. Multi-scale semantic features are extracted through the teacher model, and the multi-scale semantic features are transferred to the student model through a distillation mechanism. S3, Dual-view fusion: Predict the weakly enhanced and strongly enhanced image data separately, and calculate their average values ​​to generate a smooth probability distribution; In step S3, for each unlabeled sample, apply weak and strong enhancement to generate two enhanced versions, predict the enhanced image separately using the teacher model, generate two prediction distributions, calculate the average value of the two prediction distributions to obtain a smooth probability distribution, and sharpen the probability distribution by adjusting the temperature parameter to generate a high-confidence pseudo-label to improve the quality of the pseudo-label, thereby optimizing the learning process of unlabeled data; S4, Uncertainty Estimation: Utilize a denoising autoencoder to learn and recover a clean signal from noise, generating an uncertainty map; In step S4, the specific steps include adding noise to the segmentation mask to generate a perturbed segmentation mask, then using a denoising autoencoder to reconstruct the perturbed segmentation mask to generate a denoised segmentation mask, calculating the difference between the denoised segmentation mask and the teacher model prediction results, generating an uncertainty map, and improving the reliability of the segmentation results by focusing on the uncertain areas of the model. S5, Comprehensive Loss Calculation and Model Training: Design a comprehensive loss function to guide model training; In step S5, design the comprehensive loss function, which includes supervised loss, distillation loss, pseudo-label loss, and uncertainty estimation loss; supervised loss is calculated by processing labeled data to calculate the supervised learning loss of the model; distillation loss is calculated based on feature alignment between the teacher model and the student model; pseudo-label loss is calculated based on pseudo-labels generated by dual-view fusion; uncertainty estimation loss is calculated based on the uncertainty map to calculate the model's loss in uncertain regions.

2. The semi-supervised medical image segmentation method based on dual-view complementary consistency according to claim 1, characterized in that: In step S1, the weak enhancement includes random rotation, random flipping, random cropping, and scale variation; the strong enhancement includes brightness adjustment, Gaussian noise addition, contrast adjustment, and color perturbation.

3. The semi-supervised medical image segmentation method based on dual-view complementary consistency according to claim 2, characterized in that: In step S2, the teacher model uses three-dimensional convolution operations with kernel sizes of 1x1x1 and 3x3x3 to extract multi-scale features. It uses batch normalization and modified linear activation functions to improve the feature extraction effect. Then, it upsamples the features through trilinear upsampling to achieve matching of features at different scales, thereby enhancing the model's understanding of complex features.

4. The semi-supervised medical image segmentation method based on dual-view complementary consistency according to claim 1, characterized in that: In step S5, the model training steps include using an exponential moving average to dynamically adjust the weights of the teacher model so that they are an exponential moving average of the weights of the student model; the student model is trained using a stochastic gradient descent algorithm, and the model parameters are optimized by combining supervised loss and unsupervised loss, and continuously iterated and updated until the model's performance on the validation data converges.

Images