A staged semi-supervised medical image segmentation method based on region-mixed assisted localization

By employing a three-stage training approach and a collaborative average teacher framework, the problems of insufficient credibility of pseudo-labels and inadequate utilization of labeled data in semi-supervised medical image segmentation are addressed, achieving high accuracy and good generalization ability in medical image segmentation under conditions of limited labels.

CN122821138BActive Publication Date: 2026-11-10CHINA JILIANG UNIV +1
2 Cites 0 Cited by

Patent Information

Application Number
CN202611231252.4
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2026-08-14
Publication Date
2026-11-10
Estimated Expiration
2046-08-14

AI Technical Summary

Technical Problem

Existing semi-supervised medical image segmentation methods tend to converge or degenerate prematurely during training, have insufficient reliability of pseudo-labels, and do not fully utilize local supervision information from labeled data, making it difficult to improve the segmentation accuracy and generalization ability of the model under limited labeled conditions.

Method used

A three-stage training approach is adopted: basic augmented supervision, local region random mixing, and error localization assistance. A collaborative average teacher framework is constructed, and reliable pseudo-labels are generated by using local region mixing supervision and error localization assistance networks to suppress interference from erroneous pseudo-labels and improve the utilization quality of unlabeled images.

Benefits of technology

With limited labeled data, this study improved the utilization rate of labeled data and the reliability of pseudo-labels for unlabeled data, thereby enhancing the accuracy and generalization ability of medical image segmentation.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN122821138B_ABST
    Figure CN122821138B_ABST
Patent Text Reader

Abstract

This invention belongs to the fields of medical image processing and semi-supervised deep learning technology, and relates to a staged semi-supervised medical image segmentation method based on region-mixed assisted localization. This method acquires unlabeled medical images, labeled medical images, and their corresponding labels. It constructs a collaborative average teacher framework including one teacher network and two student networks, and sequentially performs basic augmented supervision training, local region random mixing training, and error localization-assisted training. Hybrid labels are generated using teacher pseudo-labels and labeled medical image labels. Unreliable pixels are masked using a binary reliability mask before loss calculation. After training, the teacher network outputs a predicted medical image segmentation map. This invention can improve the utilization rate of labeled data and the reliability of pseudo-labels for unlabeled images, and improve the segmentation accuracy and generalization ability of medical images under conditions of limited labeling.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This invention belongs to the field of medical image processing and semi-supervised deep learning technology, and relates to a staged semi-supervised medical image segmentation method based on region hybrid assisted localization. Background Technology

[0002] Medical image segmentation is a crucial component of computer-aided diagnostic systems, aiming to locate and separate target tissues or lesion regions from medical images. In medical image segmentation tasks, such as colorectal polyp segmentation, the morphology, scale, and boundaries of target regions vary significantly, and the segmentation results directly affect the reliability of subsequent detection, analysis, and clinical diagnostic support.

[0003] Deep learning methods such as convolutional neural networks have been widely applied to medical image segmentation, but data-driven supervised segmentation networks typically rely on a large number of accurately labeled samples. Medical image annotation requires specialized knowledge, is costly and time-consuming, and the complex lesion morphology and background conditions in different images make it difficult to obtain sufficient high-quality labeled data. At the same time, a large number of unlabeled medical images exist in medical institutions and public datasets. How to effectively utilize unlabeled images with a limited number of labeled images is an important issue for improving the performance of medical image segmentation.

[0004] Semi-supervised medical image segmentation methods typically utilize labeled data to provide supervision and leverage the latent distribution information in unlabeled data to enhance the model's generalization ability. Existing semi-supervised methods mainly fall into two categories: pseudo-label generation and consistency regularization. Pseudo-label generation methods predict unlabeled images using a trained model and use the prediction results as pseudo-labels for subsequent training; consistency regularization methods utilize unlabeled data by constraining the prediction results of the same image to remain consistent under data perturbation or model perturbation.

[0005] The average teacher model constructs a teacher network through self-ensemble of student network weights, enabling the teacher network to provide more stable supervision information to the student networks. However, multi-network cross-supervision is prone to problems such as premature convergence or degeneration of co-training into self-training during training. Furthermore, if the pseudo-labels of unlabeled images contain erroneous pixels, these erroneous pseudo-labels will continuously interfere with network optimization during loss calculation, leading to insufficient reliability of the pseudo-labels. Existing methods also suffer from insufficient utilization of local supervision information in labeled data, making it difficult to obtain high-confidence pseudo-labels while maintaining model differentiation. Summary of the Invention

[0006] To address the aforementioned technical problems in the existing technology, this invention proposes a staged semi-supervised medical image segmentation method based on region hybrid assisted localization, the specific technical solution of which is as follows:

[0007] A staged semi-supervised medical image segmentation method based on region hybrid assisted localization includes: acquiring training data containing unlabeled medical images, labeled medical images and their corresponding labels; constructing and training a collaborative average teacher framework including one teacher network and two student networks; and outputting a medical image segmentation prediction map by the teacher network after training. The training is performed in the following three stages:

[0008] Phase 1, Basic Enhanced Supervised Training: Calculate the supervision loss of each network, and train the collaborative average teacher framework with the goal of minimizing the supervision loss, so that the two student networks can gain initial segmentation ability and form stable supervision through the teacher network;

[0009] Phase 2, Local Region Random Mixing Training: Labeled and unlabeled medical image data are randomly mixed locally and then used together in supervised training between two student networks and a teacher network to provide reliable regional supervision for unlabeled medical images;

[0010] Phase 3, Error Localization Assisted Training: A binary reliability mask is used to shield unreliable pixels in the student network segmentation results. Based on the corrected segmentation results after masking, supervised training is performed between the two student networks and between the two student networks and the teacher network to suppress the interference of erroneous pseudo-labels on network training.

[0011] Furthermore, the teacher network and the two student networks adopt the same medical image segmentation network structure, but have different initialization states; the weights of the teacher network are derived from the average self-integration of the weights of the two student networks, and the average self-integration weights are all set to 0.5, so that the teacher network can integrate the learning states of the two student networks.

[0012] Furthermore, in both Phase 1 and Phase 2, basic data augmentation is performed on the data of both unlabeled and labeled medical images. The basic data augmentation includes random horizontal flipping, random vertical flipping, and random rotation within a preset angle range.

[0013] Furthermore, in Phase 1, the calculation of the supervised loss for each network includes: calculating the supervised loss for the two student networks based on labeled medical images and their corresponding labels; calculating the cross-pseudo-supervised loss between the two student networks based on each other's outputs; calculating the average teacher supervised loss between the two student networks and the teacher network respectively; and obtaining the total loss for Phase 1 by weighted summing of the supervised loss, cross-pseudo-supervised loss, and average teacher supervised loss.

[0014] Furthermore, in stage two, a random candidate region mask with the same size as the input data is first generated. Guided by this random candidate region mask, labeled and unlabeled medical images are cropped, swapped, and recombined to obtain two sets of mixed region data. Simultaneously, the unlabeled medical images are input into the teacher network to generate teacher pseudo-labels. The teacher pseudo-labels are then mixed with the labels of the labeled medical images according to the random candidate region mask to generate mixed labels corresponding to the mixed data regions. The two sets of mixed region data are then input into two student networks to obtain corresponding segmentation results. Based on the corresponding segmentation results and mixed labels, the average teacher supervision loss between the two student networks and the teacher network is calculated to train the two student networks.

[0015] Furthermore, in stage three, the unlabeled medical image is input into two student networks and a teacher network respectively to obtain the first student segmentation result, the second student segmentation result, and the teacher segmentation result. The first student segmentation result and the second student segmentation result are input into the error localization auxiliary network with the corresponding freezing parameters. The error localization auxiliary network performs pixel-by-pixel reliability discrimination on the potential erroneous pixels in the segmentation results, i.e., the pseudo-labels of the unlabeled medical image, and outputs a binary reliability mask.

[0016] Furthermore, in stage three, after using the binary reliability mask to mask unreliable pixels, the corrected student network segmentation result is obtained. Based on the corrected segmentation results of the two student networks, the cross-pseudo-supervision loss between the two student networks is calculated, and the average teacher supervision loss between the two student networks and the teacher network is calculated based on the teacher network segmentation result and the corrected segmentation results of the two student networks. The cross-pseudo-supervision loss and the average teacher supervision loss are weighted and summed to obtain the total loss.

[0017] Furthermore, the collaborative average teacher framework performs repeated splicing or length alignment processing on labeled data during the data loading stage to ensure that the number of aligned labeled data is consistent with the number of unlabeled data.

[0018] Furthermore, the collaborative average teacher framework scales the test medical image to 256×256 resolution during the inference phase and inputs it into the trained teacher network to generate the medical image segmentation prediction map.

[0019] Beneficial effects: This invention completes preliminary collaborative supervision, region mixing supervision, and error localization-assisted supervision through a three-stage training method, which can maintain the differences between the two student networks during training. By randomly mixing local regions, local regions of labeled images and their label information are used to participate in the training of unlabeled images, improving the utilization rate of labeled data. By generating a binary reliability mask and shielding unreliable pixels through the error localization-assisted network, the interference of erroneous pseudo-labels on loss calculation is reduced, thereby improving the utilization quality of pseudo-labels in unlabeled images and improving the segmentation accuracy and generalization ability of medical images under conditions of limited labeled data. Attached Figure Description

[0020] Figure 1 This is a flowchart of a staged semi-supervised medical image segmentation method based on region hybrid assisted localization according to the present invention.

[0021] Figure 2 This is a schematic diagram of the first training phase of the present invention.

[0022] Figure 3 This is a schematic diagram of the second training phase of the present invention.

[0023] Figure 4 This is a schematic diagram of the third training phase of the present invention.

[0024] Figure 5 This is a schematic diagram of the random mixing strategy for local data regions according to the present invention.

[0025] Figure 6 A visual comparison chart of the experiment under the condition of 10% labeled data.

[0026] Figure 7 A visual comparison chart of the experiment under the condition of 5% labeled data. Detailed Implementation

[0027] To make the objectives, technical solutions, and technical effects of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments.

[0028] like Figure 1 As shown, this embodiment provides a staged semi-supervised medical image segmentation method based on region hybrid assisted localization. This method can be used to generate medical image segmentation prediction maps under the condition that a small number of labeled medical images and a large number of unlabeled medical images participate in the training. Specifically, it includes the following steps:

[0029] S1, Obtain training data The training data is divided into labeled datasets. and unlabeled datasets .in, This represents the i-th labeled medical image. This represents the label corresponding to the i-th labeled medical image. Let N represent the j-th unlabeled medical image, and let M represent the number of labeled and unlabeled data, respectively.

[0030] To ensure that labeled data is aligned with unlabeled data in training batches, the labeled data is processed during the data loading phase. Repeated splicing or length alignment is performed to ensure that the number of labeled data points after alignment is consistent with the number of unlabeled data points. All training data is input into the teacher network as unlabeled data, and the aligned labeled data is used as input to the two student networks.

[0031] S2, Construct a collaborative average teacher framework. This framework includes one teacher network and two student networks. All three networks employ a medical image segmentation network structure, in one implementation using a basic U-shaped network structure. The three networks have identical structures but different initialization states to avoid excessive consistency in the outputs of the two student networks in the early stages of training. The teacher network weights are derived from the average self-ensemble of the weights of student network one and student network two. The average self-ensemble weights of student network one and student network two are both set to 0.5, enabling the teacher network to synthesize the learning states of the two student networks. Specifically:

[0032] = + ,

[0033] in For the teacher model, and For two student models, =0.5.

[0034] S3, Execute Training Phase One: Basic Enhanced Supervised Training, such as Figure 2 As shown. After performing basic data augmentation on the training data, it is input into the collaborative averaging teacher framework. The basic data augmentation includes random horizontal flipping, random vertical flipping, and random rotation within a preset angle range.

[0035] Two student networks based on labeled medical images and their corresponding tags The supervised loss is calculated by: calculating the cross-pseudo-supervised loss between the two student networks based on each other's outputs; calculating the average teacher-supervised loss between each student network and the teacher network; and then weighted summing the above losses to obtain the total loss for the first training phase, as shown in the following expression:

[0036] ,

[0037] ,

[0038] ,

[0039] in Indicates the partition loss. For the purpose of supervision loss, Indicates the loss due to cross-monitoring. This represents the average teacher supervision loss. The total loss is obtained by weighted summation of the losses from each component. This is used to enable the two student networks to acquire initial segmentation capabilities and to form stable supervision through the teacher network. The specific expression is:

[0040] ,

[0041] in This is the regularization parameter.

[0042] S4, Perform training phase two: local region random mixed training, such as Figure 3 and Figure 5 As shown, basic data augmentation and random mixing of local regions are performed on labeled and unlabeled medical image data to generate mixed region data; and pseudo-labels of teachers generated by the teacher network based on unlabeled medical images are used. Labels with labeled medical images Generate mixed tags by mixing tags in the same area.

[0043] Specifically, a local region random mixing strategy is introduced, which first generates random candidate region masks with the same size as the input data. In the random candidate region mask Under guidance, labeled and unlabeled medical images were cropped, interchanged, and recombined to obtain two sets of mixed region data. and The specific expression is:

[0044] ,

[0045] ,

[0046] ,

[0047] in This indicates the generation of random candidate region masks.

[0048] By randomly mixing local regions, the local annotation information in labeled images and the image content in unlabeled images are used together in the training, which improves the guiding role of labeled data in the segmentation process of unlabeled data.

[0049] Inputting unlabeled medical images into a teacher network generates teacher pseudo-labels. Teacher pseudo-labels Labels on medical images Perform region blending in the same manner as image region blending to obtain the blended region data. and Corresponding tags and The specific expression is:

[0050] ,

[0051] .

[0052] Finally, mixed region data was obtained. .

[0053] The two sets of mixed region data are then input into the two student networks to obtain corresponding segmentation results. Based on the corresponding segmentation results and the mixed labels, the average teacher supervision loss between the two student networks and the teacher network is calculated, with the specific expression as follows:

[0054] .

[0055] S5, Execute Training Phase Three: Error Localization Assisted Training, such as Figure 4 As shown, unlabeled medical images are input into student network 1, student network 2, and teacher network respectively, yielding segmentation results from the first student, the second student, and the teacher. The outputs of the two student networks are then input into the corresponding error localization auxiliary network with frozen parameters. This network performs pixel-by-pixel reliability discrimination on potentially erroneous pixels in the segmentation results (i.e., pseudo-labels of the unlabeled medical images), outputting a binary reliability mask. This mask distinguishes between reliable and unreliable pixels, ensuring that only high-reliability pixels are retained for subsequent loss calculations during training.

[0056] Based on the binary reliability mask, reliable and unreliable pixels are identified. After masking unreliable pixels, the corrected student network segmentation result is obtained. Then, based on the corrected student network segmentation result, the cross-pseudo-supervision loss between the two student networks and the average teacher supervision loss between the two student networks and the teacher network are calculated. The specific expressions are as follows:

[0057] ,

[0058] ,

[0059] in This represents the average teacher supervision loss. Indicates the loss due to cross-monitoring. and Let represent the corrected network segmentation results for the two students, respectively. The total loss is obtained by weighted summation of the two loss components. The specific expression is:

[0060] ,

[0061] in =0.5× .

[0062] In the third training phase, the error localization auxiliary network is used to suppress the interference of erroneous pseudo-labels on network training, thereby improving the quality of pseudo-label utilization.

[0063] S6 trains a collaborative average teacher framework based on the loss from three training phases and outputs a medical image segmentation prediction map using a separate teacher network during the testing phase.

[0064] In one implementation, this method is based on the PyTorch deep learning framework. The model optimization uses the AdamW optimizer, with an initial learning rate of 1e-4, a training batch size of 4, and a training epoch of 200. During the inference phase, the test medical images are scaled to 256×256 resolution and input into the trained medical image segmentation network. The teacher network outputs a medical image segmentation prediction map.

[0065] like Figure 6 and Figure 7 As shown, under conditions of limited labeled data, this method improves the utilization of labeled data and the reliability of pseudo-labels on unlabeled data through staged multi-model collaborative optimization, local region random mixing, and error localization assistance. Experimental results demonstrate that this method exhibits relatively stable segmentation performance and good generalization ability based on small-sample training on multiple polyp segmentation benchmark datasets.

[0066] exist Figure 6 and Figure 7 Among them, U-Net is a basic U-shaped convolutional neural network; PraNet (Parallel Reverse Attention Network) is a parallel reverse attention network, a classic fully supervised polyp segmentation network model; BCP (Bidirectional Copy-Paste) is a data-augmented semi-supervised / supervised learning method; MIDSS is an existing semi-supervised contrastive method; UCMT (Uncertainty-guided Collaborative Mean-Teacher) is a semi-supervised semantic segmentation method for medical images.

[0067] To demonstrate the effectiveness of this method, experimental verification was conducted on publicly available polyp segmentation datasets such as Kvasir, CVC-ClinicDB, CVC-ColonDB, ETIS, and CVC-300, and the results were compared with existing medical image segmentation methods. The features of the publicly available datasets used in the experiments are shown in Table 1.

[0068] Table 1:

[0069]

[0070] The comparison results of the present invention and the comparison method on five public datasets under the condition of 10% labeled data are shown in Table 2.

[0071] Table 2:

[0072]

[0073] The comparison results of the present invention and the comparison method on five public datasets under the condition of 5% labeled data are shown in Table 3.

[0074] Table 3:

[0075]

[0076] Wherein, mDice (Mean Dice Coefficient): the average Dice coefficient; mIoU (Mean Intersection over Union): Mean Intersection over Union Ratio; Sm (S-measure): Structural similarity measure; Fm (F-measure): F-measure / weighted F-score; meanEm (Mean E-measure): Average enhanced alignment metric; maxEm (Maximum E-measure): Maximum enhanced alignment metric; MAE (Mean Absolute Error): Mean Absolute Error; mDice and mIoU are commonly used evaluation metrics in medical image segmentation tasks. S-measure, F-measure, Mean E-measure, and Max E-measure are used to measure the consistency between the predicted segmentation results and the ground truth annotations from different dimensions. MAE is used to measure the mean absolute error. Except for MAE, the higher the value of the other metrics, the better the segmentation results.

[0077] In summary, this invention provides a semi-supervised medical image segmentation method that combines staged collaborative training, random mixing of local regions, and error localization assistance to make full use of labeled and unlabeled images, improve the reliability of pseudo-labels during training, and enhance segmentation accuracy and generalization ability under conditions of a small number of labeled samples.

[0078] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention in any way. Although the implementation process of the present invention has been described in detail above, those skilled in the art can still modify the technical solutions described in the foregoing examples or make equivalent substitutions for some of the technical features. All modifications and equivalent substitutions made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A staged semi-supervised medical image segmentation method based on region hybrid assisted localization, comprising: Training data containing unlabeled medical images, labeled medical images, and their corresponding labels is acquired. A collaborative average teacher framework, comprising one teacher network and two student networks, is constructed and trained. After training, the teacher network outputs a medical image segmentation prediction map. The training is characterized by sequentially executing the following three stages: Phase 1, Basic Enhanced Supervised Training: Calculate the supervision loss of each network, and train the collaborative average teacher framework with the goal of minimizing the supervision loss, so that the two student networks can gain initial segmentation ability and form stable supervision through the teacher network; Phase 2, Local Region Random Mixing Training: Labeled and unlabeled medical image data are randomly mixed locally and then used together in supervised training between two student networks and a teacher network to provide reliable regional supervision for unlabeled medical images; Phase 3, Error Localization Assisted Training: A binary reliability mask is used to shield unreliable pixels in the student network segmentation results. Based on the corrected segmentation results after shielding, supervised training is performed between the two student networks and between the two student networks and the teacher network to suppress the interference of erroneous pseudo-labels on network training. In Phase 1, the calculation of the supervised loss for each network includes: calculating the supervised loss for two student networks based on labeled medical images and their corresponding labels; calculating the cross pseudo-supervised loss between the two student networks based on each other's outputs; calculating the average teacher supervised loss for each of the two student networks and the teacher network; and obtaining the total loss for Phase 1 by weighted summing of the supervised loss, cross pseudo-supervised loss, and average teacher supervised loss. In Phase 2, a random candidate region mask with the same size as the input data is first generated. Guided by this random candidate region mask, labeled and unlabeled medical images are cropped, swapped, and recombined to obtain two sets of mixed region data. Simultaneously, the unlabeled medical images are input into the teacher network to generate teacher pseudo-labels. The teacher pseudo-labels are then mixed with the labels of the labeled medical images according to the random candidate region mask to generate mixed labels corresponding to the mixed data regions. The two sets of mixed region data are then input into two student networks to obtain corresponding segmentation results. Based on the corresponding segmentation results and mixed labels, the average teacher supervision loss between the two student networks and the teacher network is calculated to train the two student networks. In stage three, the unlabeled medical image is input into two student networks and a teacher network respectively to obtain the first student segmentation result, the second student segmentation result, and the teacher segmentation result. The first student segmentation result and the second student segmentation result are input into the error localization auxiliary network with the corresponding freezing parameters. The error localization auxiliary network performs pixel-by-pixel reliability discrimination on the potential erroneous pixels in the segmentation results, i.e., the pseudo-labels of the unlabeled medical image, and outputs a binary reliability mask.

2. The method as described in claim 1, characterized in that, The teacher network and the two student networks adopt the same medical image segmentation network structure, but have different initialization states. The weights of the teacher network are derived from the average self-integration of the weights of the two student networks. The average self-integration weights are all set to 0.5, so that the teacher network can integrate the learning states of the two student networks.

3. The method as described in claim 1, characterized in that, In both Phase 1 and Phase 2, basic data augmentation is performed on the data of both unlabeled and labeled medical images. The basic data augmentation includes random horizontal flipping, random vertical flipping, and random rotation within a preset angle range.

4. The method as described in claim 1, characterized in that, In stage three, the corrected student network segmentation results are obtained by using the binary reliability mask to block unreliable pixels. The cross-pseudo-supervision loss between the two student networks is calculated based on the corrected student network segmentation results, and the average teacher supervision loss between the two student networks and the teacher network is calculated based on the teacher network segmentation results and the corrected student network segmentation results. The total loss is obtained by weighted summation of the cross-pseudo-supervision loss and the average teacher supervision loss.

5. The method as described in claim 1, characterized in that, The collaborative average teacher framework performs repeated splicing or length alignment on labeled data during the data loading phase to ensure that the number of aligned labeled data is consistent with the number of unlabeled data.

6. The method as described in claim 1, characterized in that, The collaborative average teacher framework scales the test medical image to 256×256 resolution during the inference phase and inputs it into the trained teacher network to generate the medical image segmentation prediction map.

Citation Information

Patent Citations

  • Semi-supervised medical image segmentation method based on region selection enhancement

    CN122223041A

  • Semi-supervised medical image segmentation method based on boundary information enhancement

    CN122391110A