Staged semi-supervised medical image segmentation method based on region hybrid aided positioning
Patent Information
- Application Number
- CN202611231252.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-08-14
- Publication Date
- 2026-09-25
AI Technical Summary
然而，多网络交叉监督在训练过程中容易出现网络过早收敛或共同训练退化为自训练的问题；同时，若无标签图像的伪标签存在错误像素，错误伪标签会在损失计算中持续干扰网络优化，导致伪标签可信度不足
[0019]有益效果：本发明通过三阶段训练方式分别完成初步协同监督、区域混合监督以及误差定位辅助监督，能够在训练过程中保持两个学生网络之间的差异性；通过局部区域随机混合，使有标签图像的局部区域及其标签信息参与无标签图像训练，提高有标签数据利用率；通过误差定位辅助网络生成二值可靠性掩码并屏蔽不可靠像素，降低错误伪标签对损失计算的干扰，从而提高无标签图像伪标签的利用质量，并改善少量有标签数据条件下的医学图像分割精度和泛化能力。
Smart Images
Figure CN122821138A_ABST
Abstract
Description
TECHNICAL FIELD
[0001] The application belongs to the technical field of medical image processing and semi-supervised deep learning, and relates to a phased semi-supervised medical image segmentation method based on regional hybrid auxiliary positioning. BACKGROUND
[0002] Medical image segmentation is a key link in a computer-aided diagnosis system, and its goal is to locate and separate target tissues or lesion regions from medical images. In medical image segmentation tasks represented by colon polyp segmentation, the morphology, scale and boundary of the target region are quite different, and the segmentation result directly affects the reliability of subsequent detection, analysis and clinical auxiliary judgment.
[0003] Deep learning methods such as convolutional neural networks have been widely used in medical image segmentation, but data-driven supervised segmentation networks usually rely on a large number of accurately labeled samples. Medical image labeling requires professional knowledge, and the labeling cost is high and the cycle is long. The morphology of the lesion and the background condition in different images are complex, making it difficult to obtain sufficient high-quality labeled data. At the same time, there are a large number of unlabeled medical images in medical institutions and public data sets. How to effectively use unlabeled images under the condition of a small amount of labeled images is an important problem to improve the performance of medical image segmentation.
[0004] Semi-supervised medical image segmentation methods usually use labeled data to provide supervision information and use the potential distribution information in unlabeled data to improve the generalization ability of the model. Existing semi-supervised methods mainly include two types of pseudo-label generation and consistency regularization. The pseudo-label generation method uses the trained model to predict the unlabeled image, and uses the prediction result as the pseudo-label for subsequent training; the consistency regularization method uses the constraint that the prediction results of the same image under data disturbance or model disturbance remain consistent to utilize the unlabeled data.
[0005] The average teacher model constructs a teacher network through the self-integration of the student network weights, so that the teacher network provides more stable supervision information to the student network. However, multi-network cross-supervision is prone to premature convergence of the network or degeneration into self-training during the training process; at the same time, if the pseudo-label of the unlabeled image has an error pixel, the false pseudo-label will continuously interfere with the network optimization in the loss calculation, resulting in insufficient credibility of the pseudo-label. The existing method also has the problem of insufficient utilization of local supervision information of labeled data, which is difficult to obtain high-confidence pseudo-labels while maintaining the diversity of the model. SUMMARY
[0006] In order to solve the above technical problems in the prior art, the application provides a phased semi-supervised medical image segmentation method based on regional hybrid auxiliary positioning, and the specific technical scheme is as follows:
[0007] The application discloses a phased semi-supervised medical image segmentation method based on regional mixed auxiliary positioning, and relates to the technical field of medical image segmentation.
[0008] In the first phase, basic enhancement supervision training is performed, the supervision loss of each network is calculated, the collaborative average teacher framework is trained to minimize the supervision loss, the two student networks obtain preliminary segmentation capability, and stable supervision is formed through the teacher network.
[0009] In the second phase, local region random mixing training is performed, the data of the labeled medical image and the unlabeled medical image are randomly mixed in local regions, and then are jointly used in the supervision training between the two student networks and the teacher network to provide regional reliable supervision for the unlabeled medical image.
[0010] In the third phase, error positioning auxiliary training is performed, the binary reliability mask is used to shield unreliable pixels in the segmentation result of the student network, the corrected segmentation result after shielding is used to perform supervision training between the two student networks and between the two student networks and the teacher network, and the interference of the error pseudo-label on the network training is suppressed.
[0011] Further, the teacher network and the two student networks adopt the same medical image segmentation network structure but have different initialization states, the teacher network weight is derived from the average self-integration of the two student network weights, and the average self-integration weight is set to 0.5, so that the teacher network can integrate the learning states of the two student networks.
[0012] Further, in the first phase and the second phase, the data of the unlabeled medical image and the labeled medical image are subjected to basic data enhancement, and the basic data enhancement includes random horizontal flipping, random vertical flipping and random rotation in a preset angle range.
[0013] Further, in the first phase, the calculation of the supervision loss of each network includes the following steps: the two student networks calculate the supervised loss based on the labeled medical image and the corresponding label; the two student networks calculate the cross pseudo-supervision loss according to the outputs of each other; the two student networks and the teacher network calculate the average teacher supervision loss; and the total loss of the first phase is obtained by weighted summation of the supervised loss, the cross pseudo-supervision loss and the average teacher supervision loss.
[0014] Further, in the second stage, first, a random candidate region mask with the same size as the input data is generated, and the labeled medical images and unlabeled medical images are regionally cropped, interchanged and recombined under the guidance of the random candidate region mask to obtain two groups of mixed region data; at the same time, the unlabeled medical images are input into the teacher network to generate teacher pseudo labels, and the teacher pseudo labels and the labels of the labeled medical images are regionally mixed according to the random candidate region mask to generate mixed labels corresponding to the mixed data regions; the two groups of mixed region data are input into the two student networks respectively to obtain corresponding segmentation results; based on the corresponding segmentation results and the mixed labels, the average teacher supervision loss between the two student networks and the teacher network is calculated to train the two student networks.
[0015] Further, in the third stage, the unlabeled medical images are input into the two student networks and the teacher network respectively to obtain the first student segmentation result, the second student segmentation result and the teacher segmentation result; the first student segmentation result and the second student segmentation result are input into the error positioning auxiliary network with the corresponding frozen parameters, and the error positioning auxiliary network performs pixel-by-pixel reliability discrimination on the potential error pixels in the segmentation results, i.e. the pseudo labels of the unlabeled medical images, and outputs a binary reliability mask.
[0016] Further, in the third stage, the binary reliability mask is used to shield unreliable pixels to obtain corrected student network segmentation results, and the cross pseudo supervision loss between the two student networks is calculated based on the corrected student network segmentation results, and the average teacher supervision loss between the two student networks and the teacher network is calculated based on the teacher network segmentation result and the corrected student network segmentation results; the cross pseudo supervision loss and the average teacher supervision loss are weighted and summed to obtain the total loss.
[0017] Further, the cooperative average teacher framework performs repeated splicing or length alignment processing on the labeled data in the data loading stage, so that the number of aligned labeled data is consistent with the number of unlabeled data.
[0018] Further, in the inference stage of the cooperative average teacher framework, the test medical images are scaled to 256x256 resolution and input into the trained teacher network to generate the medical image segmentation prediction map.
[0019] Beneficial effects: The present application can maintain the difference between the two student networks during the training process by completing preliminary cooperative supervision, regional mixed supervision and error positioning auxiliary supervision through a three-stage training method; by local region random mixing, the local region of the labeled image and its label information are involved in the training of the unlabeled image, improving the utilization rate of labeled data; by the error positioning auxiliary network, a binary reliability mask is generated to shield unreliable pixels, reducing the interference of false labels on loss calculation, thereby improving the utilization quality of pseudo labels of unlabeled images and improving the medical image segmentation precision and generalization ability under the condition of a small amount of labeled data. BRIEF DESCRIPTION OF DRAWINGS
[0020] Figure 1 A regional mixed auxiliary positioning-based phased semi-supervised medical image segmentation method flowchart of the present application.
[0021] Figure 2 A schematic diagram of training phase one of the present application.
[0022] Figure 3 A schematic diagram of training phase two of the present application.
[0023] Figure 4 A schematic diagram of training phase three of the present application.
[0024] Figure 5 A schematic diagram of the data local region random mixing strategy of the present application.
[0025] Figure 6 An experimental visualization comparison chart under the condition of 10% labeled data.
[0026] Figure 7 An experimental visualization comparison chart under the condition of 5% labeled data. DETAILED DESCRIPTION
[0027] In order to make the purpose, technical scheme and technical effects of the present application clearer, the present application is further described in detail below in combination with the drawings and examples of the specification.
[0028] As Figure 1 shown, the present embodiment provides a regional mixed auxiliary positioning-based phased semi-supervised medical image segmentation method, which can be used to generate a medical image segmentation prediction map under the condition that a small amount of labeled medical images and a large number of unlabeled medical images participate in training together, and specifically includes the following steps:
[0029] S1, obtaining training data , dividing the training data into a labeled data set and an unlabeled data set . Among them, Xi represents the i-th labeled medical image, This represents the label corresponding to the i-th labeled medical image. Let N represent the j-th unlabeled medical image, and let M represent the number of labeled and unlabeled data, respectively.
[0030] To ensure that labeled data is aligned with unlabeled data in training batches, the labeled data is processed during the data loading phase. Repeated splicing or length alignment is performed to ensure that the number of labeled data points after alignment is consistent with the number of unlabeled data points. All training data is input into the teacher network as unlabeled data, and the aligned labeled data is used as input to the two student networks.
[0031] S2, Construct a collaborative average teacher framework. This framework includes one teacher network and two student networks. All three networks employ a medical image segmentation network structure, in one implementation using a basic U-shaped network structure. The three networks have identical structures but different initialization states to avoid excessive consistency in the outputs of the two student networks in the early stages of training. The teacher network weights are derived from the average self-ensemble of the weights of student network one and student network two. The average self-ensemble weights of student network one and student network two are both set to 0.5, enabling the teacher network to synthesize the learning states of the two student networks. Specifically:
[0032] = + ,
[0033] in For the teacher model, and For two student models, =0.5.
[0034] S3, Execute Training Phase One: Basic Enhanced Supervised Training, such as Figure 2 As shown. After performing basic data augmentation on the training data, it is input into the collaborative averaging teacher framework. The basic data augmentation includes random horizontal flipping, random vertical flipping, and random rotation within a preset angle range.
[0035] Two student networks based on labeled medical images and their corresponding tags The supervised loss is calculated by: calculating the cross-pseudo-supervised loss between the two student networks based on each other's outputs; calculating the average teacher-supervised loss between each student network and the teacher network; and then weighted summing the above losses to obtain the total loss for the first training phase, as shown in the following expression:
[0036] ,
[0037] ,
[0038] ,
[0039] wherein denotes the segmentation loss, is a supervised loss, denotes the cross pseudo-supervised loss, denotes the average teacher supervision loss. The total loss is obtained by weighted sum of each part loss, i.e. , which is used to promote the two student networks to obtain preliminary segmentation ability and form stable supervision through the teacher network, and the specific expression is:
[0040] ,
[0041] wherein is a regularization parameter.
[0042] S4, performing training phase two: local region random mixing training, as shown in Figure 3 and Figure 5 . The labeled medical image data and unlabeled medical image data are subjected to basic data enhancement and local region random mixing to generate mixed region data; and the teacher pseudo-label generated by the teacher network based on the unlabeled medical image is mixed with the label of the labeled medical image to generate a mixed label in the same region mixing manner.
[0043] Specifically, a local region random mixing strategy is introduced. The strategy first generates a random candidate region mask with the same size as the input data Under the guidance of the random candidate region mask , the labeled medical image and the unlabeled medical image are subjected to region cropping, interchanging and recombination to obtain two groups of mixed region data , and the specific expression is:
[0044] ,
[0045] ,
[0046] ,
[0047] wherein denotes random candidate region mask generation.
[0048] Through local region random mixing, the local annotation information in the labeled image and the image content in the unlabeled image jointly participate in the training, thereby improving the guiding role of the labeled data on the segmentation process of the unlabeled data.
[0049] input the unlabeled medical image into the teacher network to generate a teacher pseudo label , the teacher pseudo label , the label of the labeled medical image , and the mixed region data is obtained by mixing the image region in the same way as the image region mixing and , the corresponding label and , and the specific expression is
[0050] ,
[0051] .
[0052] Finally, the mixed region data is obtained .
[0053] Then, the two sets of mixed region data are respectively input into the two student networks to obtain the corresponding segmentation results, and the average teacher supervision loss between the two student networks and the teacher network is calculated based on the corresponding segmentation results and the mixed label, and the specific expression is
[0054] .
[0055] S5, execute the third training phase: error positioning auxiliary training, as shown in Figure 4 , the unlabeled medical image is input into the student network one, the student network two and the teacher network to obtain the first student segmentation result, the second student segmentation result and the teacher segmentation result. The output results of the two student networks are input into the error positioning auxiliary network with the corresponding frozen parameters, and the error positioning auxiliary network performs pixel-by-pixel reliability discrimination on the potential error pixels in the segmentation result, i.e. the pseudo label of the unlabeled medical image, and outputs a binary reliability mask. The binary reliability mask is used to distinguish between reliable pixels and unreliable pixels, so that only high reliability pixels are retained in the training process to participate in subsequent loss calculation.
[0056] According to the binary reliability mask, the reliable pixels and the unreliable pixels are identified, the corrected student network segmentation result is obtained after shielding the unreliable pixels, and the cross pseudo supervision loss between the two student networks and the average teacher supervision loss between the two student networks and the teacher network are calculated based on the corrected student network segmentation result, and the specific expression is
[0057] ,
[0058] ,
[0059] wherein represents the average teacher supervision loss, represents the cross pseudo supervision loss, and The two corrected student network segmentation results are represented by and respectively. The total loss is obtained by weighted summation of the two parts of loss , and the specific expression is:
[0060] ,
[0061] wherein = 0.5 x .
[0062] The third training stage suppresses the interference of false pseudo-labels on network training through error positioning auxiliary network, thereby improving the quality of pseudo-label utilization.
[0063] S6, according to the loss of the three training stages, the collaborative mean teacher framework is trained, and a single teacher network is used to output a medical image segmentation prediction map in the test stage.
[0064] In an embodiment, the method is implemented based on the PyTorch deep learning framework, the AdamW optimizer is used for model optimization, the initial learning rate is set to 1e-4, the training batch size is 4, and the training round number is set to 200; in the inference stage, the test medical image is scaled to 256x256 resolution and input into the trained medical image segmentation network, and the medical image segmentation prediction map is generated by the teacher network output.
[0065] As shown in Figure 6 and Figure 7 , under the condition of a small amount of labeled data, the method improves the utilization rate of labeled data and the reliability of pseudo-labels of unlabeled data through stage-based multi-model collaborative optimization, local region random mixing and error positioning assistance. Experimental results show that the method shows relatively stable segmentation performance and good generalization ability based on small sample training on multiple polyp segmentation benchmark datasets.
[0066] In Figure 6 and Figure 7 , U-Net is a basic U-shaped convolutional neural network; PraNet (Parallel Reverse Attention Network) is a classic fully supervised polyp segmentation network model; BCP (Bidirectional Copy-Paste) is a semi-supervised / supervised learning method based on data augmentation; MIDSS is an existing semi-supervised comparison method; UCMT (Uncertainty-guided Collaborative Mean-Teacher) is a semi-supervised semantic segmentation method for medical images.
[0067] To illustrate the effectiveness of the present method, the staged semi-supervised medical image segmentation method is experimentally verified on public polyp segmentation datasets such as Kvasir, CVC-ClinicDB, CVC-ColonDB, ETIS and CVC-300, and compared with existing medical image segmentation methods. The characteristics of the public datasets used in the experiment are shown in Table 1.
[0068] Table 1:
[0069]
[0070] The comparison results of the present application and the comparative method on five public datasets under the condition of 10% labeled data are shown in Table 2.
[0071] Table 2:
[0072]
[0073] The comparison results of the present application and the comparative method on five public datasets under the condition of 5% labeled data are shown in Table 3.
[0074] Table 3:
[0075]
[0076] mDice (Mean Dice Coefficient): average Dice coefficient; mIoU (Mean Intersection over Union): average intersection over union; Sm (S-measure): structural similarity measure; Fm (F-measure): F measure / weighted F score; meanEm (Mean E-measure): average enhanced alignment measure; maxEm (Maximum E-measure): maximum enhanced alignment measure; MAE (Mean Absolute Error): mean absolute error; mDice and mIoU are common evaluation indicators in medical image segmentation tasks, S-measure, F-measure, Mean E-measure and Max E-measure are used to measure the consistency between the predicted segmentation result and the true label from different dimensions, and MAE is used to measure the average absolute error; except MAE, the higher the value of the remaining indicators, the better the segmentation result.
[0077] In conclusion, the semi-supervised medical image segmentation method can combine stage-wise co-training, local region random mixing and error positioning assistance, fully utilize the labeled images and unlabeled images, improve the reliability of pseudo labels in the training process, and improve the segmentation accuracy and generalization ability under the condition of a small number of labeled samples.
[0078] The above merely describes the preferred embodiments of the present application, and does not limit the present application in any form. Although the implementation process of the present application has been described in detail above, those skilled in the art can modify the technical solutions recorded in the foregoing examples, or replace some of the technical features equivalently. Any modification, equivalent replacement, etc. within the spirit and principles of the present application shall be included in the protection scope of the present application.
Claims
1. A staged semi-supervised medical image segmentation method based on region hybrid assisted localization, comprising: Training data containing unlabeled medical images, labeled medical images, and their corresponding labels is acquired. A collaborative average teacher framework, comprising one teacher network and two student networks, is constructed and trained. After training, the teacher network outputs a medical image segmentation prediction map. The training is characterized by the following three stages performed sequentially: Phase 1, Basic Enhanced Supervised Training: Calculate the supervision loss of each network, and train the collaborative average teacher framework with the goal of minimizing the supervision loss, so that the two student networks can gain initial segmentation ability and form stable supervision through the teacher network; Phase 2, Local Region Random Mixing Training: Labeled and unlabeled medical image data are randomly mixed locally and then used together in supervised training between two student networks and a teacher network to provide reliable regional supervision for unlabeled medical images; Phase 3, Error Localization Assisted Training: A binary reliability mask is used to shield unreliable pixels in the student network segmentation results. Based on the corrected segmentation results after masking, supervised training is performed between the two student networks and between the two student networks and the teacher network to suppress the interference of erroneous pseudo-labels on network training.
2. The method as described in claim 1, characterized in that, The teacher network and the two student networks adopt the same medical image segmentation network structure, but have different initialization states; the weights of the teacher network are derived from the average self-integration of the weights of the two student networks, and the average self-integration weights are all set to 0.5, so that the teacher network can integrate the learning states of the two student networks.
3. The method as described in claim 1, characterized in that, In both Phase 1 and Phase 2, basic data augmentation is performed on the data of both unlabeled and labeled medical images. The basic data augmentation includes random horizontal flipping, random vertical flipping, and random rotation within a preset angle range.
4. The method as described in claim 1, characterized in that, In Phase 1, the calculation of the supervised loss for each network includes: for the two student networks, the supervised loss is calculated based on the labeled medical images and their corresponding labels; the cross pseudo-supervised loss is calculated between the two student networks based on each other's outputs; the average teacher supervised loss is calculated between the two student networks and the teacher network respectively; and the total loss for Phase 1 is obtained by weighted summation of the supervised loss, cross pseudo-supervised loss, and average teacher supervised loss.
5. The method as described in claim 1, characterized in that, In Phase 2, a random candidate region mask with the same size as the input data is first generated. Guided by this random candidate region mask, labeled and unlabeled medical images are cropped, swapped, and recombined to obtain two sets of mixed region data. Simultaneously, the unlabeled medical images are input into the teacher network to generate teacher pseudo-labels. The teacher pseudo-labels are then mixed with the labels of the labeled medical images according to the random candidate region mask to generate mixed labels corresponding to the mixed data regions. The two sets of mixed region data are then input into two student networks to obtain corresponding segmentation results. Based on the corresponding segmentation results and mixed labels, the average teacher supervision loss between the two student networks and the teacher network is calculated to train the two student networks.
6. The method as described in claim 1, characterized in that, In stage three, the unlabeled medical image is input into two student networks and a teacher network respectively to obtain the first student segmentation result, the second student segmentation result, and the teacher segmentation result. The first student segmentation result and the second student segmentation result are input into the error localization auxiliary network with the corresponding freezing parameters. The error localization auxiliary network performs pixel-by-pixel reliability discrimination on the potential erroneous pixels in the segmentation results, i.e., the pseudo-labels of the unlabeled medical image, and outputs a binary reliability mask.
7. The method as described in claim 1, characterized in that, In stage three, the corrected student network segmentation results are obtained by using the binary reliability mask to block unreliable pixels. The cross-pseudo-supervision loss between the two student networks is calculated based on the corrected student network segmentation results, and the average teacher supervision loss between the two student networks and the teacher network is calculated based on the teacher network segmentation results and the corrected student network segmentation results. The total loss is obtained by weighted summation of the cross-pseudo-supervision loss and the average teacher supervision loss.
8. The method as described in claim 1, characterized in that, The collaborative average teacher framework performs repeated splicing or length alignment on labeled data during the data loading phase to ensure that the number of aligned labeled data is consistent with the number of unlabeled data.
9. The method as described in claim 1, characterized in that, The collaborative average teacher framework scales the test medical image to 256×256 resolution during the inference phase and inputs it into the trained teacher network to generate the medical image segmentation prediction map.