A semi-supervised image segmentation method based on uncertain pseudo label correction

Abstract

The application provides a semi-supervised image segmentation method based on uncertain pseudo label correction. The method comprises the following steps: training an initialized semantic segmentation model using a labeled data set to obtain a teacher model; generating pseudo labels for an unlabeled data set using the teacher model to form an unlabeled data set with pseudo labels, and mixing the unlabeled data set with an original labeled data set to obtain a mixed labeled training set; training a student model using the mixed labeled training set and the labeled data set through an uncertainty pseudo label correction algorithm; inputting an original image to be classified into the trained student model, and generating a mask segmentation result by the student model; and classifying the original image according to the mask segmentation result. The method of the application introduces labeled images and reliable labeling information to effectively eliminate noise in the pseudo labels, and realizes the generation of unlabeled image training samples with different scale effects, thereby suppressing noise and errors in the pseudo labels from different regional scales.

Description

Technical Field

[0001] This invention relates to the field of image segmentation technology, and in particular to a semi-supervised image segmentation method based on uncertain pseudo-label correction. Background Technology

[0002] Image semantic segmentation is a crucial branch of image understanding within image processing and machine vision technologies, and also an important branch of AI (Artificial Intelligence). Semantic segmentation classifies each pixel in an image, determining its category to achieve region division. The rapid development of deep learning technology over the past decade has also driven advancements in image semantic segmentation. A key factor in image semantic segmentation in the deep learning era is the emergence of large-scale, fine-grained pixel-level labeled datasets. However, labeling a large-scale pixel-level labeled dataset is extremely costly, limiting the multi-scenario generalization ability of image semantic segmentation methods. Semi-supervised semantic segmentation methods aim to alleviate this data dilemma by utilizing large-scale unlabeled data to reduce data dependence.

[0003] Recent semi-supervised semantic segmentation algorithms can be broadly categorized into two types: consistency regularization and entropy minimization. Consistency regularization constrains the network to produce similar predictions for copies of the same unlabeled image under different transformations, thus effectively utilizing unlabeled images. However, consistency regularization requires complex regularization techniques, such as contrastive learning and class balancing strategies. Entropy minimization, on the other hand, is implemented through self-training, using pseudo-labels from unlabeled data for model retraining. However, this retraining method is prone to overfitting to noise in the pseudo-labels, leading to performance degradation.

[0004] Currently, one existing method to mitigate the impact of pseudo-label noise includes erasing the noisy areas. The drawbacks of this method are that noise in the pseudo-labels is difficult to locate directly, and random erasure can also erase valid labeled areas simultaneously, resulting in the loss of effective supervisory information from the pseudo-labels and performance degradation. Summary of the Invention

[0005] The embodiments of the present invention provide a semi-supervised image segmentation method based on uncertain pseudo-label correction, so as to realize a semi-supervised image semantic segmentation algorithm that is robust to noise and has high generalization.

[0006] To achieve the above objectives, the present invention adopts the following technical solution.

[0007] A semi-supervised image segmentation method based on uncertain pseudo-label correction includes:

[0008] The teacher model is obtained by training the initial semantic segmentation model using a labeled dataset;

[0009] The teacher model is used to generate pseudo-labels for the unlabeled dataset, forming an unlabeled dataset with pseudo-labels. The unlabeled dataset with pseudo-labels is then mixed with the original labeled dataset to obtain a hybrid labeled training set.

[0010] The student model is trained using the hybrid labeled training set and the labeled dataset through an uncertainty pseudo-label correction algorithm to obtain a well-trained student model.

[0011] The original image to be classified is input into the trained student model, which generates a mask segmentation result and classifies the original image based on the mask segmentation result.

[0012] Preferably, the step of training the student model using the hybrid labeled training set and the labeled dataset through an uncertainty pseudo-label correction algorithm to obtain a trained student model includes:

[0013] A certain number of images are sampled from the labeled dataset as labeled samples. These labeled samples use manually labeled masks as supervision information. The same number of images are sampled from the unlabeled dataset as unlabeled samples. The unlabeled samples use pseudo-labels generated from the hybrid labeled training set as supervision information. An uncertainty pseudo-label correction algorithm is used to enhance the unlabeled samples and their corresponding pseudo-labels. The uncertainty pseudo-label correction algorithm calculates pixel-level uncertainty scores for the pseudo-labels, merges the pixel-level uncertainty scores of each region to obtain the uncertainty of each region of the pseudo-label, and replaces the pseudo-label region with the accurate mask label of the labeled sample to generate enhanced samples.

[0014] The student model is trained under supervision using the labeled samples and the augmented samples, and the training of the student model is constrained using the cross-entropy loss function to obtain a well-trained student model.

[0015] Preferably, the method of using an uncertainty pseudo-label correction algorithm to enhance unlabeled samples and their corresponding pseudo-labels involves calculating pixel-level uncertainty scores for the pseudo-labels, merging the pixel-level uncertainty scores in each region to obtain the uncertainty of each region of the pseudo-label, and replacing the pseudo-label region with the accurate mask label of the labeled sample to generate enhanced samples, including:

[0016] Each batch of input for training the student model includes labeled samples and unlabeled samples in a 1:1 ratio. The unlabeled samples use a noisy pseudo-label mask generated by the mixed labeled training set as supervision information. For each unlabeled sample and its pseudo-label mask, the uncertainty pseudo-label correction algorithm calculates the uncertainty of each pixel on its pseudo-label mask matrix to obtain an uncertainty matrix. The uncertainty pseudo-label correction algorithm performs the same block processing on the unlabeled image matrix, pseudo-label mask matrix and uncertainty matrix. Each matrix is ​​divided into 16 4x4 sub-regions in the same way. The length and width of each sub-region are 1 / 4 of the original matrix. For each sub-region of the uncertainty matrix, the uncertainty of the sub-region is obtained by combining the uncertainty of all pixels in the sub-region.

[0017] The uncertainty pseudo-label correction algorithm randomly scales the labeled sample image and label mask by a factor of 0.5 to 1.0. It then randomly selects k non-overlapping regions of the same size from the scaled labeled sample and replaces the unlabeled image sample region and pseudo-label mask region corresponding to the k sub-regions with the highest uncertainty using the k non-overlapping regions of the same size.

[0018] For each k value, different labeled samples are selected to generate multiple sets of different augmented unlabeled samples.

[0019] Preferably, the step of inputting the original image to be classified into the trained student model, the student model generating a mask segmentation result, and classifying the original image based on the mask segmentation result includes:

[0020] The trained student model is saved, and the original image to be classified is input into the trained student model. The trained student model generates a mask segmentation result for the input original image, and classifies the original image according to the mask segmentation result. As can be seen from the technical solution provided by the embodiments of the present invention above, the method of the embodiments of the present invention utilizes an uncertainty-guided image content stitching enhancement method to introduce labeled images and their reliable annotation information to effectively remove noise in pseudo-labels; through multiple sets of transformation combinations, the present invention achieves the generation of unlabeled image training samples with different scale effects, suppressing noise and errors in pseudo-labels at different regional scales.

[0021] Additional aspects and advantages of the invention will be set forth in part in the description which follows, and will become apparent from the description or may be learned by practice of the invention. Attached Figure Description

[0022] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0023] Figure 1 This is a flowchart of the semi-supervised image segmentation method based on uncertain pseudo-label correction according to an embodiment of the present invention.

[0024] Figure 2 This is a flowchart illustrating the processing of the Uncertainty Pseudo-Label Correction UAC algorithm as described in an embodiment of the present invention.

[0025] Figure 3 The image shows the effect of the semi-supervised segmentation method based on the uncertainty pseudo-label correction method described in the embodiment of the present invention.

[0026] Figure 4 This is a comparison of the performance of the semi-supervised image segmentation method (represented as UAC in the figure) based on uncertain pseudo-label correction described in the embodiments of the present invention with the fully supervised segmentation method. Detailed Implementation

[0027] Embodiments of the present invention are described in detail below, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and should not be construed as limiting the present invention.

[0028] Those skilled in the art will understand that, unless specifically stated otherwise, the singular forms “a,” “an,” “the,” and “the” used herein may also include the plural forms. It should be further understood that the term “comprising” as used in this specification means the presence of the stated features, integers, steps, operations, elements, and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof. It should be understood that when we say an element is “connected” or “coupled” to another element, it can be directly connected or coupled to the other element, or there may be intermediate elements. Furthermore, “connected” or “coupled” as used herein can include wireless connections or couplings. The term “and / or” as used herein includes any and all combinations of one or more of the associated listed items.

[0029] It will be understood by those skilled in the art that, unless otherwise defined, all terms used herein (including technical and scientific terms) have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. It should also be understood that terms such as those defined in general dictionaries should be understood to have the same meaning as in the context of the prior art, and should not be interpreted in an idealized or overly formal sense unless defined as herein.

[0030] To facilitate understanding of the embodiments of the present invention, the following will provide further explanation and description with reference to the accompanying drawings and several specific embodiments. These embodiments do not constitute a limitation on the embodiments of the present invention.

[0031] This invention provides a semi-supervised image segmentation method based on Noise-robust Semi-supervised Segmentation via Uncertainty-aware CutMix (UAC). The processing flow of this method is as follows: Figure 1 As shown, the processing steps include the following:

[0032] Step S1: Train the initialized semantic segmentation model using the labeled dataset to obtain the teacher model.

[0033] This invention does not limit the specific structure of the semantic segmentation model. In the specific method performance verification section, we selected two common semantic segmentation networks: DeepLab v3+ and Mask2Former. Both of these semantic segmentation models are encoder-decoder structures. The encoder part of both uses a ResNet-101 network, while the decoder parts differ. The DeepLabv3+ encoder uses a decoder design based on dilated convolution, while the Mask2Former encoder uses a Transformer-based decoder structure.

[0034] Step S2: Use the teacher model from step S1 to generate pseudo-labels for the unlabeled dataset, forming an unlabeled dataset with pseudo-labels. Mix the unlabeled dataset with pseudo-labels with the original labeled dataset to obtain a hybrid labeled training set.

[0035] The aforementioned hybrid labeled training set provides pseudo-labels generated by the teacher model's predictions for the original unlabeled data, which are then used as supervisory information. These pseudo-labels possess high information content but also exhibit noticeable noise in some regions. In subsequent semi-supervised training, the unlabeled data with pseudo-labels in the hybrid labeled training set will be utilized as supervised training samples to train the student model.

[0036] Step S3: Train a student model using a hybrid labeled training set.

[0037] In each step of training, a certain number of image training samples are sampled from the labeled dataset to obtain labeled samples, which use manually labeled masks as supervision information. The same number of unlabeled samples are sampled from the unlabeled dataset, which use pseudo-labels generated from the aforementioned hybrid labeled training set as supervision information. The Uncertainty Pseudo-Label Correction Algorithm (UAC) proposed in this invention is used to enhance the unlabeled image samples and their corresponding pseudo-labels. The UAC algorithm calculates pixel-level uncertainty scores for the pseudo-label masks and merges the pixel-level uncertainty scores in each region to obtain the uncertainty of each region of the pseudo-label mask. For the pseudo-label mask region with the highest uncertainty, the accurate mask label of the labeled sample is introduced for replacement, generating enhanced samples with less noise. The above transformation process also occurs on the unlabeled sample images, achieving replacement on both the image and the pseudo-label mask.

[0038] The enhanced unlabeled samples and their pseudo-labeled masks obtained by the above method have less noise information and higher reliability. They can be used together with labeled samples for fully supervised semantic segmentation model training. When training the student model, the labeled samples in the original batch input and these enhanced unlabeled samples are used for supervised training. The cross-entropy loss function is used for constraints to achieve the training of the student model, resulting in a well-trained student model.

[0039] Step S4: Save the trained student model. This student model is the final output model of the semi-supervised semantic segmentation algorithm of this invention. Similar to the fully supervised semantic segmentation model, this trained student model generates mask segmentation results for the input original image, classifies the original image based on the mask segmentation results, and can be deployed in various application scenarios. In step S1, when training the teacher model using the labeled dataset, the labeled data samples do not need to use the aforementioned uncertainty-guided image stitching enhancement algorithm UAC;

[0040] In step S2, when generating pseudo-labels for unlabeled data using the teacher model, to improve the accuracy of the pseudo-labels and mitigate the overfitting effect of the segmentation algorithm, each unlabeled image is scaled and flipped at different sizes to form multiple image samples with different pixel counts and flipped content. The teacher model is then used to predict the category of each of the multiple image samples, and the category prediction results of the multiple image samples are averaged to generate the pseudo-labels corresponding to the aforementioned unlabeled images.

[0041] In step S3, the processing flowchart of the UAC algorithm based on uncertainty pseudo-label correction provided by this embodiment of the invention is as follows: Figure 2As shown. In this process, each batch of input for training the student model includes labeled and unlabeled samples in a 1:1 ratio. The unlabeled samples use noisy pseudo-labeled masks from S2 as supervision information. For each unlabeled sample and its pseudo-labeled mask, the UAC algorithm calculates the uncertainty per pixel on its pseudo-labeled mask matrix, obtaining an uncertainty matrix. The UAC algorithm performs the same block processing on the unlabeled image matrix, pseudo-labeled mask matrix, and uncertainty matrix. Each matrix is ​​divided into 16 4x4 sub-regions in the same way, with each sub-region having a length and width that are 1 / 4 of the original matrix. There is a one-to-one correspondence between the unlabeled image matrix, pseudo-labeled mask matrix, and uncertainty matrix. For each sub-region of the uncertainty matrix, the uncertainty of that sub-region is obtained by combining the uncertainties of all pixels within that sub-region. The unlabeled image sample regions and pseudo-labeled mask regions corresponding to the k sub-regions with the highest uncertainties are removed by replacement.

[0042] The User-Augmented Process (UAC) algorithm randomly selects a labeled sample from the input labeled samples. This labeled sample uses a manually labeled mask as supervision information. The image and mask of the labeled sample are randomly scaled by a factor of 0.5 to 1.0. From this scaled labeled sample, k non-overlapping regions of the same size are randomly selected and used to replace k regions in the image and pseudo-labeled mask of the unlabeled sample. Furthermore, this invention uses multiple sets of UAC enhancement algorithms with different k values ​​for the unlabeled image samples. For each k value, different labeled samples are selected to generate multiple sets of enhanced unlabeled samples. This multi-set enhancement strategy is quite effective in improving the performance of the student model. In this invention, one set of k values ​​is 2, 3, and 5.

[0043] The effects of the above-mentioned enhanced change process are as follows: Figure 3 As shown, error processing is the calculation of the loss function during the training of the image semantic segmentation network. Specifically, it involves processing the error between the final predicted class probability distribution map and the mask annotations of the input samples, minimizing this error value through backpropagation to achieve optimal training. This error calculation is generally implemented using the cross-entropy loss function.

[0044] The general expression for the cross-entropy loss function is:

[0045]

[0046] y ic For the label of sample i, class c, p ic The probability of sample i belonging to class c.

[0047] In this invention, the aforementioned cross-entropy function is also used to calculate the error between the student model's prediction and the pseudo-label of the corrected augmented sample. The student model is then trained through backpropagation to optimize this error value, thereby achieving the training of the student model.

[0048] The aforementioned UAC semi-supervised image segmentation algorithm is a plug-and-play image data augmentation method. It is not limited by specific network structures; it only requires augmentation of the input data to achieve excellent semi-supervised image segmentation results. Furthermore, this algorithm eliminates the need for redundant repetitive training and complex structural designs, achieving a simple and efficient design principle.

[0049] Experimental results

[0050] (1) Training and testing process

[0051] Experiments were conducted on the PyTorch framework. The base segmentation network used the commonly used deeplab v3+ and mask2former. The experimental datasets were chosen based on the homodistribution and heterodistribution of unlabeled datasets. Under the homodistributed unlabeled dataset setting, experiments were conducted on the visual challenge dataset PascalVOC and the street scene recognition dataset Cityscapes. The PascalVOC dataset initially consisted of 1464 images for training and 1449 images for validation. Similar to previous semi-supervised image segmentation algorithms, this invention used the SBD edge dataset as the augmentation set, containing 9118 training images, resulting in a total of 10582 labeled training images. This invention samples labeled data only from the original labeled training set and also from the mixed training set containing 10582 images; the remaining images in the 10582 training set are used as unlabeled data. The Cityscapes dataset contains 2975 training set images and 500 validation set images. For the two datasets with the same distribution, this invention extracts 1 / 2, 1 / 4, 1 / 8, and 1 / 16 of the training set from the aforementioned datasets as labeled samples, respectively, to conduct semi-supervised image semantic segmentation experiments with the same distribution. The experimental comparison algorithm is evaluated and compared on the PascalVOC and Cityscapes test datasets.

[0052] In this heterogeneous, unlabeled dataset setting, the experiment uses the complete PascalVOC training set as the labeled training set and the MSCOCO large-scale image dataset with different distributions as the large-scale unlabeled dataset. Under this heterogeneous semi-supervised semantic segmentation experimental setting, 10,582 labeled images and 118,000 unlabeled images were used. The experiment was then evaluated and compared on the PascalVOC test dataset.

[0053] In detail, regarding the network structure used, this invention employs two semantic segmentation network structures: deeplabv3+ and mask2former, both using the ResNet-101 network as the basic backbone network, achieving fair comparison with other methods. The experiments used the Mean Intersection of the Union (mIoU) as the evaluation metric to measure the degree of overlap between the model's output mask and the ground truth labeled mask, i.e., accuracy.

[0054] (2) Comparison of experimental results

[0055] First, we compare the UAC method with recent state-of-the-art semi-supervised image segmentation algorithms, including Mean-Teacher, CCT, GCT, PseudoSeg, CPS, and PC. 2 Seg,AEL,U 2 PL, ST++ (listed in order of publication date from earliest to latest). PseudoSeg and ST++ employ entropy minimization, while the remaining methods use consistency regularization. In this section, we use ResNet-101 as the basic backbone network and DeepLab v3+ as the basic segmentation network structure to achieve a fair comparison with the methods mentioned above. We also provide comparison results using Mask2former (M2F) as the basic segmentation network.

[0056] First, we conducted a performance comparison experiment of the semi-supervised algorithm on the PascalVOC dataset. In Table 1, we present an objective performance comparison of the mIoU values, with the best results highlighted in bold. The 1 / 16 (92) in the first row of Table 1 indicates that 1 / 16 of the labeled data was extracted from the labeled data source, with a quantity of 92. The other columns are similar, and Tables 2, 3, and 4 also use this representation method.

[0057] Table 1 Comparison of semi-supervised image semantic segmentation performance using the Pascal VOC original annotation dataset as the annotation data source.

[0058]

[0059] As shown in Table 1, UAC demonstrates a significant leading advantage in various labeled data sampling scenarios of PascalVOC. This advantage was not realized only in the 1 / 16 semi-supervised experimental setting, where it lagged slightly behind UAC. 2PL. Meanwhile, it can be observed that the UAC method has a significant leading advantage under both different network structures, verifying the effectiveness of the method of this invention. It is worth noting that, with its simple and effective method design, the UAC method not only surpasses the complex and time-consuming self-training method ST++, but also achieves consistency regularization for complex structure designs using CPS,U 2 PL, PC 2 The breakthrough of Seg demonstrates that image denoising achieved through uncertainty-guided image enhancement can effectively achieve superior semi-supervised image segmentation performance.

[0060] Table 2 compares the performance of semi-supervised image semantic segmentation using the Pascal VOC hybrid annotation dataset as the annotation data source.

[0061]

[0062] As shown in Table 2, when using a hybrid labeled dataset as the annotation data source, the UAC method also outperformed several comparable methods in various semi-supervised experimental settings. This advantage is consistent with that shown in Table 1. Notably, in this set of semi-supervised experiments, the UAC method, using Mask2former (M2F) as the base segmentation network, outperformed the recent state-of-the-art method UAC in all settings. 2 PL and ST++.

[0063] UAC also competes with U on the cityscapes street view dataset. 2 The PL and ST++ methods were compared. In this set of comparisons, UAC, ST++, and U... 2 All three PLs used ResNet-101 as the basic backbone network and Mask2Former (M2F) as the basic segmentation network. Their performance was compared in three semi-supervised experimental settings: 1 / 8, 1 / 4, and 1 / 2.

[0064] Table 3 Comparison of semantic segmentation performance of semi-supervised images on the Cityscapes dataset with identically distributed data.

[0065]

[0066] As shown in Table 4, on the Cityscapes dataset, the UAC method outperformed the comparison method U in all three semi-supervised experimental settings: 1 / 8, 1 / 4, and 1 / 2. 2 PL,ST++. Meanwhile, the lead of UAC increases with the amount of labeled data; under a 1 / 2 semi-supervised data setting, UAC leads U... 2 The mIoU performance of PL2.18.

[0067] Tables 1, 2, and 3 above demonstrate the performance of semi-supervised image semantic segmentation on identically distributed data. UAC exhibits excellent performance on identically distributed unlabeled data. Table 4 provides a performance comparison for more complex semi-supervised image segmentation tasks on heterogeneously distributed unlabeled data.

[0068] Table 4 Comparison of semantic segmentation performance of Pascal VOC and MSCOCO heterogeneous unlabeled data for semi-supervised image segmentation

[0069]

[0070] As shown in Table 4, labeled training data comes from the Pascal VOC training set, while unlabeled training data comes from the heterogeneous MSCOCO dataset. The size of the unlabeled dataset is approximately 10 times that of the labeled dataset. All methods were tested on the Pascal VOC validation set. UAC was compared using the DeepLab v3+ network and the PseduoSeg method, and the Mask2Former network and UAC were compared. 2 Comparing PL and ST++, UAC significantly outperforms the comparison method in both networks, verifying the superior noise resistance and strong generalization performance of the method of this invention under heterogeneous unlabeled data.

[0071] like Figure 4 As shown, we present a comparison of the prediction performance of the UAC method proposed in this invention with that of the fully supervised algorithm and the real labeled mask. Figure 4 The columns from left to right in the image are, in order: the input image, the ground truth mask, the performance of the fully supervised algorithm, the prediction results of the UAC method for identically distributed data, and the prediction results of the UAC method for heterogeneous data. Comparing the predicted mask of the UAC method with the prediction results of the fully supervised algorithm and the ground truth mask reveals that the UAC method performs better in terms of class misclassification and contour smoothness, showing a significant improvement over the fully supervised algorithm.

[0072] In summary, the method of this invention utilizes an uncertainty-guided image content stitching enhancement method to introduce labeled images and their reliable annotation information to effectively remove noise in pseudo-labels. Through multiple transformation combinations, this invention achieves the generation of unlabeled image training samples with different scale effects, suppressing noise and errors in pseudo-labels at different regional scales.

[0073] By accurately and effectively replacing noise in pseudo-labels, this invention ensures the validity of pseudo-label information, reduces overfitting caused by pseudo-label noise, and achieves strong noise resistance. This invention outperforms classic semi-supervised image segmentation algorithms on various semi-supervised image segmentation performance metrics for both similarly and disseminated unlabeled data. Furthermore, this invention is a general-purpose semi-supervised image segmentation technique that requires no modification to the image segmentation network, operating only on the data, and is theoretically applicable to all current segmentation models.

[0074] This invention addresses pseudo-label noise reduction by implementing a heuristic denoising mechanism that effectively mitigates the impact of pseudo-label noise in semi-supervised image segmentation. The algorithm is concise and efficient, requiring no complex hyperparameter tuning or repetitive training processes to achieve high-efficiency semi-supervised image segmentation. It achieves superior performance in both identically and heterogeneously distributed unlabeled data scenarios, significantly outperforming similar methods. Thanks to its simple and efficient design, this invention offers plug-and-play functionality, making it suitable for various image segmentation networks.

[0075] Those skilled in the art will understand that the accompanying drawings are merely schematic diagrams of one embodiment, and the modules or processes shown in the drawings are not necessarily essential for implementing the present invention.

[0076] As can be seen from the above description of the embodiments, those skilled in the art can clearly understand that the present invention can be implemented by means of software plus necessary general-purpose hardware platforms. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in various embodiments or some parts of the embodiments of the present invention.

[0077] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, for apparatus or system embodiments, since they are basically similar to method embodiments, the description is relatively simple; relevant parts can be referred to the descriptions in the method embodiments. The apparatus and system embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without creative effort.

[0078] The above description is merely a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A semi-supervised image segmentation method based on uncertain pseudo-label correction, characterized in that, include: The teacher model is obtained by training the initial semantic segmentation model using a labeled dataset; The teacher model is used to generate pseudo-labels for the unlabeled dataset, forming an unlabeled dataset with pseudo-labels. The unlabeled dataset with pseudo-labels is then mixed with the original labeled dataset to obtain a hybrid labeled training set. The student model is trained using the hybrid labeled training set and the labeled dataset through an uncertainty pseudo-label correction algorithm to obtain a well-trained student model. The original image to be classified is input into the trained student model, which generates a mask segmentation result, and the original image is classified according to the mask segmentation result. The aforementioned algorithm for enhancing unlabeled samples and their corresponding pseudo-labels involves using an uncertainty pseudo-label correction algorithm. This algorithm calculates pixel-level uncertainty scores for each pseudo-label, merges these scores across regions to obtain the uncertainty of each region, and replaces the pseudo-label region with the accurate mask label of the labeled sample to generate enhanced samples. This process includes: Each batch of input for training the student model includes labeled samples and unlabeled samples in a 1:1 ratio. The unlabeled samples use a noisy pseudo-label mask generated by the mixed labeled training set as supervision information. For each unlabeled sample and its pseudo-label mask, the uncertainty pseudo-label correction algorithm calculates the uncertainty of each pixel on its pseudo-label mask matrix to obtain an uncertainty matrix. The uncertainty pseudo-label correction algorithm performs the same block processing on the unlabeled image matrix, pseudo-label mask matrix and uncertainty matrix. Each matrix is ​​divided into 16 4x4 sub-regions in the same way. The length and width of each sub-region are 1 / 4 of the original matrix. For each sub-region of the uncertainty matrix, the uncertainty of the sub-region is obtained by combining the uncertainty of all pixels in the sub-region. The uncertainty pseudo-label correction algorithm randomly scales the labeled sample image and label mask by a factor of 0.5 to 1.

0. It then randomly selects k non-overlapping regions of the same size from the scaled labeled sample and replaces the unlabeled image sample region and pseudo-label mask region corresponding to the k sub-regions with the highest uncertainty using the k non-overlapping regions of the same size. For each k value, different labeled samples are selected to generate multiple sets of different augmented unlabeled samples.

2. The method according to claim 1, characterized in that, The process of training a student model using the hybrid labeled training set and the labeled dataset through an uncertainty pseudo-label correction algorithm to obtain a trained student model includes: A certain number of images are sampled from the labeled dataset as labeled samples. These labeled samples use manually labeled masks as supervision information. The same number of images are sampled from the unlabeled dataset as unlabeled samples. The unlabeled samples use pseudo-labels generated from the hybrid labeled training set as supervision information. An uncertainty pseudo-label correction algorithm is used to enhance the unlabeled samples and their corresponding pseudo-labels. The uncertainty pseudo-label correction algorithm calculates pixel-level uncertainty scores for the pseudo-labels, merges the pixel-level uncertainty scores of each region to obtain the uncertainty of each region of the pseudo-label, and replaces the pseudo-label region with the accurate mask label of the labeled sample to generate enhanced samples. The student model is trained under supervision using the labeled samples and the augmented samples, and the training of the student model is constrained using the cross-entropy loss function to obtain a well-trained student model.

3. The method according to claim 1 or 2, characterized in that, The process of inputting the original image to be classified into the trained student model, the student model generating a mask segmentation result, and classifying the original image based on the mask segmentation result includes: Save the trained student model, input the original image to be classified into the trained student model, the trained student model generates a mask segmentation result for the input original image, and classifies the original image according to the mask segmentation result.

Images