A semi-supervised medical image segmentation method based on progressive pseudo-label enhancement

Abstract

The application provides a kind of semi-supervised medical image segmentation method based on progressive pseudo label enhancement.The method comprises: dividing the original medical image data set after labeling into a labeled data set and an unlabeled data set, constructing a PPE network model comprising a student model and a teacher model, iteratively training the PPE network model using the labeled data set and the unlabeled data set, in each iteration, using the SAT mechanism to generate an adaptive threshold, filtering out the pseudo labels in the student model and the teacher model below a certain confidence range, obtaining the trained student model and the teacher model; input the preprocessed medical image into the trained student model, and output the image category segmentation result of the medical image.The application uses the SAT mechanism to generate an adaptive threshold to improve the utilization rate of reliable pixels, then combines the collaboration and consistency regularization method of the teacher model to effectively utilize unreliable pixels, and gradually realizes the comprehensive utilization of unlabeled data.

Description

Technical Field

[0001] This invention relates to the field of image segmentation technology, and in particular to a semi-supervised medical image segmentation method based on progressive pseudo-label enhancement. Background Technology

[0002] Medical image segmentation is a fundamental task in medical imaging, aiming to accurately delineate regions of interest, such as lesions, tissues, and organs, for quantitative analysis, thereby providing more precise support for clinicians in diagnostic and treatment decisions. Currently, many advanced medical image segmentation methods rely heavily on training with large amounts of high-quality, pixel-level labeled data. However, labeling medical images is both tedious and requires a high level of expertise, making high-quality, pixel-level labeled data quite scarce. In stark contrast, acquiring unlabeled medical images is relatively simple. Therefore, developing semi-supervised segmentation methods that effectively utilize both limited labeled data and large amounts of unlabeled data is particularly important.

[0003] Pseudo-labeling is a typical semi-supervised method widely used in various medical image segmentation tasks. Currently, one existing semi-supervised medical image segmentation method employs a thresholding strategy to generate pseudo-labels from weakly enhanced unlabeled images, retaining only pseudo-labels with a confidence level higher than a fixed threshold (typically 0.95). Subsequently, this method uses high-confidence pseudo-labels to guide the prediction of strongly enhanced versions of the same image to maintain consistency.

[0004] One drawback of the aforementioned semi-supervised medical image segmentation method is its reliance on a fixed threshold. Threshold-based pseudo-labeling methods directly filter out unreliable pixels, often located in difficult-to-segment boundary regions, leading to insufficient model learning in these areas. Furthermore, many pseudo-labels deemed unreliable are actually correct predictions, mistakenly identified as unreliable simply because a high threshold is used. This erroneous judgment limits the model's effective use of data information, thus impacting segmentation performance. Summary of the Invention

[0005] This invention provides a semi-supervised medical image segmentation method based on progressive pseudo-label enhancement, which can make full use of unlabeled data.

[0006] To achieve the above objectives, the present invention adopts the following technical solution.

[0007] A semi-supervised medical image segmentation method based on progressive pseudo-label enhancement includes:

[0008] The original medical image dataset is labeled, and the labeled original medical image dataset is divided into labeled dataset and unlabeled dataset. Weak augmentation and strong augmentation are used to augment the unlabeled data.

[0009] A progressive pseudo-label enhancement framework (PPE) network model is constructed, which includes a student model and a teacher model. The PPE network model is iteratively trained using the labeled and unlabeled datasets. In each iteration, an adaptive threshold is generated using the adaptive anchor threshold (SAT) mechanism. The adaptive threshold mechanism is used to filter out pseudo-labels in the student and teacher models that are below a certain confidence range, thus obtaining the trained student and teacher models.

[0010] The preprocessed medical image is input into a trained student model, which then outputs the image category segmentation result of the medical image.

[0011] Preferably, the annotation of the original medical image dataset, which divides the annotated original medical image dataset into labeled and unlabeled datasets, includes:

[0012] Raw medical image data was collected, and min-max normalization was applied to convert the 3D medical images into 2D slice images. All images were cropped to a uniform size of 256×256, and a subset of images was randomly selected for pixel-level annotation to generate a labeled dataset D. L The unlabeled images constitute the unlabeled dataset D. U In semi-supervised learning, D U The amount of data in is greater than D L The amount of data in it.

[0013] Preferably, the data augmentation of unlabeled data using weak and strong augmentation includes:

[0014] Using weak and strong augmentation on unlabeled data D U Data augmentation can be performed in several ways. Weak augmentation methods include flipping and rotating the image with a 50% probability, while strong augmentation methods include performing Dropout operations on the model's features and performing CutMix operations on the image, which involves cutting out a region of one image and replacing it with the corresponding region of another image.

[0015] Preferably, the construction of the Progressive Pseudo-Label Enhancement (PPE) network model, which includes a student model and a teacher model, comprises:

[0016] Construct a PPE network model that adopts a teacher-student structure, including a student model. Teacher Model Two models, the student model and the teacher model, have the same structure but different initializations. Both models generate pseudo-labels using weakly enhanced images.

[0017] Using the labeled dataset D L The student model is trained, and its parameters are updated via backpropagation, using weakly augmented unlabeled data D. U The student model is trained, and the student model and the teacher model output a model prediction probability map, a confidence map, and pseudo-labels. After each iteration of training, the teacher model updates the parameters from the student model using the exponential smoothed moving average (EMA) algorithm with a smoothing factor β.

[0018] Preferably, the iterative training of the PPE network model using the labeled and unlabeled datasets involves generating an adaptive threshold using an adaptive anchor threshold (SAT) mechanism in each iteration. This adaptive threshold mechanism filters out pseudo-labels in the student and teacher models that fall below a certain confidence range. The parameters of the student and teacher models are then iteratively updated to obtain the trained student and teacher models.

[0019] In each iteration of training the PPE network model, an adaptive threshold is generated using the SAT mechanism. The confidence level of anchor pixels is calculated based on the model prediction probability map and confidence map output by the teacher model. The learning state of the PPE network model is estimated using the confidence level of the anchor pixels, and the learning state w of the PPE network model is estimated based on the average confidence level of all anchor pixels in a batch. t , is represented as:

[0020]

[0021] Where t represents the number of iterations, and B represents the batch size in each iteration, i.e., the number of samples used in each iteration;

[0022] An adaptive threshold is generated for each training iteration by using the estimated model learning state as a weighting function. Represented as:

[0023]

[0024] in It is a predefined maximum threshold;

[0025] Define the lower limit of the adaptive anchor threshold. Where C represents the number of categories.

[0026] The threshold locking strategy is expressed as follows:

[0027] w t―u t >ξ#(5)

[0028] Among them, u t This indicates that all w in the first t iterations t The average value of ξ is a local minimum. If this condition is met, the threshold value increases with w. t Change, or otherwise locked in

[0029] Update u using the EMA method with a smoothing factor of α. t , is represented as:

[0030] u t =αu t―1 +(1―α)u t #(6)

[0031] Where α = 0.9;

[0032] The SAT mechanism is represented as follows:

[0033]

[0034] The student model only retains students whose scores are greater than their adaptive threshold. The teacher model only retains pseudo-labels that are greater than its adaptive threshold. The pseudo-labels are used, and a filtering mask is used to retain some pixels in the prediction of the student model. The teacher model also retains the predicted values ​​of the corresponding pixels. The parameters of the student model and the teacher model are iteratively updated to obtain the trained student model and teacher model.

[0035] Preferably, the iterative updating of the parameters of the student model and the teacher model to obtain the trained student model and teacher model includes:

[0036] The student model receives data from the labeled dataset and generates a prediction P; the student model receives weakly augmented images from the unlabeled dataset and generates a prediction P. Pseudo-labels The student model receives strongly augmented images from an unlabeled dataset and generates predictions. The teacher model receives weakly augmented images from an unlabeled dataset and generates predictions. Pseudo-labels

[0037] For labeled data, perform supervised training with supervised loss. Represented as:

[0038]

[0039] Where B represents the batch size in each iteration, i.e., the number of samples used in each iteration, P represents the model prediction map, and Y represents the pixel-level label. Represents cross-entropy loss, Indicates Dice loss;

[0040] Pseudo-labels generated using student models To guide the prediction of student models False label loss Represented as:

[0041]

[0042] in A confidence plot representing the model's predictions. This represents a filter mask; only masks that meet certain conditions are allowed. Loss calculation will only be performed on pixels that meet the minimum requirements.

[0043] Generating pseudo-labels from another perspective using a collaborative teacher model The collaborative average teacher model shares the same structure as the student model. The collaborative average teacher model updates parameters from the student model using the EMA (Equal Access Modifier) ​​mechanism, as follows:

[0044]

[0045] Where β = 0.99, The parameters represent the collaborative average teacher model. This represents the parameters of the student model.

[0046] Collaborative pseudo-label loss between pseudo-labels generated by the collaborative average teacher model and predictions from the student model. Represented as:

[0047]

[0048] Consistency regularization loss Represented as:

[0049]

[0050] in This represents the mean square error loss.

[0051] The total loss of the PPE network model is expressed as follows:

[0052]

[0053] Where λ1=0.5, λ2=0.4, λ3=0.1.

[0054] Preferably, the step of inputting the preprocessed medical image into a trained student model, wherein the trained student model outputs the image category segmentation result of the medical image, includes:

[0055] Minimum-maximum normalization is performed on the medical images, and the image size is uniformly adjusted to 256×256. The three-dimensional medical images are sliced ​​into two-dimensional images to obtain the preprocessed medical images.

[0056] Using PyTorch, a pre-trained student model is loaded. The pre-processed medical image is input into the pre-trained student model. Based on the probability map generated by the student model, each pixel of the medical image is assigned a corresponding category, a segmentation mask is generated, contours are extracted according to the segmentation mask, and the extracted contours are superimposed on the original medical image to generate a visualized image category segmentation result.

[0057] As can be seen from the technical solutions provided by the embodiments of the present invention above, the PPE framework proposed in the embodiments of the present invention first uses the SAT mechanism to generate an adaptive threshold to improve the utilization rate of reliable pixels, and then combines the collaboration and consistency regularization methods of the teacher model to effectively utilize unreliable pixels, thus progressively realizing the full utilization of unlabeled data.

[0058] Additional aspects and advantages of the invention will be set forth in part in the description which follows, and will become apparent from the description or may be learned by practice of the invention. Attached Figure Description

[0059] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the following description of the embodiments will be briefly introduced. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0060] Figure 1 A flowchart illustrating a semi-supervised medical image segmentation method based on progressive pseudo-label enhancement, provided in an embodiment of the present invention;

[0061] Figure 2 This is a framework diagram for training on unlabeled data according to the present invention;

[0062] Figure 3 This is a computational diagram of the model state estimation in this invention;

[0063] Figure 4 This is a comparison of the effects of the present invention with fully supervised learning;

[0064] Figure 5 This is a comparison diagram of the segmentation effect of the present invention. Detailed Implementation

[0065] Embodiments of the present invention are described in detail below, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and should not be construed as limiting the present invention.

[0066] Those skilled in the art will understand that, unless specifically stated otherwise, the singular forms “a,” “an,” “the,” and “the” used herein may also include the plural forms. It should be further understood that the term “comprising” as used in this specification means the presence of the stated features, integers, steps, operations, elements, and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof. It should be understood that when we say an element is “connected” or “coupled” to another element, it can be directly connected or coupled to the other element, or there may be intermediate elements. Furthermore, “connected” or “coupled” as used herein can include wireless connections or couplings. The term “and / or” as used herein includes any and all combinations of one or more of the associated listed items.

[0067] It will be understood by those skilled in the art that, unless otherwise defined, all terms used herein (including technical and scientific terms) have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. It should also be understood that terms such as those defined in general dictionaries should be understood to have the same meaning as in the context of the prior art, and should not be interpreted in an idealized or overly formal sense unless defined as herein.

[0068] To facilitate understanding of the embodiments of the present invention, the following will provide further explanation and description with reference to the accompanying drawings and several specific embodiments. These embodiments do not constitute a limitation on the embodiments of the present invention.

[0069] This invention classifies pixels (pseudo-labels) with confidence levels above a threshold as reliable and pixels (pseudo-labels) with confidence levels below a threshold as unreliable.

[0070] The processing flow of a semi-supervised medical image segmentation (PPE) method based on progressive pseudo-label enhancement provided in this embodiment of the invention is as follows: Figure 1 As shown, it includes the following steps:

[0071] Step 1: Data Preparation

[0072] Collect image data and label the image data, then divide the training set into labeled datasets D. L and unlabeled dataset D U .

[0073] This invention first preprocesses and partitions the acquired raw medical images. Specifically, it first performs min-max normalization on the raw medical images. For three-dimensional medical images, they need to be converted into two-dimensional slice images, and then all images are cropped to a uniform size of 256×256. Finally, a portion of the images is randomly selected for pixel-level annotation to generate a labeled dataset D. L The unlabeled images constitute the unlabeled dataset D. U In semi-supervised learning, there is usually D. U >>D L .

[0074] Step 2: Data Augmentation

[0075] This invention uses weak and strong enhancement to enhance unlabeled data D. U Perform data augmentation.

[0076] The specific method of weak enhancement is to flip and rotate the image with a 50% probability.

[0077] The specific methods of strong enhancement are: performing Dropout operation on the features of the model; and performing CutMix operation on the image, that is, cutting out a region of one image and replacing it with the corresponding region of another image.

[0078] Step 3: Construct the PPE (Progressive Pseudo-labeling Enhancement) network model.

[0079] Figure 2 This is a framework diagram for training on unlabeled data in this invention, as shown below. Figure 2 As shown, the PPE model adopts a teacher-student structure, including a student model. Teacher Model Two models. The student model and the teacher model have the same structure but different initializations. Student model parameters are updated via backpropagation.

[0080] Teacher model parameter updates: The teacher model inherits parameters from the student model via EMA after each iteration. For the labeled dataset D... L Only the student model is trained. The teacher model uses only the unlabeled dataset D. U To assist student model training, the teacher model takes only weakly enhanced images as input and outputs a model prediction probability map, a confidence map, and pseudo-labels.

[0081] Both the student and teacher models use the U-Net network architecture, and both generate pseudo-labels through weakly enhanced images. Using the aforementioned labeled dataset D... L and unlabeled dataset D U The PPE network model is trained iteratively. During the training phase, the student model updates its parameters through backpropagation, while the teacher model updates its parameters from the student model using an EMA (Exponential Moving Average) algorithm with a smoothing factor β. During the inference phase, only the student model is used.

[0082] Step 4: Generation of adaptive threshold.

[0083] A key feature of this invention is the use of the SAT (Adaptive Feature Pooling) mechanism to generate an adaptive threshold in each iteration of model training. The process includes: calculating the confidence score of anchor pixels based on the prediction results of the weakly enhanced image; estimating the model's learning state using the confidence scores of the anchor pixels; and generating the adaptive threshold using the learning state as a weighting function. Finally, the adaptive anchor threshold is corrected using threshold protection and threshold locking strategies.

[0084] The SAT mechanism dynamically adjusts the threshold based on the model's learning state. In threshold-based pseudo-labeling methods, a relatively high fixed threshold is typically used to filter pseudo-labels, but this leads to some correctly predicted pseudo-labels being misclassified as unreliable. In contrast, the adaptive thresholding mechanism designed in this invention aims to retain more correct pseudo-labels, rather than just high-confidence pseudo-labels. Since the model's confidence is typically low and its accuracy insufficient in the early stages of training, a lower threshold is needed to retain pseudo-labels. As training progresses, the model's confidence level increases and it can provide more accurate judgments; in this case, a higher threshold is appropriate for retaining high-confidence pseudo-labels.

[0085] Based on the above analysis, the SAT mechanism of this invention includes the following steps:

[0086] Step 4.1: Calculate the confidence score for anchor pixels. For each image in a batch, the model generates a probability map representing the likelihood of each pixel belonging to a specific category. Confidence Map The confidence map is generated by selecting the highest probability value across all categories for each pixel. To effectively reflect the model's learning state, this invention introduces the concept of anchor pixels. For each pixel in the confidence map, the anchor pixel is defined as the pixel falling into the first percentile with the lowest confidence value. This is achieved by sorting all pixels by confidence. The confidence value of the anchor pixel serves as an indicator of the model's learning state, representing the model's predictive ability. The sorted confidence sequence is shown below. Represented as:

[0087]

[0088] Where H and W represent the height and width of the input image. According to the confidence plot Q w The resulting sequence, q i,j Represents the confidence plot Q w The confidence value at position (i,j).

[0089] This invention uses the confidence value q of the anchor pixel anchor q is defined as the confidence value at the first percentile in the sorted confidence sequence S. anchor The calculation method is as follows:

[0090] q anchor =S[0.01×N]#(2)

[0091] Where N = H × W, and [0.01 × N] represents the position index of the first percentile in sequence S.

[0092] Step 4.2: Estimate the model's learning state. Estimate the model's learning state w based on the average confidence score of all anchor pixels in a batch. t , is represented as:

[0093]

[0094] Where t represents the number of iterations, and B represents the batch size in each iteration, i.e., the number of samples used in each iteration.

[0095] Step 4.3: Generate an adaptive threshold.

[0096] An adaptive threshold is generated for each training iteration by using the estimated model learning state as a weighting function. Represented as:

[0097]

[0098] in It is a predefined maximum threshold. In this invention,

[0099] Step 4.4: Threshold Protection Strategy. In the early stages of training, due to the random initialization of model parameters, the confidence level of anchor pixels may be too low, resulting in a large number of incorrect pseudo-labels being retained. To address this issue, a lower threshold is defined. Where C represents the number of categories. This strategy prevents the use of unreliable pseudo-labels due to excessively low thresholds and balances the selection probabilities of different categories.

[0100] Step 4.5: Threshold Locking Strategy. To effectively filter out erroneous pseudo-labels and improve generalization performance when the model achieves high stability and accuracy, SAT needs to maintain a stable and relatively high threshold. This invention implements a threshold locking strategy, where the threshold is locked at a fixed high value when the estimated model learning state does not increase significantly. This strategy is expressed as follows:

[0101] w t ―u t >ξ#(5)

[0102] Among them, u t This indicates that all w in the first t iterations t The average value of ξ is a local minimum, which in this invention is ξ = 0.02. If this condition is met, the threshold value increases with w. t Change, or otherwise locked in

[0103] In order to reduce u t To address the fluctuations, this invention uses the EMA method with a smoothing factor of α to update u. t , is represented as:

[0104] u t =αu t―1 +(1―α)u t #(6)

[0105] Where α = 0.9.

[0106] Step 4.6: In summary, the SAT mechanism is represented as follows:

[0107]

[0108] Step 5: Filter pseudo-labels and model predictions based on adaptive thresholds:

[0109] The student model only retains pseudo-labels that are greater than its adaptive threshold;

[0110] The teacher model only retains pseudo-labels that are greater than its adaptive threshold;

[0111] The student model retains only predictions smaller than its adaptive threshold, while the teacher model retains predictions for the corresponding positions. The predictions from both the student and teacher models have the same size and are filtered using a single mask. Some pixels from the student model's predictions were retained, and the teacher model also retained the predicted values ​​of the corresponding pixels.

[0112] For example, if the student model retains the prediction at position (i,j), the teacher model's prediction at the same position (i,j) will also be retained and used in the loss function.

[0113] The superscripts w and s refer to weakly enhanced and strongly enhanced data, respectively, for example:

[0114] The student model is based on the weakly augmented input image. Generate prediction probability map

[0115] The student model is based on the strongly augmented input image. Generate prediction probability map

[0116] (H and W are the dimensions of the image, and C is the number of categories).

[0117] A prediction probability map is a model's prediction for each pixel, providing a probability value for each pixel in an image to belong to a different category.

[0118] Prediction confidence plot It is a probability graph The highest probability value for each pixel. Compare this to a confidence plot and a threshold.

[0119] Pseudo-labels It is based on the predicted probability map For each pixel, the category with the highest probability is selected as the pseudo-label, meaning the pseudo-label is the index of the category.

[0120] However, the teacher model's input only includes weakly enhanced images.

[0121] The teacher model is based on the weakly enhanced input image. Generate prediction probability map (H and W are the dimensions of the image, and C is the number of categories).

[0122] Predicted probability map It is the model's prediction for each pixel, providing the probability that each pixel in the image belongs to a different category.

[0123] Prediction confidence plot It represents the highest probability value for each pixel in the probability graph. A confidence graph is then used to compare it with a threshold.

[0124] Pseudo-labels It is based on the predicted probability map For each pixel, the category with the highest probability is selected as the pseudo-label, meaning the pseudo-label is the index of the category.

[0125] Step 6: PPE model training and parameter update:

[0126] Model training aims to optimize the student model, while the teacher model plays only a supporting role. After each iteration, the teacher model directly inherits the parameters of the student model; the teacher model's parameters are not updated through training.

[0127] The training set includes both labeled and unlabeled data. Labeled data will only be fed into the student model.

[0128] Specific iterative training process:

[0129] Initialization: Randomly initialize the parameters of the student model and the teacher model.

[0130] Student model training: The student model receives data from the labeled dataset and generates a prediction P.

[0131] The student model receives weakly augmented images from an unlabeled dataset and generates predictions. Pseudo-labels

[0132] The student model receives strongly augmented images from an unlabeled dataset and generates predictions.

[0133] The teacher model receives weakly augmented images from an unlabeled dataset and generates predictions. Pseudo-labels

[0134] The student model is updated according to the loss function in formula (13).

[0135] Teacher model parameter update: After each iteration, i.e. after the student model parameters are updated, the teacher model inherits the parameters of the student model through formula (10).

[0136] Repeated iterations: The above process is repeated in multiple training iterations to continuously optimize the student model and update the teacher model.

[0137] Supervised training is performed on labeled data, and supervised loss is applied.

[0138] For unlabeled data:

[0139] Pseudo-labels generated by the student model Student model prediction Apply pseudo-label loss between

[0140] Pseudo-labels generated by the teacher model Student model prediction Applying collaborative pseudo-label loss between them

[0141] Apply a consistency loss between the predictions of the student model and the teacher model.

[0142] The student model parameters are updated based on the loss function described above. The teacher model updates its parameters from the student model using the EMA method. Finally, the best model is selected as the final student model and teacher model based on the test results, and the parameter file of the student model is saved.

[0143] Step 6.1: For labeled data, this invention performs supervised training with supervised loss. Represented as:

[0144]

[0145] Where B represents the batch size in each iteration, i.e., the number of samples used in each iteration, P represents the model prediction map, and Y represents the pixel-level label. Represents cross-entropy loss, This indicates Dice's loss.

[0146] Step 6.2: PPE uses pseudo-labels generated by the student model. To guide the prediction of student models This results in pseudo-label loss. Represented as:

[0147]

[0148] in A confidence plot representing the model's predictions. This represents a filter mask; only masks that meet certain conditions are allowed. Only pixels that meet the criteria will undergo loss calculation. It includes the three losses in equation (14).

[0149] Step 6.3: Furthermore, PPE also utilized a collaborative teacher model to generate pseudo-labels from another perspective. The collaborative average teacher model used in this invention shares the same structure as the student model. The collaborative average teacher model updates parameters from the student model via EMA, as shown below:

[0150]

[0151] Where β = 0.99, The parameters represent the collaborative average teacher model. This represents the parameters of the student model.

[0152] The pseudo-labels generated by the student model and the collaborative average teacher model jointly guide the student model's learning, providing dual supervision and thus enhancing the comprehensive representation of the pseudo-labels. In this process, the collaborative pseudo-label loss between the pseudo-labels generated by the collaborative average teacher model and the predictions of the student model is considered. It can be represented as:

[0153]

[0154] in It includes the three losses in equation (14).

[0155] Step 6.4: Further, PPE applies a consistency regularization loss to the predictions of unreliable pixels in both models, achieving consistency for predictions below a threshold (i.e., ...). The utilization of pixels. Consistency regularization loss corrects predictions before and after data or model perturbations, avoiding the accumulation of errors caused by low confidence. Consistency regularization loss. Represented as:

[0156]

[0157] in This represents the mean square error loss.

[0158] Step 6.5: By using consistency regularization loss, PPE extends the threshold-based pseudo-label method to achieve full utilization of unlabeled data, and enhances the effectiveness of pseudo-labels through collaborative pseudo-label loss. The total loss of the PPE framework is expressed as follows:

[0159]

[0160] Where λ1 = 0.5, λ2 = 0.4, and λ3 = 0.1

[0161] The focus of this invention is the utilization of unlabeled data. For unlabeled data, this invention uses data augmentation techniques to generate five forward data streams. Two of these weakly augmented data streams generate pseudo-labels using student and teacher models. and The three strongly augmented data streams generate predictions through the student model. Weak augmentations include image flipping and rotation, while strong augmentations encompass Dropout and CutMix (random cropping and pasting) in feature enhancement. For the strongly augmented data streams, The prediction generated after feature enhancement, and and This is the prediction after the image is CutMixed. The pseudo-label-based loss applies Dice loss to the three strong enhancements mentioned above simultaneously, as shown in equation (9) in step 6.2. And equation (11) in step 6.3 The following three Dice losses are included, denoted as:

[0162]

[0163] Wherein, γ1=0.5, γ2=0.25, γ3=0.25.

[0164] Step 6.6: Select the best model as the final student model and teacher model based on the test results, and save the parameter file of the student model.

[0165] The dataset includes a training set, a validation set, and a test set. During model training, the model performance is evaluated using the validation set every 30 epochs, with the Dice coefficient as the primary metric and IoU and HD95 as secondary metrics. The parameter file of the best-performing student model is also saved.

[0166] Step 7: Reasoning Stage

[0167] Use the student model to perform the segmentation task.

[0168] Student model reasoning process:

[0169] Data preprocessing: First, the medical images are subjected to min-max normalization, and the image size is uniformly adjusted to 256×256. If it is a 3D image, it is sliced ​​into a 2D image.

[0170] Model inference: Load the trained model using PyTorch and input the preprocessed image into the deep learning model for inference.

[0171] Output generation: Based on the probability map generated by the model, a corresponding category is assigned to each pixel to generate a segmentation mask. Subsequently, contour extraction is performed based on the segmentation mask, and the extracted contours are superimposed on the original image to generate a visualization result.

[0172] Input to the student model during the inference phase: preprocessed medical images.

[0173] The student model's output during the inference phase consists of two segmentation result images. The first image is a 2D mask of the same size as the input image, where each pixel is labeled with its corresponding category. For example, in the ACDC dataset, the categories include background, left ventricle, myocardium, and right ventricle, so the value of each pixel in the mask is 0 (background), 1 (left ventricle), 2 (myocardium), and 3 (right ventricle). The second image is an overlay of the original input image and the extracted contour lines, derived from the first image. By displaying the contour lines in RGB colors on top of the original image, a more intuitive visualization is provided.

[0174] Example 1

[0175] (1) Dataset and label settings:

[0176] This invention utilizes two public datasets: ACDC and PROMISE12, to evaluate the effectiveness of the method. The ACDC dataset consists of 200 MRI scan images from 100 subjects, labeled with three cardiac regions: left ventricle (LV), myocardium (Myo), and right ventricle (RV). In the experiments, this invention converted the three-dimensional MRI scan images into two-dimensional slices, generating a total of 1902 slices, which were then cropped to a uniform size of 256×256. 70 subjects (140 scans, 1312 slices) were assigned to the training dataset, and 30 subjects (60 scans, 590 slices) were assigned to the testing dataset. The PROMISE12 dataset includes 50 transverse T2-weighted MR prostate images from 50 subjects. This invention converted the three-dimensional images into two-dimensional slices, generating a total of 1377 slices, which were then cropped to a size of 256×256. 40 subjects (1124 slices) were assigned to the training dataset, and 10 subjects (253 slices) were assigned to the testing dataset. 5%, 10%, and 20% of the data in the training set are randomly sampled as labeled datasets, and a uniform random seed is used to ensure that all methods use the same labeled dataset.

[0177] (2) Experimental details:

[0178] To ensure a fair comparison with existing research, the method of this invention and all comparative methods use U-Net as the segmentation model. All evaluation methods are implemented using the PyTorch framework and experiments are conducted on a workstation equipped with an NVIDIA 4090 GPU. Each experiment is trained for 300 epochs, with a batch size of 24, using the Adam optimizer, and an initial learning rate of 0.001.

[0179] (3) Evaluation indicators:

[0180] This invention employs three evaluation metrics to assess segmentation performance: the Dice coefficient, the intersection-over-union (IoU) ratio, and the 95th percentile of the Hausdorff distance (HD95). Dice and IoU are used to evaluate segmentation accuracy, while HD95 is used to evaluate boundary segmentation performance.

[0181] (4) Comparison method:

[0182] This invention selected seven advanced semi-supervised learning methods for comparison: Mean Teacher Model (MT), Uncertainty-Aware Mean Teacher Model (UA-MT), Adversarial Entropy Minimization (ADVENT), Interpolation Consistency Training (ICT), Fixed Thresholding (FixMatch), Cross-Pseudo-Supervision (CPS), and Two-Stream Perturbation Fixed Thresholding (UniMatch). Furthermore, the performance of U-Net in a purely supervised learning environment was evaluated. This includes U-Net-All, trained using full-label data, and U-Net-L, trained using only partial-label data. Theoretically, U-Net-All can represent the upper limit of performance for semi-supervised learning methods, while U-Net-L represents the lower limit.

[0183] (5) Experimental results:

[0184] Table 1 presents the quantitative evaluation results for the ACDC dataset. This invention demonstrates significant improvements in the 5%, 10%, and 20% data partitions. For example, compared to the fixed-threshold-based UniMatch method, PPE improves Dice from 88.45% to 90.08%, IoU from 80.02% to 82.61%, and reduces HD95 from 1.38 to 0.84 when using 5% labeled data. Notably, when using 20% ​​labeled data, PPE achieves a Dice of 91.42%, approaching the upper limit of 91.57% for fully supervised datasets. Figure 4 (a) shows the comparison results between PPE and the supervision method under different label ratios.

[0185] Table 2 presents the quantitative results of the PROMISE12 dataset. As... Figure 4 As shown in (b), significant segmentation performance can be achieved even using only 5% labeled data. Specifically, with only 5% labeled data, PPE significantly outperforms the fully supervised U-Net-L, with a 65.67% increase in Dice, a 57.88% increase in IoU, and a 66.31 reduction in HD95. These results validate the effectiveness of improving segmentation performance using unlabeled data, demonstrating the effectiveness of this invention even with limited labeled data.

[0186] (6) Results visualization:

[0187] Figure 5 The first two rows present the segmentation visualization results for the ACDC dataset. It can be observed that all methods achieve satisfactory results in segmenting the central region (especially the left ventricle). Notably, the proposed method shows significant improvement in segmenting the peripheral region (especially the right ventricle). Although the comparative method UniMatch performs well in overall segmentation performance, it still falls short in handling boundary regions.

[0188] Figure 5 The last two lines show the segmentation visualization results for the PROMISE12 dataset. It can be seen that when using only 5% of the labeled data, most comparison methods (such as MT, ICT, FixMatch, and CPS) do not achieve ideal segmentation results. However, this invention maintains the integrity of the segmentation target while achieving a high degree of matching with the ground truth.

[0189] Table 1 Quantitative evaluation results of the ACDC dataset

[0190]

[0191] Table 2 Quantitative evaluation results of the PROMISE12 dataset

[0192]

[0193] The above description is merely an embodiment of the present invention and is not intended to limit the invention. For those skilled in the art, the present invention...

[0194] In summary, unlike existing pseudo-labeling methods that use a fixed threshold, this invention proposes an adaptive anchor thresholding (SAT) mechanism that can dynamically adjust the threshold based on the model learning state reflected by the anchor pixel, thereby generating reliable pixel-level pseudo-labels more effectively.

[0195] This invention proposes a progressive pseudo-label enhancement framework (PPE) that combines pseudo-labels with consistency regularization to effectively utilize unreliable pixels with confidence levels below an adaptive threshold.

[0196] This invention not only utilizes more reliable pixels, but also improves the training effect of the model in semi-supervised medical image segmentation by optimizing the use of unreliable pixels, especially when dealing with challenging regions such as boundary regions, thus improving the model's segmentation performance.

[0197] Those skilled in the art will understand that the accompanying drawings are merely schematic diagrams of one embodiment, and the modules or processes shown in the drawings are not necessarily essential for implementing the present invention.

[0198] As can be seen from the above description of the embodiments, those skilled in the art can clearly understand that the present invention can be implemented by means of software plus necessary general-purpose hardware platforms. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in various embodiments or some parts of the embodiments of the present invention.

[0199] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, for apparatus or system embodiments, since they are basically similar to method embodiments, the description is relatively simple; relevant parts can be referred to the descriptions in the method embodiments. The apparatus and system embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without creative effort.

[0200] The above description is merely a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A semi-supervised medical image segmentation method based on progressive pseudo-label enhancement, characterized in that, include: The original medical image dataset is labeled, and the labeled original medical image dataset is divided into labeled dataset and unlabeled dataset. Weak augmentation and strong augmentation are used to augment the unlabeled data. A progressive pseudo-label enhancement framework (PPE) network model is constructed, comprising a student model and a collaborative average teacher model. The PPE network model is iteratively trained using the labeled and unlabeled datasets. In each iteration, an adaptive threshold is generated using the adaptive anchor threshold (SAT) mechanism. The adaptive threshold mechanism is used to filter out pseudo-labels in the student model and the collaborative average teacher model that are below a certain confidence range. The parameters of the student model and the collaborative average teacher model are iteratively updated to obtain the trained student model and the collaborative average teacher model. The preprocessed medical image is input into the trained student model, and the trained student model outputs the image category segmentation result of the medical image; The method of iteratively training the PPE network model using the labeled and unlabeled datasets, wherein in each iteration, an adaptive threshold is generated using an adaptive anchor threshold (SAT) mechanism, and pseudo-labels below a certain confidence range in the student model and collaborative average teacher model are filtered out using the adaptive threshold mechanism, and the parameters of the student model and collaborative average teacher model are iteratively updated to obtain the trained student model and collaborative average teacher model, includes: In each iteration of training the PPE network model, an adaptive threshold is generated using the SAT mechanism. The confidence score of the anchor pixels is calculated based on the model prediction probability map and confidence map output by the collaborative average teacher model. The learning state of the PPE network model is estimated using the confidence scores of the anchor pixels, and the learning state of the PPE network model is estimated based on the average confidence score of all anchor pixels in a batch. , represented as: in Indicates the number of iterations. This represents the batch size in each iteration, i.e., the number of samples used in each iteration. For batch Confidence level value; An adaptive threshold is generated for each training iteration by using the estimated model learning state as a weighting function. , represented as: in It is a predefined maximum threshold; Define the lower limit of the adaptive anchor threshold. ,in Indicates the number of categories. The threshold locking strategy is expressed as follows: in, Indicates preceding All in the next iteration The average value, If the condition is met, the threshold value is a local minimum. Change, or otherwise locked in ; Use a smoothing factor as EMA method update , represented as: in ; The SAT mechanism is represented as follows: The student model only retains students whose scores are greater than their adaptive threshold. The pseudo-labels, the collaborative average teacher model only retains those greater than its adaptive threshold. The pseudo-labels are used, and a filtering mask is used to retain some pixels in the prediction of the student model. The collaborative average teacher model also retains the predicted value of the corresponding pixel. The parameters of the student model and the collaborative average teacher model are iteratively updated to obtain the trained student model and collaborative average teacher model.

2. The method according to claim 1, characterized in that, The process of labeling the original medical image dataset, dividing the labeled original medical image dataset into labeled and unlabeled datasets, includes: Raw medical image data is collected, and min-max normalization is applied to the raw medical images to convert the 3D medical images into 2D slice images. All images are then cropped to a uniform size. Randomly select a portion of images for pixel-level annotation to generate a labeled dataset. Unlabeled images constitute an unlabeled dataset. In semi-supervised learning In .

3. The method according to claim 2, characterized in that, The aforementioned data augmentation of unlabeled data using weak and strong augmentation includes: Using weak and strong augmentation on unlabeled data Data augmentation can be performed in several ways. Weak augmentation methods include flipping and rotating the image with a 50% probability, while strong augmentation methods include performing Dropout operations on the model's features and performing CutMix operations on the image, which involves cutting out a region of one image and replacing it with the corresponding region of another image.

4. The method according to claim 1, 2, or 3, characterized in that, The aforementioned construction of a progressive pseudo-label enhancement framework (PPE) network model, comprising a student model and a collaborative average teacher model, includes: Construct a PPE network model, which adopts a teacher-student structure and includes a student model. and Collaborative Average Teacher Model The two models, the student model and the collaborative average teacher model, have the same structure but different initializations. The student model and the collaborative average teacher model generate pseudo-labels through weakly enhanced images. Using a labeled dataset The student model is trained, and its parameters are updated via backpropagation, using weakly augmented unlabeled data. The student model is trained, and the student model and the collaborative average teacher model output a model prediction probability map, a confidence map, and pseudo-labels. The collaborative average teacher model is then trained using a smoothing factor after each iteration. The Exponential Moving Average (EMA) algorithm updates parameters from the student model.

5. The method according to claim 1, characterized in that, The iterative update of the parameters of the student model and the collaborative average teacher model to obtain the trained student model and collaborative average teacher model includes: The student model receives data from a labeled dataset and generates predictions. The student model receives weakly augmented images from an unlabeled dataset and generates predictions. pseudo-tags The student model receives strongly augmented images from an unlabeled dataset and generates predictions. The collaborative average teacher model receives weakly augmented images from an unlabeled dataset and generates predictions. pseudo-tags ; For labeled data, perform supervised training with supervised loss. Represented as: in This represents the batch size in each iteration, i.e., the number of samples used in each iteration. Represents the model's prediction graph. Represents pixel-level tags. Represents cross-entropy loss, Indicates Dice loss; Pseudo-labels generated using student models To guide the prediction of student models This leads to pseudo-label loss. , represented as: in A confidence plot representing the model's predictions. This represents a filter mask; only masks that meet certain conditions are allowed. Loss calculation will only be performed on pixels that meet the minimum requirements. Generating pseudo-labels from another perspective using a collaborative average teacher model. The collaborative average teacher model shares the same structure as the student model. The collaborative average teacher model updates parameters from the student model using the EMA method, as shown below: in, , The parameters represent the collaborative average teacher model. Represents the parameters of the student model; Collaborative pseudo-label loss between pseudo-labels generated by the collaborative average teacher model and predictions from the student model. Represented as: Consistency regularization loss Represented as: in Indicates the mean square error loss; The total loss of the PPE network model is expressed as follows: in, .

6. The method according to claim 1, characterized in that, The process of inputting the preprocessed medical image into a trained student model, and the training student model outputting the image category segmentation result of the medical image, includes: Perform min-max normalization on medical images and resize the images uniformly. The three-dimensional medical image is sliced ​​into a two-dimensional image to obtain a preprocessed medical image; Using PyTorch, a pre-trained student model is loaded. The pre-processed medical image is input into the pre-trained student model. Based on the probability map generated by the student model, each pixel of the medical image is assigned a corresponding category, a segmentation mask is generated, contours are extracted according to the segmentation mask, and the extracted contours are superimposed on the original medical image to generate a visualized image category segmentation result.

Images