Semi-supervised medical image segmentation method based on progressive pseudo label enhancement

Abstract

The invention provides a semi-supervised medical image segmentation method based on progressive pseudo label enhancement. The method comprises the steps of dividing a labeled original medical image data set into a labeled data set and a non-labeled data set, constructing a PPE network model comprising a student model and a teacher model, performing iterative training on the PPE network model by using the labeled data set and the non-labeled data set, generating a self-adaptive threshold value by using an SAT mechanism in each iteration, and outputting the self-adaptive threshold value. Filtering out false labels lower than a certain confidence coefficient range in the student model and the teacher model to obtain a trained student model and a trained teacher model; and inputting the preprocessed medical image into the trained student model, and outputting an image category segmentation result of the medical image. According to the method, the SAT mechanism is used for generating the self-adaptive threshold value, the utilization rate of reliable pixels is improved, then the cooperation and consistency regularization method of the teacher model is combined, unreliable pixels are effectively utilized, and comprehensive utilization of label-free data is achieved progressively.

Description

Technical Field

[0001] The present invention relates to the technical field of image segmentation, and in particular to a semi-supervised medical image segmentation method based on progressive pseudo-label enhancement. Background Art

[0002] Medical image segmentation is a fundamental task in medical imaging, which aims to accurately delineate regions of interest, such as lesions, tissues, and organs, for quantitative analysis, thereby providing more precise support to clinicians in diagnosis and treatment decisions. Currently, many advanced medical image segmentation methods rely mainly on a large amount of high-quality pixel-level annotated data for training. However, the process of annotating medical images is cumbersome and requires a high degree of expertise, so high-quality pixel-level annotated data is quite scarce. In sharp contrast, it is relatively simple to obtain unlabeled medical images. Therefore, it is particularly important to develop semi-supervised segmentation methods that can effectively utilize limited labeled data and a large amount of unlabeled data.

[0003] The pseudo-label method is a typical method in semi-supervised methods and is widely used in various medical image segmentation tasks. At present, a semi-supervised medical image segmentation method in the prior art uses a threshold strategy to generate pseudo-labels from weakly enhanced unlabeled images and only retains pseudo-labels with confidence higher than a fixed threshold (usually 0.95). Subsequently, the method uses high-confidence pseudo-labels to guide the prediction of the strongly enhanced version of the same image to maintain consistency.

[0004] The disadvantages of the above-mentioned prior art semi-supervised medical image segmentation method include: a potential limitation of the method is its reliance on a fixed threshold. The threshold-based pseudo-labeling method directly filters out unreliable pixels, which are usually located in boundary areas that are difficult to segment, resulting in insufficient learning of the model in these areas. At the same time, many pseudo-labels that are judged to be unreliable are actually correctly predicted, but are mistakenly regarded as unreliable only because a higher threshold is used. This misjudgment limits the model's effective use of data information, thereby affecting segmentation performance. Summary of the invention

[0005] The present invention provides a semi-supervised medical image segmentation method based on progressive pseudo-label enhancement, which can fully utilize unlabeled data.

[0006] In order to achieve the above object, the present invention adopts the following technical scheme.

[0007] A semi-supervised medical image segmentation method based on progressive pseudo-label enhancement, comprising:

[0008] Annotate the original medical image dataset, divide the annotated original medical image dataset into labeled dataset and unlabeled dataset, and use weak enhancement and strong enhancement to enhance the unlabeled data;

[0009] A progressive pseudo-label enhancement framework PPE network model is constructed, the PPE network model includes a student model and a teacher model, the PPE network model is iteratively trained using the labeled data set and the unlabeled data set, in each iteration, an adaptive anchor threshold SAT mechanism is used to generate an adaptive threshold, and the adaptive threshold mechanism is used to filter out pseudo-labels in the student model and the teacher model that are below a certain confidence range, so as to obtain a trained student model and a teacher model;

[0010] The preprocessed medical image is input into a trained student model, and the trained student model outputs an image category segmentation result of the medical image.

[0011] Preferably, the step of labeling the original medical image dataset and dividing the labeled original medical image dataset into a labeled dataset and an unlabeled dataset includes:

[0012] Collect the original medical image data, perform minimum-maximum normalization on the original medical images, convert the three-dimensional medical images into two-dimensional slice images, crop all images to a uniform size of 256×256, randomly select some images for pixel-level annotation, and generate a labeled dataset D L , the unlabeled images constitute the unlabeled dataset D U , in semi-supervised learning D U The amount of data in is greater than D L The amount of data in .

[0013] Preferably, the method of performing data enhancement on unlabeled data using weak enhancement and strong enhancement includes:

[0014] Use weak and strong augmentation to train unlabeled data D U Data enhancement is performed. The specific methods of weak enhancement include: flipping and rotating the image with a probability of 50%, and the specific methods of strong enhancement include: performing a Dropout operation on the features of the model; performing a CutMix operation on the image, that is, cutting an area of ​​an image and replacing it with the corresponding area of ​​another image.

[0015] Preferably, the PPE network model of the progressive pseudo-label enhancement framework is constructed, and the PPE network model includes a student model and a teacher model, including:

[0016] Construct a PPE network model, which adopts a teacher-student structure and includes a student model and teacher model Two models, the student model and the teacher model have the same structure but different initializations, and the student model and the teacher model generate pseudo labels through weakly enhanced images.

[0017] Using the labeled dataset D L Train the student model, whose parameters are updated by back propagation, using weakly enhanced unlabeled data D U The student model is trained, and the student model and the teacher model output model prediction probability maps, confidence maps and pseudo labels. After each iterative training, the teacher model updates parameters from the student model through an exponential moving average algorithm EMA with a smoothing factor β.

[0018] Preferably, the PPE network model is iteratively trained using the labeled data set and the unlabeled data set. In each iteration, an adaptive anchor threshold SAT mechanism is used to generate an adaptive threshold, and the adaptive threshold mechanism is used to filter out pseudo labels below a certain confidence range in the student model and the teacher model, and the parameters of the student model and the teacher model are iteratively updated to obtain the trained student model and the teacher model, including:

[0019] In each iterative training of the PPE network model, the SAT mechanism is used to generate an adaptive threshold, the confidence of the anchor pixel is calculated according to the model prediction probability map and the confidence map output by the teacher model, the learning state of the PPE network model is estimated by the confidence of the anchor pixel, and the learning state w of the PPE network model is estimated according to the average confidence of all anchor pixels in a batch. t , expressed as:

[0020]

[0021] Where t represents the number of iterations, and B represents the batch size in each iteration, that is, the number of samples used in each iteration;

[0022] The estimation of the model learning state is used as a weighted function of the threshold to generate an adaptive threshold for each training iteration. It is expressed as:

[0023]

[0024] in is the predefined maximum threshold;

[0025] Define the lower threshold of the adaptive anchor point Where C represents the number of categories,

[0026] The threshold locking strategy is expressed as follows:

[0027] w t―u t >ξ#(5)

[0028] Among them, u t Indicates that all w in the first t iterations t The average value of ξ is a minimum value. If this condition is met, the threshold value increases with w t changes, otherwise locked in

[0029] Update u using the EMA method with a smoothing factor α t , expressed as:

[0030] u t =αu t―1 +(1―α)u t #(6)

[0031] Where α = 0.9;

[0032] The SAT mechanism is expressed as follows:

[0033]

[0034] The student model only retains the values ​​greater than its adaptive threshold. The teacher model only retains pseudo labels greater than its adaptive threshold The pseudo labels of the student model are obtained by using a filtering mask to retain some pixels in the prediction of the student model, and the teacher model also retains the predicted values ​​of the pixels at the corresponding positions. The parameters of the student model and the teacher model are iteratively updated to obtain the trained student model and the teacher model.

[0035] Preferably, the iterative updating of the parameters of the student model and the teacher model to obtain the trained student model and the teacher model includes:

[0036] The student model receives data from the labeled dataset and generates a prediction P. The student model receives weakly enhanced images from the unlabeled dataset and generates a prediction Pseudo Labels The student model receives strongly augmented images from the unlabeled dataset and generates predictions The teacher model receives weakly augmented images from the unlabeled dataset and generates predictions Pseudo Labels

[0037] For labeled data, perform supervised training with supervised loss It is expressed as:

[0038]

[0039] Where B represents the batch size in each iteration, that is, the number of samples used in each iteration, P represents the model prediction map, and Y represents the pixel-level label. represents the cross entropy loss, represents Dice loss;

[0040] Using pseudo labels generated by the student model To guide the prediction of the student model Forming pseudo-label loss It is expressed as:

[0041]

[0042] in represents the confidence map of the model predictions, Represents a filter mask, only those that meet the conditions The loss calculation will be performed only for the pixels;

[0043] Generate pseudo labels from another perspective using collaborative teacher model The collaborative average teacher model shares the same structure as the student model. The collaborative average teacher model updates parameters from the student model via EMA, which can be expressed as:

[0044]

[0045] Among them, β = 0.99, represents the parameters of the collaborative average teacher model, Represents the parameters of the student model.

[0046] Collaborative pseudo-label loss between the pseudo-labels produced by the collaboratively averaged teacher model and the predictions of the student model It is expressed as:

[0047]

[0048] Consistency Regularization Loss It is expressed as:

[0049]

[0050] in represents the mean square error loss.

[0051] The total loss of the PPE network model is expressed as follows:

[0052]

[0053] Among them, λ1=0.5, λ2=0.4, λ3=0.1.

[0054] Preferably, the step of inputting the preprocessed medical image into a trained student model, wherein the trained student model outputs an image category segmentation result of the medical image, comprises:

[0055] Perform minimum-maximum normalization on the medical image, adjust the image size to 256×256, slice the three-dimensional medical image into a two-dimensional image, and obtain the preprocessed medical image;

[0056] Use PyTorch to load the trained student model, input the preprocessed medical image into the trained student model, assign the corresponding category to each pixel of the medical image based on the probability map generated by the student model, generate a segmentation mask, extract contours based on the segmentation mask, and superimpose the extracted contours on the original medical image to generate a visual image category segmentation result.

[0057] It can be seen from the technical solutions provided by the above-mentioned embodiments of the present invention that the PPE framework proposed in the embodiments of the present invention first uses the SAT mechanism to generate an adaptive threshold to improve the utilization rate of reliable pixels, and then combines the collaboration of the teacher model and the consistency regularization method to effectively utilize unreliable pixels, thereby gradually realizing the comprehensive utilization of unlabeled data.

[0058] Additional aspects and advantages of the present invention will be given in part in the following description, which will become obvious from the following description, or may be learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0059] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative work.

[0060] Figure 1 A processing flow chart of a semi-supervised medical image segmentation method based on progressive pseudo-label enhancement provided by an embodiment of the present invention;

[0061] Figure 2 A framework diagram for training unlabeled data in the present invention;

[0062] Figure 3 A calculation diagram for estimating the model state according to the present invention;

[0063] Figure 4 The effect comparison between the present invention and fully supervised learning is shown;

[0064] Figure 5 This is a comparison diagram of the segmentation effect of the present invention. DETAILED DESCRIPTION

[0065] The embodiments of the present invention are described in detail below, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and cannot be interpreted as limiting the present invention.

[0066] It will be understood by those skilled in the art that, unless expressly stated, the singular forms "one", "said", and "the" used herein may also include plural forms. It should be further understood that the term "comprising" used in the specification of the present invention refers to the presence of the features, integers, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or groups thereof. It should be understood that when we refer to an element as being "connected" or "coupled" to another element, it may be directly connected or coupled to the other element, or there may be intermediate elements. In addition, the "connection" or "coupling" used herein may include wireless connection or coupling. The term "and / or" used herein includes any unit and all combinations of one or more associated listed items.

[0067] It will be understood by those skilled in the art that, unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as those generally understood by those skilled in the art in the art to which the present invention belongs. It should also be understood that terms such as those defined in common dictionaries should be understood to have meanings consistent with the meanings in the context of the prior art, and will not be interpreted with idealized or overly formal meanings unless defined as herein.

[0068] To facilitate understanding of the embodiments of the present invention, several specific embodiments will be further explained below with reference to the accompanying drawings, and each embodiment does not constitute a limitation on the embodiments of the present invention.

[0069] The present invention classifies pixels (pseudo labels) with confidence levels higher than a threshold as reliable, and pixels (pseudo labels) with confidence levels lower than the threshold as unreliable.

[0070] The processing flow of a semi-supervised medical image segmentation method (PPE) based on progressive pseudo-label enhancement provided by an embodiment of the present invention is as follows: Figure 1 As shown, the following steps are included:

[0071] Step 1: Data Preparation:

[0072] Collect image data, annotate the image data, and divide the training set into labeled data sets D L and the unlabeled dataset D U .

[0073] The present invention first performs data preprocessing and data set division on the acquired original medical images. Specifically, the original medical images are first subjected to minimum-maximum normalization. For three-dimensional medical images, they need to be converted into two-dimensional slice images, and then all images are cropped to a uniform size of 256×256. Finally, some images are randomly selected for pixel-level annotation to generate a labeled data set D L , and the unlabeled images constitute the unlabeled dataset D U In semi-supervised learning, there is usually D U >>D L .

[0074] Step 2: Data Augmentation:

[0075] The present invention uses weak enhancement and strong enhancement to the unlabeled data D U Perform data augmentation.

[0076] The specific method of weak enhancement is to flip and rotate the image with a probability of 50%.

[0077] The specific method of strong enhancement is: perform Dropout operation on the features of the model; perform CutMix operation on the image, that is, cut an area of ​​an image and replace it with the corresponding area of ​​another image.

[0078] Step 3: Construct the PPE (Progressive Pseudo-labeling Enhancement) network model.

[0079] Figure 2 This is a framework diagram for training unlabeled data in the present invention, such as Figure 2 As shown, the PPE model adopts a teacher-student structure, including the student model and teacher model Two models. The student model and the teacher model have the same structure but different initializations. Update of student model parameters: Parameter update is performed through back propagation.

[0080] Update of teacher model parameters: The teacher model inherits parameters from the student model through EMA after each iteration. L , only train the student model. The teacher model only uses the unlabeled dataset D U To assist student model training, the input of the teacher model is only weakly enhanced images, and the output includes model prediction probability maps, confidence maps, and pseudo labels.

[0081] The network structures of both the student model and the teacher model are U-Net, and both models generate pseudo labels through weakly enhanced images. L and the unlabeled dataset D U The PPE network model is iteratively trained. In the training phase, the student model updates parameters through back propagation, and the teacher model updates parameters from the student model through EMA (Exponential Moving Average) with a smoothing factor β; in the inference phase, only the student model is used.

[0082] Step 4: Generation of adaptive threshold.

[0083] An important feature of the present invention is that in each iteration of model training, an adaptive threshold is generated using the SAT (Adaptive feature pooling, adaptive anchor threshold) mechanism. The processing process includes: calculating the confidence of the anchor pixel based on the prediction result of the weakly enhanced image. Then, the learning state of the model is estimated by the confidence of the anchor pixel, and the adaptive threshold is generated using the learning state as a weighted function. Then, the adaptive anchor threshold is corrected by the threshold protection strategy and the threshold locking strategy.

[0084] The SAT mechanism dynamically adjusts the threshold according to the learning state of the model. In the threshold-based pseudo-labeling method, a higher fixed threshold is usually used to filter the pseudo-labels, but this causes some pseudo-labels that are actually predicted to be correct to be misjudged as unreliable. In contrast, the adaptive threshold mechanism designed by the present invention is intended to retain more correct pseudo-labels rather than only retaining high-confidence pseudo-labels. Since the confidence of the model in the early stage of training is usually low and the judgment accuracy is insufficient, a lower threshold is required to retain the pseudo-labels. As the training deepens, the model confidence level increases and can provide more accurate judgments. In this case, a higher threshold is suitable for retaining high-confidence pseudo-labels.

[0085] Based on the above analysis, the SAT mechanism of the present invention includes the following steps:

[0086] Step 4.1: Calculate the confidence of the anchor pixel. For each image in a batch, the model generates a probability map that represents the likelihood that each pixel belongs to a specific class. Confidence map It is generated by selecting the maximum probability value of all categories for each pixel. In order to effectively reflect the learning state of the model, the present invention introduces the concept of anchor pixels. For each pixel in the confidence map, the anchor pixel is defined as the pixel that falls in the first percentile with the lowest confidence value. This is achieved by sorting all pixels by confidence. The confidence value of the anchor pixel serves as an indicator of the learning state of the model and represents the predictive ability of the model. The sorted confidence sequence It is expressed as:

[0087]

[0088] Where H, W represent the height and width of the input image, According to the confidence graph Q w The obtained sequence, q i,j Represents the confidence map Q w The confidence value of position (i, j) in .

[0089] The present invention converts the confidence value q of the anchor pixel into anchor Defined as the confidence value at the first percentile in the sorted confidence sequence S, q anchor The calculation method is as follows:

[0090] q anchor =S[0.01×N]#(2)

[0091] Where N = H × W, [0.01 × N] represents the position index of the first percentile in sequence S.

[0092] Step 4.2: Estimate the model learning state. Estimate the model learning state w according to the average confidence of all anchor pixels in a batch t , expressed as:

[0093]

[0094] Where t represents the number of iterations, and B represents the batch size during each iteration, that is, the number of samples used in each iteration.

[0095] Step 4.3: Generate adaptive threshold.

[0096] The estimation of the model learning state is used as a weighted function of the threshold to generate an adaptive threshold for each training iteration. It is expressed as:

[0097]

[0098] in is a predefined maximum threshold. In the present invention,

[0099] Step 4.4: Threshold protection strategy. In the early stage of training, due to the random initialization of model parameters, the anchor pixel confidence may be too low, resulting in a large number of incorrect pseudo labels being retained. To solve this problem, define the lower threshold Where C represents the number of categories. This strategy prevents the use of unreliable pseudo labels due to too low a threshold and balances the selection probabilities of different categories.

[0100] Step 4.5: Threshold locking strategy. In order to effectively filter out erroneous pseudo-labels and improve generalization performance when the model achieves high stability and accuracy, SAT needs to maintain a stable and high threshold. The present invention implements a threshold locking strategy. When the estimation of the model learning state does not increase significantly, the locking threshold is a fixed high value. The strategy is expressed as follows:

[0101] w t ―u t >ξ#(5)

[0102] Among them, u t Indicates that all w in the first t iterations t The average value of w is ξ, and ξ is a minimum value. In the present invention, ξ=0.02. If this condition is met, the threshold value increases with w t changes, otherwise locked in

[0103] In order to reduce u t The present invention uses the EMA method with a smoothing factor of α to update u t , expressed as:

[0104] u t =αu t―1 +(1―α)u t #(6)

[0105] Where α=0.9.

[0106] Step 4.6: In summary, the SAT mechanism is expressed as follows:

[0107]

[0108] Step 5: Filter pseudo labels and model predictions based on adaptive thresholds:

[0109] The student model only retains pseudo labels that are larger than its adaptive threshold;

[0110] The teacher model only retains pseudo labels that are larger than its adaptive threshold;

[0111] The student model only retains predictions that are less than its adaptive threshold, and the teacher model retains predictions at the corresponding positions. The predictions of the student model and the teacher model have the same size, using a filtering mask Some pixels in the student model's prediction are retained, and the teacher model also retains the predicted values ​​of the pixels at the corresponding positions.

[0112] For example, if the student model retains the prediction at position (i, j), the teacher model's prediction at the same position (i, j) will also be retained for the loss function

[0113] The superscripts w and s refer to weakly enhanced and strongly enhanced data, respectively, for example:

[0114] The student model is based on the weakly enhanced image input Generate predicted probability plot

[0115] The student model is based on the input strongly enhanced image Generate predicted probability plot

[0116] (H and W are the dimensions of the image, and C is the number of classes).

[0117] The prediction probability map is the model's prediction for each pixel, which provides the probability value of belonging to different categories for each pixel in the image.

[0118] Prediction confidence graph It is a probability graph The highest probability value for each pixel in . Use the confidence map to compare with the threshold.

[0119] Pseudo Labels Based on the predicted probability map For each pixel in , the category with the highest probability is selected as the pseudo label, that is, the pseudo label is the index of the category.

[0120] But the input of the teacher model only includes weakly enhanced images.

[0121] The teacher model is based on the weakly enhanced image of the input Generate predicted probability plot (H and W are the dimensions of the image, and C is the number of classes).

[0122] Prediction probability plot is the model’s prediction for each pixel, which provides the probability of belonging to different categories for each pixel in the image.

[0123] Prediction confidence graph is the highest probability value for each pixel in the probability map. Use the confidence map to compare with the threshold.

[0124] Pseudo Labels Based on the predicted probability map For each pixel in , the category with the highest probability is selected as the pseudo label, that is, the pseudo label is the index of the category.

[0125] Step 6: PPE model training and parameter update:

[0126] Model training aims to optimize the student model, while the teacher model only plays an auxiliary role. After each iteration, the teacher model directly inherits the parameters of the student model, and the parameters of the teacher model will not be updated through training.

[0127] The training set includes labeled data and unlabeled data. For labeled data, only it is input into the student model.

[0128] Specific iterative training process:

[0129] Initialization: Randomly initialize the parameters of the student model and the teacher model.

[0130] Student model training: The student model receives data from the labeled dataset and generates predictions P.

[0131] The student model receives weakly augmented images from the unlabeled dataset and generates predictions Pseudo Labels

[0132] The student model receives strongly augmented images from the unlabeled dataset and generates predictions

[0133] The teacher model receives weakly augmented images from the unlabeled dataset and generates predictions Pseudo Labels

[0134] Update the student model according to the loss function of formula (13).

[0135] Update of teacher model parameters: After each iteration, that is, after the student model parameters are updated, the teacher model inherits the parameters of the student model through formula (10).

[0136] Repeated Iteration: The above process is repeated in multiple training iterations to continuously optimize the student model and update the teacher model.

[0137] Perform supervised training on labeled data and perform supervised loss

[0138] For unlabeled data:

[0139] Pseudo labels generated by the student model and the student model predicts Apply pseudo-label loss between

[0140] The pseudo labels generated by the teacher model and the student model predicts Apply collaborative pseudo-label loss between

[0141] Apply a consistency loss between the predictions of the student model and the teacher model

[0142] The student model parameters are updated according to the above loss function, and the teacher model parameters are updated from the student model through EMA. Finally, the best model is selected as the final student model and teacher model according to the test results, and the parameter file of the student model is saved.

[0143] Step 6.1: For labeled data, the present invention performs supervised training with supervised loss It is expressed as:

[0144]

[0145] Where B represents the batch size in each iteration, that is, the number of samples used in each iteration, P represents the model prediction map, and Y represents the pixel-level label. represents the cross entropy loss, Denotes Dice loss.

[0146] Step 6.2: PPE uses pseudo labels generated by the student model To guide the prediction of the student model Thus forming a pseudo label loss It is expressed as:

[0147]

[0148] in represents the confidence map of the model predictions, Represents a filter mask, only those that meet the conditions The loss calculation will be performed only for the pixels It includes the three losses in formula (14).

[0149] Step 6.3: Furthermore, PPE also uses the collaborative teacher model to generate pseudo labels from another perspective The collaborative average teacher model used in the present invention shares the same structure as the student model. The collaborative average teacher model updates parameters from the student model through EMA, which is expressed as:

[0150]

[0151] Among them, β = 0.99, represents the parameters of the collaborative average teacher model, Represents the parameters of the student model.

[0152] The pseudo-labels generated by the student model and the collaborative average teacher model jointly guide the learning of the student model, providing dual supervision for the student model, thereby enhancing the comprehensive representation of the pseudo-labels. In this process, the collaborative pseudo-label loss between the pseudo-labels generated by the collaborative average teacher model and the predictions of the student model is It can be expressed as:

[0153]

[0154] in It includes the three losses in formula (14).

[0155] Step 6.4: Further, PPE applies a consistency regularization loss to the predictions of unreliable pixels in both models to achieve a loss of ) pixels. The consistency regularization loss corrects the predictions before and after the data or model perturbations, and does not accumulate errors caused by low confidence. It is expressed as:

[0156]

[0157] in represents the mean square error loss.

[0158] Step 6.5: Through the consistency regularization loss, PPE extends the threshold-based pseudo-labeling method to achieve full utilization of unlabeled data, and enhances the effectiveness of pseudo-labeling through collaborative pseudo-labeling loss. The total loss of the PPE framework is expressed as follows:

[0159]

[0160] Among them, λ1=0.5, λ2=0.4, λ3=0.1

[0161] The focus of the present invention is the utilization of unlabeled data. For unlabeled data, the present invention uses data enhancement technology to generate 5 forward data streams. Among them, 2 weakly enhanced data streams generate pseudo labels through student and teacher models. and The three strong enhancement data streams generate predictions through the student model. Weak enhancement includes image flipping and rotation, while strong enhancement covers Dropout in feature enhancement and CutMix in random cropping and pasting. For the strong enhancement data stream, is the prediction generated after feature enhancement, and and is the prediction of the image after CutMix. The pseudo-label-based loss will apply Dice loss to the above three strong enhancements at the same time. and equation (11) in step 6.3 It includes the following three Dice losses, expressed as:

[0162]

[0163] Among them, γ1=0.5, γ2=0.25, γ3=0.25.

[0164] Step 6.6: Select the best model as the final student model and teacher model based on the test results, and save the parameter file of the student model.

[0165] The dataset includes training set, validation set and test set. During the model training process, the validation set is used to evaluate the model performance every 30 epochs, using the Dice coefficient as the main indicator, and IoU and HD95 as auxiliary indicators, and the parameter file of the best performing student model is saved.

[0166] Step 7: Inference Phase

[0167] Use the student model to perform the segmentation task.

[0168] Student model reasoning process:

[0169] Data preprocessing: First, perform minimum-maximum normalization on the medical images and resize the images to 256 × 256. If they are 3D images, slice them into 2D images.

[0170] Model inference: Use PyTorch to load the trained model and input the preprocessed images into the deep learning model for inference.

[0171] Output generation: Based on the probability map generated by the model, each pixel is assigned a corresponding category to generate a segmentation mask. Subsequently, contours are extracted based on the segmentation mask and the extracted contours are superimposed on the original image to generate visualization results.

[0172] Input to the student model during the inference phase: preprocessed medical images.

[0173] Output of the student model in the inference phase: includes two segmentation result images. The first image is a two-dimensional mask of the same size as the input image, in which each pixel is labeled with the corresponding category. For example, in the ACDC dataset, the categories include background, left ventricle, myocardium, and right ventricle, so the value of each pixel in the mask is 0 (background), 1 (left ventricle), 2 (myocardium), and 3 (right ventricle). The second image is an overlay of the original input image and the extracted contour lines. The contour lines are obtained based on the first image. By displaying the contour lines in RGB colors on the original image, a more intuitive visualization effect is provided.

[0174] Embodiment 1

[0175] (1) Dataset and label settings:

[0176] The present invention uses two public datasets: ACDC and PROMISE12 to evaluate the effectiveness of the method. The ACDC dataset consists of 200 MRI scan images of 100 subjects, containing labels of three cardiac regions: left ventricle (LV), myocardium (Myo), and right ventricle (RV). In the experiment, the present invention converts the three-dimensional MRI scan images into two-dimensional slices, generates a total of 1902 slices, and cuts them to a uniform size of 256×256. The present invention allocates 70 subjects (140 scans, 1312 slices) for training and 30 subjects (60 scans, 590 slices) for testing. The PROMISE12 dataset includes 50 transverse T2-weighted MR prostate images of 50 subjects. The present invention converts the three-dimensional images into two-dimensional slices, generates a total of 1377 slices, and cuts them to 256×256 size. Among them, 40 subjects (1124 slices) are used for training and 10 subjects (253 slices) are used for testing. 5%, 10%, and 20% of the data are randomly selected from the training set as the labeled data set, and a unified random seed is used to ensure that each method uses the same labeled data set.

[0177] (2) Experimental details:

[0178] To ensure a fair comparison with existing research results, the method of the present invention and all the comparative methods use U-Net as the segmentation model. All evaluation methods are implemented using the PyTorch framework and experiments are conducted on a workstation equipped with an NVIDIA 4090 GPU. Each experiment is trained for 300 epochs, the batch size is set to 24, the optimizer is Adam, and the initial learning rate is 0.001.

[0179] (3) Evaluation indicators:

[0180] The present invention uses three evaluation indicators to evaluate the segmentation performance: Dice Coefficient, Intersection over Union (IoU) and 95th Percentile of the Hausdorff Distance (HD95). Among them, Dice and IoU are used to evaluate the accuracy of segmentation, while HD95 is used to evaluate the boundary segmentation performance.

[0181] (4) Comparison method:

[0182] This paper selects 7 advanced semi-supervised learning methods for comparison, namely: Average Teacher Model (MT), Uncertainty-Aware Average Teacher Model (UA-MT), Adversarial Entropy Minimization (ADVENT), Interpolation Consistency Training (ICT), Fixed Threshold Method (FixMatch), Cross Pseudo-Supervision (CPS) and Fixed Threshold Method for Dual-Stream Perturbation (UniMatch). In addition, the performance of U-Net in a purely supervised learning environment is also evaluated. This includes U-Net-All trained with complete labeled data, and U-Net-L trained with only partial labeled data. In theory, U-Net-All can represent the upper limit of the performance of semi-supervised learning methods, while U-Net-L represents the lower limit of performance.

[0183] (5) Experimental results:

[0184] Table 1 shows the quantitative evaluation results of the ACDC dataset. The present invention shows significant improvements in 5%, 10% and 20% data partitions. For example, compared with the fixed threshold-based method UniMatch, PPE improves Dice from 88.45% to 90.08% when using 5% labeled data, improves IoU from 80.02% to 82.61%, and reduces HD95 from 1.38 to 0.84. It is worth noting that when using 20% ​​labeled data, PPE's Dice reaches 91.42%, which is close to the upper limit of 91.57% for full supervision. Figure 4 (a) shows the comparison results of PPE and supervised methods under different label ratios.

[0185] Table 2 shows the quantitative results of PROMISE12 dataset. Figure 4 As shown in (b), even with only 5% labeled data, significant segmentation performance can be achieved. Specifically, with only 5% labeled data, PPE significantly outperforms the fully supervised U-Net-L, with a 65.67% increase in Dice, a 57.88% increase in IoU, and a 66.31 decrease in HD95. These results verify the effectiveness of using unlabeled data to improve segmentation performance, indicating that the present invention is still effective in the context of limited labeled data.

[0186] (6) Result visualization:

[0187] Figure 5 The first two rows of Figure 3 show the segmentation visualization results of the ACDC dataset. It can be observed that all methods have achieved satisfactory results in segmenting the central region (especially the left ventricle). It is worth noting that the proposed method shows a large improvement in segmenting the peripheral region (especially the right ventricle). Although the comparison method UniMatch performs well in overall segmentation performance, it still has shortcomings in handling boundary areas.

[0188] Figure 5 The last two rows show the segmentation visualization results of the PROMISE12 dataset. It can be seen that when only 5% of the labeled data is used, the segmentation effect of most of the comparison methods (such as MT, ICT, FixMatch and CPS) is not ideal. However, the present invention has a high match with the ground truth while maintaining the integrity of the segmentation target.

[0189] Table 1 Quantitative evaluation results of ACDC dataset

[0190]

[0191] Table 2 Quantitative evaluation results of PROMISE12 dataset

[0192]

[0193] The above description is only an embodiment of the present invention and is not intended to limit the present invention.

[0194] In summary, the embodiments of the present invention are different from the existing pseudo-labeling methods that use fixed thresholds. They propose an adaptive anchor threshold mechanism (SAT), which can dynamically adjust the threshold according to the model learning state reflected by the anchor pixels, thereby more effectively generating reliable pixel-level pseudo-labels.

[0195] In this paper, we propose a progressive pseudo-label enhancement framework (PPE) that combines pseudo-labels with consistency regularization to effectively utilize unreliable pixels whose confidence is below an adaptive threshold.

[0196] The present invention can not only utilize more reliable pixels, but also improve the training effect of the model in semi-supervised medical image segmentation by optimizing the use of unreliable pixels, especially when dealing with challenging areas such as boundary areas, thereby improving the segmentation performance of the model.

[0197] Those skilled in the art can understand that the accompanying drawings are only schematic diagrams of an embodiment, and the modules or processes in the accompanying drawings are not necessarily required to implement the present invention.

[0198] It can be known from the description of the above implementation methods that those skilled in the art can clearly understand that the present invention can be implemented by means of software plus a necessary general hardware platform. Based on such an understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which can be stored in a storage medium such as ROM / RAM, a magnetic disk, an optical disk, etc., and includes a number of instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in the various embodiments of the present invention or certain parts of the embodiments.

[0199] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the device or system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment. The device and system embodiments described above are merely schematic, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0200] The above is only a preferred specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any changes or substitutions that can be easily thought of by a person skilled in the art within the technical scope disclosed by the present invention should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.

Claims

1. A semi-supervised medical image segmentation method based on progressive pseudo-label enhancement, characterized in that: include: Annotate the original medical image dataset, divide the annotated original medical image dataset into labeled dataset and unlabeled dataset, and use weak enhancement and strong enhancement to enhance the unlabeled data; A progressive pseudo-label enhancement framework PPE network model is constructed, the PPE network model includes a student model and a teacher model, the PPE network model is iteratively trained using the labeled data set and the unlabeled data set, in each iteration, an adaptive anchor threshold SAT mechanism is used to generate an adaptive threshold, and the adaptive threshold mechanism is used to filter out pseudo-labels in the student model and the teacher model that are below a certain confidence range, so as to obtain a trained student model and a teacher model; The preprocessed medical image is input into a trained student model, and the trained student model outputs an image category segmentation result of the medical image.

2. The method according to claim 1, characterized in that The step of labeling the original medical image dataset and dividing the labeled original medical image dataset into a labeled dataset and an unlabeled dataset includes: Collect the original medical image data, perform minimum-maximum normalization on the original medical images, convert the three-dimensional medical images into two-dimensional slice images, crop all images to a uniform size of 256×256, randomly select some images for pixel-level annotation, and generate a labeled dataset D L , the unlabeled images constitute the unlabeled dataset D U , in semi-supervised learning D U The amount of data in is greater than D L The amount of data in .

3. The method according to claim 2, characterized in that The method of using weak enhancement and strong enhancement to enhance the unlabeled data includes: Use weak and strong augmentation to train unlabeled data D U Data enhancement is performed. The specific methods of weak enhancement include: flipping and rotating the image with a probability of 50%, and the specific methods of strong enhancement include: performing a Dropout operation on the features of the model; performing a CutMix operation on the image, that is, cutting an area of ​​an image and replacing it with the corresponding area of ​​another image.

4. The method according to claim 1, 2 or 3, characterized in that: The PPE network model of the progressive pseudo-label enhancement framework is constructed, and the PPE network model includes a student model and a teacher model, including: Construct a PPE network model, which adopts a teacher-student structure and includes a student model and teacher model Two models, the student model and the teacher model have the same structure but different initializations, and the student model and the teacher model generate pseudo labels through weakly enhanced images. Using the labeled dataset D L Train the student model, whose parameters are updated by back propagation, using weakly enhanced unlabeled data D U The student model is trained, and the student model and the teacher model output model prediction probability maps, confidence maps and pseudo labels. After each iterative training, the teacher model updates parameters from the student model through an exponential moving average algorithm EMA with a smoothing factor β.

5. The method according to claim 4, characterized in that The method of iteratively training the PPE network model using the labeled data set and the unlabeled data set, generating an adaptive threshold using an adaptive anchor threshold SAT mechanism in each iteration, filtering out pseudo labels below a certain confidence range in the student model and the teacher model using the adaptive threshold mechanism, iteratively updating the parameters of the student model and the teacher model, and obtaining a trained student model and teacher model, includes: In each iterative training of the PPE network model, the SAT mechanism is used to generate an adaptive threshold, the confidence of the anchor pixel is calculated according to the model prediction probability map and the confidence map output by the teacher model, the learning state of the PPE network model is estimated by the confidence of the anchor pixel, and the learning state w of the PPE network model is estimated according to the average confidence of all anchor pixels in a batch. t , expressed as: Where t represents the number of iterations, and B represents the batch size in each iteration, that is, the number of samples used in each iteration; The estimation of the model learning state is used as a weighted function of the threshold to generate an adaptive threshold for each training iteration. It is expressed as: in is the predefined maximum threshold; Define the lower threshold of the adaptive anchor point Where C represents the number of categories, The threshold locking strategy is expressed as follows: w t ―u t >ξ#(5) Among them, u t Indicates that all w in the first t iterations t The average value of ξ is a minimum value. If this condition is met, the threshold value increases with w t changes, otherwise locked in Update u using the EMA method with a smoothing factor α t , expressed as: you t =au t―1 +(1-α)u t #(6) Where α = 0.9; The SAT mechanism is expressed as follows: The student model only retains the values ​​greater than its adaptive threshold. The teacher model only retains pseudo labels greater than its adaptive threshold The pseudo labels of the student model are obtained by using a filtering mask to retain some pixels in the prediction of the student model. The teacher model also retains the predicted values ​​of the pixels at the corresponding positions. The parameters of the student model and the teacher model are iteratively updated to obtain the trained student model and the teacher model.

6. The method according to claim 5, characterized in that The iterative updating of the parameters of the student model and the teacher model to obtain the trained student model and the teacher model includes: The student model receives data from the labeled dataset and generates a prediction P. The student model receives weakly enhanced images from the unlabeled dataset and generates a prediction Pseudo Labels The student model receives strongly augmented images from the unlabeled dataset and generates predictions The teacher model receives weakly augmented images from the unlabeled dataset and generates predictions Pseudo Labels For labeled data, perform supervised training with supervised loss It is expressed as: Where B represents the batch size in each iteration, that is, the number of samples used in each iteration, P represents the model prediction map, and Y represents the pixel-level label. represents the cross entropy loss, represents Dice loss; Using pseudo labels generated by the student model To guide the prediction of the student model Forming pseudo-label loss It is expressed as: in represents the confidence map of the model predictions, Represents a filter mask, only those that meet the conditions The loss calculation will be performed only for the pixels; Generate pseudo labels from another perspective using collaborative teacher model The collaborative average teacher model shares the same structure as the student model. The collaborative average teacher model updates parameters from the student model via EMA, which can be expressed as: Among them, β = 0.99, represents the parameters of the collaborative average teacher model, Represents the parameters of the student model. Collaborative pseudo-label loss between the pseudo-labels produced by the collaboratively averaged teacher model and the predictions of the student model It is expressed as: Consistency Regularization Loss It is expressed as: in represents the mean square error loss. The total loss of the PPE network model is expressed as follows: Among them, λ1=0.5, λ2=0.4, λ3=0.

1.

7. The method according to claim 1, characterized in that The step of inputting the preprocessed medical image into the trained student model, wherein the trained student model outputs the image category segmentation result of the medical image, comprises: Perform minimum-maximum normalization on the medical image, adjust the image size to 256×256, slice the three-dimensional medical image into a two-dimensional image, and obtain the preprocessed medical image; Use PyTorch to load the trained student model, input the preprocessed medical image into the trained student model, assign the corresponding category to each pixel of the medical image based on the probability map generated by the student model, generate a segmentation mask, extract contours based on the segmentation mask, and superimpose the extracted contours on the original medical image to generate a visual image category segmentation result.

Images