Semi-supervised medical image semantic segmentation method based on dual-uncertainty guidance

Abstract

The invention discloses a semi-supervised medical image semantic segmentation method based on dual-uncertainty guidance. The method comprises the following steps: 1, constructing a semi-supervised medical image semantic segmentation model STP-DUDG; 2, performing preprocessing and data division on the medical images in the medical image data set of different sources, dividing the images in the training set into annotation data X1 and non-annotation data Xu, and performing pre-enhancement to form a mixed input image as two groups of training images Xa and Xb which are input in parallel; 3, a semi-supervised medical image semantic segmentation model STP-DUDG is trained; 4, the performance of the STP-DUDG is evaluated, and parameters are optimized; and 5, inputting a medical image to be segmented into the trained segmentation model STP-DUDG, and outputting a segmentation result. According to the method, the information diversity and stability between teacher and student models are enhanced, deep exploration of feature space is promoted, meanwhile, the adaptability to difficult pixels and regions is improved, and the discrimination ability to complex regions is enhanced.

Description

Technical Field

[0001] This invention belongs to the fields of image processing and computer vision, specifically relating to a semi-supervised medical image semantic segmentation method based on dual uncertainty guidance. Background Technology

[0002] Medical image segmentation is a core task in medical image analysis, aiming to accurately segment regions of interest from medical images. Precise and automatic segmentation of human anatomical structures or lesions is crucial for applications such as image-guided intervention, radiotherapy, and computer-aided diagnosis. However, the blurred boundaries between different organs and tissues in medical images, along with significant variations in the shape and scale of lesion regions, make efficient segmentation difficult. Furthermore, the high cost and professional barriers of manual annotation of medical images severely restrict its clinical application and are highly susceptible to subjective influences. Therefore, achieving high-precision, low-cost medical image segmentation has always been a key research focus and hot topic in the field of intelligent healthcare. Currently, with the continuous development of artificial intelligence and the advancement of medical technology, deep learning-based medical image segmentation methods have become a research hotspot, capable of capturing deep abstract features in medical images, thereby improving the accuracy of image segmentation.

[0003] Traditional deep learning methods for medical image segmentation mainly include those based on Convolutional Neural Networks (CNNs) and Transformers. CNNs typically rely on fixed-size convolutional kernels for feature extraction, thus limiting their receptive field to local regions. This makes CNNs strong at extracting local details, but they are insufficient in capturing global contextual information and modeling spatial location information in medical images. This limitation is particularly pronounced when dealing with targets with significant spatial variations. To alleviate the problem of limited receptive field, researchers have introduced dilated convolutions to expand the receptive field without significantly increasing computation, thereby enhancing the model's perception of global information. However, dilated convolutions still lack the ability to explicitly model global spatial structure, and their performance improvement is limited when dealing with structurally complex and semantically rich medical images. Unlike CNNs, Transformer models introduce relative position encoding through a self-attention mechanism, which can more effectively model the spatial relationships between different regions in an image and significantly improve the ability to express long-range dependencies. However, at the same time, Transformers face the problem of high computational cost when processing high-resolution medical images, and the efficiency of its global information modeling remains a challenge that urgently needs to be addressed.

[0004] Traditional deep learning methods rely on large amounts of labeled data, but the high cost and professional barriers of medical data labeling severely restrict its clinical application. To alleviate the bottleneck caused by insufficient labeled data, semi-supervised learning (SSL), a rapidly developing research direction in recent years, provides a new solution for medical image analysis. This method trains the model by jointly using a small number of labeled samples and a large number of unlabeled samples, fully mining the potential structural information in the unlabeled data, thereby reducing the dependence on large amounts of manual annotation while maintaining or even improving segmentation performance. Thanks to its high data utilization efficiency and low training cost, the application of semi-supervised learning in medical image segmentation has received increasing attention and has become one of the current research hotspots. Especially in real clinical environments, SSL provides strong technical support for improving the deployability and generalization ability of models, opening up new paths for the development of intelligent healthcare.

[0005] However, current semi-supervised medical image segmentation methods still face two challenges: First, the coupling between teacher and student models leads to inefficient learning. Teacher-student models are a mainstream architecture in semi-supervised learning, where the teacher model provides learnable supervisory signals to the student model. However, updating a single-teacher model via exponential moving average (EMA) can cause rapid coupling between the parameters of the teacher and student models, limiting the teacher model's ability to transfer additional knowledge to the student. Although ensemble teacher methods alleviate this problem through multi-model collaboration, their reliance on single-student updates makes it difficult to maintain the necessary diversity among teachers, resulting in inefficient knowledge transfer. Second, there is a lack of targeted processing for complex regions in medical images. The boundaries between different organs and tissues in medical images are blurred, and the shape and scale of lesion areas vary greatly, making it difficult for models to effectively capture the features of complex regions in medical images. Most existing medical image segmentation networks perform well when handling simple target segmentation. However, due to a lack of additional attention to difficult and complex regions, their performance is poor when facing challenging complex regions such as blurred edges, overlapping tissues, or occlusion. Summary of the Invention

[0006] The purpose of this invention is to provide a semi-supervised medical image semantic segmentation method based on dual uncertainty guidance, which enhances the information diversity and stability between teacher and student models, promotes in-depth exploration of the feature space, improves adaptability to difficult pixels and regions, and strengthens the ability to discriminate complex regions.

[0007] To achieve the above objectives, the present invention provides the following technical solution:

[0008] A semi-supervised medical image semantic segmentation method based on dual uncertainty guidance includes the following steps:

[0009] Step 1: Construct a semi-supervised medical image semantic segmentation model, STP-DUDG;

[0010] The semi-supervised medical image semantic segmentation model STP-DUDG includes a stochastic teacher framework (STP), a cross-scale dual uncertainty guided learning (DUG) module, and a divergence guided learning (DG) module. The STP is constructed based on a dual student model and a stochastic teacher model, both of which are Unet networks.

[0011] Step 2: Preprocess the medical images from different medical image datasets, dividing them into training, test, and validation sets. The images in the training set are further divided into labeled data X. l And unlabeled data X u The training set data is pre-augmented to form two sets of training images X, which are used as parallel inputs by a mixed input image. a X b ;

[0012] Step 3: Train the semi-supervised medical image semantic segmentation model STP-DUDG;

[0013] Step 3.1: Transfer the unlabeled data X u Input the random teacher model in STP, generate pseudo-labels, and compare them with the labeled data X. l The real labels are combined into a hybrid label Y a Y b ;

[0014] Step 3.2: Combine the two sets of training images X a X b Simultaneously, student model 1 and student model 2 are input to obtain corresponding prediction results. The DUG module is used to perform uncertainty analysis on the predictions of student model 1 and student model 2 at both the pixel and region scales to guide the weight mapping of different pixels and guide the loss calculation. Finally, the region uncertainty is used to guide the construction of targeted new samples X. á and

[0015] Step 3.3: Use the DG module to find the prediction discrepancies between student model 1 and student model 2 on the same samples, and use them as the focus of optimization to guide the student model to perform additional optimization on the discrepancy pixels and calculate the loss of the discrepancy region.

[0016] Step 3.4: Calculate the total loss for Student Model 1 and Student Model 2, and independently optimize and update the parameters;

[0017] Step 4: Evaluate the performance of the segmentation model STP-DUDG using the validation set and optimize the parameters;

[0018] Step 5: Input the medical image to be segmented into the trained segmentation model STP-DUDG and output the segmentation result.

[0019] Furthermore, the specific process of step 2 includes:

[0020] Step 2.1: Cropping and partitioning medical images from different sources of medical image datasets;

[0021] Step 2.2: Analyze the labeled data X in the training set. l And unlabeled data X u Perform copy-paste data augmentation to create a mixed input image;

[0022] The training set D contains N labeled data points and M unlabeled data points, where N << M, and D is derived from the labeled dataset. and unlabeled datasets Composition, in which This represents the i-th labeled image. This indicates its corresponding label. This represents the i-th unlabeled image;

[0023] For a set of labeled data with batch size B, number of categories C, height H and width W respectively. and a set of unlabeled data Generate mixed samples:

[0024]

[0025] in, represents a randomly generated zero-center mask, and ⊙ represents pixel-wise dot product;

[0026] We obtained new data by mixing the two sets of labeled and unlabeled data. and

[0027] Furthermore, in step 3.1, the mixed label Y a Y b It is obtained through the following steps:

[0028] The STP framework includes pre-trained alternative teacher models. With independently updated student models Forming a bond between teachers and students and Introduce a random activation strategy in STP: randomly select teachers in each iteration. Only active teachers are allowed to interact with their associated student models: α is a hyperparameter between 0 and 1, representing the weights of the teacher model parameters. This represents the parameters of the i-th teacher model. This represents the parameters of the i-th student model;

[0029] Activate teachers For unlabeled X u Generate pseudo tags Constructing a supervision signal using a CP hybrid strategy:

[0030]

[0031] in, Y represents a randomly generated zero-center mask. l Y represents the true label of the labeled data. a and Y b These represent different supervision signals used to supervise the training image X. a and X b .

[0032] Furthermore, the specific content of step 3.2 includes:

[0033] Step 3.2.1, for image X a and X b Two student models in STP and For X respectively a and X b Prediction is obtained These represent the i-th student model's response to the input image X. a and X b The prediction results are then mapped to the class space to obtain... and

[0034]

[0035] in, and This means that for each pixel x i (1≤i≤B*H*W) is the probability that it belongs to category c (1≤c≤C);

[0036] Step 3.2.2: Calculate the uncertainty assessment:

[0037]

[0038] in, and Representing sample X respectively a and X b The prediction uncertainty diagram;

[0039] Step 3.2.3: At the pixel level, combining the uncertainty guidance map, adaptively assign different levels of attention intensity to pixel regions of varying difficulty. The specific process is as follows:

[0040]

[0041] Among them, NE [a / b] (x) represents the uncertainty graph after normalization. U represents the uncertainty threshold used to distinguish between difficult and simple pixels, t and T represent the current iteration number and the total number of iterations, respectively, and τ represents the initial threshold. [a / b] (x) represents the original uncertainty graph, where x is a pixel location in the image, min(U [a / b] (x)) and max(U [a / b] (x) represents the uncertainty graph U. [a / b] The minimum and maximum values ​​of all pixel values ​​in (x), W l and W h These represent the lower and upper limits of the weight, respectively. X represents a or X b The pixel-by-pixel attention guidance map is used to guide the model to focus on key regions during the prediction process and to guide the calculation of the loss between the predicted value and the supervision signal, as described below:

[0042]

[0043] in, Let Y represent the loss function of the i-th student model in the segmentation task, corresponding to Y respectively. a and Y b Two different monitoring signals; L d&c It represents a combination of Dice loss and cross-entropy loss: L d&c =0.5×(L) ce +L dice ), L ce L represents the cross-entropy loss. dice This represents the Dice loss; η and β are two hyperparameters used to control the ratio of the losses before and after the two terms.

[0044] Step 3.2.4: Convert the pixel uncertainty map into a region uncertainty map:

[0045]

[0046] in, and Representing sample X respectively a and X b Patch-level prediction uncertainty, Ph and P w These represent the height and width of the patch area, respectively. This represents the index of the patch in the uncertainty graph;

[0047] Step 3.2.5: Guide patch swapping through regional uncertainty, and apply sample augmentation at the patch level:

[0048]

[0049] Among them, X á and These are new samples generated through a cross-image patch exchange strategy guided by patch uncertainty. This represents the number of swapped patches, defined as a decreasing variable that changes with the number of iterations.

[0050] Step 3.2.6: Transfer the original monitoring signal Y a and Y b Perform the same guide patch swapping to generate the supervision signal Y corresponding to the new sample. á and

[0051]

[0052] After guided enhancement, a new sample X is obtained. á and and X a and X b The new sample X is fed into the network in parallel for training. á and Having the same loss term:

[0053]

[0054] Furthermore, the specific content of step 3.3 includes:

[0055] First, argmax is applied to the prediction results obtained from step 3.2. and In the process, the final predicted label is obtained. and The prediction discrepancy of the student model can be expressed as:

[0056]

[0057] in This indicates the XOR operation. express and The region of absolute class divergence in predictions for the same sample;

[0058] Next, students will undergo additional learning to predict absolute class divergence regions, with the following specific loss:

[0059]

[0060] Among them, L mse Indicates MSE loss.

[0061] For the newly generated sample X á and Similarly, calculate the divergence-driven loss:

[0062]

[0063] Furthermore, the loss function for the total loss in step 3.4 is:

[0064]

[0065] Where ψ and λ are hyperparameters used to control the original sample X a and X b Predictive loss and guided augmentation of new sample X á and The balance between predicted losses.

[0066] Compared with the prior art, the present invention has the following beneficial effects:

[0067] (1) This invention proposes a Stochastic Teacher Strategy (STP): To enhance the information differentiation between the teacher model and the student model, a random teacher selection method is designed to supervise dual-student learning. Specifically, in each training iteration, a pre-trained reliable teacher model is randomly selected to generate high-quality supervision signals, and learning is conducted through the collaboration of the dual-student models. This random teacher selection strategy not only alleviates the problem of excessively rapid coupling between the teacher and student models, but also fully utilizes the perspective diversity of the teacher model, preventing the model from learning towards specific biases, thereby significantly improving the network model's utilization of rich semantic information.

[0068] (2) This invention proposes a cross-scale dual uncertainty guided learning (DUG): To generate high-quality training samples and focus on complex regions such as boundaries and lesions in medical images, a dual uncertainty guided strategy for pixels and regions is designed. Specifically, in the pixel dimension, the attention level of pixels is adaptively adjusted according to the uncertainty of pixels; in the region dimension, samples with difficult regions that match the model prediction are guided to be generated through cross-image patch exchange of region uncertainty. Based on the model prediction, the attention level is dynamically adjusted for different pixel regions and targeted learning samples are generated, improving the model's adaptability to difficult pixels and regions.

[0069] (3) This invention proposes a divergence-guided learning (DG): In order to better utilize the advantages of dual-student collaborative training, the prediction divergence regions are obtained by comparing the prediction results between student models, and then additional guided learning is performed to improve the semantic understanding ability of the model when dealing with these learning difficulties. Divergence-guided learning is introduced to guide the student model to learn the prediction category divergence in complex regions, and further correct potential cognitive biases. Attached Figure Description

[0070] Figure 1 This is the overall flowchart of the present invention;

[0071] Figure 2 This is the STP architecture diagram of the present invention;

[0072] Figure 3 This is a diagram of the DUG (pixel) module of the present invention;

[0073] Figure 4 This is a diagram of the DUG (patch) module of the present invention;

[0074] Figure 5 This is a diagram of the DG module of the present invention;

[0075] Figure 6 This is a visual comparison of the segmentation results of the present invention and existing mainstream methods on the ACDC dataset;

[0076] Figure 7 This is a visual comparison of the segmentation results of the present invention and existing mainstream methods on the ISIC2018 dataset and the MoNuSeg dataset. Detailed Implementation

[0077] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the present invention will be described in detail below with reference to the accompanying drawings and specific embodiments.

[0078] like Figure 1As shown in the figure, the specific steps of the semi-supervised medical image semantic segmentation method based on dual uncertainty guidance described in this embodiment are as follows:

[0079] Step 1: Construct a semi-supervised medical image semantic segmentation model STP-DUDG

[0080] The semi-supervised medical image semantic segmentation model STP-DUDG comprises a randomized teacher framework (STP), a cross-scale dual uncertainty guided learning (DUG) module, and a divergence guided learning (DG) module. STP is constructed based on a dual-student model (student model 1 and student model 2) and a randomized teacher model, both of which are Unet networks. The DUG module guides the model at both the pixel and region scales based on uncertainty information, dynamically adjusting the level of attention for different pixel regions and guiding the generation of targeted learning samples, thereby improving the model's perception ability in key regions and its ability to learn fine-grained features. The DG module, by mining the prediction divergence between the dual-student models, guides the model to focus on regions with significant prediction differences, effectively enhancing its ability to recognize complex structures and model boundaries, thus further improving overall segmentation performance. These three modules work together to effectively improve the segmentation accuracy and generalization ability of medical images.

[0081] Step 2: Preprocess the medical images from different medical image datasets, dividing them into training, test, and validation sets. The images in the training set are further divided into labeled data X. l And unlabeled data X u The training set data is pre-augmented to form two sets of training images X, which are used as parallel inputs by a mixed input image. a X b

[0082] First, the image size is standardized by random cropping to meet the size requirements of the network input. Then, the image data is normalized to make its distribution more standardized and improve the training stability of the model. Finally, the data is divided into training set, validation set and test set according to a certain ratio to ensure the reasonable distribution and comprehensive coverage of the data during the training process, providing a reliable foundation for the subsequent training, validation and performance evaluation of the model.

[0083] Step 2.1: Cropping and partitioning medical images from different medical image datasets.

[0084] This invention uses multiple publicly available medical image segmentation datasets, including ACDC, ISIC2018, and MoNuSeg, which respectively cover cardiac MRI, skin lesions, and multi-organ cell nucleus images, and have the characteristics of strong representativeness and structural diversity.

[0085] The ACDC dataset contains dynamic magnetic resonance imaging (MRI) images of the heart from 100 patients and their corresponding ground truth labels, covering three anatomical structures: right ventricle, left ventricle, and myocardium. Images from 70 patients were used for training, 10 for validation, and the remaining 20 for testing. During training, the 3D scan images were converted into 2D slices and uniformly resized to 256×256 pixels to fit the input size requirements of the model.

[0086] The ISIC2018 dataset originates from the 2018 International Skin Image Collaboration (ISIC-2018) Skin Lesion Segmentation Challenge. It contains 2,594 training images and 100 validation images, covering various skin lesion types and resolutions. Following common settings, 10% (259 images) and 20% (519 images) of the training set were randomly selected as labeled data, while the remaining images were used as unlabeled data for semi-supervised training.

[0087] The MoNuSeg dataset originates from the 2018 Multi-Organ Nucleus Segmentation (MoNuSeg) Challenge and contains 30 training images and 14 validation images. The images are H&E-stained tissue sections from different organs, where the contrast between the target cell nuclei and the background tissue is low, making segmentation challenging. The original images have a resolution of 1000×1000 pixels. Before training, they are uniformly cropped into non-overlapping 250×250 pixel sub-images and then uniformly adjusted to 256×256 pixels. Following existing methods, one image (corresponding to 16 sub-images) and two images (corresponding to 32 sub-images) are randomly selected as labeled samples, while the rest are used as unlabeled samples for training.

[0088] Step 2.2: Analyze the labeled data X in the training set. l And unlabeled data X u Perform copy-paste data augmentation to create a mixed input image.

[0089] The training set D contains N labeled data points and M unlabeled data points, where N << M, and D is derived from the labeled dataset. and unlabeled datasets Composition, in which This represents the i-th labeled image. This indicates its corresponding label. This represents the i-th unlabeled image.

[0090] In this invention, copy-paste data augmentation is first performed between training images. This is done for a set of labeled data with batch size B, number of classes C, and height and width H and W, respectively. and a set of unlabeled data The generated mixed samples are shown in equation (1):

[0091]

[0092] in, represents a randomly generated zero-center mask, and ⊙ represents pixel-wise dot product.

[0093] We obtained new data by mixing the two sets of labeled and unlabeled data. and Two sets of training images were used as parallel inputs to the STP-DUDG model.

[0094] Step 3: Train the semi-supervised medical image semantic segmentation model STP-DUDG

[0095] Step 3.1: Transfer the unlabeled data X u Input the random teacher model in STP, generate pseudo-labels, and compare them with the labeled data X. l The real labels are combined into a hybrid label Y a Y b

[0096] To address the core issues of model coupling and cognitive bias in existing semi-supervised methods, this invention proposes an STP architecture, such as... Figure 2 As shown, this framework achieves synergistic optimization of model diversity and training stability through a dynamic decoupling mechanism.

[0097] Specifically, the STP framework includes pre-trained candidate teacher models. With independently updated student models Forming a bond between teachers and students and Unlike the unidirectional parameter passing of traditional multi-teacher frameworks, STP introduces a random activation strategy: teachers are randomly selected in each iteration. Only the activated teacher is allowed to interact with the student model it is bound to, as shown in Equation (2).

[0098]

[0099] Here, α is a hyperparameter between 0 and 1, representing the weights of the teacher model parameters. This represents the parameters of the i-th teacher model. This represents the parameters of the i-th student model.

[0100] This peer-to-peer update mechanism, by physically isolating gradient propagation paths between different teacher-student pairs, greatly alleviates the parameter coupling problem, such as... Figure 2As shown, this method enables the teacher model to have a temporally differentiated representation, maintaining the necessary diversity between teaching sessions and improving the generalization ability of the student model. Furthermore, by adding random Gaussian noise only to samples between bound teacher-student pairs, this strategy allows the model to form complementary perturbation responses in the feature space. Bound students emphasize local texture robustness, while unbound students maintain global structural consistency. This is presented as a favorable method for injecting diversity into the network to address coupling issues. As positive feedback, the teacher model can then provide more refined supervisory signals to the student model.

[0101] Activate teachers For unlabeled X u Generate pseudo tags Constructing a supervision signal using a CP hybrid strategy:

[0102]

[0103] in, Y represents a randomly generated zero-center mask. l Y represents the true label of the labeled data. a and Y b These represent different supervision signals used to supervise the training image X. a and X b .

[0104] Step 3.2: Combine the two sets of training images X a X b Simultaneously, student model 1 and student model 2 are input to obtain corresponding prediction results. The DUG module is used to perform uncertainty analysis on the predictions of student model 1 and student model 2 at both the pixel and region scales to guide the weight mapping of different pixels and guide the loss calculation. Finally, the region uncertainty is used to guide the construction of targeted new samples X. á and

[0105] First, for image X a and X b Two student models in STP and For X respectively a and X b Prediction is obtained and These represent the i-th student model's response to the input image X. a and X b The prediction results are then mapped to the class space using formula (4) to obtain the results. and

[0106]

[0107] in, and This means that for each pixel x i (1≤i≤B*H*W) represents the probability of belonging to category c (1≤c≤C). These mapped predicted values ​​are the probability distribution of each pixel in each category.

[0108] Secondly, to increase the reliability of uncertainty assessment and reduce its randomness, an uncertainty graph is obtained by integrating the outputs of two collaboratively trained student models. This process can be described as follows:

[0109]

[0110] in, and Representing sample X respectively a and X b The prediction uncertainty map quantifies the ensemble uncertainty of two independently trained student models on the same sample. By using two uncertainty guidance maps, it is possible to reveal pixel regions of varying difficulty within the sample and the regions most likely to result in missegmentation.

[0111] Finally, at the pixel level, by combining the uncertainty guidance map, different levels of attention intensity are adaptively assigned to pixel regions of varying difficulty, thereby optimizing the learning process and improving segmentation accuracy. Figure 3 As shown. The specific process can be described as follows:

[0112]

[0113]

[0114] Among them, NE [a / b] (x) represents the uncertainty graph after normalization. U represents the uncertainty threshold used to distinguish between difficult and simple pixels, t and T represent the current iteration number and the total number of iterations, respectively, and τ represents the initial threshold. [a / b] (x) represents the original uncertainty graph, where x is a pixel location in the image, min(U [a / b] (x)) and max(U [a / b] (x) represents the uncertainty graph U. [a / b] The minimum and maximum values ​​of all pixel values ​​in (x), W l and W h These represent the lower and upper limits of the weight, respectively. X represents a or X bThe pixel-by-pixel attention guidance map is used to guide the model to focus on key regions during the prediction process and to guide the calculation of the loss between the predicted value and the supervision signal, as described below:

[0115]

[0116] in, Let Y represent the loss function of the i-th student model in the segmentation task, corresponding to Y respectively. a and Y b Two different monitoring signals; L d&c It represents a combination of Dice loss and cross-entropy loss: L d&c =0.5×(L) ce +L dice ), L ce L represents the cross-entropy loss. dice This represents the Dice loss. η and β are two hyperparameters used to control the ratio of the losses before and after the two terms.

[0117] Pixel weights with pixel uncertainty exceeding a threshold will smoothly approach higher weights W. h Pixels with pixel uncertainty below a threshold will have their weights shifted to lower weights W. l Because of the nature of the sigmoid function, the function values ​​on both sides of the threshold always change relatively drastically. Therefore, setting a dynamic threshold can make the pixels on both sides of the threshold show a more obvious difference in focus. This threshold size gradually increases with the number of training iterations. The strategy of dynamically increasing the uncertainty threshold is essentially a process of gradually changing the scope of focus. In the early stages of training, a large area of ​​high uncertainty is strictly focused on; in the middle stages, it is gradually compressed to a medium uncertainty area; and in the later stages, only extremely difficult areas with entropy values ​​still higher than the threshold are focused on. This approach can effectively avoid the model overfitting to simple regions in the later stages of training, thereby improving the model's generalization ability.

[0118] By employing an adaptive pixel attention strategy, the model can accurately handle image regions of varying complexity. Simple pixels are easier to predict, thus receiving relatively low attention and weight; while difficult pixels typically contain higher uncertainty or boundary information, requiring more attention and thus receiving greater weight. This differentiated learning strategy effectively allocates learning resources, improving the prediction accuracy and stability of challenging regions.

[0119] On the other hand, to leverage the dynamic information of model inference and achieve a high degree of matching between the data augmentation process and the model learning process, effectively guiding sample learning, a cross-image virtual difficult sample generation strategy based on patch uncertainty assessment is proposed. In this strategy, the model's inference results are used as guiding information to interchange easy and difficult patches across images, generating samples containing a large number of prediction-difficult regions for model training. The specific process can be described as follows:

[0120] First, convert the pixel uncertainty map into a region uncertainty map:

[0121]

[0122] in, and Representing sample X respectively a and X b Patch-level prediction uncertainty, P h and P w These represent the height and width of the patch area, respectively. This indicates the index of the patch in the uncertainty graph.

[0123] Secondly, patch swapping is guided by regional uncertainty, such as... Figure 4 As shown, to avoid excessive interference with the structural information of the image itself when performing data augmentation at the pixel level, we choose to discuss sample augmentation at the patch dimension. This can effectively reduce the potential negative impact of data augmentation. The specific operation is described below:

[0124]

[0125] in, and The new sample was generated through a cross-image patch exchange strategy guided by patch uncertainty. This represents the number of swapped patches, defined as a decreasing variable that changes with the number of iterations.

[0126] These new samples address the challenges encountered during model inference, effectively improving the model's adaptability and discrimination capabilities in difficult regions.

[0127] Finally, in order to effectively supervise the newly generated difficult samples, the original supervision signal Y is... a and Y b Perform the same guide patch swapping to generate the supervision signal Y corresponding to the new sample. á and

[0128]

[0129] It is worth noting that in this method, a new sample X is obtained after guided enhancement. á and and X a and X b They are fed into the network in parallel for training, and they all have the same loss term, as shown in the following formula:

[0130]

[0131] Step 3.3: Use the DG module to identify the prediction discrepancies between Student Model 1 and Student Model 2 on the same samples, and use these discrepancies as the focus of optimization. Guide the student models to perform additional optimization on the discrepancy pixels and calculate the loss in the discrepancy region.

[0132] In the STP framework of this invention, two student models can learn and infer from different perspectives. Since the two student models are optimized independently, they may exhibit significant prediction discrepancies when dealing with prediction challenges such as boundaries. In this case, if cross-supervision is used between the student models, it can lead to one-way false confidence in the student models when learning in areas of discrepancy, resulting in unclear model learning objectives.

[0133] To better leverage the synergistic effect of the dual-student model and fully utilize the potential complementarity between the models, a learning strategy guided by predicting divergence regions is proposed, such as... Figure 5 As shown, by comparing the prediction results of the two student models, the absolute class divergence regions predicted by the student models are found. These regions reveal the model's uncertainty regarding the samples and potential learning difficulties, guiding the student models to pay extra attention to these challenging regions instead of relying solely on the overall prediction of the samples, thereby enhancing the model's robustness and discriminative ability. The specific process can be described as follows:

[0134] First, argmax is applied to the prediction results obtained from step 3.2. and In the process, the final predicted label is obtained. The prediction discrepancy of the student model can be expressed as:

[0135]

[0136] in This indicates the XOR operation. express and The region of absolute class divergence for the same sample.

[0137] Next, students will undergo additional learning to predict absolute class divergence regions, with the following specific loss:

[0138]

[0139] Among them, L mse This represents the MSE loss. Furthermore, for the newly generated sample X... á and Similarly, calculate the divergence-driven loss:

[0140]

[0141] Step 3.4: Calculate the total loss for Student Model 1 and Student Model 2, and independently optimize and update the parameters.

[0142] In the method of this invention, the two students are updated independently, therefore their respective loss functions can be summarized as follows:

[0143]

[0144] Where ψ and λ are hyperparameters used to control the original sample X a and X b Predictive loss and guided augmentation of new sample X á and The balance between predicted losses.

[0145] Furthermore, during the inference phase, for a given image X test We selected the student model with the higher Dice score for inference, specifically: i∈{1,2}, where θ are the parameters of the better model. Of course, if computational cost and inference time are not considered during the inference phase, the option of using the student model for average ensemble inference is still retained:

[0146] Step 4: Evaluate the performance of the segmentation model STP-DUDG using the validation set and optimize the parameters.

[0147] Step 5: Input the medical image to be segmented into the trained segmentation model STP-DUDG, and output the segmentation result.

[0148] The effects of this invention can be further illustrated by the following experiments.

[0149] The STP-DUDG method of this invention is implemented in PyTorch, and all experiments were conducted on a server equipped with an NVIDIA GeForce RTX 3090 24GB graphics card. UNet was used as the segmentation network on the ACDC, ISIC-2018, and MoNuSeg datasets, with batch size, pre-training iterations, and self-training iterations set to 24 (half labeled data, half unlabeled data), 10k, and 30k, respectively. For the LA dataset, VNet was used as the segmentation network, with batch size, pre-training iterations, and self-training iterations set to 8 (half labeled data, half unlabeled data), 2k, and 15k, respectively. The SGD optimizer was used to train the model with a learning rate of 0.01, weight decay of 0.0001, and momentum of 0.9. Based on the best results on the validation set, the hyperparameters were set as follows: η = 0.5, β = 1.0, W... h =2.0, W l =0.5, P h =P w =16,N start =16, ψ=λ=0.5. Regarding hyperparameters in the experiment... The settings are based on the standards used in BCP.

[0150] Table 1 shows the average performance of the four-class segmentation task on the ACDC dataset with 5% and 10% labeling ratios. The algorithm of this invention exhibits best performance in both settings. Specifically, at a 5% labeling ratio, the method of this invention improves the Dice value by 2.34% and the Jaccard metric by 3.56% compared to BCP; at a 10% labeling ratio, the method of this invention achieves best performance across all four evaluation metrics, improving Dice by 1.29% and Jaccard by 1.97% compared to the recent work ABD. This result validates the effectiveness of the method of this invention in learning difficult pixels, particularly its focus on challenging regions, which helps improve the model's generalization ability and overall performance.

[0151] Tables 2 and 3 present the experimental results of the STP-DUDG method of this invention on the ISIC-2018 and MoNuSeg datasets, respectively. As can be seen from the results in Tables 2 and 3, the method of this invention generally outperforms other competing methods under all settings with different numbers of labeled images. Specifically, on the Skin Lesion dataset, especially with 5% labeled training data, the method of this invention achieves a 1.54% improvement in Dice compared to the recent work UFC, and a 1.34% improvement in Jaccard. On the MoNuSeg dataset, especially with 2 / 30 labeled training data, the method of this invention surpasses UFC in Dice value by 1.23%.

[0152] Table 1 compares the performance metrics of this invention with other state-of-the-art methods on the ACDC multi-organ segmentation dataset.

[0153]

[0154]

[0155] Table 2 compares the performance metrics of this invention with other advanced methods on the ISIC-2018 dataset.

[0156]

[0157]

[0158] Table 3 compares the performance metrics of this invention with other state-of-the-art methods on the MoNuSeg dataset.

[0159]

[0160]

[0161] Figure 6 This is a visual comparison of the segmentation results of the present invention and existing mainstream methods on the ACDC dataset. In the figure: a) represents the ground truth label; b) represents the segmentation result using the BCP method; c) represents the segmentation result using the ABD method; d) represents the segmentation result using the AD-MT method; e) represents the segmentation result using the PMT method; and f) represents the segmentation result using the method of the present invention. Figure 6 It can be seen that the STP-DUDG method of the present invention exhibits more refined results in the difficult-to-segment edge regions and easily mis-segmented regions in multi-organ segmentation.

[0162] Figure 7 This is a visualization comparison of the segmentation results of the present invention and existing mainstream methods on the ISIC2018 and MoNuSeg datasets. In the figure: a) is the ground truth label; b) is the segmentation result of the UA-MT method; c) is the segmentation result of the CPS method; d) is the segmentation result of the DTC method; e) is the segmentation result of the ASE-Net method; f) is the segmentation result of the MCF method; g) is the segmentation result of the DTC method; h) is the segmentation result of the BCP method; i) is the segmentation result of the UFC method; j) is the segmentation result of the present invention. Furthermore, to better display the segmentation results, the present invention uses different colors for visualization. Green and red pixels represent the predicted result and the ground truth label, respectively, while yellow pixels represent the overlapping area between the predicted and ground truth labels; from Figure 7 As can be seen from the present invention, the actual segmentation effect of the STP-DUDG method is closer to the real label, reducing incorrect segmentation and missed segmentation.

Claims

1. A semi-supervised medical image semantic segmentation method based on dual uncertainty guidance, characterized in that, Includes the following steps: Step 1: Construct a semi-supervised medical image semantic segmentation model, STP-DUDG; The semi-supervised medical image semantic segmentation model STP-DUDG includes a stochastic teacher framework (STP), a cross-scale dual uncertainty guided learning (DUG) module, and a divergence guided learning (DG) module. The STP is constructed based on a dual student model and a stochastic teacher model, both of which are Unet networks. Step 2: Preprocess the medical images from different medical image datasets, dividing them into training, test, and validation sets. The images in the training set are further divided into labeled data X. l And unlabeled data X u The training set data is pre-augmented to form two sets of training images X, which serve as parallel inputs, using a mixed input image. a X b ; Step 3: Train the semi-supervised medical image semantic segmentation model STP-DUDG; Step 3.1: Transfer the unlabeled data X u Input the random teacher model in STP, generate pseudo-labels, and compare them with the labeled data X. l The real labels are combined into a hybrid label Y a Y b ; Step 3.2: Combine the two sets of training images X a X b Simultaneously, student model 1 and student model 2 are input to obtain corresponding prediction results. The DUG module is used to perform uncertainty analysis on the predictions of student model 1 and student model 2 at both the pixel and region scales to guide the weight mapping of different pixels and guide the loss calculation. Furthermore, the region uncertainty is used to guide the construction of targeted new samples. and Step 3.3: Use the DG module to find the prediction discrepancies between student model 1 and student model 2 on the same samples, and use them as the focus of optimization to guide the student model to perform additional optimization on the discrepancy pixels and calculate the loss of the discrepancy region. Step 3.4: Calculate the total loss for Student Model 1 and Student Model 2, and independently optimize and update the parameters; Step 4: Evaluate the performance of the segmentation model STP-DUDG using the validation set and optimize the parameters; Step 5: Input the medical image to be segmented into the trained segmentation model STP-DUDG and output the segmentation result.

2. The semi-supervised medical image semantic segmentation method based on dual uncertainty guidance according to claim 1, characterized in that, The specific process of step 2 includes: Step 2.1: Cropping and partitioning medical images from different sources of medical image datasets; Step 2.2: Analyze the labeled data X in the training set. l And unlabeled data X u Perform copy-paste data augmentation to create a mixed input image; The training set D contains N labeled data points and M unlabeled data points, where N << M, and D is derived from the labeled dataset. and unlabeled datasets Composition, in which This represents the i-th labeled image. This indicates its corresponding label. This represents the i-th unlabeled image; For a set of labeled data with batch size B, number of categories C, height H and width W respectively. and a set of unlabeled data Generate mixed samples: in, represents a randomly generated zero-center mask, and ⊙ represents pixel-wise dot product; We obtained new data by mixing the two sets of labeled and unlabeled data. and 3. The semi-supervised medical image semantic segmentation method based on dual uncertainty guidance according to claim 2, characterized in that, In step 3.1, the mixed label Y a Y b It is obtained through the following steps: The STP framework includes pre-trained alternative teacher models. With independently updated student models Forming a bond between teachers and students and Introduce a random activation strategy in STP: randomly select teachers in each iteration. Only active teachers are allowed to interact with their associated student models: α = 0.99, where α is a hyperparameter between 0 and 1, representing the weights of the teacher model parameters. This represents the parameters of the i-th teacher model. This represents the parameters of the i-th student model; Activate teachers For unlabeled X u Generate pseudo tags Constructing a supervision signal using a CP hybrid strategy: in, Y represents a randomly generated zero-center mask. l Y represents the true label of the labeled data. a and Y b These represent different supervision signals used to supervise the training image X. a and X b .

4. The semi-supervised medical image semantic segmentation method based on dual uncertainty guidance according to claim 3, characterized in that, The specific content of step 3.2 includes: Step 3.2.1, for image X a and X b Two student models in STP and For X respectively a and X b Prediction is obtained and These represent the i-th student model's response to the input image X. a and X b The prediction results are then mapped to the class space to obtain... and in, and This means that for each pixel x i (1≤i≤B*H*W) is the probability that it belongs to category c (1≤c≤C); Step 3.2.2: Calculate the uncertainty assessment: in, and Representing sample X respectively a and X b The prediction uncertainty diagram; Step 3.2.3: At the pixel level, combining the uncertainty guidance map, adaptively assign different levels of attention intensity to pixel regions of varying difficulty. The specific process is as follows: Among them, NE [a / b] (x) represents the uncertainty graph after normalization. U represents the uncertainty threshold used to distinguish between difficult and simple pixels, t and T represent the current iteration number and the total number of iterations, respectively, and τ represents the initial threshold. [a / b] (x) represents the original uncertainty graph, where x is a pixel location in the image, min(U [a / b] (x)) and max(U [a / b] (x) represents the uncertainty graph U. [a / b] The minimum and maximum values ​​of all pixel values ​​in (x), W l and W h These represent the lower and upper limits of the weight, respectively. X represents a or X v The pixel-by-pixel attention guidance map is used to guide the model to focus on key regions during the prediction process and to guide the calculation of the loss between the predicted value and the supervision signal, as described below: in, Let Y represent the loss function of the i-th student model in the segmentation task, corresponding to Y respectively. a and Y b Two different monitoring signals; L d&c It consists of Dice loss and cross-entropy loss: L d&c =0.5×(L) ce +L dice ), L ce L represents the cross-entropy loss. dice This represents the Dice loss; η and β are two hyperparameters used to control the ratio of the losses before and after the two terms. Step 3.2.4: Convert the pixel uncertainty map into a region uncertainty map: in, and Representing sample X respectively a and X b Patch-level prediction uncertainty, P h and P w These represent the height and width of the patch area, respectively. This represents the index of the patch in the uncertainty graph; Step 3.2.5: Guide patch swapping through regional uncertainty, and apply sample augmentation at the patch level: in, and These are new samples generated through a cross-image patch exchange strategy guided by patch uncertainty. This represents the number of swapped patches, defined as a decreasing variable that changes with the number of iterations. Step 3.2.6: Transfer the original monitoring signal Y a and Y b Perform the same guide patch swapping to generate supervision signals corresponding to the new samples. and New samples were obtained after guided enhancement. and and X a and X b New samples are fed into the network in parallel for training. and Having the same loss term:

5. The semi-supervised medical image semantic segmentation method based on dual uncertainty guidance according to claim 4, characterized in that, The specific content of step 3.3 includes: First, argmax is applied to the prediction results obtained from step 3.

2. and In the process, the final predicted label is obtained. and The prediction discrepancy of the student model can be expressed as: in This represents the XOR operation. express and The region of absolute class divergence in predictions for the same sample; Next, students will undergo additional learning to predict absolute class divergence regions, with the following specific loss: Among them, L mse Indicates MSE loss. For newly generated samples and Similarly, calculate the divergence-driven loss:

6. The semi-supervised medical image semantic segmentation method based on dual uncertainty guidance according to claim 5, characterized in that, The loss function for the total loss in step 3.4 is: Where ψ and λ are hyperparameters used to control the original sample X a and X b Predicted loss and new samples after guidance enhancement and The balance between predicted losses.

Images