A semi-supervised segmentation method combining dual segmentation head frequency decoupling learning and entropy change pseudo label screening

Abstract

The application discloses a kind of semi-supervised segmentation methods of dual segmentation head frequency decoupling learning and entropy change pseudo label screening combination, it is applied to image processing field, for the problem that the quality of pseudo label generation is difficult to guarantee in existing semi-supervised semantic segmentation method, and model is single for enhanced disturbance response, it is difficult to distinguish global structure change and local detail change, leading to pseudo label error is high, unstable training;Student model of the present application adopts dual segmentation head structure, respectively by prototype prediction head and linear prediction head is formed, and realizes function division by frequency domain decomposition mechanism: prototype head receives the low-frequency component feature of encoder, linear head receives the high-frequency component feature of encoder, for unlabeled sample, after applying random enhancement, the class distribution of sample before and after enhancement is predicted by two segmentation heads of teacher model respectively, and the change ratio of average prediction entropy is calculated.Based on the ratio of entropy change before and after enhancement, the reaction of prototype head and linear head is integrated, and the prediction stability after enhancement is jointly evaluated.

Description

Technical Field

[0001] This invention belongs to the field of image processing, and specifically relates to a semi-supervised segmentation technique. Background Technology

[0002] In recent years, deep learning technology has made significant progress in medical image segmentation tasks, especially methods based on convolutional neural networks (CNNs), which have demonstrated excellent performance on various medical image datasets. However, these methods typically rely on large amounts of accurate pixel-level labeled data for supervised learning. To reduce the dependence on large amounts of labeled data, researchers have proposed various semi-supervised learning strategies that incorporate unlabeled data into the training process to improve the model's generalization ability and data utilization. The core of semi-supervised segmentation methods lies in how to effectively mine the potential information in unlabeled data and combine it with limited labeled samples to jointly optimize the model. However, medical images are characterized by modal diversity (such as MRI, CT, ultrasound, etc.), complex organ structures, and large differences in lesion morphology, making semi-supervised segmentation face greater challenges in this field. At the same time, the annotation process of medical images is not only time-consuming but also requires the participation of radiologists or other professionals, which greatly limits the efficiency and scale of obtaining high-quality labeled data.

[0003] Semi-supervised learning has achieved considerable success in the field of natural image processing, with applications such as consistency regularization, self-training, and contrastive learning. These ideas are increasingly being incorporated into medical image segmentation to address the shortage of high-quality labeled data. For example, using a teacher-student structure for consistency constraints or leveraging pseudo-labels to enhance model training have become mainstream design approaches for semi-supervised segmentation frameworks. Improving the reliability of pseudo-labels and guiding the model to learn valuable feature information from unlabeled samples are crucial for enhancing the effectiveness of semi-supervised segmentation.

[0004] In medical image segmentation tasks, the feature distribution between categories often exhibits high variability and ambiguity, especially in small sample or weakly supervised scenarios. To improve the model's discriminative ability, a class of segmentation methods based on prototype learning has emerged in recent years. These methods construct representative features (i.e., prototypes) for each category, guiding the model to align within the feature space, thereby achieving more robust pixel-level classification. A prototype is typically defined as the center vector of a category in the feature space, obtainable through the mean or weighted aggregation of features from labeled samples. In semi-supervised segmentation, introducing category prototypes allows for similarity calculation or matching between the pixel features of unlabeled samples and the prototypes, thus assisting in pseudo-label generation, feature alignment, or consistency modeling, significantly enhancing the supervisory signal for unlabeled data. Some works further propose dynamic prototype update mechanisms, continuously optimizing category prototypes during training to better reflect the current model's representation of the features for that category, effectively avoiding the bias caused by pseudo-label errors in prototype calculation.

[0005] In practical medical image segmentation tasks, data often originates from different hospitals, equipment, or imaging protocols, resulting in significant inter-domain differences between images. This causes models trained solely on the source domain (i.e., labeled data) to exhibit significantly reduced performance on the target domain (unlabeled data), severely impacting their generalization performance. To address this issue, researchers have begun introducing the concept of Domain Adaptation (DA) into semi-supervised segmentation. By bridging the distributional differences between the source and target domains, this improves the model's adaptability to unlabeled target domain data. In semi-supervised scenarios, a particularly common strategy is to combine self-training with domain adaptation: on the one hand, the model generates pseudo-labels on the target domain for self-supervision; on the other hand, alignment constraints, such as prototype alignment, are applied to the feature layer or output layer to reduce the source-target distribution gap.

[0006] Existing technology 1, "Semi-supervised Semantic Segmentation with Prototype-based Consistency Regulation (PbCR), a prototype-based consistency regularization mechanism," employs two segmentation heads: a prototype-based segmentation head and a linear projection-based segmentation head. The prototype-based segmentation head constructs semantic category prototypes in the feature space and matches these prototypes with features extracted from labeled images by the student model to obtain segmentation results. The linear projection-based segmentation head generates pixel-level predictions using a learnable linear classifier. Losses are calculated for both segmentation results of labeled images and the original image labels, and then backpropagated. Simultaneously, consistency regularization is applied to the two segmentation predictions of unlabeled images output by the student model and the segmentation predictions obtained from the teacher model's linear segmentation head to achieve stronger discriminative power and more robust pseudo-supervision.

[0007] This technique first uses labeled images to perform fully supervised pre-training on a model with a linear prediction head. Then, k-means clustering is used to cluster the image features extracted by the pre-trained model, obtaining initial prototypes. Afterwards, conventional segmentation prediction and loss calculation are performed on the labeled data. Unlabeled images are weakly enhanced through flipping, rotation, etc., and then subjected to a cut-mix operation to obtain X~u, which is input into the student model. Two different segmentation predictions are obtained for the same batch of unlabeled images using a prototype segmentation head based on the prototype and feature distance, and a linear segmentation head using linear projection. The teacher model inputs X~u and outputs the segmentation prediction from the preceding segmentation head as pseudo-labels, which are then used to calculate cross-entropy loss with the two predictions output by the student model. In each iteration, the model again uses the k-means algorithm to cluster the features extracted by the encoder, obtaining new prototypes and performing momentum updates.

[0008] The performance of existing technology PbCR is heavily reliant on pre-training, resulting in significant training overhead. This method only applies simple consistency constraints to the results of two different segmentation methods without exploring the relationship between them in greater depth. Furthermore, it ignores the impact of potential class imbalance in the dataset on the prototype formation and update process, thus limiting its ability to achieve more generalized performance.

[0009] Existing technology two, "Upcol: Uncertainty-informed prototype consistency learning for semi-supervised medical image segmentation," proposes a semi-supervised medical image segmentation framework based on Uncertainty-informed Prototype Consistency Learning (UPCoL). It aims to improve the model's ability to recognize complex medical structures by optimizing the distribution of the embedding space through the fusion of feature prototypes from labeled and unlabeled data. This method is built on a student-teacher framework, where the student model optimizes parameters through supervised learning, and the teacher model updates using exponential moving average (EMA) and generates pseudo-labels and uncertainty estimates. The core innovation lies in introducing an uncertainty-guided prototype fusion mechanism and consistency constraints on the feature space. The specific process is as follows:

[0010] First, the uncertainty assessment module integrates the prediction results of multiple classifiers (cross-entropy, focus loss, Dice loss, and IoU loss) to calculate the entropy value of each voxel to quantify its reliability. Voxels with high entropy values ​​indicate ambiguous predictions and their weights should be reduced; voxels with low entropy values ​​are considered reliable samples. Based on this, a reliability map is generated for weighted calculations in subsequent prototype fusion.

[0011] Secondly, prototype extraction and fusion are divided into two parts: labeled data and unlabeled data. For labeled data, the feature maps are directly masked average pooling using the real labels to obtain initial prototypes for each category. For unlabeled data, prototypes are extracted using attention-weighted pooling, combining features and reliability maps generated by the teacher model. The two types of prototypes are combined using a dynamic fusion strategy: a time-dependent Gaussian heating function is used to adjust the fusion ratio, initially prioritizing labeled prototypes and gradually increasing the weight of unlabeled prototypes until a balanced fusion is achieved. This strategy ensures the reliability of the prototypes in the initial stage while gradually introducing the diversity information from the unlabeled data.

[0012] Furthermore, prototype consistency learning constructs a consistency loss by calculating the cosine similarity between feature embeddings and fused prototypes. For labeled data, prototype predictions are required to be consistent with the true labels; for unlabeled data, a reliability graph-weighted constraint is used to ensure consistency between prototype predictions and the pseudo-labels of the teacher model. The final total loss combines the segmentation loss and the prototype consistency loss, improving model performance through end-to-end optimization. This process forces feature representations to be more compact among similar samples and more separable among dissimilar samples, thereby alleviating common problems in medical images such as large intra-class differences and blurred boundaries.

[0013] Furthermore, the paper proposes a nonlinear prototype update mechanism that ensures stable iteration of the prototype during training by gradually adjusting the fusion ratio. Compared with existing methods (such as updating labeled / unlabeled prototypes separately), the fused prototype can more comprehensively capture the data distribution and avoid information fragmentation.

[0014] The existing technique 2, which uses average pooling to obtain the prototype, is rather primitive. Due to the existence of intra-class differences, simply averaging all pixels or voxels will ignore these differences and reduce the generalization ability of the prototype, further leading to a decrease in the precision of model segmentation. At the same time, the high-entropy regions calculated by the reliability map do not necessarily represent non-target regions; they only indicate that the model has a low degree of fit to this region, and cannot truly solve the problems of intra-class differences and cross-domain inconsistencies. Summary of the Invention

[0015] To address the aforementioned technical challenges, this invention proposes a semi-supervised segmentation method that combines dual-segmentation head frequency decoupling learning with entropy change pseudo-label selection. This method systematically integrates frequency division of labor, enhanced response sensitivity, and a pseudo-label selection mechanism. While ensuring the diversity and stability of feature representation, it dynamically controls the quality of pseudo-label usage, thereby improving the performance and robustness of the semi-supervised semantic segmentation model in low-label scenarios.

[0016] The technical solution adopted in this invention is as follows: a semi-supervised segmentation method combining dual-segmentation head frequency decoupling learning and entropy change pseudo-label screening, including a student model and a teacher model. Both the student model and the teacher model adopt a dual-segmentation head structure, which specifically includes a prototype head and a linear head. The student model and the teacher model also include an encoder and a feature frequency domain separation module with the same structure. After the input data is processed by the encoder, the resulting feature map is processed by the feature frequency domain separation module to obtain a low-frequency feature map and a high-frequency feature map, respectively. The low-frequency feature map is input into the prototype head, and the high-frequency feature map is input into the linear head.

[0017] Labeled and unlabeled data are simultaneously input into the student model to obtain prototype head prediction results and linear head prediction results;

[0018] The teacher model also includes a fusion module, which inputs unlabeled data and its augmented batch data into the teacher model to obtain four different segmentation prediction results. Specifically, these include the prototype head prediction result and the linear head prediction result for the unlabeled data, and the prototype head prediction result and the linear head prediction result for the augmented batch data. These four different segmentation prediction results are then input into the fusion module to obtain pseudo-labels.

[0019] Furthermore, it also includes backpropagation optimization of the encoder of the student model, specifically:

[0020] Extract the high-frequency and low-frequency components of the labels or pseudo-labels in the labeled data; calculate the prototype segmentation loss by comparing the high-frequency components of the labels or pseudo-labels in the labeled data with the prototype head prediction results corresponding to the labeled data or unlabeled data; calculate the linear segmentation loss by comparing the low-frequency components of the labels or pseudo-labels in the labeled data with the linear head prediction results of the student model corresponding to the labeled data or unlabeled data.

[0021] It also includes calculating the cluster evaluation loss;

[0022] The encoder of the student model is optimized by backpropagation based on prototype segmentation loss, linear segmentation loss, and cluster evaluation loss.

[0023] Furthermore, the implementation process of the feature frequency domain separation module is as follows: after the feature map is transformed into a frequency domain representation by the DCT (Discrete Cosine Transform) algorithm, the masks corresponding to low-frequency and high-frequency features are extracted from the frequency domain map. For the feature map of each channel, the low-frequency feature map and the high-frequency feature map are obtained by using the IDCT (Inverse Discrete Cosine Transform) algorithm with the help of the mask.

[0024] Furthermore, the implementation process of the fusion module is as follows: calculate the proportion of entropy change in the segmentation prediction of the prototype head and the linear head of the teacher model before and after the enhancement, respectively. If the proportion of entropy change of the prototype head and the linear head of the teacher model both meet the set conditions, then the prototype head prediction result and the linear head prediction result of the same batch of data after the unlabeled data enhancement are used to generate pseudo-labels; otherwise, the current batch of data after the unlabeled data enhancement is discarded.

[0025] Furthermore, it also includes dynamically updating the prototype head in the student model.

[0026] The beneficial effects of this invention are as follows: This invention designs a semi-supervised segmentation method that combines frequency decoupling learning with entropy change pseudo-label selection using a dual-segmentation head. It employs a parallel structure of a prototype segmentation head and a linear segmentation head. By applying frequency domain filtering to the encoder output features, low-frequency and high-frequency features are extracted separately. This ensures that the prototype segmentation head focuses on modeling the global structure while optimizing clustering performance based on clustering index loss, while the linear segmentation head focuses on extracting local edges and details. This frequency division strategy allows the model to accommodate semantic information representation at different scales. Furthermore, after applying random enhancement to the input image, this invention proposes using the ratio of the predicted entropy changes of the two segmentation heads before and after enhancement as an evaluation index for the reasonableness of the perturbation. If both entropy ratios are within a reasonable range, the enhancement is considered not to damage the original structural information, and pseudo-labels can be safely used; otherwise, they are filtered out, dynamically controlling the quality of pseudo-labels. Finally, pseudo-labels are generated by fusing the prediction results of the two segmentation heads, thus balancing global consistency and local accuracy, improving the overall reliability and guiding value of the pseudo-labels. Attached Figure Description

[0027] Figure 1 This is a flowchart of the method of the present invention.

[0028] Figure 2 This is a schematic diagram of the student model and teacher model structure in this invention. Detailed Implementation

[0029] To facilitate understanding of the present invention by those skilled in the art, the following technical terms will be explained first:

[0030] Pseudo-labels: These refer to the predictions made by the teacher model on unlabeled data, which are then used as "labels" for training the student model, in order to achieve supervised use of unlabeled data.

[0031] Prototype: Represents the feature center or representative vector of a certain class, usually generated by the aggregation of intra-class features, and is used to calculate similarity.

[0032] Contrastive loss: A loss function that brings similar samples closer together and pushes different samples further apart, used to optimize the discriminativeness of the feature space.

[0033] Entropy: used to measure the uncertainty of a probability distribution; high entropy indicates unstable prediction; in this invention, it is used to determine whether a pseudo-label is credible.

[0034] The invention will be further explained below with reference to the accompanying drawings:

[0035] The student and teacher models of this invention are designed with a dual-head structure, such as... Figure 2 As shown, the dual-head structure specifically includes a prototype segmentation head and a linear segmentation head. The prototype segmentation head is used to output a prediction based on the distance or similarity between the prototype and the extracted features; the linear segmentation head is used to output a prediction based on linear projection.

[0036] The data processed in this invention can refer to most medical image datasets, such as brain tumor image data or heart image data. The medical image dataset is divided into two parts of 2D data, labeled and unlabeled, for training. At the same time, all data is normalized and then cropped to a uniform size of 224*224 to complete the image data preprocessing.

[0037] In the student model, the prototype prediction head receives low-frequency component features from the encoder, focusing on global structural region modeling and improving the ability to perceive large-scale semantic coherence; the linear prediction head receives high-frequency component features from the encoder, focusing on modeling local edges and details, and improving the sensitivity to target boundaries and fine-grained changes.

[0038] For unlabeled samples, after applying random augmentation, the two segmentation heads of the teacher model predict the class distribution of the samples before and after augmentation, and calculate the ratio of change in average prediction entropy. Based on the ratio of entropy change before and after augmentation, the prediction stability after augmentation is jointly evaluated by combining the responses of the prototype head and the linear head. When the entropy changes of both heads are within a reasonable range, it is considered that the augmentation has not caused excessive perturbation, and pseudo-labels are generated using the augmented image; otherwise, the sample is discarded to prevent pseudo-label contamination. The pseudo-labels fuse the output prediction results of the two heads, taking into account both global structure and local boundary accuracy, further improving the reliability of the pseudo-labels.

[0039] The teacher model uses an exponential moving average algorithm to obtain parameter weights from the student model and updates them to provide stable pseudo-labels. The update is represented as:

[0040] θt←α·θt+(1-α)·θs

[0041] Where θt on the left represents the updated teacher parameters, θt on the right represents the teacher parameters before the update, θs is the student parameters, and α is the attenuation coefficient, which is 0.996 in this embodiment.

[0042] like Figure 2 The outputs of the student and teacher models are explained below:

[0043] The input image x is encoded by encoder ε(·) to obtain intermediate features.

[0044] The features extracted by the encoder are transformed into the frequency domain using the DCT algorithm to extract high-frequency and low-frequency feature masks. Then, the IDCT algorithm is used to convert these masks back into the original spatial domain feature maps, yielding low-frequency and high-frequency feature maps. These are then input into the prototype segmentation head and the linear segmentation head, respectively, to the p... proto and p linear The specific process can be represented as follows:

[0045] Perform 2D DCT on each channel:

[0046]

[0047] in, This represents the DCT algorithm, where c represents the channel number, C represents the total number of channels, and f... (c) This represents the intermediate features of channel c;

[0048] For extracting the masks corresponding to low-frequency and high-frequency features from the frequency domain graph:

[0049]

[0050]

[0051] M high (u,v)=1-M low (u,v)

[0052] Where ρ refers to the radius of the boundary between low and high frequencies, taken as an empirical value of 0.25, M low and M high This is the mask, where u and v are coordinate indices in the frequency domain.

[0053] For each channel's feature map, the IDCT algorithm is used with a mask to obtain low-frequency and high-frequency feature maps:

[0054]

[0055]

[0056] Where ⊙ represents element-wise product.

[0057] Then input the prototype header and the linear header:

[0058] Prototype branch:

[0059] Linear branch:

[0060] like Figure 2 As shown, the implementation process of the fusion module is as follows:

[0061] In semi-supervised learning, generating pseudo-labels using unlabeled samples is a common strategy. However, excessive random augmentation can cause the sample distribution to deviate from the true distribution, leading to an increased pseudo-label error rate and affecting training performance. Observations reveal that different types of segmentation heads have varying sensitivities to augmented inputs: prototype heads are more sensitive to global augmentations such as brightness, contrast, and style changes, while linear heads are more sensitive to local perturbations, such as edge perturbations, CutMix, and CutPaste. Therefore, combining the degree of change in the augmented prediction results from both types of heads allows for dynamic evaluation of whether the augmentation perturbation is within a reasonable range, thus enabling the selection of high-quality pseudo-labels.

[0062] For each predicted image p, the pixel-level entropy is defined as:

[0063]

[0064] p c (i,j) is used to refer to p in general. proto p linear One of the channels.

[0065] Taking the global average yields the average entropy of the entire image:

[0066]

[0067] Calculate the entropy before and after the enhancement:

[0068]

[0069]

[0070] in, The enhancement function is used here; random enhancement in this embodiment refers to one or more random enhancement operations among global enhancements such as brightness and contrast, and local enhancement operations such as edge perturbation, CutMix, and CutPaste performed on the original image. The specific enhancement process is a known existing technology and will not be described in detail here.

[0071] Define the entropy change ratio (i.e., the ratio of entropy change before and after enhancement):

[0072]

[0073] Note that the average entropy increases as the enhanced prediction becomes more uncertain. The entropy will increase, therefore the entropy change ratio r > 1; conversely, an entropy change ratio r < 1 indicates that the enhanced prediction is more certain. Set upper and lower thresholds γ for the reasonableness of the enhancement.min and γ max (If the value is less than 1 to encourage the model to learn underfitting patterns) then the appropriate enhancement condition is:

[0074] γ min ≤r proto ≤γ max andγ min ≤r linear ≤γ max

[0075] In this embodiment, γ min The value is 0.85, γ max The value is 1.2.

[0076] Only when all of the above conditions are met is the augmented sample x considered to be augmented. ′ It can be used to generate pseudo-labels for training. Pseudo-label generation formula:

[0077]

[0078] That is, the pseudo-label is obtained by summing the prediction results from both ends of the teacher model and taking the category with the highest probability.

[0079] like Figure 1 As shown, the present invention also includes a backpropagation optimization process, specifically:

[0080] The prototype vector is continuously optimized and updated during training. The prototype update process can be better controlled based on the clustering index of the prototype and the extracted features, resulting in a more generalized prototype and preventing the prototype from becoming too sparse or collapsing.

[0081] First, define intra-class distance:

[0082]

[0083] Inter-class distance:

[0084]

[0085] Clustering evaluation loss is

[0086]

[0087] in is the mean of the j-th prototype, and β is used to balance the influence coefficients of different terms in the loss; in this embodiment, β is set to 2. This loss term encourages prototypes of different categories, as well as prototypes of the same category, to maintain semantic distinguishability.

[0088] The encoder of the student model is optimized through backpropagation based on clustering evaluation loss, prototype segmentation loss, and linear segmentation loss. The specific backpropagation optimization process is a known existing technique and will not be described in detail here.

[0089] The calculation process for prototype segmentation loss and linear segmentation loss is as follows:

[0090] The labeled and pseudo-labeled image data are processed to obtain corresponding high-frequency and low-frequency region labels. The prototype segmentation loss is calculated by comparing the labels of the labeled data with the corresponding prototype head prediction results. The prototype segmentation loss is also calculated by comparing the high-frequency components of the pseudo-labeled data with the corresponding prototype head prediction results of the unlabeled data. The linear segmentation loss is calculated by comparing the low-frequency components of the labeled data with the corresponding linear head prediction results of the student model. The specific calculation process is as follows:

[0091] Let the original label image or pseudo label be: To extract edge-change regions (high-frequency regions) from the labels, a local class change detection operation is defined. For any pixel location, the class difference between it and its four neighboring pixels is calculated:

[0092]

[0093] Where: δ represents the neighborhood offset (up, down, left, right), It is an indicator function that takes the value 1 when the condition is true and 0 otherwise; y gt (i,j)≠y gt (i+δ i ,j+δ j This indicates whether the label at the current position is different from the label of one of its neighboring pixels. It is used to detect category boundaries and extract structural change information.

[0094] Based on the results of local changes, the binary mask for the high-frequency region is defined as M. high (i,j)=G(i,j), and the binary mask for the low-frequency region is M. low (i,j)=1-M high Since edge features in high-frequency regions can be specialized and learned using only a linear segmentation head, this may lead to unstable learning. Therefore, the linear head is learned using full labels / pseudo-labels.

[0095] Based on the masks for different regions calculated above, the final high-frequency and low-frequency feature collaborative segmentation loss can be calculated as follows:

[0096]

[0097]

[0098]

[0099] in, This represents the loss calculated in the low-frequency region, i.e. Figure 1 The prototype segmentation loss in the model; This represents the loss calculated in the high-frequency region, i.e. Figure 1 The linear segmentation loss is used to balance the importance of different losses. low express The corresponding coefficient, λ high express The corresponding coefficient, λ in this embodiment low With λ high The value is 0.5.

[0100] This also includes dynamic management of the prototype head in the student model:

[0101] 1) Prototype Definition

[0102] Define the prototype set for each class k∈{1,…,K} as follows: in N represents the i-th prototype vector in the k-th class. k This represents the number of prototypes possessed by the k-th class. Let represent the d-dimensional real space.

[0103] For each feature vector Calculate the cosine similarity with the prototype of the class:

[0104]

[0105] Among them, f j f represents the feature vector corresponding to the j-th pixel in the encoder output of the student model. j The prototype is ‖·‖2 denotes the 2-norm of a vector;

[0106] like If the number of prototypes does not exceed 10, then create a new prototype p. new =f j ;

[0107] Where, δ new The threshold value is 0.5 in this embodiment.

[0108] If a certain prototype is in T inactive If the number of activations in a given iteration is less than the threshold τ, then it is deleted; in this embodiment, T inactive The value is 200, and τ is 20.

[0109] If the cosine similarity between two prototypes is greater than the merging threshold δ merge If the two prototypes are similar, then they are merged; the cosine similarity between the two prototypes is calculated as follows:

[0110]

[0111] Where the superscript T denotes the transpose of a matrix or vector, and in this embodiment δ merge The value is 0.9.

[0112] Meanwhile, to prevent an excessive number of prototypes from reducing modeling capabilities, the number of prototypes per category is limited to no more than 10.

[0113] 2) Prototype Update

[0114] An online clustering strategy is employed to dynamically update the prototypes, drawing inspiration from [Zhou et al., 2022]. For each category C, a similarity matrix between its prototype and feature map can be calculated. S k Each element in the representation represents the pixel x at the feature location (h, w). h,w p between the m-th prototype c,m The cosine similarity is calculated, where M represents the number of prototypes in each class, and H and W represent the length and width of the image, respectively. To improve the smoothness of prototype assignment, a temperature factor κ is introduced, which affects the similarity matrix S. c Applying the softmax operation, we obtain the soft allocation matrix L. c :

[0115]

[0116] Where u and v are normalized vectors to ensure that the matrix satisfies the assignment constraints. In this embodiment, κ is set to 1.

[0117] In obtaining the soft allocation matrix L c Subsequently, the present invention is based on feature F, which is consistent with the current label. c For each prototype (i.e., the correctly predicted pixels), a weighted average update is performed, with the update rule as follows:

[0118]

[0119] L c,h,w L represents c The value at (h,w).

[0120] This strategy can dynamically integrate more discriminative prototype representations during training, thereby improving the semantic consistency and category discrimination ability of the prototypes.

[0121] Those skilled in the art will recognize that the embodiments described herein are intended to help the reader understand the principles of the invention, and should be understood that the scope of protection of the invention is not limited to such specific statements and embodiments. Various modifications and variations can be made to the invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the invention should be included within the scope of the claims of the invention.

Claims

1. A semi-supervised image segmentation method combining dual-segmentation head frequency decoupling learning and entropy change pseudo-label selection, characterized in that, It includes a student model and a teacher model, both of which adopt a dual-segment head structure, specifically including a prototype head and a linear head. The student model and the teacher model also include an encoder and a feature frequency domain separation module with the same structure. After the input data is processed by the encoder, the resulting feature map is processed by the feature frequency domain separation module to obtain a low-frequency feature map and a high-frequency feature map, respectively. The low-frequency feature map is input into the prototype head, and the high-frequency feature map is input into the linear head. Labeled and unlabeled data are simultaneously input into the student model to obtain prototype head prediction results and linear head prediction results; The teacher model also includes a fusion module, which inputs unlabeled data and its augmented batch data into the teacher model to obtain four different segmentation prediction results. Specifically, these include the prototype head prediction result and the linear head prediction result for the unlabeled data, and the prototype head prediction result and the linear head prediction result for the augmented batch data. These four different segmentation prediction results are then input into the fusion module to obtain pseudo-labels. The implementation process of the fusion module is as follows: calculate the proportion of entropy change of the segmentation prediction of the prototype head and the linear head in the teacher model before and after the enhancement, respectively. If the proportion of entropy change of the prototype head and the linear head in the teacher model both meet the set conditions, then the prototype head prediction result and the linear head prediction result of the same batch of data after the unlabeled data enhancement are used to generate pseudo-labels; otherwise, the current batch of data after the unlabeled data enhancement is discarded.

2. The semi-supervised image segmentation method combining dual-segmentation head frequency decoupling learning and entropy change pseudo-label selection according to claim 1, characterized in that, This also includes backpropagation optimization of the encoder of the student model, specifically: Extract the high-frequency and low-frequency components of the labels or pseudo-labels in the labeled data; calculate the prototype segmentation loss by comparing the high-frequency components of the labels or pseudo-labels in the labeled data with the prototype head prediction results corresponding to the labeled data or unlabeled data; calculate the linear segmentation loss by comparing the low-frequency components of the labels or pseudo-labels in the labeled data with the linear head prediction results of the student model corresponding to the labeled data or unlabeled data. It also includes calculating the cluster evaluation loss; The encoder of the student model is optimized by backpropagation based on prototype segmentation loss, linear segmentation loss, and cluster evaluation loss.

3. The semi-supervised image segmentation method combining dual-segmentation head frequency decoupling learning and entropy change pseudo-label selection according to claim 2, characterized in that, The high-frequency part is represented as: ； ； in: Indicates the neighborhood offset; It is an indicator function that takes the value 1 when the condition is true and 0 otherwise. It indicates whether the label at the current location is different from the label of one of its neighboring pixels.

4. The semi-supervised image segmentation method combining dual-segmentation head frequency decoupling learning and entropy change pseudo-label selection according to claim 3, characterized in that, The expression for cluster evaluation loss is: ； in, Indicates the clustering assessment loss. Indicates intra-class distance. Indicates the distance between classes. express The coefficient; The calculation formula is: ； in, Indicates the total number of categories. Indicates the first The number of prototypes a class has. This represents the feature vector corresponding to the j-th pixel in the encoder output of the student model. express Belonging to The first in the class One prototype vector, Represents the 2-norm; The calculation formula is: ； in, Indicates the first Class prototype mean Indicates the first Class prototype mean.

5. The semi-supervised image segmentation method combining dual-segmentation head frequency decoupling learning and entropy change pseudo-label selection according to claim 4, characterized in that, The implementation process of the feature frequency domain separation module is as follows: after the feature map is transformed into a frequency domain representation by the DCT algorithm, the masks corresponding to low-frequency and high-frequency features are extracted for the frequency domain map, and the low-frequency feature map and high-frequency feature map are obtained by using the IDCT algorithm with the help of the mask for the feature map of each channel.

6. The semi-supervised image segmentation method combining dual-segmentation head frequency decoupling learning and entropy change pseudo-label selection according to claim 5, characterized in that, It also includes using an online clustering strategy to dynamically update the prototype head in the student model.

Images