Semi-supervised segmentation method combining double-segmentation-head frequency decoupling learning and entropy change pseudo-label screening

Abstract

The invention discloses a semi-supervised segmentation method combining double segmentation head frequency decoupling learning and entropy change pseudo label screening, is applied to the field of image processing, and aims to solve the problems that in an existing semi-supervised semantic segmentation method, pseudo label generation quality is difficult to guarantee, a model is single in enhanced disturbance response, and the efficiency is low. The problems of high false label error and unstable training caused by difficulty in distinguishing global structure change and local detail change are solved; the student model adopts a double-segmentation-head structure and is composed of a prototype prediction head and a linear prediction head, and function division is realized through a frequency domain decomposition mechanism: the prototype head receives low-frequency component characteristics of an encoder, the linear head receives high-frequency component characteristics of the encoder, and for a label-free sample, after random enhancement is applied, the original prediction head and the linear prediction head are divided into two segments; and respectively predicting category distribution of samples before and after enhancement by two segmentation heads of the teacher model, and calculating a change ratio of average prediction entropy. And based on the entropy change ratio before and after enhancement, synthesizing the reaction of the prototype head and the linear head, and jointly evaluating the prediction stability after enhancement.

Description

Technical Field

[0001] The invention belongs to the field of image processing, and in particular relates to a semi-supervised segmentation technology. Background Art

[0002] In recent years, deep learning techniques have made significant progress in medical image segmentation tasks. In particular, methods based on convolutional neural networks (CNNs) have demonstrated excellent performance on a variety of medical image datasets. However, these methods typically rely on large amounts of accurately labeled pixel-level data for supervised learning. To reduce this reliance on large amounts of labeled data, researchers have proposed various semi-supervised learning strategies that incorporate unlabeled data into the training process to improve model generalization and data utilization. The core of semi-supervised segmentation methods lies in how to effectively mine the potential information in unlabeled data and optimize the model using limited labeled samples. However, medical images are characterized by modality diversity (such as MRI, CT, and ultrasound), complex organ structures, and widely varying lesion morphology, making semi-supervised segmentation a particularly challenging task in this field. Furthermore, the annotation process for medical images is not only time-consuming but also requires the involvement of radiologists or other professionals, which significantly limits the efficiency and scale of obtaining high-quality labeled data.

[0003] Semi-supervised learning has achieved considerable success in the field of natural images, with applications such as consistency regularization, self-training, and contrastive learning. These concepts are also gradually being introduced into medical image segmentation to address the shortage of high-quality annotated data. For example, the use of a teacher-student framework for consistency constraints or the use of pseudo-labels to enhance model training have become the current mainstream design approaches for semi-supervised segmentation frameworks. Improving the reliability of pseudo-labels and guiding the model to learn valuable feature information from unlabeled samples are key to improving semi-supervised segmentation performance.

[0004] In the task of medical image segmentation, the feature distribution between categories is often highly variable and ambiguous, especially in small sample or weakly supervised scenarios. In order to improve the discriminative ability of the model, a class of segmentation methods based on prototype learning has emerged in recent years. This type of method guides the model to align in the feature space by constructing representative features of each category (i.e., prototypes), thereby achieving more robust pixel-level classification. The prototype is usually defined as the center vector of a certain category in the feature space, which can be obtained by the feature mean or weighted aggregation of labeled samples. In semi-supervised segmentation, by introducing category prototypes, the pixel features of unlabeled samples can be similarly calculated or matched with the prototype, thereby assisting in pseudo-label generation, feature alignment or consistency modeling, and greatly enhancing the supervisory signal of unlabeled data. Some work has further proposed a dynamic prototype update mechanism to continuously optimize the category prototype during the training process, making it closer to the current model's expression of the category features, effectively avoiding the offset of pseudo-label errors in prototype calculation.

[0005] In actual medical image segmentation tasks, data often comes from different hospitals, equipment or imaging protocols, resulting in significant inter-domain differences between images. This causes the performance of models trained only on the source domain (i.e., labeled data) to decline significantly on the target domain (unlabeled data), seriously affecting the generalization performance of the model. To solve this problem, researchers began to introduce the idea of ​​domain adaptation (DA) into semi-supervised segmentation, bridging the distribution differences between the source and target domains to improve the model's adaptability to unlabeled target domain data. In semi-supervised scenarios, the strategy of combining self-training and domain adaptation is particularly common: on the one hand, the model itself generates pseudo labels on the target domain for self-supervision, and on the other hand, alignment constraints such as prototype alignment are imposed on the feature layer or output layer to narrow the source-target distribution gap.

[0006] Prior art 1, "Semi-supervised Semantic Segmentation with Prototype-based Consistency Regulation (PbCR), a prototype-based consistency regularization mechanism (Prototype-based Consistency Regularization)". The model has two segmentation heads, namely the prototype-based segmentation head and the linear projection-based segmentation head. The prototype-based segmentation head obtains the segmentation result by constructing a prototype of the semantic category in the feature space, and matches the prototype with the features extracted by the student model for the labeled image. The linear projection-based segmentation head generates pixel-level predictions through a learnable linear classifier. The losses of the two segmentation results of the labeled image are calculated and back-propagated with the original label of the image. At the same time, the two segmentation predictions of the unlabeled image output by the student model and the segmentation predictions obtained by the linear segmentation head of the teacher model are subjected to consistency regularization to obtain stronger discriminability and more robust pseudo-supervision effects.

[0007] This technology first uses labeled images to fully supervise pre-train a model with a linear prediction head. The k-means clustering algorithm is then used to cluster the image features extracted by the pre-trained model to obtain an initial prototype. Conventional segmentation prediction and loss calculation are then performed on the labeled data. After weak enhancements such as flipping and rotation are performed on the unlabeled images, a cut-mix operation is performed to obtain X~u, which is input into the student model. Two different segmentation predictions are obtained for the same batch of unlabeled images using a prototype segmentation head that makes segmentation predictions based on the distance between the prototype and the features, and a linear segmentation head that makes predictions using linear projection. The teacher model inputs X~u and outputs the segmentation predictions of the previous segmentation head as pseudo-labels. The cross-entropy loss is then calculated with the two predictions output by the student model. In each iteration, the model again uses the k-means algorithm to cluster the features extracted by the encoder to obtain a new prototype and perform a momentum update.

[0008] The effectiveness of existing techniques, such as PbCR, relies heavily on pre-training and carries significant training overhead. This approach simply imposes a consistency constraint on the results of two different segmentation methods, without further exploring their relationship. It also ignores the impact of potential class imbalance in the dataset on prototype formation and updating, limiting its ability to achieve more generalized performance.

[0009] Prior art 2, "Upcol: Uncertainty-informed prototype consistency learning for semi-supervised medical image segmentation," proposes a semi-supervised medical image segmentation framework based on uncertainty-informed prototype consistency learning (UPCoL). This approach aims to improve the model's ability to recognize complex medical structures by fusing feature prototypes from labeled and unlabeled data to optimize the distribution of the embedding space. This approach is based on a student-teacher framework, where the student model optimizes parameters through supervised learning, and the teacher model is updated using an exponential moving average (EMA) to generate pseudo-labels and uncertainty estimates. The core innovation lies in the introduction of an uncertainty-guided prototype fusion mechanism and consistency constraints in the feature space. The specific process is as follows:

[0010] First, the uncertainty assessment module integrates the predictions of multiple classifiers (cross entropy, focal loss, Dice loss, and IoU loss) to calculate the entropy of each voxel to quantify its reliability. Voxels with high entropy values ​​indicate ambiguous predictions and should be weighted down; voxels with low entropy values ​​are considered reliable. Based on this, a reliability map is generated for weighted calculations in subsequent prototype fusion.

[0011] Secondly, prototype extraction and fusion are divided into two parts: labeled data and unlabeled data. For labeled data, the feature maps are directly masked and averaged pooled using the true labels to obtain the initial prototypes for each category. For unlabeled data, prototypes are extracted through attention-weighted pooling, combining the features and reliability maps generated by the teacher model. The two types of prototypes are combined through a dynamic fusion strategy: a time-dependent Gaussian ramping function is used to adjust the fusion ratio. Initially, the labeled prototypes are the primary focus, and the weight of the unlabeled prototypes is gradually increased to ultimately achieve balanced fusion. This strategy ensures the reliability of the prototypes in the initial stage while gradually introducing the diversity information of the unlabeled data.

[0012] Furthermore, prototype consistency learning constructs a consistency loss by calculating the cosine similarity between the feature embedding and the fused prototype. For labeled data, the prototype predictions are required to be consistent with the true labels. For unlabeled data, a reliability graph is used to weight the consistency between the prototype predictions and the pseudo-labels of the teacher model. The final total loss combines the segmentation loss and the prototype consistency loss to improve model performance through end-to-end optimization. This process forces the feature representation to be more compact between samples of the same class and more separable between samples of different classes, thereby alleviating the problems of large intra-class variability and blurred boundaries common in medical images.

[0013] In addition, the paper proposes a nonlinear prototype update mechanism that gradually adjusts the fusion ratio to ensure stable iteration of prototypes during training. Compared with existing methods (such as updating labeled and unlabeled prototypes separately), the fused prototype can more comprehensively capture the data distribution and avoid information fragmentation.

[0014] The second existing technique, average pooling, is a relatively primitive method for obtaining prototypes. Due to intra-class variations, simply averaging all pixels or voxels ignores these differences, reducing the generalization ability of the prototype and further decreasing the precision of the model segmentation. Furthermore, high-entropy regions calculated in the reliability map do not necessarily represent non-target areas; rather, they simply indicate that the model has a poor fit to these areas, failing to truly address intra-class variations and cross-domain inconsistencies. Summary of the Invention

[0015] To solve the above technical problems, the present invention proposes a semi-supervised segmentation method that combines dual-segmentation head frequency decoupling learning with entropy change pseudo-label screening; it systematically combines frequency division of labor, enhanced response sensitivity and pseudo-label screening mechanism, while ensuring the diversity and stability of feature expression, dynamically controls the quality of pseudo-label usage, thereby improving the performance and robustness of the semi-supervised semantic segmentation model in low-label scenarios.

[0016] The technical solution adopted by the present invention is: a semi-supervised segmentation method that combines dual-segmentation head frequency decoupling learning with entropy change pseudo-label screening, including a student model and a teacher model, both of which adopt a dual-segmentation head structure, and the dual-segmentation head structure specifically includes a prototype head and a linear head; the student model and the teacher model also include an encoder and a feature frequency domain separation module with the same structure. After the input data is processed by the encoder, the feature map obtained is passed through the feature frequency domain separation module to obtain a low-frequency feature map and a high-frequency feature map, respectively. The low-frequency feature map is input into the prototype head, and the high-frequency feature map is input into the linear head;

[0017] Input labeled and unlabeled data into the student model at the same time to obtain the prototype head prediction results and the linear head prediction results;

[0018] The teacher model also includes a fusion module, which inputs the unlabeled data and the enhanced same batch data into the teacher model respectively to obtain four different segmentation prediction results, specifically including the prototype head prediction result and the linear head prediction result of the unlabeled data, and the prototype head prediction result and the linear head prediction result of the same batch data after the unlabeled data is enhanced; these four different segmentation prediction results are input into the fusion module to obtain pseudo labels.

[0019] Furthermore, it also includes back-propagation optimization of the encoder of the student model, specifically:

[0020] Extract the high-frequency and low-frequency parts of the labels or pseudo-labels of the labeled data; calculate the prototype segmentation loss by respectively using the high-frequency parts of the labels or pseudo-labels of the labeled data and the prototype head prediction results corresponding to the labeled data or the unlabeled data, and calculate the linear segmentation loss by respectively using the linear head prediction results of the student model corresponding to the labeled data or the unlabeled data;

[0021] It also includes calculating cluster evaluation loss;

[0022] The encoder of the student model is back-propagated and optimized based on prototype segmentation loss, linear segmentation loss, and clustering evaluation loss.

[0023] Furthermore, the implementation process of the feature frequency domain separation module is as follows: after the feature map is transformed into a frequency domain representation by the DCT (Discrete Cosine Transform) algorithm, the masks corresponding to the low-frequency and high-frequency features are extracted from the frequency domain map, and the feature map of each channel is subjected to the mask using the IDCT (Inverse discrete cosine transform) algorithm to obtain the low-frequency feature map and the high-frequency feature map.

[0024] Furthermore, the implementation process of the fusion module is: respectively calculating the ratio of the entropy change of the segmentation prediction of the data before and after the output enhancement of the prototype head and the linear head in the teacher model; if the ratio of the entropy change of the prototype head and the linear head in the teacher model meets the set conditions, the prototype head prediction results and the linear head prediction results of the same batch of data after the unlabeled data enhancement are used to generate pseudo labels; otherwise, the same batch of data after the current unlabeled data enhancement is discarded.

[0025] Furthermore, it also includes dynamically updating the prototype head in the student model.

[0026] Beneficial effects of the present invention: The present invention designs a semi-supervised segmentation method that combines dual-segmentation head frequency decoupling learning with entropy change pseudo-label screening. A prototype segmentation head and a linear segmentation head are used in parallel, and frequency domain filtering operations are applied to the encoder output features to extract low-frequency and high-frequency features respectively, ensuring that the prototype segmentation head focuses on modeling the global structure while optimizing the clustering effect according to the clustering index loss, while the linear segmentation head focuses on the extraction of local edges and details. This frequency division of labor strategy enables the model to take into account the expression of semantic information at different scales. At the same time, after applying random enhancement to the input image, the present invention proposes to use the ratio of the predicted entropy changes of the two segmentation heads before and after enhancement as an evaluation index for the rationality of the disturbance. If the entropy ratios of the two are within a reasonable range, it is considered that the enhancement will not destroy the original structural information, and the pseudo-label can be used safely. Otherwise, it is screened out to dynamically control the quality of the pseudo-label. Finally, the pseudo-label is generated by fusing the prediction results of the two segmentation heads, thereby taking into account global consistency and local accuracy, and improving the overall reliability and guidance value of the pseudo-label. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Figure 1 Flowchart of the method of the present invention.

[0028] Figure 2 Schematic diagram of the student model and teacher model structure in the present invention. DETAILED DESCRIPTION

[0029] To facilitate those skilled in the art to understand the content of the present invention, the following technical terms are first explained:

[0030] Pseudo-labels: refer to the prediction results of the teacher model on unlabeled data, which are used as training "labels" for the student model to achieve supervised use of unlabeled data.

[0031] Prototype: Represents the feature center or representative vector of a class, usually generated by aggregating features within the class and used to calculate similarity.

[0032] Contrastive loss: A loss function that brings similar samples closer and pushes different samples further away, used to optimize the discriminability of feature space.

[0033] Entropy: It is used to measure the uncertainty of probability distribution. High entropy indicates unstable prediction. In this paper, it is used to determine whether the pseudo-label is credible.

[0034] The present invention will be further explained below with reference to the accompanying drawings:

[0035] The student model and teacher model of the present invention are designed as a double-headed structure, such as Figure 2 As shown, the dual-head structure specifically includes: a prototype segmentation head and a linear segmentation head, wherein the prototype segmentation head is used to output a prediction based on the distance or similarity between the prototype and the extracted features; the linear segmentation head is used to output a prediction based on linear projection.

[0036] The data processed in the present invention can refer to most medical image datasets, such as brain tumor image data or heart image data. The image data preprocessing is completed by dividing the medical image dataset into two parts of 2D data, labeled and unlabeled. At the same time, all the data are normalized and then cropped to a uniform size of 224*224.

[0037] The prototype prediction head in the student model receives the low-frequency component features of the encoder, focuses on modeling the global structure area, and improves the perception of large-scale semantic coherence; the linear prediction head receives the high-frequency component features of the encoder, focuses on modeling local edges and detail information, and improves sensitivity to target boundaries and fine-grained changes.

[0038] For unlabeled samples, after random augmentation, the two segmentation heads of the teacher model predict the category distribution of the samples before and after augmentation, and calculate the ratio of the change in the average predicted entropy. Based on the ratio of the entropy change before and after augmentation, the responses of the prototype head and the linear head are combined to jointly evaluate the stability of the augmented prediction. When the entropy change of both heads is within a reasonable range, the augmentation is considered to have not caused excessive perturbation, and the enhanced image is used to generate a pseudo-label. Otherwise, the sample is discarded to prevent pseudo-label contamination. The pseudo-label combines the output prediction results of the two heads, taking into account both global structure and local boundary accuracy, further improving the reliability of the pseudo-label.

[0039] The teacher model uses the exponential moving average algorithm to obtain parameter weights from the student model and update it to provide stable pseudo labels. The update is expressed as:

[0040] θt←α·θt+(1-α)·θs

[0041] Among them, θt on the left of ← represents the updated teacher parameter, θt on the right of ← represents the teacher parameter before the update, θs is the student parameter, α is the attenuation coefficient, and in this embodiment, the value of α is 0.996.

[0042] like Figure 2 As shown, the output of the student model and the teacher model are described as follows:

[0043] The input image x is passed through the encoder ε(·) to obtain the intermediate features

[0044] The features extracted by the encoder are converted into frequency domain using the DCT algorithm to extract high-frequency and low-frequency feature masks, and then converted into the original spatial domain feature map using the IDCT algorithm to obtain low-frequency and high-frequency feature maps. proto and p linear The specific process can be expressed as follows:

[0045] Do 2D DCT on each channel:

[0046]

[0047] in, represents the DCT algorithm, c represents the channel number, C represents the total number of channels, f (c) represents the intermediate features of channel c;

[0048] For the frequency domain image, extract the masks corresponding to the low-frequency and high-frequency features:

[0049]

[0050]

[0051] M high (u,v)=1-M low (u,v)

[0052] Where ρ refers to the radius of the boundary between low frequency and high frequency, and the empirical value is 0.25, M low and M high That is the mask, u, v are the coordinate indices in the frequency domain.

[0053] The IDCT algorithm is used to obtain low-frequency and high-frequency feature maps for the feature map of each channel with the help of a mask:

[0054]

[0055]

[0056] where ⊙ represents the element-wise product

[0057] Then enter the prototype header and linear header:

[0058] Prototype branch:

[0059] Linear branching:

[0060] like Figure 2 As shown in Figure 2, the implementation process of the fusion module is as follows:

[0061] In semi-supervised learning, it is a common strategy to generate pseudo labels using unlabeled samples. However, excessive random augmentation may cause the sample distribution to deviate from the true distribution, thereby increasing the pseudo label error rate and affecting the training effect. It is observed that different types of segmentation heads have different sensitivities to the augmented input: the prototype segmentation head is more sensitive to global augmentations such as brightness, contrast, and style changes. The linear segmentation head is more sensitive to local perturbations such as edge perturbations, CutMix, and CutPaste. Therefore, the degree of change in the prediction results after augmentation can be combined to dynamically evaluate whether the augmentation perturbation is within a reasonable range, thereby screening high-quality pseudo labels.

[0062] For each prediction graph p, the pixel-level entropy is defined as:

[0063]

[0064] p c (i,j) is used to refer to p proto 、p linear A channel in the.

[0065] Take the global average to get the average entropy of the entire image:

[0066]

[0067] Calculate the entropy before and after enhancement:

[0068]

[0069]

[0070] in, is an enhancement function. The random enhancement in this embodiment refers to performing global enhancements such as brightness and contrast on the original image, as well as one or more random enhancement operations such as edge perturbation, CutMix, and CutPaste. The specific enhancement process is well known in the art and will not be elaborated in detail in this invention.

[0071] Define the entropy change ratio (i.e. the ratio of entropy change before and after enhancement):

[0072]

[0073] Note that as the prediction becomes more uncertain after augmentation, the average entropy will increase, so the entropy change ratio r>1, and conversely, the entropy change ratio r<1 indicates that the prediction is more certain after enhancement. Set the upper and lower thresholds γ for the rationality of enhancementmin and γ max (less than 1 to encourage the model to learn the underfitting mode) then the reasonable conditions for enhancement are:

[0074] γ min ≤r proto ≤γ max andγ min ≤r linear ≤γ max

[0075] In this embodiment, min The value is 0.85, γ max The value is 1.2.

[0076] Only when the above conditions are met at the same time, the enhanced sample x is considered ′ Can be used to generate pseudo labels for training. Pseudo label generation formula:

[0077]

[0078] That is, the prediction results of both ends of the teacher model are summed up and the category corresponding to the maximum probability is taken as the pseudo label.

[0079] like Figure 1 As shown, the present invention also includes a back propagation optimization process, specifically:

[0080] The prototype vector is continuously optimized and updated during training. The prototype update process can be better controlled based on the clustering indicators of the prototype and the extracted features to obtain a more generalized prototype and prevent the prototype from being too sparse or collapsing.

[0081] First, define the intra-class distance:

[0082]

[0083] And the distance between classes:

[0084]

[0085] The clustering evaluation loss is

[0086]

[0087] in is the mean value of the j-th prototype, and β is used to balance the influence coefficients of different items in the loss. In this embodiment, β is set to 2. Through this loss term, prototypes of different categories and prototypes of the same category are encouraged to maintain semantic distinction.

[0088] Backpropagation optimization is performed on the encoder of the student model based on cluster evaluation loss, prototype segmentation loss, and linear segmentation loss. The specific backpropagation optimization process is well known in the art and will not be elaborated in detail in this invention.

[0089] The calculation process of prototype segmentation loss and linear segmentation loss is:

[0090] The labels and pseudo-labels of the labeled image data are processed to obtain the corresponding high-frequency and low-frequency region labels. The prototype segmentation loss is calculated by combining the label of the labeled data with the prototype head prediction result corresponding to the labeled data, the prototype segmentation loss is calculated by combining the high-frequency part of the pseudo-label with the prototype head prediction result corresponding to the unlabeled data, and the linear segmentation loss is calculated by combining the low-frequency part of the label of the labeled data with the linear head prediction result of the student model corresponding to the labeled data; the linear segmentation loss is calculated by combining the low-frequency part of the pseudo-label with the linear head prediction result of the student model corresponding to the unlabeled data. The specific calculation process is as follows:

[0091] Let the original label map or pseudo label be: In order to extract the edge change area (high-frequency area) in the label, a local category change detection operation is defined. For any pixel position, the category difference between it and the four neighboring pixels is calculated:

[0092]

[0093] Among them: δ represents the neighborhood offset (up, down, left, and right), Is an indicator function, which takes 1 when the condition is met, otherwise it takes 0; gt (i,j)≠y gt (i+δ i ,j+δ j ) indicates whether the label of the current position is different from the label of one of its neighboring pixels. It is used to detect category boundaries and extract structural change information.

[0094] According to the local change results, the binary mask of the high-frequency area is defined as M high (i, j) = G(i, j), the binary mask of the low-frequency region is M low (i,j)=1-M high Although the edge features in high-frequency regions can be specialized and learned using only the linear segmentation head, this may lead to unstable learning. Therefore, the linear head is learned using complete labels / pseudo-labels.

[0095] According to the masks of different regions calculated above, the final high-frequency and low-frequency feature collaborative segmentation loss can be calculated as

[0096]

[0097]

[0098]

[0099] in, represents the loss calculated in the low-frequency region, i.e. Figure 1 Prototype segmentation loss in ; represents the loss calculated in the high-frequency region, i.e. Figure 1 The linear segmentation loss in

[15] ; λ is the importance hyperparameter used to balance different losses, λ low express The corresponding coefficient, λ high express The corresponding coefficient, in this embodiment, is low and λ high The value is 0.5.

[0100] It also includes dynamic management of prototype headers in the student model:

[0101] 1) Prototype definition

[0102] Define the prototype set of each class k∈{1,…,K} as in represents the i-th prototype vector in the k-th class, N k Indicates the number of prototypes of the kth class, represents the d-dimensional real space.

[0103] For each eigenvector Calculate the cosine similarity with the prototype of the class:

[0104]

[0105] Among them, f j Represents the feature vector corresponding to the jth pixel of the output of the encoder in the student model, where f j The prototype is ‖·‖2 represents the 2-norm of the vector;

[0106] like If the number of prototypes does not exceed 10, a new prototype p is created. new =f j ;

[0107] Among them, δ new is the threshold value, which is 0.5 in this embodiment.

[0108] If a prototype is in T inactive If the number of activations in the iterations is less than the threshold τ, it will be deleted; in this embodiment, T inactive The value is 200 and the value of τ is 20.

[0109] If the cosine similarity between the two prototypes is greater than the merge threshold δ merge , then merge; the cosine similarity calculation formula between two prototypes is:

[0110]

[0111] Wherein, the superscript T represents the transpose of a matrix or vector. In this embodiment, δ merge The value is 0.9.

[0112] At the same time, in order to prevent the excessive number of prototypes, which would lead to a decrease in modeling capabilities, the number of prototypes is limited to no more than 10 per category.

[0113] 2) Prototype update

[0114] The prototype is dynamically updated using an online clustering strategy, which is inspired by the idea of ​​[Zhou et al., 2022]. For each category C, the similarity matrix between its prototype and feature map can be calculated. S k Each element in represents the pixel x at the feature position (h,w). h,w Between the mth prototype c,m The cosine similarity of M represents the number of prototypes in each category, H and W represent the length and width of the image respectively. In order to improve the smoothness of the prototype assignment, the temperature factor κ is introduced to the similarity matrix S c Apply the softmax operation to obtain the soft assignment matrix L c :

[0115]

[0116] Where u and v are normalized vectors to ensure that the matrix satisfies the allocation constraint. In this embodiment, κ is set to 1.

[0117] After obtaining the soft assignment matrix L c Afterwards, the present invention is based on the feature F that is consistent with the current label c (i.e. predict the correct pixel) and perform a weighted average update on each prototype. The update rule is:

[0118]

[0119] L c,h,w Indicates L c The value at (h,w) in .

[0120] This strategy can dynamically integrate more discriminative prototype representations during training, improving the semantic consistency and category discrimination ability of the prototype.

[0121] Those skilled in the art will appreciate that the embodiments described herein are intended to aid the reader in understanding the principles of the present invention, and it should be understood that the scope of the present invention is not limited to such specific descriptions and embodiments. Various modifications and variations are readily apparent to those skilled in the art. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention are intended to be included within the scope of the claims.

Claims

1. A semi-supervised segmentation method combining dual-segmentation head frequency decoupling learning with entropy change pseudo-label screening, characterized by: The system includes a student model and a teacher model. Both the student model and the teacher model adopt a dual-segmentation head structure, which specifically includes a prototype head and a linear head. The student model and the teacher model also include an encoder and a feature frequency domain separation module with the same structure. After the input data is processed by the encoder, the feature map obtained is passed through the feature frequency domain separation module to obtain a low-frequency feature map and a high-frequency feature map respectively. The low-frequency feature map is input into the prototype head, and the high-frequency feature map is input into the linear head. Input labeled and unlabeled data into the student model at the same time to obtain the prototype head prediction results and the linear head prediction results; The teacher model also includes a fusion module, which inputs the unlabeled data and the enhanced same batch data into the teacher model respectively to obtain four different segmentation prediction results, specifically including the prototype head prediction result and the linear head prediction result of the unlabeled data, and the prototype head prediction result and the linear head prediction result of the same batch data after the unlabeled data is enhanced; these four different segmentation prediction results are input into the fusion module to obtain pseudo labels.

2. A semi-supervised segmentation method combining dual-segmentation head frequency decoupling learning with entropy change pseudo-label screening according to claim 1, characterized in that: It also includes backpropagation optimization of the encoder of the student model, specifically: Extract the high-frequency and low-frequency parts of the labels or pseudo-labels of the labeled data; calculate the prototype segmentation loss by respectively using the high-frequency parts of the labels or pseudo-labels of the labeled data and the prototype head prediction results corresponding to the labeled data or the unlabeled data, and calculate the linear segmentation loss by respectively using the linear head prediction results of the student model corresponding to the labeled data or the unlabeled data; It also includes calculating cluster evaluation loss; The encoder of the student model is back-propagated and optimized based on prototype segmentation loss, linear segmentation loss, and clustering evaluation loss.

3. A semi-supervised segmentation method combining dual-segmentation head frequency decoupling learning with entropy change pseudo-label screening according to claim 2, characterized in that: The high frequency part is expressed as: M high (i,j)=G(i,j) Where: δ represents the neighborhood offset; Is an indicator function, which takes 1 when the condition is met, otherwise it takes 0; gt (i,j)≠y gt (i+δ i,j +δ j ) indicates whether the label of the current position is different from the label of one of its neighboring pixels.

4. A semi-supervised segmentation method combining dual-segmentation head frequency decoupling learning with entropy change pseudo-label screening according to claim 3, characterized in that: The clustering evaluation loss expression is: in, represents the clustering evaluation loss, D intra represents the intra-class distance, D inter represents the distance between classes, and β represents D inter The coefficient of D intra The calculation formula is: Among them, K represents the total number of categories, N k Indicates the number of prototypes of the kth class, f j represents the feature vector corresponding to the j-th pixel of the output of the encoder in the student model, represents f j The i-th prototype vector in the k-th class, ‖·‖2 represents the 2-norm; D inter The calculation formula is: in, represents the prototype mean of the k1 class, represents the prototype mean of the k2 class.

5. A semi-supervised segmentation method combining dual-segmentation-head frequency decoupling learning with entropy-change pseudo-label screening according to claim 4, characterized in that: The implementation process of the feature frequency domain separation module is as follows: after the feature map is transformed into a frequency domain representation through the DCT algorithm, the masks corresponding to the low-frequency and high-frequency features are extracted from the frequency domain map, and the IDCT algorithm is used to obtain the low-frequency feature map and the high-frequency feature map for the feature map of each channel with the help of the mask.

6. A semi-supervised segmentation method combining dual-segmentation head frequency decoupling learning with entropy change pseudo-label screening according to claim 5, characterized in that: The implementation process of the fusion module is as follows: respectively calculate the ratio of the entropy change of the segmentation prediction of the data before and after the output enhancement of the prototype head and the linear head in the teacher model; if the ratio of the entropy change of the prototype head and the linear head in the teacher model meets the set conditions, then use the prototype head prediction results and the linear head prediction results of the same batch of data after the unlabeled data enhancement to generate pseudo labels; otherwise, discard the same batch of data after the current unlabeled data enhancement.

7. A semi-supervised segmentation method combining dual-segmentation-head frequency decoupling learning with entropy-change pseudo-label screening according to claim 6, characterized in that: It also includes the use of online clustering strategies to dynamically update the prototype heads in the student model.

Images