Semi-supervised segmentation method based on graph prototype learning

Abstract

The invention discloses a semi-supervised segmentation method based on graph prototype learning, and the method comprises the steps: collecting a data set, carrying out the preprocessing of the data set, carrying out the mapping of a pixel space to a graph semantic space for the bottom features of an encoder, carrying out the multi-stage graph convolution compression of the number of vertexes, and carrying out the alignment to a labeled image category prototype; sending an unlabeled image into the segmentation network to obtain a category prototype of a pseudo label, and aligning the centroid of the most similar subclass after the category prototype of the unlabeled image is subjected to self-clustering with the category prototype of the labeled image; fusing the structure information of the image with the label and the style information of the image without the label to obtain a composite image; screening prediction results of a non-label image and a plurality of weak disturbance images through a voting strategy, determining reliable and fuzzy pseudo labels, and optimizing a segmentation model and category prototype sampling; and updating network parameters through training to obtain a final segmentation network. According to the method, accurate segmentation can be realized in scenes of scarcity of high-quality labeled data, complex structure and shape of a target and the like.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image processing, in particular to a semi-supervised segmentation method based on graph prototype learning. BACKGROUND

[0002] Alopecia areata (AA) is a common hair loss disease that affects the quality of life and mental health, and accurate segmentation of alopecia areata (AA) lesions is crucial for diagnosing and grading this non-scarring autoimmune disease. However, due to limited labeled datasets, uneven hair distribution, fuzzy lesion boundaries, differences in scalp and hair color, and low image quality, reliable segmentation remains challenging.

[0003] To address these challenges, many semi-supervised methods utilize unlabeled data, which are roughly divided into pseudo-label methods and unsupervised regularization. Although these methods mostly emphasize the utilization of unlabeled data, they often ignore the explainability and reliability of data augmentation strategies. For example, random cropping is commonly used to reduce overfitting and increase data diversity, but its random strategy can undermine the explainability of the method and compromise the robustness of the model, which reduces the reliability of medical image analysis. SUMMARY

[0004] The technical problem to be solved by the present application is to provide a semi-supervised segmentation method based on graph prototype learning, which can achieve accurate segmentation in scenarios where high-quality labeled data is scarce and the structure and shape of the target are complex.

[0005] To solve the above technical problems, the present application provides a semi-supervised segmentation method based on graph prototype learning, comprising the following steps:

[0006] Step 1) Collecting scalp images of alopecia areata patients as a dataset, labeling the segmentation targets in the dataset, and dividing the training set, which includes labeled images and unlabeled images;

[0007] Step 2) Constructing class prototypes based on labeled images, mapping image features extracted by the encoder from pixel space to graph semantic space to obtain graph prototypes, compressing graph prototypes to class numbers through multi-layer graph convolution and full connection, and aligning graph prototypes of the labeled branch with class prototypes at multiple levels;

[0008] Step 3) Clustering each class prototype into several sub-classes, feeding the unlabeled images into the segmentation network to obtain segmentation results as pseudo-labels to extract class prototypes, and aligning the class prototype of each unlabeled image with the most similar sub-class centroid of the labeled image;

[0009] Step 4) Decompose the labeled and unlabeled images at the prototype level to obtain their respective style and structural information, fuse the structural information of the labeled images with the style information of the unlabeled images, and add the synthesized image to the segmentation network for training.

[0010] Step 5) Apply different weak perturbations to the input unlabeled image to obtain several corresponding weakly perturbated images. Then, combine the weakly perturbated images with the original image. Figure One The input is used to segmentation network, resulting in multiple predictions.

[0011] Subsequently, a voting strategy was used to identify two pseudo-labels with different confidence levels;

[0012] Step 6) Calculate the total loss based on labeled images, synthetic images, and unlabeled images, update the network parameters, and enhance the network's robustness, enabling the network to perform well in segmentation on datasets with only a small number of labeled images. The final result is a semi-supervised segmentation network for segmentation.

[0013] Furthermore, in step 1), the images in the training set, test set, and validation set are cropped or scaled to a uniform size.

[0014] Furthermore, in step 1), N+M scalp images of patients with alopecia areata are collected as a dataset, where N represents N images as a labeled dataset. in, It is the i-th image in the labeled images of the training set. Let be the label corresponding to the i-th image in the labeled image set;

[0015] M represents M images as an unlabeled dataset. in, It is the i-th image in the unlabeled image set.

[0016] Furthermore, in step 2), the encoder of the segmentation network is used to extract features from the labeled and unlabeled images to obtain features f;

[0017] Perform soft clustering on feature f and calculate vertex features;

[0018] Calculate the corresponding adjacency matrix based on the obtained vertex features, perform attention map convolution and fully connected layers, and compress the number of vertices:

[0019] The penultimate layer of the decoder in the segmentation network extracts the features f and category label m of the labeled image. k , where k represents the k-th category, and the distribution space of the category prototypes of labeled images of multiple categories is obtained;

[0020] The vertex features are matched and aligned to the centroids of the class prototype space, and the alignment loss is calculated.

[0021] Furthermore, in step 3), the unlabeled image is fed into the segmentation network to obtain pseudo-labels, and the category prototypes are extracted based on the obtained pseudo-labels to obtain the category prototypes of the unlabeled image to be aligned.

[0022] The category prototypes of the labeled images obtained in step 2) are clustered by category, and several sub-centroids are obtained for each category. The sub-centroids that are closest to the category prototypes of the unlabeled images to be aligned are found as positive samples for contrastive learning, while the centroids of different categories are used as negative samples for calculating the contrastive loss.

[0023] Calculate the contrastive loss between the class prototypes to be aligned in the unlabeled image and the positive and negative samples.

[0024] Furthermore, in step 4), the mean and variance of the prototype features of the labeled image and the prototype features of the unlabeled image are calculated respectively. The mean and variance of the prototype extracted from the labeled image are aligned with the prototype of the unlabeled image and input into the reconstruction decoder to obtain a synthetic image with the structural information of the labeled image and the style information of the unlabeled image.

[0025] The semantic structural similarity between the synthesized image and the corresponding labeled image is calculated using a structural filter. Reliable labeled images and their corresponding labels are then added to the training process for data augmentation.

[0026] Furthermore, in step 5), the unlabeled image and the corresponding perturbation image are input into the segmentation network to obtain the prediction probability map. Then, the prediction probability scores of the original image and its perturbation image are added together to form a classification voting histogram, and the voting matrix is ​​calculated.

[0027] Based on the voting matrix, a dual-threshold voting matrix filtering strategy is set to divide the prediction results into reliable pseudo-labels and fuzzy pseudo-labels.

[0028] Extract the prototype of the unlabeled image using reliable pseudo-labels and update the category prototype space;

[0029] Strong consistency loss is calculated based on reliable pseudo-labels, and entropy minimization loss is calculated based on fuzzy pseudo-labels, which are used to optimize the training of unlabeled branches.

[0030] Furthermore, the dual-threshold voting matrix filtering strategy specifically includes:

[0031] Set a high threshold and a low threshold. Based on the voting of each category of the pixel, if the voting score exceeds the high threshold, it is designated as a reliable pseudo-label and there can only be one. If the voting score is within the range of the high threshold and the low threshold, it is designated as a fuzzy pseudo-label and there can be at most two. If the voting score is less than the low threshold, the pixel is considered completely unreliable and is masked during the loss calculation.

[0032] Furthermore, the minimum entropy loss of the fuzzy label set is calculated based on the fuzzy pseudo-labels and used as the fuzzy loss to reduce the negative optimization caused by strong consistency constraints on the fuzzy boundary.

[0033] The beneficial effects of this invention are:

[0034] 1. This invention proposes a graph prototype learning method, which employs a prototype-based pixel-level contrastive learning strategy. Vertex features in the image graph semantic space are aligned to the closest class in the category prototype, while simultaneously using a prototype-guided contrastive learning strategy to align to the closest subclass, thus reducing the distance to other classes. This ensures both inter-class differences and intra-class diversity.

[0035] 2. This invention proposes a data augmentation (SDA) strategy with structural protection, which enriches the visual diversity of labeled samples while maintaining the semantic structure of labeled samples. It can make more effective use of unlabeled images and make the visual style of labeled samples more diverse.

[0036] 3. This invention proposes a confidence-aware dual consistency regularization voting strategy, which assigns different confidence levels to the segmentation results, uses high-confidence regions to optimize prototype extraction, and uses low-confidence regions to optimize the segmentation of fuzzy boundaries, thereby improving the segmentation accuracy of pixels with fuzzy categories. Attached Figure Description

[0037] Figure 1 This is a schematic diagram of the overall process of the present invention;

[0038] Figure 2 This is a segmentation result diagram comparing the alopecia areata dataset of this invention with existing methods;

[0039] Figure 3 This is a segmentation result diagram comparing the ISIC2018 dataset of this invention with existing methods. Detailed Implementation

[0040] The present invention will be further described below with reference to the accompanying drawings and specific embodiments, so that those skilled in the art can better understand and implement the present invention. However, the embodiments described are not intended to limit the present invention.

[0041] Reference Figure 1 As shown, an embodiment of the semi-supervised segmentation method based on graph prototype learning of the present invention is described, with the following specific steps:

[0042] Taking the alopecia areata dataset as an example, scalp images of alopecia areata patients were collected as the dataset. Professional medical personnel labeled the segmentation targets in the data. A portion of the data was used as the training and validation set, while the remainder was used as the test set. All images were uniformly cropped or scaled, and the input images were uniformly processed to a size of 512×512, divided into labeled image x... l and unlabeled image x u .

[0043] There are a total of N+M training data sets, where N represents the number of images used as the labeled dataset. in, It is the i-th image in the labeled images of the training set. Let M be the label corresponding to the i-th image in the labeled image set; M represents the M images as an unlabeled dataset. in, It is the i-th image in the unlabeled image set.

[0044] At the lowest level of the segmentation network, features are mapped from pixel space to graph semantic space. The goal is to map semantically similar regions in pixel space to the same graph node, forming a shared graph prototype representation. The graph prototypes with labeled branches are then aligned with the category prototypes at multiple levels. This allows the model to effectively capture category semantics while suppressing the effects of noise and blurred boundaries.

[0045] Specifically, an encoder is used to process the labeled image x. l and unlabeled image x u Feature extraction is performed, and the obtained features f are then soft-clustered. Let f∈R c×h×w Mapping from pixel space to graph G(Z,A), where Z is the vertex feature and A is the adjacency matrix; using a soft assignment matrix Q∈R hw×|V| For each pixel, perform soft clustering. V represents the number of vertices in graph G, which is the number of clusters. The calculation process for Q is expressed as follows:

[0046]

[0047] Where w v It is the centroid of the v-th cluster, σ v This is a scale parameter used to control the clustering degree in the pixel allocation process; exp() represents the exponential function of e. Then, vertex features are calculated. The calculation method for the v-th vertex feature is as follows:

[0048]

[0049] Calculate the adjacency matrix A based on the obtained Z, and define it as:

[0050]

[0051] Among them, Z T Let z represent the transpose of Z. Then, multi-layer graph convolution is performed, where z is the input to the l-th layer graph convolution. l and A l Its output z l+1 The calculation method is defined as follows:

[0052]

[0053] Z l+1 =Z l ·W l T ;

[0054] d is Z l The dimension, α l It is a constant.

[0055] Extract the label image x in the penultimate layer of the decoder. l The feature f, and through the label y l Obtain the category-wise binary label m k To obtain the category prototype u k Constituting the category prototype distribution space:

[0056]

[0057] The vertex features of each level graph are mapped to the centroid u of the category prototype distribution space. k* Perform matching alignment (u k* For u k (centroid of the centroid), and calculate the alignment loss L. align This ensures that the network relies more on category features during clustering. The specific matching method between vertices and categories is achieved by minimizing the Hungarian matching cost matrix, which is calculated as follows:

[0058]

[0059] Where O is the matching scheme, if vertex v matches category k, then After obtaining the matching scheme, calculate the alignment loss:

[0060]

[0061] The class prototypes of labeled images are extracted to form a prototype distribution space. Each class prototype is then clustered into several subclasses. Unlabeled images are fed into a shared segmentation network to obtain segmentation results as pseudo-labels. The class prototype of each unlabeled image is aligned with the centroid of the most similar subclass of the labeled image. This local alignment preserves individual differences while enhancing semantic consistency and reducing the risk of prototype shift.

[0062] Specifically, for the unlabeled image x u The data is fed into a segmentation network to obtain pseudo-labels. Category prototypes are then extracted based on these pseudo-labels. During this process, the pseudo-labels are considered to be related to the aforementioned y... l Same function:

[0063]

[0064] Cluster the category prototype distribution space category by category, and obtain c subcentroids for each category. And find the relationship with u k The nearest subcentroid is used as a positive sample for contrastive learning. + Meanwhile, the centroids of different categories are used as negative samples N for calculating the contrast loss. k ={u j* ,j≠k}.

[0065] Calculate the contrast loss L for a single class k contrastive :

[0066]

[0067] Where τ represents the temperature hyperparameter, it is set to 0.01.

[0068] Labeled and unlabeled images are decomposed in prototype space to obtain their respective style and structural information. The structural information of the labeled images is fused with the style information of the unlabeled images, and a structure preservation strategy is used to ensure the reliability of the synthesized images (labeled samples with multiple visual styles). The synthesized images are added to the network training to improve the robustness of the segmentation network. The synthesized images are also added to the training set as a data augmentation method.

[0069] Specifically, the mean and variance of the prototype features of labeled and unlabeled images are calculated separately. The mean and variance of the prototypes extracted from labeled images are aligned with those of the unlabeled prototypes and input into the reconstruction decoder (set in the early stages of network model building, i.e., the segmentation network includes an encoder, decoder, and reconstruction decoder) to obtain a synthetic image S with the structure of labeled images and the style of unlabeled images. u→l .

[0070] A structure filter is introduced to evaluate the synthesized image S u→l The semantic structure, which is associated with the corresponding label image x l The structural similarity score is calculated as follows:

[0071]

[0072] Where ε = 0.01 is the stability constant. and They represent x respectivelyl and S u→l The local variance of σ. 12 Their covariance is represented by a Gaussian kernel filter W with a standard deviation of 1.5. G The specific calculation process for performing local structural convolution is as follows:

[0073]

[0074] σ 12 =(x l ·S u→l *W G )-(x l *W G (S) u→l *W G );

[0075] S s The higher the score, the better. u→l With x l The better the structural consistency of X and x, the higher its reliability as labeled data in training. l Consistent. During network training, synthesized images above a threshold of 0.8 are considered reliable, and their corresponding x values ​​are assigned accordingly. l The labels are added to the training as a data augmentation technique to improve the robustness of the segmentation network.

[0076] Different weak perturbations are applied to the input unlabeled image to obtain the corresponding weakly perturbated image. This image, along with the original image (i.e., the corresponding unlabeled image), is input into the segmentation network to obtain multiple prediction results. Subsequently, a voting strategy is used to determine pseudo-labels with two different confidence levels using two thresholds. The high-confidence pseudo-labels are used to constrain the extraction of prototypes, while the low-confidence regions are used to optimize the segmentation of fuzzy boundaries. Different regularization methods are applied to pseudo-labels with different confidence levels to avoid strong consistency from negatively impacting the network at fuzzy boundaries.

[0077] Specifically, x u And its i perturbation images respectively obtain the predicted probability map p u , Then, by summing the predicted probability scores of the original image and its perturbed image to form a classification voting histogram, the voting matrix E is calculated. p ∈R h ×w×k .

[0078] Based on the voting matrix, this invention proposes a dual-threshold voting matrix filtering strategy to classify prediction results into reliable pseudo-labels and fuzzy pseudo-labels. Specifically, it includes employing a dual-threshold filtering strategy with one high and one low threshold on the voting matrix, i.e., a high threshold T... highand low threshold T low This strategy divides predictions into reliable and fuzzy pseudo-labels. For each pixel (i,j), if the voting score of some category k exceeds a high threshold T... high If so, then this type is selected as the reliable pseudo-label. Based on the theoretical upper limit of voting scores, at most one class can exceed this threshold. If no class meets the high threshold, then it belongs to class T. low To T high Within the range, scores from a maximum of two classes are allowed to fall within T. low To T high Between. Then designate these classes as fuzzy pseudo-labels. That is, using a moderate confidence level to represent an uncertain prediction, if it is less than T low If the pixel is not found to be trustworthy, it will be masked during the loss calculation.

[0079] Reliable pseudo-labels and fuzzy pseudo-labels are defined using a dual threshold system. For the prototype sampling process of unlabeled data, use As a pseudo-label, it ensures the accuracy of the prototype's category (not belonging to). (The pixels will be masked during the prototype sampling process).

[0080] Based on reliable pseudo-labels The strong consistency loss is calculated as the reliability loss and used to optimize the unlabeled branch. The specific loss calculation is as follows:

[0081]

[0082] in, N represents the voting scores for each category at pixel positions i and j. p This represents the upper limit of the score.

[0083] Calculate the weight of each pixel in each fuzzy category based on the fuzzy pseudo-label. The specific calculation method is as follows:

[0084]

[0085] To mitigate the negative optimization problem that may arise from imposing strong consistency constraints on fuzzy boundaries, a fuzzy pseudo-label-based approach is proposed. The minimum entropy loss of the fuzzy set is calculated as the fuzzy loss, which is used to solve the problem of category uncertainty at the fuzzy boundary. The specific loss calculation is as follows:

[0086]

[0087] Among them, C i，j Let i be the set of fuzzy labels belonging to pixels i and j.

[0088] Finally, calculate the total loss:

[0089] L total =L sup +λ1L align +λ2L contrastive +λ3(L reliable +L pending )

[0090] λ1, λ2, and λ3 are hyperparameters set based on experience.

[0091] After calculating the total loss, the network parameters are updated to enable the network to perform well in segmentation on datasets with only a small number of labeled data, thereby improving the robustness of the method.

[0092] The technical solutions of the present invention will be further described and explained below with reference to specific embodiments.

[0093] Reference Figure 1 The diagram shown is an overall flowchart of the present invention. The dataset used in this invention includes two sources:

[0094] The alopecia areata dataset was sourced from a clinical hospital. A total of 971 images from 281 patients with alopecia areata were collected via digital photography, using four normalized views: vertex (top), occipital (back), left lateral, and right lateral. Each image was resized to 512×512 pixels. All images were annotated by experienced dermatologists into five categories: background, hair (H), mild alopecia (Mi), moderate alopecia (Mo), and complete or severe alopecia (S). Of the dataset, 750 images were used for training, 91 for validation, and 130 for testing. There was no patient overlap in the data folds.

[0095] The ISIC 2018 Skin Lesion Segmentation Dataset is a dermoscopic image segmentation dataset released by the International Society for Dermatologic Imaging (ISIC) in its 2018 challenge. This dataset contains 2,594 high-quality dermoscopic images, each accompanied by a manually annotated binary segmentation mask. The images originate from various devices and clinical environments, covering a wide range of skin colors, lesion types (such as melanoma and benign nevi), lighting conditions, and image resolutions, demonstrating strong diversity and clinical representativeness. The dataset is divided into training and testing sets. The original images are stored in JPEG format, and the corresponding masks are saved in PNG format. In the experiments, 2000 images were used for training, 194 for validation, and 400 for testing. All input images were uniformly resized to 256×256 pixels.

[0096] Experimental environment:

[0097] This invention trains and tests the model on an NVIDIA RTX3090. The entire network is trained 300 times on both datasets with a learning rate of 0.001. The SGD optimizer is used to train the segmentation model. The batch size for the bald patch dataset is 6, and the batch size decays every 25 epochs with a coefficient of 0.9. UNet, based on ResNet50, is used as the encoder.

[0098] The experimental results of this invention are as follows:

[0099] To quantitatively evaluate the performance of the method proposed in this invention, the Dice similarity coefficient (DSC), an evaluation metric, was selected to assess the segmentation performance of the network. Its calculation formula is as follows:

[0100]

[0101] Where, N TP N is the number of true positive segmentation pixels. FN N is the number of segmented pixels that produce false positives. FP It represents the number of pixels that segmented the data as false negatives.

[0102] The present invention compares the segmentation results with existing semi-supervised models, as shown in Tables 1 and 2. Table 1 compares the present invention with other state-of-the-art methods on the alopecia areata dataset, and Table 2 compares the present invention with other methods on the ISIC2018 dataset.

[0103] Table 1.

[0104]

[0105]

[0106] Table 2.

[0107]

[0108]

[0109] As shown in Table 1, the method of this application achieved the highest average Dice score on the AA dataset across all labeled proportions. Specifically, with only 5% labeled data, the method of this application outperformed the second-best method by 1.2%; with 10% labeled data, it outperformed the second-best method by 5.4%, with a 3.6% improvement for severe baldness (S); and with 30% labeled data, the improvement remained significant at 4.0%, with a 5.3% improvement for mild baldness (Mi).

[0110] As shown in Table 2, the method in this application achieved a higher average Dice score. Specifically, on the ISIC2018 dataset, using 5% labeled data for training, the method in this application outperformed the second-best method on Dice by 0.5%; with 10% labeled data, the performance was 0.6% better; and with 30% labeled data, the performance was 0.5% better.

[0111] Figure 2 and Figure 3 These are segmentation results of the alopecia areata dataset and the ISIC2018 dataset compared with other methods. Figure 2 The first line shows that the method in this application has better segmentation results on samples with reflective noise. Figure 2 As can be seen from the second line, the method of this application can more accurately locate the boundary in the case of a completely bald sample. Figure 2 The fourth line shows that the method in this application has a better segmentation effect when the category boundaries are blurred. Figure 3 As can be seen from the first line, when the segmentation target itself has diversity, the method of this application shows better segmentation results. Figure 3 The second line demonstrates that the method of this application also exhibits superior performance on small and fuzzy targets. Figure 3 The third line indicates that when interfered with by hair or other factors, the method of this application can more accurately locate the segmentation target.

[0112] Figure 2 The performance comparison of the present invention on the alopecia areata dataset is shown. The first row shows that the method of the present application has good noise resistance in the presence of reflective noise. The second row shows that the method of the present application has good boundary localization ability in the absence of obvious boundaries. At the same time, in the fourth row, the method of the present application shows good performance in the case of diffuse hair loss and other categories with messy boundaries and large differences within the categories.

[0113] Figure 3 The performance comparison of this invention on the ISIC dataset is shown. The first row demonstrates that even when there are significant differences within the target category, the method of this application can still completely identify the category region. The second row shows that when the category boundary is blurred or indistinct, the method of this application can locate the boundary better.

[0114] The embodiments described above are merely preferred embodiments for fully illustrating the present invention, and the scope of protection of the present invention is not limited thereto. Equivalent substitutions or modifications made by those skilled in the art based on the present invention are all within the scope of protection of the present invention.

Claims

1. A semi-supervised segmentation method based on graph prototype learning, characterized in that, Includes the following steps: Step 1) Collect scalp images of patients with alopecia areata as a dataset, label the segmentation targets in the dataset, and divide the dataset into a training set, which includes labeled images and unlabeled images. Step 2) Construct category prototypes based on labeled images. Map the image features extracted by the encoder from pixel space to graph semantic space to obtain graph prototypes. Compress the graph prototypes to the number of categories through multi-layer graph convolution and fully connected layers. Align the graph prototypes with labeled branches with the category prototypes at multiple levels. Step 3) Cluster each category prototype into several subclasses, feed the unlabeled image into the segmentation network to obtain the segmentation result as a pseudo label, and align the category prototype of each unlabeled image with the centroid of the most similar subclass of the labeled image. Step 4) Decompose the labeled and unlabeled images at the prototype level to obtain their respective style and structural information, fuse the structural information of the labeled images with the style information of the unlabeled images, and add the synthesized image to the segmentation network for training. Step 5) Apply different weak perturbations to the input unlabeled image to obtain several corresponding weakly perturbated images. Combine the weakly perturbated images and the original image into an unlabeled image set and input it into the segmentation network for training to obtain multiple prediction results. Subsequently, a voting strategy was used to determine two pseudo-labels with different confidence levels, which were then used to optimize the segmentation model and category prototype sampling. Step 6) Calculate the total loss based on the labeled image, the synthesized image, and the unlabeled image, update the network parameters, enhance the robustness of the network, and obtain a semi-supervised segmentation network.

2. The semi-supervised segmentation method based on graph prototype learning as described in claim 1, characterized in that, In step 1), the images in the training set, test set, and validation set are cropped or scaled to a uniform size.

3. The semi-supervised segmentation method based on graph prototype learning as described in claim 1, characterized in that, In step 1), scalp images of N+M patients with alopecia areata are collected as a dataset, where N represents the number of images used as a labeled dataset. in, It is the i-th image in the labeled images of the training set. Let be the label corresponding to the i-th image in the labeled image set; M represents M images as an unlabeled dataset. in, It is the i-th image in the unlabeled image set.

4. The semi-supervised segmentation method based on graph prototype learning as described in claim 1, characterized in that, In step 2), the encoder of the segmentation network is used to extract features from the labeled and unlabeled images to obtain features f; Perform soft clustering on feature f, mapping it to a graph, and calculate its vertex features; Calculate the corresponding adjacency matrix based on the obtained vertex features, then perform multi-layer graph convolution and fully connected layers, and compress the number of nodes to be the same as the number of classes: The penultimate layer of the decoder in the segmentation network extracts the features f and category label m of the labeled image. k , where k represents the k-th category, to obtain category prototypes of labeled images of multiple categories; Align the vertex features of the graph to the centroids of the class prototype space at multiple levels, and calculate the alignment loss.

5. The semi-supervised segmentation method based on graph prototype learning as described in claim 4, characterized in that, In step 3), the unlabeled image is fed into the segmentation network to obtain pseudo-labels. Based on the obtained pseudo-labels, the category prototypes are extracted to obtain the category prototypes of the unlabeled image to be aligned. Perform category-by-category clustering on the category prototypes of the labeled images obtained in step 2), and obtain several sub-centroids for each category; Find the subcentroid that is closest to the class prototype to be aligned with the unlabeled image as a positive sample for contrastive learning, and use the centroids of different classes as negative samples for calculating the contrastive loss; Calculate the contrastive loss between the unlabeled image to be aligned with the class prototype and the positive and negative samples.

6. The semi-supervised segmentation method based on graph prototype learning as described in claim 1, characterized in that, In step 4), the mean and variance of the prototype features of the labeled image and the unlabeled image are calculated respectively. The mean and variance of the prototype extracted from the labeled image are aligned with the prototype of the unlabeled image and input into the reconstruction decoder to obtain a synthetic image with structural information of the labeled image and style information of the unlabeled image. The semantic structural similarity between the synthesized image and the corresponding labeled image is calculated using a structural filter. Reliable labeled images and their corresponding labels are then added to the training process for data augmentation.

7. The semi-supervised segmentation method based on graph prototype learning as described in claim 1, characterized in that, In step 5), the unlabeled image and the corresponding perturbation image are input into the segmentation network to obtain the prediction probability map. Then, the prediction probability scores of the original image and its perturbation image are added together to form a classification voting histogram, and the voting matrix is ​​calculated. Based on the voting matrix, a dual-threshold voting matrix filtering strategy is set to divide the prediction results into reliable pseudo-labels and fuzzy pseudo-labels. Extract category prototypes from unlabeled images using reliable pseudo-labels and optimize category prototype sampling; Strong consistency loss is calculated based on reliable pseudo-labels, and entropy minimization loss is calculated based on fuzzy pseudo-labels, which are used to optimize the training of unlabeled branches.

8. The semi-supervised segmentation method based on graph prototype learning as described in claim 7, characterized in that, The dual-threshold voting matrix filtering strategy specifically includes: Set a high threshold and a low threshold. Based on the voting of each category of the pixel, if the voting score exceeds the high threshold, it is designated as a reliable pseudo-label and there can only be one. If the voting score is within the range of the high threshold and the low threshold, it is designated as a fuzzy pseudo-label and there can be at most two. If the voting score is less than the low threshold, the pixel is considered completely unreliable and is masked during the loss calculation.

9. The semi-supervised segmentation method based on graph prototype learning as described in claim 7, characterized in that, The minimum entropy loss of the fuzzy label set is calculated based on the fuzzy pseudo-label and used as the fuzzy loss to reduce the negative optimization caused by strong consistency constraints on the fuzzy boundary.

Images