A semi-supervised multi-class medical image semantic segmentation method based on prototype contrast learning

Abstract

The application discloses a kind of based on prototype contrast learning's semi-supervised multiclass medical image semantic segmentation method, comprising: again labeled image dataset and unlabeled image dataset are divided into training sample set and test sample set;Deep learning model of constructing semi-supervised semantic segmentation, the class prototype feature library of main branch network is constructed, the total loss function of deep learning model is constructed, training sample set is input into the deep learning model and is trained to obtain semantic segmentation model, wherein random rotation and random flip data enhancement scheme are applied in each round of training;The deep learning model that has been trained is tested using test sample, and the test sample image segmentation result is output.This method reduces the workload of doctors needing to annotate data in daily medical image processing tasks, solves the problem of data annotation scarcity widely existing in the field of medical image processing.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of artificial intelligence, and particularly relates to a semi-supervised multi-class medical image semantic segmentation method based on prototype contrast learning. BACKGROUND

[0002] Multi-class medical image segmentation plays an irreplaceable role in modern clinical practice and can be applied to multiple clinical tasks such as disease diagnosis and disease monitoring. The core is to accurately classify and segment different tissues, organs or lesion regions in medical images to provide accurate and insightful diagnostic basis for clinicians. Compared with traditional single-class segmentation, multi-class segmentation can simultaneously identify and distinguish multiple anatomical structures or pathological features of interest, greatly improving the depth and breadth of image analysis.

[0003] At present, the low precision of multi-class medical image semantic segmentation based on deep learning mainly exists in the following three reasons: (1) low foreground and background distinction degree leads to fuzzy segmentation boundary; (2) there are similar foreground categories, and the features of these categories are difficult to distinguish; (3) because the training samples are often taken by different machines or using different parameters, thereby leading to the problem of low intra-class consistency of samples in the same training sample set. For the problems of low foreground and background distinction degree and similar foreground categories, contrast learning is an effective solution, but existing related methods cannot simultaneously consider the problem of low intra-class consistency of samples in the same training sample set.

[0004] In addition, medical image data labeling often requires a lot of manpower and time, and requires that the labeling personnel must have relevant medical knowledge, therefore, the problem of scarcity of medical image data labeling is particularly significant, and the scarcity of data labeling greatly limits the improvement of the precision of medical image semantic segmentation. Recently, semi-supervised learning has made significant progress in solving the problem of data labeling scarcity. These methods train a small amount of labeled data and a large amount of unlabeled data, and achieve performance close to or even surpassing that of fully supervised learning in the medical image segmentation task. For example, a semi-supervised medical image segmentation model based on uncertainty correction pyramid consistency regularization, a teacher-student network based on uncertainty estimation, a semi-supervised medical image segmentation method based on decoupled consistency, etc. However, the current semi-supervised medical image segmentation algorithm is mainly applied to the segmentation task of a single class, and the algorithm for multi-class medical image segmentation task is very limited. SUMMARY

[0005] In view of the problems in the prior art, the application discloses a semi-supervised multi-class medical image semantic segmentation method based on prototype contrast learning, which aims to solve the problem of low semantic segmentation accuracy in a multi-class medical image semantic segmentation task. The method uses a semi-supervised learning framework, uses prototype contrast learning to promote inter-class separation and force intra-class compactness, and simultaneously solves the problems of low segmentation boundary caused by low foreground and background differentiation, similar foreground classes, low intra-class consistency of multi-source samples in the same training sample set, and data annotation scarcity. The method specifically comprises the following steps:

[0006] The obtained multiple CT / MR slice images are preprocessed and partially annotated, so as to obtain a labeled image dataset and an unlabeled image dataset, and then the labeled image dataset and the unlabeled image dataset are divided into a training sample set and a test sample set;

[0007] A deep learning model for semi-supervised semantic segmentation is constructed, the model comprising a left main branch network and a right main branch network, each main branch network comprising an encoder and a decoder, and the two main branch networks using different initialization strategies;

[0008] A class prototype feature library of the main branch network is constructed, wherein the left main branch network comprises a class prototype feature library constructed under the guidance of labels of labeled data, and the right main branch network comprises a class prototype feature library constructed under the guidance of high-quality pseudo labels;

[0009] A total loss function of the deep learning model is constructed, wherein the loss function is constructed based on consistency loss of cross pseudo supervision of different upsampling, consistency loss of cross pseudo supervision of different initialization, contrast loss of prototype contrast learning guided by labels of labeled data, and contrast loss of prototype contrast learning guided by high-quality pseudo labels;

[0010] The training sample set is input into the deep learning model for training to obtain a semantic segmentation model, wherein a data enhancement scheme of random rotation and random flipping is applied in each round of training;

[0011] The test sample is used to test the trained deep learning model, and a test sample image segmentation result is output.

[0012] Further, when the main branch network is constructed:

[0013] Class prototype features obtained from feature maps output by the decoder of the left main branch network are stored in the class prototype feature library guided by the labels of the labeled data; and class prototype features obtained from feature maps output by the decoder of the right main branch network are stored in the class prototype feature library guided by the high-quality pseudo labels;

[0014] When the current capacity of the prototype feature library does not reach the set maximum capacity, the current prototype feature to be stored is directly stored in the corresponding prototype feature library;

[0015] When the current capacity of the prototype feature library reaches the set maximum capacity, the cosine similarity of the current prototype feature to be stored and the existing prototype feature in the corresponding prototype feature library is calculated, and the prototype feature with the highest cosine similarity is replaced by the current prototype feature to be stored.

[0016] Further, the total loss function of the deep learning model is represented as:

[0017]

[0018] Where the supervision loss is a loss directly supervised using the labels of the labeled data The consistency loss based on cross pseudo supervision includes the consistency loss based on cross pseudo supervision of different up-sampling And the consistency loss based on cross pseudo supervision of different initialization The contrast loss based on prototype contrast learning includes the contrast loss based on prototype contrast learning guided by the labels of the labeled data And the contrast loss based on prototype contrast learning guided by high-quality pseudo labels λ dc , λ TC And λ PC are three weights for balancing the respective losses.

[0019] Further, the specific steps for calculating the consistency loss based on cross pseudo supervision of different up-sampling are as follows:

[0020] SB1: For the input training sample image x, use the soft label as the supervision signal in the cross pseudo supervision method based on different up-sampling, represented as:

[0021]

[0022] Where, p(y pred |x; θ) is defined as the generation mapping probability of x, θ represents the parameters of the model f θ , f θ is a model composed of an encoder and a decoder in the same main branch network, T upsc is a hyperparameter controlling the sharpening temperature.

[0023] SB2: Use the soft label obtained by the model f θ to supervise the outputs of the other two decoders in the same main branch network;

[0024] SB3: using the three models f corresponding to the left main branch network θ Repeat step SB2 respectively, cross-supervision, using the mean square error to quantify the difference, denoted as:

[0025]

[0026] wherein is the model corresponding to the left main branch network, θ i and θ i indicate that different up-sampling methods are used;

[0027] SB4: using the three models f corresponding to the right main branch network θ Repeat step SB2 respectively, cross-supervision, using the mean square error to quantify the difference, denoted as:

[0028]

[0029] wherein is the model corresponding to the right main branch network, θ i and θ i indicate that different up-sampling methods are used;

[0030] SB5: calculate the consistency loss of the cross pseudo-supervision based on different up-sampling, denoted as:

[0031]

[0032] Further, calculate the consistency loss of the cross pseudo-supervision based on different initialization The specific steps include:

[0033] SC1: for the input training sample image x, using the class probability map obtained by one-hot encoding the output of the model or as a supervision signal, denoted as:

[0034]

[0035] wherein K is the total number of classes (including the background class);

[0036] SC2: using the class probability map obtained by the model corresponding to the left main branch network as a supervision signal, supervising the output of the model corresponding to the right main branch network using the same up-sampling method, and performing the above operation on the three models corresponding to the left main branch network respectively, using entropy loss and dice loss to quantify the difference, denoted as:

[0037]

[0038] SC3: Use the model corresponding to the right main branch network. The obtained class probability graph As a supervisory signal, the same upsampling method is used for the model corresponding to the left main branch network. The output is supervised, and the three models corresponding to the right main branch network are respectively evaluated. The above operations are performed, and the differences are quantified using entropy loss and dice loss, expressed as follows:

[0039]

[0040] SC4: Calculate the consistency loss based on the cross-pseudo-supervision with different initializations, expressed as:

[0041]

[0042] Furthermore, the contrastive loss based on prototype contrastive learning guided by labeled data and the contrastive loss based on prototype contrastive learning guided by high-quality pseudo-labels are used to enhance inter-class separability and intra-class compactness in semantic segmentation tasks. This is achieved by comparing the class feature prototypes obtained in each round with prototypes in the class prototype feature library. For class feature prototypes obtained by the left main branch network... Where k is The category to which it belongs, the contrastive loss based on prototype contrastive learning guided by labels from labeled data, is expressed as:

[0043]

[0044] in, It is a class prototype feature library guided by the labels of the labeled data. The set of class prototype features belonging to the same category k. It is a class prototype feature library guided by the labels of labeled data and related to The set of class prototype features that do not belong to category k. yes One of the class prototype features, yes One of the prototype features is τ, which is a hyperparameter controlling the sharpening temperature, where s(F i, F j ) is the similarity in the feature space, expressed as:

[0045]

[0046] Among them, F i and F j It is a class prototype feature;

[0047] For class feature prototypes obtained from the right main branch network Where k is The corresponding category, the contrastive loss based on prototype contrastive learning guided by high-quality pseudo-labels, is expressed as follows:

[0048]

[0049] in, It is a class prototype feature library guided by high-quality pseudo-tags and The set of class prototype features belonging to the same category k. It is a class prototype feature library guided by the labels of labeled data and related to The set of class prototype features that do not belong to category k. yes One of the class prototype features, yes One of the prototype features is τ, which is a hyperparameter controlling the sharpening temperature, where s(F i ,F j ) is the similarity in the feature space, expressed as:

[0050]

[0051] Among them, F i and F j It is a prototype feature.

[0052] Furthermore, the contrastive loss based on prototype contrastive learning guided by the labels of the labeled data is used only for the labeled data in the training sample set, and is expressed as:

[0053]

[0054] Where, N L K is the number of labeled data, and K is the total number of categories; the contrastive loss based on prototype contrastive learning guided by the labels of the labeled data is used for the labeled and unlabeled data in the training sample set, and is expressed as:

[0055]

[0056] Where, N L N is the number of labeled data. U It represents the number of unlabeled data.

[0057] Furthermore, the class prototype feature library includes multiple prototype feature libraries, wherein the number of prototype feature libraries in each class prototype feature library is equal to the number of categories, each prototype feature library corresponds to one category, and each prototype feature library will be set with the same maximum capacity.

[0058] Furthermore, the decoder in the left main branch network outputs a feature map for each training sample image. Based on the labels of the marked data in the training sample images, the features corresponding to each category in the corresponding training sample images are selected. Then, the average feature corresponding to each category of each training sample image is calculated for the above features to obtain the class prototype features.

[0059] Furthermore, the decoder in the right main branch network outputs a feature map for each training sample image. The decoder output is used as a pseudo-label, and the uncertainty of prediction for each pixel in each training sample image is evaluated by information entropy. Then, an uncertainty threshold is set, and high-quality pseudo-labels are selected. The feature maps corresponding to pixels with uncertainty below the uncertainty threshold are processed. Based on the high-quality pseudo-labels, the features corresponding to each category in the corresponding training sample image are selected. Then, the average feature corresponding to each category in each training sample image is calculated for the above features to obtain the class prototype features.

[0060] By employing the aforementioned technical solutions, this invention provides a semi-supervised multi-class medical image semantic segmentation method based on prototype contrastive learning. This method utilizes a semi-supervised learning framework, aiming to minimize the deep learning model's dependence on user labels. Only a small amount of data labels from the user is required to obtain the accurate edges of the desired objects, reducing the workload of doctors in daily medical image processing tasks and solving the widespread problem of scarce data labels in the field of medical image processing. Furthermore, this method utilizes prototype contrastive learning to force inter-class separation, improving the segmentation accuracy of each class in the semi-supervised multi-class medical image semantic segmentation task, while enhancing the reliability of segmentation results for edge regions, providing more accurate and reliable judgment criteria for clinical surgery and clinical diagnosis. In addition, this method applies a class prototype feature library to the deep learning model, thereby improving the deep learning model's inclusiveness of multi-source data in medical image datasets, giving the model better generalization and robustness, and effectively alleviating the problem of class imbalance. Attached Figure Description

[0061] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0062] Figure 1 This is a flowchart of a semi-supervised multi-class medical image semantic segmentation method based on multi-guided prototype contrastive learning in this invention.

[0063] Figure 2This is a network model structure diagram of a semi-supervised multi-class medical image semantic segmentation method based on multi-guided prototype contrastive learning in this invention.

[0064] Figure 3 This is a flowchart of the method for constructing a class prototype feature library guided by tags of labeled data in this invention;

[0065] Figure 4 This is a flowchart of the method for constructing a class prototype feature library guided by high-quality pseudo-tags in this invention. Detailed Implementation

[0066] To make the technical solutions and advantages of the present invention clearer, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention:

[0067] like Figure 1 The method shown is a semi-supervised multi-class medical image semantic segmentation method based on prototype contrastive learning, which specifically includes the following steps:

[0068] S1: Preprocess several CT / MR image slices of a certain part of the body from the same shooting position, and label some of the image slices to obtain labeled datasets and unlabeled datasets. Then, divide the labeled datasets and unlabeled datasets into training sample sets and test sample sets.

[0069] S2: Constructing a deep learning model architecture for semi-supervised semantic segmentation, as follows Figure 2 As shown, it includes two main branch networks, left and right. The parameters of the two main branch networks are initialized by one kaiming initialization and two kaiming initializations, respectively. Each main branch network is an encoder-decoder structure, consisting of a shared encoder and three decoders using different upsampling methods. The three decoders use three upsampling methods: nearest neighbor interpolation, bilinear interpolation, and transposed convolution, respectively.

[0070] S3: Construct two class prototype feature libraries, each guided by labels on labeled data and high-quality pseudo-labels. The method for constructing the class prototype feature library guided by labels on labeled data is as follows: Figure 3 As shown:

[0071] S31: Obtain the feature map of each labeled data output by the decoder in the left main branch network;

[0072] S32: Get the labels from the tag data;

[0073] S33: Based on the labels of the labeled data in the training sample images, filter out the features corresponding to each category in the labeled data;

[0074] S34: Calculate the average feature corresponding to each category of each labeled data using the above features to obtain the prototype feature;

[0075] S35: Determine whether the current capacity of the prototype feature library for the corresponding category in the class prototype feature library guided by the labels of the labeled data has reached the maximum capacity V. max ;

[0076] S36: If the current capacity has not reached the maximum capacity V max The prototype features that need to be stored will be stored. Directly add it to the prototype feature library corresponding to category k in the class prototype feature library guided by the labels of the labeled data; if the current capacity has reached the maximum capacity V. max It will calculate the prototype features that need to be stored. The existing prototype features in the prototype feature library corresponding to category k in the prototype feature library guided by the labels of the labeled data. cosine similarity

[0077]

[0078] Then use cosine similarity The highest-level prototype feature is replaced with the prototype feature that needs to be stored.

[0079] Methods for building class prototype feature libraries guided by high-quality pseudo-tags, such as Figure 4 As shown:

[0080] S31: Obtain the feature map of each training sample image output by the decoder in the right main branch network;

[0081] S32: Obtain the probability vector from the decoder's output h.

[0082]

[0083] Where K is the total number of categories, including foreground and background categories. The uncertainty μ of the prediction for each pixel in each training sample image is evaluated using information entropy:

[0084]

[0085] Here, ε is a local minimum value used to prevent errors in calculating the logarithm. A higher uncertainty μ indicates a less reliable prediction result for that pixel.

[0086] S33: Determine whether the uncertainty of pixels in the training sample image is lower than the uncertainty threshold;

[0087] S34: If the pixel uncertainty is below the uncertainty threshold, the prediction of that pixel is defined as a high-quality pseudo-label; otherwise, the pixel prediction will not participate in the subsequent construction of the prototype feature library guided by the high-quality pseudo-label.

[0088] S35: Obtain the prediction results output by the decoder and combine them with the pixels selected in step S34 to obtain high-quality pseudo-labels;

[0089] S36: Select the features corresponding to each category in the corresponding training sample images based on high-quality pseudo-labels;

[0090] S37: Calculate the average feature corresponding to each category of each training sample image using the above features to obtain the prototype feature;

[0091] S38: Determine whether the current capacity of the prototype feature library for the corresponding category in the prototype feature library guided by high-quality pseudo-labels has reached the maximum capacity V. max ;

[0092] S39: If the current capacity has not reached the maximum capacity V max The prototype features that need to be stored will be stored. Add directly to the prototype feature library corresponding to category k in the class prototype feature library guided by high-quality pseudo-labels; if the current capacity has reached the maximum capacity V. max It will calculate the prototype features that need to be stored. The existing prototype features in the prototype feature library corresponding to category k in the prototype feature library guided by high-quality pseudo-labels. cosine similarity

[0093]

[0094] Then use cosine similarity The highest-level prototype feature is replaced with the prototype feature that needs to be stored.

[0095] S4: Construct the total loss function for the entire process Including monitoring losses Consistency loss based on cross-pseudo-supervision with different upsampling methods Consistency loss based on cross-pseudo-supervision with different initializations Contrastive loss based on prototype contrastive learning guided by labels from labeled data And contrastive loss based on prototype contrastive learning guided by high-quality pseudo-labels. Represented as:

[0096]

[0097] Where, λdc , λ TC and λ PC These are three weights that balance the corresponding losses;

[0098] S5: A semantic segmentation model is trained on the training sample set using the deep learning model architecture, and a data augmentation scheme of random rotation and random flipping is applied in each round of training.

[0099] S6: Test the trained model using test samples and output the segmentation results of the test sample images.

[0100] In step S4, monitoring loss Direct supervision of labeled data using the labels of labeled data is represented as:

[0101]

[0102] Where p(y pred |x t ;θ) is defined as x t The generation mapping probability, θ represents the model f θ The parameter, f θ It is a model consisting of an encoder and a decoder in the same main branch network. This is the model corresponding to the left main branch network. This is the model corresponding to the right main branch network, N L It is the number of labeled data, y l These are labels used to mark data.

[0103] In step S3, the class prototype feature library aims to more comprehensively summarize the characteristics of each category in the dataset by having one category correspond to multiple prototypes, improve the inclusiveness of prototypes for multi-source samples, and further solve the problem of low intra-class consistency of multi-source samples in multi-class medical image semantic segmentation tasks by combining prototype contrastive learning.

[0104] The prototype feature library guided by high-quality pseudo-labels and the prototype feature library guided by labeled data provide positive and negative samples for the prototype contrastive learning method based on high-quality pseudo-labels and the prototype contrastive learning method based on labeled data in step S4, respectively. The high-quality pseudo-labels in the prototype feature library guided by high-quality pseudo-labels are obtained by filtering the output of the decoder in the right main branch network based on uncertainty estimation. The prototype feature library constructed using high-quality pseudo-labels is more accurate and can effectively reduce the negative impact of noise sampling on subsequent prototype contrastive learning.

[0105] In step S4, a double-cross pseudo-supervision is used to strengthen the perturbation and improve the effect of semi-supervised learning.

[0106] One aspect of dual cross-pseudo-supervision is cross-pseudo-supervision based on different upsampling methods, used to fuse information from multiple perspectives. For the input training sample image x, soft labels are used... The signal used as the supervision signal in the cross-pseudo-supervision method based on different upsampling is expressed as:

[0107]

[0108] Where p(y pred |x;θ) is defined as the generation mapping probability of x, and θ represents the model f θ The parameter, f θ The model consists of an encoder and a decoder in the same main branch network, T upsc It is a hyperparameter that controls the sharpening temperature. It is derived from model f. θ Received soft tags The outputs of the other two decoders in the same main branch network are supervised separately. The three models f corresponding to the left main branch network are used respectively. θ Cross-supervision is performed on the outputs of the other two decoders in the same main branch network, and the differences are quantized using mean squared error, as follows:

[0109]

[0110] in This is the model corresponding to the left main branch network, θ i and θ i This indicates the use of different upsampling methods; using the three models f corresponding to the right main branch network respectively. θ Cross-supervision is performed on the outputs of the other two decoders in the same main branch network, and the differences are quantized using mean squared error, as follows:

[0111]

[0112] in This is the model corresponding to the right main branch network, θ i and θ i This indicates the use of different upsampling methods; finally, the consistency loss of the cross-pseudo-supervision based on different upsampling is calculated, and expressed as:

[0113]

[0114] Another aspect of double cross-supervision is that, for the input training sample image x, the model uses... or The class probability map obtained by one-hot encoding the output is used as a supervision signal and is represented as follows:

[0115]

[0116] Where K is the total number of categories (including the background category); the model corresponding to the left main branch network is used. The obtained class probability graph As a supervisory signal, the same upsampling method is used for the model corresponding to the right main branch network. The output is supervised, and the three models corresponding to the left main branch network are respectively evaluated. The above operations are performed, and the differences are quantified using entropy loss and dice loss, expressed as follows:

[0117]

[0118] Use the model corresponding to the right main branch network. The obtained class probability graph As a supervisory signal, the same upsampling method is used for the model corresponding to the left main branch network. The output is supervised, and the three models corresponding to the right main branch network are respectively evaluated. The above operations are performed, and the differences are quantified using entropy loss and dice loss, expressed as follows:

[0119]

[0120] Finally, the consistency loss based on the cross-pseudo-supervision with different initializations is calculated and expressed as:

[0121]

[0122] In step S4, a contrastive loss based on multi-guided prototype contrastive learning is adopted. By constructing positive and negative samples using a class prototype feature library, the prototype features obtained in each round of training are compared with the positive and negative samples to enhance intra-class compactness and inter-class separability. This solves the problems of low foreground-background distinction leading to blurred segmentation boundaries and the existence of similar foreground categories that are difficult to distinguish between category features in multi-class medical image semantic segmentation tasks.

[0123] One aspect of multi-guided prototype contrastive learning is prototype contrastive learning guided by labels on labeled data. This involves class feature prototypes obtained from the left main branch network. Where k is The category to which it belongs, the contrastive loss based on prototype contrastive learning guided by labels from labeled data, is expressed as:

[0124]

[0125] in, It is a class prototype feature library guided by the labels of the labeled data. The set of class prototype features belonging to the same category k. It is a class prototype feature library guided by the labels of labeled data and related to The set of class prototype features that do not belong to category k. yes One of the class prototype features, yes One of the prototype features is τ, which is a hyperparameter controlling the sharpening temperature, where s(F i ,F j ) is the similarity in the feature space, expressed as:

[0126]

[0127] Among them, F i and F j These are class prototype features. The contrastive loss based on prototype contrastive learning guided by the labels of labeled data is used only for the labeled data in the training sample set, and is expressed as:

[0128]

[0129] Where, N L K represents the number of labeled data, and K represents the total number of categories (including background categories).

[0130] Another aspect of prototype contrastive learning based on multiple guidance is prototype contrastive learning guided by high-quality pseudo-labels. This involves class feature prototypes obtained from the right main branch network. Where k is The corresponding category, the contrastive loss based on prototype contrastive learning guided by high-quality pseudo-labels, is expressed as follows:

[0131]

[0132] in, It is a class prototype feature library guided by high-quality pseudo-tags and The set of class prototype features belonging to the same category k. It is a class prototype feature library guided by the labels of labeled data and related to The set of class prototype features that do not belong to category k. yes One of the class prototype features, yes One of the prototype features is τ, which is a hyperparameter controlling the sharpening temperature, where s(F i ,F j ) is the similarity in the feature space, expressed as:

[0133]

[0134] Among them, Fi and F j These are class prototype features. A contrastive loss based on prototype contrastive learning guided by the labels of labeled data is used for both labeled and unlabeled data in the training sample set, and is expressed as:

[0135]

[0136] Where, N L N is the number of labeled data. U K is the number of unlabeled data, and K is the total number of categories (including background categories).

[0137] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A semi-supervised multi-class medical image semantic segmentation method based on prototype contrastive learning, characterized in that... include: Multiple CT / MR slice images were preprocessed and partially annotated to obtain labeled image datasets and unlabeled image datasets. The labeled image datasets and unlabeled image datasets were then divided into training sample sets and test sample sets. A semi-supervised deep learning model for semantic segmentation is constructed, which includes a left main branch network and a right main branch network. Each main branch network includes an encoder and a decoder, and the two main branch networks adopt different initialization strategies. Construct a class prototype feature library for the main branch network, wherein the left main branch network includes a class prototype feature library built guided by labels of labeled data, and the right main branch network includes a class prototype feature library built guided by high-quality pseudo-labels. Construct the total loss function of the deep learning model, which is based on the consistency loss of cross pseudo-supervision with different upsampling, the consistency loss of cross pseudo-supervision with different initialization, the contrastive loss of prototype contrastive learning guided by the labels of the labeled data, and the contrastive loss of prototype contrastive learning guided by high-quality pseudo-labels. The training sample set is input into the deep learning model for training to obtain the semantic segmentation model, in which random rotation and random flipping data augmentation schemes are applied in each round of training; The trained deep learning model is tested using test samples, and the test sample image segmentation results are output.

2. The semi-supervised multi-class medical image semantic segmentation method based on prototype contrastive learning according to claim 1, characterized in that: When building the main branch network: The class prototype features obtained from the feature map output by the decoder of the left main branch network are stored in a class prototype feature library guided by the labels of the labeled data; the class prototype features obtained from the feature map output by the decoder of the right main branch network are stored in a class prototype feature library guided by high-quality pseudo-labels. When the current capacity of the prototype feature library has not reached the set maximum capacity, the prototype features that need to be stored are directly stored in the corresponding prototype feature library. When the current capacity of the prototype feature library reaches the set maximum capacity, calculate the cosine similarity between the prototype feature to be stored and the corresponding existing prototype feature in the prototype feature library, and then replace the prototype feature with the highest cosine similarity with the prototype feature to be stored.

3. The semi-supervised multi-class medical image semantic segmentation method based on prototype contrastive learning according to claim 1, characterized in that: The total loss function of the deep learning model is expressed as: The supervision loss is the loss incurred by directly supervising the labeled data using the labels. Consistency loss based on cross-pseudo-supervision includes consistency loss based on cross-pseudo-supervision with different upsampling methods. Consistency loss based on cross-pseudo-supervision with different initializations Contrastive loss based on prototype contrastive learning includes contrastive loss based on label-guided prototype contrastive learning using labeled data. Comparative loss based on prototype contrastive learning guided by high-quality pseudo-labels λ dc , λ TC and λ PC These are the three weights that balance the corresponding losses.

4. The semi-supervised multi-class medical image semantic segmentation method based on prototype contrastive learning according to claim 3, characterized in that: Calculate the consistency loss based on cross-pseudo-supervision with different upsampling. Specific steps: SB1: For the input training sample image x, use soft labels. The signal used as the supervision signal in the cross-pseudo-supervision method based on different upsampling is expressed as: Where p(y pred |x;θ) is defined as the generation mapping probability of x, and θ represents the model f θ The parameter, f θ The model consists of an encoder and a decoder in the same main branch network, T upsc It is a hyperparameter that controls the sharpening temperature; SB2: Using model f θ Received soft tags The outputs of the other two decoders in the same main branch network are supervised separately; SB3: Using the three models f corresponding to the left main branch network θ Repeat step SB2 for cross-supervision, and quantify the differences using mean squared error, as follows: in This is the model corresponding to the left main branch network, θ i and θ i This indicates the use of different upsampling methods; SB4: Using the three models f corresponding to the right main branch network θ Repeat step SB2 for cross-supervision, and quantify the differences using mean squared error, as follows: in This is the model corresponding to the right main branch network, θ i and θ i This indicates the use of different upsampling methods; SB5: Calculate the consistency loss based on the cross-pseudo-supervision using different upsampling methods, expressed as:

5. The semi-supervised multi-class medical image semantic segmentation method based on prototype contrastive learning according to claim 3, characterized in that: Calculate the consistency loss based on cross-pseudo-supervision with different initializations. The specific steps include: SC1: For the input training sample image x, use the model... or The class probability map obtained by one-hot encoding the output is used as a supervision signal and is represented as follows: Where K is the total number of categories, including the background category; SC2: Use the model corresponding to the left main branch network. The obtained class probability graph As a supervisory signal, the same upsampling method is used for the model corresponding to the right main branch network. The output is supervised, and the three models corresponding to the left main branch network are respectively evaluated. The above operations are performed, and the differences are quantified using entropy loss and dice loss, expressed as follows: SC3: Use the model corresponding to the right main branch network. The obtained class probability graph As a supervisory signal, the same upsampling method is used for the model corresponding to the left main branch network. The output is supervised, and the three models corresponding to the right main branch network are respectively evaluated. The above operations are performed, and the differences are quantified using entropy loss and dice loss, expressed as follows: SC4: Calculate the consistency loss based on the cross-pseudo-supervision with different initializations, expressed as:

6. The semi-supervised multi-class medical image semantic segmentation method based on prototype contrastive learning according to claim 3, characterized in that: The contrastive loss based on prototype contrastive learning guided by labeled data and the contrastive loss based on prototype contrastive learning guided by high-quality pseudo-labels are used to enhance inter-class separability and intra-class compactness in semantic segmentation tasks. This is achieved by comparing the class feature prototypes obtained in each round with prototypes in the class prototype feature library. For class feature prototypes obtained from the left main branch network... Where k is The category to which it belongs, the contrastive loss based on prototype contrastive learning guided by labels from labeled data, is expressed as: in, It is a class prototype feature library guided by the labels of the labeled data. The set of class prototype features belonging to the same category k. It is a class prototype feature library guided by the labels of labeled data and related to The set of class prototype features that do not belong to category k. yes One of the class prototype features, yes One of the prototype features is τ, which is a hyperparameter controlling the sharpening temperature, where s(F i ,F j ) is the similarity in the feature space, expressed as: Among them, F i and F j It is a class prototype feature; For class feature prototypes obtained from the right main branch network Where k is The corresponding category, the contrastive loss based on prototype contrastive learning guided by high-quality pseudo-labels, is expressed as follows: in, It is a class prototype feature library guided by high-quality pseudo-tags and The set of class prototype features belonging to the same category k. It is a class prototype feature library guided by the labels of labeled data and related to The set of class prototype features that do not belong to category k. yes One of the class prototype features, yes One of the prototype features is τ, which is a hyperparameter controlling the sharpening temperature, where s(F i ,F j ) is the similarity in the feature space, expressed as: Among them, F i and F j It is a prototype feature.

7. The semi-supervised multi-class medical image semantic segmentation method based on prototype contrastive learning according to claim 6, characterized in that: The contrastive loss based on prototype contrastive learning guided by labels from labeled data is used only for labeled data in the training sample set, and is expressed as: Where, N L K is the number of labeled data, and K is the total number of categories; the contrastive loss based on prototype contrastive learning guided by the labels of the labeled data is used for the labeled and unlabeled data in the training sample set, and is expressed as: Where, N L N is the number of labeled data. U It represents the number of unlabeled data.

8. The semi-supervised multi-class medical image semantic segmentation method based on prototype contrastive learning according to claim 1, characterized in that: The class prototype feature library includes multiple prototype feature libraries, where the number of prototype feature libraries in each class prototype feature library is equal to the number of categories. Each prototype feature library corresponds to one category, and each prototype feature library will be set with the same maximum capacity.

9. The semi-supervised multi-class medical image semantic segmentation method based on prototype contrastive learning according to claim 1, characterized in that: The decoder in the left main branch network outputs a feature map for each training sample image. Based on the labels of the marked data in the training sample images, the features corresponding to each category in the corresponding training sample images are selected. Then, the average feature corresponding to each category of each training sample image is calculated for the above features to obtain the class prototype features.

10. A semi-supervised multi-class medical image semantic segmentation method based on prototype contrastive learning according to claim 1, characterized in that: The decoder in the right main branch network outputs a feature map for each training sample image. The decoder output is used as a pseudo-label. The uncertainty of prediction for each pixel in each training sample image is evaluated by information entropy. An uncertainty threshold is then set to select high-quality pseudo-labels. The feature maps corresponding to pixels with uncertainty below the uncertainty threshold are processed. Based on the high-quality pseudo-labels, the features corresponding to each category in the corresponding training sample image are selected. The average feature corresponding to each category in each training sample image is then calculated to obtain the class prototype features.

Images