Semi-supervised multi-class medical image semantic segmentation method based on prototype comparative learning

Abstract

The invention discloses a semi-supervised multi-class medical image semantic segmentation method based on prototype comparative learning. The method comprises the following steps: dividing a marked image data set and an unmarked image data set into a training sample set and a test sample set; constructing a deep learning model of semi-supervised semantic segmentation, constructing a prototype-like feature library of a main and branch network, constructing a total loss function of the deep learning model, inputting the training sample set into the deep learning model for training to obtain a semantic segmentation model, wherein a data enhancement scheme of random rotation and random flipping is applied in each round of training; and testing the trained deep learning model by using the test sample, and outputting a test sample image segmentation result. According to the method, the workload that a doctor needs to label the data in a daily medical image processing task is reduced, and the problem that data labeling is scarce generally existing in the field of medical image processing is solved.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to a semi-supervised multi-category medical image semantic segmentation method based on prototype contrast learning. Background Art

[0002] Multi-category medical image segmentation plays an irreplaceable role in modern clinical practice and can be applied to multiple clinical tasks such as disease diagnosis and condition monitoring. Its core lies in accurately classifying and segmenting different tissues, organs or lesion areas in medical images, providing clinicians with accurate and insightful diagnostic evidence. Compared with traditional single-category segmentation, multi-category segmentation can simultaneously identify and distinguish multiple anatomical structures or pathological features of interest, greatly improving the depth and breadth of image analysis.

[0003] At present, there are three main reasons for the low accuracy of semantic segmentation of multi-category medical images based on deep learning: (1) the low distinction between foreground and background leads to blurred segmentation boundaries; (2) there are similar foreground categories, and the features of these categories are difficult to distinguish; (3) because the training samples are often taken by different machines or with different parameters, the samples in the same training sample set have low intra-class consistency. Contrastive learning is an effective solution to the problems of low distinction between foreground and background and similar foreground categories, but the existing related methods cannot simultaneously take into account the problem of low intra-class consistency of samples in the same training sample set.

[0004] In addition, medical image data annotation often requires a lot of manpower and time, and requires annotators to have relevant medical knowledge. Therefore, the problem of scarcity of medical image data annotation is particularly significant, and the scarcity of data annotation greatly limits the improvement of the accuracy of medical image semantic segmentation. Recently, semi-supervised learning has made significant progress in solving the problem of scarce data annotation. These methods achieve performance close to or even exceeding that of fully supervised learning in medical image segmentation tasks by training a small amount of labeled data and a large amount of unlabeled data. For example, semi-supervised medical image segmentation models based on uncertainty-corrected pyramid consistency regularization, teacher-student networks based on uncertainty estimation, and semi-supervised medical image segmentation methods based on decoupled consistency. However, the current semi-supervised medical image segmentation algorithms are mainly used in single-category segmentation tasks, and the algorithms for multi-category medical image segmentation tasks are very limited. Summary of the invention

[0005] In view of the problems existing in the prior art, the present invention discloses a semi-supervised multi-category medical image semantic segmentation method based on prototype contrast learning. The method solves the problem of low semantic segmentation accuracy in the multi-category medical image semantic segmentation task. The method adopts a semi-supervised learning framework and uses prototype contrast learning to promote inter-class separation and force intra-class compactness. At the same time, it solves the problems of blurred segmentation boundaries caused by low foreground and background distinction, similar foreground categories, low intra-class consistency of multi-source samples in the same training sample set, and scarce data annotation. The method specifically includes the following steps:

[0006] Preprocessing and partially annotating multiple acquired CT / MR slice images to obtain a labeled image dataset and an unlabeled image dataset, and then dividing the labeled image dataset and the unlabeled image dataset into a training sample set and a test sample set;

[0007] Construct a deep learning model for semi-supervised semantic segmentation. The model includes a left main branch network and a right main branch network. Each main branch network includes an encoder and a decoder. The two main branch networks use different initialization strategies.

[0008] Constructing a prototype feature library of the main branch network, wherein the left main branch network includes a prototype feature library guided by the label of the labeled data, and the right main branch network includes a prototype feature library guided by the high-quality pseudo label;

[0009] Constructing a total loss function of a deep learning model, wherein the loss function is constructed based on a consistency loss of cross pseudo-supervision with different upsampling, a consistency loss of cross pseudo-supervision with different initializations, a contrast loss of prototype contrastive learning guided by labels of labeled data, and a contrast loss of prototype contrastive learning guided by high-quality pseudo-labels;

[0010] The training sample set is input into the deep learning model for training to obtain a semantic segmentation model, wherein a random rotation and random flipping data augmentation scheme is applied in each round of training;

[0011] Use the test sample to test the trained deep learning model and output the test sample image segmentation result.

[0012] Furthermore, when building the main branch network:

[0013] The prototype features obtained from the feature map output by the decoder of the left main branch network are stored in a prototype feature library guided by the label of the labeled data; the prototype features obtained from the feature map output by the decoder of the right main branch network are stored in a prototype feature library guided by high-quality pseudo labels;

[0014] When the current capacity of the prototype feature library does not reach the set maximum capacity, the prototype features currently to be stored are directly stored in the corresponding prototype feature library;

[0015] When the current capacity of the prototype feature library reaches the set maximum capacity, the cosine similarity between the prototype feature currently to be stored and the corresponding prototype feature existing in the prototype feature library is calculated, and then the prototype feature with the highest cosine similarity is replaced with the prototype feature currently to be stored.

[0016] Furthermore, the total loss function of the deep learning model is expressed as:

[0017]

[0018] The supervised loss is the loss of direct supervision using the labels of the labeled data. The consistency loss based on cross pseudo supervision includes the consistency loss based on cross pseudo supervision with different upsampling and consistency loss based on cross-pseudo supervision with different initializations Contrastive loss based on prototype contrastive learning includes contrastive loss based on prototype contrastive learning guided by the labels of labeled data. and a contrastive loss based on prototype contrastive learning guided by high-quality pseudo-labels λ dc , TC and λ PC are the three weights that balance the corresponding losses.

[0019] Furthermore, the consistency loss of cross pseudo supervision based on different upsampling is calculated Specific steps:

[0020] SB1: For the input training sample image x, use soft label As the supervisory signal in the cross pseudo-supervision method based on different upsampling, it is expressed as:

[0021]

[0022] Among them, p(y pred |x;θ) is defined as the probability of generating a mapping of x, and θ represents the model f θ The parameter, f θ is a model consisting of an encoder and a decoder in the same main branch network, T upsc is a hyperparameter that controls the sharpening temperature.

[0023] SB2: Use the model f θ The obtained soft label Supervise the outputs of the other two decoders in the same main branch network respectively;

[0024] SB3: Use the three models corresponding to the left main branch network f θ Repeat step SB2 separately for cross supervision and use mean square error to quantify the difference, expressed as:

[0025]

[0026] in is the model corresponding to the left main branch network, θ i and θ i Indicates the use of different upsampling methods;

[0027] SB4: Use the three models corresponding to the right main branch network f θ Repeat step SB2 separately for cross supervision and use mean square error to quantify the difference, expressed as:

[0028]

[0029] in is the model corresponding to the right main branch network, θ i and θ i Indicates the use of different upsampling methods;

[0030] SB5: Calculate the consistency loss of the cross pseudo supervision based on different upsampling, expressed as:

[0031]

[0032] Furthermore, we calculate the consistency loss of cross-pseudo supervision based on different initializations The specific steps include:

[0033] SC1: For the input training sample image x, use the model or The class probability map obtained by one-hot encoding the output of is used as the supervisory signal and is expressed as:

[0034]

[0035] Among them, K is the total number of categories (including background categories);

[0036] SC2: Use the model corresponding to the left main branch network The resulting class probability map As a supervisory signal, the model corresponding to the right main branch network uses the same upsampling method The output of Perform the above operations and use entropy loss and dice loss to quantify the difference, expressed as:

[0037]

[0038] SC3: Use the model corresponding to the right main branch network The resulting class probability map As a supervisory signal, the model corresponding to the left main branch network uses the same upsampling method The output of Perform the above operations and use entropy loss and dice loss to quantify the difference, expressed as:

[0039]

[0040] SC4: Calculate the consistency loss of the cross pseudo-supervision based on different initializations, expressed as:

[0041]

[0042] Furthermore, the contrastive loss based on prototype contrastive learning guided by the label of the labeled data and the contrastive loss based on prototype contrastive learning guided by the high-quality pseudo-label are used to enhance the inter-class separability and intra-class compactness in the semantic segmentation task. By comparing the class feature prototype obtained in each round with the prototype in the class prototype feature library, the class feature prototype obtained by the left main branch network is Where k is The contrast loss of the prototype contrast learning guided by the label of the labeled data is expressed as:

[0043]

[0044] in, It is the class prototype feature library guided by the label of the labeled data. The set of class prototype features belonging to the same category k, is the class prototype feature library guided by the label data and The set of prototype features that do not belong to category k, yes A class prototype feature in yes is a prototype feature in , τ is a hyperparameter that controls the sharpening temperature, where s(F i, F j ) is the similarity in the feature space, expressed as:

[0045]

[0046] Among them, F i and F j It is a class prototype feature;

[0047] For the class feature prototype obtained by the right main branch network Where k is The contrast loss based on prototype contrast learning guided by high-quality pseudo labels is expressed as:

[0048]

[0049] in, It is the prototype feature library guided by high-quality pseudo labels. The set of class prototype features belonging to the same category k, is the class prototype feature library guided by the label data and The set of prototype features that do not belong to category k, yes A class prototype feature in yes is a prototype feature in , τ is a hyperparameter that controls the sharpening temperature, where s(F i ,F j ) is the similarity in the feature space, expressed as:

[0050]

[0051] Among them, F i and F j It is a class prototype feature.

[0052] Furthermore, the contrastive loss based on prototype contrastive learning guided by the label of the labeled data is only used for the labeled data in the training sample set, which is expressed as:

[0053]

[0054] Among them, N L is the number of labeled data, K is the total number of categories; the contrastive loss based on prototype contrastive learning guided by the labels of the labeled data is used for the labeled data and the unlabeled data in the training sample set, expressed as:

[0055]

[0056] Among them, N L is the number of labeled data, N U is the number of unlabeled data.

[0057] Furthermore, the class prototype feature library includes multiple prototype feature libraries, wherein the number of prototype feature libraries in each class prototype feature library is the number of categories, each prototype feature library corresponds to one category, and each prototype feature library will be set with the same maximum capacity.

[0058] Furthermore, the decoder in the left main branch network outputs a feature map of each training sample image, filters out features corresponding to each category in the corresponding training sample image according to the labels of the marked data in the training sample image, and then calculates the average features corresponding to each category of each training sample image for the above features to obtain the class prototype features.

[0059] Furthermore, the decoder in the right main branch network outputs a feature map of each training sample image, and the result of the decoder output is used as a pseudo-label. The uncertainty of the prediction of each pixel in each training sample image is evaluated by information entropy, and then an uncertainty threshold is set to select high-quality pseudo-labels. The feature maps corresponding to pixels below the uncertainty threshold are operated, and the features corresponding to each category in the corresponding training sample image are screened out according to the high-quality pseudo-labels. The average features corresponding to each category of each training sample image are then calculated for the above features to obtain the class prototype features.

[0060] Due to the adoption of the above technical solution, the present invention provides a semi-supervised multi-category medical image semantic segmentation method based on prototype contrast learning. This method adopts a semi-supervised learning framework, which aims to minimize the dependence of the deep learning model on user labels. Only a small amount of data labels are required by the user to obtain the accurate edge of the required object, which reduces the workload of doctors in daily medical image processing tasks to annotate data, and solves the problem of scarce data annotation in the field of medical image processing. In addition, this method uses prototype contrast learning to force inter-class separation, improves the segmentation accuracy of each category in the semi-supervised multi-category medical image semantic segmentation task, and enhances the reliability of the segmentation results of the segmented edge area, providing a more accurate and reliable judgment basis for clinical surgery and clinical diagnosis. In addition, this method applies the prototype feature library to the deep learning model, thereby improving the tolerance of the deep learning model to multi-source data in the medical image data set, so that the model has better generalization and robustness, and can effectively alleviate the problem of class imbalance. BRIEF DESCRIPTION OF THE DRAWINGS

[0061] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0062] Figure 1 A flowchart of a semi-supervised multi-category medical image semantic segmentation method based on multi-guided prototype contrast learning in the present invention;

[0063] Figure 2A network model structure diagram of a semi-supervised multi-category medical image semantic segmentation method based on multi-guided prototype contrast learning in the present invention;

[0064] Figure 3 It is a flow chart of the method for constructing a class prototype feature library guided by the label of the marked data in the present invention;

[0065] Figure 4 This is a flowchart of the method for constructing a prototype-like feature library guided by high-quality pseudo-labels in the present invention. DETAILED DESCRIPTION

[0066] In order to make the technical solutions and advantages of the present invention more clear, the technical solutions in the embodiments of the present invention are clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present invention:

[0067] like Figure 1 A semi-supervised multi-category medical image semantic segmentation method based on prototype contrast learning is shown, which specifically includes the following steps:

[0068] S1: Preprocess several CT / MR image slices of a certain part at the same shooting orientation, and annotate some image slices to obtain a labeled data set and an unlabeled data set, and then divide the labeled data set and the unlabeled data set into a training sample set and a test sample set;

[0069] S2: Build a deep learning model architecture for semi-supervised semantic segmentation Figure 2 As shown, it includes two main branch networks on the left and right. The two main branch networks use one kaiming initialization and two kaiming initializations to initialize parameters respectively. Each main branch network is an encoder-decoder structure, consisting of a shared encoder and three decoders using different upsampling methods. The three decoders use three upsampling methods: nearest neighbor interpolation, bilinear interpolation, and transposed convolution.

[0070] S3: Construct two class prototype feature libraries. The two class prototype feature libraries are respectively guided by the labels of the labeled data and the high-quality pseudo labels. The method for constructing the class prototype feature library guided by the labels of the labeled data is as follows: Figure 3 As shown:

[0071] S31: Obtain a feature map of each labeled data output by the decoder in the left main branch network;

[0072] S32: Obtaining labels for the marked data;

[0073] S33: filtering out features corresponding to each category in the corresponding labeled data according to the labels of the labeled data in the training sample image;

[0074] S34: using the above features to obtain the average features corresponding to each category of each labeled data, and obtain the prototype features;

[0075] S35: Determine whether the current capacity of the prototype feature library of the corresponding category in the class prototype feature library guided by the label of the marked data has reached the maximum capacity V max ;

[0076] S36: If the current capacity does not reach the maximum capacity V max , the prototype features currently to be stored Directly add to the prototype feature library corresponding to category k in the prototype feature library guided by the label of the labeled data; if the current capacity has reached the maximum capacity V max , will calculate the prototype features that need to be stored currently The prototype features already in the prototype feature library corresponding to category k in the prototype feature library guided by the labels of the labeled data The cosine similarity of

[0077]

[0078] Then the cosine similarity The highest prototype feature is replaced by the prototype feature currently to be stored

[0079] The method of constructing a prototype feature library guided by high-quality pseudo labels is as follows: Figure 4 As shown:

[0080] S31: Obtain a feature map of each training sample image output by the decoder in the right main branch network;

[0081] S32: Obtain the probability vector through the decoder output h

[0082]

[0083] Among them, K is the total number of categories, including foreground categories and background categories. The uncertainty μ of each pixel prediction in each training sample image is evaluated by information entropy:

[0084]

[0085] Among them, ε is a minimum value used to prevent errors in calculating logarithms. The higher the uncertainty μ, the less reliable the prediction result of the pixel;

[0086] S33: Determine whether the uncertainty of the pixel in the training sample image is lower than the uncertainty threshold;

[0087] S34: If the pixel uncertainty is lower than the uncertainty threshold, the prediction of the pixel is defined as a high-quality pseudo label; otherwise, the pixel prediction does not participate in the subsequent construction of the class prototype feature library guided by the high-quality pseudo label;

[0088] S35: Obtain the prediction result output by the decoder, and combine it with the pixels screened in step S34 to obtain a high-quality pseudo label;

[0089] S36: Filter out the features corresponding to each category in the corresponding training sample image according to the high-quality pseudo labels;

[0090] S37: Using the above features, calculate the average features corresponding to each category of each training sample image to obtain prototype features;

[0091] S38: Determine whether the current capacity of the prototype feature library of the corresponding category in the class prototype feature library guided by the high-quality pseudo-label has reached the maximum capacity V max ;

[0092] S39: If the current capacity does not reach the maximum capacity V max , the prototype features currently to be stored Directly add it to the prototype feature library corresponding to category k in the prototype feature library guided by high-quality pseudo labels; if the current capacity has reached the maximum capacity V max , will calculate the prototype features that need to be stored currently The existing prototype features in the prototype feature library corresponding to category k in the prototype feature library guided by high-quality pseudo labels The cosine similarity of

[0093]

[0094] Then the cosine similarity The highest prototype feature is replaced by the prototype feature currently to be stored

[0095] S4: Construct the total loss function of the whole process Including monitoring loss Consistency loss based on cross pseudo-supervision with different upsampling Cross-Pseudo-Supervision Consistency Loss with Different Initializations Contrastive loss based on prototype contrastive learning guided by labels from labeled data and a contrastive loss based on prototype contrastive learning guided by high-quality pseudo-labels It is expressed as:

[0096]

[0097] Among them, λdc , TC and λ PC are the three weights that balance the corresponding losses;

[0098] S5: training a semantic segmentation model on the training sample set by using the deep learning model architecture, and applying a random rotation and random flipping data enhancement scheme in each round of training;

[0099] S6: Use the test sample to test the trained model and output the test sample image segmentation result.

[0100] In step S4, the supervision loss The label of the labeled data is used to directly supervise the labeled data, which is expressed as:

[0101]

[0102] Among them, p(y pred |x t ; θ) is defined as x t The probability of generating a mapping, θ represents the model f θ The parameter, f θ It is a model consisting of an encoder and a decoder in the same main branch network. is the model corresponding to the left main branch network, is the model corresponding to the right main branch network, N L is the number of labeled data, y l are the labels of the labeled data.

[0103] In step S3, the class prototype feature library aims to more comprehensively summarize the characteristics of each category in the data set by corresponding multiple prototypes to one category, improve the inclusiveness of the prototype to multi-source samples, and combine with prototype comparative learning to further solve the problem of low intra-class consistency of multi-source samples in the multi-category medical image semantic segmentation task.

[0104] The prototype feature library guided by high-quality pseudo labels and the prototype feature library guided by the labels of the labeled data provide positive and negative samples for the prototype contrast learning method guided by high-quality pseudo labels and the prototype contrast learning method guided by the labels of the labeled data in step S4, respectively. Among them, the high-quality pseudo labels in the prototype feature library guided by high-quality pseudo labels are obtained by screening the output of the decoder in the right main branch network based on uncertainty estimation. The prototype feature library constructed by the guidance of high-quality pseudo labels is more accurate and can effectively reduce the negative impact of noise sampling on subsequent prototype contrast learning.

[0105] In step S4, double cross pseudo-supervision is used to strengthen the perturbation and improve the effect of semi-supervised learning.

[0106] One aspect of the double cross pseudo-supervision is the cross pseudo-supervision based on different upsampling, which is used to fuse information from multiple perspectives. For the input training sample image x, use the soft label As the supervisory signal in the cross pseudo-supervision method based on different upsampling, it is expressed as:

[0107]

[0108] Among them, p(y pred |x;θ) is defined as the probability of generating a mapping of x, and θ represents the model f θ The parameter, f θ is a model consisting of an encoder and a decoder in the same main branch network, T upsc is a hyperparameter that controls the sharpening temperature. θ The obtained soft label The outputs of the other two decoders in the same main branch network are supervised respectively. The three models f corresponding to the left main branch network are used respectively. θ The outputs of the other two decoders in the same main branch network are cross-supervised, and the difference is quantified using the mean square error, expressed as:

[0109]

[0110] in is the model corresponding to the left main branch network, θ i and θ i Indicates the use of different upsampling methods; the three models f corresponding to the right main branch network are used respectively θ The outputs of the other two decoders in the same main branch network are cross-supervised, and the difference is quantified using the mean square error, expressed as:

[0111]

[0112] in is the model corresponding to the right main branch network, θ i and θ i Indicates the use of different upsampling methods; finally, the consistency loss of the cross pseudo-supervision based on different upsampling is calculated, which is expressed as:

[0113]

[0114] Another aspect of double cross pseudo supervision is to use the input training sample image x, which is generated by the model or The class probability map obtained by one-hot encoding the output of is used as the supervisory signal and is expressed as:

[0115]

[0116] Among them, K is the total number of categories (including background categories); use the model corresponding to the left main branch network The resulting class probability map As a supervisory signal, the model corresponding to the right main branch network uses the same upsampling method The output of Perform the above operations and use entropy loss and dice loss to quantify the difference, expressed as:

[0117]

[0118] Use the model corresponding to the right main branch network The resulting class probability map As a supervisory signal, the model corresponding to the left main branch network uses the same upsampling method The output of Perform the above operations and use entropy loss and dice loss to quantify the difference, expressed as:

[0119]

[0120] Finally, the consistency loss of the cross pseudo-supervision based on different initializations is calculated and expressed as:

[0121]

[0122] In step S4, a contrast loss based on multi-guided prototype contrastive learning is adopted. By using the class prototype feature library to construct positive and negative samples, and then comparing the prototype features obtained in each round of training with the positive and negative samples, the intra-class compactness and inter-class separability are enhanced, thereby solving the problem of low foreground and background distinction in the multi-category medical image semantic segmentation task, resulting in blurred segmentation boundaries, similar foreground categories, and difficult to distinguish category features.

[0123] One aspect of prototype contrastive learning based on multi-guided learning is prototype contrastive learning based on label guidance of labeled data. Where k is The contrast loss of the prototype contrast learning guided by the label of the labeled data is expressed as:

[0124]

[0125] in, It is the class prototype feature library guided by the label of the labeled data. The set of class prototype features belonging to the same category k, is the class prototype feature library guided by the label data and The set of prototype features that do not belong to category k, yes A class prototype feature in yes is a prototype feature in , τ is a hyperparameter that controls the sharpening temperature, where s(F i ,F j ) is the similarity in the feature space, expressed as:

[0126]

[0127] Among them, F i and F j is the class prototype feature. The contrastive loss based on prototype contrastive learning guided by the labels of labeled data is only used for the labeled data in the training sample set and is expressed as:

[0128]

[0129] Among them, N L is the number of labeled data, K is the total number of categories (including background categories);

[0130] Another aspect of prototype contrastive learning based on multiple guidance is prototype contrastive learning based on high-quality pseudo labels. Where k is The contrast loss based on prototype contrast learning guided by high-quality pseudo labels is expressed as:

[0131]

[0132] in, It is the prototype feature library guided by high-quality pseudo labels. The set of class prototype features belonging to the same category k, is the class prototype feature library guided by the label data and The set of prototype features that do not belong to category k, yes A class prototype feature in yes is a prototype feature in , τ is a hyperparameter that controls the sharpening temperature, where s(F i ,F j ) is the similarity in the feature space, expressed as:

[0133]

[0134] Among them, Fi and F j is the class prototype feature. The contrastive loss based on prototype contrastive learning guided by the label of the labeled data is used for the labeled data and unlabeled data in the training sample set, expressed as:

[0135]

[0136] Among them, N L is the number of labeled data, N U is the number of unlabeled data, and K is the total number of categories (including background categories).

[0137] The above description is only a preferred specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can make equivalent replacements or changes according to the technical scheme and inventive concept of the present invention within the technical scope disclosed by the present invention, which should be covered by the protection scope of the present invention.

Claims

1. A semi-supervised multi-category medical image semantic segmentation method based on prototype contrastive learning, characterized by include: Preprocessing and partially annotating multiple acquired CT / MR slice images to obtain a labeled image dataset and an unlabeled image dataset, and then dividing the labeled image dataset and the unlabeled image dataset into a training sample set and a test sample set; Construct a deep learning model for semi-supervised semantic segmentation. The model includes a left main branch network and a right main branch network. Each main branch network includes an encoder and a decoder. The two main branch networks use different initialization strategies. Constructing a prototype feature library of the main branch network, wherein the left main branch network includes a prototype feature library guided by the label of the labeled data, and the right main branch network includes a prototype feature library guided by the high-quality pseudo label; Constructing a total loss function of a deep learning model, wherein the loss function is constructed based on a consistency loss of cross pseudo-supervision with different upsampling, a consistency loss of cross pseudo-supervision with different initializations, a contrast loss of prototype contrastive learning guided by labels of labeled data, and a contrast loss of prototype contrastive learning guided by high-quality pseudo-labels; The training sample set is input into the deep learning model for training to obtain a semantic segmentation model, wherein a random rotation and random flipping data augmentation scheme is applied in each round of training; Use the test sample to test the trained deep learning model and output the test sample image segmentation result.

2. The method for semi-supervised multi-category medical image semantic segmentation based on prototype contrastive learning according to claim 1, characterized in that: When building a main branch network: The prototype features obtained from the feature map output by the decoder of the left main branch network are stored in a prototype feature library guided by the label of the labeled data; the prototype features obtained from the feature map output by the decoder of the right main branch network are stored in a prototype feature library guided by high-quality pseudo labels; When the current capacity of the prototype feature library does not reach the set maximum capacity, the prototype features currently to be stored are directly stored in the corresponding prototype feature library; When the current capacity of the prototype feature library reaches the set maximum capacity, the cosine similarity between the prototype feature currently to be stored and the corresponding prototype feature existing in the prototype feature library is calculated, and then the prototype feature with the highest cosine similarity is replaced with the prototype feature currently to be stored.

3. The method for semi-supervised multi-category medical image semantic segmentation based on prototype contrastive learning according to claim 1, characterized in that: The total loss function of the deep learning model is expressed as: The supervised loss is the loss of direct supervision using the labels of the labeled data. The consistency loss based on cross pseudo supervision includes the consistency loss based on cross pseudo supervision with different upsampling and consistency loss based on cross-pseudo supervision with different initializations Contrastive loss based on prototype contrastive learning includes contrastive loss based on prototype contrastive learning guided by the labels of labeled data. and a contrastive loss based on prototype contrastive learning guided by high-quality pseudo-labels λ dc , TC and λ PC are the three weights that balance the corresponding losses.

4. The method for semi-supervised multi-category medical image semantic segmentation based on prototype contrastive learning according to claim 3, characterized in that: Compute the consistency loss of cross pseudo-supervision based on different upsampling Specific steps: SB1: For the input training sample image x, use soft label As the supervisory signal in the cross pseudo-supervision method based on different upsampling, it is expressed as: Among them, p(y pred |x;θ) is defined as the probability of generating a mapping of x, and θ represents the model f θ The parameter, f θ is a model consisting of an encoder and a decoder in the same main branch network, T upsc is a hyperparameter that controls the sharpening temperature; SB2: Use the model f θ The obtained soft label Supervise the outputs of the other two decoders in the same main branch network respectively; SB3: Use the three models corresponding to the left main branch network f θ Repeat step SB2 separately for cross supervision and use mean square error to quantify the difference, expressed as: in is the model corresponding to the left main branch network, θ i and θ i Indicates the use of different upsampling methods; SB4: Use the three models corresponding to the right main branch network f θ Repeat step SB2 separately for cross supervision and use mean square error to quantify the difference, expressed as: in is the model corresponding to the right main branch network, θ i and θ i Indicates the use of different upsampling methods; SB5: Calculate the consistency loss of the cross pseudo supervision based on different upsampling, expressed as:

5. The method for semi-supervised multi-category medical image semantic segmentation based on prototype contrastive learning according to claim 3, characterized in that: Compute the consistency loss of cross-pseudo supervision based on different initializations The specific steps include: SC1: For the input training sample image x, use the model or The class probability map obtained by one-hot encoding the output of is used as the supervisory signal and is expressed as: Among them, K is the total number of categories (including background categories); SC2: Use the model corresponding to the left main branch network The resulting class probability map As a supervisory signal, the model corresponding to the right main branch network uses the same upsampling method The output of Perform the above operations and use entropy loss and dice loss to quantify the difference, expressed as: SC3: Use the model corresponding to the right main branch network The resulting class probability map As a supervisory signal, the model corresponding to the left main branch network uses the same upsampling method The output of Perform the above operations and use entropy loss and dice loss to quantify the difference, expressed as: SC4: Calculate the consistency loss of the cross pseudo-supervision based on different initializations, expressed as:

6. The method for semi-supervised multi-category medical image semantic segmentation based on prototype contrastive learning according to claim 3, characterized in that: The contrastive loss based on prototype contrastive learning guided by the label of the labeled data and the contrastive loss based on prototype contrastive learning guided by high-quality pseudo labels are used to enhance the inter-class separability and intra-class compactness in the semantic segmentation task. By comparing the class feature prototype obtained in each round with the prototype in the class prototype feature library, the class feature prototype obtained by the left main branch network is Where k is The contrast loss of the prototype contrast learning guided by the label of the labeled data is expressed as: in, It is the class prototype feature library guided by the label of the labeled data. The set of class prototype features belonging to the same category k, is the class prototype feature library guided by the label data and The set of prototype features that do not belong to category k, yes A class prototype feature in yes is a prototype feature in , τ is a hyperparameter that controls the sharpening temperature, where s(F i ,F j ) is the similarity in the feature space, expressed as: Among them, F i and F j It is a class prototype feature; For the class feature prototype obtained by the right main branch network Where k is The contrast loss based on prototype contrast learning guided by high-quality pseudo labels is expressed as: in, It is the prototype feature library guided by high-quality pseudo labels. The set of class prototype features belonging to the same category k, is the class prototype feature library guided by the label data and The set of prototype features that do not belong to category k, yes A class prototype feature in yes is a prototype feature in , τ is a hyperparameter that controls the sharpening temperature, where s(F i ,F j ) is the similarity in the feature space, expressed as: Among them, F i and F j It is a class prototype feature.

7. The method for semi-supervised multi-category medical image semantic segmentation based on prototype contrastive learning according to claim 6, characterized in that: The contrastive loss based on prototype contrastive learning guided by the label of the labeled data is only used for the labeled data in the training sample set and is expressed as: Among them, N L is the number of labeled data, K is the total number of categories; the contrastive loss based on prototype contrastive learning guided by the labels of the labeled data is used for the labeled data and the unlabeled data in the training sample set, expressed as: Among them, N L is the number of labeled data, N U is the number of unlabeled data.

8. The method for semi-supervised multi-category medical image semantic segmentation based on prototype contrastive learning according to claim 1, characterized in that: The class prototype feature library includes multiple prototype feature libraries, wherein the number of prototype feature libraries in each class prototype feature library is the number of categories, each prototype feature library corresponds to one category, and each prototype feature library will be set with the same maximum capacity.

9. The method for semi-supervised multi-category medical image semantic segmentation based on prototype contrastive learning according to claim 1, characterized in that: The decoder in the left main branch network outputs a feature map of each training sample image, filters out features corresponding to each category in the corresponding training sample image according to the labels of the marked data in the training sample image, and then calculates the average features corresponding to each category of each training sample image for the above features to obtain class prototype features.

10. The method for semi-supervised multi-category medical image semantic segmentation based on prototype contrastive learning according to claim 1, characterized in that: The decoder in the right main branch network outputs a feature map of each training sample image, and the result of the decoder output is used as a pseudo label. The uncertainty of the prediction of each pixel in each training sample image is evaluated by information entropy, and then an uncertainty threshold is set to select high-quality pseudo labels. The feature maps corresponding to pixels below the uncertainty threshold are operated, and the features corresponding to each category in the corresponding training sample image are screened out according to the high-quality pseudo labels. The average features corresponding to each category of each training sample image are then calculated for the above features to obtain class prototype features.

Images