A semi-supervised medical image segmentation method and system based on a visual language model

Abstract

A kind of semi-supervised medical image segmentation method and system based on visual language model, method includes: S1, obtaining the medical image dataset to be segmented containing labeled and unlabeled sample;S2, pre-training visual language model;S3, teacher model is trained using labeled sample, then unlabeled sample is input into the trained teacher model, and pseudo label is obtained;S4, pre-trained visual language model is trained, and the visual language model after training is obtained;S5, the visual language model after training is optimized;S6, segmentation model is trained;S7, the trained segmentation model is optimized;S8, adjust learning efficiency, batch size, cycle S1 to S7, the prediction result of multiple models is integrated, and the final segmentation result is obtained.The present application can learn multiple visual and language features, provide a good foundation for transfer learning in medical image segmentation task, so that model can adapt to different image features.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to a semi-supervised medical image segmentation method and system based on a visual language model. Background Technology

[0002] Medical image segmentation plays a crucial role in extracting important organs or lesions from abnormal images, contributing significantly to computer-aided diagnosis and treatment research. In recent years, many supervised learning-based medical image segmentation encoder-decoder networks have achieved remarkable results, such as U-Net, U-Net++, and H-DenseUNet. However, these supervised learning methods heavily rely on large amounts of pixel-level labeled data, making medical image annotation typically very expensive in practice. Furthermore, medical image annotation requires more specialized knowledge than natural image annotation. Compared to supervised learning, semi-supervised learning offers a novel learning paradigm for addressing the problem of incomplete data supervision. It uses a small amount of labeled data and a large amount of unlabeled data for joint training. Clearly, in medical image segmentation, semi-supervised learning is more important than supervised learning and better aligns with the requirements of real-world clinical scenarios. In semi-supervised medical image segmentation, segmentation performance is closely related to the quality of pseudo-labels. Current semi-supervised medical image segmentation methods may suffer from insufficient complementarity between models during joint training, leading to inconsistent predictions for the same sample and thus affecting the quality of pseudo-labels.

[0003] However, a major problem facing semi-supervised image segmentation is the scarcity of labeled samples in the training data. Summary of the Invention

[0004] This invention provides a semi-supervised medical image segmentation method and system based on a visual language model to solve the technical problems mentioned in the background.

[0005] To achieve the above objectives, the technical solution of the present invention is implemented as follows:

[0006] This invention provides a semi-supervised medical image segmentation method based on a visual language model, comprising the following steps:

[0007] S1. Obtain the medical image dataset containing labeled and unlabeled samples to be segmented, and preprocess the medical image dataset;

[0008] S2. Pre-train the visual language model to obtain the pre-trained visual language model.

[0009] S3. Select a segmentation model as the teacher model, and train the teacher model using labeled samples to obtain the trained teacher model.

[0010] S4. Use the medical image dataset to train the pre-trained visual language model to obtain the trained visual language model.

[0011] S5. Enhance the original medical image to obtain a variant image. Then, use the original medical image and the variant image to optimize the trained visual language model to obtain an optimized visual language model. Input the unlabeled samples into the optimized visual language model and input the output back into the trained teacher model to obtain optimized pseudo-labels.

[0012] S6. Select a segmentation model, and train the segmentation model using labeled samples and pseudo-labels obtained in S5 to obtain the trained segmentation model.

[0013] S7. Calculate unit terms based on the preliminary segmentation probability map of each pixel in the original medical image, and define interaction terms based on the similarity between adjacent pixels. Then, optimize the trained segmentation model based on the unit terms and interaction terms to obtain the optimized segmentation model.

[0014] S8. Adjust the learning efficiency and batch size of the optimized segmentation model, repeat S1 to S7 to obtain multiple segmentation models, integrate the prediction results of multiple models to obtain the final segmentation result.

[0015] Furthermore, the preprocessing in S1 includes at least image normalization, denoising and enhancement, and data augmentation;

[0016] Image standardization is used to resize the input image to a uniform size.

[0017] Denoising and enhancement are used to enhance image contrast using contrast-limited adaptive histogram equalization, and to denoise the input image using image filters;

[0018] Data augmentation is used to enhance input images using methods such as rotation, flipping, scaling, and translation.

[0019] Furthermore, step S2 specifically includes the following steps:

[0020] S21. Select multiple original medical images from the medical image dataset. The original medical images are either labeled or unlabeled samples. Perform data augmentation on the multiple original medical images to obtain multiple augmented images.

[0021] S22. Provide textual descriptions of multiple original medical images and multiple enhanced images;

[0022] S23. Use a convolutional neural network (CNN) or a visual network (Transformer) as the first visual encoder; use a BERT model or a GPT model as the first text encoder, and then construct a momentum visual encoder and a momentum text encoder respectively.

[0023] S24. Input the original medical image into visual encoder one to obtain the original image embedding; input the enhanced image into momentum visual encoder to obtain the enhanced image embedding;

[0024] S25. Input the original medical image into text encoder one to obtain the original text embedding; input the enhanced image into momentum text encoder to obtain momentum text embedding;

[0025] S26. Calculate the first comparative loss function based on the outputs of S24 and S25;

[0026] S27. Repeat S24 to S26 to maximize the similarity between the original image embedding and the corresponding text embedding, while minimizing the similarity between the original image embedding and the enhanced image embedding.

[0027] S28. Use a momentum update strategy to update the parameters of the momentum visual encoder and the momentum text encoder respectively, so that the current parameters of the momentum visual encoder approximate the weights of the visual encoder one, and the current parameters of the momentum text encoder approximate the weights of the text encoder one.

[0028] The momentum update strategy is expressed by a formula, as follows:

[0029] θ momentum =m×θ momentum +(1-m)×θ main

[0030] Where m is the momentum decay factor, θ momentum These are the current parameters of the momentum encoder, which includes a momentum visual encoder and a momentum text encoder, θ. main The weights are those of the original encoder; the original encoder includes visual encoder 1 and text encoder 1.

[0031] Furthermore, step S3 specifically includes the following steps:

[0032] S31. Select the U-Net segmentation model as the teacher model and initialize the teacher model;

[0033] S32. Train the initialized teacher model using labeled samples to obtain the trained teacher model. The training objective is to minimize the following formula:

[0034]

[0035] Where N1 is the number of labeled samples, f is the teacher model, and L teacher The loss function for the teacher model. and These represent the i-th original medical image in the labeled sample and its corresponding segmentation labeling mask, respectively.

[0036] S33. Input the unlabeled samples into the trained teacher model to obtain the corresponding pseudo-labels;

[0037] S34. Set a reliability threshold T, and then use the reliability threshold T to filter multiple pseudo-labels to obtain the retained pseudo-labels. The filtering process is expressed by a formula, as follows:

[0038]

[0039] Among them, X u f(X) represents unlabeled data. u () represents pseudo-labels generated by the teacher model from unlabeled data.

[0040] Furthermore, step S4 specifically includes the following steps:

[0041] S41. Input the original medical image X into the pre-trained visual language model, use the visual encoder in the pre-trained visual language model to extract a pair of features, and obtain the image feature embedding h(X); then use the text encoder in the pre-trained visual language model to extract text features from the corresponding text description T of the original medical image X, and obtain the text feature embedding g(T).

[0042] S42. The similarity between image feature embedding h(X) and text feature embedding g(T) is calculated based on cosine similarity, and expressed by the formula as follows:

[0043]

[0044] Where h(X)·g(T) represents the vector dot product, and ||·|| is the Euclidean norm of the vector;

[0045] S43. Construct a second contrastive loss function based on the similarity obtained in S42. The second contrastive loss function is as follows:

[0046]

[0047] Among them, L contrastive T represents the loss value of the second contrastive loss function; T is the text description matched with the original medical image X. j For all candidate text descriptions; N is the total number of labeled and unlabeled samples;

[0048] S44. Repeat S41 to S43, maximize the similarity between the image and the corresponding text by contrastive loss function 2, and minimize the similarity between the image and the non-matching text; optimize the parameters in visual encoder 1 and text encoder 1 by backpropagation algorithm and according to contrastive loss function 2 to obtain the trained visual language model.

[0049] Furthermore, step S5 specifically includes the following steps:

[0050] S51. Enhance the original medical image X to generate a variant image X. aug Enhancement methods include rotation, cropping, or brightness adjustment.

[0051] S52, Combine the original medical image X and the variant image X aug The images are input into visual encoder one, and the image feature embeddings h(X) and h(X) are extracted from them respectively. aug );

[0052] S53. Embed h(X) and h(X) based on their image features. aug And the corresponding text embeddings of both construct the loss function three;

[0053] S54. Repeat steps S51 to S53 to minimize the loss function 3. During this process, update the weights of the trained visual language model using the backpropagation algorithm to obtain the optimized visual language model.

[0054] S55, Unlabeled sample X u Input the optimized visual language model and extract image features h(X) u ) and text features g(T) u ), where T u Is it the same as the unlabeled sample X? u Corresponding text description; image features h(X) u ) and text features g(T) u The pixel-level features of the original medical image X are input into the teacher model of S3 to generate optimized pseudo-labels.

[0055] Furthermore, step S6 specifically includes the following steps:

[0056] S61. Based on the segmentation task requirements, select one model from the segmentation model U-Net and the segmentation model Mask R-CNN as the selected segmentation model.

[0057] S62. Mix the pseudo-labels obtained in S5 with the labeled samples and input them into the selected segmentation model to obtain the model prediction output.

[0058] S63. Predict output based on model The fourth loss function is constructed as follows:

[0059]

[0060] Among them, L seg Y represents the loss value of loss function four. i The true labels corresponding to the labeled samples and the pseudo labels corresponding to the unlabeled samples in S3;

[0061] S64. Repeat S62 to S63 to minimize the loss function 4, and update the weights of the segmentation model through backpropagation during this process to obtain the trained segmentation model.

[0062] Furthermore, step S7 specifically includes the following steps:

[0063] S71. Calculate the unit term U(y) based on the preliminary segmentation probability map of each pixel in the original medical image X. i The formula is:

[0064] U(y i )=-logP(y i )

[0065] Wherein, P(y i ) indicates that pixel i belongs to label y i The probability of;

[0066] S72. Based on the similarity between adjacent pixels of the original medical image X, define the interaction term V(y). i y j The formula is:

[0067]

[0068] In the formula, μ(y i ≠y j ) represents the penalty term, indicating the energy value of the different labels of adjacent pixels; p i p j Let be the position coordinates of pixels i and j; σ is the smoothing parameter of the Gaussian kernel;

[0069] S73, Based on unit term U(y) i ), interaction term V(y) i y j Construct an energy function and iterate through steps S71 to S72 to minimize the energy function, obtaining the optimized segmentation model. The specific steps for minimizing the energy function are as follows:

[0070]

[0071] Among them, E CRF (Y) represents the energy function; Y represents the set of segmentation labels; y i y j Let i and y represent the segmentation labels for pixel i and y, respectively.

[0072] Furthermore, step S8 specifically includes the following steps:

[0073] S81. Dynamically adjust the learning rate, batch size, and structure of the optimized segmentation model;

[0074] S82. Repeat S1 to S7 to obtain multiple segmentation models. Train multiple segmentation models and then perform a weighted average of the results of multiple segmentation models to improve the segmentation accuracy of the final model and obtain the final segmentation result.

[0075] In another aspect, the present invention provides a semi-supervised medical image segmentation system based on a visual language model, including a computer device programmed or configured to perform the above-described semi-supervised medical image segmentation method.

[0076] The beneficial effects of this invention are:

[0077] 1. This invention is based on the fact that visual language models can extract richer contextual information, thereby enabling more accurate identification and segmentation of key structures or abnormal areas when processing medical images. In addition, text descriptions provide additional semantic information to visual language models, making it easier to identify clinically relevant features. Furthermore, visual language models can be pre-trained on large-scale datasets and can learn a variety of visual and linguistic features, providing a good foundation for transfer learning in medical image segmentation tasks, enabling visual language models to adapt to different image features.

[0078] 2. This invention reduces reliance on expensive annotation resources by using a large amount of unlabeled data for training, thereby improving the robustness of the segmentation model. Furthermore, the visual language model combines visual and linguistic information, enabling it to understand not only the visual features of an image but also related textual descriptions (such as medical reports), thus enhancing its ability to analyze complex medical images.

[0079] 3. This invention effectively increases the diversity of training data through data augmentation, enabling visual language models and segmentation models to access a wider range of sample variations during training. This helps improve the generalization ability of the segmentation model. Furthermore, in situations where labeled data is scarce, direct training may lead to overfitting of the model to the training samples. However, by applying data augmentation, the model can see different image variations in each training iteration, reducing its dependence on specific samples and thus improving its performance on unseen samples. At the same time, image augmentation technology can highlight important anatomical structures or pathological changes, enabling the model to learn features more effectively during training. Attached Figure Description

[0080] Figure 1 This is a flowchart of the semi-supervised medical image segmentation method in this invention. Detailed Implementation

[0081] To facilitate understanding of the present invention, a more complete description will be given below with reference to the accompanying drawings. Preferred embodiments of the invention are shown in the drawings. However, the invention can be implemented in many other different forms and is not limited to the embodiments described herein. Rather, these embodiments are provided to provide a thorough and complete understanding of the disclosure of the invention.

[0082] Reference Figure 1 This application provides a semi-supervised medical image segmentation method based on a visual language model, comprising the following steps:

[0083] S1. Obtain the medical image dataset containing labeled and unlabeled samples to be segmented, and preprocess the medical image dataset;

[0084] S2. Pre-train the visual language model to obtain the pre-trained visual language model; the visual language model is any one of CLIP model, BLIP model or ViLT model.

[0085] S3. Select a segmentation model as the teacher model, and train the teacher model using labeled samples to obtain the trained teacher model. Then, input the unlabeled samples into the trained teacher model to obtain pseudo-labels.

[0086] S4. Use a medical image dataset to train the pre-trained visual language model to improve the visual language model's ability to model the matching relationship between images and text, thus obtaining an optimized visual language model.

[0087] S5. Enhance the original medical image to obtain a variant image. Then, use the original medical image and the variant image to optimize the trained visual language model to obtain an optimized visual language model. Improve the quality of pseudo-label generation through the optimized visual language model. Input the unlabeled sample into the optimized visual language model and input the output into the trained teacher model again to obtain optimized pseudo-labels. The goal of S5 optimization is to improve the model's ability to understand image and text features through self-supervised learning, so that it can better capture the semantic relationship between the two.

[0088] S6. Select a segmentation model, and train the segmentation model using labeled samples and pseudo-labels obtained in S5 to obtain the trained segmentation model.

[0089] S7. Calculate unit terms based on the preliminary segmentation probability map of each pixel in the original medical image, and define interaction terms based on the similarity between adjacent pixels. Then, optimize the trained segmentation model based on the unit terms and interaction terms to obtain the optimized segmentation model.

[0090] S8. Adjust the learning efficiency and batch size of the optimized segmentation model, repeat S1 to S7 to obtain multiple segmentation models, integrate the prediction results of multiple models to obtain the final segmentation result.

[0091] In some embodiments, the preprocessing in S1 includes at least image normalization, denoising and enhancement, and data enhancement;

[0092] Image standardization is used to resize the input image to a uniform size.

[0093] Denoising and enhancement are used to enhance image contrast using contrast-limited adaptive histogram equalization, and to denoise the input image using image filters;

[0094] Data augmentation is used to enhance input images using methods such as rotation, flipping, scaling, and translation.

[0095] This invention effectively increases the diversity of training data through data augmentation, enabling visual language models and segmentation models to access a wider range of sample variations during training. This helps improve the generalization ability of the models. Furthermore, in situations where labeled data is scarce, direct training may lead to overfitting of the model to the training samples. However, by applying data augmentation, the model can see different image variations in each training iteration, reducing its dependence on specific samples and thus improving its performance on unseen samples. At the same time, image augmentation technology can highlight important anatomical structures or pathological changes, enabling the model to learn features more effectively during training.

[0096] In some embodiments, S2 specifically includes the following steps:

[0097] S21. Select multiple original medical images from the medical image dataset. The original medical images are either labeled or unlabeled samples. Perform data augmentation on the multiple original medical images to obtain multiple augmented images.

[0098] S22. Provide textual descriptions of multiple original medical images and multiple enhanced images;

[0099] S23. Use a convolutional neural network (CNN) or a visual network (Transformer) as the first visual encoder; use a BERT model or a GPT model as the first text encoder, and then construct a momentum visual encoder and a momentum text encoder respectively.

[0100] S24. Input the original medical image into visual encoder one to obtain the original image embedding; input the enhanced image into momentum visual encoder to obtain the enhanced image embedding;

[0101] S25. Input the original medical image into text encoder one to obtain the original text embedding; input the enhanced image into momentum text encoder to obtain momentum text embedding;

[0102] S26. Calculate the first comparative loss function based on the outputs of S24 and S25;

[0103] S27. Repeat S24 to S26 to maximize the similarity between the original image embedding and the corresponding text embedding, while minimizing the similarity between the original image embedding and the enhanced image embedding.

[0104] S28. Use a momentum update strategy to update the parameters of the momentum visual encoder and the momentum text encoder respectively, so that the current parameters of the momentum visual encoder approximate the weights of the visual encoder one, and the current parameters of the momentum text encoder approximate the weights of the text encoder one.

[0105] The momentum update strategy is expressed by a formula, as follows:

[0106] θ momentum =m×θ momentum +(1-m)×θ main

[0107] Where m is the momentum decay factor, θ momentum These are the current parameters of the momentum encoder, which includes a momentum visual encoder and a momentum text encoder, θ. main The weights are those of the original encoder; the original encoder includes visual encoder 1 and text encoder 1.

[0108] In some embodiments, S3 specifically includes the following steps;

[0109] S31. Select the U-Net segmentation model as the teacher model and initialize the teacher model;

[0110] S32. Train the initialized teacher model using labeled samples to obtain the trained teacher model. The training objective is to minimize the following formula:

[0111]

[0112] Where N1 is the number of labeled samples, f is the teacher model, and L teacher The loss function for the teacher model. and These represent the i-th original medical image in the labeled sample and its corresponding segmentation labeling mask, respectively.

[0113] S33. Input the unlabeled samples into the trained teacher model to obtain multiple pseudo-labels;

[0114] S34. Set a reliability threshold τ, and then use the reliability threshold T to filter multiple pseudo-labels to obtain the retained pseudo-labels. The filtering process is expressed by a formula, as follows:

[0115]

[0116] Among them, X u f(X) represents unlabeled data. u () represents pseudo-labels generated by the teacher model from unlabeled data.

[0117] In some embodiments, S4 specifically includes the following steps:

[0118] S41. Input the original medical image X into the pre-trained visual language model, use the visual encoder in the pre-trained visual language model to extract a pair of features, and obtain the image feature embedding h(X); then use the text encoder in the pre-trained visual language model to extract text features from the corresponding text description T of the original medical image X, and obtain the text feature embedding g(T).

[0119] S42. The similarity between image feature embedding h(X) and text feature embedding g(T) is calculated based on cosine similarity, and expressed by the formula as follows:

[0120]

[0121] Where h(X)·g(T) represents the vector dot product, and ||·|| is the Euclidean norm of the vector;

[0122] S43. Construct a second contrastive loss function based on the similarity obtained in S42. The second contrastive loss function is as follows:

[0123]

[0124] Among them, L contrastive T represents the loss value of the second contrastive loss function; T is the text description matched with the original medical image X. j For all candidate text descriptions; N is the total number of labeled and unlabeled samples;

[0125] S44. Repeat S41 to S43, maximize the similarity between the image and the corresponding text by contrastive loss function 2, and minimize the similarity between the image and the non-matching text. Optimize the parameters in visual encoder 1 and text encoder 1 through backpropagation algorithm based on contrastive loss function 2 to improve the visual language model's ability to model the matching relationship between images and text, and obtain the trained visual language model.

[0126] This invention reduces reliance on expensive annotation resources and improves the robustness of the model by using a large amount of unlabeled data for training. Furthermore, the visual language model combines visual and linguistic information, enabling it to understand not only the visual features of an image but also the associated textual descriptions (such as medical reports), thereby enhancing its ability to analyze complex medical images.

[0127] Visual language models extract richer contextual features by combining visual features of medical images with semantic information from text descriptions. The text provides semantic guidance for key structures or abnormal areas (such as location and shape). Meanwhile, the visual and linguistic features obtained by the model through pre-training on large-scale data enhance its generalization ability, enabling it to adapt to different scenarios and complex structures, thereby more accurately identifying and segmenting target regions.

[0128] In some embodiments, S5 specifically includes the following steps:

[0129] S51. Enhance the original medical image X to generate a variant image X. aug Enhancement methods include rotation, cropping, or brightness adjustment.

[0130] S52, Combine the original medical image X and the variant image X aug The images are input into visual encoder one, and the image feature embeddings h(X) and h(X) are extracted from them respectively. aug );

[0131] S53. Embed h(X) and h(X) based on their image features. aug And the corresponding text embeddings of both construct the loss function three;

[0132] S54. Repeat steps S51 to S53 to minimize the loss function 3. During this process, update the weights of the trained visual language model using the backpropagation algorithm to obtain the optimized visual language model.

[0133] S55, Unlabeled sample X u Input the optimized visual language model and extract image features h(X) u ) and text features g(T) u ), where T u Is it the same as the unlabeled sample X? u Corresponding text description; image features h(X) u ) and text features g(T) u The pixel-level features of the original medical image X are input into the teacher model of S3 to generate optimized pseudo-labels.

[0134] In some embodiments, S6 specifically includes the following steps:

[0135] S61. Based on the segmentation task requirements, select one model from the segmentation model U-Net and the segmentation model Mask R-CNN as the selected segmentation model; among which, the segmentation model U-Net is suitable for smaller datasets and has efficient feature extraction and context information fusion capabilities, while the segmentation model Mask R-CNN is suitable for more complex tasks and can perform object detection and segmentation simultaneously.

[0136] S62. Mix the pseudo-labels obtained in S5 with the labeled samples and input them into the selected segmentation model to obtain the model prediction output.

[0137] S63. Predict output based on model The fourth loss function is constructed as follows:

[0138]

[0139] Among them, L seg Y represents the loss value of loss function four. i The true labels corresponding to the labeled samples and the pseudo labels corresponding to the unlabeled samples in S3;

[0140] S64. Repeat S62 to S63 to minimize the loss function 4, and update the weights of the segmentation model through backpropagation during this process to obtain the trained segmentation model.

[0141] In some embodiments, S7 specifically includes the following steps:

[0142] S71. Calculate the unit term U(y) based on the preliminary segmentation probability map of each pixel in the original medical image X.i The formula is:

[0143] U(y i )=-logP(y i )

[0144] Wherein, P(y i ) indicates that pixel i belongs to label y i The probability of;

[0145] S72. Based on the similarity between adjacent pixels of the original medical image X, define the interaction term V(y). i y j The formula is:

[0146]

[0147] In the formula, μ(y i ≠y j ) represents the penalty term, indicating the energy value of the different labels of adjacent pixels; p i p j Let be the position coordinates of pixels i and j; σ is the smoothing parameter of the Gaussian kernel;

[0148] S73, Based on unit term U(y) i ), interaction term V(y) i y j Construct an energy function and iterate through steps S71 to S72 to minimize the energy function, resulting in a smoother and more accurate segmentation model. The specific steps for minimizing the energy function are as follows:

[0149]

[0150] Among them, E CRF (Y) represents the energy function; Y represents the set of segmentation labels; y i y j Let U(y) represent the segmentation labels for pixels i and j, respectively. i ) indicates that pixel i belongs to segmentation label y i The energy value, V(y) i y j ) represents the interaction energy between pixel i and pixel .

[0151] In some embodiments, S8 specifically includes the following steps:

[0152] S81. Dynamically adjust the learning rate, batch size, and structure of the optimized segmentation model;

[0153] S82. Repeat S1 to S7 to obtain multiple segmentation models. Train multiple segmentation models and then perform a weighted average of the results of multiple segmentation models to improve the segmentation accuracy, performance and robustness of the final model, and obtain the final segmentation result.

[0154] In another aspect, the present invention provides a semi-supervised medical image segmentation system based on a visual language model, including a computer device programmed or configured to perform the above-described semi-supervised medical image segmentation method.

[0155] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Furthermore, the technical solutions of the various embodiments of the present invention can be combined with each other, but this must be based on the ability of those skilled in the art to implement them. When the combination of technical solutions is contradictory or cannot be implemented, it should be considered that such a combination of technical solutions does not exist and is not within the scope of protection claimed by the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A semi-supervised medical image segmentation method based on a visual language model, characterized in that, The method comprises the following steps: S1, obtaining a medical image dataset to be segmented containing labeled and unlabeled samples, and preprocessing the medical image dataset; S2, pre-training a visual language model to obtain a pre-trained visual language model; S3, selecting a segmentation model as a teacher model, and training the teacher model using labeled samples to obtain a trained teacher model; S4, training the pre-trained visual language model using the medical image dataset to obtain a trained visual language model; S5, enhancing the original medical image to obtain a variant image, then optimizing the trained visual language model using the original medical image and the variant image to obtain an optimized visual language model, inputting the unlabeled sample into the optimized visual language model to extract image features and text features, and inputting the image features, the text features and the pixel-level features of the original medical image into the trained teacher model again to obtain optimized pseudo labels; S6, selecting a segmentation model, training the segmentation model using the labeled samples and the pseudo labels obtained in S5 to obtain a trained segmentation model; S7, calculating a cell term according to a preliminary segmentation probability map of each pixel point of the original medical image, defining an interaction term according to the similarity between adjacent pixel points, and then optimizing the trained segmentation model according to the cell term and the interaction term to obtain an optimized segmentation model; S8, adjusting the learning efficiency and batch size of the optimized segmentation model, and repeating S1 to S7 to obtain multiple segmentation models, integrating the prediction results of the multiple models to obtain a final segmentation result; The formula of the cell term is: U(y i ) = -log P(y i ) where U(y i ) denotes a unit term; P(y i ) denotes a probability that a pixel point i belongs to a label y i ; and P(y The formula of the interaction term is: where V(y i , y j ) represents the interaction term; μ(y i ≠ y j ) is the penalty term, representing the energy value of the adjacent pixel points with different labels; p i , p j are the position coordinates of the pixel points i and j; σ is the smoothing parameter of the Gaussian kernel, and ||·|| is the Euclidean norm of the vector.

2. The semi-supervised medical image segmentation method of claim 1, wherein, The preprocessing in S1 at least includes image standardization, denoising and enhancement, and data enhancement; The image standardization is used to adjust the input image to a uniform size; The denoising and enhancement are used to enhance the contrast of the image by using a contrast limited adaptive histogram equalization, and to denoise the input image by using an image filter; The data enhancement is used to enhance the input image by using rotation, flipping, scaling and translation methods.

3. The semi-supervised medical image segmentation method of claim 1, wherein, S2 specifically comprises the following steps: S21, selecting multiple original medical images from the medical image dataset, the original medical images being labeled samples or unlabeled samples, and performing data enhancement on the multiple original medical images to obtain multiple enhanced images; S22, performing text description on the multiple original medical images and the multiple enhanced images; S23, using a convolutional neural network (CNN) or a visual network (Transformer) as a visual encoder one, using a BERT model or a GPT model as a text encoder one, and then constructing a momentum visual encoder and a momentum text encoder; S24, inputting the original medical image into the visual encoder one to obtain an original image embedding, and inputting the enhanced image into the momentum visual encoder to obtain an enhanced image embedding; S25, inputting the original medical image into the text encoder one to obtain an original text embedding; inputting the enhanced image into the momentum text encoder to obtain a momentum text embedding; S26, calculating a contrast loss function one according to the outputs of S24 and S25; S27, circulate S24-S26, maximize the similarity between the original image embedding and the corresponding text embedding, while minimizing the similarity between the original image embedding and the enhanced image embedding; S28, use a momentum update strategy to update the parameters of the momentum visual encoder and the momentum text encoder respectively, so that the current parameters of the momentum visual encoder approximate the weights of the visual encoder one, and the current parameters of the momentum text encoder approximate the weights of the text encoder one; The momentum update strategy adopts a formula, which is as follows: θ momentum = m x θ momentum + (1 - m) x θ main where m is a momentum decay factor, θ momentum is the current parameter of the momentum encoder, which includes a momentum visual encoder and a momentum text encoder, θ main is the weight of the original encoder; the original encoder includes a visual encoder one and a text encoder one.

4. The semi-supervised medical image segmentation method of claim 1, wherein, The S3 specifically comprises the following steps: S31, selecting a U-Net segmentation model as a teacher model and initializing the teacher model; S32, training the initialized teacher model using the labeled samples to obtain a trained teacher model, and the training target is to minimize the following formula: wherein N1 is the number of labeled samples, f is the teacher model, L teacher is the loss function of the teacher model, and respectively represent the i-th original medical image in the labeled samples and its corresponding segmentation label mask. S33, inputting the unlabeled sample into the trained teacher model to obtain the corresponding pseudo label; S34, set a confidence threshold T, and then filter the plurality of pseudo labels using the confidence threshold T to obtain retained pseudo labels The filtering process is expressed by a formula, and is specifically as follows: wherein X u represents an unlabeled sample; f(X u ) represents the pseudo label generated by the teacher model for the unlabeled sample.

5. The semi-supervised medical image segmentation method of claim 4, wherein, The S4 specifically comprises the following steps: S41, inputting the original medical image X into the pre-trained visual language model, using the visual encoder one in the pre-trained visual language model to extract the features to obtain the image feature embedding h(X); and then using the text encoder one in the pre-trained visual language model to extract the text features of the corresponding text description T of the original medical image X to obtain the text feature embedding g(T); S42, calculating the similarity between the image feature embedding h(X) and the text feature embedding g(T) based on the cosine similarity, which adopts a formula, which is as follows: Wherein, h(X)·g(T) represents the vector inner product, and ||·|| is the Euclidean norm of the vector; S43, constructing a contrast loss function two according to the similarity obtained in S42, which is as follows: wherein, L contrastive represents the loss value of the contrast loss function two; T is a text description matched with the original medical image X, T j is all candidate text descriptions; N is the total number of labeled samples and unlabeled samples; S44, circulating S41-S43, maximizing the similarity between the image and the corresponding text while minimizing the similarity between the image and the unmatched text through the contrast loss function two; and optimizing the parameters in the visual encoder one and the text encoder one through the back propagation algorithm and according to the contrast loss function two to obtain the trained visual language model.

6. The semi-supervised medical image segmentation method of claim 5, wherein, The S5 specifically comprises the following steps: S51, enhancing the original medical image X to generate a variant image X aug ; the enhancement method includes rotation, cropping, or brightness adjustment; S52, the original medical image X and the variant image X aug are input to the visual encoder one, image feature embeddings h(X) and h(X aug ), respectively, are extracted S53、Depend on the image feature embedding h(X) and h(X aug ) of both and the corresponding text embedding of both to build loss function three; S54, circulating S51-S53, minimizing the loss function three, and updating the weights of the trained visual language model through the back propagation algorithm in the process to obtain the optimized visual language model; S55, the unlabeled sample X u input the optimized visual language model, extract the image feature h(X u ) and the text feature g(T u ), wherein T u is the text description corresponding to the unlabeled sample X u ; input the image feature h(X u ) and the text feature g(T u ) to the teacher model trained in S3 together with the pixel-level feature of the original medical image X, to generate the optimized pseudo label 7. The semi-supervised medical image segmentation method of claim 6, wherein, The S6 specifically comprises the following steps: S61, selecting one model as a selected segmentation model from the segmentation model U-Net and the segmentation model Mask R-CNN according to the segmentation task requirements; S62, mix the pseudo label obtained in S5 with the labeled samples, and input into the selected segmentation model to obtain model prediction output S63, predict output according to the model The loss function four is constructed, and the loss function four is specifically as follows: wherein L seg is a loss value of the loss function four; Y i is a real label corresponding to the labeled sample and a pseudo label corresponding to the unlabeled sample in S3; S64, circulating S62-S63, minimizing the loss function four, and updating the weights of the segmentation model through the back propagation algorithm in the process to obtain the trained segmentation model.

8. The semi-supervised medical image segmentation method of claim 7, wherein, The S7 specifically comprises the following steps: S71, a preliminary segmentation probability map calculation unit item U(y i ) is calculated for each pixel point of the original medical image X S72, define the interaction term V(y i , y j ) according to the similarity between adjacent pixel points of the original medical image X; S73、According to the unit item U(y i ), the interaction item V(y i , y j ) to construct an energy function, and loop S71 to S72 to minimize the energy function to obtain an optimized segmentation model; the energy function is minimized as follows: wherein E CRF (Y) represents an energy function; Y represents a set of segmentation labels; y i , y j respectively represent the segmentation labels of pixel points i and j.

9. The semi-supervised medical image segmentation method of claim 8, wherein, The S8 specifically comprises the following steps: S81, dynamically adjusting the learning rate, batch size and structure of the optimized segmentation model; S82, repeat S1 to S7 to obtain a plurality of segmentation models, train the plurality of segmentation models, and then perform weighted average on results of the plurality of segmentation models to improve segmentation accuracy of a final model, and obtain a final segmentation result. 10.A semi-supervised medical image segmentation system based on a visual language model, comprising a computer device, characterized in that, The computer device is programmed or configured to perform the semi-supervised medical image segmentation method of any one of claims 1 to 9.

Images