Semi-supervised medical image segmentation method and system based on visual language model

Abstract

SOLUTION: A semi-supervised medical image segmentation method based on a visual language model includes the steps of: obtaining a medical image; inputting an unlabeled image and a text description into a visual language model, and obtaining a text-guided mask based on obtained dense image embedding and text embedding; inputting a labeled image into a student model, and calculating supervised loss by using obtained labeled image prediction; respectively inputting the unlabeled image into the student model and a teacher model to obtain unlabeled image prediction and a pseudo label, merging the text-guided mask with the pseudo label, and calculating semi-supervised loss by using the merged pseudo label and unlabeled image prediction; and performing medical image segmentation by using a trained student model on the basis of the supervised loss and the semi-supervised loss.EFFECT: A target segmentation region can be accurately identified by using advantages of text descriptions.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] (CROSS-REFERENCE TO RELATED APPLICATIONS) This invention claims priority to a Chinese patent application bearing application number 202410321989.X and entitled "Semi-supervised medical image segmentation method and system based on visual language model," filed with the State Intellectual Property Office of the People's Republic of China on March 20, 2024, the entire contents of which are incorporated by reference into and constitute a part of the present invention for all purposes.

[0002] The present disclosure relates to the technical field of image processing, and in particular to a method and system for semi-supervised medical image segmentation based on a visual language model. [Background technology]

[0003] The discussion in this section merely provides background information related to the present disclosure and does not necessarily constitute prior art.

[0004] Medical image segmentation can be used to extract important organs and lesions in abnormal images, playing an important role in computer-aided diagnosis and treatment research. In recent years, many supervised learning-based medical image segmentation codec networks, such as U-Net, U-Net++, and H-DenseUNet, have achieved remarkable results. However, these supervised learning methods rely heavily on large amounts of pixel-level labeled data, making practical medical image annotation typically very expensive. Furthermore, medical image annotation requires a higher level of expertise than natural images. Semi-supervised learning is a new learning paradigm that overcomes the problem of incomplete data supervision compared to supervised learning. It enables joint training using small amounts of labeled data and large amounts of unlabeled data. It is clear that semi-supervised learning is more important than supervised learning in medical image segmentation and better meets the demands of real-world clinical situations. In semi-supervised medical image segmentation, segmentation performance is closely related to the quality of pseudo-labels. Current semi-supervised medical image segmentation methods can suffer from insufficient complementarity between models during joint training, leading to inconsistent predictions for identical samples and affecting the quality of pseudo-labels.

[0005] Visual language models are an effective paradigm for improving the quality of pseudo-labels in semi-supervised medical image segmentation. Specifically, visual language models generate text-guided image segmentation masks based on input image and text information, and then merge the masks with pseudo-labels generated by traditional semi-supervised methods to improve the quality of the pseudo-labels. However, traditional visual language models suffer from cross-modal and intra-modal uncertainty during pre-training. Cross-modal uncertainty manifests as multiple images / texts corresponding to a single text / image. Intra-modal uncertainty manifests as the uncertainty of which object is being referred to when describing a visual region, since a single prompt visual region contains multiple different objects. In linguistic terms, it manifests as word ambiguity due to complex word relationships, such as synonyms and hierarchical relationships between words. Traditional uncertainty-aware visual language modeling methods primarily use distributions to represent semantic embeddings to solve the uncertainty problem. However, distribution-based methods face two challenges. First, although the variance of the distribution reflects the distributional difference, distribution alone cannot fully recognize abstract or complex semantic information and understand semantic uncertainty. Second, traditional distribution-based visual language models mainly focus on cross-modal contrastive learning and ignore semantic relevance within each modality, which leads to cross-modal alignment uncertainty. Summary of the Invention

[0006] In response to the above deficiencies, this disclosure proposes a method and system for semi-supervised medical image segmentation based on a visual language model, which extracts image and text embeddings using a pre-trained visual language model and transfers the rich a priori information in the pre-trained visual language model to semi-supervised learning. By aggregating text-guided masks into semi-supervised medical image segmentation, the quality deficiencies of pseudo-labels are compensated for and the advantages of text descriptions are utilized to accurately identify target segmentation regions. A semantic uncertainty constraint policy is proposed to facilitate the model's understanding of similar but semantically ambiguous data pairs. The constraint policy is embedded in triple-uncertainty contrastive learning to reduce the uncertainty of cross-modal alignment.

[0007] To achieve the above objectives, the present disclosure adopts the following technical solutions:

[0008] In a first aspect of the present disclosure, obtaining a medical image to be segmented; inputting the unlabeled images and text descriptions into a visual language model to obtain dense image embeddings and text embeddings, and obtaining a text-guided mask based on the dense image embeddings and text embeddings; The teacher-student model is a semi-supervised segmentation backbone network, and the labeled images are input to the student model to obtain labeled image predictions. The labeled image predictions and the ground truth labels are used to calculate a supervised loss. inputting unlabeled images into a student model and a teacher model, respectively, to obtain unlabeled image predictions and pseudo labels; merging the text-guided masks and the pseudo labels to obtain merged pseudo labels; and calculating a semi-supervised loss using the merged pseudo labels and the unlabeled image predictions; and training a student model based on the supervised loss and the semi-supervised loss, and performing medical image segmentation using the trained student model.

[0009] In a further implementation, the visual language model is a pre-trained visual language model, and the visual language model is specifically pre-trained as follows: Enhance the acquired original image to obtain an enhanced image; Input the original image and the augmented image into a visual encoder and a momentum visual encoder, respectively, to obtain the original image embedding and the augmented image embedding; inputting the text descriptions of the original image and the augmented image into a text encoder and a momentum text encoder, respectively, to obtain text embeddings and momentum text embeddings; We jointly perform triple contrast learning, embed an uncertainty semantic constraint policy in each contrast learning process, and calculate each contrast learning loss. Each contrastive learning loss is used to optimize the parameter weights of the visual language model to obtain a pre-trained visual language model.

[0010] In a further implementation, the triple contrast learning includes cross-modal contrast learning, intra-modal contrast learning, and global-local contrast learning.

[0011] Further implementations include cross-modal contrastive learning, which achieves effective image and text alignment; intra-modal contrastive learning, which captures potential associations between different samples within each modality, providing richer contextual relationships and internal representations; and global-local contrastive learning, which understands features at different scales, encouraging the model to focus on important local information and thereby limiting learning of irrelevant local regions.

[0012] In a further implementation, the uncertainty semantic constraint policy specifically models the text embedding and the momentum text embedding as two corresponding distribution representations, calculates the distribution distance between these two distributions as an uncertainty level, and constrains the difference between the text embedding and the momentum text embedding based on the uncertainty level.

[0013] In a further implementation, the momentum visual encoder and momentum text encoder are momentum updated by the visual encoder and the text encoder, respectively.

[0014] A further implementation further includes element-wise multiplying the text-guided mask with the dense image embedding to obtain a text-guided embedding, computing a text-guided loss using the text-guided embedding and the text embedding, and optimizing the text-guided mask using the text-guided loss.

[0015] In a second aspect of the present disclosure, a data acquisition module configured to acquire a medical image to be segmented; a text-guided mask generation module configured to input the unlabeled images and text descriptions into a visual language model to obtain dense image and text embeddings, and to obtain a text-guided mask based on the dense image and text embeddings; and a supervised loss calculation module configured to: input labeled images to the student model to obtain labeled image predictions; and calculate a supervised loss using the labeled image predictions and ground truth labels; an unsupervised loss calculation module configured to input unlabeled images to a student model and a teacher model, respectively, to obtain unlabeled image predictions and pseudo labels, merge the text-guided masks and the pseudo labels to obtain merged pseudo labels, and calculate a semi-supervised loss using the merged pseudo labels and the unlabeled image predictions; and a model training module configured to train a student model based on supervised loss and semi-supervised loss, and perform medical image segmentation using the trained student model.

[0016] In a third aspect of the present disclosure, there is provided a medium having a program stored thereon, the program, when executed by a processor, realizing steps of the method for semi-supervised medical image segmentation based on a visual language model described in the first aspect of the present disclosure.

[0017] In a fourth aspect of the present disclosure, there is provided an electronic device including a memory, a processor, and a program stored in the memory and executable by the processor, wherein when the program is executed by the processor, steps of the semi-supervised medical image segmentation method based on a visual language model described in the first aspect of the present disclosure are realized.

[0018] Compared with the prior art, the beneficial effects of the present disclosure are as follows:

[0019] The proposed method for semi-supervised medical image segmentation based on a visual language model extracts image and text embeddings using a pre-trained visual language model, generates a text-guided mask based on the embedding, and merges the mask with a pseudo-label to guide semi-supervised learning. The rich a priori knowledge in the pre-trained visual language model is transferred to semi-supervised learning. This is the first attempt to integrate text-guided masks into semi-supervised medical image segmentation, taking advantage of the text to improve the quality of the pseudo-labels and compensate for the quality deficiencies of the pseudo-labels, and taking advantage of the text description to accurately identify the target segmentation region.

[0020] The visual language model with triple semantic uncertainty constraints proposed in this disclosure constrains semantic differences using distribution-based uncertainty levels, and the semantic uncertainty constraints are embedded in each contrastive learning process to constrain semantic differences based on the corresponding uncertainty levels, thereby facilitating the model's understanding of similar but semantically ambiguous data pairs, improving semantic consistency, and reducing the impact of uncertain inputs.

[0021] The triple contrast learning method proposed in this disclosure establishes semantic associations within each modality and promotes the joint representation of images and text by capturing implicit information and contextual relationships within each modality, thereby strengthening the semantic associations between images and text and reducing the uncertainty in cross-modal alignment. The uncertainty in semantic feature alignment is resolved.

[0022] The drawings in the specification that form a part of this disclosure are intended to facilitate a further understanding of the disclosure, and the illustrative examples of the disclosure and their descriptions are intended to interpret the disclosure and are not intended to unduly limit the disclosure. [Brief explanation of the drawings]

[0023] [Figure 1] 1 is a flowchart of a method for semi-supervised medical image segmentation based on a visual language model according to the present disclosure. [Figure 2] 1 is a flowchart of pre-training a visual language model with a triple semantic uncertainty constraint according to the present disclosure. [Figure 3] 1 is a flowchart of text-guided semi-supervised medical image segmentation according to the present disclosure. DETAILED DESCRIPTION OF THE INVENTION

[0024] In the following, the present disclosure will be further explained with reference to the figures and examples.

[0025] It should be noted that the following detailed description is for illustrative purposes only and is intended to further explain the present disclosure. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the art to which this disclosure pertains.

[0026] Where not inconsistent, embodiments and features of embodiments in the present disclosure may be combined with each other.

[0027] Example 1 As shown in FIG. 1, in the first embodiment of the present disclosure, obtaining a medical image to be segmented; inputting the unlabeled images and text descriptions into a visual language model to obtain dense image embeddings and text embeddings, and obtaining a text-guided mask based on the dense image embeddings and text embeddings; The teacher-student model is a semi-supervised segmentation backbone network, and the labeled images are input to the student model to obtain labeled image predictions. The labeled image predictions and the ground truth labels are used to calculate a supervised loss. inputting unlabeled images into a student model and a teacher model, respectively, to obtain unlabeled image predictions and pseudo labels; merging the text-guided masks and the pseudo labels to obtain merged pseudo labels; and calculating a semi-supervised loss using the merged pseudo labels and the unlabeled image predictions; and training a student model based on the supervised loss and the semi-supervised loss, and performing medical image segmentation using the trained student model.

[0028] Here, the visual language model is a pre-trained visual language model, and pre-training the visual language model is specifically as follows: Step 1 to Step 4.

[0029] In step 1, image-text features are extracted. First, the acquired original image is augmented to obtain an augmented image. The original image and the augmented image are input into a visual encoder and a momentum visual encoder, respectively, to obtain original image embeddings and augmented image embeddings. The text descriptions of the original image and the augmented image are input into a text encoder and a momentum text encoder, respectively, to obtain text embeddings and momentum text embeddings. Here, the momentum visual encoder and the momentum text encoder are momentum updated by the visual encoder and the text encoder, respectively, and the momentum update process is expressed as follows:

number

number

[0030] In step 2, we construct an uncertainty semantic constraint policy, calculate the distribution distance as the uncertainty level, and constrain the semantic difference.

[0031] In the case of pair data (x1, x2), the semantic difference between them is

number

number

number

[0032] When comparing two data sets, semantic differences D s (·) and uncertainty level D u (·) are considered simultaneously. D u (·) and D s The ratio of (·) to the relative uncertainty

number

number

number

[0033] In step 3, we construct a triple contrastive learning model based on the uncertainty semantic constraint policy, namely cross-modal contrastive learning, intra-modal contrastive learning, and global-local contrastive learning, and calculate the loss of each contrastive learning model.

[0034] Cross-modal contrastive learning aims to achieve effective alignment between images and text, i.e., to bring matching image-text pairs closer and to separate unmatched image-text pairs. To avoid potential uncertainty issues, we use a semantic uncertainty constraint function D SUC We constrain the cosine similarity sim(I,T) between image-text pairs by (I,T), and the uncertainty cosine similarity

number

number

number

number

number

number

number

number

number

number

[0035] Cross-modal contrastive learning is cmc We align image and text embeddings by optimizing the . However, cross-modal alignment cannot establish semantic associations within each modality. To overcome this limitation, we employ intra-modal contrastive learning based on semantic uncertainty constraints.

[0036] Intramodal contrastive learning aims to provide richer contextual relationships and internal representations by capturing the potential associations between different samples within each modality. At the same time, it also aims to provide a robust internal representation of the uncertainty constraint function D. SUC (I,T) is embedded in intramodal contrastive learning to reduce semantic uncertainty within each modal.

[0037] For image-text pair (I1,T1) and data-augmented image-text pair (I2,T2), the intramodal contrast loss is expressed as:

number

number

number

[0038] Intramodal contrastive training establishes intramodal semantic relevance by learning internal representations of samples within each modality. However, because intramodal contrastive training is based on global features, irrelevant local features may become dominant, causing the model to deviate from the correct learning trajectory. To address this issue, we perform global-local contrastive training based on semantic uncertainty constraints.

[0039] Global-local contrastive learning encourages the model to focus on important local information by understanding the features of different scales, thereby restricting learning of irrelevant local areas. At the same time, the uncertainty constraint function D SUC (I,T) is embedded using global-local contrastive learning to reduce the uncertainty between the global embedding and the local embedding within each modality. Specifically, the global embedding of image I1 is matched with the dense local embedding of augmented image I2, and the text performs a similar operation. The global-local contrast loss is expressed as:

number

number

[0040] In step 4, each contrastive learning loss is used to optimize the parameter weights of the visual language model to obtain a pre-trained visual language model.

[0041] Generate text-guided masks based on a pre-trained visual language model, specifically: Unlabeled images and text descriptions are fed into a pre-trained visual language model for dense image embedding.

number

number

number

number

number

number

number

[0042] The teacher-student model is used as the semi-supervised segmentation backbone network, and pseudo-labels are generated based on the teacher-student semi-supervised segmentation network.

[0043] The architecture of the teacher network and the learning network is the same, both are U-Net networks, and the student network f θs is calculated by the exponential smoothing moving average method. θs The update process is expressed as follows: θ t =αθ t +(1-α)θ s (16) where α∈[0,1] controls the update rate and θ t are the model parameters of the teacher network, and θ s are the model parameters of the student network. The student network is l and unlabeled image x u The teacher network predicts the pseudo-labels of the unlabeled images.

number

number

[0044] We calculate the supervised and semi-supervised losses of the text-guided semi-supervised medical image segmentation network and optimize the semi-supervised medical image segmentation network. Specifically, we do the following:

[0045] For labeled images, the correct label y gt and labeled image prediction y l Based on the supervised loss L sup For unlabeled images, we calculate the unlabeled image prediction.

number

number

[0046] Therefore, the composite loss of the text-guided semi-supervised medical image segmentation network is expressed as follows: L overall =L sup +L semi +γL tg (20) where γ is the text guide loss L tg is the coefficient of

[0047] The parameter weights of the semi-supervised medical image segmentation network are optimized by the composite loss to obtain a trained semi-supervised medical image segmentation network.

[0048] The present disclosure designs a visual language model-driven semi-supervised learning scheme that effectively alleviates the problem of low quality pseudo labels in traditional semi-supervised medical image segmentation, and utilizes the advantages of visual language models to generate text-guided masks and compensate for the quality of pseudo labels in semi-supervised medical image segmentation, thereby improving semi-supervised learning performance.

[0049] This paper proposes a novel uncertainty-aware visual language model, i.e., a triple uncertainty semantic constraint visual language model. Compared with conventional visual language models, our proposed model can effectively solve the uncertainty problem during image-text alignment. Specifically, we propose a semantic uncertainty constraint policy that models two semantic features as two corresponding distribution representations, calculates the distance between these two distributions, and constrains the difference between the two semantic features. This semantic uncertainty constraint policy is simultaneously embedded in cross-modal contrastive training, intra-modal contrastive training, and global-local contrastive training, thereby constraining the semantic difference in each contrastive training process. Furthermore, the additional intra-modal contrastive training and global-local contrastive training strengthen the semantic correlation between cross-modal (i.e., image-text) contexts by learning internal representations within each modality, thereby mitigating the uncertainty during cross-modal alignment.

[0050] Example 2 Example 2 of the present disclosure is a data acquisition module configured to acquire a medical image to be segmented; a text-guided mask generation module configured to input the unlabeled images and text descriptions into a visual language model to obtain dense image and text embeddings, and to obtain a text-guided mask based on the dense image and text embeddings; and a supervised loss calculation module configured to: input labeled images to the student model to obtain labeled image predictions; and calculate a supervised loss using the labeled image predictions and ground truth labels; an unsupervised loss calculation module configured to input unlabeled images to a student model and a teacher model, respectively, to obtain unlabeled image predictions and pseudo labels, merge the text-guided masks and the pseudo labels to obtain merged pseudo labels, and calculate a semi-supervised loss using the merged pseudo labels and the unlabeled image predictions; and a model training module configured to train a student model based on supervised loss and semi-supervised loss, and perform medical image segmentation using the trained student model.

[0051] Since the detailed steps are the same as those in the first embodiment, a detailed description will be omitted here.

[0052] Example 3 A third embodiment of the present disclosure provides a medium having a program stored thereon, the program, when executed by a processor, realizing steps of the method for semi-supervised medical image segmentation based on a visual language model described in the first embodiment of the present disclosure.

[0053] Since the detailed steps are the same as those in the first embodiment, a detailed description will be omitted here.

[0054] Example 4 Example 4 of the present disclosure provides an electronic device including a memory, a processor, and a program stored in the memory and executable by the processor, wherein when the program is executed by the processor, steps of the semi-supervised medical image segmentation method based on a visual language model described in Example 1 of the present disclosure are realized.

[0055] Since the detailed steps are the same as those in the first embodiment, a detailed description will be omitted here.

[0056] The above are only preferred embodiments of the present disclosure, and are not intended to limit the present disclosure. Those skilled in the art can make various modifications and changes to the present disclosure. Any modifications, equivalent replacements, improvements, etc. made without departing from the spirit and principle of the present disclosure shall fall within the protection scope of the present disclosure.

Claims

1. obtaining a medical image to be segmented; inputting the unlabeled images and text descriptions into a visual language model to obtain dense image embeddings and text embeddings, and obtaining a text-guided mask based on the dense image embeddings and text embeddings; The teacher-student model is a semi-supervised segmentation backbone network, and the labeled images are input to the student model to obtain labeled image predictions. The labeled image predictions and the ground truth labels are used to calculate a supervised loss. inputting unlabeled images into a student model and a teacher model, respectively, to obtain unlabeled image predictions and pseudo labels; merging the text-guided masks and the pseudo labels to obtain merged pseudo labels; and calculating a semi-supervised loss using the merged pseudo labels and the unlabeled image predictions; training a student model based on the supervised loss and the semi-supervised loss, and performing medical image segmentation using the trained student model; The visual language model is a pre-trained visual language model, and the visual language model is specifically pre-trained as follows: Enhance the acquired original image to obtain an enhanced image; Input the original image and the augmented image into a visual encoder and a momentum visual encoder, respectively, to obtain the original image embedding and the augmented image embedding; inputting the text descriptions of the original image and the augmented image into a text encoder and a momentum text encoder, respectively, to obtain text embeddings and momentum text embeddings; We construct an uncertainty semantic constraint policy, calculate the distribution distance as the uncertainty level, and constrain the semantic difference. Specifically, Pair data (x １ , x ２ ), their semantic difference is [Equation 35] is the 2-norm, and (s １ , s ２ ) is (x １ , x ２ ) and the semantic embedding (s １ , s ２ ) is N(μ １ , σ １ ) and N(μ ２ , σ ２ ) is a distribution representation, where μ is the mean vector and σ is the standard deviation vector, and the Wasserstein-2 distance is used to measure the difference between the distributions as an uncertainty level, which is expressed as: [Equation 36] Global Semantic Features [CLS] are used to model distributional representations, and the uncertainty level between data pairs is defined as follows: [Equation 37] where a is a positive parameter for controlling the degree of uncertainty, b is the deviation value, When comparing two data, semantic differences D ｓ (·) and uncertainty level D ｕ (·) are considered simultaneously, and D ｕ (・) and D ｓ The ratio of (・) is the relative uncertainty [Equation 38] and the semantic uncertainty constraint function is defined as follows: [0.39] Here, D ＳＵＣ (·) is the semantic uncertainty constraint function, λ is a positive hyperparameter, and Eq. ＳＵＣ (・) Relative uncertainty between the two data points [Equation 40] If is large, D ｓ (x １ , x ２ ) is small, thereby constraining the semantic difference between the two data, We jointly perform triple contrast learning, embed an uncertainty semantic constraint policy in each contrast learning process, and calculate each contrast learning loss. A semi-supervised medical image segmentation method based on a visual language model, characterized in that each contrastive learning loss is used to optimize parameter weights of the visual language model to obtain a pre-trained visual language model.

2. The method for semi-supervised medical image segmentation based on a visual language model according to claim 1 , wherein the triple contrast learning includes cross-modal contrast learning, intra-modal contrast learning, and global-local contrast learning.

3. 3. The method for semi-supervised medical image segmentation based on a visual language model according to claim 2, wherein the cross-modal contrastive learning achieves effective alignment between images and text, the intra-modal contrastive learning captures potential associations between different samples within each modality to provide richer contextual relationships and internal representations, and the global-local contrastive learning understands features at different scales to encourage the model to focus on important local information, thereby limiting the learning of irrelevant local regions.

4. 2. The method for semi-supervised medical image segmentation based on a visual language model according to claim 1, wherein the uncertainty semantic constraint policy specifically models the text embedding and the momentum text embedding as two corresponding distribution representations, calculates the distribution distance between these two distributions as an uncertainty level, and constrains the difference between the text embedding and the momentum text embedding based on the uncertainty level.

5. The method for semi-supervised medical image segmentation based on a visual language model according to claim 1, wherein the momentum visual encoder and the momentum text encoder are momentum updated by the visual encoder and the text encoder, respectively.

6. 2. The method for semi-supervised medical image segmentation based on a visual language model of claim 1, further comprising the steps of: element-wise multiplying the text-guided mask and the dense image embedding to obtain a text-guided embedding; computing a text-guided loss using the text-guided embedding and the text embedding; and optimizing the text-guided mask using the text-guided loss.

7. a data acquisition module configured to acquire a medical image to be segmented; a text-guided mask generation module configured to input the unlabeled images and text descriptions into a visual language model to obtain dense image and text embeddings, and to obtain a text-guided mask based on the dense image and text embeddings; and a supervised loss calculation module configured to: input labeled images to the student model to obtain labeled image predictions; and calculate a supervised loss using the labeled image predictions and ground truth labels; an unsupervised loss calculation module configured to input unlabeled images to a student model and a teacher model, respectively, to obtain unlabeled image predictions and pseudo labels, merge the text-guided masks and the pseudo labels to obtain merged pseudo labels, and calculate a semi-supervised loss using the merged pseudo labels and the unlabeled image predictions; a model training module configured to train a student model based on the supervised loss and the semi-supervised loss, and to perform medical image segmentation using the trained student model; The visual language model is a pre-trained visual language model, and the visual language model is specifically pre-trained as follows: Enhance the acquired original image to obtain an enhanced image; Input the original image and the augmented image into a visual encoder and a momentum visual encoder, respectively, to obtain the original image embedding and the augmented image embedding; inputting the text descriptions of the original image and the augmented image into a text encoder and a momentum text encoder, respectively, to obtain text embeddings and momentum text embeddings; We construct an uncertainty semantic constraint policy, calculate the distribution distance as the uncertainty level, and constrain the semantic difference. Specifically, Pair data (x １ , x ２ ), their semantic difference is [Equation 41] is the 2-norm, and (s １ , s ２ ) is (x １ , x ２ ) and the semantic embedding (s １ , s ２ ) is N(μ １ , σ １ ) and N(μ ２ , σ ２ ) is a distribution representation, where μ is the mean vector and σ is the standard deviation vector, and the Wasserstein-2 distance is used to measure the difference between the distributions as an uncertainty level, which can be denoted as: [Equation 42] Global Semantic Features [CLS] are used to model distributional representations, and the uncertainty level between data pairs is defined as follows: [Equation 43] where a is a positive parameter for controlling the degree of uncertainty, b is the deviation value, When comparing two data, semantic differences D ｓ (·) and uncertainty level D ｕ (·) are considered simultaneously, and D ｕ (・) and D ｓ The ratio of (・) is the relative uncertainty [Equation 44] and the semantic uncertainty constraint function is defined as follows: [Equation 45] Here, D ＳＵＣ (·) is the semantic uncertainty constraint function, λ is a positive hyperparameter, and Eq. ＳＵＣ (・) Relative uncertainty between the two data points [Equation 46] If is large, D ｓ (x １ , x ２ ) is small, thereby constraining the semantic difference between the two data, We jointly perform triple contrast learning, embed an uncertainty semantic constraint policy in each contrast learning process, and calculate each contrast learning loss. A semi-supervised medical image segmentation system based on a visual language model, characterized in that each contrastive learning loss is used to optimize the parameter weights of the visual language model to obtain a pre-trained visual language model.

8. A medium on which a program is stored, the program being characterized in that, when executed by a processor, the steps of the method for semi-supervised medical image segmentation based on a visual language model according to any one of claims 1 to 6 are realized.

9. 10. An electronic device comprising: a memory; a processor; and a program stored in the memory and executable by the processor, the program, when executed by the processor, implementing the steps of the method for semi-supervised medical image segmentation based on a visual language model according to any one of claims 1 to 6.

Images