An Unsupervised Lesion Segmentation Method Based on Large Model and Pseudo-label Learning

By employing an unsupervised lesion segmentation method based on large models and class activation mapping, and utilizing a visual-language model to generate pseudo-labels combined with dynamic occlusion data augmentation, the dependence of deep learning technology on pixel-level annotation is resolved, achieving high-precision lesion segmentation, reducing costs, and improving the model's segmentation performance on complex images.

CN119762774BActive Publication Date: 2025-11-14UNIV OF ELECTRONICS SCI & TECH OF CHINA
View PDF 2 Cites 0 Cited by

Patent Information

Application Number
CN202411786169.4
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-12-06
Publication Date
2025-11-14
Estimated Expiration
2044-12-06

AI Technical Summary

Technical Problem

Existing deep learning techniques rely heavily on pixel-level annotations in lesion image segmentation, which is costly and time-consuming. Furthermore, unsupervised methods perform poorly when processing images with low contrast and unclear boundaries.

Method used

An unsupervised lesion segmentation method based on large models and class activation mapping is adopted. A visual-language model is used to generate classification pseudo-labels. Combined with dynamic occlusion data augmentation strategy, the segmentation model is optimized through self-training strategy to generate high-quality segmentation pseudo-labels.

Benefits of technology

It achieves high-precision lesion segmentation without manual annotation, improves the automation and accuracy of the segmentation model, reduces annotation costs, and enhances segmentation performance in low-contrast and unclear-boundary images.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119762774B_ABST
    Figure CN119762774B_ABST
Patent Text Reader

Abstract

This invention discloses an unsupervised lesion segmentation method based on large model and pseudo-label learning, belonging to the field of image recognition. First, a "Visual-Language" large model (CLIP) is used to generate high-quality classification pseudo-labels, which are then used to supervise the training of an image classification network. During this process, we further extract the class activation map (CAM) of the classification network and introduce a data augmentation strategy based on dynamic occlusion to further improve the performance of the classification network. Subsequently, we utilize these more accurate class activation maps to generate guidance information for the segmentation large model (SAM), thereby obtaining high-quality segmentation pseudo-labels. Finally, this invention proposes a self-training strategy to train and continuously optimize the segmentation network using these pseudo-labels to achieve higher segmentation accuracy and efficiency.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This invention relates to an unsupervised lesion segmentation technique in medical images, belonging to the field of image recognition technology. Background Technology

[0002] In the field of medical image segmentation, accurate identification and separation of lesion regions are crucial for disease diagnosis, treatment planning, and efficacy evaluation. Traditional methods, such as image segmentation techniques based on thresholding, edge detection, and active contours, are effective in some cases, but they mainly rely on low-level image features and struggle to handle complex image conditions such as low contrast, blurred boundaries, and noise interference, thus making it difficult to obtain high-precision segmentation results.

[0003] In recent years, the rise of deep learning technology has brought revolutionary progress to medical image segmentation. Deep learning-based segmentation methods, especially those using convolutional neural networks, have significantly improved the accuracy and efficiency of segmentation due to their powerful feature extraction capabilities. However, these methods rely on a large number of high-quality, pixel-level annotations. Acquiring such densely labeled data is not only costly but also time-consuming, requiring significant involvement from medical professionals. This has become a major bottleneck limiting the widespread adoption of deep learning image segmentation technology in practical applications.

[0004] To address this issue, researchers have begun exploring weakly supervised learning methods to reduce reliance on densely labeled data. Common weakly supervised annotation methods include doodle annotation, bounding box annotation, point annotation, and image-level labeling. While these methods reduce annotation costs to some extent, they still face challenges such as insufficient supervision information and limited segmentation accuracy. Especially when dealing with multi-class segmentation tasks, obtaining a high-accuracy segmentation model with limited annotation information remains extremely difficult. Furthermore, these methods still require a certain amount of manually provided annotations, placing high demands on the annotator's experience and time.

[0005] Unlike weakly supervised segmentation, unsupervised lesion segmentation methods in medical images can achieve image segmentation without manual annotation of training images. Typical unsupervised lesion segmentation methods include color thresholding, clustering, and edge detection. However, these unsupervised image segmentation techniques often perform poorly when dealing with low-contrast and poorly defined boundaries in medical images. These methods typically struggle to automatically learn sufficient features to distinguish different categories or regions from the image, limiting the accuracy and reliability of the segmentation results.

[0006] To address these challenges, this invention proposes an unsupervised lesion segmentation method based on large models and class activation mapping. This method does not rely on any manual annotation. It generates high-quality classification pseudo-labels using a "visual-language" large model (CLIP) and combines this with dynamic occlusion data augmentation strategies and guidance information generated by a segmentation large model (SAM) to achieve high-quality segmentation model construction. This innovative method not only reduces reliance on labeled data but also effectively improves the automation and accuracy of lesion segmentation tasks, providing a new technical approach for the field of medical image segmentation. Summary of the Invention

[0007] This invention aims to address the reliance on pixel-level fine-grained annotation in existing deep learning techniques for lesion image segmentation, proposing an innovative unsupervised segmentation method. This method achieves high-quality segmentation model construction without relying on any annotation costs. Specifically, this invention first uses a Visual-Language Large Model (CLIP) to generate high-quality classification pseudo-labels, which are then used to supervise the training of an image classification network. During this process, we further extract the Class Activation Map (CAM) of the classification network and introduce a data augmentation strategy based on dynamic occlusion to further improve the performance of the classification network. Subsequently, we utilize these more accurate class activation maps to generate guidance information for the Segmentation Large Model (SAM), thereby obtaining high-quality segmentation pseudo-labels. Finally, this invention proposes a self-training strategy to train and continuously optimize the segmentation network using these pseudo-labels to achieve higher segmentation accuracy and efficiency.

[0008] The objective of this invention can be achieved through the following technical solution: an unsupervised lesion segmentation method based on large model and pseudo-label learning; the specific steps are as follows:

[0009] Step 1: Collect a set of unlabeled images as a training set for the lesion segmentation task, and use a "visual-language" model to generate image classification pseudo-labels;

[0010] Step 2: Train a classification model using classification pseudo-labels to generate a preliminary class activation map of lesions on the training set;

[0011] Step 3: Use image occlusion based on class activation maps to augment the data, retrain the classification model, and generate enhanced class activation maps;

[0012] Step 4: Generate segmentation pseudo-labels using the category activation map and the large segmentation model (SAM model);

[0013] Step 5: Use images containing segmentation pseudo-labels as the training set, and train an image segmentation network using an iterative training method based on pseudo-label selection.

[0014] Furthermore, the method for generating classification pseudo-labels in step 1 is as follows:

[0015] Using an image encoder E img and text encoder E text A visual-language model; where c represents the image-level category, c=0 indicates that the image does not contain the target lesion, and c=1 indicates that the image contains the target lesion; and T... c This indicates the prompt statement corresponding to category c; for each image X i , use g c Indicates the prompt statement T c Features, f i Represents image X i Features; Feature g c and f i The calculation method is as follows:

[0016] f i =E img (X i ) Formula 1

[0017] g c =E text (Φ(T c )) Formula 2

[0018] Where Φ represents the word segmenter, which has the function of segmenting the prompt statement T c The function of converting into feature representation; based on this, using Image X i The probability of belonging to category c is calculated as follows:

[0019]

[0020] Where Sim(f) i g c ) is f i and g c The cosine similarity, where j is the class index value; X is obtained through the argmax function. i pseudo-tags y i :

[0021] y i =argmax(p i ) Formula 4.

[0022] Furthermore, the preliminary category activation map generated in step 2 based on the classification model is generated as follows:

[0023] The initial training set with pseudo-labels obtained in step 1 is denoted as . Where N sTo determine the number of 2D slices in the training set, an image classification network θ is trained on dataset D1 containing classification pseudo-labels using standard supervised learning methods and the cross-entropy loss function. c ;

[0024] Based on the trained θ c , for category label y i The image with a value of 1 is used to obtain a preliminary class activation map through gradient backpropagation, which is used to identify the regions in the image that contribute the most to the classification of lesions; on the feature map F of a certain layer in the classification network, using... X represents i The corresponding preliminary category activation map is calculated using the following formula:

[0025]

[0026] Where F k This represents the k-th channel of feature map F. It is a gradient operator; P avg The expression represents the global average pooling operation; ReLU is a linear rectified function used to filter out negative values ​​in the gradient.

[0027] Furthermore, the enhanced category activation map in step 3 is generated as follows:

[0028] For each image marked y i The image X = 1 i Activate graph Q through preliminary category 0 Perform adaptive occlusion-based data augmentation; specifically, first apply a threshold τ to Q... 0 Convert to a binary mask M i :

[0029]

[0030] Then input image X i M i The area with a value of 1 is set to 0, resulting in the occluded image:

[0031]

[0032] For the label y i The image X = 0 i Then by randomly assigning X i The occluded image is obtained by setting a sub-region to zero. The dataset after occlusion enhancement is represented as follows: Using D2, the classification network is retrained through supervised training based on the cross-entropy loss function; after training, the network is re-trained for label y using Equation 5. iThe training images with a value of 1 generate enhanced class activation maps, denoted as . and compared to, It can uncover more marginal areas of lesions, resulting in higher quality findings.

[0033] Furthermore, the method for generating the segmentation pseudo-tags in step 4 is as follows:

[0034] For y i Training image X = 1 i From its corresponding enhanced category activation graph Two types of prompts are generated: one is calculation The bounding box of the region above τ receives a bounding box cue; secondly, the center point of the bounding box is used as the foreground point, and its four vertices are used as background points to form a point cue. These two types of cues are combined and used... X represents i The corresponding prompt message;

[0035] Next and X i Inputting the large segmentation model SAM generates segmentation pseudo-labels; the SAM model contains an image encoder E m A prompt message encoder E prom and a decoder D m To obtain the segmentation mask of the image; for a given image X i The prediction mask M is obtained. i ′ The formula is as follows:

[0036]

[0037] For each y i The image X = 1 i The segmentation mask is obtained using Formula 8 and used as the segmentation pseudo-label; for the labeled y i For images where the value is 0, their segmentation mask is set to empty; finally, the masks of all 2D slices in a 3D medical image are stacked together to form a complete 3D segmentation pseudo-label S. i The corresponding 3D image segmentation training dataset is represented as follows: Where N v This indicates the number of 3D images in the training dataset.

[0038] Furthermore, the iterative training method for the segmentation model based on pseudo-label selection in step 5 is as follows:

[0039] Based on the pseudo-label segmentation, a segmentation model is trained using a two-stage iterative training method. In the first stage, the 3D image segmentation dataset D3 with pseudo-labels is used as the training set, and a 3D segmentation network θ is trained using standard supervised image segmentation model training strategies. s This process uses the Dice loss function and the cross-entropy loss function to update the model parameters;

[0040] Training θ s Then, using its model parameters, predictions are made on all training set images to obtain V. i Updated pseudo tags for For slices containing non-zero regions, calculate the bounding box for each non-zero region. Using the bounding box region, its center point, and corner points, generate SAM cue points as described in step 4. Input the slice and corresponding cue points into the SAM to obtain the corrected SAM result. Stack the results from each slice to form the corrected 3D segmentation result, denoted as...

[0041] In order to evaluate The quality was determined by calculating the SAM correction before and after. and The higher the similarity between them, the greater the similarity. The more reliable; the similarity index is defined as the Dice value between the two:

[0042]

[0043] in The value of is between 0 and 1, and the closer its value is to 1, the stronger it is. and The higher the consistency between them, the better; define a similarity threshold. Used to select highly reliable pseudo-tags; the selected 3D image and its pseudo-tag are denoted as... in Indicates using threshold The number of selected 3D image samples;

[0044] In the second stage, the segmentation model was retrained using the selected pseudo-label dataset D4. The training process also adopted the standard supervised training method, using the Dice loss function and cross-entropy loss function to update the model parameters, thereby obtaining the final automatic lesion segmentation model.

[0045] Compared with the prior art, the present invention has the following advantages:

[0046] (1) In the field of medical image segmentation today, deep learning methods rely heavily on large-scale, high-quality pixel-level labeled images. Acquiring this labeled data is not only time-consuming and labor-intensive, but also extremely costly, often requiring extensive participation from professional medical personnel. The unsupervised lesion segmentation method proposed in this invention does not require any annotations and can achieve high lesion segmentation accuracy without relying on manual annotation.

[0047] (2) Existing unsupervised image segmentation techniques often perform poorly when dealing with images with low contrast and unclear boundaries. In contrast, this invention fully utilizes the powerful feature extraction capabilities of large models and combines a series of innovative strategies such as image occlusion based on class activation maps for data augmentation and pseudo-label similarity filtering, which significantly improves the model's performance at each stage.

[0048] (3) Existing pseudo-label-based image segmentation methods ignore noise in pseudo-labels and lack effective strategies to filter low-quality pseudo-labels. This invention uses the consistency between pseudo-labels predicted by neural networks and pseudo-labels generated by the SAM model to evaluate the quality of pseudo-labels, which can filter out low-quality pseudo-labels and thus improve the performance of the final segmentation model. Attached Figure Description

[0049] Figure 1 This is the overall flowchart of the present invention.

[0050] Figure 2 The category activation diagram and segmentation pseudo-label example diagram of the present invention.

[0051] Figure 3 This image shows a comparison of the segmentation performance of the present invention and existing methods in brain tumors in MRI images; the black line represents the gold standard, and the white area represents the segmentation result of the algorithm. Detailed Implementation

[0052] In conjunction with the content of this invention, the following embodiment of tumor lesion segmentation in MRI images is provided. This embodiment is implemented on a computer with an Intel(R) Core(TM) i7-6850K CPU@3.60GHZ, a GeForce GTX1080Ti GPU, and 32.0GB of memory, and the programming language is Python.

[0053] Step 1: Dataset Construction and Classification Pseudo-label Generation. The dataset was constructed using 369 3D brain MRI images from the BraTS public dataset, with 258 images used as the training set, 37 as the validation set, and 74 as the test set. For each image, mean-standard deviation normalization (Z-score Normalization) was applied to convert the pixel values ​​of the brain region into a normal distribution with a mean of 0 and a standard deviation of 1, and the segmentation labels of the training images were discarded.

[0054] Generating classification pseudo-labels. In this example, the industry-standard trained "vision-language" model CLIP is used to generate classification pseudo-labels for images. CLIP includes an image encoder E. img and text encoder E text A visual-language model. Let c represent the image-level category, where c=0 indicates the image does not contain the target lesion, and c=1 indicates the image contains the target lesion. T... c The prompt statements corresponding to category c are: T0 = "This is an MRI image that does not contain a brain tumor, and there are no significantly highlighted lesions"; T1 = "This is an MRI image that contains a brain tumor, and there are significantly highlighted lesions." For each image X... i , use g c Indicates the prompt statement T c Features, f i Represents image X i Features. Feature g c and f i The calculation method is as follows:

[0055] f i =E img (X i ) Formula 1

[0056] g c =E text (Φ(T c )) Formula 2

[0057] Where Φ represents the word segmenter, which has the function of segmenting the prompt statement T c This involves converting the data into feature representations. Based on this, using... Image X i The probability of belonging to category c is calculated as follows:

[0058]

[0059] Where Sim(f) i g c ) is f i and g cThe cosine similarity is given by , where j is the class index. X is obtained using the argmax function. i pseudo-tags y i :

[0060] y i =argmax(p i ) Formula 4

[0061] Step 2: Train a classification network and obtain a preliminary class activation map:

[0062] The initial training set with pseudo-labels obtained in step 1 is denoted as . Where N s The number of 2D slices in the training set is given. On dataset D1 containing classification pseudo-labels, an image classification network θ is trained using standard supervised learning methods and the cross-entropy loss function. c The classification network uses a pre-trained ResNet50 architecture. The training process employs the Adam optimizer with a learning rate of 0.001 and 100 epochs.

[0063] Based on the trained θ c , for category label y i The image with a value of 1 is used to obtain a preliminary class activation map through gradient backpropagation, which is used to identify the regions in the image that contribute the most to lesion classification. On the feature map F of the third convolutional module in the ResNet50 classification network, [the following is used]... X represents i The corresponding preliminary category activation map is calculated using the following formula:

[0064]

[0065] Where F k This represents the k-th channel of feature map F. It is a gradient operator. P avg The expression represents the global average pooling operation. ReLU is a linear rectified function used to filter out negative values ​​in the gradient.

[0066] Step 3: After enhancing the training set based on the initial class activation map, retrain the classification network to obtain the enhanced class activation map:

[0067] For each image marked y i The image X = 1 i Activate graph Q through preliminary category 0 Adaptive occlusion-based data augmentation is performed. Specifically, Q is first augmented using a threshold τ = 0.5. 0 Convert to a binary mask M i :

[0068]

[0069] Then input image X i M i The area with a value of 1 is set to 0, resulting in the occluded image:

[0070]

[0071] For the label y i The image X = 0 i Then by randomly assigning X i The occluded image is obtained by setting a sub-region to zero. The dataset after occlusion enhancement is represented as follows: Using D2, the classification network is retrained through supervised training based on the cross-entropy loss function. After training, the classification network is re-trained with the label y using Equation 5. i The training images with a value of 1 generate enhanced class activation maps, denoted as . and compared to, It can uncover more marginal areas of lesions, resulting in higher quality findings.

[0072] Step 4: Generate segmentation pseudo-labels using enhanced class activation maps and SAM:

[0073] For y i Training image X = 1 i From its corresponding enhanced category activation graph Two types of prompts are generated: one is calculation The bounding box of the region above τ receives a bounding box cue; secondly, the center point of the bounding box is used as the foreground point, and its four vertices as background points to form a point cue. These two types of cue information are combined and used... X represents i The corresponding prompt message.

[0074] Next and X i Inputting the data into the large segmentation model SAM generates segmentation pseudo-labels. The SAM model contains an image encoder E. m A prompt message encoder E prom and a decoder D m This is used to obtain the image segmentation mask. In this example, the "ViT-B" pre-trained model is used for the SAM image encoder. For a given image X... i The prediction mask M is obtained. i ′ The formula is as follows:

[0075]

[0076] For each y i The image X = 1 i The segmentation mask is obtained using Formula 8 and used as the segmentation pseudo-label. For the labeled y i For images where the value is 0, the segmentation mask is set to empty. Finally, the masks of all 2D slices in a 3D medical image are stacked together to form a complete 3D segmentation pseudo-label S. i The corresponding 3D image segmentation training dataset is represented as follows: Where N v This indicates the number of 3D images in the training dataset.

[0077] Step 5: Iterative training of the segmentation model based on pseudo-label selection. Based on the segmentation pseudo-labels, an iterative training method with two stages is used to train a segmentation model. In this embodiment, the 3D image segmentation model adopts the 3DUNet structure.

[0078] In the first stage, the 3D image segmentation dataset D3 with pseudo-labels is used as the training set, and a 3D segmentation network θ is trained using standard supervised image segmentation model training strategies. s The process uses the Dice loss function and cross-entropy loss function to update the model parameters, with Adam as the optimizer, 100 iterations, and a learning rate of 0.001.

[0079] Training θ s Then, using its model parameters, predictions are made on all training set images to obtain V. i Updated pseudo tags for For slices containing non-zero regions, calculate the bounding box for each non-zero region. Using the bounding box region, its center point, and corner points, generate SAM cue points as described in step 4. Input the slice and the corresponding cue points into the SAM to obtain the corrected SAM result. Stack the results from each slice to form the corrected 3D segmentation result, denoted as [the result is missing from the original text].

[0080] In order to evaluate The quality was determined by calculating the SAM correction before and after. and The higher the similarity between them, the greater the similarity. The more reliable, the better. The similarity index is defined as the Dice value between the two:

[0081]

[0082] in The value of is between 0 and 1, and the closer its value is to 1, the stronger it is. and The higher the consistency between them, the better. Define a similarity threshold. Used to select pseudo-tags with high reliability. The selected 3D image and its pseudo-tag are denoted as... in Indicates using threshold Number of selected 3D image samples.

[0083] In the second stage, the segmentation model was retrained using the selected pseudo-label dataset D4. The training process also adopted the standard supervised training method, using the Dice loss function and cross-entropy loss function to update the model parameters, thereby obtaining the final lesion automatic segmentation model 3D UNet.

[0084] After training, the final 3D UNet model is used to infer the results from the samples in the test set.

[0085] Figure 2 These are example images of the class activation maps and pseudo-labels on the training set based on the method of the present invention. It can be seen that the initial class activation map locates the lesion region to a certain extent, and the enhanced class activation map can more fully explore the lesion region. Based on the cues generated by the class activation map, the segmentation pseudo-labels obtained using SAM have good quality.

[0086] Figure 3 This paper compares the segmentation model trained by the unsupervised learning method of this invention with other existing unsupervised learning methods in the segmentation of brain tumor lesions. It can be seen that the segmentation results obtained are more accurate compared to other methods.

Claims

1. An unsupervised lesion segmentation method based on large model and pseudo-label learning; the specific steps are as follows: Step 1: Collect a set of unlabeled images as a training set for the lesion segmentation task, and use a "visual-language" model to generate image classification pseudo-labels; Step 2: Train a classification model using classification pseudo-labels to generate a preliminary class activation map of lesions on the training set; The preliminary category activation map is generated as follows: The initial training set with pseudo-labels obtained in step 1 is denoted as . Where N s To determine the number of 2D slices in the training set, an image classification network θ is trained on dataset D1 containing classification pseudo-labels using standard supervised learning methods and the cross-entropy loss function. c ; Based on the trained θ c , for category label y i The image with a value of 1 is used to obtain a preliminary class activation map through gradient backpropagation, which is used to identify the regions in the image that contribute the most to the classification of lesions; on the feature map F of a certain layer in the classification network, using... X represents i The corresponding preliminary category activation map is calculated using the following formula: Where F k This represents the k-th channel of feature map F. It is a gradient operator; P avg The expression represents the global average pooling operation; ReLU is a linear rectified function used to filter out negative values ​​in the gradient. Step 3: Use image occlusion based on class activation maps to augment the data, retrain the classification model, and generate enhanced class activation maps; The enhanced category activation map is generated as follows: For each image marked y z The image X = 1 z Activate graph Q through preliminary category 0 Perform data augmentation based on adaptive occlusion; The specific method involves first setting Q using a threshold τ. 0 Convert to a binary mask M i : Then input image X z M z The area with a value of 1 is set to 0, resulting in the occluded image: For the label y z The image X = 0 z Then by randomly assigning X z The occluded image is obtained by setting a sub-region to zero. The dataset after occlusion enhancement is represented as follows: Using D2, the classification network is retrained through supervised training based on the cross-entropy loss function; After training is completed, proceed through step 2. The calculation formula is re-labeled as y z The training images with a value of 1 generate enhanced class activation maps, denoted as . and compared to, It can uncover more of the lesion's edge area, resulting in higher quality findings; Step 4: Generate segmentation pseudo-labels using the category activation map and the large segmentation model (SAM model); Step 5: Use images containing segmentation pseudo-labels as the training set, and train an image segmentation network using an iterative training method based on pseudo-label selection; The iterative training method based on pseudo-label selection is as follows: Based on the pseudo-label segmentation, a segmentation model is trained using a two-stage iterative training method. In the first stage, the 3D image segmentation dataset D3 with pseudo-labels is used as the training set, and a 3D segmentation network θ is trained using standard supervised image segmentation model training strategies. s This process uses the Dice loss function and the cross-entropy loss function to update the model parameters; Training θ s Then, using its model parameters, predictions are made on all training set images to obtain V. p Updated pseudo tags for For slices containing non-zero regions, calculate the bounding box for each non-zero region in the slice. Using the bounding box region and its center point and corner points as in step 4, generate SAM hints. Input the slice and the corresponding hints into SAM to obtain the SAM correction result. The results from each slice are stacked, and the resulting 3D segmentation correction result is denoted as... In order to evaluate The quality was determined by calculating the SAM before and after correction. and The higher the similarity between them, the greater the similarity. The more reliable; the similarity index is defined as the Dice value between the two: in The value of is between 0 and 1, and the closer its value is to 1, the stronger it is. and The higher the consistency between them, the better; define a similarity threshold. Used to select highly reliable pseudo-tags; the selected 3D image and its pseudo-tag are denoted as... in Indicates using threshold The number of selected 3D image samples; In the second stage, the segmentation model was retrained using the selected pseudo-label dataset D4; The training process also adopts the standard supervised training method, using the Dice loss function and cross-entropy loss function to update the model parameters, thereby obtaining the final automatic lesion segmentation model.

2. The unsupervised lesion segmentation method based on large model and pseudo-label learning as described in claim 1, characterized in that, The method for generating classification pseudo-labels in step 1 is as follows: Using an image encoder E img and text encoder E text A visual-language model; where c represents the image-level category, c=0 indicates that the image does not contain the target lesion, and c=1 indicates that the image contains the target lesion; and T... c This indicates the prompt statement corresponding to category c; for each image X n , use g c Indicates the prompt statement T c Features, f n Represents image X n Features; Feature g c and f n The calculation method is as follows: f n =E img (X n ) g c =E text (Φ(T c )) Where Φ represents the word segmenter, which has the function of segmenting the prompt statement T c The function of converting into feature representation; based on this, using Image X n The probability of belonging to category c is calculated as follows: Where Sim(f) n g c ) is f n and g c The cosine similarity, where j is the class index value; X is obtained through the argmax function. n pseudo-tags y n : y n =argmax(p n )。 3. The unsupervised lesion segmentation method based on large model and pseudo-label learning as described in claim 1, characterized in that, The method for generating segmentation pseudo-tags in step 4 is as follows: For y q Training image X = 1 q From its corresponding enhanced category activation graph Two types of prompts are generated: one is calculation The bounding box of the region above τ receives a bounding box cue; secondly, the center point of the bounding box is used as the foreground point, and its four vertices are used as background points to form a point cue. These two types of cues are combined and used... X represents q The corresponding prompt message; Next and X q Inputting the large segmentation model SAM generates segmentation pseudo-labels; the SAM model contains an image encoder E m A prompt message encoder E prom and a decoder D m To obtain the segmentation mask of the image; for a given image X q The prediction mask M is obtained. ′ q The formula is as follows: For each image X labeled q=1 q Using M ′ q The calculation formula is used to obtain the segmentation mask as the segmentation pseudo-label; for the labeled y q For images with a value of 0, set their segmentation mask to empty; Finally, the masks of all two-dimensional slices in a three-dimensional medical image are stacked together to form a complete 3D segmentation pseudo-label S. q The corresponding 3D image segmentation training dataset is represented as follows: Where N v This indicates the number of 3D images in the training dataset.

Citation Information

Patent Citations

  • Image processing method and device, equipment and computer storage medium

    CN117009560A

  • Method for few-shot learning and apparatus for executing the method

    US20210365788A1