Semi-supervised segmentation method, system, device and medium for intestinal ultrasound images

Through semi-supervised learning and Mean-Teacher model, pseudo-label and noise label are generated using labelless data. Combined with medical big model knowledge, the noise label problem in intestinal ultrasound image segmentation is solved, and the segmentation accuracy and model robustness are improved. Effective segmentation in small sample scenarios is realized.

CN118762039BActive Publication Date: 2024-12-06JIANGSU OPEN UNIVERSITY (THE CITY VOCATIONAL COLLEGE OF JIANGSU)
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202411245344.9
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-09-06
Publication Date
2024-12-06
Estimated Expiration
2044-09-06

AI Technical Summary

Technical Problem

Intestinal ultrasound image segmentation has noise, artifacts, low ratio and lack of labeling data, resulting in serious noise labeling problems in small sample scenarios and lack of effective solutions.

Method used

Using semi-supervised learning method, pseudo-labels are generated using label-free intestinal ultrasound images, and through Mean-Teacher model and noise perturbation training, the robustness and generalization of the model in small sample scenarios are optimized, combined with medical big model knowledge Dual-level training and noise tag learning.

Benefits of technology

The accuracy of intestinal ultrasound image segmentation and the robustness of the model are improved, the segmentation performance in small sample scenarios is enhanced, and the implicit knowledge is extracted through noisy tag data, which enhances the generalization ability of the model.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118762039B_ABST
    Figure CN118762039B_ABST
Patent Text Reader

Abstract

The present invention aims at the problem of little labeled data and differentiated label quality of intestinal ultrasound images, overcomes the small sample problem in the intestinal ultrasound image segmentation scenario, and discloses a semi-supervised segmentation method, system, device and medium for intestinal ultrasound images, which belongs to the field of computer vision; the method includes: using unlabeled intestinal ultrasound images and real labeled intestinal ultrasound images to perform two-stage training on a large medical model to obtain pseudo labels; adding noise disturbance to the intestinal ultrasound images to form noisy labeled data with pseudo labels; training a Mean-Teacher model using noisy labeled data and real labeled intestinal ultrasound images; using the trained Mean-Teacher model to predict the pixel-level labels of intestinal ultrasound images, and then obtaining the intestinal ultrasound image segmentation results. The present invention improves the robustness and generalization of the model.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention belongs to the field of artificial intelligence and computer vision, and specifically relates to medical large models and deep learning technology, and in particular to a semi-supervised segmentation method, system, device and medium for intestinal ultrasound images. Background Art

[0002] Applying deep learning to medical imaging, especially using image segmentation technology to assist doctors in diagnosis, has become a research hotspot today. However, intestinal ultrasound images contain noise, artifacts, and low contrast. In addition, the intestinal anatomical structure is complex and there is a lack of labeled data. These reasons lead to many challenges in intestinal ultrasound image segmentation.

[0003] Although a large amount of ultrasound data is generated in major hospitals every moment, it is very expensive to obtain a large number of accurately labeled ultrasound medical images because medical images involve patient privacy issues and understanding medical ultrasound images requires domain expertise and must be annotated by professional doctors. At present, there is no public medical ultrasound database, and deep learning cannot be separated from data-driven, so how to make full use of a large amount of unlabeled data is of great research significance.

[0004] The development of artificial intelligence technology has triggered a profound change in the field of medical image processing. Automated methods can reduce subjective differences between observers and achieve quantification of intestinal features. The literature (“Assessing small bowel stricturing and morphology in Crohn's disease using semi-automated image analysis”, Inflammatory Bowel Diseases, 2020, 26(5): 734-742.) proposed a morphomic method for measuring intestinal wall thickness and intraluminal expansion diameter, which effectively overcomes the subjective errors of manual observation. In another retrospective, multicenter CT enterography study (“Development and validation of a novel computed-tomography enterography radiomic approach for characterization of intestinal fibrosis in Crohn's disease”, Gastroenterology, 2021, 160(7): 2303-2316.), this work designed a machine learning-based radiomic model for the classification of intestinal fibrosis. The literature (“Curved planar reformatting and convolutional neural network‐based segmentation of the small bowel for visualization and quantitative assessment of pediatric Crohn's disease from MRI”, Journal of Magnetic Resonance Imaging, 2019, 49(6):1565-1576.) uses curved planar reformatting and convolutional neural network methods to segment the target area based on MRI images of patients with intestinal diseases.The paper (“Automatic segmentation and radiomics for identification and activity assessment of CTE lesions in Crohn's disease”, Inflammatory Bowel Diseases, 2023: 285.) designed an automatic segmentation model using radiomics and deep learning methods to segment intestinal targets in computed tomography images. The paper (“Gadolinium-Free Crohn's Disease Assessment from Magnetic Resonance Enterography Data”, ISBI, 2022: 1-5.) designed an anatomically constrained method to automatically segment T1 images in computed tomography images and simultaneously obtain intestinal wall thickness from T2-weighted images. The paper (“Automatic detection of Crohn's disease using quantified motility in magnetic resonance enterography: initial experiences”, Clinical Radiology, 2022, 77(2): 96-103.) proposed a fully automatic region of interest selection method and studied the contribution of motility metrics of areas outside the small intestine to intestinal ultrasound detection. It can be seen from the above work that there is still little work on intestinal ultrasound images, and the above work has not optimized the model for small sample scenarios, and the problem of possible mislabeling in the labels has not been solved. Summary of the invention

[0005] In view of the deficiencies in the prior art, the present invention provides a method, system, device and medium for semi-supervised segmentation of intestinal ultrasound images. From the perspective of data quality, the present invention uses abundant unlabeled ultrasound images to design a semi-supervised learning method, and regards the generated pseudo-labels as labels with noise. The targets in the intestine are segmented through image segmentation technology. This method integrates the knowledge of large models and solves the problem of noisy labels in small sample scenarios. On this basis, the key indicators of the intestine are accurately quantified, and the robustness and generalization of the model are further improved, thereby improving the accuracy of image segmentation.

[0006] To achieve the above object, the present invention adopts the following technical solutions:

[0007] A semi-supervised segmentation method for intestinal ultrasound images combined with medical large model knowledge includes the following steps:

[0008] S1. Use unlabeled intestinal ultrasound images and true labeled intestinal ultrasound images to perform dual-stage training on the medical large model to obtain pseudo labels;

[0009] S2, adding noise disturbance to the unlabeled intestinal ultrasound image to form noisy labeled data with pseudo labels;

[0010] S3, train the Mean-Teacher model using noisy labeled data and true labeled intestinal ultrasound images;

[0011] S4. Use the trained Mean-Teacher model to predict the pixel-level labels of the intestinal ultrasound image, and then obtain the intestinal ultrasound image segmentation results.

[0012] To optimize the above technical solutions, the specific measures taken also include:

[0013] Furthermore, in S1, the medical large model includes Totalsegmentator and SAM-Med3D; the process of the two-stage training is specifically as follows:

[0014] The unlabeled intestinal ultrasound images and the true labeled intestinal ultrasound images are sent to Totalsegmentator for training, where the true labeled intestinal ultrasound images are used as domain knowledge to guide the fine-tuning of Totalsegmentator. After the training is completed, the unlabeled intestinal ultrasound images are inferred to generate rough segmentation results, and the rough segmentation results are encoded. The unlabeled intestinal ultrasound images, the true labeled intestinal ultrasound images and the encoded rough segmentation results are input into SAM-Med3D for training. The rough segmentation results are used as prompts to guide the SAM-Med3D model training. After the SAM-Med3D training is completed, the inference results are obtained, and the inference results of SAM-Med3D are used as pseudo labels.

[0015] Further, in S3, the Mean-Teacher model includes a Teacher model and a Student model, and the process of training the Mean-Teacher model is:

[0016] In each training round, the following operations are performed until the maximum number of iterations is reached:

[0017] The Teacher model is trained using noisy labeled data, and the Student model is trained using noisy labeled data and true labeled intestinal ultrasound images; the Teacher model outputs predicted labels for annotating pixel categories;

[0018] Calculate the confidence of the predicted label output by the Teacher model, take the pixel with the lowest confidence as the mislabeled pixel, and obtain the error estimation map;

[0019] Calculate the uncertainty of the image labeled by the predicted label;

[0020] The noisy label map output by the Teacher model is denoised using the error estimation map and uncertainty to obtain a smoothed label map.

[0021] Calculate the value of the loss function of the Mean-Teacher model to update the Mean-Teacher model;

[0022] The Student model updates the parameters of the Student model through back propagation, and the Teacher model is updated based on the parameters of the Student model using the exponential sliding average strategy.

[0023] Furthermore, the specific process of calculating the confidence of the predicted label output by the Teacher model and taking the pixel with the lowest confidence as the mislabeled pixel to obtain the error estimation map is as follows:

[0024] Output the predicted label from the Teacher model and convert the pixel The actual category is equal to its predicted label The probability of the category is used as the confidence of the predicted label, expressed as , i is the predicted label The value of represents the category of the labeled pixel x; the confidence that pixel x belongs to category j , then the potential true label of pixel x is considered to be category j instead of i, t j is the category decision threshold of category j, by marking each pixel as Confidence The expected value is expressed as , where Indicates that all the images marked as The sum of pixels of ; According to this threshold strategy, the confidence joint matrix of each image X is obtained :

[0025] C y l , y l * [ i ][ j ]: = | X ̂ y l = i , y l * = j |

[0026] In the formula, C y l , y l * [ i ][ j ] is an element in the confidence joint matrix, The meaning of is the category confidence value, which means "the predicted label The value is category i, but the potential true label The value of is the confidence of the event "category j", and its expression is as follows:

[0027]

[0028] In the formula, is the confidence that pixel x belongs to category j, t j is the category decision threshold of category j, Indicates that all the images marked as The sum of pixels of The value of represents the category of the labeled pixel x. Indicates category is the confidence level in all categories The largest class, and the confidence is greater than the category decision threshold of the corresponding category c ; represents a set of m class labels;

[0029] Calibrate the confidence joint matrix:

[0030] C ̃ y l , y l * [ i ][ j ] = C y l , y l * [ i ][ j ] ∑ j ∈ C m C y l , y l * [ i ][ j ] ⋅ | X y l = i |

[0031] In the formula, C ̃ y l , y l * [ i ][ j ] represents the elements in the calibrated confidence joint matrix, Indicates that all categories in the image are The pixel value of

[0032] Normalized calibrated confidence joint matrix:

[0033] Q ̃ y l , y l * [ i ][ j ] = C ̃ y l , y l * [ i ][ j ] ∑ i ∈ C m , j ∈ C m C ̃ y l , y l * [ i ][ j ]

[0034] In the formula, Q ̃ y l , y l * [ i ][ j ] represents the elements in the normalized calibrated confidence joint matrix, Represents a set of m class labels;

[0035] The category pruning strategy is used to identify the noise labels, specifically:

[0036] For each class , choose the lowest confidence of T ⋅ ∑ j ∈ C m : j ≠ i ( Q ̃ y l , y l * [ i ][ j ]) pixels as mislabeled pixels, T represents the total number of pixels in the image, and the error estimation map is obtained. .

[0037] Furthermore, the specific process of calculating the uncertainty of the image marked with the predicted label is:

[0038] The prediction entropy is used to estimate the uncertainty of each pixel:

[0039]

[0040] In the formula, represents the probability that the t-th pixel is predicted to be the c-th class, T represents the total number of pixels in the image, Represents the uncertainty of each pixel;

[0041] The uncertainty of the overall image is expressed as U; the normalized overall image uncertainty is expressed as ;

[0042] The method of denoising the noisy label graph output by the Teacher model using the error estimation graph and uncertainty is as follows:

[0043]

[0044] In the formula, represents the error estimation graph, is a noisy label image, is the smoothed label map.

[0045] Furthermore, the loss function of the Mean-Teacher model is as follows:

[0046]

[0047] Where L represents the loss function of the Mean-Teacher model, is the supervision loss acting on the true label, is a perturbation-based consistency loss acting on noisy labeled data, is the supervised loss for calibration of noisy label data; λ and β are weights respectively, and λ is a Gaussian process independent of time. Sure, is a constant parameter, representing the maximum value that can be achieved in weight scheduling. is the maximum number of training times, and k is the number of training the Mean-Teacher model.

[0048] Furthermore, the Teacher model is updated using an exponential moving average strategy according to the parameters of the Student model as follows:

[0049]

[0050] In the formula, is the sliding mean decay rate, is the Student model weight when the number of training times is k, is the weight of the updated Teacher model, is the weight of the Teacher model before updating.

[0051] The present invention also proposes a semi-supervised segmentation system for intestinal ultrasound images combined with medical large model knowledge, comprising:

[0052] A two-stage training module is used to perform two-stage training on the medical large model using unlabeled intestinal ultrasound images and true labeled intestinal ultrasound images to obtain pseudo labels;

[0053] The noise adding module is used to add noise disturbance to the unlabeled intestinal ultrasound image to form noisy labeled data with pseudo labels;

[0054] Noisy label learning module, used to train the Mean-Teacher model using noisy label data and true label intestinal ultrasound images;

[0055] The image segmentation module is used to predict the pixel-level labels of intestinal ultrasound images using the trained Mean-Teacher model, and then obtain the intestinal ultrasound image segmentation results.

[0056] The present invention also proposes an electronic device, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the semi-supervised segmentation method of intestinal ultrasound images combined with medical large model knowledge as described above is implemented.

[0057] The present invention also proposes a computer-readable storage medium storing a computer program, wherein the computer program enables a computer to execute the semi-supervised segmentation method of intestinal ultrasound images combined with medical large model knowledge as described above.

[0058] The beneficial effects of the present invention are:

[0059] 1) In a large number of unlabeled image scenarios, based on the pre-trained medical large model, the model is applied with personalized knowledge in a specific scenario, and the large model is fine-tuned to generate refined pseudo labels. A large number of unlabeled images and a small amount of high-quality labeled data are used to generate refined pseudo labels (noisy labels) as data support for subsequent noisy label learning.

[0060] 2) This paper proposes a Mean-Teacher-assisted confidence learning method to make full use of these noisy labels, extract implicit knowledge from images in low-quality label data through unsupervised consistency based on perturbations, and the high-quality labels grasp the main optimization direction of the model, so that the model can robustly learn segmentation from a small number of high-quality labels and a large number of low-quality labels. Ultimately, the segmentation model obtains a large amount of domain knowledge and supervisory signals from unlabeled images, thereby enhancing the robustness and generalization of the model. BRIEF DESCRIPTION OF THE DRAWINGS

[0061] Figure 1Comparison of the segmentation results of manual annotation of intestinal ultrasound images and the image segmentation method proposed in the present invention.

[0062] Figure 2 This is a framework diagram of the semi-supervised segmentation method for intestinal ultrasound images that combines medical large model knowledge proposed in the present invention. DETAILED DESCRIPTION

[0063] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0064] Embodiment 1

[0065] The present invention proposes a semi-supervised segmentation method for intestinal ultrasound images combined with medical large model knowledge. The overall framework of the method is as follows: Figure 2 As shown, the following steps are included:

[0066] S1. Use unlabeled intestinal ultrasound images and real-label intestinal ultrasound images to perform two-stage training on the medical large model to obtain pseudo labels; the real-label intestinal ultrasound images are manually labeled in advance.

[0067] At present, large models emerge in an endless stream. The first large model proposed in the field of image segmentation is SAM ("Segmentanything"), and subsequently some advanced large models based on SAM were proposed in the field of medical image segmentation. The large medical image segmentation models Totalsegmentator and SAM-Med3D have been proposed recently. Totalsegmentator and SAM-Med3D have learned knowledge of multiple modalities and multiple organs through training on a large amount of medical data. The present invention rationally utilizes the medical image segmentation knowledge already contained in Totalsegmentator and SAM-Med3D, and studies a two-stage training method, which uses a large number of unlabeled ultrasound images to generate more refined pseudo-labels, thereby increasing the amount of labeled data. The detailed steps are as follows: Figure 2As shown in the upper part. In the first step, a large number of unlabeled intestinal ultrasound images and a small amount of high-quality labeled ultrasound data (real label intestinal ultrasound images) are sent to Totalsegmentator for training. The real label intestinal ultrasound images are used as domain knowledge to guide the fine-tuning of Totalsegmentator. After the training is completed, a large amount of unlabeled data is inferred to generate rough segmentation results. Further, these rough segmentation results are encoded and input into SAM-Med3D as prompts to guide model training. At the same time, a large number of unlabeled intestinal ultrasound images and a small amount of high-quality labeled data (real label intestinal ultrasound images) are also input into SAM-Med3D. The prompts contain a large amount of prior knowledge generated by Totalsegmentator, which can provide supervision information for SAM-Med3D. Through the above two stages of training, the reasoning results of SAM-Med3D are finally used as the refined pseudo-labels. Although some of the pseudo-labels have high labeling accuracy, there are still a large number of mislabeled areas.

[0068] S2. Add noise disturbance to the unlabeled intestinal ultrasound images to form noisy labeled data with pseudo labels. In the present invention, the noise label learning task is how to use a large amount of noisy labeled data and a small amount of real labeled intestinal ultrasound images to train the Mean-Teacher model.

[0069] S3, train the Mean-Teacher model using noisy labeled data and true labeled intestinal ultrasound images;

[0070] In this step, a Mean-Teacher-assisted confidence learning method is designed to robustly segment images from a small number of real labels and a large number of low-quality labels (pseudo labels). Under the same input, the Teacher model and the Student model that are subjected to different perturbations are kept consistent to explore the implicit knowledge in the noisy labels. At the same time, the network continuously describes the pixel-level label error map through confidence learning. The error map determines the confidence of whether each pixel is segmented correctly, and further refines the label according to the proposed label smoothing strategy. Figure 2 The lower part is the core steps of the algorithm, and the details of the proposed method will be introduced in detail below.

[0071] The noisy label data is represented as The data annotated by experts are high-quality labels, expressed as .in represents the input image, is a pseudo label, is the true label. Overall, the Mean-Teacher model consists of a Student model and a weighted average Teacher model, which performs self-denoising on noisy labels. The model not only benefits from the high-quality labels annotated by experts, but also can learn the implicit knowledge in the noisy labels by using consistency regularization.

[0072] The Student model in the framework updates the parameters of the Student model through back propagation, while the parallel Teacher model is updated according to the parameters of the Student model using an exponential sliding average strategy. This design can distinguish high-quality labels and noisy label data and process them separately. The real label provides the main guidance for training the Student model. The Student model uses the knowledge of the real label data to guide the training of the Teacher model and make better use of a large number of noisy labels through the Teacher model. At the same time, the self-integrated Teacher model can act as a "third party" to continuously provide out-of-sample prediction probabilities for the characterization process of class-conditional label noise during training.

[0073] Formally, this application expresses the weight of the Student model when the number of training times is k as , the weight parameter of the Teacher model Update using exponential moving average strategy:

[0074] (1)

[0075] In the formula, is the sliding average attenuation rate, which is set to 0.99 in the present invention. is the weight of the Teacher model before updating, is the weight of the updated Teacher model.

[0076] Based on the smoothness assumption, when the inputs of the two model branches are the same, the prediction results of the Teacher model are kept consistent with the output results of the Student model under different perturbations (perturbation refers to adding random Gaussian noise to the input image). Figure 2 As shown, for the Student model, the training process is subject to the fully supervised loss function and unsupervised consistency loss Supervision. Acting on expert-annotated true-label intestinal ultrasound images, The difference between the predicted probabilities of the Student model and the Teacher model is measured to ensure that the difference between the two predictions is not too large. Works on noisy labeled data.

[0077] In order to make full use of the knowledge in noisy label data, this paper designs a collaborative noisy label self-denoising method, aiming to improve the segmentation performance of the model and alleviate the misleading of noisy data on model optimization. First, this application introduces the class conditional classification noise process hypothesis: i. Given a noisy label (Observed, i.e. the refined pseudo-label obtained in the first step), the original image in the noisy data Every pixel in There is a potential true label ;ii. Labels in the class may be misidentified as The probability of misidentification is ( is a set of m class labels).

[0078] Confidence learning has been proposed in image-level classification tasks, where it is mentioned that a joint confidence matrix can robustly characterize the arbitrary uncertainty of the potential (true) label. Therefore, the present invention proposes a confidence learning module for describing a specific pixel-level error estimation map, that is, indicating which pixels are incorrectly labeled. At this time, the Teacher model acts as a "third party" to provide out-of-sample prediction probabilities Output the predicted label from the Teacher model and convert the pixel The actual category is equal to the predicted label The probability of the category is used as the confidence of the predicted label, expressed as . i is the predicted label The value of represents the category of the labeled pixel x. If the confidence is low, it can be considered that the pixel is incorrectly labeled. (Its label is ), the pixel belongs to the category Confidence , t j is the category decision threshold of category j, then pixel Potential true labels Can be identified as a category Instead of other categories. Here, the category decision threshold of category j is By labeling each pixel as Confidence The expected method is obtained, which is expressed as , where Indicates that all the labels in the image are According to this threshold strategy, each image The confidence matrix of Defined as:

[0079] C y l , y l * [ i ][ j ]: = | X ̂ y l = i , y l * = j | (2)

[0080] In the formula, C y l , y l * [ i ][ j ] is an element in the confidence joint matrix, The meaning of is the category confidence value, which means "the predicted label The value is category i, but the potential true label The value of is the confidence of the event "category j", and its expression is as follows:

[0081] (3)

[0082] In the formula is the confidence level, t j is the category decision threshold of category j, Indicates that all the images marked as The sum of pixels of The value of represents the category of the labeled pixel x. Indicates category is the confidence level in all categories The largest class, and the confidence is greater than the category decision threshold of the corresponding category c . Represents a set of m class labels.

[0083] Compared with the simple confusion matrix, the confidence joint matrix is ​​more robust to class imbalance and some predicted over-confident pixels. To perform a calibration:

[0084] C ̃ y l , y l * [ i ][ j ] = C y l , y l * [ i ][ j ] ∑ j ∈ C m C y l , y l * [ i ][ j ] ⋅ | X y l = i | (4)

[0085] In the formula Indicates that all categories in the image are The pixel value of C ̃ y l , y l * [ i ][ j ] Represents the elements in the calibrated confidence joint matrix.

[0086] The calibrated confidence joint matrix is ​​further normalized ,get :

[0087] Q ̃ y l , y l * [ i ][ j ] = C ̃ y l , y l * [ i ][ j ] ∑ i ∈ C m , j ∈ C m C ̃ y l , y l * [ i ][ j ] (5)

[0088] In the formula, Q ̃ y l , y l * [ i ][ j ] represents the elements in the normalized calibrated confidence joint matrix, Represents a set of m class labels.

[0089] Subsequently, this application adopts the category pruning strategy (“Confident learning: Estimatinguncertainty in dataset labels”, Journal of Artificial Intelligence Research, 2021, 70: 1373-1411.) to identify the noise labels. , this strategy selects the lowest confidence of T ⋅ ∑ j ∈ C m : j ≠ i ( Q ̃ y l , y l * [ i ][ j ]) pixels as mislabeled pixels (T represents the number of pixels in the image), so we can get the error estimation map If the pixel value in the error estimation map is "1", it means that the category of the pixel is incorrectly predicted by the model. The confidence module is highly efficient and model-independent (it only processes prediction probabilities). No additional training parameters are introduced. The pixel-level error estimation map can guide the subsequent label refinement process.

[0090] Furthermore, the present invention designs a label smoothing module based on the obtained error estimation map. If the prediction provided by the Teacher model is more certain, then the momentum smoothing factor can be roughly set to 1. If the Teacher model is uncertain about the prediction, the smoothing factor is reduced. Therefore, the present application introduces the confidence uncertainty of the Teacher model. First, the present application performs a series of random forward propagations in the Teacher model, adds noise interference that conforms to the standard normal distribution to the input image, and thus obtains a set of probability vectors of predicted categories. . Subsequently, the prediction entropy is used to estimate the uncertainty of each pixel, which is defined as follows:

[0091] (6)

[0092] in represents the probability that the t-th pixel is predicted to be the c-th class, T represents the total number of pixels in the image, represents the uncertainty of each pixel, Represents the uncertainty of the entire image. Since the predicted entropy has a fixed range, the above formula can be further normalized by dividing the uncertainty of the pixel by the maximum value log(2) to obtain the normalized uncertainty Uncertain pixels should be assigned a small weight, so the uncertainty-aware dynamic smooth label refinement module can be expressed as follows:

[0093] (7)

[0094] In the formula represents the error estimation graph, is a noisy label, is the smoothed label map (denoising operation is performed).

[0095] With the help of this label refinement strategy, the influence of label noise on the negative optimization of the model can be eliminated to a certain extent, so that the model can obtain more guidance from the noisy label data. Loss supervision: The objects supervised by this loss function are the smoothed labels and the labels predicted by the Student model for noisy data.

[0096] The final loss function is composed of the supervised loss (acts on high-quality data), perturbation-based consistency loss (acting on noisy labeled data) and calibrated supervision loss The three are weighted together (for noisy label data) as shown below:

[0097] (8)

[0098] Among them, L represents the loss function of the Mean-Teacher model, is the mean square error at the pixel level, and the weight Set to 5, is a gradually increasing weight, which is a time-independent Gaussian process Sure, is a constant parameter representing the maximum value that can be achieved in weight scheduling, set to 0.1. is the maximum number of training times, and k is the number of training the Mean-Teacher model. This strategy can prevent the network from being overly misled at the beginning of training.

[0099] S4. Use the trained Mean-Teacher model to predict the pixel-level labels of the intestinal ultrasound image, and then obtain the intestinal ultrasound image segmentation results.

[0100] Embodiment 2

[0101] The present invention proposes a semi-supervised segmentation system for intestinal ultrasound images combined with medical large model knowledge corresponding to the method of embodiment 1, comprising:

[0102] A two-stage training module is used to perform two-stage training on the medical large model using unlabeled intestinal ultrasound images and true labeled intestinal ultrasound images to obtain pseudo labels;

[0103] The noise adding module is used to add noise disturbance to the unlabeled intestinal ultrasound image to form noisy labeled data with pseudo labels;

[0104] Noisy label learning module, used to train the Mean-Teacher model using noisy label data and true label intestinal ultrasound images;

[0105] The image segmentation module is used to predict the pixel-level labels of intestinal ultrasound images using the trained Mean-Teacher model, and then obtain the intestinal ultrasound image segmentation results.

[0106] The implementation methods of each module and the functions of the modules in the system are completely consistent with the steps of the method in Example 1, so they will not be repeated here.

[0107] Embodiment 3

[0108] The present invention proposes an electronic device, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, a semi-supervised segmentation method of intestinal ultrasound images combined with medical large model knowledge as described in Example 1 is implemented.

[0109] Embodiment 4

[0110] The present invention proposes a computer-readable storage medium storing a computer program, wherein the computer program enables a computer to execute the semi-supervised segmentation method of intestinal ultrasound images combined with medical large model knowledge as described in the first embodiment.

[0111] The results of intestinal ultrasound image segmentation performed by the present invention are compared with those of manual annotation. Figure 1 shown. Figure 1 (a) is a comparison chart of intestinal wall thickness indicators in intestinal images. The left side shows the calculation of intestinal wall thickness by manually selecting points, and the right side shows the automatic detection of intestinal wall thickness through image segmentation. Figure 1 Middle (b) is a comparison of intestinal fat indicators in intestinal images. The left side is a subjective judgment of whether intestinal fat exists, and the right side is automatic detection of intestinal fat through image segmentation. Figure 1 Middle (c) is a comparison chart of intestinal stratification indicators in intestinal images. The left side shows the manually observed intestinal wall stratification status, and the right side shows the intestinal wall stratification automatically obtained through image segmentation.

[0112] In the embodiments disclosed in the present application, the computer storage medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or apparatus. The computer storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or equipment, or any suitable combination of the foregoing. More specific examples of computer storage media may include electrical connections based on one or more lines, portable computer disks, hard disks, random access memories (RAM), read-only memories (ROM), erasable programmable read-only memories (EPROM or flash memory), optical fibers, portable compact disk read-only memories (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0113] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed in this application can be implemented in electronic hardware or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0114] The above are only preferred embodiments of the present invention, and the protection scope of the present invention is not limited to the above embodiments. All technical solutions under the concept of the present invention belong to the protection scope of the present invention. It should be pointed out that for ordinary technicians in this technical field, some improvements and modifications without departing from the principle of the present invention should be regarded as the protection scope of the present invention.

Claims

1. A semi-supervised segmentation method for intestinal ultrasound images. It is characterized in that The following steps are involved: S1. Performing two-stage training on a large medical model using unlabeled intestinal ultrasound images and true labeled intestinal ultrasound images to obtain pseudo labels; the large medical model includes Totalsegmentator and SAM-Med3D; the two-stage training process is specifically as follows: The unlabeled intestinal ultrasound images and the true labeled intestinal ultrasound images are sent to Totalsegmentator for training, wherein the true labeled intestinal ultrasound images are used as domain knowledge to guide the fine-tuning of Totalsegmentator. After the training is completed, the unlabeled intestinal ultrasound images are inferred to generate rough segmentation results, and the rough segmentation results are encoded. The unlabeled intestinal ultrasound images, the true labeled intestinal ultrasound images and the encoded rough segmentation results are input into SAM-Med3D for training. The rough segmentation results are used as prompts to guide the SAM-Med3D model training. After the SAM-Med3D training is completed, the inference results are obtained, and the inference results of SAM-Med3D are used as pseudo labels. S2, adding noise disturbance to the unlabeled intestinal ultrasound image to form noisy labeled data with pseudo labels; S3. Train a Mean-Teacher model using noisy labeled data and true labeled intestinal ultrasound images; the Mean-Teacher model includes a Teacher model and a Student model, and the process of training the Mean-Teacher model is as follows: In each training round, the following operations are performed until the maximum number of iterations is reached: The Teacher model is trained using noisy labeled data, and the Student model is trained using noisy labeled data and true labeled intestinal ultrasound images; the Teacher model outputs predicted labels for annotating pixel categories; Calculate the confidence of the predicted label output by the Teacher model, take the pixel with the lowest confidence as the mislabeled pixel, and obtain the error estimation map; Calculate the uncertainty of the image labeled by the predicted label; The noisy label map output by the Teacher model is denoised using the error estimation map and uncertainty to obtain a smoothed label map. Calculate the value of the loss function of the Mean-Teacher model to update the Mean-Teacher model; The Student model updates the parameters of the Student model through back propagation, and the Teacher model is updated based on the parameters of the Student model using the exponential sliding average strategy; S4. Use the trained Mean-Teacher model to predict the pixel-level labels of the intestinal ultrasound image, and then obtain the intestinal ultrasound image segmentation results.

2. The semi-supervised segmentation method of intestinal ultrasound images according to claim 1, It is characterized in that The specific process of calculating the confidence of the predicted label output by the Teacher model and taking the pixel with the lowest confidence as the mislabeled pixel to obtain the error estimation map is as follows: Output the predicted label from the Teacher model and convert the pixel The actual category is equal to the predicted label The probability of the category is used as the confidence of the predicted label, expressed as , i is the predicted label The value of represents the category of the labeled pixel x; The confidence that pixel x belongs to class j , then the potential true label of pixel x is considered to be category j instead of i, t j is the category decision threshold of category j, by marking each pixel as Confidence The expected value is expressed as , where Indicates that all the images marked as The total number of pixels; According to this threshold strategy, the confidence matrix of each image X is obtained : ; In the formula, is an element in the confidence joint matrix, The meaning is the category confidence value, which means "the predicted label The value is category i, but the potential true label The value of is the confidence of the event "category j", and its expression is as follows: ; In the formula, is the confidence that pixel x belongs to category j, t j is the category decision threshold of category j, Indicates that all the images marked as A collection of pixels, i To predict the label The value of represents the category of the labeled pixel x. Indicates category is the confidence level in all categories The largest class, and the confidence is greater than the category decision threshold of the corresponding category c ; Represents a set of m class labels; Calibrate the confidence joint matrix: ; In the formula, represents the elements in the calibrated confidence joint matrix, Indicates that all the images marked as The total number of pixels; Normalized calibrated confidence joint matrix: ; In the formula, represents the elements in the normalized calibrated confidence joint matrix, Represents a set of m class labels; The category pruning strategy is used to identify the noise labels, specifically: For each class , choose the lowest confidence of pixels as mislabeled pixels, T represents the total number of pixels in the image, and the error estimation map is obtained. .

3. The semi-supervised segmentation method for intestinal ultrasound images according to claim 1, It is characterized in that The specific process of calculating the uncertainty of the image marked by the predicted label is: The prediction entropy is used to estimate the uncertainty of each pixel: ; In the formula, represents the probability that the t-th pixel is predicted to be the c-th class, T represents the total number of pixels in the image, Represents the uncertainty of each pixel; The uncertainty of the overall image is expressed as U; the normalized overall image uncertainty is expressed as ; The method of denoising the noisy label graph output by the Teacher model using the error estimation graph and uncertainty is as follows: ; In the formula, represents the error estimation graph, is a noisy label image, is the smoothed label map.

4. The semi-supervised segmentation method for intestinal ultrasound images according to claim 1, It is characterized in that The loss function of the Mean-Teacher model is as follows: ; Where L represents the loss function of the Mean-Teacher model, is the supervision loss acting on the true label, is a perturbation-based consistency loss acting on noisy labeled data, is the supervised loss for calibration of noisy label data; λ and β are weights respectively, and λ is a Gaussian process independent of time. Sure, is a constant parameter, representing the maximum value that can be achieved in weight scheduling. is the maximum number of training times, and k is the number of training the Mean-Teacher model.

5. The semi-supervised segmentation method for intestinal ultrasound images according to claim 1, It is characterized in that The Teacher model is updated using the exponential moving average strategy according to the parameters of the Student model as follows: ; In the formula, is the sliding average decay rate, is the Student model weight when the number of training times is k, is the weight of the updated Teacher model, is the weight of the Teacher model before updating.

6. A semi-supervised segmentation system for intestinal ultrasound images, It is characterized in that include: The two-stage training module is used to perform two-stage training on the medical large model using unlabeled intestinal ultrasound images and true labeled intestinal ultrasound images to obtain pseudo labels; the medical large model includes Totalsegmentator and SAM-Med3D; the two-stage training process is specifically as follows: The unlabeled intestinal ultrasound images and the true labeled intestinal ultrasound images are sent to Totalsegmentator for training, wherein the true labeled intestinal ultrasound images are used as domain knowledge to guide the fine-tuning of Totalsegmentator. After the training is completed, the unlabeled intestinal ultrasound images are inferred to generate rough segmentation results, and the rough segmentation results are encoded. The unlabeled intestinal ultrasound images, the true labeled intestinal ultrasound images and the encoded rough segmentation results are input into SAM-Med3D for training. The rough segmentation results are used as prompts to guide the SAM-Med3D model training. After the SAM-Med3D training is completed, the inference results are obtained, and the inference results of SAM-Med3D are used as pseudo labels. The noise adding module is used to add noise disturbance to the unlabeled intestinal ultrasound image to form noisy labeled data with pseudo labels; The noisy label learning module is used to train the Mean-Teacher model using the noisy label data and the true label intestinal ultrasound image; the Mean-Teacher model includes a Teacher model and a Student model, and the process of training the Mean-Teacher model is: In each training round, the following operations are performed until the maximum number of iterations is reached: The Teacher model is trained using noisy labeled data, and the Student model is trained using noisy labeled data and true labeled intestinal ultrasound images; the Teacher model outputs predicted labels for annotating pixel categories; Calculate the confidence of the predicted label output by the Teacher model, take the pixel with the lowest confidence as the mislabeled pixel, and obtain the error estimation map; Calculate the uncertainty of the image labeled by the predicted label; The noisy label map output by the Teacher model is denoised using the error estimation map and uncertainty to obtain a smoothed label map. Calculate the value of the loss function of the Mean-Teacher model to update the Mean-Teacher model; The Student model updates the parameters of the Student model through back propagation, and the Teacher model is updated based on the parameters of the Student model using the exponential sliding average strategy; The image segmentation module is used to predict the pixel-level labels of intestinal ultrasound images using the trained Mean-Teacher model, and then obtain the intestinal ultrasound image segmentation results.

7. An electronic device, It is characterized in that include: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the semi-supervised segmentation method for intestinal ultrasound images as described in any one of claims 1 to 5 is implemented.

8. A computer-readable storage medium storing a computer program, It is characterized in that The computer program enables the computer to execute the semi-supervised segmentation method for intestinal ultrasound images as described in any one of claims 1-5.