Thyroid nodule segmentation method based on large model and collaborative consensus training

Abstract

The present invention discloses a thyroid nodule segmentation method based on a large model and collaborative consistency training, which relates to the technical field of ultrasound image analysis. The general reasoning ability of the large model is organically embedded in a semi-supervised model, and the advantages of the semi-supervised model and the large model are complemented. The general prediction ability of the large model is utilized to alleviate the insufficient knowledge transfer ability of the semi-supervised model. At the same time, the feature collaborative consistency of the semi-supervised model is utilized to alleviate the high-confidence erroneous prediction of the large model, thereby improving the actual prediction performance of the model. Through the large model fine-tuning operation under the semi-supervised framework, the large model can be fine-tuned synchronously based on unsupervised images and supervised images. At the same time, based on collaborative loss and high-quality pseudo-labels, the fine-tuning convergence speed and accuracy of the model are accelerated. Through the collaborative consistency training operation, the advantages of the semi-supervised model and the SAM model are organically complemented, and a high-quality pseudo-label generation scheme and collaborative loss based on self-reliability and collaborative reliability are proposed.

Description

Technical Field

[0001] The present invention relates to the technical field of ultrasonic image analysis, and in particular to a thyroid nodule segmentation method based on a large model and collaborative consistency training. Background Art

[0002] With changes in modern lifestyles, the psychological and physical stress of young people continues to rise, and the clinical detection rate of thyroid-related diseases has also increased. Among them, the incidence of thyroid nodules remains high, and many nodules have an increased risk of malignancy. In clinical practice, the spatial localization of nodules is not only a key step in basic diagnosis but also directly related to subsequent risk assessment and the formulation of treatment strategies.

[0003] Traditional imaging methods typically involve delineating the nodule area on ultrasound images based on their clinical experience and the patient's medical history. This process relies heavily on manual judgment and is highly subjective. In recent years, with the widespread application of computer vision and deep learning technologies, research using AI models to assist in the automatic localization of thyroid nodules has achieved remarkable results. These methods are expected to improve localization efficiency, reduce human error, and promote the standardization of ultrasound-assisted diagnosis.

[0004] However, building high-performance models relies on large amounts of accurately labeled data. In reality, the proportion of images that meet the gold standard for pathology is extremely low, resulting in a data bottleneck for model training. Furthermore, fully labeling all images is a massive undertaking, costly, and difficult to scale.

[0005] Utilizing large amounts of unlabeled data is key to improving the performance of thyroid ultrasound localization models. Existing mainstream research methods can be divided into two categories: 1. Leveraging semi-supervised learning to mine the intrinsic features of unlabeled thyroid images, such as MixMatch and FixMatch; 2. Leveraging the general learning capabilities of basic visual large-scale models, such as SAM and MedSAM, to fine-tune thyroid ultrasound images. However, large amounts of unlabeled data come from complex sources, often collected from different hospitals, ultrasound equipment, and probes. This complexity makes semi-supervised learning inefficient in knowledge transfer. Furthermore, due to the lack of feature information in labeled data, semi-supervised learning is prone to overfitting on labeled data, affecting its true generalization. Furthermore, because general-purpose large-scale models are trained with reference to a large number of images from other domains and inherit extensive prior knowledge, they tend to make high-confidence erroneous predictions in the early stages of fine-tuning. These predictions are difficult to correct, hindering model performance improvement.

[0006] Based on the above analysis, it can be found that although both semi-supervised learning and general large models have the ability to utilize unlabeled data, due to factors such as the complexity of the field and the interference of prior knowledge, they both have their own disadvantages and are unable to utilize the potential of unlabeled data to a greater extent. Summary of the Invention

[0007] In order to solve the above technical problems, the present invention provides a thyroid nodule segmentation method based on a large model and coordinated consistent training, comprising the following steps:

[0008] S1. Collect thyroid ultrasound images and divide them into annotated and unannotated images based on the presence or absence of pathology reports and annotation information, forming an annotated dataset and an unannotated dataset. The annotated dataset is then divided into a training set, a validation set, and a test set in proportion.

[0009] S2: Build a semi-supervised model based on the Mean-Teacher framework and FixMatch technology, and use U-Net as the segmentation network. The U-Net network uses randomly initialized weights. Build a SAM model based on the Mean-Teacher framework, and load the official pre-trained weights into the SAM model.

[0010] S3. In the semi-supervised model, the unlabeled image is transformed into a strongly perturbed image and a weakly perturbed image through a strong transformation and a weak transformation, respectively. The weakly perturbed image is processed by the teacher network to obtain a pseudo label, and the strongly perturbed image is processed by the student network to obtain the model's prediction mask. At the same time, the student network predicts the labeled image to obtain the mask of the labeled image, and calculates the supervision loss with the labeled information.

[0011] S4. In the SAM model, unlabeled images are transformed into strongly perturbed images and weakly perturbed images through strong and weak transformations respectively. The weakly perturbed images are processed by the teacher network to obtain pseudo labels, and the strongly perturbed images are processed by the student network to obtain the model's prediction mask. At the same time, the student network predicts the labeled images to obtain the masks of the labeled images, and calculates the supervision loss with the labeled information.

[0012] S5, merge the pseudo labels generated by the semi-supervised model and the SAM model, and impose constraints on the model predictions through the consistency loss function;

[0013] S6. Calculate the loss function for the student network in the semi-supervised model and the SAM model respectively;

[0014] S7. Synchronously train the semi-supervised model and the SAM model, perform gradient backpropagation on the semi-supervised model and the SAM model to update the parameters, and the teacher network in the semi-supervised model and the SAM model inherits the weight of the student network through EMA.

[0015] S8. After each iteration, the student networks in the semi-supervised model and the SAM model are evaluated using the validation set, and the model with the highest DICE coefficient in the historical results is saved.

[0016] S9. Load the models with the highest DICE coefficient in the historical training process respectively, perform inference verification on the test set, and use the DICE coefficient of the test set to reflect the actual inference performance of the model.

[0017] The technical solution further defined in the present invention is:

[0018] Furthermore, in step S1, the labeled dataset is divided into a training set, a validation set, and a test set in a ratio of 7:1:2.

[0019] As described above, in the thyroid nodule segmentation method based on a large model and collaborative consistent training, in steps S3 and S4, the strong transformation includes hue adjustment, style adjustment, and arbitrary masking; the weak transformation includes a series of affine transformations.

[0020] As described above, in the thyroid nodule segmentation method based on a large model and coordinated consistent training, in step S3, the pseudo-labels of the unlabeled images are not directly used as supervisory information for the unlabeled images, but are subsequently processed in step S5. The specific process of step S3 is shown in the following formula:

[0021]

[0022] in, represents the pseudo label of the semi-supervised model, Represents the prediction mask of the semi-supervised model, the superscript U indicates that the network is U-Net, and the subscripts w and s represent weak transformation and strong transformation respectively; P U Represents the prediction results of the labeled image; represents the supervised loss function of the semi-supervised model; and Respectively represent the teacher network and student network in the semi-supervised model; X, Y, I s , I w They represent the labeled image, the mask of the labeled image, the strongly perturbed image, and the weakly perturbed image respectively; L DICE and L CE Represent the Dice loss function and the cross entropy loss function respectively.

[0023] As described above, in the thyroid nodule segmentation method based on a large model and collaborative consistent training, in step S4, the SAM model is adjusted synchronously using both unlabeled data and labeled data based on the Mean-Teacher framework. During the adjustment, the prompt module and decoder module in the SAM model are frozen, and the encoder module in the SAM model is adjusted using the LoRA technology.

[0024] As described above, the thyroid nodule segmentation method based on a large model and collaborative consistent training, the specific process of step S4 is as follows:

[0025]

[0026] in, represents the pseudo label of the SAM model, Represents the prediction mask of the SAM model, the superscript SAM indicates that the network is a SAM model, and the subscripts w and s represent weak transformation and strong transformation respectively; P SAM Represents the prediction results of the labeled image; Represents the supervised loss function of the SAM model; and Respectively represent the teacher network and student network in the SAM model; X, Y, I s , I w They represent the labeled image, the mask of the labeled image, the strongly perturbed image, and the weakly perturbed image respectively; L DICE and L CE Represent the Dice loss function and the cross entropy loss function respectively.

[0027] As described above, the thyroid nodule segmentation method based on a large model and coordinated consistent training, the specific process of step S5 is shown in the following formula:

[0028]

[0029] in, and They represent the pseudo labels of the semi-supervised model and the SAM model respectively, and λ represents the hyperparameter used in the combination, and the setting of this hyperparameter depends on the self-reliability and collaborative reliability;

[0030] Then, the prediction masks are compared and the prediction results are divided into two parts: consistent predictions and divergent predictions. The consistent predictions indicate that the semi-supervised model and the SAM model have common cognition, while the divergent predictions indicate that the semi-supervised model and the SAM model have different perspectives and expression capabilities. The consistency loss is introduced and is defined as follows:

[0031]

[0032] Among them, L d represents the consistency loss calculated based on information entropy, L c represents the consistency loss calculated based on MSE, represents the predicted mask obtained by the teacher network in the semi-supervised model to reason about the strongly perturbed image, It represents the predicted mask obtained by the teacher network in the SAM model to infer the strongly disturbed image, -H() represents the information entropy, M d Indicates the Boolean mask where the semi-supervised model and the SAM model predict the same region, L MSE represents the mean squared error loss.

[0033] As mentioned above, in the thyroid nodule segmentation method based on large models and collaborative training, the setting of the hyperparameter λ depends on self-reliability and collaborative reliability. Self-reliability is defined as the consistency of the input response of the student network and the teacher network in the semi-supervised model. Its formula is defined as follows:

[0034]

[0035] Among them, C represents the total number of categories; represents the mask region predicted by the teacher network mask as the i-th category, represents the mask region predicted by the student network mask as the i-th category.

[0036] As mentioned above, the thyroid nodule segmentation method based on large models and collaborative consensus training, collaborative reliability is defined as the similarity of the prediction results of the semi-supervised model and the SAM model, and its formula is defined as follows:

[0037] .

[0038] As described above, in the thyroid nodule segmentation method based on a large model and coordinated training, in step S6, the loss function of the student network in the semi-supervised model is as follows:

[0039]

[0040] in, represents the unsupervised loss of the semi-supervised model, L DICE and L CE Represents the Dice loss function and the cross entropy loss function respectively, T w represents a pseudo label, Represents the predicted mask obtained by the teacher network in the semi-supervised model to reason about the strongly perturbed image; represents the loss function of the student network in the semi-supervised model, represents the supervised loss of the semi-supervised model, L d and L c Represent the consistency loss calculated based on information entropy and MSE respectively;

[0041] The loss function of the student network in the SAM model is as follows:

[0042]

[0043] in, represents the unsupervised loss of the SAM model, Represents the predicted mask obtained by the teacher network in the SAM model to reason about the strongly perturbed image; represents the loss function of the student network in the SAM model, represents the supervised loss of the SAM model.

[0044] The beneficial effects of the present invention are:

[0045] (1) This paper proposes a semi-supervised segmentation method for thyroid nodules based on a large model and a collaborative consistency training mechanism. This method effectively solves the problems of high labeling cost and data scarcity in the current medical ultrasound image segmentation task. It innovatively and fully integrates the general reasoning ability of the large model with the semi-supervised learning framework. Through an organic modular design, it realizes the collaborative training of unlabeled data and labeled data. Compared with traditional segmentation methods, this paper not only improves the accuracy of prediction, but also significantly enhances the generalization ability and robustness of the model in real scenarios.

[0046] (2) In this invention, a large model fine-tuning strategy under a semi-supervised framework is adopted to enable large models (such as SAM, MAE, etc.) to be efficiently trained under the mixed conditions of limited labeled data and a large amount of unlabeled data; by introducing a collaborative consistency training mechanism, multiple model branches (such as teacher model and student model, large model and lightweight model) maintain output consistency during the training process, effectively alleviating the interference of pseudo-label noise on model learning; in addition, the introduction of collaborative loss promotes information complementarity and constraint between different models, thereby improving the stability of the overall segmentation results;

[0047] (3) In this invention, a high-quality pseudo-label generation mechanism combining self-reliability and collaborative reliability is designed. This mechanism measures the consistency of the output of a single model and the similarity between multiple models to select more reliable pseudo-labels for training, thereby improving the model's learning efficiency for unsupervised samples, promoting faster model convergence, and achieving performance comparable to or even better than full supervision under weak supervision conditions;

[0048] (4) The present invention has good versatility and scalability. Although the method was initially applied to ultrasound image segmentation of thyroid nodules, its training framework and collaborative mechanism are also applicable to ultrasound images of other organs, such as lymph nodes, breast, gynecology, heart, and other scenarios. Users only need to fine-tune model parameters or replace a small number of training samples to quickly migrate the technology to other tasks, significantly reducing model deployment and development costs. In addition, since the present invention is compatible with large models and lightweight model structures, it is suitable for a variety of resource environments, including clinical edge device deployment or model training on central servers, so it also has high practical value and industrialization potential. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] Figure 1 Schematic diagram of the process of the semi-supervised model in an embodiment of the present invention;

[0050] Figure 2 Schematic diagram of the process of the SAM model in an embodiment of the present invention. DETAILED DESCRIPTION

[0051] This embodiment provides a thyroid nodule segmentation method based on a large model and collaborative consistent training, comprising the following steps:

[0052] S1. Collect thyroid ultrasound images from multiple centers and multiple devices from different sources. The ultrasound images are required to have a sufficiently rich grayscale range, that is, the image grayscale value distribution is wide and the contrast is obvious, so that different tissue areas have obvious grayscale differences in the image. Generally, the image grayscale coverage range is required to exceed 50% of the total interval (grayscale coverage pixel value interval 70-200) or the grayscale standard deviation is greater than 25; based on the presence or absence of clear pathology reports and annotation information, thyroid ultrasound images are divided into annotated images and unannotated images to form annotated and unannotated datasets, and the annotated dataset is divided into training set, validation set, and test set in a ratio of 7:1:2. The training set and validation set are used to select the model, and the test set is used to evaluate the model performance.

[0053] S2. Build a semi-supervised model based on the Mean-Teacher framework and FixMatch technology, and use U-Net as the segmentation network. The U-Net network uses randomly initialized weights. Build a SAM model based on the Mean-Teacher framework, and load the officially provided pre-trained weights into the SAM model. Initialize the optimizer, scheduler, and other necessary components.

[0054] S3, such as Figure 1 As shown in the semi-supervised model, unlabeled images are transformed into strongly perturbed images and weakly perturbed images through strong transformations (tone adjustment, style adjustment, random masking, etc.) and weak transformations (a series of affine transformations, etc.); the weakly perturbed images are processed by the teacher network to obtain pseudo labels. , process the strongly disturbed image through the student network to obtain the model's prediction mask , where the superscript U indicates that the network is U-Net, and the subscripts w and s represent weak transformation and strong transformation respectively.

[0055] In addition, the student network predicts the labeled image, obtains the mask of the labeled image, and calculates the supervision loss with the labeled information (i.e., the true label); unlike traditional semi-supervised technology, the pseudo-label of the unlabeled image is not directly used as the supervision information of the unlabeled image, but will be processed in the subsequent collaborative consistency training step.

[0056] The specific process of the semi-supervised model can be described as:

[0057]

[0058] Among them, P U Represents the prediction results of the labeled image; represents the supervised loss function of the semi-supervised model; and Respectively represent the teacher network and student network in the semi-supervised model; X, Y, I s , I w They represent the labeled image, the mask of the labeled image, the strongly perturbed image, and the weakly perturbed image respectively; L DICE and L CE Represent the Dice loss function and the cross entropy loss function respectively.

[0059] S4. The general large model used in this embodiment is fine-tuned based on SAM. Unlike the conventional fine-tuning using labeled data, this embodiment draws on the semi-supervised model and uses both unlabeled data and labeled data for synchronous fine-tuning based on the Mean-Teacher framework. At the same time, in order to reduce the consumption of computing resources, the Prompt module and Decoder module of SAM are frozen during fine-tuning, and the Low Rank Addition (LoRA) technology is used to fine-tune the Encoder module of SAM.

[0060] SAM model Figure 2 As shown, its forward reasoning process is similar to that of the semi-supervised model:

[0061] Unlabeled images are transformed into strongly perturbed images and weakly perturbed images through strong transformations (tone adjustment, style adjustment, random masking, etc.) and weak transformations (a series of affine transformations, etc.); weakly perturbed images are processed through the teacher network to obtain pseudo labels. , process the strongly disturbed image through the student network to obtain the model's prediction mask , where the superscript SAM indicates that the network is SAM, and the subscripts w and s represent weak transformation and strong transformation respectively.

[0062] In addition, the student network predicts the labeled image, obtains the mask of the labeled image, and calculates the supervision loss with the labeled information; similarly, the pseudo-label information will be processed in the subsequent collaborative consistency training step.

[0063] The specific process of the SAM model can be described as:

[0064]

[0065] Among them, P SAMRepresents the prediction results of the labeled image; Represents the supervised loss function of the SAM model; and Respectively represent the teacher network and student network in the SAM model; X, Y, I s , I w They represent the labeled image, the mask of the labeled image, the strongly perturbed image, and the weakly perturbed image respectively; L DICE and L CE Represent the Dice loss function and the cross entropy loss function respectively.

[0066] S5. The pseudo labels generated by the semi-supervised model and the SAM model are merged, and the model predictions are constrained by the consistency loss function to compensate for the insufficient knowledge transfer ability of the semi-supervised model and the possible high-confidence erroneous predictions of the SAM model.

[0067] The specific process of this step is shown as follows:

[0068]

[0069] in, and They represent the pseudo labels of the U-Net model (semi-supervised model) and the SAM model respectively, and λ represents the combined hyperparameter, which is manually defined according to the task. In this embodiment, the setting of the hyperparameter is attributed to self-reliability and collaborative reliability.

[0070] Self-reliability is defined as the consistency of the input response between the student network and the teacher network in a semi-supervised model. It is difficult for the teacher and student models to reach a consensus when processing unlabeled data. Good pseudo-labels should have similar information expression. The formula is defined as follows:

[0071]

[0072] Where C represents the total number of categories. For the thyroid nodule segmentation task, C is set to 1 by default; represents the mask region predicted by the teacher network mask as the i-th category, represents the mask region predicted by the student network mask as the i-th category.

[0073] Collaborative reliability is defined as the similarity between the prediction results of the semi-supervised model and the SAM model, and its formula is defined as follows:

[0074]

[0075] The definitions of the relevant parameters in the above formula are consistent with self-reliability.

[0076] Finally, by comparing the prediction masks, the prediction results can be divided into two parts: consistent predictions and divergent predictions. The consistent predictions represent that the semi-supervised model and the SAM model have common cognition, while the divergent predictions reflect the differences in the focus perspective and expression ability between the semi-supervised model and the SAM model. To ensure the correct focus perspective of the model and improve the model's expression ability, this embodiment designs a consistency loss, which is defined as follows:

[0077]

[0078] Among them, L d represents the consistency loss calculated based on information entropy, L c represents the consistency loss calculated based on MSE, represents the predicted mask obtained by the teacher network in the semi-supervised model to reason about the strongly perturbed image, It represents the predicted mask obtained by the teacher network in the SAM model to infer the strongly disturbed image, -H() represents the information entropy, M d Indicates the Boolean mask where the semi-supervised model and the SAM model predict the same region, L MSE represents the mean squared error loss.

[0079] S6. Calculate the loss function for the student network in the semi-supervised model and the SAM model respectively.

[0080] The loss function of the student network in the semi-supervised model is as follows:

[0081]

[0082] in, represents the unsupervised loss of the semi-supervised model, L DICE and L CE Represents the Dice loss function and the cross entropy loss function respectively, T w represents a pseudo label, Represents the predicted mask obtained by the teacher network in the semi-supervised model to reason about the strongly perturbed image; represents the loss function of the student network in the semi-supervised model, represents the supervised loss of the semi-supervised model, L d and L c represent the consistency loss calculated based on information entropy and MSE respectively.

[0083] The loss function of the student network in the SAM model is as follows:

[0084]

[0085] in, represents the unsupervised loss of the SAM model, Represents the predicted mask obtained by the teacher network in the SAM model to reason about the strongly perturbed image; represents the loss function of the student network in the SAM model, represents the supervised loss of the SAM model.

[0086] S7. Synchronously train the semi-supervised model and the SAM model, and perform gradient backpropagation on the semi-supervised model and the SAM model to update the parameters; the teacher network in the semi-supervised model and the SAM model does not perform gradient backpropagation through the loss function, but inherits the weights of the student network through EMA.

[0087] S8. After each iteration, the validation set is used to evaluate the student networks in the semi-supervised model and the SAM model, and the model with the highest DICE coefficient in the historical results is saved.

[0088] S9. Load the model with the highest DICE coefficient in the historical training process respectively, and perform inference verification on the test set. During verification, the prediction masks of the semi-supervised model and the SAM model are combined by weighted average, and the DICE coefficient of the test set is used to reflect the actual inference performance of the model.

[0089] This embodiment proposes a thyroid nodule segmentation method based on a large model and collaborative consistency training, which organically embeds the general reasoning ability of the large model into the semi-supervised model, complements the advantages of the semi-supervised model and the large model, and uses the general prediction ability of the large model to alleviate the insufficient knowledge transfer ability of the semi-supervised model. At the same time, the feature collaborative consistency of the semi-supervised model is used to alleviate the high-confidence erroneous prediction of the large model, thereby improving the actual prediction performance of the model.

[0090] Through the large model fine-tuning operation under the semi-supervised framework, the large model can be fine-tuned based on unsupervised images and supervised images simultaneously. At the same time, based on collaborative loss and high-quality pseudo-labels, the fine-tuning convergence speed and accuracy of the model are accelerated; and through the collaborative consistency training operation, the advantages of the semi-supervised model and the SAM model are organically complemented, and a high-quality pseudo-label generation scheme and collaborative loss based on self-reliability and collaborative reliability are proposed; the method of this embodiment can be used for medical ultrasound images of all organs, not just the thyroid gland. It can be reused for various medical ultrasound images such as lymph nodes, breast, gynecology, and heart according to actual usage, and has high practical value and industrialization potential.

[0091] In addition to the above embodiments, the present invention may also have other implementations. Any technical solution formed by equivalent replacement or equivalent transformation falls within the scope of protection required by the present invention.

Claims

1. A thyroid nodule segmentation method based on a large model and collaborative consistent training, characterized by: The following steps are involved: S1. Collect thyroid ultrasound images and divide them into annotated and unannotated images based on the presence or absence of pathology reports and annotation information, forming an annotated dataset and an unannotated dataset. The annotated dataset is then divided into a training set, a validation set, and a test set in proportion. S2: Build a semi-supervised model based on the Mean-Teacher framework and FixMatch technology, and use U-Net as the segmentation network. The U-Net network uses randomly initialized weights. Build a SAM model based on the Mean-Teacher framework, and load the official pre-trained weights into the SAM model. S3. In the semi-supervised model, the unlabeled image is transformed into a strongly perturbed image and a weakly perturbed image through a strong transformation and a weak transformation, respectively. The weakly perturbed image is processed by the teacher network to obtain a pseudo label, and the strongly perturbed image is processed by the student network to obtain the model's prediction mask. At the same time, the student network predicts the labeled image to obtain the mask of the labeled image, and calculates the supervision loss with the labeled information. S4. In the SAM model, unlabeled images are transformed into strongly perturbed images and weakly perturbed images through strong and weak transformations respectively. The weakly perturbed images are processed by the teacher network to obtain pseudo labels, and the strongly perturbed images are processed by the student network to obtain the model's prediction mask. At the same time, the student network predicts the labeled images to obtain the masks of the labeled images, and calculates the supervision loss with the labeled information. S5, merge the pseudo labels generated by the semi-supervised model and the SAM model, and impose constraints on the model predictions through the consistency loss function; S6. Calculate the loss function for the student network in the semi-supervised model and the SAM model respectively; S7. Synchronously train the semi-supervised model and the SAM model, perform gradient backpropagation on the semi-supervised model and the SAM model to update the parameters, and the teacher network in the semi-supervised model and the SAM model inherits the weight of the student network through EMA. S8. After each iteration, the student networks in the semi-supervised model and the SAM model are evaluated using the validation set, and the model with the highest DICE coefficient in the historical results is saved. S9. Load the models with the highest DICE coefficient in the historical training process respectively, perform inference verification on the test set, and use the DICE coefficient of the test set to reflect the actual inference performance of the model.

2. The thyroid nodule segmentation method based on large model and coordinated consistent training according to claim 1, characterized in that: In step S1, the labeled data set is divided into a training set, a validation set, and a test set in a ratio of 7:1:

2.

3. The thyroid nodule segmentation method based on large model and coordinated training according to claim 1, characterized in that: In step S3 and step S4, the strong transformation includes hue adjustment, style adjustment and random masking; the weak transformation includes a series of affine transformations.

4. The thyroid nodule segmentation method based on large model and coordinated consistent training according to claim 3, characterized in that: In step S3, the pseudo-label of the unlabeled image is not directly used as the supervisory information of the unlabeled image, but is subsequently processed in step S5. The specific process of step S3 is shown in the following formula: in, represents the pseudo label of the semi-supervised model, Represents the prediction mask of the semi-supervised model, the superscript U indicates that the network is U-Net, and the subscripts w and s represent weak transformation and strong transformation respectively; P U Represents the prediction results of the labeled image; represents the supervised loss function of the semi-supervised model; and Respectively represent the teacher network and student network in the semi-supervised model; X, Y, I s , I w They represent the labeled image, the mask of the labeled image, the strongly perturbed image, and the weakly perturbed image respectively; L DICE and L CE Represent the Dice loss function and the cross entropy loss function respectively.

5. The thyroid nodule segmentation method based on large model and coordinated consistent training according to claim 3, characterized in that: In step S4, the SAM model is adjusted synchronously using both unlabeled data and labeled data based on the Mean-Teacher framework. During the adjustment, the prompt module and decoder module in the SAM model are frozen, and the encoder module in the SAM model is adjusted using LoRA technology.

6. The thyroid nodule segmentation method based on large model and coordinated consistent training according to claim 5, characterized in that: The specific process of step S4 is shown in the following formula: in, represents the pseudo label of the SAM model, represents the prediction mask of the SAM model, the superscript SAM indicates that the network is a SAM model, and the subscripts w and s represent weak transformation and strong transformation respectively; P SAM Represents the prediction results of the labeled image; Represents the supervised loss function of the SAM model; and Respectively represent the teacher network and student network in the SAM model; X, Y, I s , I w They represent the labeled image, the mask of the labeled image, the strongly perturbed image, and the weakly perturbed image respectively; L DICE and L CE Represent the Dice loss function and the cross entropy loss function respectively.

7. The thyroid nodule segmentation method based on large model and coordinated consistent training according to claim 1, characterized in that: The specific process of step S5 is shown below: in, and They represent the pseudo labels of the semi-supervised model and the SAM model respectively, and λ represents the hyperparameter used in the combination, and the setting of this hyperparameter depends on the self-reliability and collaborative reliability; Then, the prediction masks are compared and the prediction results are divided into two parts: consistent predictions and divergent predictions. The consistent predictions indicate that the semi-supervised model and the SAM model have common cognition, while the divergent predictions indicate that the semi-supervised model and the SAM model have different perspectives and expression capabilities. The consistency loss is introduced and is defined as follows: Among them, L d represents the consistency loss calculated based on information entropy, L c represents the consistency loss calculated based on MSE, represents the predicted mask obtained by the teacher network in the semi-supervised model to reason about the strongly perturbed image, It represents the predicted mask obtained by the teacher network in the SAM model to infer the strongly disturbed image, -H() represents the information entropy, M d Indicates the Boolean mask where the semi-supervised model and the SAM model predict the same region, L MSE represents the mean squared error loss.

8. The thyroid nodule segmentation method based on large model and coordinated consistent training according to claim 7, characterized in that: The setting of the hyperparameter λ depends on self-reliability and collaborative reliability. Self-reliability is defined as the consistency of the input response of the student network and the teacher network in the semi-supervised model. Its formula is defined as follows: Among them, C represents the total number of categories; represents the mask region predicted by the teacher network mask as the i-th category, represents the mask region predicted by the student network mask as the i-th category.

9. The thyroid nodule segmentation method based on large model and coordinated consistent training according to claim 8, characterized in that: The collaborative reliability is defined as the similarity between the prediction results of the semi-supervised model and the SAM model, and its formula is defined as follows: 。 10. The thyroid nodule segmentation method based on large model and coordinated training according to claim 1, characterized in that: In step S6, the loss function of the student network in the semi-supervised model is as follows: in, represents the unsupervised loss of the semi-supervised model, L DICE and L CE Represents the Dice loss function and the cross entropy loss function respectively, T w represents a pseudo label, Represents the predicted mask obtained by the teacher network in the semi-supervised model to reason about the strongly perturbed image; represents the loss function of the student network in the semi-supervised model, represents the supervised loss of the semi-supervised model, L d and L c Represent the consistency loss calculated based on information entropy and MSE respectively; The loss function of the student network in the SAM model is as follows: in, represents the unsupervised loss of the SAM model, Represents the predicted mask obtained by the teacher network in the SAM model to reason about the strongly perturbed image; represents the loss function of the student network in the SAM model, and represents the supervised loss of the SAM model.

Images