Semi-supervised segmentation method and device for thyroid nodules based on hybrid domain enhancement

Abstract

The application discloses a kind of based on mixed domain enhancement's thyroid nodule semi-supervised segmentation method and device, it is related to image segmentation technical field.The method includes: constructing semi-supervised segmentation model, including unified bidirectional copy and paste enhancement module, cross-teaching model, patch enhancement module based on information entropy and frequency domain enhancement module based on Fourier transform;Using labeled data and unlabeled data to train the model constructed, by unified bidirectional copy and paste, generate intermediate sample between labeled data and unlabeled data in space domain;Using the uncertainty estimation based on information entropy, exchange low-entropy patch in labeled data and high-entropy patch in unlabeled data;The low-frequency component of labeled data and unlabeled data is exchanged by fast Fourier transform, and the low-frequency prediction is supervised in consistency based on the pseudo-label generated based on full frequency;The thyroid nodule ultrasound image to be segmented is input into the trained segmentation model, and the nodule segmentation result is obtained.The present application can improve segmentation accuracy.

Description

Technical Field

[0001] This invention relates to the field of image segmentation technology, and in particular to a semi-supervised segmentation method and apparatus for thyroid nodules based on hybrid domain enhancement. Background Technology

[0002] In the field of medical image analysis, particularly in the segmentation of thyroid nodules using ultrasound images, accurately delineating the nodule contour is crucial for determining malignancy and making treatment decisions. Ultrasound examination is the preferred method due to its non-invasive and real-time advantages; however, inherent image noise, low contrast, and shadow artifacts often lead to blurred nodule boundaries, posing a significant challenge to automatic segmentation. Currently, fully supervised methods based on deep learning have made significant progress in this task, but their performance heavily relies on large-scale pixel-level labeled data. However, medical image annotation requires the participation of experienced physicians, a costly and time-consuming process that often results in a dilemma in clinical practice: a coexistence of limited labeled data and a large amount of unlabeled data. Therefore, semi-supervised learning, by synergistically utilizing limited labeled samples and abundant unlabeled data to alleviate the annotation bottleneck, has become a key solution.

[0003] Existing semi-supervised medical image segmentation methods mainly revolve around consistency regularization and pseudo-label generation, and their performance largely depends on effective data augmentation strategies. Current data augmentation methods can be divided into spatial domain augmentation and frequency domain augmentation, but most methods only enhance a single domain: spatial domain augmentation, while enriching local texture, struggles to capture global structure and is prone to errors in areas with blurred boundaries; frequency domain augmentation, while helpful in modeling global semantics, easily loses local details, leading to structural inconsistencies. Although some studies have attempted to promote interaction between labeled and unlabeled data through strategies such as bidirectional copy-paste, their random selection of embedding directions fails to fully and uniformly utilize both types of data and may introduce confirmation bias, affecting pseudo-label quality. How to synergistically utilize the detailed information of the spatial domain and the global semantics of the frequency domain to form complementary enhancement remains a key problem that existing technologies have not adequately solved. Therefore, there is an urgent need in this field for a semi-supervised segmentation method that can integrate spatial and frequency domain information, reduce confirmation bias, and fully exploit the potential of labeled and unlabeled data to achieve more accurate and robust thyroid nodule ultrasound image segmentation under conditions of scarce labeled data. Summary of the Invention

[0004] To address the problems of low segmentation accuracy and poor robustness in existing technologies under conditions of scarce labeled data, this invention provides a semi-supervised segmentation method and apparatus for thyroid nodules based on hybrid domain enhancement. The technical solution is as follows:

[0005] On the one hand, a semi-supervised segmentation method for thyroid nodules based on hybrid domain enhancement is provided. This method is implemented by a semi-supervised segmentation device for thyroid nodules based on hybrid domain enhancement, and includes:

[0006] S1. Construct a semi-supervised segmentation model that integrates spatial domain and frequency domain enhancements; the model includes: a unified bidirectional copy-paste enhancement module, a cross-teaching model, an information entropy-based patch enhancement module, and a Fourier transform-based frequency domain enhancement module; wherein, the cross-teaching model includes: a teacher model and two student models;

[0007] S2. Input the acquired labeled thyroid nodule ultrasound image data and unlabeled thyroid nodule ultrasound image data into the unified bidirectional copy and paste enhancement module to generate labeled and unlabeled data after weak enhancement and labeled and unlabeled data after strong enhancement. Generate intermediate samples through unified bidirectional copy and paste. Input the intermediate samples into the cross-teaching model and supervise the two student models through the pseudo-label data generated by the teacher model to calculate the overall cross-teaching loss.

[0008] S3. Input the weakly enhanced labeled data and unlabeled data into the information entropy-based patch enhancement module, swap the low-entropy patch in the weakly enhanced labeled data with the high-entropy patch in the weakly enhanced unlabeled data to generate new samples; input the new samples into the student model for supervision, and calculate the entropy patch enhancement loss.

[0009] S4. Input the intermediate samples into the frequency domain enhancement module based on Fourier transform for fast Fourier transform to obtain the frequency domain spectrum; extract the central low-frequency component of the frequency domain spectrum and swap it, then reconstruct it into a spatial domain image through inverse Fourier transform to generate new samples; stitch the new samples together to generate stitched samples; input the stitched samples into the student model, supervised by the pseudo-label data generated by the teacher model, and calculate the Fourier enhancement loss.

[0010] S5. The cross-teaching overall loss, entropy patch enhancement loss and Fourier enhancement loss are weighted and combined to obtain the total loss function of the model. The model is trained based on the total loss function to obtain the trained model.

[0011] S6. The obtained images of the thyroid nodules to be segmented are generated and input into the trained segmentation model to obtain the nodule segmentation results.

[0012] On the other hand, a hybrid domain enhancement-based semi-supervised segmentation device for thyroid nodules is provided. This device is applied to a hybrid domain enhancement-based semi-supervised segmentation method for thyroid nodules. The device includes:

[0013] A building unit is used to construct a semi-supervised segmentation model that integrates spatial domain and frequency domain enhancements; the model includes: a unified bidirectional copy-paste enhancement module, a cross-teaching model, an information entropy-based patch enhancement module, and a Fourier transform-based frequency domain enhancement module; wherein, the cross-teaching model includes: a teacher model and two student models;

[0014] A unified bidirectional copy-paste enhancement unit is used to input the acquired labeled and unlabeled thyroid nodule ultrasound image data into the unified bidirectional copy-paste enhancement module to generate weakly enhanced labeled and unlabeled data, as well as strongly enhanced labeled and unlabeled data. Intermediate samples are generated through unified bidirectional copy-paste. The intermediate samples are then input into the cross-teaching model, and the pseudo-label data generated by the teacher model is used to supervise the two student models to calculate the overall cross-teaching loss.

[0015] The information entropy-based patch enhancement unit is used to input weakly enhanced labeled data and unlabeled data into the information entropy-based patch enhancement module, swap low-entropy patches in the weakly enhanced labeled data with high-entropy patches in the weakly enhanced unlabeled data to generate new samples; the new samples are then input into the student model for supervision, and the entropy patch enhancement loss is calculated.

[0016] The Fourier transform-based frequency domain enhancement unit is used to input intermediate samples into the Fourier transform-based frequency domain enhancement module for fast Fourier transform to obtain the frequency domain spectrum; the central low-frequency component of the frequency domain spectrum is extracted and swapped, and reconstructed into a spatial domain image through inverse Fourier transform to generate new samples; the new samples are stitched together to generate stitched samples; the stitched samples are input into the student model, supervised by pseudo-label data generated by the teacher model, and the Fourier enhancement loss is calculated.

[0017] The training unit is used to weight and combine the cross-teaching overall loss, entropy patch enhancement loss and Fourier enhancement loss to obtain the total loss function of the model, and to train the model based on the total loss function to obtain a trained model.

[0018] The segmentation unit is used to generate images of the acquired thyroid nodules to be segmented, input them into the trained segmentation model, and obtain the nodule segmentation results.

[0019] On the other hand, a hybrid domain-enhanced thyroid nodule semi-supervised segmentation device is provided, the hybrid domain-enhanced thyroid nodule semi-supervised segmentation device comprising: a processor; a memory storing computer-readable instructions, which, when executed by the processor, implement any of the methods described above for hybrid domain-enhanced thyroid nodule semi-supervised segmentation.

[0020] On the other hand, a computer-readable storage medium is provided, wherein at least one instruction is stored therein, the at least one instruction being loaded and executed by a processor to implement any of the above-described methods of semi-supervised segmentation of thyroid nodules based on hybrid domain enhancement.

[0021] The beneficial effects of the technical solutions provided in the embodiments of the present invention include at least the following:

[0022] This invention constructs a semi-supervised segmentation model that integrates spatial and frequency domain enhancements. This model includes a cross-teaching model, a unified bidirectional copy-and-paste enhancement module, an information entropy-based patch enhancement module, and a Fourier transform-based frequency domain enhancement module. During training, a unified bidirectional copy-and-paste strategy is used to uniformly and synchronously implement bidirectional embedding between labeled and unlabeled data, generating more diverse intermediate samples and effectively reducing data distribution differences. The information entropy-based patch enhancement module swaps high-confidence regions of labeled data with high-uncertainty regions of unlabeled data, strengthening supervision of uncertain regions and reducing confirmation bias in semantic learning. The Fourier transform-based frequency domain enhancement module swaps low-frequency components of labeled and unlabeled data and applies additional supervision to low-frequency predictions using full-frequency pseudo-labels, guiding the model to focus on semantic content rather than style variations. This enables the collaborative mining and fusion of spatial and frequency domain information in semi-supervised learning, significantly improving the generalization ability and accuracy of the segmentation model. It can more accurately capture nodule morphology when processing ultrasound images with speckle noise, low contrast, and blurred boundaries, generating more robust and accurate segmentation results, and significantly improving segmentation performance under limited labeled data conditions. Attached Figure Description

[0023] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0024] Figure 1 This is a flowchart of a semi-supervised segmentation method for thyroid nodules based on hybrid domain enhancement provided in an embodiment of the present invention;

[0025] Figure 2 This is a schematic diagram of the structure of a semi-supervised segmentation model for thyroid nodules based on hybrid domain enhancement provided in an embodiment of the present invention;

[0026] Figure 3 This is a block diagram of a semi-supervised segmentation device for thyroid nodules based on hybrid domain enhancement provided in an embodiment of the present invention;

[0027] Figure 4This is a schematic diagram of the structure of a semi-supervised segmentation device for thyroid nodules based on hybrid domain enhancement provided in an embodiment of the present invention. Detailed Implementation

[0028] The technical solution of the present invention will now be described with reference to the accompanying drawings.

[0029] In embodiments of the present invention, words such as "exemplarily," "for example," etc., are used to indicate that something is an example, illustration, or description. Any embodiment or design described as "exemplary" in the present invention should not be construed as being more preferred or advantageous than other embodiments or designs. Specifically, the use of the word "exemplary" is intended to present the concept in a concrete manner. Furthermore, in embodiments of the present invention, the meaning expressed by "and / or" can be both, or either one.

[0030] In the embodiments of this invention, the terms "image" and "picture" may sometimes be used interchangeably. It should be noted that, without emphasizing the distinction between them, they convey the same meaning. Similarly, the terms "of," "corresponding (relevant)," and "corresponding" may sometimes be used interchangeably. It should be noted that, without emphasizing the distinction between them, they convey the same meaning.

[0031] In this embodiment of the invention, sometimes a subscript such as W1 may be written in a non-subscript form such as W1. When the difference is not emphasized, the meaning they express is the same.

[0032] To make the technical problems, technical solutions and advantages of the present invention clearer, a detailed description will be given below in conjunction with the accompanying drawings and specific embodiments.

[0033] This invention provides a semi-supervised segmentation method for thyroid nodules based on hybrid domain enhancement. This method can be implemented using a semi-supervised segmentation device for thyroid nodules based on hybrid domain enhancement, which can be a terminal or a server. Figure 1 The flowchart shown is for a semi-supervised segmentation method for thyroid nodules based on hybrid domain enhancement. The processing flow of this method may include the following steps:

[0034] S1. Construct a semi-supervised segmentation model that integrates spatial domain and frequency domain enhancements; the model includes: a unified bidirectional copy-paste enhancement module, a cross-teaching model, an information entropy-based patch enhancement module, and a Fourier transform-based frequency domain enhancement module; among which, the cross-teaching model includes: a teacher model and two student models.

[0035] In this invention, both the teacher model and the two student models use U-Net as their backbone network. The teacher model does not directly participate in training; its weights are an exponential moving average of the student model weights. U-Net is a classic encoder-decoder convolutional neural network architecture. The encoder (downsampling path) progressively extracts deep contextual features of the image through convolution and pooling operations, while the decoder (upsampling path) progressively restores the spatial resolution of the feature map through operations such as transposed convolution to achieve pixel-level localization. The core innovation of U-Net lies in the introduction of skip connections, which concatenates the high-resolution and weakly semantic shallow features from the encoder stage with the corresponding deep and strong semantic features from the decoder stage. In this embodiment, the number of encoder channels is set to 16, 32, 64, 128, and 256, respectively. This configuration significantly improves the computational efficiency of the model while ensuring feature extraction capabilities.

[0036] Among them, such as Figure 2 The diagram shows a semi-supervised segmentation structure of thyroid nodules based on hybrid domain enhancement.

[0037] S2. Input the acquired labeled thyroid nodule ultrasound image data and unlabeled thyroid nodule ultrasound image data into the unified bidirectional copy and paste enhancement module to generate labeled and unlabeled data after weak enhancement and labeled and unlabeled data after strong enhancement. Generate intermediate samples through unified bidirectional copy and paste. Input the intermediate samples into the cross-teaching model and supervise the two student models through the pseudo-label data generated by the teacher model to calculate the overall cross-teaching loss.

[0038] In this embodiment of the invention, a method comprising... A dataset of labeled images and a by An unlabeled dataset consisting of 10 unlabeled images ,in .in, Indicates a labeled image. This indicates its corresponding gold standard. Indicates an unlabeled image, where, , , These represent the number of channels, height, and width of the input data, respectively.

[0039] In one feasible implementation, labeled and unlabeled thyroid nodule ultrasound image data are simultaneously input into a unified bidirectional copy-paste module to generate intermediate sample pairs. Specifically, embedding in two directions is performed simultaneously using a unified mask: pasting labeled data regions into unlabeled data, and pasting unlabeled data regions into labeled data, generating synthetic sample pairs that incorporate information from both. The generated sample pairs are then input into two student models for prediction. The student models are trained under supervision using a supervision signal generated by the pseudo-labels produced by the teacher model and the gold standard of the labeled data through the same unified bidirectional copy-paste operation. Simultaneously, a cross-supervision loss is introduced to align the predictions of the two student models, encouraging model consistency.

[0040] In this embodiment of the invention, a training strategy based on the Bidirectional Copy-Paste (BCP) method is used, dividing the training process into two stages: pre-training and self-training. In the pre-training stage, this embodiment uses uniform bidirectional copy-paste enhancement between different labeled data to train the supervised model. In the self-training stage, this embodiment uses the pre-trained model as a teacher model to generate pseudo-labels for unlabeled data. In both training stages, this embodiment applies a specific method to each labeled data... Both the unlabeled data u and the data 'u' underwent weak and strong augmentation, respectively, yielding the original weakly augmented data. and And the original strongly enhanced data and Weak enhancement includes random rotation and flipping, while strong enhancement is based on weak enhancement and involves color dithering and cropping.

[0041] Optionally, the specific process of the unified bidirectional copy and paste enhancement module is represented by the following formula (1):

[0042] (1)

[0043] in, Indicates weak enhancement or strong enhancement. This represents labeled data. This indicates unlabeled data; This represents the labeled data after weak enhancement; This represents the annotated data after strong enhancement; This represents unlabeled data after weak enhancement; This represents the unlabeled data after strong enhancement; This indicates a zero-center mask for performing uniform bidirectional copy and paste, with the size of the zero-value region being [value missing]. ,in Indicates the size factor for the zero-value region; Indicates the height of the input data. Indicates the width of the input data; These represent pasting the marked data area into the unmarked data area; This indicates that the unmarked data area is pasted into the marked data area; This indicates a new sample after weakly enhanced labeled data regions are pasted onto weakly or strongly unlabeled data. This indicates a new sample after a region of strongly enhanced labeled data has been pasted onto weakly or strongly unlabeled data. This indicates a new sample pasted from weakly enhanced unlabeled data regions to weakly or strongly enhanced labeled data regions. This indicates a new sample pasted from a strongly enhanced unlabeled data region onto a weakly or strongly enhanced labeled data region. This indicates element-wise multiplication.

[0044] Among them, the unified bidirectional copy and paste generates richer data by uniformly and simultaneously implementing two embedding directions between labeled and unlabeled samples. While effectively reducing the differences in data distribution, it significantly improves the diversity of spatial domain information combinations in the generated data.

[0045] Optionally, S2 inputs intermediate samples into the cross-teaching model, and supervises the two student models using pseudo-label data generated by the teacher model, including:

[0046] S21. Input the generated weak augmentation sample pairs and strong augmentation sample pairs into two student models with different initializations, respectively, to generate the probability map and pseudo-label of the weak augmentation sample and the probability map and pseudo-label of the strong augmentation sample.

[0047] In one feasible implementation, the newly generated weakly enhanced samples are paired and strongly enhanced sample pairs Input student models with different initializations respectively and The corresponding probability map and pseudo-labels are generated, which are represented by the following formula (2):

[0048] (2)

[0049] in, It is a weakly enhanced new sample The probability diagram, It is a weakly enhanced new sample pseudo-tags; It is a weakly enhanced new sample The probability diagram; Weakly enhanced new samples pseudo-tags; It is a strongly enhanced new sample The probability diagram; It is a strongly enhanced new sample pseudo-tags; It is a strongly enhanced new sample The probability diagram; It is a strongly enhanced new sample The pseudo-label; c represents a category label of the input data; Represents student model 1, Represents student model 2;

[0050] S22. Input the unlabeled data into the teacher model and calculate the initial pseudo-labels for the unlabeled data, expressed by the following formula (3):

[0051] (3)

[0052] in, This indicates that the input weakly enhanced unlabeled data has been processed. The generated initial pseudo-tags; This indicates that the input weakly enhanced unlabeled data has been processed. The generated predicted probabilities; This indicates that the input strongly enhanced unlabeled data has been processed. The generated predicted probabilities; This indicates that the input strongly enhanced unlabeled data has been processed. The generated initial pseudo-tags; This represents unlabeled data after weak enhancement;

[0053] S23. Copy and paste the initial pseudo-labels of the unlabeled data and the gold standard of the labeled data in a unified bidirectional manner to generate the final pseudo-labels for supervision of the two student models;

[0054] The process of generating the final pseudo-tags is represented by the following formula (4):

[0055] (4)

[0056] in, and The gold standard for indicating weak enhancement With the initial pseudo-label The final pseudo-tag is obtained by embedding it into the gold standard and the initial pseudo-tag direction respectively through unified bidirectional copy and paste; and The gold standard indicating weak or strong enhancement With the initial pseudo-label The final pseudo-tag is obtained by embedding it into the gold standard and the initial pseudo-tag direction respectively through unified bidirectional copy and paste; The gold standard indicating weak enhancement; Indicates a strong enhancement gold standard; This indicates the zero-center mask for performing a unified bidirectional copy and paste; This indicates that the input weakly enhanced unlabeled data has been processed. The generated predicted probabilities.

[0057] The cross-teaching overall loss utilizes pseudo-labels from one model to supervise the other. It consists of a supervisory loss and a consistency loss. The supervisory loss uses pseudo-labels generated by the teacher model to supervise the predictions of the two student models with different augmented inputs, ensuring effective learning from the gold standard.

[0058] Optionally, the overall loss from cross-teaching includes: supervision loss and consistency loss;

[0059] The functional expression for the supervision loss is represented by the following formula (5):

[0060] (5)

[0061] in, Indicates monitoring losses; This represents a linear combination of the dice loss and the cross-entropy loss;

[0062] To enhance consistency, the cross-supervision loss aligns the predictions of the two models with each other's pseudo-labels.

[0063] The functional expression for consistency loss is represented by the following formula (6):

[0064] (6)

[0065] in, This indicates a loss of consistency. Indicates die loss;

[0066] The functional expression for the overall loss of cross-teaching is represented by the following formula (7):

[0067] (7)

[0068] in, This indicates the overall loss in cross-disciplinary teaching; Indicates monitoring losses; This indicates a loss of consistency.

[0069] S3. Input the weakly enhanced labeled data and unlabeled data into the information entropy-based patch enhancement module, swap the low-entropy patch in the weakly enhanced labeled data with the high-entropy patch in the weakly enhanced unlabeled data to generate new samples; input the new samples into the student model for supervision, and calculate the entropy patch enhancement loss.

[0070] Optionally, the specific implementation process of S3 includes S31-S32:

[0071] S31. Input the weakly augmented labeled data and unlabeled data into the student model to obtain the predicted probability maps of the weakly augmented labeled data and unlabeled data; divide the predicted probability maps into K×K regions of size... The patches are processed; the average entropy value of each patch is calculated;

[0072] S32. Select the k patches with the lowest entropy values ​​from the labeled data after weak augmentation and the k patches with the highest entropy values ​​from the unlabeled data after weak augmentation; swap the positions of the selected patches and reorganize them to generate new samples; input the new samples into the student model for supervision and calculate the entropy patch augmentation loss.

[0073] The process of recombining to generate new samples is represented by the following formulas (8)-(9):

[0074] (8)

[0075] (9)

[0076] in, This indicates a new sample generated after the tag data exchange patch; This represents the region in the unlabeled data corresponding to the index set; It is an index set that includes the first k selected patches with the lowest entropy values; This indicates a new sample generated after an unlabeled data exchange patch; This represents the region in the labeled data corresponding to the index set; This represents the set of indices that includes the k selected patches with the highest entropy values;

[0077] Specifically, based on the entropy distribution map, the patches with the k highest entropy values ​​and the patches with the k lowest entropy values ​​are selected.

[0078] in, and This can be expressed by the following formulas (10)-(11):

[0079] (10)

[0080] (11)

[0081] in, Represents weakly augmented labeled data The corresponding entropy distribution diagram; This indicates weakly enhanced unlabeled data. The corresponding entropy distribution diagram; This indicates the index selection for the maximum value of k items in the data; This indicates the index selection of the k minimum values ​​in the data;

[0082] in, It is the average entropy value of the m-th patch, expressed by the following formula (12):

[0083] (12)

[0084] Where C represents the number of input data channels; This indicates that weakly enhanced or unannotated data is input to... The probability that the nth pixel in the mth patch belongs to class c;

[0085] Two new samples will be generated. and Input into two student models and Cross-supervision will be conducted.

[0086] The entropy patch enhancement loss is expressed by the following formula (13):

[0087] (13)

[0088] in, This represents the entropy patch enhancement loss; and It involves inputting new samples into the student model. The obtained probability plot prediction results and Then it is the probability graph corresponding to it. The pseudo-labels obtained after the operation; and For the model The prediction results and their pseudo-labels are as follows: and .

[0089] Specifically, the information entropy-based patch enhancement module exchanges high-confidence regions of labeled data with high-uncertainty regions of unlabeled data to strengthen the supervision of uncertain regions and reduce confirmation bias in semantic learning.

[0090] S4. Input the intermediate samples into the frequency domain enhancement module based on Fourier transform for fast Fourier transform to obtain the frequency domain spectrum; extract the central low-frequency component of the frequency domain spectrum and swap it, then reconstruct it into a spatial domain image through inverse Fourier transform to generate new samples; stitch the new samples together to generate stitched samples; input the stitched samples into the student model, supervised by the pseudo-label data generated by the teacher model, and calculate the Fourier enhancement loss.

[0091] In one feasible implementation, the image after a unified bidirectional copy-paste operation is transformed from the spatial domain to the frequency domain using a fast Fourier transform, as expressed by the following formula (14):

[0092] (14)

[0093] in, Indicates the FFT operation; express The frequency domain spectrum; express The frequency domain spectrum.

[0094] The process of extracting the center low-frequency component of the frequency domain spectrum is represented by the following formula (15):

[0095] (15)

[0096] in, This represents a rectangular low-frequency mask used to extract the low-frequency region at the center of the frequency domain spectrum. The mask is only located at the center. The value is 1 within the specified area and 0 in other areas. .

[0097] Among them, exchange and The low-frequency components of the frequency domain spectrum are obtained and the spatial domain image is recovered through inverse Fourier transform.

[0098] Optionally, the process of extracting the central low-frequency component of the frequency domain spectrum of S4 and exchanging it, reconstructing it into a spatial domain image through inverse Fourier transform, and generating new samples is represented by the following formula (16):

[0099] (16)

[0100] in, and This represents the new frequency domain spectrum generated after the weakly enhanced data frequency domain spectrum is swapped with the central low-frequency component. and This indicates the new frequency domain spectrum generated after the strong enhancement data frequency domain spectrum is swapped with the central low-frequency component. This indicates new weakly enhanced unlabeled data. The frequency domain spectrum; This indicates strongly enhanced unlabeled data. The frequency domain spectrum; This indicates new weakly enhanced unlabeled data. The frequency domain spectrum; This indicates new weakly enhanced unlabeled data. The frequency domain spectrum; express The new sample after inverse Fourier transform; express The new sample after inverse Fourier transform; express The new sample after inverse Fourier transform; express The new sample after inverse Fourier transform; This indicates its low-frequency component; This indicates the inverse Fourier transform operation.

[0101] In one feasible implementation, wherein This represents the inverse Fourier transform operation. This represents its low-frequency components. The objective of this invention is to ensure that different models produce consistent outputs on diverse samples, thereby ensuring they learn a unified feature representation. This consistency helps the model maintain strong performance on unseen samples, thus enhancing its generalization ability. To further enhance the effect, the four images generated after performing the low-frequency swap operation are stitched together into a new set. As input, it is represented by the following formula (17):

[0102] (17)

[0103] in, This indicates that the data is stitched together along the batch dimension. This represents the set of data after concatenation of dimensional entropy.

[0104] Alternatively, the functional expression for the Fourier enhancement loss can be represented by the following formula (18):

[0105] (18)

[0106] in, This represents the Fourier enhancement loss function; Represents the cross-supervision loss function; Represents the unsupervised loss function;

[0107] Among them, the data-enhanced cross-supervision loss based on Fourier transform ensures that the model can produce consistent outputs for exchanging low-frequency generated data.

[0108] Among them, cross-supervision loss This can be expressed by the following formula (19):

[0109] (19)

[0110] In one feasible implementation, this embodiment of the invention introduces a consistency constraint to enhance the consistency of feature representations. The original unlabeled image is input into the teacher model, and its output is used as a supervision signal to guide the student model that only receives the low-frequency components of the image as input. The specific process includes:

[0111] (1) The low-frequency components are extracted and expressed by the following formula (20):

[0112] (20)

[0113] in, express The frequency domain spectrum; express The extracted low-frequency components;

[0114] (2) Extract the low-frequency components The pseudo-labels are generated by inputting into the student model, and supervision is performed through the pseudo-labels as expressed by the following formula (21):

[0115] (twenty one)

[0116] in, It is the probability graph generated by the new sample after Fourier transform through Student Model 1; It is a probability graph go through The pseudo-labels obtained after the operation It is the probability graph generated by the new sample after Fourier transform through Student Model 2; It is a probability graph go through The pseudo-labels obtained after the operation.

[0117] Specifically, a frequency domain enhancement module based on Fourier transform swaps the low-frequency components of labeled and unlabeled data, and combines full-frequency pseudo-labels to apply additional supervision to low-frequency predictions, guiding the model to focus on semantic content rather than style variations. This enables the collaborative mining and fusion of spatial and frequency domain information in semi-supervised learning, significantly improving the generalization ability and accuracy of the segmentation model. It can more accurately capture nodule morphology when processing ultrasound images with speckle noise, low contrast, and blurred boundaries, generating more robust and accurate segmentation results, and significantly improving segmentation performance under limited labeled data conditions.

[0118] S5 weights and combines the overall loss from cross-teaching, the entropy patch enhancement loss, and the Fourier enhancement loss to obtain the model's total loss function. The model is then trained based on this total loss function to obtain a well-trained model.

[0119] In one feasible implementation, the overall loss function of the model is obtained by combining the cross-teaching overall loss, the entropy patch enhancement loss, and the Fourier enhancement loss, which is expressed by the following formula (22):

[0120] (twenty two)

[0121] in, This represents the total loss function of the model; This represents the overall loss function for cross-teaching; This represents the Fourier enhancement loss function; This indicates that the entropy patch enhances the loss function; This represents the first weight hyperparameter; This represents the second weight hyperparameter.

[0122] S6. The obtained images of the thyroid nodules to be segmented are generated and input into the trained segmentation model to obtain the nodule segmentation results.

[0123] In this invention, a semi-supervised segmentation method for thyroid nodule ultrasound images is implemented. This method introduces a unified bidirectional copy-paste strategy to construct intermediate samples between labeled and unlabeled data, enhancing data combination diversity and promoting knowledge transfer. Furthermore, a block enhancement mechanism based on information entropy is used to generate new samples from reliable labeled regions and highly uncertain unlabeled regions, strengthening supervision of ambiguous boundaries and mitigating confirmation bias in semantic learning. Additionally, a Fourier transform enhancement module is introduced, which exchanges the low-frequency spectral components of labeled and unlabeled data and uses full-frequency pseudo-labels to supervise low-frequency prediction, improving frequency domain feature diversity and guiding the model to focus on semantic content. This invention significantly improves the model's generalization ability and segmentation accuracy with limited labeled data through a dual enhancement strategy in both the spatial and frequency domains. This invention is applicable to ultrasound image segmentation, medical image analysis, and other semi-supervised visual tasks requiring the processing of low-contrast and high-noise images.

[0124] To demonstrate the effectiveness of the embodiments of the present invention, in specific application examples, the embodiments of the present invention were used on multiple thyroid nodule ultrasound datasets and compared and evaluated with existing semi-supervised segmentation methods. Specifically, the embodiments of the present invention were tested on three datasets—TN3K, DDTI, and TUD—with 5% labeled data, further verifying their robustness and generalization ability.

[0125] Table 1 shows the segmentation results of different methods on the TN3K dataset at 5% and 10% annotation ratios. The embodiments of this invention are compared with DCT (Deep Co-training), ICT (Interpolation Consistency Training), CCT (Cross Consistency Training), CPS (Cross Pseudo-Supervised Training), URPC (Uncertainty Correction Pyramid Consistency), AD-MT (Alternating Diversified Mean Teacher), ABD (Adaptive Bidirectional Shift), UA-MT (Uncertainty-Aware Mean Teacher), UCMT (Uncertainty-Guided Co-mean Teacher), SS-Net (Smooth Separation Network), SAMT-PCL (Shape-Aware Multi-Task Co-aware Perception-Pixel-Level Contrast Learning), and EM (Entropy Minimization). In Table 1, ↑ indicates a higher value is better, while ↓ indicates the opposite. The values ​​are DSC (Dice Similarity Coefficient), JC (Jaccard Index), 95HD (95% Hausdorff Distance), and ASD (Average Surface Distance).

[0126] Table 1

[0127]

[0128] Table 2 shows the segmentation results of different methods on the DDTI dataset at 5% and 10% annotation ratios. The results demonstrate that this invention outperforms other semi-supervised segmentation methods at both annotation ratios.

[0129] Table 2

[0130]

[0131] Table 3 shows the segmentation results of different methods on the TUD dataset at 5% and 10% annotation ratios. The results demonstrate that this invention outperforms other semi-supervised segmentation methods at both annotation ratios.

[0132] Table 3

[0133]

[0134] This invention constructs a semi-supervised segmentation model that integrates spatial and frequency domain enhancements. This model includes a cross-teaching model, a unified bidirectional copy-and-paste enhancement module, an information entropy-based patch enhancement module, and a Fourier transform-based frequency domain enhancement module. During training, a unified bidirectional copy-and-paste strategy is used to uniformly and synchronously implement bidirectional embedding between labeled and unlabeled data, generating more diverse intermediate samples and effectively reducing data distribution differences. The information entropy-based patch enhancement module swaps high-confidence regions of labeled data with high-uncertainty regions of unlabeled data, strengthening supervision of uncertain regions and reducing confirmation bias in semantic learning. The Fourier transform-based frequency domain enhancement module swaps low-frequency components of labeled and unlabeled data and applies additional supervision to low-frequency predictions using full-frequency pseudo-labels, guiding the model to focus on semantic content rather than style variations. This enables the collaborative mining and fusion of spatial and frequency domain information in semi-supervised learning, significantly improving the generalization ability and accuracy of the segmentation model. It can more accurately capture nodule morphology when processing ultrasound images with speckle noise, low contrast, and blurred boundaries, generating more robust and accurate segmentation results, and significantly improving segmentation performance under limited labeled data conditions.

[0135] Figure 3 This is a block diagram of a semi-supervised segmentation device for thyroid nodules based on hybrid domain enhancement, provided in an embodiment of the present invention. This device is used in a semi-supervised segmentation method for thyroid nodules based on hybrid domain enhancement. (Refer to...) Figure 3 The device includes a construction unit 310, a unified bidirectional copy-paste enhancement unit 320, an information entropy-based patch enhancement unit 330, a Fourier transform-based frequency domain enhancement unit 340, a training unit 350, and a segmentation unit 360. Wherein:

[0136] Construction unit 310 is used to construct a semi-supervised segmentation model that integrates spatial domain and frequency domain enhancements; the model includes: a unified bidirectional copy-paste enhancement module, a cross-teaching model, an information entropy-based patch enhancement module, and a Fourier transform-based frequency domain enhancement module; wherein, the cross-teaching model includes: a teacher model and two student models;

[0137] The unified bidirectional copy-paste enhancement unit 320 is used to input the acquired labeled and unlabeled thyroid nodule ultrasound image data into the unified bidirectional copy-paste enhancement module to generate weakly enhanced labeled and unlabeled data, as well as strongly enhanced labeled and unlabeled data. Intermediate samples are generated through unified bidirectional copy-paste. The intermediate samples are then input into the cross-teaching model, and the pseudo-label data generated by the teacher model is used to supervise the two student models and calculate the overall cross-teaching loss.

[0138] The information entropy-based patch enhancement unit 330 is used to input weakly enhanced labeled data and unlabeled data into the information entropy-based patch enhancement module, exchange low-entropy patches in the weakly enhanced labeled data with high-entropy patches in the weakly enhanced unlabeled data to generate new samples; input the new samples into the student model for supervision, and calculate the entropy patch enhancement loss.

[0139] The Fourier transform-based frequency domain enhancement unit 340 is used to input intermediate samples into the Fourier transform-based frequency domain enhancement module for fast Fourier transform to obtain the frequency domain spectrum; extract the central low-frequency component of the frequency domain spectrum and swap it, reconstruct it into a spatial domain image through inverse Fourier transform to generate new samples; stitch the new samples to generate stitched samples; input the stitched samples into the student model, supervised by pseudo-label data generated by the teacher model, and calculate the Fourier enhancement loss;

[0140] Training unit 350 is used to weight and combine the cross-teaching overall loss, entropy patch enhancement loss and Fourier enhancement loss to obtain the total loss function of the model, and train the model based on the total loss function to obtain a trained model.

[0141] The segmentation unit 360 is used to generate images of the acquired thyroid nodules to be segmented, input them into the trained segmentation model, and obtain the nodule segmentation results.

[0142] Optionally, the specific process of the unified bidirectional copy and paste enhancement module is represented by the following formula (1):

[0143] (1)

[0144] in, Indicates weak enhancement or strong enhancement. This represents labeled data. This indicates unlabeled data; This represents the labeled data after weak enhancement; This represents the annotated data after strong enhancement; This represents unlabeled data after weak enhancement; This represents the unlabeled data after strong enhancement; This indicates a zero-center mask for performing uniform bidirectional copy and paste, with the size of the zero-value region being [value missing]. ,in Indicates the size factor for the zero-value region; Indicates the height of the input data. Indicates the width of the input data; These represent pasting the marked data area into the unmarked data area; This indicates that the unmarked data area is pasted into the marked data area; This indicates a new sample after weakly enhanced labeled data regions are pasted onto weakly or strongly unlabeled data. This indicates a new sample after a region of strongly enhanced labeled data has been pasted onto weakly or strongly unlabeled data. This indicates a new sample pasted from weakly enhanced unlabeled data regions to weakly or strongly enhanced labeled data regions. This indicates a new sample pasted from a strongly enhanced unlabeled data region onto a weakly or strongly enhanced labeled data region. This indicates element-wise multiplication.

[0145] Optionally, the step of inputting intermediate samples into the cross-teaching model and supervising the two student models using pseudo-label data generated by the teacher model includes:

[0146] The generated weak and strong augmentation sample pairs are input into two student models with different initializations, respectively, to generate the probability map and pseudo-label of the weak augmentation sample and the probability map and pseudo-label of the strong augmentation sample, which are expressed by the following formula (2):

[0147] (2)

[0148] in, It is a weakly enhanced new sample The probability diagram, It is a weakly enhanced new sample pseudo-tags; It is a weakly enhanced new sample The probability diagram; Weakly enhanced new samples pseudo-tags; It is a strongly enhanced new sample The probability diagram; It is a strongly enhanced new sample pseudo-tags; It is a strongly enhanced new sample The probability diagram; It is a strongly enhanced new sample The pseudo-label; c represents a category label of the input data; Represents student model 1, Represents student model 2;

[0149] Input the unlabeled data into the teacher model and calculate the initial pseudo-labels for the unlabeled data, which are expressed by the following formula (3):

[0150] (3)

[0151] in, This indicates that the input weakly enhanced unlabeled data has been processed. The generated initial pseudo-tags; This indicates that the input weakly enhanced unlabeled data has been processed. The generated predicted probabilities; This indicates that the input strongly enhanced unlabeled data has been processed. The generated predicted probabilities; This indicates that the input strongly enhanced unlabeled data has been processed. The generated initial pseudo-tags; This represents unlabeled data after weak enhancement;

[0152] The initial pseudo-labels for unlabeled data are copied and pasted bidirectionally with the gold standard for labeled data to generate the final pseudo-labels for supervising the two student models;

[0153] The process of generating the final pseudo-tags is represented by the following formula (4):

[0154] (4)

[0155] in, and The gold standard for indicating weak enhancement With the initial pseudo-label The final pseudo-tag is obtained by embedding it into the gold standard and the initial pseudo-tag direction respectively through unified bidirectional copy and paste; and The gold standard indicating weak or strong enhancement With the initial pseudo-label The final pseudo-tag is obtained by embedding it into the gold standard and the initial pseudo-tag direction respectively through unified bidirectional copy and paste; The gold standard indicating weak enhancement; Indicates a strong enhancement gold standard; This indicates the zero-center mask for performing a unified bidirectional copy and paste; This indicates that the input weakly enhanced unlabeled data has been processed. The generated predicted probabilities.

[0156] Optionally, the overall loss of cross-teaching includes: supervision loss and consistency loss;

[0157] The functional expression for the supervision loss is represented by the following formula (5):

[0158] (5)

[0159] in, Indicates monitoring losses; This represents a linear combination of the dice loss and the cross-entropy loss;

[0160] The functional expression for consistency loss is represented by the following formula (6):

[0161] (6)

[0162] in, This indicates a loss of consistency. Indicates die loss;

[0163] The functional expression for the overall loss of cross-teaching is represented by the following formula (7):

[0164] (7)

[0165] in, This indicates the overall loss in cross-disciplinary teaching; Indicates monitoring losses; This indicates a loss of consistency.

[0166] Optionally, the information entropy-based patch enhancement unit 330 is used for:

[0167] The weakly augmented labeled data and unlabeled data are input into the student model to obtain the predicted probability maps of the weakly augmented labeled data and unlabeled data; the predicted probability maps are then divided into K×K regions of size [missing information]. The patches are processed; the average entropy value of each patch is calculated;

[0168] Select the k patches with the lowest entropy values ​​from the labeled data after weak augmentation and the k patches with the highest entropy values ​​from the unlabeled data after weak augmentation; swap the positions of the selected patches and reorganize them to generate new samples; input the new samples into the student model for supervision and calculate the entropy patch augmentation loss;

[0169] The process of recombining to generate new samples is represented by the following formulas (8)-(9):

[0170] (8)

[0171] (9)

[0172] in, This indicates a new sample generated after the tag data exchange patch; This represents the region in the unlabeled data corresponding to the index set; It is an index set that includes the first k selected patches with the lowest entropy values; This indicates a new sample generated after an unlabeled data exchange patch; This represents the region in the labeled data corresponding to the index set; This represents the set of indices that includes the k selected patches with the highest entropy values;

[0173] in, and This can be expressed by the following formulas (10)-(11):

[0174] (10)

[0175] (11)

[0176] in, Represents weakly augmented labeled data The corresponding entropy distribution diagram; This indicates weakly enhanced unlabeled data. The corresponding entropy distribution diagram; This indicates the index selection for the maximum value of k items in the data; This indicates the index selection of the k minimum values ​​in the data;

[0177] in, It is the average entropy value of the m-th patch, expressed by the following formula (12):

[0178] (12)

[0179] Where C represents the number of input data channels; This indicates that weakly enhanced or unannotated data is input to... The probability that the nth pixel in the mth patch belongs to class c;

[0180] The entropy patch enhancement loss is expressed by the following formula (13):

[0181] (13)

[0182] in, This represents the entropy patch enhancement loss; and It involves inputting new samples into the student model. The obtained probability plot prediction results and Then it is the probability graph corresponding to it. The pseudo-labels obtained after the operation; and For the model The prediction results and their pseudo-labels are as follows: and .

[0183] Optionally, the process of extracting the central low-frequency component of the frequency domain spectrum, exchanging it, and reconstructing it into a spatial domain image through inverse Fourier transform to generate new samples is represented by the following formula (14):

[0184] (14)

[0185] in, and This represents the new frequency domain spectrum generated after the weakly enhanced data frequency domain spectrum is swapped with the central low-frequency component. and This indicates the new frequency domain spectrum generated after the strong enhancement data frequency domain spectrum is swapped with the central low-frequency component. This indicates new weakly enhanced unlabeled data. The frequency domain spectrum; This indicates strongly enhanced unlabeled data. The frequency domain spectrum; This indicates new weakly enhanced unlabeled data. The frequency domain spectrum; This indicates new weakly enhanced unlabeled data. The frequency domain spectrum; express The new sample after inverse Fourier transform; express The new sample after inverse Fourier transform; express The new sample after inverse Fourier transform; express The new sample after inverse Fourier transform; This indicates its low-frequency component; This indicates the inverse Fourier transform operation.

[0186] Optionally, the functional expression of the Fourier enhancement loss is represented by the following formula (15):

[0187] (15)

[0188] in, This represents the Fourier enhancement loss function; Indicates cross-monitoring loss; This indicates unsupervised loss;

[0189] Among them, cross-supervision loss This can be expressed by the following formula (16):

[0190] (16)

[0191] Among them, unsupervised losses This can be expressed by the following formula (17):

[0192] (17)

[0193] in, It is the probability graph generated by the new sample after Fourier transform through Student Model 1; It is a probability graph go through The pseudo-labels obtained after the operation It is the probability graph generated by the new sample after Fourier transform through Student Model 2; It is a probability graph go through The pseudo-labels obtained after the operation.

[0194] This invention constructs a semi-supervised segmentation model that integrates spatial and frequency domain enhancements. This model includes a cross-teaching model, a unified bidirectional copy-and-paste enhancement module, an information entropy-based patch enhancement module, and a Fourier transform-based frequency domain enhancement module. During training, a unified bidirectional copy-and-paste strategy is used to uniformly and synchronously implement bidirectional embedding between labeled and unlabeled data, generating more diverse intermediate samples and effectively reducing data distribution differences. The information entropy-based patch enhancement module swaps high-confidence regions of labeled data with high-uncertainty regions of unlabeled data, strengthening supervision of uncertain regions and reducing confirmation bias in semantic learning. The Fourier transform-based frequency domain enhancement module swaps low-frequency components of labeled and unlabeled data and applies additional supervision to low-frequency predictions using full-frequency pseudo-labels, guiding the model to focus on semantic content rather than style variations. This enables the collaborative mining and fusion of spatial and frequency domain information in semi-supervised learning, significantly improving the generalization ability and accuracy of the segmentation model. It can more accurately capture nodule morphology when processing ultrasound images with speckle noise, low contrast, and blurred boundaries, generating more robust and accurate segmentation results, and significantly improving segmentation performance under limited labeled data conditions.

[0195] Figure 4 This is a schematic diagram of a semi-supervised segmentation device for thyroid nodules based on hybrid domain enhancement, provided in an embodiment of the present invention. Figure 4 As shown, the hybrid domain-enhanced thyroid nodule semi-supervised segmentation device may include the above-mentioned Figure 3 The illustrated hybrid domain-enhanced thyroid nodule semi-supervised segmentation device 410 may optionally include a first processor 2001.

[0196] Optionally, the hybrid domain-enhanced thyroid nodule semi-supervised segmentation device 410 may also include a memory 2002 and a transceiver 2003.

[0197] The first processor 2001, memory 2002, and transceiver 2003 can be connected via a communication bus.

[0198] The following is combined with Figure 4 A detailed description of each component of the hybrid domain-enhanced thyroid nodule semi-supervised segmentation device 410 is provided below:

[0199] The first processor 2001 is the control center of the hybrid-domain enhanced thyroid nodule semi-supervised segmentation device 410. It can be a single processor or a collective term for multiple processing elements. For example, the first processor 2001 can be one or more central processing units (CPUs), application-specific integrated circuits (ASICs), or one or more integrated circuits configured to implement embodiments of the present invention, such as one or more digital signal processors (DSPs), or one or more field-programmable gate arrays (FPGAs).

[0200] Optionally, the first processor 2001 can perform various functions of the hybrid domain-enhanced thyroid nodule semi-supervised segmentation device 410 by running or executing software programs stored in the memory 2002 and calling data stored in the memory 2002.

[0201] In a specific implementation, as one example, the first processor 2001 may include one or more CPUs, for example... Figure 4 CPU0 and CPU1 are shown in the diagram.

[0202] In a specific implementation, as one example, the hybrid domain-enhanced thyroid nodule semi-supervised segmentation device 410 may also include multiple processors, such as... Figure 4 The first processor 2001 and the second processor 2004 are shown in the diagram. Each of these processors can be a single-core processor or a multi-core processor. Here, a processor can refer to one or more devices, circuits, and / or processing cores used to process data (such as computer program instructions).

[0203] The memory 2002 is used to store the software program that executes the present invention, and is controlled by the first processor 2001 to execute it. The specific implementation method can be referred to the above method embodiment, and will not be repeated here.

[0204] Optionally, the memory 2002 may be a read-only memory (ROM) or other type of static storage device capable of storing static information and instructions, random access memory (RAM) or other type of dynamic storage device capable of storing information and instructions, or electrically erasable programmable read-only memory (EEPROM), compact disc read-only memory (CD-ROM) or other optical disc storage, optical disc storage (including compressed optical discs, laser discs, optical discs, digital universal optical discs, Blu-ray discs, etc.), magnetic disk storage media or other magnetic storage devices, or any other medium capable of carrying or storing desired program code in the form of instructions or data structures and accessible by a computer, but not limited thereto. The memory 2002 may be integrated with the first processor 2001 or may exist independently, and may be connected via the interface circuit of the hybrid domain-enhanced thyroid nodule semi-supervised segmentation device 410. Figure 4 (Not shown in the image) is coupled to the first processor 2001, and this embodiment of the invention does not specifically limit this.

[0205] The transceiver 2003 is used to communicate with network devices or with terminal devices.

[0206] Alternatively, transceiver 2003 may include a receiver and a transmitter. Figure 4 (Not shown separately). The receiver is used to implement the receiving function, and the transmitter is used to implement the transmitting function.

[0207] Optionally, the transceiver 2003 can be integrated with the first processor 2001 or exist independently, and can be connected via the interface circuit of the hybrid domain-enhanced thyroid nodule semi-supervised segmentation device 410. Figure 4 (Not shown in the image) is coupled to the first processor 2001, and this embodiment of the invention does not specifically limit this.

[0208] It should be noted that, Figure 4 The structure of the hybrid domain-enhanced thyroid nodule semi-supervised segmentation device 410 shown in the figure does not constitute a limitation on the router. Actual hybrid domain-enhanced thyroid nodule semi-supervised segmentation devices may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0209] Furthermore, the technical effects of the hybrid domain-enhanced thyroid nodule semi-supervised segmentation device 410 can be referred to the technical effects of the hybrid domain-enhanced thyroid nodule semi-supervised segmentation method described in the above method embodiments, and will not be repeated here.

[0210] It should be understood that the first processor 2001 in the embodiments of the present invention may be a central processing unit (CPU), or it may be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor, or it may be any conventional processor, etc.

[0211] It should also be understood that the memory in the embodiments of the present invention can be volatile memory or non-volatile memory, or may include both volatile and non-volatile memory. The non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. The volatile memory can be random access memory (RAM), which is used as an external cache. By way of example, but not limitation, many forms of random access memory (RAM) are available, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate synchronous DRAM (DDR SDRAM), enhanced synchronous DRAM (ESDRAM), synchronous linked DRAM (SLDRAM), and direct rambus RAM (DR RAM).

[0212] The above embodiments can be implemented, in whole or in part, by software, hardware (such as circuits), firmware, or any other combination thereof. When implemented using software, the above embodiments can be implemented, in whole or in part, as a computer program product. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer programs are loaded or executed on a computer, all or part of the processes or functions described in the embodiments of the present invention are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more sets of available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium. A semiconductor medium can be a solid-state drive.

[0213] It should be understood that the term "and / or" in this article is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. A and B can be singular or plural. Additionally, the character " / " in this article generally indicates an "or" relationship between the preceding and following related objects, but it can also represent an "and / or" relationship. Please refer to the context for a more accurate understanding.

[0214] In this invention, "at least one" means one or more, and "more than one" means two or more. "At least one of the following" or similar expressions refer to any combination of these items, including any combination of a single item or a plurality of items. For example, at least one of a, b, or c can represent: a, b, c, ab, ac, bc, or abc, where a, b, and c can be a single item or multiple items.

[0215] It should be understood that, in various embodiments of the present invention, the order of the above-mentioned process numbers does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0216] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0217] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the devices, apparatuses, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0218] In the embodiments provided by this invention, it should be understood that the disclosed devices, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another device, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0219] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0220] In addition, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0221] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0222] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A semi-supervised segmentation method for thyroid nodules based on hybrid domain enhancement, characterized in that, The method includes: S1. Construct a semi-supervised segmentation model that integrates spatial domain and frequency domain enhancements; the model includes: a unified bidirectional copy-paste enhancement module, a cross-teaching model, an information entropy-based patch enhancement module, and a Fourier transform-based frequency domain enhancement module; wherein, the cross-teaching model includes: a teacher model and two student models; S2. Input the acquired labeled thyroid nodule ultrasound image data and unlabeled thyroid nodule ultrasound image data into the unified bidirectional copy and paste enhancement module to generate labeled and unlabeled data after weak enhancement and labeled and unlabeled data after strong enhancement. Generate intermediate samples through unified bidirectional copy and paste. Input the intermediate samples into the cross-teaching model and supervise the two student models through the pseudo-label data generated by the teacher model to calculate the overall cross-teaching loss. The specific process of the unified bidirectional copy and paste enhancement module is represented by the following formula (1): （1） in, Indicates weak enhancement or strong enhancement. This represents labeled data. This indicates unlabeled data; This represents the labeled data after weak enhancement; This represents the annotated data after strong enhancement; This represents unlabeled data after weak enhancement; This represents the unlabeled data after strong enhancement; This indicates a zero-center mask for performing uniform bidirectional copy and paste, with the size of the zero-value region being [value missing]. ,in Indicates the size factor for the zero-value region; Indicates the height of the input data. Indicates the width of the input data; These represent pasting the marked data area into the unmarked data area; This indicates that the unmarked data area is pasted into the marked data area; This represents a new sample after the weakly enhanced labeled data region is pasted onto the weakly unlabeled data; This indicates a new sample after pasting the strongly enhanced labeled data region onto the strongly unlabeled data region; This represents a new sample where the weakly enhanced unlabeled data region is pasted onto the weakly enhanced labeled data region. This indicates a new sample where the strongly enhanced unlabeled data region is pasted onto the strongly enhanced labeled data region; This represents element-wise multiplication; S3. Input the weakly enhanced labeled data and unlabeled data into the information entropy-based patch enhancement module, swap the low-entropy patch in the weakly enhanced labeled data with the high-entropy patch in the weakly enhanced unlabeled data to generate new samples; input the new samples into the student model for supervision, and calculate the entropy patch enhancement loss. S4. Input the intermediate samples into the frequency domain enhancement module based on Fourier transform for fast Fourier transform to obtain the frequency domain spectrum; extract the central low-frequency component of the frequency domain spectrum and swap it, then reconstruct it into a spatial domain image through inverse Fourier transform to generate new samples; stitch the new samples along the batch dimension to generate stitched samples; input the stitched samples into the student model, supervised by the pseudo-label data generated by the teacher model, and calculate the Fourier enhancement loss; S5 performs a weighted combination of the cross-teaching overall loss, entropy patch enhancement loss, and Fourier enhancement loss to obtain the total loss function of the model. The model is then trained based on the total loss function to obtain a trained model. S6. The obtained images of the thyroid nodules to be segmented are generated and input into the trained segmentation model to obtain the nodule segmentation results.

2. The semi-supervised segmentation method for thyroid nodules based on hybrid domain enhancement according to claim 1, characterized in that, The S2 step involves inputting intermediate samples into the cross-teaching model and supervising the two student models using pseudo-label data generated by the teacher model, including: S21. Input the generated weakly enhanced sample pairs and strongly enhanced sample pairs into two student models with different initializations, respectively, to generate the probability map and pseudo-label of the weakly enhanced samples and the probability map and pseudo-label of the strongly enhanced samples, as expressed by the following formula (2): （2） in, It is a weakly enhanced new sample The probability diagram, It is a weakly enhanced new sample pseudo-tags; It is a weakly enhanced new sample The probability diagram; Weakly enhanced new samples pseudo-tags; It is a strongly enhanced new sample The probability diagram; It is a strongly enhanced new sample pseudo-tags; It is a strongly enhanced new sample The probability diagram; It is a strongly enhanced new sample The pseudo-label; c represents a category label of the input data; Represents student model 1, Represents student model 2; S22. Input the unlabeled data into the teacher model and calculate the initial pseudo-labels for the unlabeled data, expressed by the following formula (3): （3） in, This indicates that the input weakly enhanced unlabeled data has been processed. The generated initial pseudo-tags; This indicates that the input weakly enhanced unlabeled data has been processed. The generated predicted probabilities; This indicates that the input strongly enhanced unlabeled data has been processed. The generated predicted probabilities; This indicates that the input strongly enhanced unlabeled data has been processed. The generated initial pseudo-tags; This represents unlabeled data after weak enhancement; S23. Copy and paste the initial pseudo-labels of the unlabeled data and the gold standard of the labeled data in a unified bidirectional manner to generate the final pseudo-labels for supervision of the two student models; The process of generating the final pseudo-tags is represented by the following formula (4): （4） in, and The gold standard for indicating weak enhancement With the initial pseudo-label The final pseudo-tag is obtained by embedding it into the gold standard and the initial pseudo-tag direction respectively through unified bidirectional copy and paste; and The gold standard indicating weak or strong enhancement With the initial pseudo-label The final pseudo-tag is obtained by embedding it into the gold standard and the initial pseudo-tag direction respectively through unified bidirectional copy and paste; The gold standard indicating weak enhancement; Indicates a strong enhancement gold standard; This indicates the zero-center mask for performing a unified bidirectional copy and paste; This indicates that the input weakly enhanced unlabeled data has been processed. The generated predicted probabilities.

3. The semi-supervised segmentation method for thyroid nodules based on hybrid domain enhancement according to claim 1, characterized in that, The overall loss in cross-teaching includes: supervision loss and consistency loss; The functional expression for the supervision loss is represented by the following formula (5): （5） in, Indicates monitoring losses; This represents a linear combination of the dice loss and the cross-entropy loss; The functional expression for consistency loss is represented by the following formula (6): （6） in, This indicates a loss of consistency. Indicates die loss; The functional expression for the overall loss of cross-teaching is represented by the following formula (7): （7） in, This indicates the overall loss in cross-disciplinary teaching; Indicates monitoring losses; This indicates a loss of consistency.

4. The semi-supervised segmentation method for thyroid nodules based on hybrid domain enhancement according to claim 1, characterized in that, S3 inputs the weakly enhanced labeled data and unlabeled data into the information entropy-based patch enhancement module, and swaps the low-entropy patch in the weakly enhanced labeled data with the high-entropy patch in the weakly enhanced unlabeled data to generate a new sample; New samples are input into the student model for supervision, and the entropy patching enhancement loss is calculated, including: S31. Input the weakly augmented labeled data and unlabeled data into the student model to obtain the predicted probability maps of the weakly augmented labeled data and unlabeled data; divide the predicted probability maps into K×K groups of size... The patches are processed; the average entropy value of each patch is calculated; S32. Select the k patches with the lowest entropy values ​​from the labeled data after weak augmentation and the k patches with the highest entropy values ​​from the unlabeled data after weak augmentation; swap the positions of the selected patches and reorganize them to generate new samples; input the new samples into the student model for supervision and calculate the entropy patch augmentation loss. The process of recombining to generate new samples is represented by the following formulas (8)-(9): （8） （9） in, This indicates a new sample generated after the tag data exchange patch; This represents the region in the unlabeled data corresponding to the index set; It is an index set that includes the first k selected patches with the lowest entropy values; This indicates a new sample generated after an unlabeled data exchange patch; This represents the region in the labeled data corresponding to the index set; This represents the set of indices that includes the k selected patches with the highest entropy values; in, and This can be expressed by the following formulas (10)-(11): （10） （11） in, Represents weakly augmented labeled data The corresponding entropy distribution diagram; This indicates weakly enhanced unlabeled data. The corresponding entropy distribution diagram; This indicates the index selection for the maximum value of k items in the data; This indicates the index selection of the k minimum values ​​in the data; in, It is the average entropy value of the m-th patch, expressed by the following formula (12): （12） Where C represents the number of input data channels; This indicates that weakly enhanced or unannotated data is input to... The probability that the nth pixel in the mth patch belongs to class c; The entropy patch enhancement loss is expressed by the following formula (13): （13） in, This represents the entropy patch enhancement loss; This indicates a new sample generated after the tag data exchange patch; This indicates a new sample generated after an unlabeled data exchange patch; and These are the new samples and Input student model The obtained probability plot prediction results and Then it is the probability graph corresponding to it. The pseudo-labels obtained after the operation; and For new samples, the model The prediction results and their pseudo-labels are as follows: and .

5. The semi-supervised segmentation method for thyroid nodules based on hybrid domain enhancement according to claim 1, characterized in that, The process of extracting the central low-frequency component of the frequency domain spectrum in S4, exchanging it, reconstructing it into a spatial domain image through inverse Fourier transform, and generating new samples is represented by the following formula (14): （14） in, and This represents the new frequency domain spectrum generated after the weakly enhanced data frequency domain spectrum is swapped with the central low-frequency component. and This indicates the new frequency domain spectrum generated after the strong enhancement data frequency domain spectrum is swapped with the central low-frequency component. This indicates new weakly enhanced unlabeled data. The frequency domain spectrum; This indicates strongly enhanced unlabeled data. The frequency domain spectrum; This indicates new weakly enhanced unlabeled data. The frequency domain spectrum; This indicates new weakly enhanced unlabeled data. The frequency domain spectrum; express The new sample after inverse Fourier transform; express The new sample after inverse Fourier transform; express The new sample after inverse Fourier transform; express The new sample after inverse Fourier transform; This indicates its low-frequency component; This indicates the inverse Fourier transform operation.

6. The semi-supervised segmentation method for thyroid nodules based on hybrid domain enhancement according to claim 1, characterized in that, The functional expression of the Fourier enhancement loss is represented by the following formula (15): （15） in, This represents the Fourier enhancement loss function; Indicates cross-monitoring loss; This indicates unsupervised loss; Among them, cross-supervision loss This can be expressed by the following formula (16): （16） Among them, unsupervised losses The following formula (17) represents （17） in, It is the probability graph generated by the new sample after Fourier transform through Student Model 1; It is a probability graph go through The pseudo-labels obtained after the operation It is the probability graph generated by the new sample after Fourier transform through Student Model 2; It is a probability graph go through The pseudo-labels obtained after the operation.

7. A semi-supervised segmentation device for thyroid nodules based on hybrid domain enhancement, wherein the semi-supervised segmentation device for thyroid nodules based on hybrid domain enhancement is used to implement the semi-supervised segmentation method for thyroid nodules based on hybrid domain enhancement as described in any one of claims 1-6, characterized in that, The device includes: A building unit is used to construct a semi-supervised segmentation model that integrates spatial domain and frequency domain enhancements; the model includes: a unified bidirectional copy-paste enhancement module, a cross-teaching model, an information entropy-based patch enhancement module, and a Fourier transform-based frequency domain enhancement module; wherein, the cross-teaching model includes: a teacher model and two student models; A unified bidirectional copy-paste enhancement unit is used to input the acquired labeled and unlabeled thyroid nodule ultrasound image data into the unified bidirectional copy-paste enhancement module to generate weakly enhanced labeled and unlabeled data, as well as strongly enhanced labeled and unlabeled data. Intermediate samples are generated through unified bidirectional copy-paste. The intermediate samples are then input into the cross-teaching model, and the pseudo-label data generated by the teacher model is used to supervise the two student models to calculate the overall cross-teaching loss. The information entropy-based patch enhancement unit is used to input weakly enhanced labeled data and unlabeled data into the information entropy-based patch enhancement module, swap low-entropy patches in the weakly enhanced labeled data with high-entropy patches in the weakly enhanced unlabeled data to generate new samples; the new samples are then input into the student model for supervision, and the entropy patch enhancement loss is calculated. The Fourier transform-based frequency domain enhancement unit is used to input intermediate samples into the Fourier transform-based frequency domain enhancement module for fast Fourier transform to obtain the frequency domain spectrum; the central low-frequency component of the frequency domain spectrum is extracted and swapped, and reconstructed into a spatial domain image through inverse Fourier transform to generate new samples; the new samples are stitched together to generate stitched samples; the stitched samples are input into the student model, supervised by pseudo-label data generated by the teacher model, and the Fourier enhancement loss is calculated. The training unit is used to weight and combine the cross-teaching overall loss, entropy patch enhancement loss and Fourier enhancement loss to obtain the total loss function of the model, and to train the model based on the total loss function to obtain a trained model. The segmentation unit is used to generate images of the acquired thyroid nodules to be segmented, input them into the trained segmentation model, and obtain the nodule segmentation results.

8. A semi-supervised segmentation device for thyroid nodules based on hybrid domain enhancement, characterized in that, The hybrid domain-enhanced thyroid nodule semi-supervised segmentation device includes: processor; A memory storing computer-readable instructions that, when executed by the processor, implement the method as described in any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium contains program code that can be invoked by a processor to execute the method as described in any one of claims 1 to 6.

Images