Thyroid nodule benign and malignant prediction method based on automatic segmentation and semi-supervised learning

Abstract

The invention discloses a thyroid nodule benign and malignant prediction method based on automatic segmentation and semi-supervised learning. The thyroid nodule benign and malignant prediction method comprises the following steps of data set establishment, image preprocessing, thyroid nodule segmentation model training, mask and image fusion processing, thyroid nodule benign and malignant classification model training and semi-supervised learning model construction. According to the method, the thyroid nodule effective area is segmented firstly, subsequent benign and malignant classification tasks are assisted, and information space is expanded and learned through semi-supervised learning, so that the problems of high medical image marking cost, insufficient data and limitation on deep learning model performance in the prior art are solved; and meanwhile, the second-generation basic model of the sliding window layered visual converter after mixing fusion is carried out on the basis of pre-training and by using the nodule region generated by the thyroid segmentation model realizes good performance, and high-precision thyroid nodule benign and malignant classification prediction can be realized.

Description

Technical Field

[0001] The present invention relates to the technical field of medical image processing and discrimination, and in particular to a method for predicting benign and malignant thyroid nodules by automatic segmentation combined with semi-supervised learning. Background Art

[0002] Thyroid nodules are common thyroid diseases. With the increasing incidence of thyroid cancer, early detection of nodules has become particularly important. Ultrasound is a non-invasive and efficient diagnostic tool and is widely used in clinical practice. However, the analysis of traditional ultrasound images mainly relies on manual segmentation and classification by doctors, which has problems such as strong subjectivity, low efficiency, and poor repeatability. In addition, the results are easily affected by the doctor's experience and image quality.

[0003] There are many challenges in classifying benign and malignant thyroid nodules based on ultrasound images. Ultrasound images often have problems such as speckle noise and low contrast. At the same time, due to the variability of ultrasound imaging angles and the lack of fixed surrounding anatomical structures as references, compared with imaging methods such as computed tomography and magnetic resonance imaging, directly using global ultrasound images as input is easily interfered by a large amount of noise. Therefore, segmenting the effective nodule area becomes a necessary step to avoid interference from irrelevant areas in subsequent classification.

[0004] Deep learning has been widely used in medical image processing, but the uncertainty and variability in ultrasound images increase the difficulty of segmentation and classification. Existing image segmentation algorithms are mainly trained on natural images, but their accuracy is limited when dealing with thyroid nodule morphological changes and blurred edges. The current image classification algorithm has poor performance when directly transferred to the task of benign and malignant thyroid nodules due to the lack of training and learning for the classification of benign and malignant thyroid nodules. However, due to the cost of acquiring and annotating medical images, it is impossible to perform large-scale training like natural images, which can easily lead to overfitting.

[0005] Currently, ultrasound-based thyroid nodule detection faces problems such as variable imaging angles and lack of fixed anatomical references, resulting in low global image classification accuracy. Existing segmentation algorithms have limited performance when nodules have variable morphology and blurred edges. Classification models lack specific training for the classification of benign and malignant thyroid nodules, and have a high risk of overfitting. In addition, the high cost of medical image annotation and insufficient data limit the performance of deep learning models. Summary of the invention

[0006] The purpose of the present invention is to address the deficiencies in the prior art and to provide a technical solution for a method for predicting benign and malignant thyroid nodules by automatic segmentation and combined with semi-supervised learning. The effective area of ​​the thyroid nodule is first segmented to assist in the subsequent benign and malignant classification task, and the information space is expanded and learned through semi-supervised learning, thereby overcoming the problems of high cost of medical image annotation, insufficient data, and limited performance of deep learning models in the prior art. At the same time, the second-generation basic version model of the sliding window hierarchical visual transformer based on pre-training and hybrid fusion of nodule areas generated by a thyroid segmentation model achieves good performance and can achieve high-precision prediction of benign and malignant classification of thyroid nodules.

[0007] In order to solve the above technical problems, the present invention adopts the following technical solutions: A method for predicting benign or malignant thyroid nodules by automatic segmentation combined with semi-supervised learning, characterized by comprising the following steps: Step 1: Dataset establishment: collect ultrasound images of patients with thyroid nodules, annotate the thyroid nodule area, divide the images into training set, validation set and independent test set for training the thyroid nodule segmentation model and the benign and malignant thyroid nodule classification model; Step 2: Image preprocessing: standardize the standard deviation and mean of the ultrasound image, crop and scale the ultrasound image uniformly, and normalize the real mask annotation of the thyroid nodule area to make it suitable for the segmentation model input; Step 3, thyroid nodule segmentation model training: Based on the segmentation model of arbitrary object segmentation pre-trained on natural images, add an adapter layer, adopt different freezing training strategies, migrate the original model parameters and network structure, and perform transfer learning on the constructed thyroid nodule segmentation ultrasound dataset to obtain a thyroid nodule segmentation model; Step 4: Mask and image fusion processing: Automatically segment the ultrasound image to extract the thyroid nodule area mask, and fuse the thyroid nodule area mask with the ultrasound image through two modes: blending and cropping, to reduce interference from surrounding areas; Step 5: Training of benign and malignant thyroid nodule classification model: Based on the sliding window hierarchical visual transformer image classification model pre-trained on natural images, the model is trained on the mixed and cropped ultrasound images to construct a benign and malignant thyroid nodule classification model; Step 6. Construction of semi-supervised learning model: Use pseudo labels and consistency regularization strategy to construct a semi-supervised learning model for the task of classifying benign and malignant thyroid nodules. Use strong and weak methods to perform data enhancement on labeled samples and unlabeled samples. Compare the outputs of unlabeled samples after the two types of data enhancement. Use the benign and malignant prediction labels of unlabeled samples with weak data enhancement as pseudo labels. Calculate the cross entropy loss between the pseudo labels whose confidence meets the preset threshold and the output of strong data enhancement.

[0008] The above-mentioned method for predicting benign and malignant thyroid nodules first segments the effective area of ​​thyroid nodules to assist in the subsequent benign and malignant classification tasks, and expands and learns the information space through semi-supervised learning, thereby overcoming the problems of high cost of medical image annotation, insufficient data, and limiting the performance of deep learning models in the existing technology. At the same time, the second-generation basic version model of the sliding window hierarchical visual transformer based on the pre-training and hybrid fusion of the nodule area generated by the thyroid segmentation model achieves good performance and can realize high-precision classification prediction of benign and malignant thyroid nodules.

[0009] The present invention has the following beneficial effects due to the adoption of the above technical solution: The present invention first segments out the effective area of ​​the thyroid nodule to assist in the subsequent benign and malignant classification task, and expands and learns the information space through semi-supervised learning, thereby overcoming the problems of high cost of medical image annotation, insufficient data, and limited performance of deep learning models in the prior art. At the same time, the second-generation basic version model of the sliding window hierarchical visual transformer based on pre-training and hybrid fusion of nodule areas generated by the thyroid segmentation model achieves good performance and can realize high-precision prediction of benign and malignant classification of thyroid nodules. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] The present invention will be further described below in conjunction with the accompanying drawings: Figure 1 This is a flow chart of a method for predicting benign and malignant thyroid nodules by automatic segmentation combined with semi-supervised learning according to the present invention; Figure 2 This is an example diagram of the results of the thyroid nodule segmentation model in the present invention. DETAILED DESCRIPTION

[0011] It should be noted that, in the absence of conflict, the embodiments and features in the embodiments of the present application can be combined with each other. The present invention will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0012] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only embodiments of a part of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0013] like Figure 1 As shown in the figure, a method for predicting benign and malignant thyroid nodules by automatic segmentation combined with semi-supervised learning of the present invention comprises the following steps: Step 1: Dataset creation: Ultrasound images of patients with thyroid nodules were collected, the thyroid nodule areas were marked, and the images were divided into training sets, validation sets, and independent test sets for the training of thyroid nodule segmentation models and benign and malignant thyroid nodule classification models.

[0014] Before establishing the data set, first build the labeled queue and the unlabeled queue: There are label queue construction inclusion criteria: ① Patients diagnosed with benign thyroid nodules or thyroid cancer (thyroid malignant tumors) by pathological histology or cytology examination; ② Preoperative thyroid ultrasound images are available, with complete ultrasound image numbers and serial numbers. Ultrasound examination items may include "thyroid ultrasound" and "thyroid, cervical lymph nodes", etc.; ③The ultrasound image is clear, without obvious noise, adverse artifacts, or obvious abnormal contrast; ④The patient or legal representative voluntarily agrees to participate in the study and signs the informed consent form.

[0015] There are exclusion criteria for labeling queue construction: ① Histopathological or cytological examinations cannot clearly identify the benign or malignant pathological type of the patient's thyroid nodules; ② Patients with other primary tumors besides thyroid cancer; ③ Those deemed unsuitable for inclusion by the researchers.

[0016] Unlabeled cohort construction inclusion criteria: ① Patients with thyroid nodules detected by ultrasound examination but without pathological histology or cytology examination; ② Preoperative thyroid ultrasound images are available, with complete ultrasound image numbers and serial numbers. Ultrasound examination items may include "thyroid ultrasound" and "thyroid, cervical lymph nodes", etc.; ③The ultrasound image is clear, without obvious noise, adverse artifacts, or obvious abnormal contrast; ④The patient or legal representative voluntarily agrees to participate in the study and signs the informed consent form.

[0017] Exclusion criteria for the unlabeled cohort: ① Pathological histology or cytology examination has been performed, but the pathological type of the patient's thyroid nodule cannot be determined to be benign or malignant; ② Patients with other primary tumors besides thyroid cancer; ③ Those deemed unsuitable for inclusion by the researchers.

[0018] Dataset establishment includes labeled dataset establishment and unlabeled dataset establishment.

[0019] The labeled dataset is established by collecting ultrasound images of patients with thyroid nodules with pathological benign and malignant basis, annotating the thyroid nodule area and constructing a labeled dataset for nodule segmentation and benign and malignant nodule classification tasks, which is used for training thyroid nodule segmentation models and thyroid nodule benign and malignant nodule classification models.

[0020] The establishment of a labeled dataset includes the following steps: A labeled dataset was constructed based on the labeled cohort, and the corresponding benign and malignant labels of thyroid nodules were annotated on the ultrasound images of each patient. The thyroid nodule region was annotated on the ultrasound images, with 255 as the tumor value and 0 as the background value. This region annotation was used for training and testing the thyroid nodule segmentation model, and was not directly used for the subsequent benign and malignant classification model.

[0021] 20% of the dataset is divided into an independent test set, which does not participate in any training stage of the thyroid nodule segmentation model and is only used for performance testing of the thyroid nodule segmentation model.

[0022] 80% of the remaining data is divided into a training set for model training.

[0023] 20% of the remaining data was divided into a validation set for verifying and screening the best thyroid nodule segmentation model.

[0024] The unlabeled dataset was established by collecting ultrasound images of patients with thyroid nodules who lacked pathological basis for distinguishing benign or malignant nodules, annotating the thyroid nodule area, and using it for training thyroid nodule segmentation models and semi-supervised auxiliary training of thyroid nodule benign or malignant nodule classification models.

[0025] The establishment of unlabeled dataset includes the following steps: An unlabeled dataset was constructed based on the unlabeled cohort. The thyroid nodule region was annotated on the ultrasound image and used for training and testing the thyroid nodule segmentation model. 255 was used as the tumor value and 0 was used as the background value. This region annotation was only used for training and testing the thyroid nodule segmentation model and was not directly used for the subsequent benign and malignant classification model. All data were used for semi-supervised training of the benign and malignant thyroid nodule classification model.

[0026] Step 2: Image preprocessing: The standard deviation and mean of ultrasound images were calculated and standardized using the normalization method of the visual image transformation library.

[0027] The image is scaled to 256*256 pixels using the resize method of the Visual Image Transformation Library. The image is center-cropped to 224*224 pixels using the random crop method of the Visual Image Transformation Library.

[0028] The true mask annotation of the thyroid nodule area is normalized to make it suitable for segmentation model input.

[0029] Step 3: Thyroid nodule segmentation model training: Based on the segmentation model of arbitrary objects pre-trained on natural images, an adapter layer is added to the key multi-head attention module, and different frozen training strategies are adopted to migrate the original model parameters and network structure. Transfer learning is performed on the constructed thyroid nodule segmentation ultrasound dataset to obtain a thyroid nodule segmentation model.

[0030] The thyroid nodule segmentation ultrasound dataset is a dataset formed by calculating the standard deviation and mean of ultrasound images in image preprocessing, normalizing, cropping and scaling.

[0031] The thyroid nodule segmentation model training uses an early stopping mechanism to select the model with the best performance in the validation set, uses the stochastic gradient descent method as the optimizer, and uses the cosine annealing method for processing. The segmentation model of the arbitrary object segmentation model is the basic architecture model of the thyroid nodule segmentation model.

[0032] The performance indicators of the thyroid nodule segmentation model include the intersection over union (IOU) and the similarity coefficient (Dice); ; ; Among them: TP – true positive; FP – False Positive; TN – true counterexample; FN – False Negative Example.

[0033] The results of the thyroid nodule segmentation model are shown in Table 1 and Figure 2 As shown: Table 1 ;

[0034] The results in Table 1 show that fine-tuning the segmentation model using the segmentation arbitrary object model has better performance than the dense convolutional segmentation network trained directly from scratch, and can achieve good automatic segmentation of thyroid nodules.

[0035] Figure 2 Taking the result examples of different models, the results show that using the model for segmenting arbitrary objects for fine-tuning can more accurately segment the thyroid nodule area than the dense convolutional segmentation network trained directly from scratch, and can realize the automatic segmentation of the thyroid nodule area.

[0036] Step 4: Mask and image fusion processing: The ultrasound image is automatically segmented to extract the thyroid nodule region mask, and the thyroid nodule segmentation mask prediction is generated for all images in the labeled and unlabeled datasets. The channel value of the nodule region is 255, and the channel value of the non-nodule region is 0.

[0037] The thyroid nodule region mask is fused with the ultrasound image through the blending and cropping modes to reduce the interference of the surrounding areas.

[0038] The hybrid mode uses the hybrid processing of the visual image transformation library to process the mask and the ultrasound image. The cropping mode multiplies the mask and the ultrasound image point by point, that is, the channel values ​​of the fused image in the non-nodule area are all 0 to focus solely on the nodule area features.

[0039] Step 5: Training of benign and malignant thyroid nodule classification model: Based on the sliding window hierarchical visual transformer image classification model pre-trained on natural images, the natural images can be used as the image classification benchmark dataset, and the input size is set to 224*224 pixels. The model for classifying benign and malignant thyroid nodules is constructed by training on ultrasound images after mixing and cropping.

[0040] Based on the images and labels processed by mask fusion of the labeled dataset, the original model parameters and network structure are transferred, and different freezing strategies are used for transfer learning.

[0041] The training of the benign and malignant thyroid nodule classification model uses an early stopping mechanism to select the best model in the validation set, uses the stochastic gradient descent method as the optimizer, and uses the cosine annealing method for processing. The sliding window hierarchical visual transformer image classification model is the basic architecture model of the benign and malignant thyroid nodule classification model.

[0042] The performance indicators of the benign and malignant thyroid nodule classification model include accuracy ACC and the area under the receiver operating characteristic curve AUC below the receiver operating characteristic curve ROC. The AUC is obtained by calculating the area under the ROC curve. The horizontal axis of the ROC curve is FPR (False Positive Rate) and the vertical axis is TPR (True Positive Rate); ; ; ; Among them: TP – true positive; FP – False Positive; TN – true counterexample; FN – False Negative Example.

[0043] The results of the benign and malignant classification model of thyroid nodules are shown in Table 2: Table 2 ;

[0044] The overall performance of the models under different methods for the classification and prediction of benign and malignant thyroid nodules was compared; the results showed that the second-generation basic version model of the sliding window hierarchical visual transformer, which was based on pre-training and hybrid fusion of nodule regions generated by the thyroid segmentation model, achieved good performance and could realize high-precision classification and prediction of benign and malignant thyroid nodules.

[0045] Step 6: Semi-supervised learning model construction: A semi-supervised learning model for the task of classifying benign and malignant thyroid nodules was constructed using pseudo labels and consistency regularization strategies. Data enhancement was performed on labeled and unlabeled samples in strong and weak ways. The outputs of unlabeled samples after the two types of data enhancement were compared. The benign and malignant prediction labels of unlabeled samples with weak data enhancement were used as pseudo labels, and the cross entropy loss was calculated between the pseudo labels whose confidence met the preset threshold and the output of strong data enhancement.

[0046] In the construction of the semi-supervised learning model, the images of the labeled data set after mask fusion processing and the images of the labeled and unlabeled data sets after mask fusion processing respectively constitute the labeled data set and unlabeled data set for training the semi-supervised learning model.

[0047] When performing data augmentation on labeled and unlabeled samples: Weak data augmentation is performed on labeled samples, including random rotation, random cropping, random horizontal flipping, and random translation; Strong and weak data enhancement are performed on unlabeled samples at the same time. Weak data enhancement is the same as labeled samples. Strong data enhancement uses a random enhancement method and is combined with random occlusion for enhancement.

[0048] First, a weak data enhancement strategy is applied to predict the unlabeled samples to obtain the predicted category with the highest probability and its probability value. The predicted category with a probability value exceeding the preset threshold is determined as an available pseudo-label to enter the model training.

[0049] Strong data enhancement for unlabeled samples specifically includes: cross entropy loss calculation with the pseudo-label of the same image to obtain loss Loss ulb , will monitor some of the losses lb And the unsupervised part loss Loss ulb Weighted addition to get the final loss Loss total , Loss total = Loss lb +λLoss ulb , where λ is a hyperparameter set artificially.

[0050] The above-mentioned method for predicting benign and malignant thyroid nodules first segments the effective area of ​​thyroid nodules to assist in the subsequent benign and malignant classification tasks, and expands and learns the information space through semi-supervised learning, thereby overcoming the problems of high cost of medical image annotation, insufficient data, and limiting the performance of deep learning models in the existing technology. At the same time, the second-generation basic version model of the sliding window hierarchical visual transformer based on the pre-training and hybrid fusion of the nodule area generated by the thyroid segmentation model achieves good performance and can realize high-precision classification prediction of benign and malignant thyroid nodules.

[0051] The above are only specific embodiments of the present invention, but the technical features of the present invention are not limited thereto. Any simple changes, equivalent replacements or modifications made based on the present invention to achieve basically the same technical effects are all included in the protection scope of the present invention.

Claims

1. A method for predicting benign and malignant thyroid nodules by automatic segmentation combined with semi-supervised learning, characterized in that The steps include: Step 1: Dataset establishment: collect ultrasound images of patients with thyroid nodules, annotate the thyroid nodule area, divide the images into training set, validation set and independent test set for training the thyroid nodule segmentation model and the benign and malignant thyroid nodule classification model; Step 2, image preprocessing: standardize the calculation of the standard deviation and mean of the ultrasound image, uniformly crop and scale the ultrasound image, and normalize the real mask annotation of the thyroid nodule area; Step 3, thyroid nodule segmentation model training: Based on the segmentation model of arbitrary object segmentation pre-trained on natural images, add an adapter layer, adopt different freezing training strategies, migrate the original model parameters and network structure, and perform transfer learning on the constructed thyroid nodule segmentation ultrasound dataset to obtain a thyroid nodule segmentation model; Step 4, mask and image fusion processing: automatically segmenting the ultrasound image to extract the thyroid nodule region mask, and fusing the thyroid nodule region mask with the ultrasound image through two modes of blending and cropping; Step 5, training of a benign and malignant thyroid nodule classification model: based on a sliding window hierarchical visual transformer image classification model pre-trained on natural images, training is performed on the ultrasound images after mixing and cropping to construct a benign and malignant thyroid nodule classification model; Step 6. Construction of semi-supervised learning model: Use pseudo labels and consistency regularization strategy to construct a semi-supervised learning model for the task of classifying benign and malignant thyroid nodules. Use strong and weak methods to perform data enhancement on labeled samples and unlabeled samples. Compare the outputs of unlabeled samples after the two types of data enhancement. Use the benign and malignant prediction labels of unlabeled samples with weak data enhancement as pseudo labels. Calculate the cross entropy loss between the pseudo labels whose confidence meets the preset threshold and the output of strong data enhancement.

2. The method for predicting benign or malignant thyroid nodules by automatic segmentation combined with semi-supervised learning according to claim 1, characterized in that: The data set establishment in step 1 includes establishment of a labeled data set and establishment of an unlabeled data set; The labeled data set is established by collecting ultrasound images of patients with thyroid nodules with pathological benign and malignant discrimination basis, marking the thyroid nodule area and constructing a labeled data set for nodule segmentation and benign and malignant nodule classification tasks, which is used for training thyroid nodule segmentation models and thyroid nodule benign and malignant nodule classification models; The unlabeled dataset is established by collecting ultrasound images of patients with thyroid nodules that lack pathological basis for distinguishing benign and malignant thyroid nodules, annotating the thyroid nodule area, and using it for training a thyroid nodule segmentation model and semi-supervised auxiliary training of a thyroid nodule benign and malignant thyroid nodule classification model.

3. The method for predicting benign or malignant thyroid nodules by automatic segmentation combined with semi-supervised learning according to claim 2, characterized in that: The establishment of the labeled data set specifically includes the following steps: A labeled dataset was constructed based on the labeled cohort, and the corresponding benign or malignant label of the thyroid nodule was annotated on the ultrasound image of each patient. The thyroid nodule area was annotated on the ultrasound image, and this area was used for training and testing the thyroid nodule segmentation model. The independent test set in the dataset was used for performance testing of the thyroid nodule segmentation model, the training set in the remaining data was used for model training, and the validation set in the remaining data was used to validate and screen the thyroid nodule segmentation model.

4. The method for predicting benign or malignant thyroid nodules by automatic segmentation combined with semi-supervised learning according to claim 2, characterized in that: The establishment of the unlabeled dataset specifically includes the following steps: An unlabeled dataset was constructed based on the unlabeled cohort. The thyroid nodule area was annotated on the ultrasound image and used for training and testing the thyroid nodule segmentation model. All the data was used for semi-supervised training of the benign and malignant thyroid nodule classification model.

5. The method for predicting benign or malignant thyroid nodules by automatic segmentation combined with semi-supervised learning according to claim 1, characterized in that: The thyroid nodule segmentation model training and the thyroid nodule benign and malignant classification model training both use an early stopping mechanism to select a validation set model, use a stochastic gradient descent method as an optimizer, and use a cosine annealing method for processing.

6. The method for predicting benign or malignant thyroid nodules by automatic segmentation combined with semi-supervised learning according to claim 5, characterized in that: The performance indicators of the thyroid nodule segmentation model include intersection over union (IOU) and similarity coefficient (Dice); ； ； Among them: TP – true positive; FP – False Positive; TN – true counterexample; FN – False Negative Example.

7. The method for predicting benign or malignant thyroid nodules by automatic segmentation combined with semi-supervised learning according to claim 5, characterized in that: The performance indicators of the benign and malignant thyroid nodule classification model include accuracy ACC and area under the receiver operating characteristic curve AUC below the receiver operating characteristic curve ROC, where AUC is obtained by calculating the area under the ROC curve, and the abscissa of the ROC curve is FPR and the ordinate is TPR; ； ； ； Among them: TP – true positive; FP – False Positive; TN – true counterexample; FN – False Negative Example.

8. The method for predicting benign or malignant thyroid nodules by automatic segmentation combined with semi-supervised learning according to claim 1, characterized in that: In step 6, in the construction of the semi-supervised learning model, the images of the labeled data set after mask fusion processing and the images of the labeled and unlabeled data sets after mask fusion processing respectively constitute the labeled data set and unlabeled data set for semi-supervised learning model training.

9. The method for predicting benign or malignant thyroid nodules by automatic segmentation combined with semi-supervised learning according to claim 8, characterized in that: When performing data augmentation on labeled and unlabeled samples in step 6: Weak data augmentation is performed on labeled samples, including random rotation, random cropping, random horizontal flipping, and random translation; Strong and weak data enhancement are performed on unlabeled samples at the same time. Weak data enhancement is the same as labeled samples. Strong data enhancement uses a random enhancement method and is combined with random occlusion for enhancement.

10. The method for predicting benign or malignant thyroid nodules by automatic segmentation combined with semi-supervised learning according to claim 9, characterized in that: Strong data enhancement for unlabeled samples specifically includes: cross entropy loss calculation with the pseudo-label of the same image to obtain loss Loss ulb , will monitor some of the losses lb And the unsupervised part loss Loss ulb Weighted addition to get the final loss Loss total , Loss total = Loss lb +λLoss ulb , where λ is a hyperparameter set artificially.

Images