A semi-supervised training method for dual-view interactive learning based on information bottleneck and contrastive learning

Abstract

The present invention provides a semi-supervised training method for dual-view interactive learning based on information bottleneck and contrastive learning. In the process of dual-view interactive learning, the data stream is divided into labeled data and unlabeled data. For the labeled data, the two segmentation networks in the dual-view interactive learning are optimized by calculating the cross-entropy loss and Dice loss between the predicted value and the true label. For the unlabeled data, the two segmentation networks generate pseudo-labels for mutual supervision, and the information bottleneck theory is used to strengthen the consistency between the feature layer and the label (or pseudo-label) in the segmentation network and weaken its consistency with the input value. In addition, the class center contrastive learning method is applied during the training process. First, the center features of each class are constructed, then a certain pixel feature is compared with the center features of all classes, and finally the pixel feature is classified into the class with the most similar center feature. Through the solution of the present invention, the performance and accuracy of semi-supervised segmentation can be significantly improved.

Description

Technical Field

[0001] The present invention relates to the technical fields of computer vision and image processing, and in particular to a semi-supervised training method for dual-view interactive learning based on information bottleneck and contrast learning. Background Art

[0002] Digestive tract diseases are a very common type of disease, such as gastric cancer, esophageal cancer, etc. These diseases seriously threaten the physical and mental health of patients. Early detection and treatment can significantly reduce the mortality rate of this disease. Currently, clinicians mainly rely on internists to identify digestive tract diseases through endoscopic imaging. This method highly depends on doctors' experience and visual perception, and it is very easy for inexperienced doctors to misdiagnose or miss a diagnosis. Computer-aided diagnosis methods based on deep learning can effectively alleviate this problem. Currently, many deep learning models have been developed for automatic segmentation of digestive tract diseases and have achieved high accuracy. However, deep learning methods highly depend on labeled data sets. Manually labeling complex digestive tract disease data by doctors is time-consuming and laborious, and not all of the increasing medical data can be manually labeled. Semi-supervised training techniques can effectively utilize unlabeled images to improve the accuracy of the segmentation model. However, the existing semi-supervised methods still have the following problems when applied to the auxiliary diagnosis of digestive tract diseases:

[0003] When the existing parallel network-based method performs semi-supervised training on digestive tract disease images, there is an easy coupling effect between parallel networks, which limits the performance of the model;

[0004] The edge parts of the lesion areas of some digestive tract diseases are very similar to the normal areas. Such fuzzy boundaries are extremely likely to cause the training model to segment incorrectly;

[0005] The lesion areas of some digestive tract diseases are very small, making it difficult for the model to effectively capture them, thereby affecting the segmentation accuracy. Summary of the Invention

[0006] In view of the above problems, the present invention proposes a semi-supervised training method for dual-view interactive learning based on information bottleneck and contrast learning, which can significantly improve the performance and accuracy of semi-supervised segmentation by introducing dual-view interactive learning, information bottleneck theory, and class center contrast learning.

[0007] A semi-supervised training method for dual-view interactive learning based on information bottleneck and contrast learning proposed by the present invention includes the following steps:

[0008] Step S1: Collect medical image data, and then preprocess the image data. The preprocessing includes data augmentation and view transformation;

[0009] Step S2: Construct a semi-supervised training model based on dual-view interactive learning, information bottleneck theory, and class center contrast learning. The segmentation model in the dual-view interactive learning consists of Network A and Network B. The U-Net is used as the segmentation network for both Network A and Network B. The collected RGB images are input into Network A, and the corresponding HSL images of the RGB images are input into Network B. In the parallel network composed of Network A and Network B, dual-view interactive learning is used to avoid the coupling effect of the parallel network. The information bottleneck theory and class center contrast learning are used to strengthen the perception and discrimination ability of the network;

[0010] Step S3: Train the segmentation model and train the preprocessed dataset through the semi-supervised training model. During the training process, parameter updates are performed through forward propagation and backward propagation, and the training is stopped when the performance of the network reaches the optimal level, so as to obtain the weight parameters with the best segmentation effect.

[0011] Further, the specific steps of Step S1 include: dividing the collected image data into labeled data and unlabeled data, then using the color space conversion algorithm to convert the collected RGB images into the corresponding HSL images. The labeled RGB images are denoted as , the labeled HSL images are denoted as , the unlabeled RGB images are denoted as , the unlabeled HSL images are denoted as , the total dataset is , where , , and are the image feature values input into the network, is the label value, and represent the height and width of the image respectively.

[0012] Further, the data augmentation methods in Step S1 include horizontal flipping, vertical flipping, and random scaling.

[0013] Further, the dual-view interactive learning in Step S2 specifically includes:

[0014] Adopt a parallel network framework composed of two segmentation networks A and B with the same structure. The segmentation network A is denoted as , the segmentation network B is denoted as , Train the collected RGB images, Train the corresponding HSL images. During the training process, for the labeled images, the two segmentation networks are optimized by calculating the cross-entropy loss and Dice loss between the predicted values and the ground truth values. For the unlabeled images, the two segmentation networks are mutually supervised and optimized through pseudo-labels, thus avoiding the coupling caused by training a single view in parallel networks and achieving complementary advantages of lesion features in different views, and improving the segmentation accuracy.

[0015] Furthermore, use to train the labeled RGB images , and use the combination of cross-entropy loss and Dice loss as the supervision loss for training , expressed as:

[0016] ;

[0017] where is the supervision loss for RGB labeled data, is the number of labeled RGB images, is the cross-entropy loss, is the Dice loss, The calculation method is as follows:

[0018] ;

[0019] The calculation method is as follows:

[0020] ;

[0021] where and respectively represent the label value and predicted probability of the input image. The label value is obtained through manual annotation, and the predicted probability is obtained by training, and respectively represent the height and width of the image, and respectively represent the label value and predicted probability of the th pixel in the image.

[0022] Furthermore, use to train the labeled HSL images. Similarly, use the combination of cross-entropy loss and Dice loss as the supervision loss for training the labeled HSL images, expressed as:

[0023] ;

[0024] where is the supervision loss for HSL labeled data, is the number of labeled HSL images.

[0025] Further, use to train the unlabeled RGB image data to obtain the corresponding predicted probability values , use to train the unlabeled HSL image data to obtain the corresponding predicted probability values , then, through the network Generate a pseudo-label supervised network , the calculation method of the loss function is as follows:

[0026] ;

[0027] Among them, is the number of unlabeled RGB images, is generated by The pseudo-label, similarly, through the network Generate a pseudo-label supervised network , the calculation method of the loss function is as follows:

[0028] ;

[0029] Among them, is the number of unlabeled HSL images, is generated by The pseudo-label.

[0030] Adding the above losses gives the total loss function of the dual-view interactive learning, expressed as:

[0031] ;

[0032] By using the dual-view interactive learning strategy, the two segmentation networks supervise each other, avoid coupling, and thus improve the segmentation ability of the semi-supervised model for the digestive tract injury area.

[0033] Further, the information bottleneck theory in the step S2 specifically includes:

[0034] The Hilbert-Schmidt Independence Criterion (HSIC) is used to define the information bottleneck theory, and HSIC is:

[0035] ;

[0036] Among them, , , is the identity matrix, is a column vector with all elements being 1, Represents a variable and the number of rows, represents the trace of the matrix, calculated it is necessary to ensure and have the same number of rows, is the kernel matrix, and the calculation method is as follows:

[0037] ;

[0038] where, and are two different variables in is an adjustable parameter, represents the 2-norm of the vector, also called the Euclidean norm, is defined in a similar way to :

[0039] ;

[0040] where, and are two different variables in

[0041] Based on the above definition of the information bottleneck, the information bottleneck loss of the RGB image labeled in

[0042] ;

[0043] where, is the labeled RGB image, is the corresponding label value, is the matrix of the feature layer (including convolutional layer and deconvolutional layer) during the training is the number of feature layers, the information bottleneck loss of the unlabeled RGB image in

[0044] ;

[0045] where, is the unlabeled RGB image, is the pseudo-label value generated by is the matrix of the feature layer during the training

[0046] Similarly, we can also obtain The information bottleneck loss of the labeled HSL image is:

[0047] ;

[0048] where is the labeled HSL image, is the corresponding label value, is the feature layer matrix during the training process.

[0049] The information bottleneck loss of the unlabeled HSL image in

[0050] ;

[0051] where is the unlabeled HSL image, is the pseudo-label generated, is the feature layer matrix during the training process.

[0052] Thus, the total information bottleneck loss is obtained as:

[0053] .

[0054] Furthermore, the class center contrast learning in step S2 specifically includes:

[0055] Construct a class center contrast learning method, calculate the center feature of each class, compare the pixel feature with the center features of all classes, and classify the pixel feature into the class with the most similar center feature according to the cosine similarity. The positive sample pair of the class center contrast learning proposed by the present invention is the contrast between the pixel feature and its corresponding class center, and the negative sample pair is the contrast between the pixel feature and the class centers of other classes.

[0056] The calculation method of the class center feature is:

[0057] The predicted probability value of the labeled RGB data is , and the feature vector at the position in is denoted as , and the feature vector at the position in the label is denoted as , and the feature vector at the position in the pseudo-label generated by is denoted as , where is the feature dimension, and the central feature of each category calculated based on RGB annotation data is:

[0058] ;

[0059] where is the central feature of category in the RGB annotation image, is the central feature of category in the RGB annotation image. The category includes two categories: the lesion area and the normal area. According to this formula, the central feature of the positive sample category (denoted as ) and the central feature of the negative sample category (denoted as ) are calculated. The cosine similarity is used to measure the similarity between the pixel feature and :

[0060] ;

[0061] || represents the norm of the vector. Similarly, the cosine similarity is used to measure the similarity between the pixel feature and :

[0062] ;

[0063] Then, based on the InfoNCE loss, the contrastive learning loss of the category center of the annotated RGB image is constructed during training :

[0064] ;

[0065] where is the number of pixels in , is the negative sample set, is the temperature parameter, which is used to adjust the scale of the similarity in contrastive learning. In the present invention, is uniformly set to 0.1.

[0066] Similarly, the central feature of each category calculated based on the RGB unannotated image is:

[0067] ;

[0068] where is the central feature of category in the RGB unannotated image, is the number of pixels in;

[0069] Training The category-centered contrastive learning loss of unlabeled RGB images is:

[0070] ;

[0071] in, yes The number of pixels in for Training on unlabeled data The resulting predicted probability distribution Middle position The characteristic vector of yes The corresponding positive sample is yes The corresponding negative samples are is a set of negative samples. Similarly, we get Network training class center contrast loss of labeled HSL data Compared with the category center loss of training unlabeled HSL data , the total category-centered contrastive learning loss is:

[0072] .

[0073] The dual-view interactive learning loss, information bottleneck loss and category center contrastive learning loss are added together to obtain the total loss function of the dual-view interactive learning semi-supervised training method based on information bottleneck and contrastive learning (denoted as DSDNet) proposed in the present invention:

[0074] ;

[0075] in, and It is the trade-off coefficient in the total loss function. Through the trade-off coefficient, the model performance can be further optimized.

[0076] The present invention proposes a semi-supervised training method for dual-view interactive learning based on information bottleneck and contrast learning. The method mainly includes three parts: dual-view interactive learning, information bottleneck theory, and class center contrast learning. The collected RGB images of digestive tract diseases are converted into the HSL view, and then the collected RGB view images and the converted HSL view are respectively used as the inputs of two parallel networks in the dual-view interactive learning. And during the dual-view interactive learning process, the data flow can be divided into a labeled path and an unlabeled path. For labeled data, the two segmentation networks are optimized by calculating the cross-entropy loss and Dice loss between the predicted value and the true label. For unlabeled data, the two segmentation networks generate pseudo-labels to supervise each other for optimization, which can effectively alleviate the coupling problem caused by training a single view in the parallel network, improve the learning ability of the parallel network, enable the model to learn richer semantic information, and improve the semi-supervised segmentation accuracy; The information bottleneck theory is used for shape perception. The information bottleneck theory is used to strengthen the consistency between the feature layers (convolutional layer and deconvolutional layer) and the label (or pseudo-label) in the segmentation network, and weaken the consistency between the feature layer and the input value. It is similar to a filter that filters out redundant information in the feature layer and only retains key information. By introducing the information bottleneck theory, the neural network is guided to perceive the shape of the lesion, accurately distinguish the lesion area from the normal area, and effectively solve the problem of incorrect segmentation caused by blurred lesion boundaries in digestive tract disease images; In addition, the present invention applies the class center contrast learning method during the training process. First, the center features of each class are constructed. Secondly, a certain pixel feature is compared with the center features of all classes. Finally, the pixel feature is classified into the class with the most similar center feature. By comparing the pixel feature with the class center feature, the intra-class compactness and inter-class separability are improved, effectively solving the problem of difficult detection of small lesions and enhancing the segmentation ability of the model for small lesions. BRIEF DESCRIPTION OF THE DRAWINGS

[0077] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for the description of the embodiments or the prior art. Obviously, the drawings in the following description are some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0078] Figure 1 It is a schematic flowchart of a semi-supervised training method for dual-view interactive learning based on information bottleneck and contrast learning provided by an embodiment of the present invention;

[0079] Figure 2 It is a schematic diagram of the principle of dual-view interactive learning provided by an embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0080] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0081] The present invention proposes a semi-supervised training method for dual-view interactive learning based on information bottleneck and contrast learning, as Figure 1 shown, the method includes the following steps:

[0082] Step S1: Collect medical image data, and then preprocess the image data. The preprocessing includes data augmentation and view transformation;

[0083] Step S2: Construct a semi-supervised training model based on dual-view interactive learning, information bottleneck theory, and class center contrast learning. The segmentation model in the dual-view interactive learning consists of Network A and Network B. The U-Net is used as the segmentation network for Network A and Network B. Input the RGB images of digestive tract diseases collected by the endoscope into Network A, and input the corresponding HSL images of the RGB images into Network B. In the parallel network composed of Network A and Network B, dual-view interactive learning is used to avoid the coupling effect of the parallel network. The information bottleneck theory and class center contrast learning are used to strengthen the perception and discrimination ability of the network;

[0084] Step S3: Train the segmentation model, and train the preprocessed data set through the semi-supervised training model. During the training process, parameter updates are performed through forward propagation and backward propagation, and the training is stopped when the performance of the network reaches the optimal, so as to obtain the weight parameters with the best segmentation effect.

[0085] The specific content of step S1 includes: dividing the collected image data into labeled data and unlabeled data, and then using the color space conversion algorithm to convert the collected RGB images into the corresponding HSL images. Denote the labeled RGB images as , the labeled HSL images as , the unlabeled RGB images as , the unlabeled HSL images as , and the total data set as , where , , and are the image feature values input to the network, is the label value, and respectively represent the height and width of the image.

[0086] The data augmentation method in step S1 includes horizontal flipping, vertical flipping, and random scaling.

[0087] As Figure 2 shown, the dual-view interactive learning in step S2 specifically includes:

[0088] A parallel network framework is composed of two segmentation networks A and B with the same structure. The segmentation network A is denoted as , and the segmentation network B is denoted as , Train the endoscope RGB images collected clinically, Train the corresponding HSL images. During the training process, for the labeled images, the two segmentation networks are optimized by calculating the cross-entropy loss and Dice loss between the predicted value and the true value. For the unlabeled images, the two segmentation networks are mutually supervised and optimized through pseudo-labels, so as to avoid the coupling caused by training a single view in the parallel network, and at the same time achieve the complementary advantages of lesion features in different views and improve the segmentation accuracy.

[0089] Furthermore, use to train the labeled RGB images of digestive tract diseases , and use the combination of cross-entropy loss and Dice loss as the supervision loss for training, expressed as:

[0090] ;

[0091] where is the supervision loss for RGB labeled data, is the number of labeled RGB images, is the cross-entropy loss, is the Dice loss, The calculation method is as follows:

[0092] ;

[0093] The calculation method is as follows:

[0094] ;

[0095] where and respectively represent the label value and predicted probability of the input image. The label value is obtained by manual annotation, and the predicted probability is obtained by training, and respectively represent the height and width of the image, and respectively represent the label value and predicted probability of the th pixel in the image.

[0096] Furthermore, use the labeled HSL images for training, and also use the combination of cross-entropy loss and Dice loss as the supervision loss for training the labeled HSL images, expressed as:

[0097] ;

[0098] where is the supervision loss for the HSL annotation data, is the number of labeled HSL images.

[0099] Furthermore, use to train the unlabeled RGB image data to obtain the corresponding predicted probability values , use to train the unlabeled HSL image data to obtain the corresponding predicted probability values , then, through the network generate the pseudo-label supervision network , and the calculation method of the loss function is as follows:

[0100] ;

[0101] where is the number of unlabeled RGB images, is the pseudo-label generated by , similarly, through the network generate the pseudo-label supervision network , and the calculation method of the loss function is as follows:

[0102] ;

[0103] where is the number of unlabeled HSL images, is the pseudo-label generated by .

[0104] Adding the above losses gives the total loss function of the dual-view interactive learning, expressed as:

[0105] ;

[0106] By using the dual-view interactive learning strategy, the two segmentation networks supervise each other, avoid coupling, and thus improve the segmentation ability of the semi-supervised model for the digestive tract injury area.

[0107] Furthermore, the information bottleneck theory in step S2 specifically includes:

[0108] The Information Bottleneck (IB) theory is a concept in information theory. The core idea of this theory is that the primary task of most learning models is to extract label-related information from raw data and remove label-unrelated information. The IB principle is as follows:

[0109] ;

[0110] Among them, represents the mutual information between two variables, represents the input value of the network, represents the label-related information extracted from In the present invention, represents and the convolutional layer and the deconvolutional layer of represents the label value, is the adjustable parameter in

[0111] Considering that the calculation of is very complex, the present invention selects a more convenient metric, namely the Hilbert-Schmidt Independence Criterion (HSIC). The definition of HSIC is as follows:

[0112] ;

[0113] Among them, , , is the identity matrix, is the column vector with all elements being 1, represents the variable and the number of rows of represents the trace of the matrix. Calculating requires ensuring that and have the same number of rows, is the kernel matrix, and the calculation method is as follows:

[0114] ;

[0115] Among them, and are two different variables in is the adjustable parameter, The 2-norm of a representative vector, also known as the Euclidean norm, is defined similarly to :

[0116] ;

[0117] where and are two different variables in

[0118] Based on the above definition of the information bottleneck, the information bottleneck loss of the RGB image labeled in

[0119] ;

[0120] where is the labeled RGB image, is the corresponding label value, is the matrix of the feature layers (convolutional layer and deconvolutional layer) during training is the number of feature layers, the information bottleneck loss of the unlabeled RGB image in

[0121] ;

[0122] where is the unlabeled RGB image, is the pseudo-label value generated by is the matrix of the feature layers during training

[0123] Similarly, the information bottleneck loss of the labeled HSL image in can also be obtained as:

[0124] ;

[0125] where is the labeled HSL image, is the corresponding label value, is the matrix of the feature layers during training

[0126] the information bottleneck loss of the unlabeled HSL image in

[0127] ;

[0128] Among them, is the unlabeled HSL image, is the generated pseudo-label, is the feature layer matrix during training.

[0129] Thus, the total loss of the information bottleneck is:

[0130] .

[0131] Furthermore, the class center contrast learning in step S2 specifically includes:

[0132] Construct a class center contrast learning method, calculate the center features of each class, compare the pixel features with the center features of all classes, and classify the pixel features into the class with the center feature that is most similar to it according to the cosine similarity. In this way, the intra-class compactness and inter-class separability of the damage classes are improved, and the model's ability to identify minute lesions is enhanced. The positive sample pairs of the class center contrast learning proposed in the present invention are the comparison between the pixel features and their corresponding class centers, and the negative sample pairs are the comparison between the pixel features and the class centers of other classes.

[0133] The calculation method of the class center features is as follows:

[0134] The predicted probability value for the labeled RGB data is , and the feature vector at the position in is denoted as , and the feature vector at the position in the label is denoted as , and the feature vector at the position in the generated pseudo-label

[0135] ;

[0136] Among them, is the class center feature of class in the RGB labeled image, is the number of pixels of class in the RGB labeled image. The classes include two classes: the lesion area and the normal area. According to this formula, the positive sample class center feature of is calculated (denoted as ), and the negative sample class center feature (denoted as ), use cosine similarity to measure the similarity between pixel feature and :

[0137] ;

[0138] || represents the norm of the vector. Similarly, use cosine similarity to measure the similarity between pixel feature and :

[0139] ;

[0140] Then, based on the InfoNCE loss, construct the contrastive learning loss of the labeled RGB image category center during training :

[0141] ;

[0142] where is the number of pixels in is the negative sample set, is the temperature parameter, which is used to adjust the scale of similarity in contrastive learning. In the present invention, is uniformly set to 0.1.

[0143] Similarly, for each category center feature calculated from the RGB unlabeled image:

[0144] ;

[0145] where is the category center feature of category in the RGB unlabeled image, is the number of pixels in;

[0146] The contrastive learning loss of the unlabeled RGB image category center during training is:

[0147] ;

[0148] where is the number of pixels in, is the predicted probability distribution generated by training the unlabeled data at position in the feature vector, is The corresponding positive samples are the corresponding negative samples is the negative sample set. Similarly, the class center contrast loss of the network training labeled HSL data and the class center contrast loss of the training unlabeled HSL data , and the total class center contrast learning loss is:

[0149] .

[0150] Adding the dual-view interaction learning loss, the information bottleneck loss, and the class center contrast learning loss, we obtain the total loss function of the dual-view interaction learning semi-supervised training method (denoted as DSDNet) based on information bottleneck and contrast learning proposed by the present invention:

[0151] ;

[0152] wherein and are the trade-off coefficients in the total loss function. Through the trade-off coefficients, the model performance can be further optimized.

[0153] The following conducts experimental verification on the method of the present invention through specific data:

[0154] Step 1: Construction of the digestive tract disease dataset. Use the private early esophageal cancer and the public colon polyp datasets as the data required for training the model. Label 10% of the data and leave the rest unlabeled.

[0155] Step 2: Preprocessing of the dataset. To improve the robustness of the model and avoid overfitting from affecting the model performance, the present invention uses data augmentation methods such as horizontal flipping, vertical flipping, and random scaling in training the model.

[0156] Step 3: Construction of the dual-view interaction learning semi-supervised training method based on information bottleneck and contrast learning. Adopt U-Net as the segmentation network for Network A and Network B. Introduce the information bottleneck theory and class center contrast learning into the parallel network composed of Network A and B to enhance the perception and discrimination capabilities of the network, and then avoid the coupling effect of the parallel network through dual-view interaction learning.

[0157] Step 4: Train the segmentation model. Train the preprocessed dataset through the dual-view interaction learning semi-supervised training method based on information bottleneck and contrast learning. During the training process, update the parameters through forward propagation and backward propagation, and stop training when the performance of the network reaches the optimal. In this way, the weight parameters with the best segmentation effect are obtained.

[0158] Use the private early esophageal cancer and public colon polyp datasets as the data required for training the model. The early esophageal cancer (EEC) dataset is a collection of early esophageal cancer images taken from a large well-known hospital, with a total of 2689 images. In this embodiment, 2189 images are randomly selected as the training set, 500 images are used as the test set, and 10% of the data is labeled, while the rest is unlabeled. The Kvasir-SEG (KS) dataset is a publicly available endoscopic dataset for pixel-level segmentation of colon polyps, consisting of 1000 lesion images and their corresponding labeled images. In this embodiment, 800 images are randomly selected as the training set, and 200 images are used as the test set.

[0159] The experiments in this embodiment are implemented using an NVIDIA A100 GPU based on the Pytorch deep learning framework. The input data is uniformly cropped to a size of 256×256 pixels. During the training process, Stochastic Gradient Descent (SGD) with a momentum of 0.9 and a weight decay of 0.0001 is used as the optimizer, and the initial learning rate is 0.03. The batch size of the training data in the experiment is set to 16, including 8 labeled data and 8 unlabeled data. The parameters in the present invention take the values of = 1.5, = 2.5, = 0.3, = 0.7. On the KS dataset, the values are = 2.4, = 3.6, = 0.5, = 0.3. In the test phase, the predicted values of Network A are used as the overall predicted values of the model. For fair comparison, all experiments are conducted using the same experimental settings.

[0160] The performance of the model is evaluated using metrics such as the Dice coefficient, Intersection over Union (IoU), Accuracy (Acc), Mean Absolute Error (MAE), and 95% Hausdorff Distance (95HD). Among them, the higher the values of the Dice, IoU, and Acc metrics, the better the model performance, and the lower the values of the MAE and 95HD metrics, the better the model performance.

[0161] The experimental results are shown in Table 1 and Table 2. Table 1 shows the test results of the method of the present invention (DSDNet) and other semi-supervised segmentation algorithms in the EEC dataset. It can be seen from the table that the method of the present invention leads other methods in terms of Dice, IoU, Acc, MAE, and 95HD metrics. The test results of other methods are not ideal. This is mainly because in some EEC images, the lesion areas are very similar to the normal areas, resulting in difficult identification of lesion boundaries and easy mis-segmentation by the model. In contrast, the method of the present invention has achieved excellent segmentation results, and each metric is significantly better than other methods, which reflects that the present invention can effectively solve the problem of mis-segmentation of the model caused by fuzzy boundaries.

[0162] Table 2 shows the test results of the method of the present invention (DSDNet) and other semi-supervised segmentation algorithms in the KS dataset. Similarly, the method of the present invention is superior to other methods in terms of Dice, IoU, Acc, MAE, and 95HD metrics. It should be noted that there are many small polyps in the KS dataset. Due to the inconspicuous features of these small lesions, it is easy to result in poor segmentation effects of the model. Therefore, the existing methods often have poor segmentation effects on small polyps. However, the method of the present invention can effectively alleviate this problem. It can be seen from Table 2 that DSDNet is significantly better than other methods in all metrics. Therefore, the method of the present invention can effectively solve the problem of difficult segmentation of small polyps. It can be seen that the test metrics of the method of the present invention in the two digestive tract disease segmentation datasets have reached the optimal results and are far ahead of the existing semi-supervised segmentation methods, indicating that the method of the present invention has excellent performance in the digestive tract disease segmentation task.

[0163] Table 1 Comparison of evaluation metrics between the method DSDNet of the present invention and other methods in the EEC dataset

[0164]

[0165] Table 2 Comparison of evaluation metrics between the method DSDNet of the present invention and other methods in the KS dataset

[0166]

[0167] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit them. Although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions described in the foregoing embodiments, or perform equivalent replacements on some or all of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A semi-supervised training method for dual-view interactive learning based on information bottleneck and contrastive learning, characterized in that The method includes: Step S1: Collect medical image data, and then preprocess the image data. The preprocessing includes data augmentation and view transformation; Step S2: Construct a semi-supervised training model based on dual-view interactive learning, information bottleneck theory, and class center contrast learning. The segmentation model in the dual-view interactive learning consists of Network A and Network B. The U-Net is used as the segmentation network for Network A and Network B. The collected RGB image is input into Network A, and the HSL image corresponding to the RGB image is input into Network B. In the parallel network composed of Network A and Network B, dual-view interactive learning is used to avoid the coupling effect of the parallel network. Information bottleneck theory and class center contrast learning are used to enhance the perception and discrimination ability of the network; Step S3: Train the segmentation model, and train the preprocessed data set through the semi-supervised training model. During the training process, parameter updates are performed through forward propagation and backward propagation, and the training is stopped when the performance of the network reaches the optimal, so as to obtain the weight parameters with the best segmentation effect; Among them, in the step S2, f a The information bottleneck loss of the RGB image marked in the figure is expressed as: Among them, HSIC is the Hilbert-Schmidt Independence Criterion, is the labeled RGB image, Y i is the corresponding label value, is f a during training feature layer matrix in the process, and L is the number of feature layers; f a The information bottleneck loss of the unlabeled RGB image is expressed as: Among them, is an unlabeled RGB image, is the pseudo-label value generated by f b and is the feature layer matrix during the trainingof f a process; ​ f b The information bottleneck loss of the HSL image marked is: Among them, is the labeled HSL image, Y i is the corresponding label value, is f b during training feature layer matrix in the process; f b The information bottleneck loss of the unlabeled HSL image is as follows: Among them, is the unlabeled HSL image, is the pseudo-label generated by f a , is the feature layer matrix during the training b process of f ; Thus, the total information bottleneck loss is: The class center contrast learning in Step S2 specifically includes: constructing a class center contrast learning method, calculating the center features of each class, comparing the pixel features with the center features of all classes, and classifying the pixel features into the class with the center feature that is most similar to it according to the cosine similarity. The positive sample pair of the class center contrast learning is the comparison between the pixel feature and its corresponding class center, and the negative sample pair is the comparison between the pixel feature and the class centers of other classes; The calculation method of the class center feature is: f a The predicted probability value for the labeled RGB data is Let P i r The feature vector at the (i, j) position in is denoted as Let the label Y i The feature vector at the (i, j) position in is denoted as Let the pseudo-label generated by f b be denoted as the feature vector at the (i, j) position in where C is the feature dimension, and the central feature of each category calculated based on the RGB annotation data is:​ Among them, is the class center feature of class k in the RGB annotation image, and N k is the number of pixels of class k in the RGB annotation image. The classes include two categories: the lesion area and the normal area. According to this formula, the positive sample class center feature and the negative sample class center feature are calculated. The cosine similarity is used to measure the similarity between the pixel feature and : || represents the norm of the vector. Similarly, the cosine similarity is used to measure the pixel features and the similarity between: Then, construct the training of f based on the InfoNCE loss a The contrastive learning loss of the class center for the labeled RGB images when Among them, N y is the number of pixels in, C is the negative sample set, and τ is the temperature parameter used to adjust the scale of similarity in contrast learning; Similarly, the center feature of each class calculated based on the unlabeled RGB image is: Among them, is the class center feature of class k in the RGB unlabeled image, and N u is is the number of pixels in; Training f a The contrastive learning loss of the class center of the unlabeled RGB image during training is as follows: Among them, N u is the number of pixels in f a the predicted probability distribution generated by training unlabeled data for the feature vector at position i, j in, is the corresponding positive sample, is the corresponding negative sample, and Q is the set of negative samples; Similarly, f is obtained b Class center contrast loss for network training labeled HSL data And class center contrast loss for training unlabeled HSL data The total class center contrast learning loss is as follows:

2. The method according to claim 1, wherein Step S1 further includes: Divide the collected image data into N labeled data and M unlabeled data, and then use the color space conversion algorithm to convert the collected RGB image into the corresponding HSL image. Denote the labeled RGB image as Denote the labeled HSL image as Denote the unlabeled RGB image as Denote the unlabeled HSL image as The total data set is where and are the image feature values input to the network, and Y i is the label value. The data augmentation method includes horizontal flipping, vertical flipping, and random scaling.

3. The method according to claim 1, wherein The dual-view interactive learning in step S2 specifically includes: A parallel network framework is composed of two segmentation networks A and B with the same structure. The segmentation network A is denoted as f a , and the segmentation network B is denoted as f b . f a trains the collected RGB images, and f b trains the corresponding HSL images. During the training process, for the labeled images, the two segmentation networks are optimized by calculating the cross-entropy loss and Dice loss between the predicted value and the true value. For the unlabeled images, the two segmentation networks are mutually supervised and optimized through pseudo-labels.

4. The method according to claim 3, characterized in that Use f a Training labeled RGB images Through the combination of cross-entropy loss and Dice loss as the training Supervised loss, and the supervised loss is expressed as: Among them, is the supervised loss of f a for the RGB annotation data, is the number of annotated RGB images, is the cross-entropy loss, is the Dice loss, which are respectively expressed as: Among them, Y and P respectively represent the label value and the predicted probability of the input image. The label value is obtained by manual annotation, and the predicted probability is obtained by training f a . H and W respectively represent the height and width of the image, and Y i and P i respectively represent the label value and the predicted probability of the i-th pixel in the image.

5. The method according to claim 4, wherein Use f b Train the labeled HSL images. Similarly, use the combination of cross-entropy loss and Dice loss as the supervised loss for training the labeled HSL images, expressed as: Among them, is f b the supervised loss of the HSL annotation data, is the number of the annotated HSL images; Use f a Train the unlabeled RGB image data to obtain the corresponding predicted probability values Use f b Train the unlabeled HSL image data to obtain the corresponding predicted probability values Then, through the network f b The generated pseudo labels supervise the network f a , and the calculation method of the loss function is as follows: Among them, is the number of unlabeled RGB images, is the pseudo-label generated by P i b ; Through network f a The pseudo-label supervised network f generated b , and the loss function is calculated as follows: Among them, is the number of unlabeled HSL images, is the pseudo-label generated by P i a ; Adding the above losses gives the total loss function for dual-view interactive learning It is expressed as:

6. The method according to claim 1, wherein HSIC is defined as: Among them, is the identity matrix, 1 is the column vector with all elements being 1, N represents the number of rows of variables X and Y, tr represents the trace of the matrix, is the kernel matrix, expressed as: where, X i and X j are two different variables in X, σ is an adjustable parameter, represents the 2-norm of the vector; K Y is expressed as: where, Y i and Y j are two different variables in Y.

7. The method according to claim 1 or 5, characterized in that, Adding the dual-view interactive learning loss, information bottleneck loss, and class center contrast learning loss to obtain the total loss function: where α and β are the trade-off coefficients in the total loss function.

Images