A semi-supervised learning method and system for classifying the degree of cervical intraepithelial neoplasia by combining generative adversarial networks

Abstract

The present invention discloses a semi-supervised learning method and system for classifying the degree of cervical intraepithelial neoplasia by combining generative adversarial networks. The method includes: generating realistic colposcopy acetic acid images based on an improved generative adversarial network GAN, and optimizing the realistic colposcopy acetic acid images through a super-resolution generative adversarial network SRGAN; generating pseudo-labels for the optimized realistic colposcopy acetic acid images based on a deep residual network ResNet, and training the ResNet network based on the optimized realistic colposcopy acetic acid images and images with real labels until the pseudo-labels no longer change, and retaining the trained ResNet network; inputting independent vaginal acetic acid images into the trained ResNet network to classify the degree of cervical intraepithelial neoplasia. The present invention can better adapt to the situation where the coverage of colposcopy image samples is insufficient and reduce the influence of data deviation.

Description

Technical Field

[0001] The present invention relates to the technical field of deep learning, and in particular, to a semi-supervised learning cervical intraepithelial neoplasia degree classification method and system combining a generative adversarial network. Background Art

[0002] Cervical cancer is one of the most common cancers in women worldwide. However, the pre-cancerous lesions of cervical cancer last for a long time and have a high probability of being cured in the early stage. The abnormal growth of cervical surface cells (potential pre-cancerous transformation) is called cervical intraepithelial neoplasia or cervical dysplasia. Professional doctors can determine whether there are pre-cancerous lesions through combined colposcopy with acetic acid and Lugol's iodine solution, make targeted diagnostic opinions, and timely detect cervical intraepithelial neoplasia (CIN) to prevent and treat cervical cancer and save women's lives.

[0003] Artificial intelligence (AI) based on deep learning has shown important application value in medical diagnosis. In recent years, many scholars have studied and tried to use deep learning methods to classify the levels of cervical lesions in colposcopy images to assist clinicians in triaging patients and improving the diagnostic efficiency of colposcopy doctors.

[0004] However, the training results of existing neural networks for classifying the levels of cervical intraepithelial neoplasia are affected by the scale of labeled training data. Training neural networks requires a large amount of professional annotation datasets, but the publicly available datasets in this field are scarce and of uneven quality, and collecting and labeling data is cumbersome, time-consuming and expensive. Manually labeled data may also be incorrect due to the level or fatigue of professional doctors. Therefore, how to more effectively utilize image data and a large amount of unlabeled image data is particularly important. Summary of the Invention

[0005] To solve the above technical problems, the present invention proposes a semi-supervised learning cervical intraepithelial neoplasia degree classification method and system combining a generative adversarial network, which can more effectively utilize image data and a large amount of unlabeled image data and improve the accuracy of the classification model.

[0006] On the one hand, to achieve the above object, the present invention provides a semi-supervised learning cervical intraepithelial neoplasia degree classification method combining a generative adversarial network, including:

[0007] Generating realistic colposcopy acetic acid images based on an improved generative adversarial network GAN, and optimizing the realistic colposcopy acetic acid images through a super-resolution generative adversarial network SRGAN to generate target colposcopy acetic acid images;

[0008] Generate pseudo - labels for the target colposcopy acetic acid images based on the deep residual network ResNet, and train the ResNet network based on the target colposcopy acetic acid images and the images with true labels until the pseudo - labels no longer change, then retain the trained ResNet network;

[0009] Input the colposcopy acetic acid images to be classified into the trained ResNet network for classifying the degree of cervical intraepithelial neoplasia.

[0010] Preferably, the improved generative adversarial network GAN is a network that introduces the Wasserstein distance to replace the JS divergence used in the traditional generative adversarial network, and modifies the fully - connected layer of the discriminator in the original generative adversarial network into a convolutional layer, and modifies the fully - connected layer of the generator into a transposed convolutional layer.

[0011] Preferably, the improved generative adversarial network GAN is trained with colposcopy acetic acid image samples, and the trained network is used to generate the realistic colposcopy acetic acid images.

[0012] Preferably, the loss functions of the discriminator and the generator in the improved generative adversarial network GAN are respectively:

[0013]

[0014] In the formula, L D is the discriminator loss function, is the gradient calculation, m is the number of discretization equal - division points in the summation interval, f is the use of the Wasserstein distance, G(z) is the fake image generated by the generator, L G is the generator loss function.

[0015] Preferably, optimize the realistic colposcopy acetic acid images through the super - resolution generative adversarial network SRGAN, including:

[0016] Train the super - resolution generative adversarial network SRGAN, and optimize the realistic colposcopy acetic acid images based on the trained super - resolution generative adversarial network SRGAN to enlarge the image size and refine the image texture details.

[0017] Preferably, train the super - resolution generative adversarial network SRGAN based on colposcopy acetic acid images. The method is:

[0018]

[0019] In the formula, θ G and θ D are the network parameters of the generator and the discriminator respectively, represents the discriminator network, Denote the generator network as I HR is the high-resolution image data, I LR is the low-resolution image data is the fake high-resolution image generated by the generator is the discriminator's discrimination of the real high-resolution image is the discriminator's discrimination of the fake high-resolution image generated by the generator, p train (I LR ) is the probability distribution of the real high-resolution image, p G (I LR ) is the probability distribution of the fake high-resolution image generated after inputting the low-resolution image into the generator is the mathematical expectation of the probability that a sample taken from the real high-resolution image distribution is determined by the discriminator to be a real high-resolution sample is for the fake data p generated by the generator G (I LR ) The negative logarithm of the expected value of the predicted probability after the fake high-resolution image obtained after sampling from the distribution of the sample generated by the generator is sent to the discriminator

[0020] Preferably, training the ResNet network based on the target colposcopy acetic acid image and the image with real labels includes:

[0021] First, train the deep residual network ResNet using the pictures with real labels in the original dataset based on the self-training method, then judge the target colposcopy acetic acid image and the unlabeled data in the original dataset to generate pseudo-labels, and add the data labeled with the obtained pseudo-labels to the original data for continuous training until the pseudo-labels no longer change, and the training is completed

[0022] Preferably, after training the deep residual network ResNet using the pictures with real labels in the original dataset, generate pseudo-labels for the unlabeled data, and then use the pseudo-label learning method in semi-supervised learning to perform self-training on ResNet. Among them, the loss function of pseudo-label learning is:

[0023]

[0024] In the formula, L is the loss function of pseudo-label learning, which is divided into the real label part and the pseudo-label part. The weight of the pseudo-label part is adjusted using α(t), α(t) is the weight factor for balancing labeled and unlabeled data, n is the number of labeled data, m is the number of unlabeled data, y i is the real label of the i-th labeled data, y′ jis the predicted label, i.e., the pseudo-label, for the j-th unlabeled data, and f(x i ) is the prediction of the model for the i-th labeled data, and f(x j ) is the prediction of the model for the j-th unlabeled data.

[0025] On the other hand, to achieve the above object, the present invention also provides a semi-supervised learning cervical intraepithelial neoplasia degree classification system combined with a generative adversarial network, including:

[0026] A dataset acquisition unit: used to acquire a colposcopy acetic acid image dataset, which includes images with biopsy result diagnosis-confirmed true labels and unlabeled images without biopsy results. The unlabeled images are used to train an improved generative adversarial network and a super-resolution generative adversarial network, and are used for self-training of a deep residual network. The images with true labels are used for self-training of the deep residual network;

[0027] A loss function calculation unit: used to calculate the loss functions of different networks. Among them, the deep residual network uses a cross-entropy loss function, and the discriminator and generator in the improved generative adversarial network and the super-resolution generative adversarial network each have different mutually adversarial loss functions;

[0028] A realistic colposcopy acetic acid image generation unit: used to train the colposcopy acetic acid images in the dataset through an improved generative adversarial network and a super-resolution generative adversarial network, and optimize the realistic colposcopy acetic acid images through the super-resolution generative adversarial network SRGAN;

[0029] A cervical intraepithelial neoplasia degree classification unit: used to extract colposcopy image features through a deep residual network for classification to obtain low-grade squamous intraepithelial lesion results or high-grade squamous intraepithelial lesion results.

[0030] Compared with the prior art, the present invention has the following advantages and technical effects:

[0031] The present invention uses a deep learning method to evaluate the degree of cervical intraepithelial neoplasia for colposcopy acetic acid images, which can be used in clinical practice to assist doctors in making judgments without errors due to fatigue and can also reduce the possibility of doctor misdiagnosis; the semi-supervised learning method effectively utilizes the past picture datasets that could not be used due to the lack of true labels, maximizes the use of data, combines the generative adversarial network for semi-supervised learning, and the generated data further expands the dataset, improves the generalization ability of the model, reduces the risk of overfitting, better adapts to the situation where the colposcopy image samples are insufficiently covered, reduces the influence of data bias, enables the model to better adapt to the data distribution in the real world, improves its practical application value, and improves the accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] The accompanying drawings, which form a part of this application, are used to provide a further understanding of this application. The schematic embodiments of this application and their descriptions are used to explain this application and do not constitute an improper limitation of this application. In the drawings:

[0033] Figure 1 It is a flowchart of a semi-supervised learning method for classifying the degree of cervical intraepithelial neoplasia by combining a generative adversarial network according to an embodiment of the present invention;

[0034] Figure 2 It is a schematic diagram of a classical generative adversarial network model according to an embodiment of the present invention;

[0035] Figure 3 It is a schematic diagram of the process of semi-supervised learning for classifying the degree of cervical intraepithelial neoplasia according to an embodiment of the present invention;

[0036] Figure 4 It is a comparison chart of the experimental accuracy results of an embodiment of the present invention and the accuracy of other classical networks;

[0037] Figure 5 It is a comparison chart of the experimental recall results of an embodiment of the present invention and the accuracy of other classical networks;

[0038] Figure 6 It is a comparison chart of the F1-Score experimental results of an embodiment of the present invention and the accuracy of other classical networks. Detailed implementation manners

[0039] It should be noted that, without conflict, the embodiments in this application and the features in the embodiments can be combined with each other. The following will refer to the accompanying drawings and combine the embodiments to detail this application.

[0040] It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although the logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in a different order than here.

[0041] The present invention proposes a semi-supervised learning method for classifying the degree of cervical intraepithelial neoplasia by combining a generative adversarial network, as Figure 1 , including:

[0042] Based on the improved generative adversarial network GAN, generate realistic colposcopy acetic acid images, and optimize the realistic colposcopy acetic acid images through the super-resolution generative adversarial network SRGAN to generate target colposcopy acetic acid images;

[0043] Generate pseudo-labels for the target colposcopy acetic acid images based on the deep residual network ResNet, and train the ResNet network based on the target colposcopy acetic acid images and images with true labels until the pseudo-labels no longer change, and retain the trained ResNet network;

[0044] Input the vaginal acetic acid mirror images to be classified into the trained ResNet network for the classification of the degree of cervical intraepithelial neoplasia.

[0045] Specifically, the improved generative adversarial network GAN is a network that introduces the Wasserstein distance to replace the JS divergence used in the traditional generative adversarial network, and modifies the fully connected layer of the discriminator in the original generative adversarial network into a convolutional layer, and modifies the fully connected layer of the generator into a transposed convolution.

[0046] The improved generative adversarial network GAN is trained with colposcopy acetic acid image samples, and the trained network is used to generate the realistic colposcopy acetic acid images.

[0047] Among them, the loss functions of the discriminator and the generator in the improved generative adversarial network GAN are respectively:

[0048]

[0049] In the formula, L D is the discriminator loss function, is the gradient calculation, m is a constant, is the number of discretized equal division points in the summation interval, f is the use of the Wasserstein distance, G(z) is the fake image generated by the generator, L G is the generator loss function.

[0050] Specifically, optimize the realistic colposcopy acetic acid images through the super-resolution generative adversarial network SRGAN, including:

[0051] Train the super-resolution generative adversarial network SRGAN, and optimize the realistic colposcopy acetic acid images based on the trained super-resolution generative adversarial network SRGAN to enlarge the image size and refine the image texture details.

[0052] Train the super-resolution generative adversarial network SRGAN based on the colposcopy acetic acid images. The method is:

[0053]

[0054] In the formula, θ G and θ D are the network parameters of the generator and the discriminator respectively, represents the discriminator network, represents the generator network, IHR is high-resolution image data, I LR is low-resolution image data, is the fake high-resolution image generated by the generator, is the discriminator's discrimination of the real high-resolution image, is the discriminator's discrimination of the fake high-resolution image generated by the generator, p train (I LR ) is the probability distribution of the real high-resolution image, p G (I LR ) is the probability distribution of the fake high-resolution image generated after inputting the low-resolution image into the generator, is the mathematical expectation of the probability that a sample taken from the real high-resolution image distribution is judged by the discriminator to be a real high-resolution, is for the fake data p generated by the generator G (I LR ) Among the samples sampled from the distribution, the generated picture obtained after being generated by the generator is sent to the discriminator, and it is the expectation of the negative logarithm of its predicted probability.

[0055] Furthermore, based on the target colposcopic acetic acid image and the image with real labels, the ResNet network is trained, including:

[0056] Based on the self-training method, use the pictures with real labels in the original dataset to train the deep residual network ResNet first, then judge the target colposcopic acetic acid image and the unlabeled data in the original dataset to generate pseudo-labels, and add the data labeled with the obtained pseudo-labels to the original data to continue training until the pseudo-labels no longer change, and the training is completed.

[0057] After using the pictures with real labels in the original dataset to train the deep residual network ResNet first, generate pseudo-labels for the unlabeled data, and then use the method of pseudo-label learning in semi-supervised learning to perform self-training on ResNet, where the loss function of pseudo-label learning can be expressed as:

[0058]

[0059] In the formula, L is the loss function of pseudo-label learning, which is divided into the real label part and the pseudo-label part. The weight of the pseudo-label part is adjusted using α(t), α(t) is the weight factor for balancing labeled and unlabeled data, n is the number of labeled data, m is the number of unlabeled data, y i is the real label of the i-th labeled data, y′ j is the predicted label, that is, the pseudo-label, of the j-th unlabeled data, f(x i) is the prediction of the model for the i-th labeled data, f(x j ) is the prediction of the model for the j-th unlabeled data.

[0060] For the classification of the degree of cervical intraepithelial neoplasia, the presence of acetowhite epithelium, punctate vessels, sleeve-like glandular openings, and inlays in the colposcopy image is comprehensively considered, and the situation is divided into low-grade squamous intraepithelial lesion or high-grade squamous intraepithelial lesion.

[0061] Furthermore, the present invention also provides a semi-supervised learning cervical intraepithelial neoplasia degree classification system combined with a generative adversarial network, including:

[0062] Dataset acquisition unit: used to acquire a colposcopy acetic acid image dataset, which includes images with biopsy result diagnosis-confirmed true labels and unlabeled images without biopsy results. The unlabeled images are used to train an improved generative adversarial network and a super-resolution generative adversarial network, and are used for self-training of a deep residual network. The images with true labels are used for self-training of the deep residual network;

[0063] Loss function calculation unit: used to calculate the loss functions of different networks. Among them, the deep residual network uses a cross-entropy loss function, and the discriminator and generator in the improved generative adversarial network and the super-resolution generative adversarial network each have different mutually adversarial loss functions;

[0064] Realistic colposcopy acetic acid image generation unit: used to train the colposcopy acetic acid images in the dataset through an improved generative adversarial network and a super-resolution generative adversarial network, and optimize the realistic colposcopy acetic acid images through the super-resolution generative adversarial network SRGAN;

[0065] Cervical intraepithelial neoplasia degree classification unit: used to extract colposcopy image features through a deep residual network for classification to obtain low-grade squamous intraepithelial lesion results or high-grade squamous intraepithelial lesion results.

[0066] To more clearly express the technical solution of the present invention, the following provides specific embodiments for introducing the solution: A semi-supervised learning cervical intraepithelial neoplasia degree classification method combined with a generative adversarial network, the process of which includes:

[0067] Introduce the Wasserstein distance to improve the generative adversarial network (GAN), replace the JS divergence used in the traditional generative adversarial network, and solve problems such as mode collapse and gradient disappearance in the training process of the generative adversarial network.

[0068] The loss functions of the discriminator and generator in the traditional generative adversarial network are as follows:

[0069]

[0070]

[0071] Among them, D represents calculating the distance using JS divergence.

[0072] After introducing the Wasserstein distance, the loss functions of the discriminator and the generator are as follows:

[0073]

[0074] Among them, f represents using the Wasserstein distance;

[0075]

[0076] Using the method of gradient penalty to make the discriminator in the network satisfy the 1-Lipschitz condition (not greater than K at the first derivative) after introducing the Wasserstein distance. After adding the penalty term to the loss function of the discriminator, its loss function is:

[0077]

[0078] Among them, P g is the generated data, and P r is the real data.

[0079] Compared with the original loss function, only the constraint term of gradient penalty is added.

[0080] Among them

[0081]

[0082] Use colposcopy acetic acid images to train the improved generative adversarial network, and use the trained network to generate realistic small-sized colposcopy images;

[0083] Use colposcopy acetic acid images to train the super-resolution generative adversarial model. The super-resolution generative adversarial model includes a generator and a discriminator module. The generator generates super-resolution images from low-resolution images, and the discriminator judges the authenticity. The goal of the generator is to make it difficult for the discriminator to judge that the generated image is fake, and the goal of the discriminator is to distinguish between real and fake images as much as possible. The two are in mutual confrontation, and their objective function is as follows:

[0084]

[0085] Among them, θ G and θ D are the network parameters of the generator and the discriminator respectively, represents the discriminator network, represents the generator network, and I HR is the high-resolution image data, and ILR is low-resolution image data, is the fake high-resolution image generated by the generator, is the discriminator's discrimination of the real high-resolution image, is the discriminator's discrimination of the fake high-resolution image generated by the generator, p train (I LR ) is the probability distribution of the real high-resolution image, p G (I LR ) is the probability distribution of the fake high-resolution image generated after inputting the low-resolution image to the generator, is the mathematical expectation of the probability that a sample taken from the real high-resolution image distribution is judged by the discriminator to be a real high-resolution, is for the fake data p generated by the generator G (I LR ) After sampling from the distribution, the generated image obtained after passing through the generator is sent to the discriminator, and it is the expectation of the negative logarithm of the predicted probability.

[0086] The ultimate goal is to train a generation function that can generate the corresponding high-resolution image from the output low-resolution image, as shown in the formula: The finally output high-resolution image

[0087] After training is completed, use this network to process the previously generated small-size images for image super-resolution, expand the image size, refine the texture details, and improve the image quality.

[0088] Select the deep residual network as the backbone network for the classification of the degree of cervical intraepithelial neoplasia. Use the Wasserstein distance to improve the generative adversarial network and the super-resolution generative adversarial network to generate realistic colposcopy images as unlabeled data to expand the dataset, and use the self-training method in semi-supervised learning to train the network.

[0089] Use the images with true labels in the original dataset to train the deep residual network for a period of time first, and then judge the generated realistic colposcopy images and the unlabeled data in the original dataset to generate pseudo-labels, that is, select the label of the sample with the most confident prediction probability as the true label, and add the data with pseudo-annotations obtained to the original data to continue training. Objectively speaking, it is hoped that regardless of whether the prediction result of the network is correct or not, it can make the judgment of the network more confident. According to the assumption of semi-supervised learning, the decision boundary should pass through the area where the data is relatively sparse as much as possible, that is, the low-density area, so as to avoid dividing the dense sample data points on both sides of the decision boundary. That is to say, the model needs to make a low-entropy prediction for the unlabeled data, that is, entropy minimization. The pseudo-label method is conducive to entropy minimization, that is, the goal of pseudo-labels is actually entropy minimization. The loss function of pseudo-label learning can be expressed as:

[0090]

[0091] In the formula, L is the loss function of pseudo-label learning, which is divided into the true label part and the pseudo-label part. The weight of the pseudo-label part is adjusted by α(t), and α(t) is the weight factor for balancing the labeled and unlabeled data. n is the number of labeled data, m is the number of unlabeled data, y i is the true label of the i-th labeled data, y j ′ is the predicted label of the j-th unlabeled data, that is, the pseudo-label, f(x i ) is the prediction of the model for the i-th labeled data, f(x j ) is the prediction of the model for the j-th unlabeled data.

[0092] After using the data with pseudo-labels and real data to train for a period of time, update the pseudo-labels, and then continue training, and loop until the pseudo-labels basically no longer change;

[0093] The deep residual network that completes self-training is finally used for the classification of the degree of cervical intraepithelial neoplasia, and it is tested on an independent colposcopy acetic acid image dataset and classified into low-grade squamous intraepithelial lesions and high-grade squamous intraepithelial lesions.

[0094] As Figure 2 shown, the schematic diagram of the classic generative adversarial network model, which consists of a discriminator and a generator, and its working principle is based on the adversarial game between two neural networks.

[0095] The task of the generator is to learn to generate data samples similar to real data. It receives a random noise as input and attempts to transform it into an output similar to real data samples. The goal of the generator is to deceive the discriminator as much as possible so that it cannot distinguish between the generated data and the real data. The task of the discriminator is to distinguish between the fake samples generated by the generator and the real data samples. It receives the input data and then attempts to classify it as real data or generated data. The goal of the discriminator is to distinguish between these two types of data as accurately as possible. During the training process, the generator and the discriminator compete against each other. The generator tries to deceive the discriminator by generating more realistic data, while the discriminator strives to improve its accuracy in distinguishing between real data and generated data. As the training progresses, the confrontation between the generator and the discriminator will cause the generator to gradually learn the skills of generating more realistic data samples, and the discriminator also becomes more precise.

[0096] Finally, when the training reaches equilibrium, the generator will be able to generate data that is very similar to real data samples and even indistinguishable, while the discriminator will not be able to effectively distinguish between the generated data and the real data.

[0097] In the first step, the goal is to generate realistic colposcopy acetic acid images. Using real colposcopy acetic acid images as real data, they are jointly input into the discriminator with the fake images generated by the generator for the discriminator to judge. Improving the network using the Wasserstein distance does not change its working principle. It is also through the confrontation between the generator and the discriminator to ultimately improve the quality of the output colposcopy acetic acid images, and finally output realistic colposcopy acetic acid images.

[0098] The super-resolution generative adversarial network has the same working principle. Its working principle is still based on the framework of the generative adversarial network, and at the same time combines the residual network and the perceptual loss function.

[0099] In this network, the realistic image that the generator needs to generate is a super-resolution image. The random noise input to the generator is a random low-resolution image. The task of the generator is to transform the low-resolution input image into a high-resolution output image. It receives a low-resolution image as input and attempts to generate an image similar to the original high-resolution image. Different from traditional upsampling methods, the generator of the super-resolution generative adversarial network achieves super-resolution by learning high-frequency textures and details. The task of the discriminator is to distinguish between the super-resolution images generated by the generator and the real high-resolution images. The realistic images it receives are the super-resolution images generated by the generator, and the real data it receives are the real high-resolution images, and it attempts to classify them as real images or generated images. The goal of the discriminator is to distinguish between these two types of images as accurately as possible.

[0100] The generator and the discriminator compete against each other and continuously improve each other's performance through a game. The generator attempts to generate more realistic high-resolution images to deceive the discriminator, while the discriminator endeavors to enhance its accuracy to distinguish between real images and generated images.

[0101] Figure 3 It is a schematic diagram of the process for semi-supervised learning of cervical intraepithelial neoplasia degree classification.

[0102] First, use the pictures with real labels in the original dataset to train the deep residual network. After iterating for a period of time, judge the generated realistic colposcopy images and the unlabeled data in the original dataset, and generate pseudo-labels for them, that is, select the label of the sample with the most confident prediction probability as the real label; after integrating the data with pseudo-annotations obtained with the original pictures dataset with real labels to form a new dataset, use the new dataset to continue training the network. After iterating for a while, use the network to generate new pseudo-labels. After integrating the dataset with new pseudo-labels with the original pictures dataset with real labels to form a new dataset, use the new dataset to continue training the network. Repeat the above process in a loop until the pseudo-labels basically no longer change.

[0103] Figures 4-6 It is the performance of various evaluation indicators of the model obtained from the experiment when the semi-supervised learning method for cervical intraepithelial neoplasia degree classification combined with a generative adversarial network is carried out with the scale of the pictures dataset with real labels being more than two thousand six hundred cases and the generative adversarial network generating five thousand images on more than eight hundred test sets. It compares the performances of several common other networks and the same network without using this method when training on the exactly same real-label dataset on the same test set.

[0104] Accuracy is the most commonly used metric in classification models, which describes the proportion of samples correctly predicted by the classifier. In this method, it is the proportion of the degree of cervical intraepithelial neoplasia represented by the correctly predicted pictures in the total number of pictures. Precision is the proportion of true positives among all samples predicted as positive by the classifier.

[0105] The expression for accuracy is:

[0106] Recall is the proportion of samples successfully predicted as positive among all true positives. In this method, it refers to the proportion of those predicted as high-grade squamous intraepithelial lesions in the total number of real high-grade squamous intraepithelial lesions.

[0107] The expression for recall is:

[0108] The F1-score is the harmonic mean of precision and recall, comprehensively considering the precision and recall of the model. The higher the F1-score, the better the comprehensive performance of the model.

[0109] The F1-score expression is as follows: where

[0110] In all expressions, TP represents the number of true positives, that is, the number of high-grade squamous intraepithelial lesions judged as high-grade squamous intraepithelial lesions; TN represents the number of true negatives, the number of low-grade squamous intraepithelial lesions judged as low-grade squamous intraepithelial lesions; FP represents the number of false positives, that is, the number of low-grade squamous intraepithelial lesions misjudged as high-grade squamous intraepithelial lesions; FN represents the number of false negatives, that is, the number of high-grade squamous intraepithelial lesions misjudged as low-grade squamous intraepithelial lesions.

[0111] This embodiment also provides a semi-supervised learning cervical intraepithelial neoplasia degree classification system combined with a generative adversarial network, including:

[0112] Dataset acquisition unit: used to acquire a colposcopy acetic acid image dataset, which includes images with biopsy result diagnosis to confirm true labels and unlabeled images without biopsy results. The unlabeled images are used to train an improved generative adversarial network and a super-resolution generative adversarial network, and are used for self-training of a deep residual network. The labeled images are used for self-training of the deep residual network;

[0113] Loss function calculation unit: used to calculate the loss functions of different networks. Among them, the deep residual network uses a cross-entropy loss function, and the discriminator and generator in the improved generative adversarial network and the super-resolution generative adversarial network each have different mutually adversarial loss functions;

[0114] Realistic colposcopy acetic acid image generation unit: used to train the colposcopy acetic acid images in the dataset through an improved generative adversarial network and a super-resolution generative adversarial network, and optimize the realistic colposcopy acetic acid images through the super-resolution generative adversarial network SRGAN;

[0115] Cervical intraepithelial neoplasia degree classification unit: used to extract colposcopy image features through a deep residual network for classification to obtain low-grade squamous intraepithelial lesion results or high-grade squamous intraepithelial lesion results.

[0116] This application uses deep learning methods to evaluate the degree of cervical intraepithelial neoplasia in colposcopy acetic acid images, which can be used clinically to assist doctors in making judgments without errors due to fatigue and can also reduce the possibility of misdiagnosis by doctors; the semi-supervised learning method effectively utilizes the picture dataset that could not be used in the past due to the lack of true labels, maximizes the use of data, combines with a generative adversarial network for semi-supervised learning, and the generated data further expands the dataset, improves the generalization ability of the model, reduces the risk of overfitting, better adapts to the situation where the coverage of colposcopy image samples is insufficient, and reduces the impact of data bias. This enables the model to better adapt to the data distribution in the real world, improves its practical application value, and enhances the accuracy rate.

[0117] The above is only the preferred specific implementation manner of this application, but the protection scope of this application is not limited thereto. Any changes or substitutions that can be easily thought of by those skilled in the art within the technical scope disclosed in this application should be covered by the protection scope of this application. Therefore, the protection scope of this application should be subject to the protection scope of the claims.

Claims

1. A semi-supervised learning method for classifying the degree of cervical intraepithelial neoplasia combined with a generative adversarial network, characterized in that: include: Generate a realistic colposcopic acetic acid image based on an improved generative adversarial network (GAN), optimize the realistic colposcopic acetic acid image through a super-resolution generative adversarial network (SRGAN), and generate a target colposcopic acetic acid image; Generate a pseudo label for the target colposcopic acetic acid image based on a deep residual network ResNet, and train the ResNet network based on the target colposcopic acetic acid image and an image with a real label until the pseudo label no longer changes, and retain the trained ResNet network; Inputting the vaginal acetoscopy image to be classified into the trained ResNet network to classify the degree of cervical intraepithelial neoplasia; The improved generative adversarial network GAN is a network that introduces Wasserstein distance to replace the JS divergence used in the traditional generative adversarial network, and the fully connected layer of the discriminator in the original generative adversarial network is modified to a convolutional layer, and the fully connected layer of the generator is modified to a transposed convolution; The improved generative adversarial network GAN is trained by colposcopic acetic acid image samples, and the realistic colposcopic acetic acid image is generated by using the trained network; The loss functions of the discriminator and the generator in the improved generative adversarial network GAN are: Where, L D is the discriminator loss function, is the gradient calculation, m is the number of discretized equal-division points in the summation interval, f is the Wasserstein distance used, G(z) is the false image generated by the generator, and L G is the generator loss function; The realistic colposcopic acetic acid image is optimized by a super-resolution generative adversarial network SRGAN, including: Training the super-resolution generative adversarial network SRGAN, optimizing the realistic colposcopic acetic acid image based on the trained super-resolution generative adversarial network SRGAN, enlarging the image size, and refining the image texture details; The super-resolution generative adversarial network SRGAN is trained based on the colposcopic acetic acid image by: In the formula, θ G and θ D are the network parameters of the generator and the discriminator, represents the discriminator network, represents the generator network, I HR is high-resolution image data, I LR is low-resolution image data, is a fake high-resolution image generated by the generator, For the discriminator to distinguish the real high-resolution image, The discriminator is used to distinguish the fake high-resolution images generated by the generator, p train (I LR ) is the probability distribution of the real high-resolution image, p G (I LR ) is the probability distribution of the false high-resolution image generated after inputting the low-resolution image to the generator, is the mathematical expectation of the probability that a sample taken from the true high-resolution image distribution is judged by the discriminator as a true high-resolution sample, For the false data p generated from the generator G (I LR )The false high-resolution image obtained after the samples sampled from the distribution are generated by the generator and sent to the discriminator is the expectation of the negative logarithm of the predicted probability.

2. According to claim 1, the semi-supervised learning method for classifying the degree of cervical intraepithelial neoplasia combined with a generative adversarial network is characterized in that: The ResNet network is trained based on the target colposcopic acetic acid image and the image with the real label, including: The self-training method uses the pictures with real labels in the original data set to first train the deep residual network ResNet, then judges the target colposcopic acetic acid image and the unlabeled data in the original data set to generate pseudo labels, and adds the obtained data annotated with pseudo labels to the original data to continue training until the pseudo labels no longer change and the training is completed.

3. The method for classifying the degree of cervical intraepithelial neoplasia by semi-supervised learning combined with a generative adversarial network according to claim 2, characterized in that: After training the deep residual network ResNet using pictures with real labels in the original data set, pseudo labels are generated for unlabeled data, and then the pseudo label learning method in semi-supervised learning is used to self-train ResNet, where the loss function of pseudo label learning is: Where L is the loss function of pseudo-label learning, which is divided into the real label part and the pseudo-label part. The weight of the pseudo-label part is adjusted using α(t), α(t) is the weight factor for balancing labeled and unlabeled data, n is the number of labeled data, m is the number of unlabeled data, and y is i is the true label of the i-th labeled data, y j ′ is the predicted label of the jth unlabeled data, i.e., the pseudo label, f(x i ) is the model’s prediction for the i-th labeled data, f(x j ) is the model’s prediction for the jth unlabeled data.

4. A semi-supervised learning cervical intraepithelial neoplasia degree classification system combined with a generative adversarial network, characterized in that: include: Dataset acquisition unit: used to acquire colposcopy acetic acid image datasets, including images with real labels for biopsy result diagnosis confirmation and unlabeled images without biopsy results. The unlabeled images are used to train the improved generative adversarial network and the super-resolution generative adversarial network, and are used as the deep residual network for self-training. The images with real labels are used for self-training of the deep residual network. Loss function calculation unit: used to calculate the loss functions of different networks, wherein the deep residual network uses a cross entropy loss function, and the discriminator and generator in the improved generative adversarial network and the super-resolution generative adversarial network each have a different mutually adversarial loss function; A realistic colposcopic acetic acid image generation unit: used to train the colposcopic acetic acid images in the data set through an improved generative adversarial network and a super-resolution generative adversarial network, and optimize the realistic colposcopic acetic acid images through a super-resolution generative adversarial network SRGAN; Cervical intraepithelial neoplasia degree classification unit: used to extract colposcopy image features through a deep residual network for classification, and obtain low-grade squamous intraepithelial lesion results or high-grade squamous intraepithelial lesion results; The improved generative adversarial network GAN is a network that introduces Wasserstein distance to replace the JS divergence used in the traditional generative adversarial network, and the fully connected layer of the discriminator in the original generative adversarial network is modified to a convolutional layer, and the fully connected layer of the generator is modified to a transposed convolution; The improved generative adversarial network GAN is trained by colposcopic acetic acid image samples, and the realistic colposcopic acetic acid image is generated by using the trained network; The loss functions of the discriminator and the generator in the improved generative adversarial network GAN are: Where, L D is the discriminator loss function, is the gradient calculation, m is the number of discretized equal-division points in the summation interval, f is the Wasserstein distance used, G(z) is the false image generated by the generator, and L G is the generator loss function; The realistic colposcopic acetic acid image is optimized by a super-resolution generative adversarial network SRGAN, including: Training the super-resolution generative adversarial network SRGAN, optimizing the realistic colposcopic acetic acid image based on the trained super-resolution generative adversarial network SRGAN, enlarging the image size, and refining the image texture details; The super-resolution generative adversarial network SRGAN is trained based on the colposcopic acetic acid image by: In the formula, θ G and θ D are the network parameters of the generator and the discriminator, represents the discriminator network, represents the generator network, I HR is high-resolution image data, I LR is low-resolution image data, is a fake high-resolution image generated by the generator, For the discriminator to distinguish the real high-resolution image, The discriminator is used to distinguish the fake high-resolution images generated by the generator, p train (I LR ) is the probability distribution of the real high-resolution image, p G (I LR ) is the probability distribution of the false high-resolution image generated after inputting the low-resolution image to the generator, is the mathematical expectation of the probability that a sample taken from the true high-resolution image distribution is judged by the discriminator as a true high-resolution sample, For the false data p generated from the generator G (I LR )The false high-resolution image obtained after the samples sampled from the distribution are generated by the generator and sent to the discriminator is the expectation of the negative logarithm of the predicted probability.

Images