Cervical cell image contrast learning and automatic identification method and system

Abstract

The invention provides a cervical cell image comparative learning and automatic identification method and system, and the method comprises the steps: carrying out the parallel fusion of a bimodal backbone network through the combination of unsupervised pre-training and supervised tuning, making full use of unlabeled data and labeled data, and cooperating with the careful selection of a loss function at a pre-training stage and a tuning stage, thereby achieving the automatic recognition of a cervical cell image. The training effect and generalization ability of the deep neural network model are significantly improved, efficient and accurate recognition and classification of the cervical cell image are realized, and the method has important application value and market prospect.

Description

Technical Field

[0001] The present invention relates to the field of biomedical cell image processing, and specifically relates to a method and system for contrastive learning and automatic recognition of cervical cell images, which are applicable to the robust contrastive learning task for ThinPrep cytologic test (TCT) images (Robust Contrastive Learning for TCT, RCL-TCT), as well as tasks such as automated classification and abnormal cell detection. Background Art

[0002] Cervical cancer is one of the malignant tumors with relatively high incidence and mortality rates among women globally. Early screening and diagnosis are of great significance for improving the survival rate of patients. ThinPrep cytologic test (TCT) is a widely used cervical cancer screening method in clinical practice. By analyzing cervical cell microscope images, abnormal cells can be detected and used for auxiliary diagnosis. However, traditional TCT image diagnosis relies on the experience and subjective judgment of pathologists, and has problems such as low diagnostic efficiency, high labor intensity, and poor consistency.

[0003] In recent years, with the development of machine learning, especially deep learning (i.e., deep neural network) technology, data-driven solutions have shown excellent performance in image classification and feature extraction. Machine learning-based automatic classification and recognition methods for cervical cell images have gradually become a research hotspot, which can significantly improve diagnostic efficiency and accuracy. However, the existing methods still have the following problems:

[0004] 1. Limitations of supervised learning - strong dependence on labeled data: The supervised training of deep neural networks requires a large amount of high-quality labeled data. The labeling work of cervical cell images needs to be manually completed by professional pathologists, which is time-consuming (it takes 5 - 10 minutes to label a single image), costly (professional pathologists are scarce), and there are subjective differences in the labeling results (the consistency of labeling by different doctors is only 70% - 80%). In addition, the lack of labeled data easily leads to model overfitting and poor generalization performance.

[0005] 2. Low utilization rate of unlabeled data: In the actual application of cervical cell images, there are a large number of unlabeled TCT image data, which are not fully and effectively utilized, resulting in waste of resources.

[0006] 3. Insufficient feature extraction ability: A single backbone network is difficult to simultaneously capture the local details of cell morphology (such as fine features like mitotic figures) and global context (such as cell distribution patterns), resulting in insufficient accuracy, sensitivity, specificity, and precision in recognition and classification.

[0007] 4. Insufficient model robustness: The existing methods have poor robustness and adaptability. When the images have noise (such as cell overlap, uneven staining), come from different devices (different microscope brands), or different laboratories (different staining reagents), the performance of the model often drops significantly. Especially when dealing with imbalanced datasets, the traditional cross-entropy loss function often performs poorly. The model is easily "flooded" by the dominant majority-class samples, while the minority-class samples may be completely ignored. Although weighted cross-entropy can alleviate this problem by adjusting class weights, it has obvious defects, that is, all minority-class samples are regarded as equally important. In reality, minority-class samples may have different difficulties: some samples may be more difficult to be correctly recognized due to blurred features, occlusion, or noise. In practice, weighted cross-entropy cannot distinguish these differences, resulting in the model may still tend to recognize the "easy" minority-class samples and ignore the more difficult ones. Summary of the Invention

[0008] The purpose of the present invention is to provide a method and system for contrastive learning and automatic recognition of cervical cell images, which is a robust contrastive learning and automatic recognition method and system for ThinPrep Cytology Test (TCT) images (Robust Contrastive Learning for TCT, RCL-TCT). By using an unsupervised pre-training + supervised fine-tuning framework and combining the advantages of a dual-modal backbone network (ResNet + ViT) and contrastive learning to solve the above problems. Unsupervised learning (or self-supervised learning), as a training method that does not rely on labeled data, can extract effective features from a large amount of unlabeled data by designing pre-training tasks, and provide good initial weights for subsequent supervised training. Therefore, combining unsupervised pre-training and supervised fine-tuning can effectively improve the utilization efficiency of training data, as well as the robustness, fairness, and generalization ability of the model, and contribute to the efficient and accurate recognition and classification of cervical cell images.

[0009] The technical solution adopted by the present invention is specifically as follows:

[0010] A method for contrastive learning and automatic recognition of cervical cell images, specifically:

[0011] Obtain the cervical cell image to be recognized and input it into the cervical cell image recognition model to obtain the recognition result; wherein, the cervical cell image recognition model is trained by the following method:

[0012] Obtain cervical cell images to construct an unlabeled dataset and a labeled dataset; each sample of the labeled dataset contains the corresponding cervical cell image and the lesion label;

[0013] Construct a pre-trained model, where the pre-trained model includes a bimodal backbone network, a splicing and fusion layer, and a fusion mapping network connected in sequence; the bimodal backbone network consists of two branches: a residual network and a vision transformer;

[0014] Based on the unlabeled dataset, use the contrastive learning SimCLR framework to perform unsupervised pre-training on the constructed pre-trained model;

[0015] After the unsupervised pre-training is completed, use the bimodal backbone network of the pre-trained model as a feature extractor to extract features from each cervical cell image in the labeled dataset, and construct a feature-label dataset;

[0016] Construct a classification mapping network and perform supervised training based on the feature-label dataset. The optimization objective of the supervised training is to minimize the focal cross-entropy loss;

[0017] After the supervised training is completed, integrate the bimodal backbone network and the classification mapping network of the pre-trained model to obtain a cervical cell image recognition model.

[0018] Furthermore, using the contrastive learning SimCLR framework to perform unsupervised pre-training on the constructed pre-trained model, specifically:

[0019] Based on the unlabeled dataset, construct positive and negative sample pairs; among them, the positive sample pair is two different views of the same cervical cell image; the negative sample pair is different cervical cell images;

[0020] Use the positive and negative sample pairs as the input of the pre-trained model, and perform unsupervised pre-training by minimizing the contrastive loss of the feature vectors of the positive and negative sample pairs output by the pre-trained model.

[0021] Furthermore, the two different views of the same cervical cell image are generated using data augmentation techniques, and the data augmentation technical solutions include one or more of random geometric transformation, random color perturbation, random image grayscaling, and random Gaussian blur.

[0022] Furthermore, the contrastive loss is expressed as follows:

[0023] ;

[0024] Among them, the subscripts i, i+, j are cervical cell image indices, (i, i+) represents a pair of positive sample pairs, (i, j), i≠j, represents a pair of negative sample pairs; e i,j represents the cosine similarity measure of two feature vectors z i , z j , , z represents the feature vector output by the pre-trained model, sim(*) represents the cosine similarity metric, ||*|| represents the norm of the vector, and τ is the temperature hyperparameter used to control the sharpness of the distribution. is the indicator function that takes the value of 1 when and 0 otherwise; N represents the number of sample pairs in a batch, and 2N represents the total number of positive and negative samples composed of a batch of samples.

[0025] Furthermore, the fusion mapping network is composed of a multi-layer perceptron.

[0026] Furthermore, the classification mapping network is composed of a multi-layer perceptron.

[0027] A cervical cell image contrast learning and automatic recognition system includes:

[0028] A data preprocessing module for obtaining cervical cell images to construct an unlabeled dataset and a labeled dataset; each sample in the labeled dataset contains the corresponding cervical cell image and the lesion label.

[0029] A pre-trained model construction module for constructing a pre-trained model, which includes a bimodal backbone network, a splicing and fusion layer, and a fusion mapping network connected in sequence; the bimodal backbone network consists of two branches, a residual network and a vision transformer.

[0030] An unsupervised pre-training module for performing unsupervised pre-training on the constructed pre-trained model based on the unlabeled dataset using the contrast learning SimCLR framework.

[0031] A feature-label dataset construction module for, after the unsupervised pre-training ends, using the bimodal backbone network of the pre-trained model as a feature extractor to extract features from each cervical cell image in the labeled dataset, and constructing a feature-label dataset.

[0032] A classification mapping network construction module for constructing a classification mapping network.

[0033] A supervised tuning module for performing supervised training based on the feature-label dataset, and the optimization objective of the supervised training is to minimize the focal cross-entropy loss.

[0034] A model integration module for, after the supervised training ends, integrating the bimodal backbone network and the classification mapping network of the pre-trained model to obtain a cervical cell image recognition model.

[0035] A recognition module for obtaining the cervical cell image to be recognized and inputting it into the cervical cell image recognition model to obtain the recognition result.

[0036] An electronic device includes a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the computer program, it implements the described method for contrast learning and automatic recognition of cervical cell images.

[0037] A storage medium containing computer-executable instructions that, when executed by a computer processor, implement the described method for contrast learning and automatic recognition of cervical cell images.

[0038] A computer program product includes a computer program / instructions that, when executed by a processor, implement the steps of the described method for contrast learning and automatic recognition of cervical cell images.

[0039] The beneficial effects of the present invention are:

[0040] 1. Improve data usage efficiency: Through unsupervised pre-training, a large amount of unlabeled data can be fully utilized to make up for the shortage of labeled data and improve the training effect of the model.

[0041] 2. Reduce the risk of overfitting: Unsupervised pre-training provides good initial weights for the model, making the tuning process of the model on labeled data more efficient and reducing the occurrence of overfitting.

[0042] 3. Enhance the robustness and fairness of the model: The contrast learning algorithm enables the model to learn features with good robustness and generalization ability from unlabeled images. The focal cross-entropy loss function can flexibly adjust the attention of the model to difficult samples and each category by introducing a focusing adjustment factor and a category weight mechanism, effectively improving the performance of the model on complex datasets, further enhancing the model performance while taking into account classification fairness, especially showing significant advantages in the cervical cell image classification task with class imbalance.

[0043] 4. Improve recognition and classification performance: By combining unsupervised pre-training and supervised tuning, the model shows higher accuracy, sensitivity, specificity, and fairness when recognizing and classifying cervical cell images.

[0044] 5. Support fast retraining: The present invention has the ability to quickly retrain a new model and can efficiently adapt to different datasets and application scenarios. When encountering new data from different sources or new classification tasks (such as requiring a finer classification level), there is no need to retrain the entire bimodal backbone network. Only the classification mapping network h needs to be fine-tuned and adapted to quickly meet the needs of new data and new tasks. Description of the Drawings

[0045] Figure 1 It is a schematic flowchart of the method for contrast learning and automatic recognition of cervical cell images of the present invention;

[0046] Figure 2 is the flowchart of the unsupervised pre-training stage of the present invention;

[0047] Figure 3 is the flowchart of the supervised training stage of the present invention;

[0048] Figure 4 is the relationship curve graph between the test accuracy rate and the number of samples per class in the embodiments of the present invention;

[0049] Figure 5 is the comparison bar graph of the test accuracy rates of the embodiments of the present invention and the ResNet and ViT comparison models under different numbers of samples;

[0050] Figure 6 is the confusion matrix of the embodiments of the present invention; in the figure, 0_neg represents that the label is negative, represented by the number 0; 1_pos represents that the label is positive, represented by the number 1;

[0051] Figure 7 is the confusion matrix of the Residual Network (ResNet); in the figure, 0_neg represents that the label is negative, represented by the number 0; 1_pos represents that the label is positive, represented by the number 1;

[0052] Figure 8 is the confusion matrix of the Vision Transformer (ViT); in the figure, 0_neg represents that the label is negative, represented by the number 0; 1_pos represents that the label is positive, represented by the number 1;

[0053] Figure 9 is the visualization analysis result graph of the Gradient-based Class Activation Mapping (Grad-CAM) of 3 sample graphs, also known as the saliency graph or heat map.

[0054] Figure 10 is the schematic structural diagram of a cervical cell image contrast learning and automatic recognition system of the present invention. Detailed implementation manners

[0055] Here, the exemplary embodiments will be described in detail, and the examples are shown in the accompanying drawings. When the following description refers to the accompanying drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The implementation manners described in the following exemplary embodiments do not represent all implementation manners consistent with the present application. On the contrary, they are merely examples of devices and methods consistent with some aspects of the present application as detailed in the appended claims.

[0056] The terms used in this application are for the purpose of describing specific embodiments only and are not intended to limit this application.

[0057] The singular forms "a", "the", and "said" used in this application and the appended claims are also intended to include the plural forms unless the context clearly dictates otherwise. It should also be understood that the term "and / or" as used herein refers to and encompasses any and all possible combinations of one or more of the associated listed items.

[0058] It should be understood that although the terms first, second, third, etc. may be used in this application to describe various information, such information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of this application, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information. Depending on the context, the word "if" as used herein may be interpreted as "when" or "while" or "in response to a determination".

[0059] The present invention provides a method for contrastive learning and automatic recognition of cervical cell images, specifically as follows:

[0060] Obtain the cervical cell image to be recognized and input it into the cervical cell image recognition model to obtain the recognition result;

[0061] Among them, the cervical cell image recognition model is obtained by training through the following method, as Figure 1 shown, specifically including the following steps:

[0062] Step 1: Obtain cervical cell images to construct an unlabeled dataset and a labeled dataset; each sample of the labeled dataset contains the corresponding cervical cell image and a lesion label; and each sample of the unlabeled dataset contains a cervical cell image.

[0063] In a preferred embodiment, the cervical cell image is a single-cell / cluster-level local image obtained by performing image cleaning operations on the whole-slide microscopic digital image of a cervical cell smear, removing the background noise of the glass slide, and then performing cell segmentation and size adjustment. The size of this image is generally 128×128.

[0064] In a preferred embodiment, the lesion labels include lesion types such as Normal, ASCUS, ASC-H, LSIL, HSIL, AGC, GIL, etc.

[0065] Step 2: Construct a pre-trained model, the pre-trained model includes a dual-modal backbone network, a splicing and fusion layer, and a fusion mapping network connected in sequence; the dual-modal backbone network is composed of two branches of a residual network and a vision transformer; Figure 2 Shown as an exemplary pre-trained model structure.

[0066] In a preferred embodiment, the fusion mapping network is composed of a multi-layer perceptron.

[0067] Step 3: Based on the unlabeled dataset, use the contrastive learning SimCLR framework to perform unsupervised pre-training on the constructed pre-trained model;

[0068] In a preferred embodiment, it specifically includes the following sub-steps:

[0069] (3.1) Based on the unlabeled dataset, construct positive and negative sample pairs; among them, the positive sample pair is two different views of the same cervical cell image; the negative sample pair is different cervical cell images; in a preferred embodiment, the two different views of the same cervical cell image are generated by using data augmentation techniques, and the data augmentation technique scheme includes one or more of random geometric transformation, random color perturbation, random image grayscale, and random Gaussian blur.

[0070] (3.2) Use the positive and negative sample pairs as the input of the pre-trained model, and perform unsupervised pre-training by minimizing the contrastive loss of the feature vectors of the positive and negative sample pairs output by the pre-trained model, so as to make the feature vectors of the two views of the same image (i.e., the positive sample pair) as close as possible after being encoded by the model, while the feature vectors of the views of different images (i.e., the negative sample pair) are as far away as possible.

[0071] In a preferred embodiment, the contrastive loss is expressed as follows:

[0072] (1)

[0073] Among them, the subscripts i, i+, j are cervical cell image indexes, (i, i+) represents a pair of positive sample pairs, (i, j), i≠j, represents a pair of negative sample pairs; z represents the feature vector output by the pre-trained model, e i,j represents the cosine similarity measure of two feature vectors z i , z j of, , sim(*) represents the cosine similarity measure, ||*|| represents the norm of the vector, this value is related to the angle between the two vectors, that is, the direction, the smaller the angle, the more consistent the direction, and the greater the cosine similarity. τ is the temperature hyperparameter used to control the sharpness of the distribution, and in this embodiment, it is set to τ = 0.07. is the indicator function, which takes the value of 1 when and 0 otherwise; N represents the number of sample pairs in a batch, 2N represents the total number of positive and negative samples composed of a batch of samples, and each sample has two views.

[0074] Step 4: After the unsupervised pre-training is completed, use the backbone network of the pre-trained model as a feature extractor to extract features from each cervical cell image in the labeled dataset, and construct a feature-label dataset;

[0075] Step 5: Construct a classification mapping network and perform supervised training based on the feature-label dataset. The optimization objective of the supervised training is to minimize the focal cross-entropy loss.

[0076] In a preferred embodiment, the classification mapping network is composed of a multi-layer perceptron.

[0077] Step 6: After the supervised training is completed, integrate the backbone network of the pre-trained model and the classification mapping network to obtain a cervical cell image recognition model.

[0078] By combining unsupervised pre-training and supervised tuning, the present invention makes full use of unlabeled data and labeled data, parallelly integrates double backbone networks, and cooperates with carefully selected loss functions in the pre-training and tuning stages, significantly improving the training effect and generalization ability of the deep neural network model, realizing efficient and accurate recognition and classification of cervical cell images, and having important application value and market prospects.

[0079] The following uses a specific embodiment to illustrate the effect of the present invention:

[0080] A method for contrastive learning and automatic recognition of cervical cell images, as Figure 1 shown, includes the following steps:

[0081] Step 1: Obtain cervical cell images to construct an unlabeled dataset and a labeled dataset; including the following sub-steps:

[0082] (1.1) Construction of the unlabeled dataset: Collect whole-slide microscopic digital images of cervical cell smears (resolution approximately 65536×71680), perform image cleaning operations, remove the background noise of the glass slide (brightness mean normalization + Gaussian filtering denoising), and then use the YOLOv7 model for cell segmentation and size adjustment, and output single-cell / cluster-level local images (size 128×128) as the subsequent cervical cell images to be recognized. Take part of the cervical cell images to construct an unlabeled dataset.

[0083] (1.2) Construction of the labeled dataset: Send part of the cervical cell images to a pathologist for lesion annotation, and grade them into lesion types such as Normal, ASCUS, ASC-H, LSIL, HSIL, AGC, GIL, etc., to obtain the corresponding cervical cell images and lesion labels, and construct a labeled dataset.

[0084] Step 2. Construct a pre-trained model, \(M = g(f(x))\). The pre-trained model includes a bimodal backbone network, a splicing fusion layer, and a fusion mapping network connected in sequence. Here, \(x\) is an input unlabeled cervical cell image; \(f(*)\) is the bimodal backbone network. As Figure 2 shown, a parallel dual-branch design is adopted:

[0085] Branch 1 of the backbone network: The Residual Network (ResNet) \(f1(*)\) focuses on extracting various morphological features of cells (such as edges and textures). Its structure is shown in Figure 2 , which is composed of 18 residual modules and an MLP connected in series. The dimension of the output feature vector is 512.

[0086] Branch 2 of the backbone network: The Vision Transformer (a.k.a. ViT) \(f2(*)\) focuses on globally modeling through the self-attention mechanism to capture the long-range correlations and dependencies of cell distribution and morphological features. Its structure is shown in Figure 2 , and the dimension of the output feature vector is 512.

[0087] The splicing fusion layer is used to generate a fused feature vector by splicing and fusing the feature vectors encoded by the two branches. Its inputs are \(f1(x)\) and \(f2(x)\), where \(x\) represents the input cervical cell image. Splicing is performed in the feature dimension, and the output is the splicing of the feature vectors \(f(x)= [f1(x), f2(x)]\).

[0088] The fusion mapping network \(g(*)\) is used to map the fused feature vector to a high-dimensional space to obtain a high-dimensional feature vector. In this embodiment, its structure is a two-layer perceptron MLP. The input is the splicing \(f(x)\) of the output feature vectors of the backbone network, and the output is the high-dimensional feature vector \(g(f(x))\). Its structure is shown in Figure 3 , which is Dropout(ReLU(Linear(1024, 1024)))→Linear(1024, 1024), including two fully connected layers with parameters \((feature\_dim × feature\_dim × num\_classes)=(1024×1024×3)\). Between the two fully connected layers are a ReLU activation layer and a Dropout layer (with a probability of 0.2).

[0089] Step 3: Based on the unlabeled dataset, use the contrastive learning SimCLR framework to perform unsupervised pre-training on the constructed pre-trained model. In this embodiment, positive and negative sample pairs are dynamically constructed by a sampler, input into the pre-trained model for encoding to obtain feature vectors, and then the contrastive loss (infoNCE loss) is calculated. Finally, the model is optimized by minimizing the contrastive loss, prompting the feature vectors encoded by the model for two views of the same image (i.e., positive sample pairs) to be as close as possible, while the feature vectors of views of different images (i.e., negative sample pairs) are as far apart as possible. Specifically, it includes the following sub-steps:

[0090] (3.1) Based on the unlabeled dataset, in each training iteration, randomly sample a batch of images; apply data augmentation techniques to each image to generate two different views of the same image. The data augmentation scheme in this embodiment includes:

[0091] Random geometric transformation: Random cropping and scaling (range=(0.08, 1.0)), random rotation (±30°), random horizontal / vertical flipping.

[0092] Random color perturbation: Randomly adjust brightness (brightness = 0.5), contrast (contrast = 0.5), saturation (saturation = 0.5), and hue (hue = 0.1), with a 50% probability of taking effect.

[0093] Random grayscale conversion of pictures: Randomly convert color pictures to grayscale pictures, with a 20% probability of taking effect.

[0094] Random Gaussian blur: Randomly add Gaussian blur (blur kernel size kernel_size = 9), with a 50% probability of taking effect.

[0095] (3.2) Use the pre-trained model to encode each image xi to obtain its feature vector zi = g(f(xi)), that is, the unnormalized log odds logits. The specific process is to first use the dual-modal backbone network f to extract the features of each view, and then generate the high-dimensional feature vector zi through the fusion mapping network g.

[0096] Then, perform unsupervised pre-training by minimizing the contrastive loss (infoNCE loss) of the feature vectors of the positive and negative sample pairs output by the pre-trained model. The mathematical representation form is formula (1).

[0097] In this embodiment, the maximum number of training iterations is max_epochs = 400, the optimizer is AdamW (initial learning rate is 5-4, weight decay weight_decay = 1-4), and the learning rate strategy adopts the Cosine Annealing strategy.

[0098] Step 4: After the unsupervised pre-training is completed, use the backbone network of the pre-trained model as a feature extractor to extract features f(x) from each cervical cell image in the labeled dataset, obtaining a paired feature-label dataset {(f(x), y)}, where y is the lesion label.

[0099] Step 5: Construct a classification mapping network h(*). In this embodiment, a two-layer perceptron MLP structure is adopted, which is used to classify the extracted labeled features and perform supervised fine-tuning training. Its structure is Dropout(ReLU(Linear(1024, 1024))) → Linear(1024, C), including two fully connected layers, with parameters (feature_dim × feature_dim × num_classes) = (1024 × 1024 × C). Between the two fully connected layers are a ReLU activation layer and a Dropout layer (with a probability of 0.5), where C is the number of classes.

[0100] Then, perform supervised training fine-tuning based on the feature-label dataset:

[0101] Use the feature-label dataset to perform supervised training on the classification mapping network h(*), and the optimization objective is to minimize the Focal Cross-Entropy Loss. The mathematical form of the Focal Cross-Entropy Loss function is as follows:

[0102] (2)

[0103] where, is the predicted probability of the true label y corresponding to the current sample, which is obtained by calculating the unnormalized log-odds z output by the model through the softmax function; γ is the focusing adjustment factor, which is used to control the focusing intensity. When γ ≥ 1, if the pc of the current sample is small (i.e., the probability of correctly predicting this sample is small), then this sample is regarded as a difficult sample, and its loss will be amplified, and the amplification degree is (1 - pc)γ. This mechanism forces the model to pay more attention to difficult samples that are difficult to classify correctly. When γ = 1, the Focal Cross-Entropy Loss degenerates into the standard Cross-Entropy Loss, and the model no longer assigns additional weights to difficult samples. wc is the weight of class c, which is used to compensate for class imbalance. For example, for a class c with a small number of samples, wc can be set to 2 - 4 to appropriately increase the weight of this class, thereby adjusting class imbalance and achieving rapid adjustment of the model's attention to the majority class and the minority class without complex data resampling.

[0104] The supervised training parameters in this embodiment are set as follows: the initial learning rate lr = 1 - 4, the weight decay weight_decay = 1 - 3, and the maximum number of iterations max_epochs = 200.

[0105] Step 6: After the supervised training is completed, integrate the backbone network f(*) of the pre-trained model (with fixed parameters after unsupervised pre-training) and the classification mapping network h(*) (with fixed parameters after supervised training) to obtain the cervical cell image recognition model M*(x) = h(f(x)), and save the final model for actual evaluation and deployment.

[0106] To prove the recognition effect of the cervical cell image recognition model of the present invention, this embodiment uses the test set to evaluate the performance of the model, and calculates indicators such as accuracy, sensitivity, specificity, precision, recall rate, f1 score, and confusion matrix, as shown in Table 1.

[0107] Table 1 Calculation formulas and medical meanings of some indicators

[0108]

[0109] Among them, TP represents True Positive, TN represents True Negative, FN represents False Negative, and FP represents False Positive.

[0110] The method of the present invention brings more accurate classification and grading performance to the TCT cell recognition and diagnosis tasks. Specifically, when only 10 labeled data examples are required for each category, the accuracy of the model can reach as high as 72.43%. This data fully demonstrates that unsupervised pre-training endows the model with a good initial state and strong feature learning ability. During the tuning stage, even when the amount of labeled data is relatively small, the model can still capture key information and make accurate judgments. When the number of examples for each category is further increased to 1000, the accuracy of the model reaches 84.59%. As the number of examples gradually increases, the performance of the model shows a steady upward trend, which not only verifies the good generalization ability of the model but also provides a solid foundation for subsequent practical applications.

[0111] To more comprehensively evaluate the effectiveness of the method proposed in the present invention, this embodiment conducts a detailed comparative experiment, and the results are as Figure 5As shown in Table 2. In this experiment, the test accuracy of the method of the present invention is deeply compared with that of the standard ResNet model and the ViT model (directly using labeled data sets for supervised training) under different sample numbers. When there are only 10 labeled data samples in each category, the standard model faces severe challenges. The small amount of data makes it impossible to directly conduct effective training and it is difficult to give accurate classification results. This highlights the unique advantages of the method of the present invention in scenarios with small amounts of data. As the number of samples gradually increases, when the samples of each category are 100, 1000, 5000, 10000, and 20000 respectively, the test accuracy of each model shows obvious differences. Among them, the test accuracy rates of ResNet are 0.6684, 0.6432, 0.7768, 0.7955, and 0.8171 respectively; the test accuracy rates of ViT are 0.3803, 0.6019, 0.6731, 0.6714, and 0.6947 respectively; and the test accuracy rates achieved by the method of the present invention are 0.7956, 0.8459, 0.8597, 0.8657, and 0.8740 respectively.

[0112] It can be clearly seen from these comparative data that the method proposed by the present invention that effectively combines unsupervised pre-training and supervised tuning has shown significant advantages in TCT cell identification. Unsupervised pre-training enables the model to mine rich potential features from massive unlabeled data, providing a solid foundation; while supervised tuning makes fine adjustments to specific labeled data, so that the model can better adapt to the needs of specific tasks. This combination fully integrates the advantages of both and effectively improves the classification performance of the model.

[0113] At the same time, the multi-model fusion strategy adopted by the method of the present invention has also made an important contribution to the improvement of performance. By fusing multiple models with different structures, the advantages of each model can be fully absorbed, and the robustness and generalization ability of the model can be further improved. Under different data volumes and complex task scenarios, this multi-model fusion method enables the method of the present invention to work more stably, providing a more reliable and efficient solution for complex tasks such as TCT cell identification.

[0114] Table 2. Performance comparison of three models and methods (1000 samples per class)

[0115]

[0116] Table 2 and Figure 6 , Figure 7 and Figure 8It is the performance evaluation of the present invention and two other representative models with 1,000 samples in each category. Through comparison, it can be found that the accuracy rate of the present invention, 0.8606, is significantly higher than that of the comparative models, 0.6432 and 0.6019. However, more importantly, when considering the precision, recall rate, and F1 score of a single category, the present invention has significant advantages. For example, taking the positive category as an example, the precision, recall rate, and F1 score of the present invention are 0.7402, 0.8330, and 0.7839 respectively. While for the comparative model ResNet, the precision, recall rate, and F1 score are 0.4532, 0.8520, and 0.5917 respectively; for ViT, the precision, recall rate, and F1 score are 0.4077, 0.6895, and 0.5124 respectively. The present invention is significantly superior to the comparative models.

[0117] In addition, the method proposed by the present invention only takes 2 minutes and 52 seconds to complete 100 rounds of tuning in the case of 10,000 samples in each category. While for the comparative models ResNet and ViT to complete 100 rounds of training, they respectively require 109 minutes and 25 seconds, and 110 minutes and 5 seconds, approximately 1 hour and 50 minutes. It can be seen that the model update speed of the method of the present invention is more than 36 times that of the comparative methods. In the rapidly developing field of artificial intelligence today, the model update speed plays a crucial role in improving performance. The present invention helps to significantly improve the optimization and update efficiency of the TCT cell-assisted diagnosis system.

[0118] Finally, in the field of clinical practice, the present invention brings all-round application value to TCT cell recognition. With its powerful feature extraction and analysis capabilities, the method of the present invention can quickly and automatically identify different levels of diseased cells and suspected diseased cells from a large number of cervical cell samples. Whether it is early and highly concealed lesions such as ASCUS and LSIL, or complex lesion types such as SIL and GIL classification of clustered cells, they can all be detected and diagnosed, greatly reducing the risk of missed diagnosis in clinical diagnosis. At the same time, it also greatly improves the screening efficiency, provides a clear target for doctors' in-depth diagnosis, and has obtained a large amount of positive feedback.

[0119] Visual analysis was also carried out, including:

[0120] The significance map of Grad-CAM (Gradient-based Class Activation Mapping) for positioning key diagnostic regions, as Figure 9 shown, is used to locate key diagnostic regions;

[0121] Finally, the model parameters or training strategies can be adjusted according to the evaluation results to further optimize the model performance.

[0122] In addition, there are individual differences in the conditions of different patients. Accurately judging the type and severity of cell lesions is the basis for achieving personalized and precise treatment. The high-precision recognition ability of the present invention can accurately judge key information such as the type and degree of cell lesions at the cell level, provide detailed diagnostic data for doctors, so as to customize the most suitable treatment plan for each patient, facilitate personalized treatment, and improve the pertinence and effectiveness of treatment.

[0123] Corresponding to the foregoing embodiments of a method for comparative learning and automatic recognition of cervical cell images, the present invention also provides a system for comparative learning and automatic recognition of cervical cell images. The system is as Figure 10 shown and includes:

[0124] A data preprocessing module, configured to obtain cervical cell images to construct an unlabeled data set and a labeled data set; each sample of the labeled data set includes a corresponding cervical cell image and a lesion label;

[0125] A pre-trained model construction module, configured to construct a pre-trained model, the pre-trained model includes a bimodal backbone network, a splicing fusion layer, and a fusion mapping network connected in sequence; the bimodal backbone network is composed of two branches of a residual network and a vision transformer;

[0126] An unsupervised pre-training module, configured to perform unsupervised pre-training on the constructed pre-trained model based on the unlabeled data set using the contrastive learning SimCLR framework;

[0127] A feature-label data set construction module, configured to, after the unsupervised pre-training ends, use the backbone network of the pre-trained model as a feature extractor to extract features from each cervical cell image of the labeled data set, and construct a feature-label data set;

[0128] A classification mapping network construction module, configured to construct a classification mapping network;

[0129] A supervised tuning module, configured to perform supervised training based on the feature-label data set, and the optimization objective of the supervised training is to minimize the focal cross-entropy loss;

[0130] A model integration module, configured to, after the supervised training ends, integrate the backbone network of the pre-trained model and the classification mapping network to obtain a cervical cell image recognition model;

[0131] An identification module, configured to obtain a cervical cell image to be identified and input it into the cervical cell image recognition model to obtain an identification result.

[0132] For the implementation processes of the functions and roles of each module in the above system, please refer to the implementation processes of the corresponding steps in the above method for details, and will not be elaborated here.

[0133] The present invention also provides an embodiment of an electronic device for implementing a method for contrastive learning and automatic recognition of cervical cell images.

[0134] An electronic device provided by an embodiment of the present invention includes one or more processors for implementing a method for contrastive learning and automatic recognition of cervical cell images in the above embodiment.

[0135] The embodiment of the electronic device of the present invention can be applied to any device with data processing capabilities, and the any device with data processing capabilities can be a device or apparatus such as a computer.

[0136] The device embodiment can be implemented by software, or by hardware or a combination of software and hardware. Taking software implementation as an example, as a logically meaningful device, it is formed by the processor of any device with data processing capabilities reading the corresponding computer program instructions in the non-volatile memory into the memory and running. In terms of hardware, in addition to the processor, memory, network interface, and non-volatile memory, the any device with data processing capabilities where the device in the embodiment is located usually includes other hardware according to the actual functions of the any device with data processing capabilities, which will not be elaborated here.

[0137] For the device embodiment, since it basically corresponds to the method embodiment, the relevant parts can refer to the partial description of the method embodiment. The device embodiments described above are merely illustrative, where the units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they can be located in one place, or distributed to multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the solution of the present invention. Those of ordinary skill in the art can understand and implement it without creative efforts.

[0138] The embodiment of the present invention also provides a computer-readable storage medium, on which a program is stored, and when the program is executed by a processor, it implements a method for contrastive learning and automatic recognition of cervical cell images in the above embodiment.

[0139] The computer-readable storage medium may be an internal storage unit of any device with data processing capabilities described in any of the foregoing embodiments, such as a hard disk or memory. The computer-readable storage medium may also be any device with data processing capabilities, such as a plug-in hard disk, a Smart Media Card (SMC), an SD card, a Flash Card, etc. equipped on the device. Further, the computer-readable storage medium may also include both an internal storage unit of any device with data processing capabilities and an external storage device. The computer-readable storage medium is used to store the computer program and other programs and data required by any device with data processing capabilities, and may also be used to temporarily store data that has been output or is to be output.

[0140] The above description is only a partial embodiment of the present invention and does not limit the present invention in any other form. Those skilled in the art can make various modifications or supplements to the specific embodiments described or use similar methods to replace them. However, any simple modification and equivalent change made to the above embodiments based on the technical essence of the present invention without departing from the content of the technical solution of the present invention still fall within the protection scope of the technical solution of the present invention.

Claims

1. A method for contrast learning and automatic recognition of cervical cell images, characterized in that: Specifically: The cervical cell image to be identified is obtained and input into the cervical cell image recognition model to obtain the recognition result; wherein the cervical cell image recognition model is trained by the following method: Acquire cervical cell images to construct an unlabeled data set and a labeled data set; each sample of the labeled data set includes a corresponding cervical cell image and a lesion label; Constructing a pre-training model, the pre-training model includes a bimodal backbone network, a splicing fusion layer and a fusion mapping network connected in sequence; the bimodal backbone network consists of two branches: a residual network and a visual transformer; Based on the unlabeled dataset, the SimCLR framework of contrastive learning is used to perform unsupervised pre-training on the constructed pre-trained model; After the unsupervised pre-training is completed, the bimodal backbone network of the pre-trained model is used as a feature extractor to extract features from each cervical cell image in the labeled dataset to construct a feature-label dataset; Construct a classification mapping network and perform supervised training based on the feature-label dataset. The optimization goal of supervised training is to minimize the focal cross entropy loss. After the supervised training, the bimodal backbone network and classification mapping network of the pre-trained model are integrated to obtain the cervical cell image recognition model.

2. The method according to claim 1, characterized in that The SimCLR framework of contrastive learning is used to perform unsupervised pre-training on the constructed pre-trained model, specifically: Based on the unlabeled dataset, construct positive and negative sample pairs; the positive sample pair is two different views of the same cervical cell image; the negative sample pair is different cervical cell images; The positive and negative sample pairs are taken as the input of the pre-trained model, and unsupervised pre-training is performed by minimizing the contrast loss of the feature vectors of the positive and negative sample pairs output by the pre-trained model.

3. The method according to claim 2, characterized in that The two different views of the same cervical cell image are generated using data enhancement technology, and the data enhancement technology scheme includes one or more of random geometric transformation, random color perturbation, random image grayscale, and random Gaussian blur.

4. The method according to claim 2, characterized in that: The contrast loss is expressed as follows: ； Where, the subscripts i, i+, and j are the cervical cell image indices, (i, i+) represents a pair of positive samples, and (i, j), i≠j, represents a pair of negative samples; i,j Represents two eigenvectors z i , z j The cosine similarity measure of , z represents the feature vector output by the pre-trained model, sim(*) represents the cosine similarity metric, ||*|| represents the modulus of the vector, and τ is the temperature hyperparameter used to control the sharpness of the distribution. is the indicator function, when The value is 1 when , otherwise it is 0; N represents the number of sample pairs in a batch, and 2N represents the total number of positive and negative samples in a batch.

5. The method according to claim 1, characterized in that The fusion mapping network is composed of a multi-layer perceptron.

6. The method according to claim 1, characterized in that The classification mapping network is composed of a multi-layer perceptron.

7. A cervical cell image comparison learning and automatic recognition system, characterized in that: include: A data preprocessing module, used to obtain cervical cell images to construct an unlabeled data set and a labeled data set; each sample of the labeled data set contains a corresponding cervical cell image and a lesion label; Constructing a pre-training model module, which is used to construct a pre-training model, wherein the pre-training model includes a bi-modal backbone network, a splicing fusion layer and a fusion mapping network connected in sequence; the bi-modal backbone network consists of two branches: a residual network and a visual transformer; Unsupervised pre-training module, which is used to perform unsupervised pre-training on the constructed pre-training model based on unlabeled datasets using the contrastive learning SimCLR framework; A feature-label dataset construction module is used to extract features from each cervical cell image in the labeled dataset using the bimodal backbone network of the pre-trained model as a feature extractor after the unsupervised pre-training is completed, so as to construct a feature-label dataset; Construct a classification mapping network module, which is used to construct a classification mapping network; A supervised tuning module is used for supervised training based on feature-label datasets. The optimization goal of supervised training is to minimize the focal cross entropy loss. A model integration module is used to integrate the bimodal backbone network and the classification mapping network of the pre-trained model to obtain a cervical cell image recognition model after the supervised training is completed; The recognition module is used to obtain the cervical cell image to be recognized and input it into the cervical cell image recognition model to obtain the recognition result.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the method for comparing, learning and automatically identifying cervical cell images as described in any one of claims 1 to 6 is implemented.

9. A storage medium comprising computer executable instructions, which, when executed by a computer processor, implement a cervical cell image comparison learning and automatic recognition method as described in any one of claims 1 to 6.

10. A computer program product comprising a computer program / instructions, characterized in that When the computer program / instruction is executed by a processor, the steps of the method for comparing, learning and automatically identifying cervical cells images described in any one of claims 1 to 6 are implemented.

Images