An unsupervised small sample cervical cell classification method based on self-supervised pre-training

Abstract

The present invention discloses an unsupervised small sample cervical cell classification method based on self-supervised pre-training, which belongs to the field of digital image processing and medical intersection technology. The method provides good initial parameters for the feature extraction network through self-supervised pre-training technology, and then, in the unsupervised small sample learning stage, generates support set and query set samples based on two different data enhancement technologies, and constructs a large number of small sample classification tasks to fine-tune the feature extraction network; finally, a distribution shift loss function is proposed, and the distribution difference between the support set and the query set is gradually expanded during the training process, increasing the difficulty of the small sample classification task, thereby improving the classification accuracy of the algorithm; compared with the existing cervical cell classification method, the present application scheme greatly reduces the data collection and annotation costs, improves the classification accuracy, and effectively reduces the work intensity of pathologists in cervical cell classification.

Description

Technical Field

[0001] The invention relates to an unsupervised small sample cervical cell classification method based on self-supervised pre-training, and belongs to the technical field of digital image processing and medical intersection. Background Art

[0002] Cervical cancer is one of the leading causes of death among women worldwide. Its pathogenesis is clear, and early diagnosis and cure rates are extremely high. Therefore, accurate classification of cervical cell images is crucial for early screening and interventional treatment of cervical cancer. Currently, the most important early screening method is cervical liquid-based cytology (TCT examination, which collects cervical cells and prepares slides to observe the morphology and structure of cells to detect abnormal cells or suspected malignant cells). The screening efficiency is low, and it is easily affected by the subjective influence of pathologists. It has low accuracy and often leads to misdiagnosis and missed diagnosis.

[0003] Although the deep learning-based cervical cell image classification algorithm has achieved automated film reading, the algorithm requires a large amount of training data. However, in actual medical scenarios, due to the privacy of patients and the small number of rare and low-incidence cases, collecting a large number of cervical cell images is not only time-consuming and laborious, but also requires professional annotation of each piece of data, which is extremely costly. Therefore, it is necessary to design a small sample learning algorithm to accurately classify cervical cells based on very little annotated data to assist doctors in disease diagnosis.

[0004] The current small sample learning algorithm is applied to the cervical cell image classification problem. Directly applied to the cervical cell image classification task still has the following problems:

[0005] (1) When the supervised small sample learning algorithm is applied to image classification tasks, an auxiliary dataset containing a large amount of labeled data is required for the meta-training process. However, in the cervical cell classification task, it is difficult to collect a large amount of labeled data, and the labeling cost is high.

[0006] (2) The current unsupervised few-shot learning algorithms construct the few-shot classification task through the same data augmentation method. Since the support set and query set images are too similar, the few-shot classification task is too simple, which makes the network prone to overfitting and the classification accuracy is low (see "Xu H, Wang J, Li H, et al. Unsupervised meta-learning for few-shot learning [J]. Pattern Recognition, 2021, 116 (6): 107951. DOI: 10.1016 / j.patcog.2021.107951."); In addition, since the unsupervised few-shot learning algorithm relies on the algorithm itself to discover patterns or structures from unlabeled data, rather than relying on pre-defined labels to guide the learning process, that is, the image label information cannot be used during meta-training, but a randomly initialized feature extraction network is used, so it is difficult to train an accurate classification model. Summary of the invention

[0007] In order to solve the above-mentioned problems that currently exist, and to break the strict condition that the small sample image classification algorithm requires a large amount of labeled data as a meta-training set, the present invention provides an unsupervised small sample cervical cell classification algorithm based on self-supervised pre-training, which realizes the training of a small sample cervical cell image classification model based on a large amount of unlabeled data.

[0008] The present invention provides an unsupervised small sample cervical cell classification algorithm based on self-supervised pre-training, comprising the following steps:

[0009] Step 1: Divide the small sample cervical cell image dataset into a meta-training set and a meta-testing set, where the meta-training set does not contain the label information of the image, and the meta-testing set contains the label information of the image;

[0010] Step 2: Build a feature extraction network;

[0011] Step 3: Based on the meta-training set, an auxiliary task and contrastive learning task for image rotation direction determination are constructed to perform self-supervised pre-training on the feature extraction network;

[0012] Step 4: Based on the meta-training set, traditional data augmentation technology and automatic data augmentation technology are used to generate support set and query set images respectively, and an N-way K-shot small sample classification task is constructed for unsupervised small sample learning. The feature extraction network is fine-tuned, where N represents the total number of categories of samples in a small sample classification task, K represents the number of support set samples contained in each category, and a distribution shift loss function is proposed;

[0013] Step 5: Use the trained feature extraction network to classify the cervical cells to be classified.

[0014] Optionally, step 3 includes:

[0015] Step 3a, constructing an auxiliary task for determining the image rotation direction;

[0016] Assume that m cervical cell images {x1, x2, …, x m}, and define a set of discrete rotation transformations Any image x i This set of discrete rotation transformations can generate four images Where r represents The rotation angle in; for the transformed image Using feature extraction network f θ (·) Extract abstract features and pass them through the classifier g that determines the rotation angle ω (·) Get the judgment probability of each direction:

[0017]

[0018] Where θ and ω represent the weights of the feature extraction network and the classifier respectively; finally, the loss function of the auxiliary task based on the rotation direction determination is expressed as

[0019]

[0020] Step 3b, construct a contrastive learning task;

[0021] Assume that m cervical cell images {x1, x2, …, x m}, through the traditional data enhancement operator A C ∈A C Get m enhanced images {A C (x1),A C (x2),…,A C (x m )}; The embedded features of the original image and the enhanced image after passing through the feature extraction network are

[0022]

[0023] Through the maximum likelihood estimation method, contrastive learning is regarded as a simple binary classification task; for any image x i , the enhanced image A C (x i ) should be considered as the same instance i, and other images x j (j≠i) are considered as different instances; image A C (x i ) is classified as instance i with a probability of

[0024]

[0025] Where τ represents the temperature coefficient in contrastive learning, which controls the penalty coefficient for difficult negative samples. Generally, a smaller temperature coefficient is used by default. j The probability of not being classified as instance i is

[0026]

[0027] Different images are classified into instances i, which are independent events, so A C (x i ) is classified into instances i and x j The joint probability of not being classified as instance i is

[0028]

[0029] The negative log-likelihood function of the joint probability is

[0030]

[0031] Therefore, the loss of each batch of contrastive learning tasks is the sum of the negative log-likelihood functions of all instances:

[0032]

[0033] In step 3c, the overall loss function of the self-supervised pre-training phase is defined as:

[0034] L ssl =L rot +L con

[0035] After pre-training is completed, save the pre-trained feature extraction network.

[0036] Optionally, step 4 includes:

[0037] Step 4a, construct an N-way K-shot small sample classification task;

[0038] Randomly sample N cervical cell images {x1, x2, …, x N}, each image is considered as a category and randomly assigned a label {(x1,1),(x2,2),…,(x N ,N)}, using K traditional data enhancement operators Generate support set images and use Q automatic data enhancement operators Generate query set images and finally construct N-way K-shot small sample classification task T = (SupportSet, QuerySet):

[0039]

[0040] Step 4b, construct a prototype-based classifier;

[0041] For the constructed N-way K-shot small sample classification task, the support set images and query set images are input into the pre-trained feature extraction network f θ (·) Get their respective embedding features, and calculate the prototype based on the embedding features of each type of support set image:

[0042]

[0043] The nearest neighbor classifier is constructed through the SNS (Semi-Normalized Similarity) similarity metric function. Assume that the query set image of the current small sample classification task is The probability of being classified into class c is

[0044]

[0045] The greater the similarity between the query image embedding feature and the prototype, the greater the possibility of belonging to that class. Therefore, the loss function of a small sample classification task is defined as

[0046]

[0047] Step 4c, calculating the distribution shift loss function;

[0048] First, the maximum mean difference algorithm is used to measure the distribution difference between the support set and the query set in the feature space:

[0049]

[0050] The distribution shift loss function should ensure that during the training process, the distribution difference between the support set and the query set gradually increases, thereby increasing the difficulty of the small sample classification task, and is therefore defined as:

[0051]

[0052] Step 4d, calculate the overall loss function of the unsupervised small sample learning phase as:

[0053] L=L few +λL dol

[0054] Update the parameters of the feature extractor through the gradient of the loss; when all small sample tasks are processed, end the learning process and save the trained feature extractor.

[0055] Optionally, step 5 includes:

[0056] In the meta-testing stage, based on the trained feature extraction network and the small sample tasks of the meta-testing set, the prototypes are calculated using the embedded features of the support set samples and the nearest neighbor classifier is constructed. The query set samples are classified according to the distance from the embedded features of the query set samples to each prototype, and the average classification accuracy on all small sample tasks is calculated.

[0057] Optionally, K traditional data augmentation operators include: scaling, random cropping, random rotation, and random adjustment of image attributes.

[0058] Optionally, the image attributes include image brightness, contrast, saturation, color temperature, hue, transparency, etc.

[0059] Optionally, the Q automatic data enhancement operators adopt a sub-strategy of the automatic data enhancement technology migrated from the ImageNet dataset; the automatic data enhancement technology adopts the best enhancement strategy searched by reinforcement learning ideas.

[0060] Optionally, sub-strategies include image cropping and flipping, color transformation, geometric transformation, etc.

[0061] The beneficial effects of the present invention are:

[0062] 1. The unsupervised small sample cervical cell classification method proposed in the present invention breaks the strict conditional restrictions of supervised learning and does not require image label information during meta-training. Compared with the supervised small sample learning algorithm, it greatly reduces the collection and annotation costs of cervical cell images.

[0063] 2. The present invention adopts a self-supervised learning algorithm to create supervisory information from the inherent structure and information of the cervical cell image itself for pre-training, which not only provides good initial parameters for the feature extraction network, but also learns more prior knowledge and more general feature representations, significantly improving the classification accuracy of cervical cells.

[0064] 3. In order to address the problem of network overfitting and low classification accuracy caused by the support set and query set images being too similar in the unsupervised small sample learning algorithm, the present invention, on the one hand, adopts traditional data enhancement technology and automatic data enhancement technology to generate support set and query set images to construct small sample classification tasks, thereby increasing the diversity of support set and query set samples. On the other hand, a distribution shift loss function is proposed, which gradually expands the distribution difference of the support set and query set during meta-training by maximizing the distribution shift loss, thereby further increasing the difficulty of the small sample classification task, effectively preventing network overfitting, and improving classification accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0065] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0066] Figure 1 It is the overall architecture diagram of the present invention.

[0067] Figure 2 This is the structural diagram of the feature extraction network.

[0068] Figure 3 Flowchart for self-supervised pre-training.

[0069] Figure 4 Flowchart for unsupervised few-shot learning

[0070] Figure 5 This is an example image of the small sample Herlev cervical cell image dataset. DETAILED DESCRIPTION

[0071] To make the objectives, technical solutions and advantages of the present invention more clear, the embodiments of the present invention will be further described in detail below with reference to the accompanying drawings.

[0072] Embodiment 1:

[0073] This embodiment provides an unsupervised small sample cervical cell classification method based on self-supervised pre-training, including:

[0074] Step 1: Divide the small sample cervical cell image dataset into a meta-training set and a meta-testing set, where the meta-training set does not contain the label information of the image, and the meta-testing set contains the label information of the image;

[0075] Step 2: Build a feature extraction network;

[0076] The feature extractor is built based on a convolutional neural network, specifically a Conv_64F network, which contains 4 convolutional blocks consisting of a convolutional layer, a batch normalization layer BN, and a Leaky ReLU activation layer, and each convolutional block is followed by a maximum pooling layer.

[0077] Step 3: Based on the meta-training set, an auxiliary task and contrastive learning task for image rotation direction determination are constructed to perform self-supervised pre-training on the feature extraction network;

[0078] Self-supervised pre-training technology is used to create supervised information from the inherent structure and information of cervical cell images for pre-training, providing good initial parameters for the feature extraction network and learning more prior knowledge and more general feature representations.

[0079] Step 4: Based on the meta-training set, traditional data augmentation technology and automatic data augmentation technology are used to generate support set and query set images, and an N-way K-shot small sample classification task is constructed for unsupervised small sample learning. The feature extraction network is fine-tuned, where N represents the total number of categories of samples in a small sample classification task, K represents the number of support set samples contained in each category, and a distribution shift loss function is proposed;

[0080] Two different data augmentation methods are used to generate support sets and query sets for small sample classification tasks, making the images between the two sets more different. In addition, a distribution shift loss function is used to gradually increase the distribution difference between the support set and the query set during the training process, thereby increasing the difficulty of the small sample classification task and promoting the feature extraction network to learn more discriminative features from cervical cell images, preventing network overfitting, and further improving classification accuracy.

[0081] Step 5: Use the trained feature extraction network to classify the cervical cells to be classified.

[0082] Embodiment 2

[0083] This embodiment provides an unsupervised small sample cervical cell classification method based on self-supervised pre-training, please refer to Figure 1 , the method comprises the following steps:

[0084] Step 1: divide the small sample cervical cell image dataset into a meta-training set and a meta-test set. The specific division method is:

[0085] 1a) The small sample cervical cell image dataset is used by D total Denote that the meta-training set is D train Denote that the meta-test set is represented by D test express;

[0086] 1b) Divide a part of the small sample cervical cell image dataset into a meta-training set D train , the remaining part is D test , where the categories contained in the meta-training set and the meta-test set cannot intersect, and the meta-training set does not contain the label information of the image, and the meta-test set contains the label information of the image.

[0087] Step 2: Build a feature extraction network based on convolutional neural network. The network structure is as follows: Figure 2 As shown in the figure, a Conv_64F network is specifically used, which includes 4 convolution blocks consisting of a convolution layer, a batch normalization layer, and a Leaky ReLU activation layer, and each convolution block is followed by a maximum pooling layer.

[0088] Step 3: Based on the meta-training set, auxiliary tasks and contrastive learning tasks for image rotation direction determination are constructed, and self-supervised pre-training of the feature extraction network is performed, such as Figure 3 As shown, the specific steps are:

[0089] 3a) Construct an auxiliary task for image rotation direction determination;

[0090] Assume that m cervical cell images {x1, x2, …, x m}, and define a set of discrete rotation transformations Any image x i This set of discrete rotation transformations can generate four images Where r represents The rotation angle in; for the transformed image Using feature extraction network f θ (·) Extract abstract features and pass them through the classifier g that determines the rotation angle ω (·) Get the judgment probability of each direction:

[0091]

[0092] Where θ and ω represent the weights of the feature extraction network and the classifier respectively, and exp represents the exponential function. represents the judgment probability of each rotation angle; finally, the loss function of the auxiliary task based on the rotation direction judgment is expressed as:

[0093]

[0094] 3b) Constructing contrastive learning tasks;

[0095] Assume that m cervical cell images {x1, x2, …, x m}, through the traditional data enhancement operator A C ∈A C Get m enhanced images {A C (x1),A C (x2),…,A C (x m )}; The embedded features of the original image and the enhanced image after the feature extraction network are:

[0096]

[0097] Through the maximum likelihood estimation method, contrastive learning is regarded as a simple binary classification task; for any image x i , the enhanced image A C (x i ) should be considered as the same instance i, and other images xj (j≠i) are considered as different instances; image A C (x i ) is classified as instance i with a probability of:

[0098]

[0099] Where τ represents the temperature coefficient in contrastive learning, which controls the penalty coefficient for difficult negative samples. Generally, a smaller temperature coefficient is used by default. j The probability of not being classified as instance i is:

[0100]

[0101] Different images are classified into instances i, which are independent events, so A C (x i ) is classified into instances i and x j The joint probability of not being classified as instance i is:

[0102]

[0103] The negative log-likelihood function of the joint probability is:

[0104]

[0105] Therefore, the loss of each batch of contrastive learning tasks is the sum of the negative log-likelihood functions of all instances:

[0106]

[0107] 3c) The overall loss function of the self-supervised pre-training phase is defined as:

[0108] L ssl =L rot +L con

[0109] After pre-training is completed, save the trained feature extraction network.

[0110] Step 4: Based on the meta-training set, traditional data augmentation technology and automatic data augmentation technology are used to generate support set and query set images respectively, and an N-way K-shot small sample classification task is constructed for unsupervised small sample learning. The feature extraction network is fine-tuned, where N represents the total number of categories of samples in a small sample classification task, K represents the number of support set samples contained in each category, and a distribution shift loss function is proposed, such as Figure 4 As shown, the specific steps are:

[0111] 4a) Construct N-way K-shot small sample classification task;

[0112] Randomly sample N cervical cell images {x1, x2, …, x N}, each image is considered as a category and randomly assigned a label {(x1,1),(x2,2),…,(x N ,N)}, using K traditional data enhancement operators Generate support set images and use Q automatic data enhancement operators Generate query set images and finally construct N-way K-shot small sample classification task T = (SupportSet, QuerySet):

[0113]

[0114] Traditional data enhancement operators include: scaling, random cropping, random rotation, and random adjustment of image attributes such as brightness and contrast.

[0115] The automatic data enhancement operator adopts a sub-strategy of the automatic data enhancement technology migrated from the ImageNet dataset; the automatic data enhancement technology uses reinforcement learning ideas to search for the best enhancement strategy, such as image cropping and flipping, color transformation, geometric transformation, etc.

[0116] 4b) Building a prototype-based classifier;

[0117] For the constructed N-way K-shot small sample classification task, the support set images and query set images are input into the pre-trained feature extraction network f θ (·) Get their respective embedding features, and calculate the prototype based on the embedding features of each type of support set image:

[0118]

[0119] The nearest neighbor classifier is constructed through the SNS (Semi-Normalized Similarity) similarity metric function. Assume that the query set image of the current small sample classification task is The probability of being classified into class c is:

[0120]

[0121] The greater the similarity between the query image embedding feature and the prototype, the greater the possibility of belonging to that class. Therefore, the loss function of a small sample classification task is defined as:

[0122]

[0123] 4c) Calculate the distribution shift loss function;

[0124] First, the maximum mean difference algorithm is used to measure the distribution difference between the support set and the query set in the feature space:

[0125]

[0126] Where φ represents the mapping function, which maps the data to the regenerated Hilbert space.

[0127] The distribution shift loss function should ensure that during the training process, the distribution difference between the support set and the query set gradually increases, thereby increasing the difficulty of the small sample classification task, and is therefore defined as:

[0128]

[0129] 4d) Calculate the overall loss function of the unsupervised small sample learning stage as:

[0130] L=L few +λL dol

[0131] Update the parameters of the feature extractor through the gradient of the loss; when all small sample tasks are processed, end the learning process and save the trained feature extractor.

[0132] The trained feature extractor is then used to classify the cervical cell images to be classified.

[0133] In order to measure the classification accuracy of the method of the present application, this method is further based on the small sample classification task on the meta-test set. Based on the trained feature extraction network, the prototype is calculated using the embedded features of the support set samples and a nearest neighbor classifier is constructed. The query set samples are classified according to the distance from the embedded features of the query set samples to each prototype, and the average classification accuracy on all small sample tasks is calculated.

[0134] The effect of this method can be further illustrated by the following simulation experiment:

[0135] 1. Experimental conditions and methods

[0136] 1) Hardware requirements:

[0137] ①CPU: Two Inter(R)Xeon(R)Silver 4210R CPU@2.40GHz 128G;

[0138] ②GPU: NVIDIA Geforce RTX 3090 24G.

[0139] 2) Software environment:

[0140] ①Programming language: Python 3.8;

[0141] ②Deep learning framework: Pytorch 1.9.0.

[0142] 3) Experimental methods:

[0143] ① Random-ProtoNet, please refer to the introduction in (Antoniou A, Storkey A. Assume, augmentation and learn: Unsupervised few-shot meta-learning via random labels and data augmentation [J]. arXiv preprint arXiv: 1902.09884, 2019.);

[0144] ②ProtoTransfer, please refer to the introduction in (Medina C, Devos A, Grossglauser M. Self-supervised prototypical transfer learning for few-shot classification[J].arXiv preprint arXiv:2006.11325,202);

[0145] ③AAL, please refer to the introduction in (Antoniou A, Storkey A. Assume, augmentation and learn: Unsupervised few-shot meta-learning via random labels and data augmentation [J]. arXiv preprint arXiv:1902.09884, 2019.);

[0146] ④CACTUs, please refer to the introduction in (Hsu K, Levine S, Finn C. Unsupervised learning via meta-learning[J].arXiv preprint arXiv:1810.02334,2018);

[0147] ⑤UMTRA, please refer to the introduction in (Khodadadeh S, Boloni L, Shah M. Unsupervised meta-learning for few-shot image classification [J]. Advances in Neural Information Processing Systems, 2019, 32: 1-11);

[0148] ⑥SimCLR, please refer to the introduction in (Chen T, Kornblith S, Norouzi M, et al. A simple framework for contrastive learning of visual representations [C]. Proceedings of the 37th International Conference on Machine Learning. New York: PMLR, 2020: 1597-1607);

[0149] ⑦LF2CS, please refer to the introduction in (Li S, Liu F, Hao Z, et al. Unsupervised few-shot image classification by learning features into clustering space [C]. Proceedings of the 19th European Conference on Computer Vision. Cham: Springer, 2022: 420-436.);

[0150] ⑨The method of the present invention

[0151] 4) Dataset

[0152] The cervical cell dataset used in this application is the public Herlev small sample cervical cell image dataset, such as Figure 5 As shown, the image set was produced by a digital camera and a microscope at Herlev University Hospital. The image resolution is 0.201 μm / pixel. Herlev's dataset contains a total of 917 images, each of which contains a cervical cell, divided into 7 categories. These 7 categories belong to two major categories: categories 1 to 3 are normal, and categories 4 to 7 are abnormal.

[0153] If you do not consider the cost of data collection, you can also choose to collect the data set yourself.

[0154] 5) Small sample classification task:

[0155] ①2-way K-shot task, that is, the support set contains 2 categories, each category contains K images, where K is 3, 5, or 10;

[0156] ②3-way K-shot task, that is, the support set contains 3 categories, each category contains K images, where K is 3, 5, or 10.

[0157] 6) Evaluation indicators

[0158] The average classification accuracy of all few-shot tasks on the meta-test set is calculated and a confidence interval of η = 95% is given.

[0159] 2. Simulation content and results

[0160] The feature extraction network of the method of the present invention selects Conv64F, and the input image is scaled to 84×84.

[0161] In the self-supervised pre-training stage, 40 cervical cell images were randomly sampled from the meta-training set in each iteration to construct the auxiliary task of image rotation direction determination and the comparative learning task. The entire pre-training process was iterated 20,000 times in total, using the Adam optimizer, with the initial learning rate set to 0.01, and the learning rate was halved at the 10,000th and 16,000th iterations.

[0162] In the unsupervised small sample learning stage, traditional data enhancement techniques and automatic data enhancement techniques are used to construct 5000 N-way K-shot small sample classification tasks from the meta-training set. Among them, the number of categories N is 2 or 3, the number of samples per category in the support set K = 3, 5 or 10, and the number of samples per category in the query set Q is set to 15. The Adam optimizer is used to fine-tune the pre-trained feature extraction network, and the learning rate is 0.0002.

[0163] In the meta-test phase, based on the trained feature extraction network, 400 small sample classification tasks are randomly sampled from the meta-test set, and the test is repeated 10 times to obtain more accurate results. Finally, the average classification accuracy of the 95% confidence interval is calculated and used as the performance evaluation index of the algorithm. The method of the present invention is trained from scratch in an end-to-end manner and does not require fine-tuning in the test phase.

[0164] The classification results of different methods on the Herlev small sample cervical cell image dataset are shown in Table 1. The bold numbers in the table represent the best results. Among them, the Random-ProtoNet algorithm randomly samples samples in the meta-training set, and then directly randomly assigns pseudo labels to construct small sample classification tasks; ProtoTransfer, AAL, and UMTRA are algorithms based on data enhancement; CACTUs and LF2CS are two clustering-based algorithms; SimCLR is an algorithm based on self-supervised learning. The data in the table show that the algorithm of the present invention has achieved the highest classification accuracy in all classification tasks, and is at least 2 percentage points higher than the algorithm with the second highest classification accuracy. In particular, the classification accuracy on the 2-way 5-shot and 2-way 10-shot classification tasks reached 90.73% and 92.97%, which fully reflects the excellent performance of the algorithm of the present invention.

[0165] Algorithms CACTUs and LF2CS use clustering methods to generate pseudo labels and construct small sample classification tasks based on pseudo labels. However, the classification performance of these algorithms is highly dependent on the quality of pseudo labels, resulting in lower classification accuracy than the algorithm of the present invention. In addition, since the Random-ProtoNet algorithm constructs small sample classification tasks through random sampling and random assignment of pseudo labels, its feature extractor cannot learn more effective information from a large number of irregular small sample classification tasks, so the classification accuracy is also significantly lower than the algorithm of the present invention. The method of the present application adopts a two-stage training technology and combines self-supervised pre-training with unsupervised small sample learning to significantly improve the classification performance. In the self-supervised pre-training stage, not only good initial parameters are learned for the feature extraction network, but also the learning of more prior knowledge and more general features is promoted. In the subsequent unsupervised small sample learning stage, the feature extractor is fine-tuned through a large number of small sample classification tasks to enhance its ability to extract discriminative features. Therefore, compared with the single-stage unsupervised small sample learning algorithms ProtoTransfer and UMTRA, the algorithm of the present invention can achieve higher classification accuracy.

[0166] Table 1 Comparison of results of different methods on the Herlev small sample cervical cell image dataset

[0167]

[0168]

[0169] The present invention provides an unsupervised small sample cervical cell classification algorithm based on self-supervised pre-training. First, an auxiliary task of determining the rotation direction of cervical cell images is constructed on a meta-training set, and a data enhancement technology is used to generate positive and negative samples of the image for comparative learning, and a good initial parameter is provided for a feature extraction network through the self-supervised pre-training technology. Subsequently, in the unsupervised small sample learning stage, support set and query set samples are generated based on two different data enhancement technologies, and a large number of small sample classification task fine-tuning feature extraction networks are constructed. Finally, a distribution shift loss function is proposed, and the distribution difference of the support set and the query set is gradually expanded during the training process, increasing the difficulty of the small sample classification task, thereby improving the classification accuracy of the algorithm. The method of the present invention can train a small sample classification model without the label information of the image during meta-training, accurately and quickly identify the cervical cell category based on very few labeled samples during meta-testing, greatly reducing the data collection and labeling costs, and having higher accuracy than the existing unsupervised small sample classification algorithm, which can effectively reduce the work intensity of pathologists for cervical cell classification, is suitable for application in engineering practice, and has broad application value and market prospects.

[0170] Some steps in the embodiments of the present invention may be implemented using software, and the corresponding software program may be stored in a readable storage medium, such as a CD or a hard disk.

[0171] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. An unsupervised small sample cervical cell classification method based on self-supervised pre-training, characterized in that: The method comprises: Step 1, dividing the small sample cervical cell image dataset into a meta-training set and a meta-testing set, wherein the meta-training set does not contain the label information of the image, and the meta-testing set contains the label information of the image; Step 2: Building a feature extraction network based on a convolutional neural network; Step 3: construct auxiliary tasks and contrastive learning tasks for image rotation direction determination based on the meta-training set, and perform self-supervised pre-training on the feature extraction network built in step 2; Step 4: Based on the meta-training set, traditional data augmentation technology and automatic data augmentation technology are used to generate support set and query set images, and an N-way K-shot small sample classification task is constructed for unsupervised small sample learning, and the feature extraction network is fine-tuned; where N represents the total number of categories of samples in a small sample classification task, K represents the number of support set samples contained in each category, and a distribution shift loss function is proposed; Step 5, using the trained feature extraction network to classify the cervical cells to be classified; The step 3 comprises: Step 3a, constructing an auxiliary task for determining the image rotation direction; Randomly sample m cervical cell images {x1,x2,…,x m }, and define a set of discrete rotation transformations Any image x i This set of discrete rotation transformations can generate four images Where r represents The rotation angle in; for the transformed image Using feature extraction network f θ (·) Extract abstract features and pass them through the classifier g that determines the rotation angle ω (·) Get the judgment probability P of each direction i,r ; The loss function based on the auxiliary task of rotation direction determination is expressed as: Step 3b, construct a contrastive learning task; Randomly sample m cervical cell images {x1,x2,…,x m }, through the traditional data enhancement operator A C ∈A C Get m enhanced images {A C (x1),A C (x2),…,A C (x m )}; The original image x i And the enhanced image A C (x i ) After passing through the feature extraction network, the corresponding embedded features are obtained; Contrastive learning is considered as a binary classification task through the maximum likelihood estimation method; for any image x i , the enhanced image A C (x i ) should be considered as the same instance i, and other images x j (j≠i) are considered as different instances; A is calculated based on the embedded features. C (x i ) is classified into instances i and x j The joint probability P of not being classified as instance i i ; Compute the negative log-likelihood function of the joint probability: Among them, P(i|A C (x i )) represents image A C (x i ) is classified as instance i, P(i|x j ) represents the image x j The probability of being classified as instance i; The sum of the negative log-likelihood functions of all instances is calculated as the loss for each batch of the contrastive learning task: Step 3c, perform self-supervised pre-training on the feature extraction network. The overall loss function of the self-supervised pre-training stage is defined as: L ssl =L rot +L con After pre-training is completed, save the trained feature extraction network; The step 4 comprises: Step 4a, construct an N-way K-shot small sample classification task; Randomly sample N cervical cell images {x1, x2, …, x N }, each image is considered as a category and randomly assigned a label {(x1,1),(x2,2),…,(x N ,N)}, using K traditional data enhancement operators Generate support set images and use Q automatic data enhancement operators Generate query set images, and finally construct N-way K-shot small sample classification task T = (SupportSet, QuerySet); Step 4b, construct a prototype-based classifier; For the constructed N-way K-shot small sample classification task, the support set images and query set images are input into the pre-trained feature extraction network f θ (·) Get their respective embedding features, and calculate the prototype based on the embedding features of each type of support set image: in, Represent the embedded features of each class of support set images; build a nearest neighbor classifier, given the query set image of the current small sample classification task The probability of being classified into class c is The loss function of a small sample classification task is defined according to the probability that the query set image of the small sample classification task is classified into the cth class: Step 4c, calculating the distribution shift loss function; The maximum mean difference algorithm is used to measure the distribution difference MMD (SupportSet, QuerySet) between the support set and the query set in the feature space; The inverse of the distribution difference between the support set and the query set in the feature space is used as the distribution shift loss function value: Step 4d, calculate the overall loss function of the unsupervised small sample learning phase as: L=L few +λL dol Update the parameters of the feature extractor through the gradient of the loss; when all small sample tasks are processed, end the learning process and save the trained feature extractor.

2. The method according to claim 1, characterized in that The K traditional data enhancement operators include: scaling, random cropping, random rotation, and random adjustment of image attributes.

3. The method according to claim 1, characterized in that The Q automatic data enhancement operators adopt sub-strategies of the automatic data enhancement technology migrated from the ImageNet dataset; the automatic data enhancement technology adopts the best enhancement strategy searched by reinforcement learning ideas.

4. The method according to claim 1, characterized in that: The distribution difference MMD(SupportSet,QuerySet) between the support set and the query set in the feature space is: Where φ represents the mapping function, which maps the data to the regenerated Hilbert space.

5. The method according to claim 4, characterized in that The classifier g after determining the rotation angle ω (·) The judgment probability P of each direction obtained i,r for: Where exp represents the exponential function, represents the judgment probability of each rotation angle.

6. The method according to claim 5, characterized in that The small sample cervical cell image dataset adopts a public dataset or a self-collected dataset.

Images