Cervical cancer detection method based on simsiam mae self-supervised model

Abstract

The application provides a cervical cancer detection method based on a Simsiam Mae self-supervised model, comprising the following steps: S1, acquiring and preprocessing a cervical cell image; S2, data preprocessing; S3, Simsiam Mae self-supervised model training, wherein the Simsiam Mae self-supervised model uses a contrast learning method to train an encoding path, a projection layer and a prediction layer to extract low-frequency information features of the cell image; a masked image modeling method is used to train the encoding path and a decoding path to extract high-frequency information features of the cell image; S4, downstream task training; and S5, using the trained Simsiam Mae self-supervised model to detect cervical cancer. The application greatly reduces the dependence on artificial labeled data, saves time and cost, and improves the classification accuracy of cervical cell images.

Description

TECHNICAL FIELD

[0001] The application relates to the technical field of medical image processing, and in particular to a cervical cancer detection method based on a Simsiam Mae self-supervised model. BACKGROUND

[0002] With the development of electronic computer technology and the field of artificial intelligence, image classification and recognition through computer algorithms can be initially realized, and image analysis and processing technology is generally used in medical imaging for object detection and classification. However, when the number of labeled images in the data set is too small, the trained artificial intelligence algorithm will often have the phenomenon of overfitting, making the model and algorithm perform sub-optimally.

[0003] Cervical cancer is the fourth most common cancer type in women and one of the most common cancers, which seriously threatens the lives and health of women, and HPV infection is the main factor causing cervical cancer. Compared with conventional Pap smear detection, liquid-based thin-layer cell detection significantly improves the specimen satisfaction and cervical abnormal cell detection rate, and thus has become the most widely used cervical lesion screening technology in the world. However, due to the very large size of the liquid-based thin-layer cell detection image, and the need to locate and classify the dispersed cells in the image, doctors need to annotate a large number of dispersed cervical cells in the digital image to supervise the training of the classification network to obtain better performance, and this process is time-consuming and labor-intensive. Existing automated cervical cell classification and detection methods cannot achieve satisfactory results on liquid-based thin-layer cell detection images. For example, Chinese patent CN114170198A discloses a liquid-based thin-layer cell smear digital pathology image detection method based on DRSN, which performs variable step length sliding window sampling on the image file, filters the sampled regions to screen out focal regions, and then uses a deep learning model to obtain the confidence of the focal regions, and finally determines the degree of lesion of the region according to the confidence threshold. Such a method needs to use a variable step length sliding window to sample the dispersed cells, which is low in efficiency, and the lesion result depends on the selection of the focal region and the size of the manually set confidence threshold, so the missed detection rate is high.

[0004] In addition, the document "MobileNet-Light: A Lightweight TCT Image Classification Model for Cervical Cancer" Pan, X., Bao, C., & Chen, D. (2023, June). MobileNet-Light: A Lightweight TCT Image Classification Model for Cervical Cancer. In 2023 International Joint Conference on Neural Networks (IJCNN) (pp. 1-7). IEEE. discloses that the accuracy it achieves on the liquid-based thin-layer cell detection dataset and the Pap smear dataset, expressed by classification accuracy, is 82.24% and 93.72%, respectively. The classification accuracy achieved by this method on the liquid-based thin-layer cell detection dataset is far from meeting the actual demand. SUMMARY

[0005] Based on the foregoing prior art defects, the present application provides a cervical cancer detection method based on a Simsiam Mae self-supervised model, which has high accuracy.

[0006] To achieve the above-mentioned purpose, the present application provides a cervical cancer detection method based on a Simsiam Mae self-supervised model, comprising the following steps:

[0007] S1, a step of acquiring and preprocessing cervical cell images, in combination with pre-provided labeling information, extracting cervical cell images from liquid-based thin-layer cell detection images, and dividing the extracted cervical cell images into a training dataset and a test dataset;

[0008] S2, a data preprocessing step, performing two different and random data augmentation operations on each cervical cell image in the training dataset to generate two different images, and inputting them into the Simsiam Mae self-supervised model;

[0009] S3, a Simsiam Mae self-supervised model training step, the Simsiam Mae self-supervised model uses a contrastive learning method to train the encoding path, the projection layer and the prediction layer to extract the low-frequency information features of the cell image, and learns the global information and cell shape features of the image; using a masked image modeling method to train the encoding path and the decoding path to extract the high-frequency information features of the cell image, and learning the local information and texture features in the image;

[0010] S4, a downstream task training step, using the test dataset to fine-tune the parameters of the encoding path of the above-mentioned Simsiam Mae self-supervised model, completing the classification of cervical cells, and obtaining the trained Simsiam Mae self-supervised model;

[0011] S5, using the trained Simsiam Mae self-supervised model to detect cervical cancer.

[0012] In an embodiment, the step S1 comprises the following steps:

[0013] S11, converting all liquid-based thin-layer cell detection images to images in the RGB color space;

[0014] S12, extracting cervical cell images from the above-mentioned images in the RGB color space according to the pre-provided cervical cell type and pixel value coordinate labeling information of the cervical cell image;

[0015] S13, filling the extracted cervical cell images, filling the length and width of all images to 512 pixel values to unify the size of all images;

[0016] S14, dividing the extracted cervical cell images into data sets according to the cervical cell type, randomly selecting part of the images as training data sets, and the other part as test data sets.

[0017] In an embodiment, the data enhancement operation in the step S2 comprises the following steps:

[0018] S211, random cropping and scaling, randomly cropping part of the image and adjusting it to a given size;

[0019] S212, color jitter, randomly changing the brightness, contrast, saturation and hue of the image;

[0020] S213, random grayscale, randomly converting the image to grayscale;

[0021] S214, random horizontal flip, randomly horizontally flipping the given image with a given probability;

[0022] S215, normalization, normalizing the image with the mean and standard deviation.

[0023] In an embodiment, the step S211 is to randomly crop a region in the image, the size of the randomly cropped region is 0.7 to 0.9 of the original image size, and the size of the randomly cropped region is adjusted to

RGB, 512, 512, 3

[0024] The step S212 is to adjust the brightness, contrast, saturation and hue of the image to 0.4, 0.4, 0.4 and 0.1 of the original image, respectively, with a probability of 80%;

[0025] The step S213 is that the operation randomly converts the color image into a grayscale image with a probability of 20%;

[0026] The step S214 is that the operation randomly flips the original image with a probability level of 50%;

[0027] The step S215 is that the operation normalizes the image with the mean and standard deviation, and the normalization formula is as follows,

[0028]

[0029] wherein Output is the normalized image output, Input is the original image input, mean is 【0.485, 0.456, 0.406】, and std is 【0.229, 0.224, 0.225】.

[0030] In an embodiment, the step S3 specifically comprises the following steps:

[0031] S31, inputting the two images obtained after the two random data augmentation operations into the encoding path of the Simsiam Mae self-supervised model, the encoding path performing feature extraction operations on the two images respectively, and outputting two initial feature tensors, and inputting the output initial feature tensors into the projection layer;

[0032] S32, after the two initial feature tensors pass through the projection layer, a first projection feature tensor and a second projection feature tensor are output, and the first projection feature tensor and the second projection feature tensor are input into the prediction layer respectively;

[0033] S33, after the first projection feature tensor and the second projection feature tensor pass through the prediction layer, a first prediction feature tensor and a second prediction feature tensor are output;

[0034] S34, using a contrastive learning method to calculate a loss value;

[0035] S35, inputting the first prediction feature tensor and the second prediction feature tensor into the decoding path respectively, outputting a reconstructed image after passing through the decoding path, and using a masked image modeling method to calculate a loss value;

[0036] S36, using a back propagation algorithm to obtain a gradient, using a gradient descent algorithm to obtain a vector composed of a plurality of partial derivatives, and then updating the model parameters through the vectors.

[0037] In an embodiment, the S31 comprises the following steps:

[0038] S311, input the image after the random data enhancement operation into the Patch Embedding layer in the encoding path, convert the two-dimensional image into a series of one-dimensional path embeddings, and form a patch embeddings tensor;

[0039] S312, add position encoding information to the patch embeddings tensor to provide position information for each path embeddings tensor;

[0040] S313, perform a mask operation on the patch embeddings tensor to mask a part of the patch embeddings tensor, and add a CLS token vector to the unmasked patch embeddings tensor;

[0041] S314, input the unmasked patch embeddings tensor into the Vision Transformer structure for feature extraction, and output two initial feature tensors;

[0042] S315, input the two output initial feature tensors into the projection layer.

[0043] In an embodiment, the step S32 includes the following steps:

[0044] S321, input the two initial feature tensors into the first convolutional layer in the projection layer;

[0045] S322, perform a normalization operation on the output of the first convolutional layer;

[0046] S323, input the result of the normalization operation into the second convolutional layer;

[0047] S324, perform a normalization operation on the output of the second convolutional layer;

[0048] S325, input the result of the normalization operation into the third convolutional layer;

[0049] S326, perform a layer normalization operation on the output of the third convolutional layer;

[0050] S327, output the first projection feature tensor and the second projection feature tensor, and input the first projection feature tensor and the second projection feature tensor into the prediction layer, respectively;

[0051] The step S33 includes the following steps:

[0052] S331, input the first projection feature tensor and the second projection feature tensor obtained above into a first convolutional layer in the prediction layer;

[0053] S332, normalize the result output by the first convolutional layer;

[0054] S333, input the result of the normalization operation into a second convolutional layer;

[0055] S334, output the first prediction feature tensor and the second prediction feature tensor.

[0056] In an embodiment, the step S34 comprises the following steps:

[0057] S341, multiply the gradient-truncated second projection feature tensor with the first prediction feature tensor to obtain a first cross-correlation matrix, and multiply the gradient-truncated first projection feature tensor with the second prediction feature tensor to obtain a second cross-correlation matrix;

[0058] S342, respectively divide the first cross-correlation matrix and the second cross-correlation matrix by the value of the batch size;

[0059] S343, calculate the loss value of the contrastive learning method using the following formula,

[0060]

[0061] where i indexes the dimension of the first cross-correlation matrix or the second cross-correlation matrix, C1 and C2 represent the first cross-correlation matrix and the second cross-correlation matrix, respectively;

[0062] C 1ii and the calculation formula of C 2ii is as follows,

[0063]

[0064] where i indexes the dimension of the first cross-correlation matrix or the second cross-correlation matrix, z is the output result of the projection layer, the first projection feature tensor and the second projection feature tensor, p is the output result of the prediction layer, the first prediction feature tensor and the second prediction feature tensor, and b indexes the batch sample.

[0065] In an embodiment, the step S35 comprises the following steps:

[0066] S351, input the first prediction feature tensor and the second prediction feature tensor into a first convolutional layer in the decoding path, respectively;

[0067] S352, splice the output of the convolutional layer with the concealed patch embedding tensor;

[0068] S353, adding position encoding information to the above spliced patch embedding tensor;

[0069] S354, inputting the above patch embedding tensor with position encoding information into the Vision transformer structure;

[0070] S355, inputting the output result of the above Vision transformer structure into the second convolutional layer in the decoding path;

[0071] S356, removing the CLS token vector in the output result of the above convolutional layer, and outputting the reconstructed image (an example of the reconstructed image); Figure 5

[0072] S357, performing a normalization operation on the output reconstructed image;

[0073] S358, calculating the loss value of the masked image modeling method using the mean square error formula, and the calculation formula is as follows,

[0074]

[0075] Where n is the number of patch embedding tensors that are masked, pred is the reconstructed image, and target is the original image.

[0076] In an embodiment, the S4 comprises the following steps:

[0077] S41, constructing a cervical cell image dataset;

[0078] S42, using the encoding path of the above Simsiam Mae self-supervised model as the classification model for this task;

[0079] S43, loading the pre-training parameters of the Simsiam Mae self-supervised model;

[0080] S44, using the above cervical cell image dataset to fine-tune the parameters of the classification model loaded with the pre-training parameters.

[0081] ​The beneficial effects of the present application mainly embody: based on a deep learning model, combining the ways of contrastive learning and masked image modeling, learning the low-frequency signal features and high-frequency signal features in the cervical cell image respectively, so as to achieve the purpose of self-supervised learning. The present application can effectively extract the feature information in the cervical cell image dataset, accurately recognize and detect the discrete distributed cells in the image; compared with the traditional supervised learning algorithm, the dependence on artificial labeled data is greatly reduced, the time and cost of work are saved, and the classification accuracy of the cervical cell image is improved. BRIEF DESCRIPTION OF DRAWINGS

[0082] The drawings described herein are for illustrative purposes only and are not intended to limit the scope of the present disclosure in any way. In addition, the shapes and scale sizes of the components in the drawings are only illustrative, used to help understand the present application, and are not specific limitations on the shapes and scale sizes of the components of the present application. Those skilled in the art can choose various possible shapes and scale sizes to implement the present application according to specific circumstances under the guidance of the present application. In the drawings:

[0083] Figure 1 A flowchart of a cervical cancer detection method based on a Simsiam Mae self-supervised model provided by the first embodiment of the present application;

[0084] Figure 2 An example of a cervical cell image after preprocessing operation;

[0085] Figure 3 An example of two images obtained after two random data augmentation operations;

[0086] Figure 4 An example of a mask image;

[0087] Figure 5 An example of a reconstructed image. DETAILED DESCRIPTION

[0088] In order for those skilled in the art to better understand the technical solutions in the present application, the technical solutions in the embodiments of the present application will be described clearly and completely below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor should be within the scope of protection of the present application.

[0089] Please refer to Figure 1 The first embodiment of the present application provides a cervical cancer detection method based on a Simsiam Mae self-supervised model, including the following steps:

[0090] S1, a cervical cell image acquisition and preprocessing step, in combination with pre-provided annotation information, extracts cervical cell images from liquid-based thin-layer cell detection images, and divides the extracted cervical cell images into a training data set and a test data set;

[0091] S2, a data preprocessing step, performing two different and random data augmentation operations on each cervical cell image in the training data set to generate two different images, and inputting the two images into a Simsiam Mae self-supervised model;

[0092] S3, a Simsiam Mae self-supervised model training step, the Simsiam Mae self-supervised model uses a contrastive learning method to train an encoding path, a projection layer and a prediction layer to extract low-frequency information features of a cell image, and learns global information and cell shape features of the image; an encoding path and a decoding path are trained using a masked image modeling method to extract high-frequency information features of the cell image, and local information and texture features in the image are learned;

[0093] S4, a downstream task training step, using the test data set to fine-tune the parameters of the encoding path of the above-mentioned Simsiam Mae self-supervised model, completing the classification of cervical cells, and obtaining a trained Simsiam Mae self-supervised model;

[0094] S5, using the trained Simsiam Mae self-supervised model to detect cervical cancer.

[0095] In an embodiment, the step S1 includes the following steps:

[0096] S11, converting all liquid-based thin-layer cell detection images to images in an RGB color space;

[0097] S12, extracting cervical cell images from the above-mentioned images in the RGB color space according to pre-provided cervical cell type and pixel value coordinate annotation information;

[0098] S13, filling the extracted cervical cell images, filling the length and width of all images to 512 pixel values to unify the size of all images;

[0099] S14, dividing the extracted cervical cell images into data sets according to cervical cell types, randomly selecting part of the images as a training data set, and the other part as a test data set.

[0100] In step S12, the annotation information can be provided by a doctor, including the type of cervical cells, the pixel value horizontal and vertical coordinates of the location of the cervical cells, and the length and width of the cervical cells. In step S14, the training data set can account for 75% of all images, which is used to train the feature extraction capability of the cervical cells; the test data set can account for 25% of all images, which is used to verify the effect. Figure 2 An example of a cervical cell image after pre-processing operation.

[0101] In an embodiment, the step S2 comprises the following steps:

[0102] S21, performing twice different and random data enhancement operations on the images in the above data set to generate two different images;

[0103] S22, inputting the two images after the data enhancement operation into the SimSiam self-supervised model.

[0104] The data enhancement operation comprises the following steps:

[0105] S211, random cropping and scaling, randomly cropping a part of the image and adjusting it to a given size;

[0106] S212, color jitter, randomly changing the brightness, contrast, saturation and hue of the image;

[0107] S213, random grayscale, randomly converting the image to grayscale;

[0108] S214, random horizontal flip, randomly horizontally flipping the given image with a given probability;

[0109] S215, normalization, normalizing the image with the mean and standard deviation.

[0110] In an embodiment, the step S211 is to randomly crop a region in the image, the size of the randomly cropped region is 0.7 to 0.9 of the original image size, and the size of the randomly cropped region is adjusted to

RGB, 512, 512, 3

[0111] The step S212 is to adjust the brightness, contrast, saturation and hue of the image to 0.4, 0.4, 0.4 and 0.1 of the original image respectively with a probability of 80%;

[0112] The step S213 is to randomly convert the color image to a grayscale image with a probability of 20%;

[0113] The step S214 is to horizontally randomly flip the original image with a probability of 50%;

[0114] The step S215 is to normalize the image using the mean and standard deviation, and the normalization formula is as follows,

[0115]

[0116] wherein Output is the normalized image, Input is the original image, mean is 【0.485, 0.456, 0.406】, and std is 【0.229, 0.224, 0.225】. Figure 3 An example of two images obtained after two random data augmentation operations.

[0117] In an embodiment, the step S3 specifically comprises the following steps:

[0118] S31, input the two images obtained after two random data augmentation operations into the encoding path of the Simsiam Mae self-supervised model, and the encoding path performs feature extraction operations on the two images respectively, and outputs two initial feature tensors, and input the output initial feature tensors into the projection layer;

[0119] S32, after the two initial feature tensors pass through the projection layer, a first projection feature tensor and a second projection feature tensor (referred to as two projection feature tensors z1 and z2) are output, and the first projection feature tensor and the second projection feature tensor are input into the prediction layer;

[0120] S33, after the first projection feature tensor and the second projection feature tensor pass through the prediction layer, a first prediction feature tensor and a second prediction feature tensor (referred to as two prediction feature tensors p1 and p2) are output;

[0121] S34, a loss value is calculated using a contrastive learning method;

[0122] S35, the first prediction feature tensor and the second prediction feature tensor are input into the decoding path respectively, and after passing through the decoding path, a reconstructed image is output, and a loss value is calculated using a masked image modeling method;

[0123] S36, using a back propagation algorithm to obtain a gradient, and then using a gradient descent algorithm to obtain a vector composed of a plurality of partial derivatives, and then updating the model parameters through the vectors.

[0124] In an embodiment, the S31 comprises the following steps:

[0125] S311, input the images after the random data augmentation operation to the Patch Embedding layer in the encoding path, convert the two-dimensional images into a series of one-dimensional path embeddings to form a patch embeddings tensor;

[0126] S312, add position encoding information to the patch embeddings tensor to provide position information for each path embeddings tensor;

[0127] S313, perform a mask operation on the patch embeddings tensor (for an example of a masked image), mask a part of the patch embeddings tensor, and add the unmasked patch embeddings tensor to the CLS token vector; Figure 4

[0128] S314, input the unmasked patch embeddings tensor to the Vision Transformer structure for feature extraction, and output two initial feature tensors;

[0129] S315, input the two output initial feature tensors to the projection layer.

[0130] Preferably, in step S311, the Patch Embedding layer is specifically,

[0131] S3111, input the two images after the random data augmentation operation to a two-dimensional convolutional layer, slide a specified number of convolutional kernels with the same depth as the two images on the original image row by row, perform convolution sum on the region after each sliding, and obtain two feature maps output by the convolutional layer;

[0132] S3112, perform dimension transformation on the two feature maps.

[0133] Preferably, in step S3111, it is specifically,

[0134] S31111, the input channel number of the two-dimensional convolutional layer is 3, the output channel number is 768, the convolutional kernel size is 16, and the convolutional kernel motion step is 16;

[0135] S31112, input the two images after the random data augmentation operation to the two-dimensional convolutional layer, and the output feature map size is 【768, 32, 32】.

[0136] Preferably, in step S3112, the dimension transformation is specifically, ​

[0137] S31121, dimensionality reduction is performed on the second and third dimensions of the feature map, and the shape is changed from 【768, 32, 32】 to 【768, 1024】;

[0138] S31122, transpose operation is performed on the feature map after dimensionality reduction, and the shape is changed from 【768, 1024】 to 【1024, 768】.

[0139] Preferably, in the step S312, the position encoding information is added to the patch embeddings tensor, specifically,

[0140] S3121, the sine-cosine position encoding matrix is calculated using the following formula,

[0141]

[0142]

[0143] where dim represents the dimension of the input, pos represents the index of the patch embedding, and i represents the index in the vector;

[0144] S3122, the sine-cosine position encoding matrix is added to the patch embeddings tensor, and position information is added to each patch embedding.

[0145] Preferably, in the step S313, the patch embeddings tensor is masked, specifically, some path embeddings are selected uniformly and without repetition, and these path embeddings are hidden.

[0146] Preferably, the number of selected path embeddings is 25% of the total number of path embeddings.

[0147] Preferably, in the step S314, the operation of adding the CLS token vector to the unhidden patch embeddings, specifically, a CLS token vector with a dimension of 【1, 768】 is added to the first position of the patch embeddings tensor, and the dimension of the patch embeddings tensor is changed to 【1024, 768】.

[0148] Preferably, in the step S315, the unhidden patch embeddings are input into the VisionTransformer structure, specifically,

[0149] S3151, input the above-mentioned patch embeddings which are not hidden into the encoding path of the Simsiam Mae self-supervised model, the encoding path is a Vision Transformer model, and the patch embeddings are subjected to feature extraction operation using the model;

[0150] S3152, input the output result of the above-mentioned Vision Transformer model into the projection layer.

[0151] Preferably, in the step S3151, the Vision Transformer model structure of the encoding path is specifically,

[0152] S31511, the Vision Transfomer is composed of 24 Vision Transformer Blocks;

[0153] S31512, the output result of the above-mentioned Vision Transformer model is subjected to normalization operation.

[0154] Preferably, in the step S31511, the Vision Transformer Block structure is specifically layer normalization-self-head attention layer-residual connection-layer normalization-multilayer perceptron-residual connection.

[0155] Preferably, the layer normalization operation is specifically to calculate the mean and variance of all parameters in all channels in a batch, and then perform normalization, the formula is as follows,

[0156]

[0157] Where x is the input feature map, y is the output feature map, and β and γ are learnable affine transformation parameters.

[0158] Preferably, the multi-head self-attention layer is specifically,

[0159] S315111, the feature map x is input into the first group of linear layers, and 8 groups of query, key and value matrices are calculated;

[0160] S315112, the transpose of the query and the key in each group is calculated to obtain an attention value matrix;

[0161] S315113, the attention value matrix is subjected to softmax operation;

[0162] S315114, the attention value matrix is subjected to dropout operation;

[0163] S315115, the attention value matrix is calculated to obtain 8 groups of feature maps y;

[0164] S315116, the feature maps y are input into a second linear layer, and a dropout operation is performed;

[0165] S315117, the 8 groups of feature maps y are spliced to obtain an output result.

[0166] Preferably, the residual connection is specifically an element-wise addition of the feature map before input into the first layer normalization and the feature map output by the multi-head self-attention layer.

[0167] Preferably, the multi-layer perceptron is specifically,

[0168] S315118, the feature map is input into a first linear layer;

[0169] S315119, the result of the first linear layer is input into a nonlinear activation function GELU;

[0170] S3151110, a dropout operation is performed on the output result of the nonlinear activation function GELU;

[0171] S3151111, the result after the dropout is input into a second linear layer;

[0172] S3151112, a dropout operation is performed on the output result of the second linear layer.

[0173] Preferably, in the step S31512, the normalization operation is specifically a layer normalization operation on the output result of the Vision Transformer model.

[0174] In an embodiment, the step S32 includes the following steps:

[0175] S321, two original feature tensors are input into a first convolutional layer in a projection layer;

[0176] S322, a normalization operation is performed on the result output by the first convolutional layer;

[0177] S323, the result of the normalization operation is input into a second convolutional layer;

[0178] S324, a normalization operation is performed on the result output by the second convolutional layer;

[0179] S325, the result of the normalization operation is input into a third convolutional layer;

[0180] S326, performing layer normalization operation on the output of the third convolutional layer;

[0181] S327, outputting the first projection feature tensor and the second projection feature tensor, and inputting the first projection feature tensor and the second projection feature tensor into the prediction layer;

[0182] Preferably, the first, second and third convolutional layers are one-dimensional convolution, the input channel number is 768, the output channel number is 768, the convolution kernel size is 1, and the convolution kernel motion step is 1.

[0183] Preferably, the first and second normalization operations are layer normalization operation on the input feature map, and then the nonlinear activation function RELU operation is performed on the layer normalization result.

[0184] The step S33 comprises the following steps:

[0185] S331, inputting the first projection feature tensor and the second projection feature tensor obtained above into the first convolutional layer in the prediction layer;

[0186] S332, performing normalization operation on the output of the first convolutional layer;

[0187] S333, inputting the result of the normalization operation into the second convolutional layer;

[0188] S334, outputting the first prediction feature tensor and the second prediction feature tensor.

[0189] Preferably, the first and second convolutional layers are one-dimensional convolution, the input channel number is 768, the output channel number is 768, the convolution kernel size is 1, and the convolution kernel motion step is 1.

[0190] Preferably, in the step S332, the normalization operation is performed on the output of the first convolutional layer, which comprises layer normalization operation on the input feature map, and then nonlinear activation function RELU operation is performed on the layer normalization result.

[0191] In an embodiment, the step S34 comprises the following steps:

[0192] S341, multiplying the gradient-truncated second projection feature tensor with the first prediction feature tensor to obtain a first cross-correlation matrix, and multiplying the gradient-truncated first projection feature tensor with the second prediction feature tensor to obtain a second cross-correlation matrix;

[0193] S342, the first cross-correlation matrix and the second cross-correlation matrix are divided by the value of the batch size respectively;

[0194] S343, the loss value of the contrast learning method is calculated using the following formula,

[0195]

[0196] where i indexes the dimension of the first cross-correlation matrix or the second cross-correlation matrix, C1 and C2 represent the first cross-correlation matrix and the second cross-correlation matrix respectively;

[0197] C 1ii and the calculation formula of C 2ii is as follows,

[0198]

[0199] where i indexes the dimension of the first cross-correlation matrix or the second cross-correlation matrix, z is the output result of the projection layer, the first projection feature tensor and the second projection feature tensor, p is the output result of the prediction layer, the first prediction feature tensor and the second prediction feature tensor, and b indexes the batch sample.

[0200] In an embodiment, the step S35 includes the following steps:

[0201] S351, the first prediction feature tensor and the second prediction feature tensor are respectively input into the first convolutional layer in the decoding path;

[0202] S352, the output of the convolutional layer is spliced with the hidden patch embedding tensor;

[0203] S353, position encoding information is added to the spliced patch embedding tensor;

[0204] S354, the patch embedding tensor with added position encoding information is input into the Vision transformer structure;

[0205] S355, the output result of the Vision transformer structure is input into the second convolutional layer in the decoding path;

[0206] S356, the CLS token vector in the output result of the convolutional layer is removed, and the reconstructed image is output;

[0207] S357, the output reconstructed image is normalized;

[0208] S358, the loss value of the masking image modeling method is calculated using the mean square error formula, and the calculation formula is as follows,

[0209]

[0210] Where n is the number of patch embedding tensors to be masked, pred is the reconstructed image, and target is the original image.

[0211] Preferably, the linear layer in step S351 has 768 input channels and 512 output channels.

[0212] Preferably, the linear layer in step S355 has 512 input channels and 768 output channels.

[0213] Preferably, the gradient descent algorithm in step S36 is AdamW algorithm.

[0214] In an embodiment, the S4 comprises the following steps:

[0215] S41, constructing a cervical cell image dataset;

[0216] S42, using the encoding path of the Simsiam Mae self-supervised model as the classification model for this task;

[0217] S43, loading the pre-training parameters of the Simsiam Mae self-supervised model;

[0218] S44, using the cervical cell image dataset to fine-tune the parameters of the classification model loaded with the pre-training parameters.

[0219] Preferably, in step S41, the cervical cell image dataset has a size of

RGB, 512, 512, 3

[0220] Preferably, in step S42, the encoding path of the Simsiam Mae self-supervised model is used as the classification model for this task, specifically, a classification head is added to the original structure of the Vision Transformer in the encoding path of the Simsiam Mae self-supervised model. The classification head is a linear layer with an input dimension of 768 and an output dimension of 7.

[0221] The beneficial effects of the present application mainly embody: based on a deep learning model, combining the way of contrastive learning and masked image modeling, respectively learning the low-frequency signal features and high-frequency signal features in the cervical cell image, so as to achieve the purpose of self-supervised learning. The present application can effectively extract the feature information in the cervical cell image dataset, accurately recognize and detect the discrete distributed cells in the image; compared with the traditional supervised learning algorithm, greatly reduces the dependence on artificial labeled data, saves the time and cost of work, and improves the classification accuracy of the cervical cell image.

[0222] It should be understood that the above description is for illustrative purposes only and is not intended to be limiting. Many embodiments and many applications other than those described herein will be readily apparent to those skilled in the art from this description. Thus, the scope of the present teachings should not be determined from the above description, but should be determined from the following claims and the full breadth permitted by equivalents thereof. The disclosure of all articles and references, including patent applications and publications, are incorporated by reference herein in their entirety. The omission of any aspect of the subject matter disclosed herein from any claim does not mean that the subject matter is disclaimer of that aspect of the subject matter, nor does it mean that the applicant did not consider that subject matter to be part of the disclosed application.

Claims

1. A cervical cancer detection method based on a Simsiam Mae self-supervised model, characterized by, The method comprises the following steps: S1, a cervical cell image acquisition and preprocessing step, combined with pre-provided annotation information, extracting cervical cell images from liquid-based thin-layer cell detection images, and segmenting the extracted cervical cell images into a training data set and a test data set; S2, a data preprocessing step, performing two different and random data augmentation operations on each cervical cell image in the training data set to generate two different images, and inputting them into the Simsiam Mae self-supervised model; S3, a Simsiam Mae self-supervised model training step, the Simsiam Mae self-supervised model uses a contrast learning method to train the encoding path, the projection layer and the prediction layer to extract the low-frequency information features of the cell image, and learns the global information and cell shape features of the image; The encoding path and the decoding path are trained using a masked image modeling method to extract the high-frequency information features of the cell image, and the local information and texture features in the image are learned; S4, a downstream task training step, using the test data set to fine-tune the parameters of the encoding path of the above-mentioned Simsiam Mae self-supervised model, completing the classification of cervical cells, and obtaining a trained Simsiam Mae self-supervised model; S5, using the trained Simsiam Mae self-supervised model to detect cervical cancer; The step S3 specifically comprises the following steps: S31, inputting the two images obtained after the two random data augmentation operations into the encoding path of the Simsiam Mae self-supervised model, and the encoding path performing feature extraction operations on the two images to output two initial feature tensors, and inputting the output initial feature tensors into the projection layer; S32, after the two initial feature tensors pass through the projection layer, first projection feature tensors and second projection feature tensors are output, and the first projection feature tensors and the second projection feature tensors are input into the prediction layer; S33, after the first projection feature tensors and the second projection feature tensors pass through the prediction layer, first prediction feature tensors and second prediction feature tensors are output; S34, using a contrast learning method to calculate a loss value; S35, inputting the first prediction feature tensors and the second prediction feature tensors into the decoding path, and outputting reconstructed images after the decoding path, and using a masked image modeling method to calculate a loss value; S36, using a back propagation algorithm to obtain gradients, and then using a gradient descent algorithm to obtain a vector composed of a plurality of partial derivatives, and then updating model parameters through the vector; The S31 comprises the following steps: S311, inputting the image after the random data augmentation operation into the Patch Embedding layer in the encoding path, converting the two-dimensional image into a series of one-dimensional path embeddings to form a patch embeddings tensor; S312, adding position encoding information to the patch embeddings tensor to provide position information for each path embeddings tensor; S313, mask the patch embeddings tensor, hide a part of the patch embeddings tensor, and add the CLS token vector to the unhidden patch embeddings tensor; S314, input the unhidden patch embeddings tensor into the Vision Transformer structure for feature extraction, and output two initial feature tensors; S315, input the output two initial feature tensors into the projection layer; The step S35 includes the following steps: S351, input the first predicted feature tensor and the second predicted feature tensor into the first convolution layer in the decoding path respectively; S352, splice the output of the convolution layer with the hidden patch embedding tensor; S353, add position encoding information to the spliced patch embedding tensor; S354, input the patch embedding tensor with position encoding information into the Vision transformer structure; S355, input the output result of the Vision transformer structure into the second convolution layer in the decoding path; S356, remove the CLS token vector in the output result of the convolution layer, and output the reconstructed image; S357, normalize the output reconstructed image; S358, the loss value of the masking image modeling method is calculated using the mean square error formula, and the calculation formula is as follows, , Where n is the number of hidden patch embedding tensors, pred is the reconstructed image, and target is the original image.

2. The Simsiam Mae self-supervised model-based cervical cancer detection method of claim 1, wherein, The step S1 includes the following steps: S11, convert all liquid-based thin-layer cell detection images to images in the RGB color space; S12, according to the pre-provided cervical cell type and cervical cell pixel value coordinate labeling information, extract the cervical cell image from the RGB color space image; S13, fill the extracted cervical cell image, fill the length and width of all images to 512 pixel value size, and unify the size of all images; S14, divide the extracted cervical cell image into data sets according to the cervical cell type, randomly select part of the image as the training data set, and the other part as the test data set.

3. The Simsiam Mae self-supervised model-based cervical cancer detection method of claim 1, wherein, The data enhancement operation in step S2 includes the following steps: S211, random crop scaling, randomly crop part of the image and adjust it to a given size; S212, color jitter, randomly change the brightness, contrast, saturation and hue of the image; S213, random grayscale, randomly convert the image to grayscale level; S214, random horizontal flip, randomly horizontally flip the given image with a given probability; S215, normalization, normalize the image with the mean and standard deviation.

4. The cervical cancer detection method based on the Simsiam Mae self-supervised model according to claim 3, wherein The step S211 is to randomly crop a region in the image, the size of the randomly cropped region is 0.7 to 0.9 of the original image size, and the size of the randomly cropped region is adjusted to 【RGB, 512, 512, 3】 in a bilinear interpolation manner; The step S212 is to adjust the brightness, contrast, saturation and hue of the image to 0.4, 0.4, 0.4 and 0.1 of the original image respectively with a probability of 80%; The step S213 is to randomly convert the color image to a grayscale image with a probability of 20%; The step S214 is to horizontally randomly flip the original image with a probability of 50%; The step S215 is that the operation normalizes the image with the average value and the standard deviation, and the normalization formula is as follows, , Wherein, Output is the normalized image output, Input is the original image input, mean is 【0.485, 0.456, 0.406】, and std is 【0.229, 0.224, 0.225】.

5. The method for cervical cancer detection based on the Simsiam Mae self-supervised model as claimed in claim 1, wherein, The step S32 includes the following steps: S321, input two original feature tensors into the first convolutional layer in the projection layer; S322, perform a normalization operation on the output of the first convolutional layer; S323, input the result of the normalization operation into the second convolutional layer; S324, perform a normalization operation on the output of the second convolutional layer; S325, input the result of the normalization operation into the third convolutional layer; S326, perform a layer normalization operation on the output of the third convolutional layer; S327, output the first projection feature tensor and the second projection feature tensor, which are input into the prediction layer respectively; The step S33 includes the following steps: S331, input the first projection feature tensor and the second projection feature tensor obtained into the first convolutional layer in the prediction layer; S332, perform a normalization operation on the output of the first convolutional layer; S333, input the result of the normalization operation into the second convolutional layer; S334, output the first prediction feature tensor and the second prediction feature tensor.

6. The Simsiam Mae self-supervised model-based cervical cancer detection method of claim 1, wherein, The step S34 includes the following steps: S341, multiply the gradient-truncated second projection feature tensor with the first prediction feature tensor to calculate a first cross-correlation matrix, and multiply the gradient-truncated first projection feature tensor with the second prediction feature tensor to calculate a second cross-correlation matrix; S342, divide the first cross-correlation matrix and the second cross-correlation matrix by the value of the batch size respectively; S343, calculate the loss value of the contrast learning method using the following formula, , Wherein, i indexes the dimension of the first cross-correlation matrix or the second cross-correlation matrix, C1 and C2 represent the first cross-correlation matrix and the second cross-correlation matrix respectively; C 1ii With C 2ii The calculation formula is as follows, , Wherein, i indexes the dimension of the first cross-correlation matrix or the second cross-correlation matrix, z is the output result of the projection layer, that is, the first projection feature tensor and the second projection feature tensor, p is the output result of the prediction layer, that is, the first prediction feature tensor and the second prediction feature tensor, and b indexes the batch sample.

7. The Simsiam Mae self-supervised model-based cervical cancer detection method of claim 1 wherein, The S4 includes the following steps: S41, construct a cervical cell image dataset; S42, using the encoding path of the above-mentioned Simsiam Mae self-supervised model as the classification model of the task; S43, loading the pre-training parameters of the Simsiam Mae self-supervised model; S44, using the above-mentioned cervical cell image dataset to fine-tune the parameters of the classification model loaded with the pre-training parameters.

Images