A cervical OCT image classification method and system based on mask self-supervised learning

Abstract

The application discloses a kind of based on mask self-supervised learning cervical OCT image classification method and system, first by mask image generation and the self-supervised learning network of combined contrast learning obtains the pre-training weight of hierarchical Transformer classification model;Then using labeled data is supervised training to hierarchical Transformer classification model obtains final weight;Then all two-dimensional OCT images in three-dimensional cervical OCT image are classified using classification model;Finally, according to the class label of all OCT images in OCT image, obtain classification result.The application effectively utilizes a large number of unlabeled cervical OCT images for self-supervised pre-training by using a self-distillation model that combines mask generation model and contrast learning, alleviates the problem of requiring a large number of labeled data for supervised learning, and improves the performance of the model. By using the mixed cropping data enhancement method specified by the class, the recognition ability of the model for high-grade lesion images is improved.

Description

Technical Field

[0001] This invention belongs to the fields of medical image analysis and computer-aided diagnosis, and relates to an image classification method and system, specifically a cervical OCT image classification method and system based on mask self-supervised learning. Background Technology

[0002] Cervical cancer is one of the most common malignant tumors of the female reproductive system, ranking fourth among cancer-related deaths in women worldwide. The World Health Organization has adopted a global strategy to eliminate cervical cancer as a public health issue. Studies have shown that the development of cervical cancer takes a relatively long time (ranging from several years to decades), therefore, effective cervical cancer screening and treatment of precancerous lesions play a crucial role in preventing cervical cancer and reducing mortality.

[0003] Currently, various clinical screening and diagnostic methods are used to examine cervical lesions, including human papillomavirus (HPV) testing, thin-layer liquid-based cytology (TCT), colposcopy, and cervical biopsy (also known as surgical pathology). However, each method has its own drawbacks. For example, HPV testing has a high false-positive rate; TCT has low sensitivity in detecting cervical intraepithelial neoplasia (CIN); colposcopy heavily relies on the experience and skill of the colpologist; and while biopsy is currently the "gold standard" for cervical disease diagnosis, it is invasive, prone to bleeding and infection, and with a maximum of four cervical tissue samples obtainable, it is easy to miss cervical lesions. Therefore, developing a rapid, non-invasive, efficient, and intelligent method for cervical cancer detection is extremely important for cervical cancer prevention.

[0004] Optical coherence tomography (OCT) is a three-dimensional tomographic imaging technique based on the principle of low correlation of optical interference. It can obtain cross-sectional images of biological tissues with micron-level resolution and a penetration depth of up to 2 millimeters, making it the "gold standard" for clinical examination of ophthalmic diseases. Furthermore, recent medical research has demonstrated that OCT images can reveal the morphological characteristics of cervical tissue both in vivo and in vitro (with results approaching those of pathological slides), including squamous epithelium, basement membrane, cervical stroma, glands, cysts, low-grade squamous intraepithelial lesions (LSIL), high-grade squamous intraepithelial lesions (HSIL), and cervical cancer. This makes it possible to use OCT technology for non-invasive and accurate screening and detection of cervical lesions.

[0005] Given that OCT technology is not widely used in gynecology, researchers have designed and developed computer-aided diagnostic models based on convolutional neural networks (CNNs) and transform neural networks (Transformers) in recent years to improve doctors' interpretation efficiency. However, existing methods use supervised learning to train classification models, requiring a large number of labeled cervical OCT images. It is difficult to obtain a large number of high-quality labeled OCT images in a short period of time because labeling cervical OCT images is time-consuming, labor-intensive, and requires specialized medical knowledge. Gynecologists who are proficient in OCT technology are few and have limited time and energy. Summary of the Invention

[0006] To alleviate the problem of limited labeled cervical OCT image data and improve the performance and training effect of classification models, this invention proposes an OCT image classification method and system based on mask self-supervised learning.

[0007] The technical solution adopted by the method of the present invention is: a cervical OCT image classification method based on mask self-supervised learning, comprising the following steps:

[0008] Step 1: Obtain the pre-trained weights of the hierarchical Transformer classification model through a self-supervised learning network that combines mask image generation and contrastive learning;

[0009] The self-supervised learning network includes an encoder for extracting mixed image features, an average pooling layer, a demixing operation layer, an implicit context regressor, a decoder, and a teacher branch. The encoder, implicit context regressor, average pooling layer, and demixing operation layer constitute the student branch, and the teacher branch consists of the same encoder and average pooling layer as the student branch. The encoder is used to address the inconsistency between the pre-training and fine-tuning processes. The implicit context regressor is used to reconstruct the representation of the mask block. The decoder is used to reconstruct the original pixel values ​​of the mask image block, and the representations of the teacher branch and the student branch are aligned using a self-distillation method, thereby separating the encoder's feature extraction task from the decoder's reconstruction task.

[0010] Step 2: Use labeled data to perform supervised training on the hierarchical Transformer classification model to obtain the final weights;

[0011] Step 3: Classify all OCT images in the 3D cervical OCT imaging using a classification model;

[0012] Using a hierarchical Transformer classification model, the weights obtained in step 2 are loaded, and the cervical OCT image is input into the hierarchical Transformer classification model to extract feature vectors. Then, the feature vectors are input into a linear layer and a Softmax layer to obtain the probability distribution of 7 categories. The 7 categories include normal, cervicitis, cyst, cervical columnar epithelial eversion, low-grade squamous intraepithelial lesion, high-grade squamous intraepithelial lesion, and cervical cancer.

[0013] Step 4: Use the "voting mechanism" and threshold to determine the category label of all two-dimensional cervical OCT images in the three-dimensional cervical OCT image, and obtain the classification result.

[0014] The technical solution adopted by the system of this invention is: a cervical OCT image classification system based on mask self-supervised learning, comprising the following modules:

[0015] Module 1 is used to obtain pre-trained weights for a hierarchical Transformer classification model through a self-supervised learning network that combines mask image generation and contrastive learning.

[0016] The self-supervised learning network includes an encoder for extracting mixed image features, an average pooling layer, a demixing operation layer, an implicit context regressor, a decoder, and a teacher branch. The encoder, implicit context regressor, average pooling layer, and demixing operation layer constitute the student branch, and the teacher branch consists of the same encoder and average pooling layer as the student branch. The encoder is used to address the inconsistency between the pre-training and fine-tuning processes. The implicit context regressor is used to reconstruct the representation of the mask block. The decoder is used to reconstruct the original pixel values ​​of the mask image block, and the representations of the teacher branch and the student branch are aligned using a self-distillation method, thereby separating the encoder's feature extraction task from the decoder's reconstruction task.

[0017] Module 2 is used to perform supervised training on a hierarchical Transformer classification model using labeled data to obtain the final weights;

[0018] Module 3 is used to classify all OCT images in a 3D cervical OCT imaging system using a classification model;

[0019] Using a hierarchical Transformer classification model, the weights obtained in step 2 are loaded, and the cervical OCT image is input into the hierarchical Transformer classification model to extract feature vectors. Then, the feature vectors are input into a linear layer and a Softmax layer to obtain the probability distribution of 7 categories. The 7 categories include normal, cervicitis, cyst, cervical columnar epithelial eversion, low-grade squamous intraepithelial lesion, high-grade squamous intraepithelial lesion, and cervical cancer.

[0020] Module 4 is used to determine the category label of all two-dimensional cervical OCT images in the three-dimensional cervical OCT image using a "voting mechanism" and a threshold, and obtain the classification result.

[0021] The beneficial effects of this invention are as follows: On the one hand, by using a self-distillation model that combines a mask generation model and contrastive learning, a large number of unlabeled cervical OCT images are effectively utilized for self-supervised pre-training, fully training the model weights, alleviating the problem that supervised learning requires a large amount of labeled data, and improving the model's performance. On the other hand, by using a category-specified hybrid cropping data augmentation method, the diversity of high-grade lesion images in the training set is increased, improving the model's ability to recognize high-grade lesion images. Attached Figure Description

[0022] Figure 1 This is a flowchart illustrating the classification of cervical OCT images according to an embodiment of the present invention;

[0023] Figure 2 This is a diagram of a self-supervised learning framework that combines mask image generation and contrastive learning in an embodiment of the present invention.

[0024] Figure 3 This is a schematic diagram illustrating the use of a mask self-attention mechanism to extract image features in encoder self-supervised learning according to an embodiment of the present invention.

[0025] Figure 4 A schematic diagram illustrating the cropping and blending data enhancement specified for categories in embodiments of the present invention. Detailed Implementation

[0026] To facilitate understanding and implementation of the present invention by those skilled in the art, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the embodiments described herein are for illustration and explanation only and are not intended to limit the present invention.

[0027] Please see Figure 1 This invention provides a cervical OCT image classification method based on mask self-supervised learning, comprising the following steps:

[0028] Step 1: Obtain the pre-trained weights of the hierarchical Transformer classification model through a self-supervised learning network that combines mask image generation and contrastive learning;

[0029] Please see Figure 2The self-supervised learning network in this embodiment includes an encoder for extracting mixed image features, an average pooling layer, a demixing operation layer, an implicit context regressor, a decoder, and a teacher branch. The encoder, implicit context regressor, average pooling layer, and demixing operation layer constitute the student branch, and the teacher branch consists of the same encoder and average pooling layer as the student branch. The encoder is used to solve the problem of inconsistency between the pre-training and fine-tuning processes. The implicit context regressor is used to reconstruct the representation of the mask block. The decoder is used to reconstruct the original pixel values ​​of the mask image block. The representations of the teacher branch and the student branch are aligned using a self-distillation method, thereby separating the encoder's feature extraction task from the decoder's reconstruction task.

[0030] The implicit context regressor in this embodiment consists of two Transformer blocks with cross-attention and a multilayer perceptron.

[0031] In this embodiment, the encoders are Swin-T and Swin-S, and the pooling layer is a 2×2 average pooling layer.

[0032] In this embodiment, step 1 is specifically implemented by the following sub-steps:

[0033] Step 1.1: Collect a number of unlabeled and labeled 3D cervical OCT images to obtain a dataset; and divide the dataset into a training set and a test set;

[0034] In this embodiment, the acquired 3D OCT images of cervical tissue are divided into an unlabeled dataset and a labeled dataset. The labeled dataset is divided into a training set and a test set in an 80:20 ratio. The 3D OCT images of the same patient can only appear in the training set or the test set, while ensuring that the proportion of patients in each category in the training set and the test set is roughly the same.

[0035] The three-dimensional OCT images used in this embodiment are in the TIFF (Tagged Image File Format) format, which conforms to the Medical Digital Imaging and Communications (DICOM) specification, and the two-dimensional OCT images used are in the Portable Network Graphics (PNG) format.

[0036] Step 1.2: Randomly mix pairs of data from the training set to achieve the effect of image masking, and input the resulting mixed image into the self-supervised learning network;

[0037] (1) Randomly mix two pairs of image data from the same batch, and divide the images into sets of blocks of size 4×4. The sets of blocks for two images are represented as follows:

[0038] (2) Generate a random binary mask matrix M, and obtain the mixed image according to the following formula.

[0039]

[0040] Step 1.3: Use the Swin-Transformer classification model as an encoder to extract features from mixed and normal images, followed by a 2×2 average pooling layer to further reduce the size of the feature map;

[0041] The Swin-Transformer model is used as the feature extractor for the mixed image. In order to explicitly distinguish the blocks of the two images, mask self-attention is calculated through the mask matrix M, so that the blocks can only aggregate the information of the blocks in the same image and avoid interference from the information of the blocks in the other image. Then, a 2×2 average pooling layer is applied to downsample the feature map of the image by 2 times, further reducing the size of the feature map. The final feature map is downsampled by 64 times compared to the original image, and the feature dimension is 768.

[0042] Figure 2 The process of extracting features using the Swin-Transformer, which accepts mixed image input, is as follows: Figure 3 As shown, the two images are first blended to generate a hybrid image. In each stage of the Swin-Transformer, to explicitly distinguish information from the two images, an attention mask is generated. This attention mask is then integrated into the self-attention calculation of the Swin-Transformer block. This mask self-attention calculation ensures that the information from each block in the hybrid image is only fused with information from the image to which that block belongs, avoiding interference from the other image. Then, an average pooling layer further reduces the feature map size.

[0043] Step 1.4: The features of the two images constituting the blended image are separated from the blended image through the demixing operation layer, and the vector representation of the initialized mask block is concatenated with the extracted image features to form a full-size image feature map;

[0044] The concatenated feature map is input into the implicit context regressor. All mask blocks share a global vector representation with a dimension of 768 during initialization. The implicit context regressor consists of two blocks containing cross-attention layers and multilayer perceptrons. During computation, the representation of the mask block is used as the query, and the representation of the unmasked block extracted by the encoder is used as the key and value. The key and value remain unchanged during computation. The mask block is updated by aggregating the information of the unmasked block, and finally outputs the representation of the mask block with aggregated information.

[0045] Step 1.5: Input the stitched image feature map into the implicit context regressor to reconstruct the features of the masked block and obtain the masked reconstructed feature map;

[0046] Step 1.6: Input the feature map obtained in Step 1.5 into the decoder to reconstruct the original pixel values ​​of the mask block. The decoder consists of a standard Transformer block and a linear regression layer. The Transformer block contains a standard self-attention layer and a multilayer perceptron with a dimension of 768. The weights of the linear layer are 768×4096. The loss function for image reconstruction is shown below:

[0047]

[0048] in, It is represented by the set of blocks that divide the image x1 and x2. and It is a set representation of the reconstructed image patches, and M is a binary mask matrix.

[0049] Step 1.7: Input the unmasked image into the teacher branch encoder to extract complete image features, and then apply a 2×2 average pooling layer to reduce the size of the feature map to obtain a complete image feature map;

[0050] Step 1.8: Align the feature map reconstructed from the mask obtained in Step 1.5 with the complete image feature map in Step 1.7, calculate the alignment loss, and prevent the teacher branch encoder parameters from participating in the update by gradient stopping. The encoder parameters of the teacher branch are obtained by exponential moving average with the encoder parameters of the student branch. The final student branch encoder parameters obtained after training are used as the pre-trained weights.

[0051] In this embodiment, during gradient backpropagation, the teacher branch's output is detached (detach()) to prevent updates to its parameters. The encoder parameters of the teacher branch are obtained by performing an exponential moving average with the encoder model parameters of the student branch. The update rules are as follows:

[0052] θ t ←λθ t +(1-λ)θ s ;

[0053] Where, θ t This refers to the parameters θ of the teacher branching model. s This refers to the parameters of the student branch model, where λ refers to the update speed, which is set to 0.999 here.

[0054] In this embodiment, the alignment loss includes block-level loss and image-level loss;

[0055] The block-level loss consists of the block alignment loss and the block classification loss; the block alignment loss is derived from the representation of the mask block reconstructed by the implicit context regressor. The representation Z of the block corresponding to the teacher branch output. m The calculated loss function Mean squared error loss is used;

[0056]

[0057] and Z m Input their respective linear classification layers and The probability distribution across the 7 categories is obtained using the Softmax layer. P(Z m The self-distillation losses are classified as follows:

[0058]

[0059] Here, H(a,b) = -alogb;

[0060] The final block-level loss function is

[0061] Image-level loss is used to obtain the feature map output by the implicit context regressor, followed by global average pooling to obtain the reconstructed image representation. The feature map extracted from the teacher branch is subjected to global average pooling to obtain the image representation z. z and z are respectively passed through their respective linear classification layers and And the Softmax layer yields the probability distribution across the 7 categories. P(z) is then used to calculate the image-level self-distillation loss by calculating the cross-entropy between the two.

[0062] The final training loss function is L = L patch +L img +L rec .

[0063] Step 2: Use labeled data to perform supervised training on the hierarchical Transformer classification model to obtain the final weights;

[0064] In this embodiment, labeled data is used to supervise the training of the hierarchical Transformer classification model to obtain the final weights. During the training process, a category-specified pruning and hybrid data augmentation method is used to increase the diversity of high-grade squamous intraepithelial lesion data and improve the classification model's ability to identify positive data.

[0065] Please see Figure 4 The cropping and hybrid data augmentation method in this embodiment specifically includes the following sub-steps:

[0066] (1) Filter out images of normal, cervicitis, and cysts;

[0067] (2) Filter out images of high-grade lesions and cervical cancer marked in the images;

[0068] (3) Determine the size and position of the cropping box, namely the coordinates of the upper left corner (x1, y1) and the coordinates of the lower right corner (x2, y2); where y1 is a randomly selected integer from the height H×0.1 to H×0.5 of the entire image through uniform distribution, and x1 is a randomly selected integer from the width W of the entire image through uniform distribution, i.e., from 0 to W; y2 is directly set to H, and the width w of the cropping box is taken from the range of W×0.2 to W×0.7, and finally x2=x1+w;

[0069] (4) The positions of the cropping boxes determined by (x1,y1) and (x2,y2) are cropped from the images of high-grade lesions and cervical cancer to the images of normal, cervicitis and cysts to form a new image, which is labeled as high-grade lesion;

[0070] (5) After data augmentation (including random scaling and cropping, random horizontal flipping, and random Gaussian blur) and standardization of the image, the image size is set to 512×1024. When loading batch data, each batch is set to perform mixed cropping augmentation with a probability of 0.5.

[0071] (6) Using the Swin-Transformer classification model, load the weights obtained from self-supervised pre-training, input the cervical OCT image into the Swin-Transformer classification model to extract the feature vector with a dimension of 768, and then input the feature vector into a linear layer and a Softmax layer to obtain the probability distribution of 7 categories; calculate the cross-entropy loss with the real label, update the model parameters through backpropagation, and use the optimal parameters as the final weights.

[0072] Step 3: Classify all OCT images in the 3D cervical OCT imaging using a classification model;

[0073] Using the Swin-Transformer classification model, the weights obtained in step 2 are loaded, and the cervical OCT image is input into the Swin-Transformer classification model to extract a feature vector with a dimension of 768. Then, the feature vector is input into a linear layer and a softmax layer to obtain the probability distribution of 7 categories. The 7 categories include normal, cervicitis, cyst, cervical columnar epithelial eversion, low-grade squamous intraepithelial lesion, high-grade squamous intraepithelial lesion, and cervical cancer.

[0074] Step 4: Use the "voting mechanism" and threshold to determine the category label of the 3D cervical OCT image and obtain the classification result.

[0075] In this embodiment, a voting mechanism is used to determine the final category of the three-dimensional cervical OCT image. When the positive count ratio in the prediction result of the cervical OCT image is greater than the set threshold, the predicted category of the three-dimensional image is set to positive; otherwise, it is set to negative. If the category is positive, the images predicted to be positive are taken out, and their positive probabilities are averaged to obtain the positive probability of the three-dimensional OCT image. If the category is negative, the images predicted to be negative are taken out, and their negative probabilities are averaged to obtain the negative probability of the three-dimensional OCT image.

[0076] It should be understood that the above description of the preferred embodiments is quite detailed, but it should not be considered as a limitation on the scope of protection of this invention. Those skilled in the art, under the guidance of this invention, can make substitutions or modifications without departing from the scope of protection of the claims of this invention, and all such substitutions or modifications fall within the scope of protection of this invention. The scope of protection of this invention should be determined by the appended claims.

Claims

1. A cervical OCT image classification method based on mask self-supervised learning, characterized in that, Includes the following steps: Step 1: Obtain the pre-trained weights of the hierarchical Transformer classification model through a self-supervised learning network that combines mask image generation and contrastive learning; The self-supervised learning network includes an encoder for extracting mixed image features, an average pooling layer, a demixing operation layer, an implicit context regressor, a decoder, and a teacher branch. The encoder, implicit context regressor, average pooling layer, and demixing operation layer constitute the student branch, and the teacher branch consists of the same encoder and average pooling layer as the student branch. The encoder is used to address the inconsistency between the pre-training and fine-tuning processes. The implicit context regressor is used to reconstruct the representation of the mask block. The decoder is used to reconstruct the original pixel values ​​of the mask image block, and the representations of the teacher branch and the student branch are aligned using a self-distillation method, thereby separating the encoder's feature extraction task from the decoder's reconstruction task. Step 1 includes the following sub-steps: Step 1.1: Collect a number of unlabeled and labeled 3D cervical OCT images to obtain a dataset; and divide the dataset into a training set and a test set; Step 1.2: Randomly mix pairs of data from the training set to achieve the effect of image masking, and input the resulting mixed image into the self-supervised learning network; Step 1.3: Use a hierarchical Transformer classification model as an encoder to extract mixed and normal images, followed by a 2×2 average pooling layer to further reduce the size of the feature map; Step 1.4: The features of the two images constituting the blended image are separated from the blended image through the demixing operation layer, and the vector representation of the initialized mask block is concatenated with the extracted image features to form a full-size image feature map; Step 1.5: Input the stitched image feature map into the implicit context regressor to reconstruct the features of the masked block and obtain the masked reconstructed feature map; Step 1.6: Input the image representation obtained in Step 1.5 into the decoder to reconstruct the original pixel values ​​of the mask block; Step 1.7: Input the unmasked image into the teacher branch encoder to extract complete image features, and then apply an average pooling layer to reduce the size of the feature map to obtain a complete image feature map; Step 1.8: Align the feature map reconstructed from the mask obtained in Step 1.5 with the complete image feature map in Step 1.7, calculate the alignment loss, and prevent the teacher branch encoder parameters from participating in the update by gradient stopping. The encoder parameters of the teacher branch are obtained by exponential moving average with the encoder parameters of the student branch. The final student branch encoder parameters obtained after training are used as the pre-trained weights. Step 2: Use labeled data to perform supervised training on the hierarchical Transformer classification model to obtain the final weights; Step 3: Classify all OCT images in the 3D cervical OCT imaging using a classification model; Using a hierarchical Transformer classification model, the weights obtained in step 2 are loaded, and the two-dimensional cervical OCT image is input into the hierarchical Transformer classification model to extract feature vectors. Then, the feature vectors are input into a linear layer and a Softmax layer to obtain the probability distribution of 7 categories. The 7 categories include normal, cervicitis, cyst, cervical columnar epithelial eversion, low-grade squamous intraepithelial lesion, high-grade squamous intraepithelial lesion, and cervical cancer. Step 4: Obtain the classification results based on the category labels of all two-dimensional cervical OCT images in the three-dimensional cervical OCT image.

2. The cervical OCT image classification method based on mask self-supervised learning according to claim 1, characterized in that: In step 2, labeled data is used to supervise the training of the hierarchical Transformer classification model to obtain the final weights. During the training process, a category-specified pruning and mixing data augmentation method is used to increase the diversity of high-grade squamous intraepithelial lesion data and improve the classification model's ability to identify positive data. The cropping and hybrid data augmentation method specifically includes the following sub-steps: (1) Filter out images of normal, cervicitis, and cysts; (2) Filter out images of high-grade lesions and cervical cancer marked in the images; (3) Determine the size and position of the cropping frame, i.e., the coordinates of the upper left corner of the cropping frame. And the coordinates of the bottom right corner of the cropping frame ; (4) and The determined cropping frame position is used to crop images of high-grade lesions and cervical cancer onto images of normal, cervicitis, and cysts to form a new image labeled as high-grade lesion; (5) After data augmentation and standardization of the image, the image size is set to 512×1024. When loading batch data, each batch is set to perform mixed cropping augmentation with a probability of 0.

5. (6) Using the hierarchical Transformer classification model, load the pre-trained weights obtained by self-supervision, input the two-dimensional cervical OCT image into the hierarchical Transformer classification model to extract the feature vector, and then input the feature vector into a linear layer and a Softmax layer to obtain the probability distribution of 7 categories; calculate the cross-entropy loss with the real label, update the parameters of the hierarchical Transformer classification model through backpropagation, and use the optimal parameters as the final weights.

3. The cervical OCT image classification method based on mask self-supervised learning according to claim 1, characterized in that: The implicit context regressor is two Transformer blocks with cross-attention and multilayer perceptron.

4. The cervical OCT image classification method based on mask self-supervised learning according to claim 1, characterized in that: In step 1.8, the alignment loss includes block-level loss and image-level loss; The block-level loss consists of block alignment loss and block classification loss; the block-level alignment loss is derived from the representation of the mask block reconstructed by the implicit context regressor. Representation of the block corresponding to the teacher branch output The calculated loss function Mean squared error loss is used; ； and Input their respective linear classification layers and The probability distribution across the 7 categories is obtained using the Softmax layer. , The self-distillation losses are classified as follows: here, ; The final block-level loss function is The image-level loss, after obtaining the feature map output by the implicit context regressor, is then subjected to global average pooling to obtain the reconstructed image representation. The feature maps extracted from the teacher branch are subjected to global average pooling to obtain the image representation. , and Each through its own linear classification layer and And the Softmax layer yields the probability distribution across the 7 categories. , Then, the image-level self-distillation loss is obtained by calculating the cross-entropy of the two. ; The final training loss function is ; ； in, The loss function for image reconstruction. , It is an image and The set of partitioned blocks is represented as follows: and It is a set representation of the reconstructed image patches. It is a binary mask matrix.

5. The cervical OCT image classification method based on mask self-supervised learning according to claim 1, characterized in that: In step 1, the encoder parameters of the teacher branch are obtained by performing an exponential moving average with the encoder model parameters of the student branch; the update rules are as follows: in, This refers to the parameters of the teacher branching model. This refers to the parameters of the student branching model. Refers to update speed.

6. The cervical OCT image classification method based on mask self-supervised learning according to any one of claims 1-5, characterized in that: In step 4, a voting mechanism is used to determine the final category of the 3D cervical OCT image. When the proportion of positive counts in the prediction results of the 3D cervical OCT image is greater than the set threshold, the predicted category of the 3D cervical OCT image is set to positive; otherwise, it is set to negative. If the category is positive, the images predicted to be positive are taken out, and their positive probabilities are averaged to obtain the positive probability of the 3D cervical OCT image. If the category is negative, the images predicted to be negative are taken out, and their negative probabilities are averaged to obtain the negative probability of the 3D cervical OCT image.

7. A cervical OCT image classification system based on mask self-supervised learning, characterized in that, Includes the following modules: Module 1 is used to obtain pre-trained weights for a hierarchical Transformer classification model through a self-supervised learning network that combines mask image generation and contrastive learning. The self-supervised learning network includes an encoder for extracting mixed image features, an average pooling layer, a demixing operation layer, an implicit context regressor, a decoder, and a teacher branch. The encoder, implicit context regressor, average pooling layer, and demixing operation layer constitute the student branch, and the teacher branch consists of the same encoder and average pooling layer as the student branch. The encoder is used to address the inconsistency between the pre-training and fine-tuning processes. The implicit context regressor is used to reconstruct the representation of the mask block. The decoder is used to reconstruct the original pixel values ​​of the mask image block, and the representations of the teacher branch and the student branch are aligned using a self-distillation method, thereby separating the encoder's feature extraction task from the decoder's reconstruction task. Module 1 includes the following sub-modules: Module 1.1 is used to collect several unlabeled and labeled 3D cervical OCT image data to obtain a dataset; and divide the dataset into a training set and a test set; Module 1.2 is used to randomly mix pairs of data in the training set to achieve the effect of image masking, and input the resulting mixed image into the self-supervised learning network; Module 1.3 is used to use a hierarchical Transformer classification model as an encoder to extract mixed and normal images, followed by a 2×2 average pooling layer to further reduce the size of the feature map; Module 1.4 is used to separate the features of the two images that make up the blended image from the blended image through the demixing operation layer, and to concatenate the vector representation of the initialized mask block with the features of the extracted image to form a full-size image feature map; Module 1.5 is used to input the stitched image feature map into the implicit context regressor to reconstruct the features of the masked block and obtain the feature map of the mask reconstruction. Module 1.6 is used to input the image representation obtained in Module 1.5 into the decoder to reconstruct the original pixel values ​​of the mask block; Module 1.7 is used to input the unmasked image into the teacher branch encoder to extract complete image features, followed by an average pooling layer to reduce the size of the feature map and obtain a complete image feature map. Module 1.8 is used to align the feature map reconstructed from the mask obtained in Module 1.5 with the complete image feature map in Module 1.7, calculate the alignment loss, and prevent the teacher branch encoder parameters from participating in the update by gradient stopping. The encoder parameters of the teacher branch are obtained by exponential moving average with the encoder parameters of the student branch. The final student branch encoder parameters obtained after training are used as the pre-trained weights. Module 2 is used to perform supervised training on a hierarchical Transformer classification model using labeled data to obtain the final weights; Module 3 is used to classify all OCT images in a 3D cervical OCT imaging system using a classification model; Using a hierarchical Transformer classification model, the weights obtained from module 2 are loaded, and a two-dimensional cervical OCT image is input into the hierarchical Transformer classification model to extract feature vectors. Then, the feature vectors are input into a linear layer and a Softmax layer to obtain the probability distribution of 7 categories. The 7 categories include normal, cervicitis, cyst, cervical columnar epithelial eversion, low-grade squamous intraepithelial lesion, high-grade squamous intraepithelial lesion, and cervical cancer. Module 4 is used to obtain classification results based on the category labels of all two-dimensional cervical OCT images in the three-dimensional cervical OCT image.

Images