A TCT smear automatic detection method
By constructing a PVTv2-based DeepTCT model and combining it with an online hard sample mining module, the problems of missed and false detections in automatic cervical cell screening were solved, achieving more efficient cervical cell classification and localization and improving detection accuracy.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- CHONGQING MEDICAL UNIVERSITY
- Filing Date
- 2022-06-02
- Publication Date
- 2026-05-05
AI Technical Summary
Existing automated cervical cell screening methods, especially those based on deep convolutional neural networks, are insufficient to meet the accuracy and efficiency requirements for clinical applications, particularly when processing TCT smears, where there are issues of missed and false detections.
A DeepTCT model was constructed using a network architecture based on Pyramid Vision Transformer (PVTv2) and combined with an online hard sample mining module. The classification and localization performance of cervical cells was optimized through iterative training and data augmentation.
It significantly improves the accuracy of cervical cell classification and localization, reduces the rate of missed and false detections, and is superior to the existing Cascade R-CNN network.
Smart Images
Figure CN114998264B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of neural network technology, and in particular to an automatic detection method for TCT smears. Background Technology
[0002] The growth process and morphological changes of cervical cells are closely related to the occurrence and development of cervical cancer. Cervical cytology is an effective screening method for cervical cancer. Cervical cytology screening is usually performed by a pathologist under a microscope, using methods such as the traditional Pap smear and the ThinPrep Cytologic Test (TCT). The Pap smear usually uses the Pap five-level classification system; while the TCT uses the TBS reporting method, classifying the diagnostic results into seven categories. Typically, pathologists determine whether there are abnormalities in the cells based on the morphological characteristics of exfoliated cervical cells. These morphological characteristics include cell nuclear size, shape, staining intensity, nuclear-cytoplasmic ratio, etc.; these characteristics play a very important role in the doctor's decision-making.
[0003] Traditional Pap smears often result in blurry samples due to blood, mucus, and inflammation, affecting the doctor's interpretation of morphological characteristics and leading to inaccurate screening results. TCT, however, is not affected by these factors, significantly improving sample quality and the detection rate of abnormal cervical cells. Furthermore, TCT can detect precancerous lesions and microbial infections such as fungi, trichomonas, viruses, and chlamydia. It can be said that the advent of TCT has brought cervical cytology screening to a new level. However, current TCT screening is primarily manual, which is time-consuming, labor-intensive, and its accuracy is easily affected by the doctor's skill level and subjective emotions. Therefore, it is necessary to utilize machines to achieve automated cervical cell screening to assist in medical diagnosis. To achieve automated cervical cytology screening, the machine needs to be trained to automatically acquire these characteristics and then classify and locate cervical cells based on these characteristics.
[0004] Traditional methods for detecting cervical cell morphology features include threshold-based methods, clustering-based methods, contour-based methods, and traditional machine learning methods. Due to the complexity of cervical cell morphology features, the performance of these methods often fails to meet practical clinical needs. In recent years, researchers have begun to apply deep learning to cervical cytology screening, achieving results superior to traditional algorithms in most tasks. Among them, region-based deep convolutional neural networks (R-DCNN) have become the mainstream network model in cervical cytology screening due to their superior performance. These networks first obtain candidate regions, then classify and regress bounding boxes within these regions. Their advantage lies not only in achieving content recognition and classification of images but also in more accurately locating the target. However, when applied to cervical cytology screening, their performance still falls short of the requirements for widespread clinical application. The emerging Pyramid Vision Transformer (PVT) technology offers the potential for performance improvement in target detection. However, how to utilize this technology (including PVTv2) to build a higher-performance automated screening model remains a pressing issue. Summary of the Invention
[0005] In view of this, the purpose of this invention is to provide an automated TCT smear detection method to solve the technical problem of classifying and locating cervical cells.
[0006] The TCT smear automatic detection method of the present invention includes the following steps:
[0007] 1) Construct the cell detection model DeepTCT, which includes a backbone network PVTv2 with a feature pyramid network for extracting feature images, a region proposal network for generating preliminary region proposals, three detection branches for obtaining the classification and localization of cervical cells, and an online hard sample mining module.
[0008] The three detection branches have the same structure. Each detection branch includes a RoIAlign layer, an average pooling layer connected to the RoIAlign layer, a Flatten layer connected to the average pooling layer, a first fully connected layer connected to the Flatten layer, a second fully connected layer for bounding box regression connected to the first fully connected layer, and a third fully connected layer for bounding box classification connected to the first fully connected layer.
[0009] The output of PVTv2 is used as the input to the region proposal network. The output of PVTv2 and the output of the region proposal network together form a region of interest of arbitrary size, which is used as the input to the first detection branch. The output of the second fully connected layer of the first detection branch is used as the input to the second detection branch, and the output of the second fully connected layer of the second detection branch is used as the input to the third detection branch.
[0010] The online hard sample mining module includes an average pooling layer, a Flatten layer connected to the average pooling layer, a first fully connected layer connected to the Flatten layer, a second fully connected layer for bounding box regression connected to the first fully connected layer, and a third fully connected layer for bounding box classification connected to the first fully connected layer. The average pooling layer, Flatten layer, first fully connected layer, second fully connected layer, and third fully connected layer in both the online hard sample mining module and the third detection branch have the same structure, and these layers share weights. The online hard sample mining module also includes a hard region of interest (ROI) filtering module. The outputs of the second and third fully connected layers of the third detection branch serve as the inputs to the hard ROI filtering module, and the output of the hard ROI filtering module serves as the input to the third detection branch. The RoIAlign layer of the third detection branch processes the output of the hard ROI filtering module and then outputs it as the input to the average pooling layer of the online hard sample mining module.
[0011] 2) Training and testing DeepTCT includes the following steps:
[0012] a) Basic training and testing: The TCT images in the training set are processed with data augmentation and class balance, and then the images in the training set are input into DeepTCT for training. Finally, the trained network DeepTCT is applied to the test set to output the classification and localization results of cervical cells.
[0013] b) Iterative training and testing of the basic trained DeepTCT, which includes the following steps:
[0014] b1) Search for cervical cells detected by DeepTCT in each image of the test set. If the classification score of the cervical cell is greater than the set threshold, retain the label of the cervical cell; otherwise, discard the label.
[0015] b2) Samples generated by DeepTCT that have reached the set classification score are put into the training set and used together with the existing images in the training set for DeepTCT training.
[0016] b3) Apply the trained DeepTCT to the test set to output the classification and localization results of cervical cells;
[0017] b4) Repeat steps b1) - b3) until the preset number of iterations is reached;
[0018] 3) Apply the DeepTCT obtained in step 2) to cervical cell morphology detection.
[0019] 2. The automatic detection method for TCT smears according to claim 1, wherein the set threshold in step b1) is 0.8.
[0020] The beneficial effects of this invention are:
[0021] 1. The TCT smear automatic detection method of the present invention uses DeepTCT, a cervical cell detection model, which adopts a network based on vision transformer technology instead of a deep convolutional neural network. By combining it with an online hard sample mining module, the detection accuracy of the network is improved and the false negatives and false positives are reduced.
[0022] 2. The TCT smear automatic detection method of the present invention innovatively puts the samples generated by DeepTCT into the training set, and the cell detection model DeepTCT constructed through multiple iterations of training significantly improves the classification and localization performance of DeepTCT for cervical cells, and also has obvious advantages over existing deep neural networks such as Cascade R-CNN. Attached Figure Description
[0023] Figure 1 This is a flowchart of an automated TCT smear detection method based on DeepTCT.
[0024] Figure 2 This is the architecture diagram of DeepTCT;
[0025] Figure 3 This is the architecture diagram of PVTv2;
[0026] Figure 4 This is a diagram showing the results of cell detection. Detailed Implementation
[0027] The present invention will be further described below with reference to the accompanying drawings and embodiments.
[0028] The TCT smear automatic detection method in this embodiment includes the following steps:
[0029] 1) Construct the cell detection model DeepTCT, such as... Figure 2As shown, DeepTCT includes a backbone network PVTv2 with FPN for extracting feature images, a region proposal network for generating preliminary region proposals, three detection branches for obtaining the classification and localization of cervical cells, and an online hard example mining module (OHEM).
[0030] like Figure 3 As shown, PVTv2 comprises four stages for generating feature maps at different scales. Each stage consists of an Overlapping Patch Embedding (OPE) layer, a Transformer encoder, a Position Embedding (PE) module, and an Element-wise addition module. OPE encodes image information, the Transformer encoder decodes and enriches feature information, the Position Embedding module supplements positional information, and the Element-wise addition module adds the output of the OPE layer to the input of the Position Embedding module. Assuming an input image of size H×W×3, after stages 1, 2, 3, and 4, the feature map sizes are (H / 4)×(W / 4)×C1, (H / 8)×(W / 8)×C2, (H / 16)×(W / 16)×C3, and (H / 32)×(W / 32)×C4, respectively. PVTv2 achieves its pyramid structure in such a way that it can be easily inserted into the entire DeepTCT architecture while also achieving better performance than convolution.
[0031] The three detection branches have the same structure. Each detection branch includes a RoIAlign layer, an average pooling layer connected to the RoIAlign layer, a Flatten layer connected to the average pooling layer, a first fully connected layer connected to the Flatten layer, a second fully connected layer for bounding box regression connected to the first fully connected layer, and a third fully connected layer for bounding box classification connected to the first fully connected layer. The first fully connected layer has a two-layer structure.
[0032] The output of PVTv2 is used as the input to the region proposal network. The output of PVTv2 and the output of the region proposal network together form a region of interest (RoI) of arbitrary size, which is used as the input to the first detection branch. The output of the second fully connected layer of the first detection branch is used as the input to the second detection branch, and the output of the second fully connected layer of the second detection branch is used as the input to the third detection branch.
[0033] The online hard sample mining module includes an average pooling layer, a Flatten layer connected to the average pooling layer, a first fully connected layer connected to the Flatten layer, a second fully connected layer for bounding box regression connected to the first fully connected layer, and a third fully connected layer for bounding box classification connected to the first fully connected layer. The average pooling layer, Flatten layer, first fully connected layer, second fully connected layer, and third fully connected layer in both the online hard sample mining module and the third detection branch have the same structure, and these layers share weights. The online hard sample mining module also includes a hard region of interest (ROI) filtering module. The outputs of the second and third fully connected layers of the third detection branch serve as inputs to the hard ROI filtering module, and the output of the hard ROI filtering module serves as input to the third detection branch. The RoIAlign layer of the third detection branch processes the output of the hard ROI filtering module and then uses the processed output as input to the average pooling layer of the online hard sample mining module. The higher the loss of a Region of Interest (RoI), the worse the RoI performs with the current model, meaning the RoI is more difficult to train. Therefore, RoIs with losses exceeding a certain threshold are considered difficult RoIs. OHEM filters difficult RoIs based on their loss values and categorizes them along... Figure 2 Gradient updates are performed in the direction indicated by the arrow within the online hard sample mining module.
[0034] 2) Training and testing DeepTCT includes the following steps:
[0035] a) Basic training and testing: The TCT images in the training set are processed with data augmentation and class balance, and then the images in the training set are input into DeepTCT for training. Finally, the trained network DeepTCT is applied to the test set to output the classification and localization results of cervical cells.
[0036] The data augmentation methods in this embodiment include random translation, scaling, ZCA whitening, feature standardization, horizontal flip and vertical flip, and rotation, which increase the number of samples in the training set.
[0037] Training set class balancing ensures that the number of cell images for each class in the training set is equal. Specifically, training set class balancing involves generating new samples using seven methods from the data augmentation module, then extracting the required number of samples from these new samples and adding them to the original training set, ensuring that the number of cell images (equivalent to the number of samples) for each class is equal. It is important to note that the number of new samples added by the training set balancing method should not be too large, otherwise it will lead to overfitting of the deep neural network.
[0038] b) Iterative training and testing of the basic trained DeepTCT, which includes the following steps:
[0039] b1) Search for cervical cells detected by DeepTCT in each image of the test set. If the classification score of the cervical cell is greater than the set threshold, retain the label of the cervical cell; otherwise, discard the label. In this step, the threshold is set to 0.8. Labels with a classification score less than 0.8 are discarded, and those with a classification score greater than 0.8 are retained and used as new training samples to be added to the training set for retraining.
[0040] b2) Samples generated by DeepTCT that have reached the set classification score are put into the training set and used together with the existing images in the training set for DeepTCT training.
[0041] b3) Apply the trained DeepTCT to the test set to output the classification and localization results of cervical cells;
[0042] b4) Repeat steps b1)-b3) until the preset number of iterations is reached. In this embodiment, the number of iterations is set to 3. Basic training and testing can be considered as 0 iterations.
[0043] 3) Apply the DeepTCT obtained in step 2) to cervical cell morphology detection.
[0044] The effectiveness of the automatic TCT smear detection method proposed in the above embodiments will be verified through experiments below:
[0045] The experiments were conducted using the CQMU dataset. The CQMU dataset, created by Chongqing Medical University, contains 979 images. The images in this dataset are saved in PNG format. The cells labeled in each image belong to the same cell category. According to the TBS 2014 reporting method, the CQMU dataset contains seven types of atypical cells: ① Negative for Intraepithelial Lesion or Malignancy (NILM); ② Atypical Squamous Cells of Undetermined Significance (ASC-US); ③ Atypical Squamous Cells, cannot exclude HSIL (ASC-H); ④ Low-grade Squamous Intraepithelial Lesion (LSIL); ⑤ High-grade Squamous Intraepithelial Lesion (HSIL); ⑥ Squamous Cell Carcinoma (SCC); ⑦ Adenocarcinoma Glandular Carcinoma, AGC), where ASC-US and above are considered abnormal cells (positive smears). This example uses four types of abnormal cells—ASC-US, ASC-H, LSIL, and HSIL—from the CQMU dataset for experiments. The entire dataset is divided into training, validation, and test sets. Table 1 lists the sample size of the training, validation, and test sets of the original CQMU dataset.
[0046] Table 1. Information on the original CQMU dataset
[0047]
[0048] Note: Each image is treated as a sample, and the cells in each image are all of the same category.
[0049] The results obtained by performing data augmentation and class balance on the original CQMU dataset are shown in Table 2.
[0050] Table 2. Number of additional samples required for each class during training set balancing.
[0051]
[0052] The cervical cell detection task (classification and localization) was implemented using MMDetection, an open-source object detection toolkit based on PyTorch. The final dataset information input into MMDetection is shown in Table 3. A subset of samples (5% of the training set, with the same number of samples per class) was also labeled as a validation set.
[0053] Table 3 shows the dataset information input to MMDetection.
[0054]
[0055] Hyperparameters have a significant impact on the performance of DeepTCT. The key hyperparameter settings in DeepTCT are shown in Table 4.
[0056] Table 4 Optimized DeepTCT hyperparameters
[0057]
[0058] The experiment used mean average precision (mAP) and mean average recall (mAR) as evaluation metrics. The experimental results are shown in Table 5. Table 5 shows that the iterative process significantly improves detection performance. More iterations result in better performance.
[0059] Table 5. Cell detection results
[0060]
[0061] We also compared the performance of the state-of-the-art network Cascade R-CNN under the same operating environment. Cascade R-CNN had an mAP of 6.9% and an mAR of 35.6%. Our TCT smear automatic detection method significantly outperformed Cascade R-CNN in both mAP and mAR.
[0062] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.
Claims
1. An automatic detection method for TCT smears, characterized in that: Includes the following steps: 1) Construct the cell detection model DeepTCT, which includes a backbone network PVTv2 with a feature pyramid network for extracting feature images, a region proposal network for generating preliminary region proposals, three detection branches for obtaining the classification and localization of cervical cells, and an online hard sample mining module. The three detection branches have the same structure. Each detection branch includes a RoIAlign layer, an average pooling layer connected to the RoIAlign layer, a Flatten layer connected to the average pooling layer, a first fully connected layer connected to the Flatten layer, a second fully connected layer for bounding box regression connected to the first fully connected layer, and a third fully connected layer for bounding box classification connected to the first fully connected layer. The output of PVTv2 is used as the input to the region proposal network. The output of PVTv2 and the output of the region proposal network together form a region of interest of arbitrary size, which is used as the input to the first detection branch. The output of the second fully connected layer of the first detection branch is used as the input to the second detection branch, and the output of the second fully connected layer of the second detection branch is used as the input to the third detection branch. The online hard sample mining module includes an average pooling layer, a Flatten layer connected to the average pooling layer, a first fully connected layer connected to the Flatten layer, a second fully connected layer for bounding box regression connected to the first fully connected layer, and a third fully connected layer for bounding box classification connected to the first fully connected layer. The average pooling layer, Flatten layer, first fully connected layer, second fully connected layer, and third fully connected layer in both the online hard sample mining module and the third detection branch have the same structure, and these layers share weights. The online hard sample mining module also includes a hard region of interest (ROI) filtering module. The outputs of the second and third fully connected layers of the third detection branch serve as the inputs to the hard ROI filtering module, and the output of the hard ROI filtering module serves as the input to the third detection branch. The RoIAlign layer of the third detection branch processes the output of the hard ROI filtering module and then outputs it as the input to the average pooling layer of the online hard sample mining module. 2) Training and testing DeepTCT includes the following steps: a) Basic training and testing: The TCT images in the training set are processed with data augmentation and class balance, and then the images in the training set are input into DeepTCT for training. Finally, the trained network DeepTCT is applied to the test set to output the classification and localization results of cervical cells. b) Iterative training and testing of the basic trained DeepTCT, which includes the following steps: b1) Search for cervical cells detected by DeepTCT in each image of the test set. If the classification score of the cervical cell is greater than the set threshold, retain the label of the cervical cell; otherwise, discard the label. b2) Samples generated by DeepTCT that have reached the set classification score are put into the training set and used together with the existing images in the training set for DeepTCT training. b3) Apply the trained DeepTCT to the test set to output the classification and localization results of cervical cells; b4) Repeat steps b1) - b3) until the preset number of iterations is reached; 3) Apply the DeepTCT obtained in step 2) to cervical cell morphology detection.
2. The automatic detection method for TCT smears according to claim 1, characterized in that: The threshold value set in step b1) is 0.8.