TCT smear automatic detection system based on multi-level hard polarization feature fusion
By introducing multi-level hard polarization feature fusion and hard polarization self-attention mechanism in the YOLO-TCT model, the problems of time-consuming and labor-intensive and information loss of traditional cervical cancer screening methods are solved, and efficient and accurate cervical cell detection is achieved. .
Patent Information
- Application Number
- CN202411012065.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-26
- Publication Date
- 2024-11-22
- Estimated Expiration
- 2044-07-26
AI Technical Summary
Traditional cervical cancer screening methods rely on experienced doctors to manually review smears, which are time-consuming and labor-intensive and susceptible to subjective factors. The loss of information in deep learning models in high-resolution images leads to insufficient detection efficiency and accuracy.
Using the YOLO-TCT model based on multi-level hard polarization feature fusion, the C2f module in the YOLOv8 backbone network is replaced with the MLHP-C2f module, and combining the multi-level feature fusion and hard polarization self-attention mechanism to enhance the high-resolution image. Characterization and reduce information loss.
The accuracy and efficiency of cervical cell detection are significantly improved, and the YOLO-TCT model performs better than other models in high-resolution images, achieving higher detection accuracy and lower information loss.
Smart Images

Figure CN119007194A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the fields of medical technology and image processing technology, and in particular to a width learning system for image classification and a training method thereof and an image classification method. Background Art
[0002] Cervical cancer is one of the most common gynecological malignancies, and its morbidity and mortality rates remain high in many countries. However, the development of cervical cancer is relatively slow, and it usually takes years or even decades from precancerous lesions to invasive cancer. Studies have shown that through early screening and effective treatment, the clinical cure rate of cervical precancerous lesions can be close to 100%. This not only effectively prevents further deterioration of the disease, but also significantly reduces the morbidity and mortality of cervical cancer. Therefore, early screening of cervical lesions is extremely important in preventing the occurrence of cervical cancer.
[0003] With the continuous advancement of medical technology, cervical cancer screening methods have also been significantly improved. Among them, Thinprep Cytologic Test (TCT), as the most advanced cervical cytology screening technology, plays a key role in the early screening and diagnosis of cervical cancer. However, traditional TCT screening relies on experienced doctors to manually review smears under a microscope. This method is not only time-consuming and laborious, but also easily limited by the doctor's subjective factors and professional level.
[0004] In recent years, deep learning has made significant progress in the field of medical image processing, especially in object detection. Many methods use deep learning models to solve the task of cervical cell detection, such as Faster R-CNN, Cascade R-CNN, SSD, etc. Although these methods have improved the detection accuracy of cervical cells to a certain extent, they take a long time to train on the data set. In contrast, the YOLO series of algorithms have attracted attention due to their efficient real-time detection performance. Previously, many studies have used the YOLO series of models for cervical cell detection, such as YOLOv3, YOLOv5, etc. However, the information loss problem caused by a series of operations such as overly deep network layers and convolution makes these methods often perform unsatisfactory in high-resolution images.
[0005] Therefore, developing an efficient automatic detection system for TCT smear cervical cells remains an urgent problem to be solved. Summary of the invention
[0006] In view of this, the purpose of the present invention is to provide a TCT smear automatic detection system based on multi-level hard polarization feature fusion to solve the technical problem of improving the efficiency and accuracy of TCT smear detection.
[0007] The TCT smear automatic detection system based on multi-level hard polarization feature fusion of the present invention includes a YOLO-TCT model, wherein the YOLO-TCT model is obtained by improving YOLOv8, wherein the improvement is to replace the C2f module in the YOLOv8 backbone network with an MLHP-C2f module, wherein the MLHP-C2f module includes a first convolutional layer, a Split layer, n Bottleneck layers, a Concat layer, a second convolutional layer and a hard polarization self-attention layer connected in sequence, wherein the outputs of the first convolutional layer, the Split layer and each Bottleneck layer are directly input into the Concat layer, and the input of the first convolutional layer and the output of the second convolutional layer are fused and then input into the hard polarization self-attention layer; wherein the hard polarization self-attention layer is obtained by replacing the Sigmoid function in the polarization self-attention module with the Hard Sigmoid function, and the formula of the Hard Sigmoid function is as follows:
[0008] HardSigmoid(x)=max(0,min(1,α*x+β)) (1)
[0009] Among them, α is the slope parameter and β is the bias parameter; α is set to 1 / 6 and β is set to 1 / 3 to ensure that when x is in the range of [-3,3], the Hard Sigmoid function can closely approximate the Sigmoid function. For inputs outside this range, the output is limited to between 0 and 1.
[0010] Furthermore, the YOLO-TCT model is trained, tested and validated by a dataset consisting of the following seven categories of cell images:
[0011] ① No intraepithelial lesions or malignant cells were seen;
[0012] ②Image of atypical squamous cells of unknown significance;
[0013] ③ Images of atypical squamous cells that cannot exclude high-grade squamous intraepithelial lesions;
[0014] ④ Images of low-grade squamous intraepithelial lesion cells;
[0015] ⑤Image of high-grade squamous intraepithelial lesion cells;
[0016] ⑥Image of squamous cell carcinoma;
[0017] ⑦ Images of atypical adenocarcinoma of unclear diagnostic significance.
[0018] Beneficial effects of the present invention:
[0019] The present invention is based on the TCT smear automatic detection system of multi-level hard polarization feature fusion, which uses TCT smear images as input during detection and outputs cell location and classification results. The YOLO-TCT model included in the TCT smear automatic detection system based on multi-level hard polarization feature fusion of the present invention adopts an original MLHP-C2f module in the backbone network. This module includes two aspects: first, the features are passed to subsequent layers through multi-level feature fusion operations to help the deep network obtain useful information; second, the hard polarization self-attention is used to ensure that the information loss of the image is as small as possible while enhancing the feature expression of high-resolution images. Ablation experiments have shown that feature fusion when adding ML, PSA or HPSA separately on the basis of YOLOv8-L will have a negative impact on the performance of the model. Only when ML and HPSA are combined, the model achieves the best mAP. 0.5:0.95 and mAP 0.5 , demonstrating the rationality and superiority of the MLHP structure. The experimental results of different models show that in the detection of high-resolution cervical cell TCT smear images, the YOLO-TCT detection accuracy using the MLHP-C2f module is significantly better than other model systems. BRIEF DESCRIPTION OF THE DRAWINGS
[0020] Figure 1 This is the YOLO-TCT architecture diagram;
[0021] Figure 2 It is the backbone network structure.
[0022] Figure 3 It is a MLHP-C2f structure.
[0023] Figure 4 It is a HPSA structure.
[0024] Figure 5 It is a Head structure.
[0025] Figure 6 This is a flow chart of the TCT smear detection method.
[0026] Figure 7 This is a diagram showing the results of cervical cell testing. DETAILED DESCRIPTION
[0027] The present invention will be further described below in conjunction with the accompanying drawings and embodiments.
[0028] In this embodiment, the TCT smear automatic detection system based on multi-level hard polarization feature fusion includes a YOLO-TCT model, and the YOLO-TCT model is obtained by improving YOLOv8, and the improvement is to replace the C2f module in the YOLOv8 backbone network with an MLHP-C2f module.
[0029] like Figure 1 As shown in the figure, the YOLO-TCT model improved based on YOLOv8 consists of three main components: backbone network (Backbone), neck (Neck) and head (Head).
[0030] The main function of the backbone network of the YOLO-TCT model is to extract features from the input image. Figure 2 As shown, the backbone network in YOLOv8 includes the stem node (Stem) layer, stage (Stage) layer 1, stage layer 2, stage layer 3 and stage layer 4, which correspond to Figure 1 The P1, P2, P3, P4, and P5 layers in the backbone network. The C2f module plays an important role in the feature fusion of the backbone network. It is responsible for integrating features from different layers to capture richer semantic information. In TCT image detection, the detection task faces many small-sized cells and a small number of medium- and large-sized cells. Optimizing and enhancing the C2f module is crucial to improving the detection capability of the model.
[0031] like Figure 3 As shown, in this embodiment, the MLHP-C2f module that replaces the C2f module includes a first convolutional layer, a Split layer, n Bottleneck layers, a Concat layer, a second convolutional layer, and a hard polarization self-attention layer that are connected in sequence. The outputs of the first convolutional layer, the Split layer, and each Bottleneck layer are also directly input into the Concat layer. The input of the first convolutional layer and the output of the second convolutional layer are fused and input into the hard polarization self-attention layer; the hard polarization self-attention layer is obtained by replacing the Sigmoid function in the polarization self-attention module with the Hard Sigmoid function. The formula of the Hard Sigmoid function is as follows:
[0032] HardSigmoid(x)=max(0,min(1,α*x+β)) (1)
[0033] Among them, α is the slope parameter and β is the bias parameter; α is set to 1 / 6 and β is set to 1 / 3 to ensure that when x is in the range of [-3,3], the Hard Sigmoid function can closely approximate the Sigmoid function. For inputs outside this range, the output is limited to between 0 and 1.
[0034] The MLHP-C2f module combines two key technologies: Multi-Level feature fusion and Hard Polarized Self-Attention (HPSA).
[0035] In multi-level feature fusion, skip connections are used to directly pass input information to subsequent layers of the network, facilitating residual fusion between multiple levels. This approach not only maintains the strength of the gradient, but also enhances the efficiency of gradient flow, which helps to build deeper networks to learn more complex features. In addition, the presence of residual connections allows each layer to focus on learning the residual between input and output, rather than the complete output, thereby simplifying the learning task to a certain extent. Specifically, in the MLHP-C2f structure, the residual connections are located before the HPSA. These connections enable the HPSA at a deeper level to learn the residual component more effectively, thereby improving the feature expression ability of the network.
[0036] The Hard Polarized Self-Attention (HPSA) layer is a fine-grained enhanced attention mechanism proposed on the basis of Polarized Self-Attention (PSA), combining polarized filtering and high dynamic range (HDR)-like enhancement techniques. Polarized filtering: This technique achieves high internal resolution by selectively collapsing the input tensor along specific dimensions while maintaining the resolution in the orthogonal dimensions, ensuring the integrity of the spatial resolution during the filtering process and facilitating detailed feature extraction. HDR enhancement: After polarized filtering, PSA adopts an HDR-like enhancement strategy to increase the dynamic range of the attention signal. This is achieved by performing a Softmax normalization on the bottleneck tensor and then using a sigmoid function for tone mapping. This operation significantly enhances the dynamic range of the attention mechanism, thereby improving the recovery of details and contrast in feature representation. The purpose of PSA is to enhance fine-grained computer vision tasks with minimal computational overhead. To further reduce the computational cost, the sigmoid function in PSA is replaced with a hard sigmoid function to obtain the Hard Polarized Self-Attention layer.
[0037] Compared to the Sigmoid function, the Hard Sigmoid function is computationally simpler and more efficient because it only involves maximum and minimum operations and simple linear transformations, which reduces the computational burden. At the same time, the calculation of Hard Sigmoid is more stable because when the absolute value of the input of the Sigmoid function becomes larger, its output approaches 0 or 1, causing the gradient to disappear. In contrast, the output of Hard Sigmoid fluctuates between 0 and 1, avoiding this problem. In addition, since the output of Hard Sigmoid is truncated, inputs exceeding a certain threshold are cut to 1, thereby enhancing the sensitivity of neurons to the input. This is conducive to the network to produce sparse representations.
[0038] The specific structure of the hard polarization self-attention layer (HPSA) is as follows Figure 4As shown in Figure 2, HPSA consists of two branches: a channel-only branch and a space-only branch. Each branch applies polarization filtering and HDR enhancement strategies, but effectively captures and enhances relevant features in different dimensions (channel and space).
[0039] The attention calculation formula for the channel branch only is as follows:
[0040]
[0041] Among them, σ1 and σ2 represent two tensor reshaping operators used to adjust the shape of the tensor for subsequent calculations, F SM represents Softmax, F HSG Represents Hard Sigmoid. A 1×1 convolution W is used. q (X) and W v (X) The input feature X is converted into Q and V, where the channel of Q is completely compressed, while the channel dimension of V remains at a relatively high level (that is, C / 2). Because the channel dimension of Q is compressed, as mentioned above, it is necessary to enhance the information through HDR, so Softmax is used here to enhance the information of Q. Then Q and V are matrix multiplied, and then 1×1 Convolution and Layer Normalization are added to increase the C / 2 dimension on the channel to C. Finally, the Hard Sigmoid function is used to keep all parameters between 0 and 1.
[0042] The attention calculation formula for the spatial branch only is as follows
[0043] A sp (X) = F HSG |σ3(F SM (σ1(F GP (W q (X))))×σ2(W v (X)))] (1)
[0044] Among them, σ1, σ2 and σ3 represent three tensor reshaping operators, F GP represents GlobalPooling, F SM represents Softmax, F HSG It represents Hard Sigmoid. You can see that a 1×1 convolution W is used here first. q (X) and W v(X) The input feature X is converted to Q and V. For the Q feature, GlobalPooling is used to compress the spatial dimension and convert it to a size of 1×1; while the spatial dimension of the V feature is kept at a relatively large level (H×W). Since the spatial dimension of Q is compressed, Softmax is used to enhance the information of Q. Then Q and K are matrix multiplied, and finally reshape and HardSigmoid are connected to keep all parameters between 0 and 1.
[0045] The spatial-only branch is connected in series after the channel-only branch, so the attention calculation formula for the entire HPSA is as follows:
[0046] HPSA series (X) = Z sp (Z ch )=A sp (A ch (X)⊙ ch X)⊙ sp A ch (X)⊙ ch X (4)
[0047] In the TCT cervical cell detection task, the HPSA mechanism is able to maintain relatively good performance in such a fine-grained pixel-level task, largely due to the minimal compression applied in both spatial and channel dimensions, resulting in minimal information loss.
[0048] The neck can be regarded as the layer connecting the backbone network and the head, which is mainly responsible for effectively fusing the single-scale or multi-scale features output by the backbone network. The neck in YOLO-TCT shows the structure of Feature Pyramid Network (FPN) combined with Path Aggregation Network (PAN), such as Figure 1 As shown in the Neck part. FPN uses multi-scale features to detect objects of different sizes, while PAN aggregates feature maps from different levels. In this combined structure, the FPN layer propagates strong semantic features from top to bottom, while the feature pyramid transmits strong positioning features from bottom to top, promoting the aggregation of features from different detection layers between various backbone network layers.
[0049] The head is responsible for generating the final cell localization and classification results. Part of the head structure of YOLO-TCT is as follows Figure 5 As shown, three Figure 6 The parts shown in together constitute the head of YOLO-TCT, similar to Figure 2The structure presented in the head part. The loss of the head includes border loss and category loss. Border loss (regression loss): used to calculate the difference between the predicted box and the actual box in target detection.
[0050] Distribution Focal Loss (DFL) and Complete Intersection over Union (CIoU) are used as regression loss functions to measure the distance between the predicted box and the actual box.
[0051] Category loss (classification loss): used to calculate the dissimilarity between the predicted category and the actual category in target detection. Binary Cross Entropy (BCE) loss is used as the classification loss function to measure the gap between the predicted category and the actual category.
[0052] The YOLO-TCT model in this embodiment is trained, tested and verified by a data set consisting of the following seven categories of cell images:
[0053] ① No intraepithelial lesions or malignant cells were seen;
[0054] ②Image of atypical squamous cells of unknown significance;
[0055] ③ Images of atypical squamous cells that cannot exclude high-grade squamous intraepithelial lesions;
[0056] ④ Images of low-grade squamous intraepithelial lesion cells;
[0057] ⑤Image of high-grade squamous intraepithelial lesion cells;
[0058] ⑥Image of squamous cell carcinoma;
[0059] ⑦ Images of atypical adenocarcinoma of unclear diagnostic significance.
[0060] When the number of data sets is insufficient, a variety of data enhancement methods can be used to enhance the data set. This embodiment uses six data enhancement methods including Mosaic, Cutout, Rotation, Crop, Horizontal and Vertical Flip. Among these various data enhancement methods, the introduction of Mosaic is particularly important. It helps the model better understand the relative positions and background information between different cells by randomly selecting and scaling four images for splicing.
[0061] Table 1 Original dataset information
[0062]
[0063] Table 2 Dataset information after data enhancement
[0064]
[0065] The following experiments are conducted to verify the effect of the TCT smear automatic detection system based on multi-level hard polarization feature fusion in this embodiment.
[0066] In the experiment, all methods were implemented using Ultralytics and run on an NVIDIA GTX 3090Ti graphics card with 10,752 cores and 24G global memory. For the cervical cell detection task, mean Average Precision (mAP) and mean Average Recall (mAR) were used as evaluation indicators. Each model was trained for 200 epochs, and the highest mAP was selected among these 200 epochs. 0.5 The results are retained as the final test results. The experimental results of all models are obtained by averaging the results of five experiments. The bold data shows the best performance in a column. Some important hyperparameters of the model in the experiment are shown in Table 3.
[0067] Table 3 Hyperparameters optimized by YOLO-TCT
[0068]
[0069] Ablation experiments verify the effects of each module of YOLO-TCT. Table 4 shows the results of the ablation experiments.
[0070] Table 4 Ablation experiment of YOLO-TCT
[0071]
[0072] When there is no additional module (i.e., only YOLOv8-L is used), the detection performance of the model is shown in the first row of Table 4, which we use as the baseline. When only ML, PSA, or HPSA is added, the mAP of the model is 2.3 times that of the baseline. 0.5:0.95 and mAP 0.5 Both decreased, indicating that using only these modules for feature fusion had a negative impact on the performance of the model. Only when ML and HPSA were combined, as shown in the sixth row of Table 5, the model achieved the best mAP 0.5:0.95 and mAP 0.5 , demonstrating the rationality and superiority of the MLHP structure, which provides the greatest help in enhancing the detection performance of the model.
[0073] In addition, the experiments also compare the performance of the most advanced models (Faster R-CNN, Cascade R-CNN, RT-DETR, Deformable DETR, YOLOv3-U, YOLOv5-X, YOLOv6-L and YOLOv8-L) on the test set.
[0074] Table 5 Cervical cell detection performance of different methods
[0075]
[0076] Table 5 shows the results of the comparative experiment. As can be seen from Table 5, YOLO-TCT's mAP 0.5:0.95 and mAP 0.5 It is the highest among all models, indicating that the YOLO-TCT model in this embodiment is more suitable for the TCT smear cervical cell detection task.
[0077] In order to more intuitively demonstrate the detection performance of the YOLO-TCT model on TCT smear cervical cells, some detection results of the YOLO-TCT model were selected and Figure 7 A visual display is given in . Figure 7 There are five categories included, each of which gives an example, and different colored borders are used to represent different categories. Figure 7 The left column shows the ground truth of cervical cells, which is completed by professional doctors. Figure 7 The right column shows the detection result of abnormal cervical cells in TCT smear by YOLO-TCT. Figure 7 It can be seen that each type of cell is accurately located and classified.
[0078] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present invention can be modified or replaced by equivalents without departing from the purpose and scope of the technical solutions of the present invention, which should be included in the scope of the claims of the present invention.
Claims
1. TCT smear automatic detection system based on multi-level hard polarization feature fusion, characterized by: The invention comprises a YOLO-TCT model, wherein the YOLO-TCT model is obtained by improving YOLOv8, wherein the C2f module in the YOLOv8 backbone network is replaced by an MLHP-C2f module, wherein the MLHP-C2f module comprises a first convolutional layer, a Split layer, n Bottleneck layers, a Concat layer, a second convolutional layer and a hard polarization self-attention layer connected in sequence, wherein the outputs of the first convolutional layer, the Split layer and each Bottleneck layer are directly input into the Concat layer, and the input of the first convolutional layer and the output of the second convolutional layer are fused and then input into the hard polarization self-attention layer; the hard polarization self-attention layer is obtained by replacing the Sigmoid function in the polarization self-attention module with the Hard Sigmoid function, and the formula of the Hard Sigmoid function is as follows: HardSigmoid (x)=max(0,min(1,α*x+β)) (1) Among them, α is the slope parameter and β is the bias parameter; α is set to 1 / 6 and β is set to 1 / 3 to ensure that when x is in the range of [-3,3], the Hard Sigmoid function can closely approximate the Sigmoid function. For inputs outside this range, the output is limited to between 0 and 1.
2. The TCT smear automatic detection system based on multi-level hard polarization feature fusion according to claim 1 is characterized in that: The YOLO-TCT model is trained, tested and validated by a dataset consisting of seven categories of cell images: ① No intraepithelial lesions or malignant cells were seen; ②Image of atypical squamous cells of unknown significance; ③ Images of atypical squamous cells that cannot exclude high-grade squamous intraepithelial lesions; ④ Images of low-grade squamous intraepithelial lesion cells; ⑤Image of high-grade squamous intraepithelial lesion cells; ⑥Image of squamous cell carcinoma; ⑦ Images of atypical adenocarcinoma of unclear diagnostic significance.
Citation Information
Patent Citations
Cervical cancer cell detection method based on Yolov5l model
CN117274220A
Digital image classification method for cervical fluid-based cells based on a deep learning detection model
US20220083762A1
Cited By
YOLOv11-based cervical fluorescent gray-scale image lesion area detection method and related device
CN121053081A
