TCT smear automatic detection system based on multi-level hard polarization feature fusion

Abstract

The application discloses a TCT smear automatic detection system based on multi-level hard polarization feature fusion, which comprises a YOLO-TCT model, the YOLO-TCT model is obtained by improving YOLOv8, and the improvement lies in that a C2f module in a YOLOv8 backbone network is replaced by an MLHP-C2f module. Through an ablation experiment, it is proved that feature fusion when ML, PSA or HPSA is separately added on the basis of YOLOv8-L has a negative effect on the performance of the model, and only when ML and HPSA are combined, the model realizes the best mAP 0.5:0.95 and mAP 0.5 , which shows the rationality and superiority of the MLHP structure. The comparison experiment results of different models show that in the detection of high-resolution cervical cell TCT smear images, the detection accuracy of the YOLO-TCT is significantly better than those of other model systems.

Description

Technical Field

[0001] This invention relates to the fields of medical technology and image processing technology, and in particular to a width learning system for image classification, its training method, and an image classification method. Background Technology

[0002] Cervical cancer is one of the most common gynecological malignancies, and its incidence and mortality rates remain high in many countries. However, the progression of cervical cancer is relatively slow, typically taking years or even decades from precancerous lesions to invasive cancer. Studies have shown that through early screening and effective treatment, the clinical cure rate for precancerous cervical lesions can approach 100%. This not only effectively prevents further deterioration of the disease but also significantly reduces the incidence and mortality of cervical cancer. Therefore, early screening for cervical lesions is of paramount importance in preventing the development of cervical cancer.

[0003] With the continuous advancement of medical technology, cervical cancer screening methods have also seen significant improvements. Among them, the Thinprep Cytologic Test (TCT), as the most advanced cervical cytology screening technology, plays a crucial role in the early screening and diagnosis of cervical cancer. However, traditional TCT screening relies on experienced physicians manually reviewing smears under a microscope. This method is not only time-consuming and labor-intensive, but also easily limited by the physician's subjective factors and professional level.

[0004] In recent years, deep learning has made significant progress in the field of medical image processing, especially in object detection. Many methods employ deep learning models to solve cervical cell detection tasks, such as Faster R-CNN, Cascade R-CNN, and SSD. While these methods have improved the accuracy of cervical cell detection to some extent, they require long training times on datasets. In contrast, the YOLO series of algorithms has attracted attention due to its efficient real-time detection performance. Previous studies have also utilized YOLO models for cervical cell detection, such as YOLOv3 and YOLOv5. However, the information loss caused by excessively deep network layers and convolutional operations often results in unsatisfactory performance in high-resolution images.

[0005] Therefore, developing an efficient automated TCT cervical cell detection system remains an urgent problem to be solved. Summary of the Invention

[0006] In view of this, the purpose of this invention is to provide an automatic TCT smear detection system based on multi-level hard polarization feature fusion, so as to solve the technical problem of improving the efficiency and accuracy of TCT smear detection.

[0007] This invention relates to an automatic TCT smear detection system based on multi-level hard polarization feature fusion, comprising a YOLO-TCT model. The YOLO-TCT model is an improvement upon YOLOv8, specifically replacing the C2f module in the YOLOv8 backbone network with an MLHP-C2f module. The MLHP-C2f module comprises a first convolutional layer, a split layer, n Bottleneck layers, a concat layer, a second convolutional layer, and a hard polarization self-attention layer, connected sequentially. The outputs of the first convolutional layer, the split layer, and each Bottleneck layer are directly input to the concat layer. The input of the first convolutional layer and the output of the second convolutional layer are fused and then input to the hard polarization self-attention layer. The hard polarization self-attention layer is obtained by replacing the Sigmoid function in the polarization self-attention module with a Hard Sigmoid function. The formula for the Hard Sigmoid function is as follows:

[0008] HardSigmoid(x)=max(0,min(1,α*x+β)) (1)

[0009] Where α is the slope parameter and β is the bias parameter; α is set to 1 / 6 and β is set to 1 / 3 to ensure that the Hard Sigmoid function can closely approximate the Sigmoid function when x is in the range of [-3,3]. For inputs outside this range, the output is restricted to between 0 and 1.

[0010] Furthermore, the YOLO-TCT model is trained, tested, and validated using a dataset consisting of cell images from the following seven categories:

[0011] ① No images of intraepithelial lesions or malignant cells were observed;

[0012] ② Images of atypical squamous cells of unclear significance;

[0013] ③ Images of atypical squamous cells that cannot rule out high-grade squamous intraepithelial lesions;

[0014] ④ Images of cells with low-grade squamous intraepithelial lesions;

[0015] ⑤ Images of cells with high-grade squamous intraepithelial lesions;

[0016] ⑥ Images of squamous cell carcinoma;

[0017] ⑦ Images of atypical adenocarcinoma with no clear diagnostic significance.

[0018] The beneficial effects of this invention are:

[0019] This invention presents an automatic TCT smear detection system based on multi-level hard polarization feature fusion. During detection, the system uses TCT smear images as input and outputs cell localization and classification results. The YOLO-TCT model incorporated in this system employs a unique MLHP-C2f module in its backbone network. This module has two aspects: first, it passes features to subsequent layers through multi-level feature fusion operations, helping deep networks acquire useful information; second, it enhances the feature representation of high-resolution images while minimizing information loss through hard polarization self-attention. Ablation experiments demonstrate that adding ML, PSA, or HPSA alone to YOLOv8-L for feature fusion negatively impacts model performance; only when ML and HPSA are combined does the model achieve optimal mAP. 0.5:0.95 and mAP 0.5 This demonstrates the rationality and superiority of the MLHP structure. Comparative experiments with different models show that, in detecting high-resolution cervical cell TCT smear images, the YOLO-TCT system using the MLHP-C2f module significantly outperforms other model systems in terms of detection accuracy. Attached Figure Description

[0020] Figure 1 YOLO-TCT architecture diagram;

[0021] Figure 2 It forms the backbone network structure.

[0022] Figure 3 It is an MLHP-C2f structure.

[0023] Figure 4 It has an HPSA structure.

[0024] Figure 5 It is a Head structure.

[0025] Figure 6 This is a flowchart of the TCT smear testing method.

[0026] Figure 7 This is a diagram showing the results of cervical cell testing. Detailed Implementation

[0027] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0028] In this embodiment, the TCT smear automatic detection system based on multi-level hard polarization feature fusion includes the YOLO-TCT model, which is obtained by improving YOLOv8. The improvement is that the C2f module in the YOLOv8 backbone network is replaced with the MLHP-C2f module.

[0029] like Figure 1 As shown, the YOLO-TCT model based on YOLOv8 consists of three main components: the backbone, the neck, and the head.

[0030] The main function of the backbone network in the YOLO-TCT model is to extract features from the input image. For example... Figure 2 As shown, the backbone network in YOLOv8 includes a stem layer, stage layer 1, stage layer 2, stage layer 3, and stage layer 4, which correspond to... Figure 1 Layers P1, P2, P3, P4, and P5 in the backbone network. The C2f module plays a crucial role in feature fusion within the backbone network, integrating features from different layers to capture richer semantic information. In TCT image detection, the task faces numerous small cells and a small number of medium- and large cells. Optimizing and enhancing the C2f module is essential for improving the model's detection capabilities.

[0031] like Figure 3 As shown, the MLHP-C2f module replacing the C2f module in this embodiment includes a first convolutional layer, a split layer, n Bottleneck layers, a concat layer, a second convolutional layer, and a hard polarization self-attention layer connected in sequence. The outputs of the first convolutional layer, the split layer, and each Bottleneck layer are directly input to the concat layer. The input of the first convolutional layer and the output of the second convolutional layer are fused and then input to the hard polarization self-attention layer. The hard polarization self-attention layer is obtained by replacing the Sigmoid function in the polarization self-attention module with the Hard Sigmoid function. The formula for the Hard Sigmoid function is as follows:

[0032] HardSigmoid(x)=max(0,min(1,α*x+β)) (1)

[0033] Where α is the slope parameter and β is the bias parameter; α is set to 1 / 6 and β is set to 1 / 3 to ensure that the Hard Sigmoid function can closely approximate the Sigmoid function when x is in the range of [-3,3]. For inputs outside this range, the output is restricted to between 0 and 1.

[0034] The MLHP-C2f module combines two key technologies: multi-level feature fusion and hard polarized self-attention (HPSA).

[0035] In multi-level feature fusion, skip connections are used to directly pass input information to subsequent layers of the network, facilitating residual fusion across multiple layers. This approach not only preserves the strength of gradients but also enhances the efficiency of gradient flow, enabling the construction of deeper networks to learn more complex features. Furthermore, the presence of residual connections allows each layer to focus on learning the residual between the input and output, rather than the complete output, thus simplifying the learning task to some extent. Specifically, in the MLHP-C2f architecture, residual connections precede HPSA. These connections enable deeper HPSA layers to learn residual components more effectively, thereby improving the network's feature representation capabilities.

[0036] Hard Polarized Self-Attention (HPSA) is a fine-grained attention enhancement mechanism proposed based on Polarized Self-Attention (PSA), combining polarized filtering and High Dynamic Range (HDR) enhancement techniques. Polarized filtering: This technique achieves high internal resolution by selectively folding the input tensor along specific dimensions while maintaining resolution in orthogonal dimensions, ensuring the integrity of spatial resolution during filtering and facilitating detailed feature extraction. HDR enhancement: After polarized filtering, PSA employs an HDR-like enhancement strategy to increase the dynamic range of the attention signal. This is achieved by performing Softmax normalization on the bottleneck tensor and then using a sigmoid function for tone mapping. This operation significantly enhances the dynamic range of the attention mechanism, thereby improving the recovery of details and contrast in the feature representation. The goal of PSA is to enhance fine-grained computer vision tasks with minimal computational overhead. To further reduce computational cost, the sigmoid function in PSA is replaced with a Hard Sigmoid function, resulting in the Hard Polarized Self-Attention layer.

[0037] Compared to the Sigmoid function, the Hard Sigmoid function is computationally simpler and more efficient because it only involves maximization and minimization operations and simple linear transformations, thus reducing the computational burden. Furthermore, the Hard Sigmoid is more computationally stable because when the absolute value of the Sigmoid input increases, its output approaches 0 or 1, leading to gradient vanishing. In contrast, the Hard Sigmoid output fluctuates between 0 and 1, avoiding this problem. Moreover, because the Hard Sigmoid output is truncated, inputs exceeding a certain threshold are pruned to 1, thereby enhancing the neuron's sensitivity to input. This is beneficial for the network to generate sparse representations.

[0038] The specific structure of the Hard Polarized Self-Attention Layer (HPSA) is as follows: Figure 4As shown, HPSA consists of two branches: a channel-only branch and a spatial-only branch. Each branch applies polarization filtering and HDR enhancement strategies, but effectively captures and enhances relevant features in different dimensions (channel and spatial).

[0039] The formula for calculating attention for channel-only branches is as follows:

[0040]

[0041] Where σ1 and σ2 represent two tensor reshaping operators used to adjust the shape of the tensor for subsequent calculations, F SM Represents Softmax, F HSG This represents Hard Sigmoid. A 1×1 convolution W was used. q (X) and W v (X) transforms the input features X into Q and V, where the channels of Q are fully compressed, while the channel dimension of V remains at a relatively high level (C / 2). Because the channel dimension of Q is compressed, as mentioned earlier, information enhancement is required through HDR; therefore, Softmax is used to enhance the information of Q. Then, matrix multiplication is performed on Q and V, followed by 1×1 Convolution and Layer Normalization to increase the channel dimension from C / 2 to C. Finally, the Hard Sigmoid function is used to keep all parameters between 0 and 1.

[0042] The formula for calculating attention in spatial branch only is as follows:

[0043] A sp (X)=F HSG |σ3(F SM (σ1(F GP (W q (X))))×σ2(W v (X)))] (1)

[0044] Where σ1, σ2, and σ3 represent three tensor reshaping operators, F GP Indicates GlobalPooling, F SM Represents Softmax, F HSG This represents Hard Sigmoid. As you can see, a 1×1 convolution W is used first. q (X) and W v(X) transforms the input features X into Q and V. For Q features, GlobalPooling is used to compress the spatial dimension, transforming it into a 1×1 size; while the spatial dimension of V features remains at a relatively large level (H×W). Since the spatial dimension of Q is compressed, Softmax is used to enhance the information of Q. Then, Q and K are multiplied by matrix, and finally reshape and HardSigmoid are applied to keep all parameters between 0 and 1.

[0045] The spatial branch is connected in series after the channel branch, therefore the attention calculation formula for the entire HPSA is as follows:

[0046] HPSA series (X)=Z sp (Z ch ) = A sp (A ch (X)⊙ ch X)⊙ sp A ch (X)⊙ ch X (4)

[0047] In the TCT cervical cell detection task, the HPSA mechanism can maintain relatively good performance in such a fine-grained, pixel-level task, largely because it applies minimal compression in both spatial and channel dimensions, resulting in minimal information loss.

[0048] The neck layer can be viewed as connecting the backbone network and the head layer, primarily responsible for effectively fusing single-scale or multi-scale features output from the backbone network. In YOLO-TCT, the neck layer exhibits a structure combining a Feature Pyramid Network (FPN) and a Path Aggregation Network (PAN), such as... Figure 1 The Neck section is shown. FPN utilizes multi-scale features to detect objects of different sizes, while PAN aggregates feature maps from different levels. In this combined structure, the FPN layer propagates strong semantic features from top to bottom, while the feature pyramid propagates strong localization features from bottom to top, facilitating the aggregation of features from different detection layers across various backbone network layers.

[0049] The head is responsible for generating the final cell localization and classification results. A portion of the YOLO-TCT head structure is shown below. Figure 5 As shown, three Figure 6 The parts shown in the image together form the header of the YOLO-TCT, similar to... Figure 2The structure presented in the head portion. The loss of the head includes bounding box loss and class loss. Bounding box loss (regression loss): used to calculate the difference between the predicted box and the actual box in object detection.

[0050] Distribution Focal Loss (DFL) and Complete Intersection over Union (CIoU) are used as regression loss functions to measure the distance between the predicted bounding box and the actual bounding box.

[0051] Category loss (classification loss): Used to calculate the dissimilarity between the predicted and actual categories in object detection. Binary Cross Entropy (BCE) loss is used as the classification loss function to measure the difference between the predicted and actual categories.

[0052] In this embodiment, the YOLO-TCT model was trained, tested, and validated using a dataset consisting of cell images from the following seven categories:

[0053] ① No images of intraepithelial lesions or malignant cells were observed;

[0054] ② Images of atypical squamous cells of unclear significance;

[0055] ③ Images of atypical squamous cells that cannot rule out high-grade squamous intraepithelial lesions;

[0056] ④ Images of cells with low-grade squamous intraepithelial lesions;

[0057] ⑤ Images of cells with high-grade squamous intraepithelial lesions;

[0058] ⑥ Images of squamous cell carcinoma;

[0059] ⑦ Images of atypical adenocarcinoma with no clear diagnostic significance.

[0060] When the dataset is insufficient, various data augmentation methods can be employed to enhance it. This embodiment utilizes six data augmentation methods, including Mosaic, Cutout, Rotation, Crop, Horizontal, and Vertical Flip. Among these methods, the introduction of Mosaic is particularly important. By randomly selecting and scaling four images for stitching, it helps the model better understand the relative positions of different cells and background information.

[0061] Table 1 Original Dataset Information

[0062]

[0063] Table 2 Information on the data augmented dataset

[0064]

[0065] The following experiment verifies the effectiveness of the TCT smear automatic detection system based on multi-level hard polarization feature fusion in this embodiment.

[0066] In the experiments, all methods were implemented using Ultralytics and run on an NVIDIA GTX 3090Ti graphics card with 10,752 cores and 24GB of global memory. For the cervical cell detection task, mean average precision (mAP) and mean average recall (mAR) were used as evaluation metrics. Each model was trained for 200 epochs, and the model with the highest mAP was selected from these 200 epochs. 0.5 The results are retained as the final detection results. The experimental results for all models are obtained by averaging the results of five experiments. The bolded data shows the best performance in one column. Some important hyperparameters of the models in the experiments are shown in Table 3.

[0067] Table 3. Hyperparameters of YOLO-TCT optimization

[0068]

[0069] Ablation experiments were conducted to verify the effectiveness of each module of YOLO-TCT. Table 4 shows the results of the ablation experiments.

[0070] Table 4 Ablation Experiments of YOLO-TCT

[0071]

[0072] When no additional modules are used (i.e., only YOLOv8-L), the model's detection performance is shown in the first row of Table 4, which we use as the baseline. When only ML, PSA, or HPSA is added, the model's mAP compared to the baseline is... 0.5:0.95 and mAP 0.5 The decrease in all parameters indicates that using these modules alone for feature fusion negatively impacts model performance. Only when ML and HPSA are combined, as shown in the sixth row of Table 5, does the model achieve optimal mAP. 0.5:0.95 and mAP 0.5 This demonstrates the rationality and superiority of the MLHP structure, which provides the greatest help in enhancing the detection performance of the model.

[0073] In addition, the experiments compared the performance of state-of-the-art models (Faster R-CNN, Cascade R-CNN, RT-DETR, Deformable DETR, YOLOv3-U, YOLOv5-X, YOLOv6-L and YOLOv8-L) on the test set.

[0074] Table 5 Performance of different methods for cervical cell detection

[0075]

[0076] Table 5 shows the results of the comparative experiment. As can be seen from Table 5, the mAP of YOLO-TCT... 0.5:0.95 and mAP 0.5 The highest value was achieved across all models, indicating that the YOLO-TCT model in this embodiment is more suitable for TCT cervical cell detection tasks.

[0077] To more intuitively demonstrate the detection performance of the YOLO-TCT model on cervical cells in TCT smears, a portion of the detection results from the YOLO-TCT model were selected and presented in [the following text is missing from the original]: Figure 7 The data was visualized in the document. Figure 7 It includes five categories, each with a sample, and different colored borders are used to represent the different categories. Figure 7 The column on the left shows the ground truth annotations of cervical cells, completed by professional doctors; Figure 7 The column on the right shows the detection results of YOLO-TCT on abnormal cervical cells in TCT smears. Figure 7 As you can see, each type of cell has been accurately located and classified.

[0078] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. A TCT smear automatic detection system based on multi-level hard polarization feature fusion, characterized in that: This includes the YOLO-TCT model, which is an improvement on YOLOv8. The improvement involves replacing the C2f module in the YOLOv8 backbone network with the MLHP-C2f module. The MLHP-C2f module comprises a first convolutional layer, a split layer, n Bottleneck layers, a concat layer, a second convolutional layer, and a hard polarization self-attention layer, connected sequentially. The outputs of the first convolutional layer, the split layer, and each Bottleneck layer are directly input to the concat layer. The input of the first convolutional layer and the output of the second convolutional layer are fused and then input to the hard polarization self-attention layer. The hard polarization self-attention layer is obtained by replacing the sigmoid function in the polarization self-attention module with a hard sigmoid function. The formula for the hard sigmoid function is as follows: Where α is the slope parameter and β is the bias parameter; α is set to 1 / 6 and β is set to 1 / 3 to ensure that the Hard Sigmoid function can closely approximate the Sigmoid function when x is in the range of [-3, 3], and the output is restricted to 0 to 1 for inputs outside this range; The hard-polarized self-attention layer consists of two branches: a channel-only branch and a spatial-only branch. The attention calculation formula for the channel-only branch is as follows: where σ1and σ2represent two tensor reshaping operators used to adjust the shape of the tensor for subsequent computations, F SM denotes Softmax, F HSG denotes Hard Sigmoid; The formula for calculating attention for spatial branches is as follows: Where σ3 represents the tensor reshaping operator; The spatial branch is connected in series after the channel branch, and the attention calculation formula for the entire hard polarization self-attention layer is as follows: In the formula.

2. The TCT smear automatic detection system based on multi-level hard polarization feature fusion according to claim 1, characterized in that: The YOLO-TCT model was trained, tested, and validated using a dataset consisting of cell images from the following seven categories: ① No images of intraepithelial lesions or malignant cells were observed; ② Images of atypical squamous cells of unclear significance; ③ Images of atypical squamous cells that cannot rule out high-grade squamous intraepithelial lesions; ④ Images of cells with low-grade squamous intraepithelial lesions; ⑤ Image of cells with high degree of squamous intraepithelial lesions; ⑥ Image of squamous cell carcinoma; ⑦ Images of atypical adenocarcinoma with no clear diagnostic significance.

Images