A cervical cell detection method based on multi-scale spatial information

Abstract

The application discloses a cervical cell detection method based on multi-scale space information and relates to the technical field of medical images. The application combines the advantages of high speed and high precision of Sparse R-CNN, the extraction ability of a multi-scale space information extraction branch and a channel attention module for space features and channel features, and better detection performance is obtained, so that the problem of insufficient detection precision of existing cervical lesion cells is solved.

Description

Technical Field

[0001] This invention relates to the field of medical imaging technology, specifically to a method for detecting cervical cells based on multi-scale spatial information. Background Technology

[0002] The purpose of early cervical cancer screening is to detect precancerous lesions of the cervix, enabling timely treatment and reducing the risk of cervical cancer progressing to intermediate or advanced stages and mortality. Almost all cases of cervical cancer are caused by abnormal cell growth in the cervical region due to a virus called human papillomavirus (HPV). According to the 2018 global cancer statistics released by the World Health Organization (WHO), cervical cancer ranks 14th in incidence among all cancers and 4th in both incidence and mortality among women's cancers. Cervical cytology screening has always been an effective method for diagnosing and preventing cervical cancer. This method requires cytologists to prepare and stain exfoliated cervical lesion cells, and finally, based on the subtle differences in morphology and structure between cervical lesion cells and normal cells, observe the sample under a microscope to obtain a preliminary diagnostic result. However, manually interpreting thousands of cells in a slide of cytology images with hundreds of millions of pixels is usually time-consuming and error-prone.

[0003] With the development of image processing methods and computing power, computer-aided diagnostic methods based on deep learning have received widespread attention. Currently, deep learning-based object detection algorithms are being applied to detect abnormal cells in cervical lesion cytology images, and have achieved a series of remarkable results in this field. Addressing a series of problems inherent in manual interpretation, using deep learning to assist in the interpretation of cervical lesion cytology images has great potential. It can reduce the workload of physicians, minimize the influence of subjective emotions, improve work efficiency, and enhance the reliability of screening results. However, we still note the limitations of deep learning in cytology images.

[0004] In recent years, the aforementioned problems have seen some improvement due to the gradual maturation and widespread application of deep learning technology. In 2021, Liang et al. proposed an end-to-end object detection method. This strategy directly learns prototype representations of background and foreground categories, solving the problem of cervical lesion cell detection under limited data conditions. However, this method only utilizes local background and foreground information, ignoring useful spatial contextual information. Therefore, it is necessary to develop more advanced algorithms in the field of cervical classification to improve the efficiency of cervical lesion cell screening in practical applications.

[0005] To address this issue, we designed a cervical lesion cell image detection algorithm based on multi-scale spatial information, providing an alternative technical solution. Summary of the Invention

[0006] The purpose of this invention is to provide a cervical cell detection method based on multi-scale spatial information to solve the technical problems mentioned in the background art.

[0007] To achieve the above objectives, the present invention provides the following technical solution: a method for detecting cervical cells based on multi-scale spatial information, comprising at least the following steps:

[0008] S1: Obtain a dataset of cervical lesion cells, which includes a test set and a training set;

[0009] S2: Preprocess the cervical cell image dataset in the training dataset;

[0010] S3: Construct a multi-scale spatial information extraction branch module and a channel attention module, and based on the multi-scale spatial information extraction branch module and the channel attention module, construct an improved Sparse R-CNN network;

[0011] S4: Input the preprocessed training set into the improved Sparse R-CNN network and output the training results;

[0012] S5: Train the Sparse R-CNN detection network by inputting cervical lesion cell images from the training dataset into the Sparse R-CNN network and saving the trained model.

[0013] S6: Use the optimal improved Sparse R-CNN network model to detect cervical lesion cells and retain the detection result images.

[0014] Furthermore, S2 includes at least the following steps:

[0015] Cropping images to a uniform size: uniformly crop the dataset into images of the same size;

[0016] Data resampling balances the number of samples from different classes by replicating samples from a minority of classes, preventing the model from focusing excessively on the majority of classes.

[0017] The training set was preprocessed using Mosaic data augmentation and multi-scale training methods.

[0018] Furthermore, the construction of the multi-scale spatial information extraction branch module in S3 includes at least two methods;

[0019] The first method includes the following steps:

[0020] The input feature map (W×H×C) is passed through an adaptive pooling layer with an invariant ratio to dynamically generate feature maps of different sizes. These different sizes include at least (a1W×a1H×C), (a2W×a2H×C), and (a... n W×a n H×C);

[0021] Feature maps of different sizes are upsampled to obtain feature maps of size (W×H×NC), and then all feature maps are stitched together.

[0022] The stitched feature map is processed in two ways: one is to output the feature map (W×H×NC) directly without processing the feature map;

[0023] The second method includes the following steps:

[0024] The feature map is reduced in dimensionality using a 1×1 convolution to obtain a feature map (W×H×C). Then, a 3×3 convolution is used to extract more complex nonlinear features from the input features to obtain richer global spatial information, resulting in a new global feature (W×H×N). Finally, the output feature (W×H×NC) is obtained through a sigmoid activation function and a repeat layer.

[0025] The two processed features are connected by a dot product operation, and then fused after dimensionality reduction to obtain the final global spatial information feature map (W×H×C).

[0026] Furthermore, the construction of the channel attention module in S3 includes at least the following steps:

[0027] The input feature map (W×H×C) is compressed to 1×1 by global average pooling to obtain a channel feature (1×1×C). Then, the obtained channel feature (1×1×C) is further extracted by 1×1 convolution to obtain a new global spatial information feature (1×1×C).

[0028] The input feature map (W×H×C) is compressed to 1×1 by global max pooling to obtain a channel feature (1×1×C). Then, the obtained channel feature (1×1×C) is further extracted by 1×1 convolution to obtain a new global spatial information feature (1×1×C).

[0029] The two feature maps are added element by element to obtain a new feature map (1×1×C);

[0030] This feature map is passed through a sigmoid activation function to obtain the final channel attention weight map (C×1×1).

[0031] Furthermore, in S3, the improved Sparse R-CNN network is constructed by integrating the multi-scale spatial information extraction branch module and the channel attention construction module into the original Sparse R-CNN model, forming an improved Sparse R-CNN network model. The Sparse R-CNN network model consists of a feature pyramid network part based on the ResNet architecture as the backbone network, the multi-scale spatial information extraction branch module, and the channel attention module.

[0032] Furthermore, S4 includes at least the following steps:

[0033] The features of the highest layer of the feature pyramid network from bottom to top are used as output, and more global spatial information is extracted through the multi-scale spatial information extraction branch module.

[0034] The obtained global spatial information features and the original features of each layer of the top-down part of the feature pyramid network are fused to obtain new features.

[0035] The top-level features of the top-down part of the feature pyramid network are used as the output, and the channel attention weights are obtained through the channel attention module.

[0036] The features of each layer after fusion are combined with the obtained channel weights to output the final result.

[0037] Furthermore, S5 includes at least the following steps:

[0038] The training and validation sets are divided into multiple batches using a batch training method, with the training and validation batches set to 2.

[0039] One iteration consists of traversing all images in the training and test sets.

[0040] Sparse R-CNN was used as the base network for training, which includes a backbone network, a dynamic instance interaction head, and a classification and regression network.

[0041] Furthermore, training using Sparse R-CNN as the base network includes at least the following steps:

[0042] First, the backbone network is used to generate multi-scale feature maps from the input image, and a pyramid-level output feature map with P2 to P5 levels is constructed.

[0043] Next, the RoIAlign operation is performed to extract features for each bounding box, thus obtaining RoI features;

[0044] Then, through the dynamic instance interaction header, each RoI feature interacts with the corresponding output feature to filter out invalid information and output the final object features;

[0045] Finally, classification is performed using fully connected and softmax in the classification and regression network, and then higher-precision regression is obtained through bounding box regression.

[0046] Furthermore, S6 includes at least the following steps:

[0047] Input the test set to be classified, and use the trained improved Sparse R-CNN detection model to detect cervical cell images and obtain feature vectors;

[0048] The input classifier is used to determine the cervical type, and the probability of the cervical cell image in each class is obtained. The cervical category with the highest probability value is the corresponding cervical category.

[0049] Detect targets in an image, label and predict their categories, and save the detection results.

[0050] Compared with the prior art, the beneficial effects of the present invention are:

[0051] This invention provides a cervical cell detection method based on multi-scale spatial information. It combines the advantages of Sparse-RCNN's high speed and high accuracy, multi-scale spatial information extraction branches, and channel attention module's ability to extract spatial and channel features, thereby achieving better detection performance and solving the problem of insufficient accuracy in existing cervical lesion cell detection methods. Attached Figure Description

[0052] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0053] Figure 1 This is a flowchart of the present invention;

[0054] Figure 2 This is a flowchart of the cervical lesion cell image detection method of the present invention;

[0055] Figure 3 This is an example diagram of cervical lesion cells according to the present invention;

[0056] Figure 4 This is a structural diagram of the multi-scale spatial information extraction branch module of the present invention;

[0057] Figure 5 This is a structural diagram of the channel attention enhancement module of the present invention;

[0058] Figure 6 This is a model structure diagram of the improved Sparse R-CNN backbone network of the present invention;

[0059] Figure 7 The image shows the effect of using the improved Sparse R-CNN network in this invention. Detailed Implementation

[0060] The technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments.

[0061] Reference Figures 1-7 A cervical lesion cell image detection algorithm based on multi-scale spatial information, the steps of which are as follows:

[0062] S1: Obtain a dataset of cervical lesion cells, the dataset including a test set and a training set; in this application, the obtained test set and training set are for model construction and training, and after generating the optimal model, actual operation is performed, and a dataset of images of cervical lesion cells to be detected is also obtained.

[0063] S2: Preprocess the colposcopy image dataset in the training dataset, including: cropping to a uniform image size, resampling the data, randomly flipping horizontally, randomly flipping vertically, rotating at random angles, and dividing the dataset into training set, validation set, and test set.

[0064] To compensate for the impact of imbalanced datasets on model recognition performance and to avoid network overfitting, this invention performs augmentation processing on the smaller number of sample data before training. The augmentation methods used include: 1) cropping to uniform image size: uniformly cropping the dataset to 224×224 images; 2) data resampling: balancing the samples from different classes by copying samples from a minority of classes, preventing the model from over-focusing on the larger class; 3) random horizontal flipping: randomly flipping images horizontally with a probability of 0.4 during training; 4) random vertical flipping: randomly flipping images vertically with a probability of 0.4 during training; 5) preprocessing the training set using Mosaic data augmentation and multi-scale training methods.

[0065] S3: Construct a multi-scale spatial information extraction branch module and a channel attention enhancement module; and based on the multi-scale spatial information extraction branch module and the channel attention enhancement module, construct an improved Sparse R-CNN network;

[0066] In this application, the multi-scale spatial information extraction branch module is constructed by combining the idea of ​​global spatial context information, utilizing spatial information that the original model could not access, and enriching the original feature representation. The specific construction process is as follows:

[0067] The input feature map (W×H×C) is passed through an adaptive pooling layer with an invariant ratio to dynamically generate feature maps of different sizes, such as (a1W×a1H×C), (a2W×a2H×C), and (anW×anH×C).

[0068] Feature maps of different sizes are upsampled to obtain feature maps of size (W×H×NC), and then all feature maps are stitched together.

[0069] The concatenated feature maps are processed in two ways: one is to output the feature map directly without any processing (W×H×NC); the other is to reduce the dimensionality of the feature map through 1×1 convolution to obtain the feature map (W×H×C), and then extract more complex nonlinear features from the input features through 3×3 convolution to obtain richer global spatial information, resulting in new global features (W×H×N); finally, the output features (W×H×NC) are obtained through the sigmoid activation function and repeat layer.

[0070] The two processed features are connected by a dot product operation, and then fused after dimensionality reduction to obtain the final global spatial information feature map (W×H×C).

[0071] In Table 1 below, this application designed a set of comparative experiments to examine the performance of the multi-scale spatial information extraction branch module in feature extraction, specifically as follows:

[0072] Table 1 shows the results of feature extraction using the multi-scale spatial information extraction branch module.

[0073]

[0074] In the experimental results in Table 1, using the multi-scale spatial information extraction branch for feature extraction improved the performance by 1.3% on AP50 and 0.9% on AP75 compared to the original model.

[0075] Based on the above performance, the process of constructing the multi-scale spatial information extraction branch module can be described as follows:

[0076] First, combining the ideas of spatial context information, global features, and feature extraction branches, a multi-scale spatial information extraction branch is proposed. Then, a multi-scale spatial information extraction module is introduced to obtain global spatial information. Finally, the top-level feature map C5 of the bottom-up part of the FPN is used as input to extract global spatial information. The complete multi-scale spatial information extraction branch module is as follows: Figure 4As shown.

[0077] The multi-scale spatial information extraction branch module first takes the top-level feature map C5 of the FPN as input, and dynamically generates multiple feature maps of different sizes by passing the input feature map through an adaptive pooling layer with an invariant ratio. Then, the feature maps of different sizes are upsampled to the same size feature map, and all feature maps are concatenated.

[0078] The concatenated feature maps are processed in two ways: one is to output the feature map directly without any processing; the other is to reduce the dimensionality of the feature map through 1×1 convolutions, transforming the high-dimensional feature mapping into a low-dimensional one to reduce computation and the number of parameters. This helps improve network performance and reduce the risk of overfitting. Then, 3×3 convolutional layers are used to extract more complex non-linear features from the input features. Through convolution operations, the network can learn local features in the image, such as edges and textures, and generate richer feature representations, resulting in new global features. Normalization is then achieved through a sigmoid activation function and repeat layers, which improves data comparability and allows for better fusion of feature layers at different levels, ultimately yielding the output features.

[0079] After the multi-scale spatial information extraction branch module is constructed, a channel attention enhancement module is then built. This module is used to consider the impact on the object detection algorithm results, such as... Figure 5 As shown, specifically:

[0080] We employ global average pooling to compress the spatial dimension of each channel of the input feature map, i.e., averaging the feature maps of each channel to reduce their size to 1×1, thereby generating a vector representing the average response of each channel. Subsequently, a 1×1 convolutional layer is used to further extract and transform the obtained channel feature vectors. This reduces the number of parameters, enables cross-channel information integration and feature recalibration, and generates new channel feature vectors containing new global spatial information.

[0081] The input feature map is then subjected to global max pooling to capture global contextual information in another way. Similar to global average pooling, global max pooling also compresses the feature map of each channel to 1×1, but retains the maximum response value in each channel. The channel feature vectors generated in this step are then passed through a 1×1 convolutional layer for feature extraction and transformation to generate another set of new channel feature vectors containing different information.

[0082] The two feature maps are added element-wise to obtain a new feature map. The two new channel feature vectors obtained by global average pooling and global max pooling are added element-wise to fuse the information captured by the two global pooling strategies, generating a more comprehensive feature vector.

[0083] This feature map is passed through a sigmoid activation function to obtain the final channel attention weight map. Finally, another sigmoid activation function is passed through to generate a weight map with the same number of channels as the input feature map, where each element represents the importance weight of the corresponding channel.

[0084] In Table 2 below, this application designed a set of comparative experiments to examine the performance of the channel attention module in feature extraction, specifically as follows:

[0085] Table 1 shows the results of feature extraction using the channel attention module.

[0086]

[0087] In the experimental results in Table 2, using the multi-scale spatial information extraction branch for feature extraction improved the performance by 1.0% on AP50 and 1.9% on AP75 compared to the original model. In AP75, the effect of adding the channel attention module was better because the channel attention module can enable the model to focus on key features and obtain more refined results.

[0088] In Table 3, we designed a set of ablation experiments to analyze the impact of the channel attention module on the generated channel attention weights when inputting feature maps at different levels. These experiments were conducted using an improved Sparse R-CNN with added multi-scale spatial information extraction branches and channel attention modules. Specific locations are shown in Table 3. Figure 5 As shown in the figure. Experimental results show that the channel attention module achieves the best detection performance when the feature map of layer P2 is used as input, with an AP50 of 65.3% and an AP75 of 35.4%. When the channel attention module uses the feature map of layer P3 as input, the average accuracy is slightly lower than the best, with an AP50 of 64.9%, but it is 0.3% higher than that of layer P4. Therefore, the analysis concludes that the channel attention module achieves the best performance when the feature map of layer P2 is used as input.

[0089]

[0090]

[0091] S4: Input the preprocessed training set into the improved Sparse R-CNN network and output the training results;

[0092] S5: Train the Sparse R-CNN detection network; input the cervical lesion cell images from the training dataset into the Sparse R-CNN network for training, and save the trained model;

[0093] In this application, preprocessing employs methods such as Mosaic data augmentation and multi-scale training. Then, the images from the training set are input into the improved YOLOv7 network model. The structure of the improved YOLOv7 model is as follows: Figure 5 As shown, the model weights are initialized using pre-trained weights from Sparse R-CNN on ImageNet, the training results are viewed, and the model weights are saved.

[0094] In Table 4, we designed a set of comparative experiments to compare the effectiveness of the improved Sparse R-CNN compared to the baseline Sparse R-CNN. The experimental results show that the improved Sparse R-CNN achieves a 1.9% improvement on AP50 and a 1.2% improvement on AP75 compared to the baseline Sparse R-CNN.

[0095]

[0096] S6: The optimal improved Sparse R-CNN network model was used to detect cervical lesion cells, and the detection result images were preserved. The effect image generated by the detection method of this application is shown below. Figure 7 As shown.

[0097] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the invention can be implemented in other specific forms without departing from its spirit or essential characteristics. Therefore, the embodiments should be considered in all respects as exemplary and non-limiting, and the scope of the invention is defined by the appended claims rather than the foregoing description. Thus, all variations falling within the meaning and scope of equivalents of the claims are intended to be included within the present invention. No reference numerals in the claims should be construed as limiting the scope of the claims.

Claims

1. A method for cervical cell detection based on multi-scale spatial information, characterized in that: At least comprising the following steps: S1: obtaining a data set of cervical lesion cells, the data set comprising a test set and a training set; S2: preprocessing the cervical cell image data set in the training data set; S3: constructing a multi-scale spatial information extraction branch module and a channel attention module, and based on the multi-scale spatial information extraction branch module and the channel attention module, constructing an improved Sparse R-CNN network; The S3 constructing a multi-scale spatial information extraction branch module comprises: The input feature map (W×H×C) is passed through an adaptive pooling layer with an invariant ratio to dynamically generate feature maps of different sizes. These different sizes of feature maps include at least (a1W×a1H×C), (a2W×a2H×C), and (a... n W×a n H×C); The feature maps of different sizes are all up-sampled to obtain feature maps of size (WxHxNC), and then all the feature maps are spliced; The spliced feature maps are processed in two ways, one of which is to directly output the feature maps (WxHxNC) without processing the feature maps; The other processing is: The feature maps are reduced in dimension through 1x1 convolution to obtain feature maps (WxHxC), and then more complex nonlinear features are extracted from the input features through 3x3 convolution to obtain more rich global spatial information, and new global features (WxHxN) are obtained; then the output features (WxHxNC) obtained through the sigmoid activation function and the repeat layer are obtained; The two processed features are connected using point multiplication operation, and the final global spatial information feature map (WxHxC) is obtained through fusion after dimension reduction; S4: inputting the preprocessed training set into the improved Sparse R-CNN network to output the training result; S5: learning and training the Sparse R-CNN detection network, inputting the cervical lesion cell images in the training data set into the Sparse R-CNN network for training, and saving the trained model; S6: using the optimal improved Sparse R-CNN network model to detect cervical lesion cells, and retaining the detection result picture.

2. The method of claim 1, wherein the method comprises: The S2 at least comprises the following steps: Crop the uniform image size, and crop the data set to the same size image; Resample the data, duplicate the samples of the minority class to balance the samples of different classes, and prevent the model from paying too much attention to the class with more quantity; Using the method of Mosaic data enhancement and multi-scale training to preprocess the training set.

3. The method of claim 1, wherein the method comprises: The S3 constructing a channel attention module at least comprises the following steps: The input feature map (WxHxC) is compressed to 1x1 through global average pooling to obtain a channel feature (1x1xC), and then the obtained channel feature (1x1xC) is further extracted through 1x1 convolution to obtain new global spatial information feature (1x1xC); The input feature map (WxHxC) is compressed to 1x1 through global maximum pooling to obtain a channel feature (1x1xC), and then the obtained channel feature (1x1xC) is further extracted through 1x1 convolution to obtain new global spatial information feature (1x1xC); The two obtained feature maps are added element by element to obtain a new feature map (1x1xC); The feature map is passed through a sigmoid activation function to obtain the final channel attention weight map (C×1×1).

4. The method of claim 3, wherein the method comprises: The S3 includes a multi-scale spatial information extraction branch module and a channel attention module, and the improved SparseR-CNN network is formed by fusing the multi-scale spatial information extraction branch module and the channel attention module into the original SparseR-CNN model.

5. The method of claim 4, wherein the method further comprises: The S4 includes at least the following steps: The features of the highest layer of the feature pyramid network from bottom to top are taken as the output, and more global spatial information is extracted through the multi-scale spatial information extraction branch module; The obtained global spatial information features and the original features of each layer of the top-down part of the feature pyramid network are fused to obtain new features; The features of the topmost layer of the top-down part of the feature pyramid network are taken as the output, and channel attention weights are obtained through the channel attention module; The fused features of each layer are fused with the obtained channel weights, and the final result is output.

6. The method of claim 5, wherein the method further comprises: The S5 includes at least the following steps: The training set and the validation set are divided into multiple batches by batch training method, and the training and validation batch is set to 2; All pictures in the training and testing set are traversed once as an iteration; Sparse R-CNN is used as the basic network for training, and the basic network includes a backbone network, a dynamic instance interaction head, and a classification and regression network.

7. The method of claim 6, wherein the method further comprises: Sparse R-CNN is used as the basic network for training, which includes at least the following steps: First, the input image is output by the backbone network to generate a multi-scale feature map, and a pyramid level output feature map with P2 to P5 is constructed; Then, the RoI Align operation is performed to extract features for each box, and RoI features are obtained; Then, through the dynamic instance interaction head, each RoI feature interacts with the corresponding output feature to filter out invalid information and output the final object feature; Finally, the full connection and softmax in the classification and regression network are used for classification, and the bounding box regression is used to obtain higher precision regression.

8. The method of claim 7, wherein the method further comprises: The S6 includes at least the following steps: The test set to be classified is input, and the cervical cell image is detected by the trained improved Sparse R-CNN detection model to obtain a feature vector; The classifier is input to determine the cervical type, and the probability of the cervical cell image under each class is obtained, and the maximum probability value is the corresponding cervical category; The target in the image is detected, labeled and classified, and the detection result is saved.

Images