Cervical cell detection method based on multi-scale spatial information

Abstract

The invention discloses a cervical cell detection method based on multi-scale spatial information, and relates to the technical field of medical images. According to the method, a filtering type feature selection method and a package type feature selection method are effectively combined, association among all features is fully considered, and population initialization is carried out by adopting three filtering type feature selection methods including mutual information, variance threshold and chi-square verification; the problems of high calculation cost and overfitting of a traditional single package type feature selection method are solved; a multi-objective optimization algorithm-UNSGAIII package type method is adopted, and different from other optimization algorithms, the UNSGAIII method is an algorithm suitable for multi-objective tradeoff solving problems, so that the problems that multi-scheme solving can be provided for special users, and single filtering type feature selection prediction precision is low are solved; and the convergence speed and the global search capability are improved.

Description

Technical Field

[0001] The present invention relates to the field of medical image technology, and in particular to a cervical cell detection method based on multi-scale spatial information. Background Art

[0002] The purpose of early screening for cervical cancer is to detect precancerous lesions of cervical cancer, provide timely treatment, and reduce the risk and mortality of cervical cancer lesions in the middle and late stages. Almost all cases of cervical cancer are caused by abnormal growth of cells in the cervical area caused by a virus called human papillomavirus (HPV). According to the global cancer statistics released by the World Health Organization (WHO) in 2018, the incidence of cervical cancer ranks 14th among all cancers, and the incidence and mortality rates among female cancers are both fourth. Cervical lesion cytology screening has always been an effective method for diagnosing and preventing cervical cancer. This method requires cytologists to prepare and stain the exfoliated cervical lesion cells, and finally observe the samples under a microscope based on the subtle differences in morphology and structure between cervical lesion cells and normal cells to obtain preliminary diagnostic results. However, manual interpretation of thousands of cells in billion-pixel cytology image slides is usually time-consuming and error-prone.

[0003] With the development of image processing methods and computing power, computer-aided diagnosis methods based on deep learning have received widespread attention. At present, target detection algorithms based on deep learning are applied to detect abnormal cells in cervical lesion cytology images, and a series of remarkable results have been achieved in this field. In view of a series of problems existing in manual interpretation, the use of deep learning to assist in the interpretation of cervical lesion cytological medical images has great potential. It can reduce the workload of physicians, reduce the impact of subjective emotions, improve work efficiency, and improve the reliability of screening results. However, we still note the limitations of deep learning in cytological medical images.

[0004] In recent years, the above problems have been improved to a certain extent with the gradual maturity and widespread application of deep learning technology. In 2021, Liang et al. proposed an end-to-end target detection method. This strategy directly learns the prototype representation of background and foreground categories, solving the problem of cervical lesion cell detection under limited data. However, this method only utilizes the local background and foreground, while ignoring useful spatial context information. Therefore, it is necessary to develop more advanced algorithms in the field of cervical classification to improve the efficiency of cervical lesion cell screening in practical applications.

[0005] To this end, we designed a cervical lesion cell image detection algorithm based on multi-scale spatial information to provide another technical solution to the above technical problems. Summary of the invention

[0006] The purpose of the present invention is to provide a cervical cell detection method based on multi-scale spatial information to solve the technical problems raised in the background technology.

[0007] To achieve the above object, the present invention provides the following technical solution: a cervical cell detection method based on multi-scale spatial information, comprising at least the following steps:

[0008] S1: Acquire a data set of cervical lesion cells, wherein the data set includes a test set and a training set;

[0009] S2: preprocessing the cervical cell image dataset in the training dataset;

[0010] S3: Construct a multi-scale spatial information extraction branch module and a channel attention module, and construct an improved Sparse R-CNN network based on the multi-scale spatial information extraction branch module and the channel attention module;

[0011] S4: Input the preprocessed training set into the improved Sparse R-CNN network and output the training results;

[0012] S5: Perform learning and training on the Sparse R-CNN detection network, input the cervical lesion cell images in the training data set into the Sparse R-CNN network for training, and save the trained model;

[0013] S6: Use the optimal improved Sparse R-CNN network model to detect cervical lesion cells and retain the detection result images.

[0014] Furthermore, the S2 at least includes the following steps:

[0015] Crop the image to a uniform size and crop the dataset into images of the same size;

[0016] Resample the data to balance the samples of different categories by duplicating the samples of the minority category to prevent the model from focusing too much on the category with a large number of samples;

[0017] The training set is preprocessed using Mosaic data enhancement and multi-scale training methods.

[0018] Furthermore, constructing the multi-scale spatial information extraction branch module in S3 includes at least two methods;

[0019] The first method includes the following steps:

[0020] The input feature map (W×H×C) is passed through a ratio-invariant adaptive pooling layer to dynamically generate feature maps of different sizes, wherein the feature maps of different sizes at least include (a1 W×a 1 H×C), (a 2 W×a2H×C), (a n W×a n H × C);

[0021] Feature maps of different sizes are upsampled to obtain feature maps of (W×H×NC) size, and then all feature maps are concatenated.

[0022] The concatenated feature maps are processed in two ways. One is to directly output the feature map (W×H×NC) without processing the feature map.

[0023] The second method includes the following steps:

[0024] The feature map is reduced in dimension by 1×1 convolution to obtain a feature map (W×H×C), and then more complex nonlinear features are extracted from the input features by 3×3 convolution to obtain richer global spatial information and obtain new global features (W×H×N); the output features (W×H×NC) are obtained by sigmoid activation function and repeat layer;

[0025] The two processed features are connected using a dot product operation, and the final global spatial information feature map (W×H×C) is obtained by fusion after dimensionality reduction.

[0026] Furthermore, constructing the channel attention module in S3 at least includes the following steps:

[0027] The input feature map (W×H×C) is subjected to global average pooling, and the feature map of each channel is compressed to 1×1 to obtain a channel feature (1×1×C). Then, the obtained channel feature (1×1×C) is further extracted through 1×1 convolution to obtain a new global spatial information feature (1×1×C).

[0028] The input feature map (W×H×C) is subjected to global maximum pooling, and the feature map of each channel is compressed to 1×1 to obtain a channel feature (1×1×C). Then, the obtained channel feature (1×1×C) is further extracted through 1×1 convolution to obtain a new global spatial information feature (1×1×C).

[0029] Add the two feature maps element by element to obtain a new feature map (1×1×C);

[0030] This feature map is passed through a sigmoid activation function to obtain the final channel attention weight map (C×1×1).

[0031] Furthermore, in S3, based on the multi-scale spatial information extraction branch module and the channel attention module, an improved Sparse R-CNN network is constructed by integrating the multi-scale spatial information extraction branch module and the channel attention module into the original Sparse R-CNN model to form an improved Sparse R-CNN network model. The Sparse R-CNN network model includes a feature pyramid network part based on the ResNet architecture as a backbone network, a multi-scale spatial information extraction branch module and a channel attention module.

[0032] Furthermore, the S4 at least includes the following steps:

[0033] The top-level features of the feature pyramid network from bottom to top are taken as output, and more global spatial information is extracted through the multi-scale spatial information extraction branch module;

[0034] The obtained global spatial information features are fused with the original features of each layer of the top-down part of the feature pyramid network to obtain new features.

[0035] The top-level features of the top-down part of the feature pyramid network are used as output, and the channel attention weights are obtained through the channel attention module;

[0036] The fused features of each layer are fused with the obtained channel weights to output the final result.

[0037] Furthermore, the S5 at least includes the following steps:

[0038] The batch training method is used to divide the training set and the validation set into multiple batches, where the training validation batch is set to 2;

[0039] Traversing all the images in the training and test sets once is considered an iteration;

[0040] Sparse R-CNN is used as the basic network for training, and the basic network includes a backbone network, a dynamic instance interaction head and a classification regression network.

[0041] Furthermore, using Sparse R-CNN as the basic network for training includes at least the following steps:

[0042] Firstly, the backbone network is used to generate multi-scale feature maps for the input image output, and a pyramid-level output feature map with levels from P2 to P5 is constructed;

[0043] Then perform the RoIAlign operation to extract features for each box and obtain the RoI features;

[0044] Then, through the dynamic instance interaction head, each RoI feature interacts with the corresponding output feature to filter out invalid information and output the final object feature;

[0045] Finally, classification is performed through full connection and softmax in the classification regression network, and then bounding box regression is used to obtain higher precision regression.

[0046] Further, the S6 at least includes the following steps:

[0047] Input the test set to be classified, detect the cervical cell image through the trained improved Sparse R-CNN detection model, and obtain the feature vector;

[0048] The input classifier is used to determine the cervical type and the probability of the cervical cell image in each category is obtained. The one with the largest probability value is the corresponding cervical category.

[0049] Detect objects in images, mark and predict categories, and save detection results.

[0050] Compared with the prior art, the present invention has the following beneficial effects:

[0051] The present invention provides a cervical cell detection method based on multi-scale spatial information, which combines the advantages of Sparse-RCNN's high speed and high precision, the multi-scale spatial information extraction branch and the channel attention module's ability to extract spatial features and channel features, thereby achieving better detection performance and solving the problem of insufficient accuracy in existing cervical lesion cell detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for describing the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative work.

[0053] Figure 1 It is a flowchart of the present invention;

[0054] Figure 2 This is a flow chart of the cervical lesion cell image detection method of the present invention;

[0055] Figure 3 This is an example diagram of cervical lesion cells of the present invention;

[0056] Figure 4 This is a structural diagram of the multi-scale spatial information extraction branch module of the present invention;

[0057] Figure 5 This is a structural diagram of the channel attention enhancement module of the present invention;

[0058] Figure 6 The model structure diagram of the improved Sparse R-CNN backbone network of the present invention;

[0059] Figure 7 This is a diagram showing the effect of the improved Sparse R-CNN network used in the present invention. DETAILED DESCRIPTION

[0060] The technical solutions in the embodiments of the present invention will be described clearly and completely below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments.

[0061] Reference Figure 1-Figure 7 , a cervical lesion cell image detection algorithm based on multi-scale spatial information, the steps are as follows:

[0062] S1: Obtain a data set of cervical lesion cells, wherein the data set includes a test set and a training set; in the present application, the test set and training set are obtained for the purpose of model construction and training. After the optimal model is generated, actual operation is performed, and a data set of images of cervical lesion cells to be detected is also obtained.

[0063] S2: Preprocess the colposcopy image dataset in the training dataset, including cropping the image to a uniform size, resampling the data, randomly flipping horizontally, randomly flipping vertically, and randomly rotating the data, and dividing the dataset into a training set, a validation set, and a test set.

[0064] In order to compensate for the impact of unbalanced data set samples on model recognition performance and avoid network overfitting, the present invention enhances the sample data with a small number before training. The enhancement methods used are: 1) Crop the uniform image size: Crop the data set to a uniform image of 224×224 size; 2) Resample the data: Balance samples of different categories by copying samples of a few categories to prevent the model from paying too much attention to categories with a large number. 3) Random horizontal flip: In training mode, randomly flip the image horizontally with a probability of 0.4; 4) Random vertical flip: In training mode, randomly flip the image vertically with a probability of 0.4; 5) Preprocess the training set using Mosaic data enhancement and multi-scale training methods.

[0065] S3: Construct a multi-scale spatial information extraction branch module and a channel attention enhancement module; and based on the multi-scale spatial information extraction branch module and the channel attention enhancement module, construct an improved Sparse R-CNN network;

[0066] In this application, the construction of a multi-scale spatial information extraction branch module combines the idea of ​​global spatial context information, utilizes the spatial information that the original model cannot use and enriches the original feature representation. The specific construction process is:

[0067] The input feature map (W×H×C) is passed through a ratio-invariant adaptive pooling layer to dynamically generate feature maps of different sizes such as (a1W×a1H×C), (a2W×a2H×C), and (anW×anH×C);

[0068] Feature maps of different sizes are upsampled to obtain feature maps of (W×H×NC) size, and then all feature maps are concatenated.

[0069] The concatenated feature maps are processed in two ways. One is to directly output the feature map (W×H×NC) without processing the feature map; the other is to reduce the dimension of the feature map through 1×1 convolution to obtain the feature map (W×H×C), and then extract more complex nonlinear features from the input features through 3×3 convolution to obtain richer global spatial information and obtain new global features (W×H×N); and then obtain the output features (W×H×NC) through the sigmoid activation function and repeat layer;

[0070] The two processed features are connected using a dot product operation, and the final global spatial information feature map (W×H×C) is obtained by fusion after dimensionality reduction.

[0071] In the following Table 1, the present application designs a set of comparative experiments to test the performance of the branch module with multi-scale spatial information extraction in feature extraction, specifically:

[0072] Table 1 Results of feature extraction with multi-scale spatial information extraction branch module

[0073]

[0074] In the experimental results in Table 1, the use of the multi-scale spatial information extraction branch for feature extraction improves AP50 by 1.3% and AP75 by 0.9% compared to the original model.

[0075] Based on the above performance, the process of constructing a multi-scale spatial information extraction branch module can be described as follows:

[0076] Firstly, combining the ideas of spatial context information, global features and feature extraction branches, a multi-scale spatial information extraction branch is proposed, and then a multi-scale spatial information extraction module is introduced to obtain global spatial information; finally, the top-level feature map C5 from the bottom-up part of FPN is used as input to extract global spatial information. The complete multi-scale spatial information extraction branch module is shown in Figure 4shown.

[0077] The multi-scale spatial information extraction branch module first takes the top-level feature map C5 of FPN as input, passes the input feature map through a ratio-invariant adaptive pooling layer, and dynamically generates multiple feature maps of different sizes. Then, feature maps of different sizes are upsampled to feature maps of the same size, and all feature maps are concatenated.

[0078] The concatenated feature maps are processed in two ways. One is to directly output the feature maps without processing them. The other is to reduce the dimension of the feature maps through 1×1 convolution to obtain feature maps, and convert high-dimensional feature maps to low-dimensional feature maps to reduce the amount of calculation and the number of parameters. This helps to improve network performance and reduce the risk of overfitting; then the 3×3 convolution layer is used to extract more complex nonlinear features from the input features. Through the convolution operation, the network can learn local features in the image, such as edges, textures, etc., and generate richer feature representations to obtain new global features; then the sigmoid activation function and repeat layer are used to achieve normalization, which helps the comparability of the data, so that feature layers at different levels can be better integrated to finally obtain the output features.

[0079] After the multi-scale spatial information extraction branch module is constructed, the channel attention enhancement module is then constructed. The channel attention enhancement module is used to consider the impact on the results of the target detection algorithm, such as Figure 5 As shown, specifically:

[0080] We use global average pooling to compress the spatial dimension of each channel of the input feature map, that is, average the feature map of each channel to reduce its size to 1×1, and then generate a vector representing the average response of each channel. Subsequently, a 1×1 convolutional layer is used to further extract and transform the channel feature vector obtained above. The number of parameters is reduced, cross-channel information integration and feature recalibration are achieved, and a new channel feature vector containing new global spatial information is generated.

[0081] The input feature map is pooled with the input feature map through global maximum pooling to capture global context information in another way. Similar to global average pooling, global maximum pooling also compresses the feature map of each channel to 1×1, but retains the maximum response value in each channel. The channel feature vector generated by this step is then also extracted and transformed through a 1×1 convolution layer to generate another set of new channel feature vectors containing different information.

[0082] The two feature maps are added element by element to obtain a new feature map; we add the two sets of new channel feature vectors obtained by global average pooling and global maximum pooling respectively, fuse the information captured by the two global pooling strategies, and generate a more comprehensive feature vector.

[0083] This feature map is passed through a sigmoid activation function to obtain the final channel attention weight map. Finally, a sigmoid activation function is passed to generate a weight map with the same number of channels as the input feature map, where each element represents the importance weight of the corresponding channel.

[0084] In the following Table 2, this application designs a set of comparative experiments to test the performance of the channel attention module in feature extraction, specifically:

[0085] Table 1 Results of feature extraction with channel attention module

[0086]

[0087] In the experimental results in Table 2, the use of the multi-scale spatial information extraction branch for feature extraction improves AP50 by 1.0% and AP75 by 1.9% compared to the original model. In AP75, the effect of adding the channel attention module is better because the channel attention module enables the model to focus on key features and obtain more refined results.

[0088] In Table 3, we designed a set of ablation experiments to analyze the impact of the channel attention module on the generated channel attention weights when inputting feature maps at different levels. The experiments were conducted on the improved Sparse R-CNN with the addition of a multi-scale spatial information extraction branch and a channel attention module. Figure 5 As shown in the figure. The experimental results show that the channel attention module achieves the best detection effect when the feature map of the P2 layer is used as input, with an AP50 of 65.3% and an AP75 of 35.4%. When the channel attention module uses the feature map of the P3 layer as input, the average precision is slightly lower than the best, with an AP50 of 64.9%, and 0.3% higher than P4. Therefore, the analysis shows that the channel attention module achieves the best effect when the feature map of the P2 layer is used as input.

[0089]

[0090]

[0091] S4: Input the preprocessed training set into the improved Sparse R-CNN network and output the training results;

[0092] S5: Learning and training the Sparse R-CNN detection network; inputting the cervical lesion cell images in the training data set into the Sparse R-CNN network for training, and saving the trained model;

[0093] In this application, the preprocessing method uses Mosaic data enhancement and multi-scale training, and then the images in the training set are input into the improved yolov7 network model. The improved yolov7 model structure is as follows: Figure 5 As shown, use the pre-trained weights of Sparse R-CNN on ImageNet for initialization, view the training results, and save the model weights.

[0094] In Table 4, we designed a set of comparative experiments to compare the effectiveness of the improved Sparse R-CNN with the baseline Sparse R-CNN. The experimental results show that the improved Sparse R-CNN improves 1.9% on AP50 and 1.2% on AP75 over the baseline Sparse R-CNN.

[0095]

[0096] S6: Use the optimal improved Sparse R-CNN network model to detect cervical lesion cells, retain the detection result image, and the effect image generated by the detection method of this application is as follows: Figure 7 shown.

[0097] It will be apparent to those skilled in the art that the invention is not limited to the details of the exemplary embodiments described above and that the invention can be implemented in other specific forms without departing from the spirit or essential features of the invention. Therefore, the embodiments should be considered exemplary and non-limiting in all respects, and the scope of the invention is defined by the appended claims rather than the foregoing description, and it is intended that all variations falling within the meaning and scope of the equivalent elements of the claims be included in the invention. Any reference numeral in a claim should not be considered as limiting the claim to which it relates.

Claims

1. A cervical cell detection method based on multi-scale spatial information, characterized in that: At least the following steps are included: S1: Acquire a data set of cervical lesion cells, wherein the data set includes a test set and a training set; S2: preprocessing the cervical cell image dataset in the training dataset; S3: Construct a multi-scale spatial information extraction branch module and a channel attention module, and construct an improved Sparse R-CNN network based on the multi-scale spatial information extraction branch module and the channel attention module; S4: Input the preprocessed training set into the improved Sparse R-CNN network and output the training results; S5: Perform learning and training on the Sparse R-CNN detection network, input the cervical lesion cell images in the training data set into the Sparse R-CNN network for training, and save the trained model; S6: Use the optimal improved Sparse R-CNN network model to detect cervical lesion cells and retain the detection result images.

2. The cervical cell detection method based on multi-scale spatial information according to claim 1, characterized in that: The S2 at least comprises the following steps: Crop the image to a uniform size and crop the dataset into images of the same size; Resample the data to balance the samples of different categories by duplicating the samples of the minority category to prevent the model from focusing too much on the category with a large number of samples; The training set is preprocessed using Mosaic data enhancement and multi-scale training methods.

3. The cervical cell detection method based on multi-scale spatial information according to claim 2, characterized in that: The multi-scale spatial information extraction branch module in S3 includes at least two methods; The first method includes the following steps: The input feature map (W×H×C) is passed through a ratio-invariant adaptive pooling layer to dynamically generate feature maps of different sizes, wherein the feature maps of different sizes include at least (a1W×a1H×C), (a2W×a2H×C), (a n W×a n H × C); Feature maps of different sizes are upsampled to obtain feature maps of (W×H×NC) size, and then all feature maps are concatenated. The concatenated feature maps are processed in two ways. One is to directly output the feature map (W×H×NC) without processing the feature map. The second method includes the following steps: The feature map is reduced in dimension by 1×1 convolution to obtain a feature map (W×H×C), and then more complex nonlinear features are extracted from the input features by 3×3 convolution to obtain richer global spatial information and obtain new global features (W×H×N); the output features (W×H×NC) are obtained by sigmoid activation function and repeat layer; The two processed features are connected using a dot product operation, and the final global spatial information feature map (W×H×C) is obtained by fusion after dimensionality reduction.

4. The cervical cell detection method based on multi-scale spatial information according to claim 3, characterized in that: Constructing the channel attention module in S3 at least includes the following steps: The input feature map (W×H×C) is subjected to global average pooling, and the feature map of each channel is compressed to 1×1 to obtain a channel feature (1×1×C). Then, the obtained channel feature (1×1×C) is further extracted through 1×1 convolution to obtain a new global spatial information feature (1×1×C). The input feature map (W×H×C) is subjected to global maximum pooling, and the feature map of each channel is compressed to 1×1 to obtain a channel feature (1×1×C). Then, the obtained channel feature (1×1×C) is further extracted through 1×1 convolution to obtain a new global spatial information feature (1×1×C). Add the two feature maps element by element to obtain a new feature map (1×1×C); This feature map is passed through a sigmoid activation function to obtain the final channel attention weight map (C×1×1).

5. The cervical cell detection method based on multi-scale spatial information according to claim 4, characterized in that: In the S3, based on the multi-scale spatial information extraction branch module and the channel attention module, an improved SparseR-CNN network is constructed by integrating the multi-scale spatial information extraction branch module and the channel attention module into the original SparseR-CNN model to form an improved Sparse R-CNN network model. The Sparse R-CNN network model includes a feature pyramid network part based on the ResNet architecture as a backbone network, a multi-scale spatial information extraction branch module and a channel attention module.

6. The cervical cell detection method based on multi-scale spatial information according to claim 5, characterized in that: The S4 at least comprises the following steps: The top-level features of the feature pyramid network from bottom to top are taken as output, and more global spatial information is extracted through the multi-scale spatial information extraction branch module; The obtained global spatial information features are fused with the original features of each layer of the top-down part of the feature pyramid network to obtain new features. The top-level features of the top-down part of the feature pyramid network are used as output, and the channel attention weights are obtained through the channel attention module; The fused features of each layer are fused with the obtained channel weights to output the final result.

7. The method for detecting cervical cells based on multi-scale spatial information according to claim 6, characterized in that: The S5 at least comprises the following steps: The batch training method is used to divide the training set and the validation set into multiple batches, where the training validation batch is set to 2; Traversing all the images in the training and test sets once is considered an iteration; Sparse R-CNN is used as the basic network for training, and the basic network includes a backbone network, a dynamic instance interaction head and a classification regression network.

8. The cervical cell detection method based on multi-scale spatial information according to claim 7, characterized in that: Using Sparse R-CNN as the base network for training includes at least the following steps: Firstly, the backbone network is used to generate multi-scale feature maps for the input image output, and a pyramid-level output feature map with levels from P2 to P5 is constructed; Then perform the RoIAlign operation to extract features for each box and obtain the RoI features; Then, through the dynamic instance interaction head, each RoI feature interacts with the corresponding output feature to filter out invalid information and output the final object feature; Finally, classification is performed through full connection and softmax in the classification regression network, and then bounding box regression is used to obtain higher precision regression.

9. The cervical cell detection method based on multi-scale spatial information according to claim 7, characterized in that: The S6 at least comprises the following steps: Input the test set to be classified, detect the cervical cell image through the trained improved Sparse R-CNN detection model, and obtain the feature vector; The input classifier is used to determine the cervical type and the probability of the cervical cell image in each category is obtained. The one with the largest probability value is the corresponding cervical category. Detect objects in images, mark and predict categories, and save detection results.

Images