A Ki67 index calculation method based on deep learning
Hot spot areas were extracted through deep learning methods and cell detection using YOLOv5 model, which solved the time-consuming and subjective problems of Ki67 index calculation, improved detection accuracy and efficiency, especially achieved more accurate local calculations in pancreatic cancer neuroendocrine tumors .
Patent Information
- Application Number
- CN202210102370.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-01-27
- Publication Date
- 2024-12-06
- Estimated Expiration
- 2042-01-27
AI Technical Summary
In the prior art, manual counting is time-consuming and subjective when calculating the Ki67 index, especially in pancreatic cancer neuroendocrine tumors, and the cell data set is small in size and the cell overlap is severe, resulting in unpractical detection results.
Using a deep learning-based method, hot spot areas were extracted through density regression model, cell detection was performed using YOLOv5 model, and multi-scale feature fusion module and dual-task learning paradigm were combined to segment the slice images for cell detection, and the Ki67 index was calculated.
It improves the accuracy and efficiency of cell detection, solves the overlap problem caused by small cell data sets, and reduces the huge calculation amount of the entire slice calculation, achieving more accurate and efficient Ki67 index calculation.
Smart Images

Figure CN114494204B_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the field of computer vision, and specifically relates to a Ki67 index calculation method based on deep learning. Background Art
[0002] Histopathological slide images are considered the gold standard for evaluating a variety of tumors and are crucial for the differential diagnosis, treatment, and prognosis of complex diseases. The Ki67 index obtained based on histopathological slide images also has important guiding significance. The Ki67 index refers to the ratio of the number of cells with proliferative activity to the number of tumor cells. For example, the percentage of tumor-positive cells containing the Ki67 molecular biomarker on the IHC staining image indicates the degree of tumor deterioration. In order to obtain this index, pathologists need to manually count at least 500 to 1,000 cells of different categories in the hotspot area of the entire whole slide image (WSI). This process is always time-consuming and subjective. Therefore, in order to provide accurate and objective evaluation, higher requirements are placed on automatic cell detection.
[0003] The existing automatic Ki67 counting method is based on the whole slice image, and the counting is performed on the whole slice image. However, in pancreatic neuroendocrine tumors, doctors need to calculate the hotspot area more, and use the Ki67 index of the representative hotspot area to replace the index calculated on the whole slice. And the calculation is performed on the whole slice. A slice has billions of pixels, which requires a lot of calculation, so local calculation is very necessary.
[0004] In recent years, with the continuous development of deep learning, especially the excellent performance of convolutional neural networks in image classification, more and more image classification tasks can be efficiently automated. However, cell datasets are often small in size, and the scale of cell data is relatively consistent. There is also a serious overlap between cells. Therefore, it is very difficult to obtain a practical detection result on cells. Summary of the invention
[0005] In order to solve the above problems, a method for automatically extracting hotspot areas for densely populated cells and relatively small data sets is provided. The present invention adopts the following technical solutions:
[0006] The present invention provides a Ki67 index calculation method based on deep learning, which is used for analyzing and calculating neuroendocrine tumor pathological section images to obtain Ki67 index, and is characterized in that it comprises the following steps: step S1, obtaining a pathological image with a resolution of 1.5x based on the neuroendocrine tumor pathological section image, and preprocessing the pathological image to generate corresponding point labels; step S2, generating a Gaussian density map corresponding to the pathological image with a resolution of 1.5x based on a density regression model, and extracting three heat areas based on the Gaussian density map; step S3, extracting corresponding 40x resolution hot spot images for the three heat areas, and cutting the 40x resolution hot spot images into a plurality of 512-pixel hot spot pictures; step S4, inputting the preprocessed plurality of hot spot pictures of the three heat areas into a cell detection model to obtain a plurality of corresponding cell detection results of different heat areas; step S5, statistically accumulating a plurality of cell detection results of each heat area to obtain the corresponding Ki67 index of the heat area, and taking the highest Ki67 value as the final Ki67 index result.
[0007] The deep learning-based Ki67 index calculation method provided by the present invention may also have such a technical feature, wherein the cell detection model is obtained by training through the following steps: step T1, frame annotation is performed based on the pancreatic neuroendocrine tumor pathological section image to obtain a data set, and the data set is divided into a training set and a verification set; step T2, a cell frame detection model is constructed; step T3, the cell frame detection model is iteratively trained using the training set; step T4, the cell frame monitoring model in the training process is verified using the verification set, and the model with the highest index is selected as the trained model, i.e., the cell detection model.
[0008] The deep learning-based Ki67 index calculation method provided by the present invention may also have such a technical feature, wherein, in step T3, the loss error is calculated based on the model parameters of the last layer of the cell frame detection model, the loss error is back-propagated and the stochastic gradient descent algorithm is used to optimize the model parameters.
[0009] The deep learning-based Ki67 index calculation method provided by the present invention may also have such technical features, wherein the cell detection model uses YOLOv5 as the backbone network, has a multi-scale feature fusion module and a dual-task learning paradigm.
[0010] The present invention also provides a computer-readable storage medium for storing a computer program, characterized in that the computer program is configured to implement the steps of a Ki67 index calculation method based on deep learning when executed.
[0011] Function and Effect of the Invention
[0012] According to the method for calculating the Ki67 index based on deep learning of the present invention, it is carried out based on two stages. First, the pathological slice image data is preprocessed to obtain a low-resolution image, the hot spot area is extracted on the 1.5x low-resolution image, and mapped to the 40x high-resolution image, the hot spot area on the high-resolution image is divided into several small pictures of 512 pixels, and the small pictures are input into the cell detection model to obtain the results of cell detection, and finally the results of all small pictures are accumulated and counted to obtain the Ki67 index of the hot spot area, and the highest Ki67 value in all hot spot areas is taken as the final result. Since the constructed cell detection model is constructed based on the characteristics of small pathological cell detection data set and cell scale consistency, it has a multi-scale feature fusion module and a dual-task learning paradigm, thus solving the overlapping defects caused by the small cell data set in the existing detection technology, and also improving the accuracy of cell detection. In addition, since the whole slice is segmented and then detected based on the trained cell detection model, the detection of the whole large picture is avoided, so the problem of large calculation amount of the existing whole slice calculation method is also solved, and the efficiency of detection calculation is improved. BRIEF DESCRIPTION OF THE DRAWINGS
[0013] Figure 1 is a flow chart of a method for calculating the Ki67 index based on deep learning in an embodiment of the present invention;
[0014] Figure 2 is a schematic diagram of a box standard data set of cell categories in an embodiment of the present invention;
[0015] Figure 3 It is a structural schematic diagram of a cell detection model based on deep learning in an embodiment of the present invention. DETAILED DESCRIPTION
[0016] In order to make the technical means, creative features, objectives and effects achieved by the present invention easy to understand, the Ki67 index calculation method based on deep learning of the present invention is specifically described below in combination with embodiments and drawings.
[0017] <Example>
[0018] In this embodiment, pancreatic neuroendocrine tumor pathological section image data is used to run a deep learning-based Ki67 index calculation method through a computer. The computer requires a graphics card for GPU acceleration to complete the model training process. The trained hot spot area extraction, density regression model, cell detection model and image detection process are stored in the computer in the form of executable code.
[0019] Figure 1 It is a flowchart of a method for calculating the Ki67 index based on deep learning in an embodiment of the present invention.
[0020] like Figure 1 As shown, the Ki67 index calculation method based on deep learning includes the following steps:
[0021] Step S1, acquiring a pathological image with a 1.5x resolution based on a pathological slice image of a pancreatic neuroendocrine tumor, and preprocessing the pathological image with a 1.5x resolution to generate corresponding point labels.
[0022] Step S2, inputting the pathological image with a resolution of 1.5x into a density regression model to generate a Gaussian density map corresponding to the pathological image with a resolution of 1.5x, and extracting three heat areas based on the Gaussian density map.
[0023] Step S3: extract corresponding 40x resolution hotspot images for the three heat areas, and cut the 40x resolution hotspot images into several 512-pixel hotspot images.
[0024] In this embodiment, the 1.5x point labels of the image dataset are calculated from the box labels detected on the 40x slice images, the heat map is regressed at low resolution (1.5x resolution), and the detection is performed at high resolution (40x resolution).
[0025] Step S4, inputting a plurality of preprocessed hot spot images of the three heat areas into the cell detection model to obtain a plurality of corresponding cell detection results of different heat areas.
[0026] In this step S4, the training process of the cell detection model is as follows:
[0027] Step T1, obtaining a data set by labeling the pathological slice images of pancreatic neuroendocrine tumors.
[0028] The data set in step T1 above comes from the clinical diagnosis data of Changhai Hospital. Through the annotation of professional clinicians, the frame labels of cells (such as Figure 2 As shown). The dataset has a total of 1256 images, 1046 of which are used as training sets, and the remaining 210 images are used as validation sets.
[0029] Step T2, constructing a cell frame detection model.
[0030] In this embodiment, according to the characteristics of small pathological cell detection data set, cell scale consistency, and easy overfitting in training, a neural network model with YOLOv5 as the backbone network, multi-scale feature fusion module and dual-task learning paradigm is designed. The cell detection model improves the accuracy of cell detection based on the cell scale consistency and the small size of the pathological cell data set.
[0031] Step T3, iteratively train the cell box detection model using the training set.
[0032] In the training process of the cell frame detection model mentioned above, after each iteration (i.e., the training image passes through the model), the model parameters of the last layer of the model will calculate the loss error (i.e., GIoU Loss) respectively, and then the calculated loss error will be back-propagated, and the stochastic gradient descent algorithm will be used for parameter optimization. In addition, the training completion conditions of the model training are the same as those of the conventional convolutional neural network model, that is, the training is completed after the model parameters of each layer converge.
[0033] Step T4, using the validation set to validate the cell frame monitoring model in the training process, and selecting the model with the highest index as the trained model, i.e., the cell detection model (e.g. Figure 3 shown).
[0034] The experimental results in this embodiment show that the average accuracy of the final cell detection model reaches 48.9%.
[0035] Step S5, statistically accumulating several cell detection results in each heat area to obtain the corresponding heat area Ki67 index, and taking the highest Ki67 value as the final Ki67 index result.
[0036] Example Function and Effect
[0037] According to the deep learning-based Ki67 index calculation method provided in this embodiment, the pathological slice image data is first preprocessed to obtain a low-resolution image, the hotspot area is extracted from the 1.5x low-resolution image, and mapped to a 40x high-resolution image, the hotspot area on the high-resolution image is divided into several small pictures of 512 pixels, and these small pictures are input into the cell detection model to obtain the results of cell detection. Finally, the results of all small pictures are accumulated and counted to obtain the Ki67 index of the hotspot area, and the one with the highest Ki67 value among all hotspot areas is taken as the final result.
[0038] In the embodiment, since the constructed cell detection model is constructed based on the characteristics of small pathological cell detection data set and consistent cell scale, it has a multi-scale feature fusion module and a dual-task learning paradigm, thus solving the overlapping defects caused by the small cell data set in the existing detection technology and improving the accuracy of cell detection.
[0039] In the embodiment, since a calculation method is adopted in which the whole slice is segmented and then the detection is performed based on the trained cell detection model to accumulate the results, the problem of large amount of calculation in the existing whole slice calculation method is solved, and the detection calculation efficiency is improved.
[0040] The above embodiments are only used to illustrate specific implementation modes of the present invention, and the present invention is not limited to the description scope of the above embodiments.
Claims
1. A Ki67 index calculation method based on deep learning, used to analyze and calculate the pathological section images of neuroendocrine tumors to obtain the Ki67 index, characterized in that: The following steps are involved: Step S1, acquiring a pathological image with a resolution of 1.5x based on the neuroendocrine tumor pathological slice image, and preprocessing the pathological image to generate corresponding point labels; Step S2, generating a Gaussian density map corresponding to the pathological image with a resolution of 1.5x based on a density regression model, and extracting three heat areas based on the Gaussian density map; Step S3, extracting corresponding 40x resolution hotspot images for the three heat areas, and cutting the 40x resolution hotspot images into a plurality of 512-pixel hotspot images; Step S4, inputting the preprocessed hot spot images of the three heat areas into the cell detection model to obtain the corresponding cell detection results of the different heat areas; Step S5, statistically accumulating the cell detection results of each heat area to obtain the corresponding heat area Ki67 index, and taking the highest Ki67 value as the final Ki67 index result. Wherein, the cell detection model is trained by the following steps: Step T1, performing frame annotation based on the neuroendocrine tumor pathological section image to obtain a data set, and dividing the data set into a training set and a validation set; Step T2, constructing a cell frame detection model; Step T3, iteratively training the cell frame detection model using the training set; Step T4, using the verification set to verify the cell frame monitoring model in the training process, and selecting the model with the highest index as the trained model, i.e., the cell detection model, The cell detection model uses YOLOv5 as the backbone network, with a multi-scale feature fusion module and a dual-task learning paradigm.
2. The method for calculating the Ki67 index based on deep learning according to claim 1, characterized in that: in, In step T3, the loss error is calculated based on the model parameters of the last layer of the cell frame detection model, the loss error is back-propagated and a stochastic gradient descent algorithm is used to optimize the model parameters.
3. A computer-readable storage medium for storing a computer program, characterized in that: The computer program is configured to implement the steps of a Ki67 index calculation method based on deep learning described in any one of claims 1 to 2 when executed.
