Pathological type identification method and device based on ROSE cell morphology

Abstract

The invention provides a pathology type identification method and device based on ROSE cell morphology, and the method is characterized in that the method comprises the following steps: S1, constructing a training data set according to a plurality of existing pathology diagrams; s2, constructing a classification model, and training the classification model through the training data set to obtain a trained classification model; s3, sampling the pathological image to be identified to generate a plurality of sampling images; and S4, inputting all the sampling images into the trained classification model in sequence to obtain a pathological category identification result corresponding to each sampling image. In a word, the ROSE result can be accurately interpreted by the method.

Description

Technical Field

[0001] The present invention relates to a method and device for identifying pathological types, and in particular to a method and device for identifying pathological types based on ROSE cell morphology. Background Art

[0002] ROSE is a rapid cytological interpretation technology that accompanies the sampling process in real time. When sampling from the target site, without losing the tissue specimen, part of the sample is printed on a glass slide, quickly stained and observed under a microscope, and then the clinical information is combined to immediately interpret and diagnose. The interpretation includes: 1. Cell morphology and classification count; 2. Cell composition and background composition; 3. Identification and diagnosis of identifiable pathogens.

[0003] However, due to the lack of cytopathologists, the interpretation of ROSE results is limited by the individual level of physicians, and most medical institutions are unable to fully utilize the value and advantages of ROSE.

[0004] Therefore, there is an urgent need for a method and device that can accurately interpret ROSE results. Summary of the invention

[0005] The present invention is made to solve the above-mentioned problems, and aims to provide a method and device for pathological type identification based on ROSE cell morphology.

[0006] The present invention provides a pathological type recognition method based on ROSE cell morphology, which is used for pathological category recognition of pathological images collected by a rapid on-site evaluation ROSE method, and has the following characteristics, including the following steps: step S1, constructing a training data set according to multiple existing pathological images; step S2, constructing a classification model, and training the classification model through the training data set to obtain a trained classification model; step S3, sampling the pathological image to be identified to generate multiple sampling images; step S4, inputting all the sampling images into the trained classification model in sequence to obtain the pathological category recognition results corresponding to each sampling image, wherein step S1 includes the following sub-steps: step S1-1, marking a selection box and a corresponding pathological category for abnormal cells in the pathological image; step S1-2, intercepting multiple sub-images from the pathological image according to the selection box, and combining the corresponding pathological category to obtain multiple positive samples; step S1-3, intercepting multiple sub-images from the non-selection box area of ​​the pathological image, and setting the corresponding pathological category to normal cells to obtain multiple negative samples; step S1-4, constructing a training data set according to all positive samples and negative samples.

[0007] The pathological type identification method based on ROSE cell morphology provided by the present invention may also have the following features: in step S1-2, when the selection box is smaller than the sub-image, the sub-image contains all images within the selection box; when the selection box is larger than the sub-image, a point on the edge of the selection box is randomly selected, and another point is set according to the shape and size of the sub-image, and the rectangular area formed by these two points as the diagonal is used as the sub-image.

[0008] In the pathological type identification method based on ROSE cell morphology provided by the present invention, it can also have the following characteristics: wherein, in step S2, the training data set is divided into a training subset and a test subset, the classification model is trained by the training subset, and the classification model is judged whether the training is completed by the test subset, if so, the trained classification model is obtained, if not, the classification model is retrained using the training data set. In the test phase using the test subset, the test image is extracted by a sliding window of a fixed size, and the moving distance of the sliding window is set to be less than the side length of the sliding window, so that there is an overlapping area between the test images.

[0009] The pathology type identification method based on ROSE cell morphology provided by the present invention may also have the following feature: for the overlapping area, the classification model uses the mean of the predicted values ​​corresponding to all test images containing the overlapping area as the predicted value of the overlapping area.

[0010] The pathological type identification method based on ROSE cell morphology provided by the present invention may also have the following features: wherein, in step S1-4, data enhancement is performed on sub-images in positive samples and negative samples to obtain data enhanced samples, the training data set includes data enhanced samples, positive samples and negative samples, data enhancement includes flipping, rotation, scaling, cropping, shifting and Gaussian noise, and data enhancement is achieved through a deep learning library.

[0011] The pathological type identification method based on ROSE cell morphology provided by the present invention may also have the following feature: wherein the backbone network of the classification model is a residual network.

[0012] The pathological type identification method based on ROSE cell morphology provided by the present invention may also have the following characteristics: wherein, in step S2, the classification model is trained using back propagation by calculating the loss function according to the calculation result, and the expression of the loss function is: Where p i is the pathological category corresponding to sample i, is the predicted category probability of the classification model for sample i in the training data set, L i is the calculation result of sample i calculated by the loss function.

[0013] The present invention also provides a pathological type identification device based on ROSE cell morphology, which is used to identify the pathological category of pathological images collected by the rapid on-site evaluation ROSE method, and has the following characteristics, including: a training data generation module, which is used to construct a training data set based on multiple existing pathological images; a model training module, which is used to construct a classification model and train the classification model through the training data set to obtain a trained classification model; a sampling module, which is used to sample the pathological image to be identified and generate multiple sampling images; a recognition module, which is used to input all the sampling images into the trained classification model in sequence to obtain the pathological category identification results corresponding to each sampling image, wherein the training data generation module includes: a marking unit, which is used to mark the abnormal cells in the pathological image with a selection box and the corresponding pathological category; a positive sample generation unit, which is used to intercept multiple sub-images from the pathological image according to the selection box, and combine the corresponding pathological category to obtain multiple positive samples; a negative sample generation unit, which is used to intercept multiple sub-images from the non-selection box area in the pathological image, and set the corresponding pathological category to normal cells to obtain multiple negative samples; a training set construction unit, which is used to construct a training data set based on all positive samples and negative samples.

[0014] The pathological type identification device based on ROSE cell morphology provided by the present invention may also have the following features, further comprising: a display module for displaying the pathological image and the pathological category identification result.

[0015] Functions and Effects of the Invention

[0016] According to the pathological type identification method and device based on ROSE cell morphology involved in the present invention, on the one hand, according to the size relationship between the selection box and the sub-image, a corresponding strategy is adopted to collect abnormal cells; on the other hand, when testing the trained classification model, the adjacent test images have overlapping areas, so that the cell category of the overlapping area is predicted by using the mean of the predicted value, and then a classification model with good prediction performance is trained. Therefore, the pathological type identification method and device based on ROSE cell morphology of the present invention can more accurately interpret the ROSE results. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 is a block diagram of a pathology type identification device based on ROSE cell morphology in an embodiment of the present invention;

[0018] Figure 2 is a schematic diagram of collecting sub-images when a selection box is smaller than a sub-image in an embodiment of the present invention;

[0019] Figure 3 is a schematic diagram of collecting sub-images when a selection box is larger than a sub-image in an embodiment of the present invention;

[0020] Figure 4 4 is a flow chart of a method for identifying pathological types based on ROSE cell morphology in an embodiment of the present invention. DETAILED DESCRIPTION

[0021] In order to make the technical means, creative features, objectives and effects achieved by the present invention easy to understand, the following embodiments and the accompanying drawings specifically illustrate the pathological type identification method and device based on ROSE cell morphology of the present invention.

[0022] In this embodiment, a pathological type identification device based on ROSE cell morphology is provided, which is used to identify the pathological category of the pathological image collected by the rapid on-site evaluation ROSE method. In this embodiment, the pathological categories are divided into two categories: normal cells and abnormal cells. Abnormal cells can be further classified according to specific cell types, such as small cell carcinoma, adenocarcinoma, squamous cell carcinoma, etc.

[0023] Figure 1 4 is a block diagram of a pathology type identification device based on ROSE cell morphology in an embodiment of the present invention.

[0024] like Figure 1 As shown, the pathological type identification device 100 based on ROSE cell morphology includes a training data generation module 10, a model training module 20, a sampling module 30, a recognition module 40, a display module 50 and a control module 60 for controlling the operation of the above modules.

[0025] The training data generation module 10 is used to construct a training data set according to multiple existing pathological images.

[0026] The training data generating module 10 comprises a labeling unit 101 , a positive sample generating unit 102 , a negative sample generating unit 103 and a training set constructing unit 104 .

[0027] The marking unit 101 is used to mark the selection box and the corresponding pathological category for the abnormal cells in the pathological image. In this embodiment, the selection box is an irregular closed circle formed around the edge of the abnormal cell.

[0028] The positive sample generating unit 102 is used to capture multiple sub-images from the pathology image according to the selection box, and obtain multiple positive samples in combination with corresponding pathology categories.

[0029] In this embodiment, the pixel area of ​​the abnormal cells in the selection box is compared with the pixel area of ​​the sub-image, so as to determine the relative size of the selection box and the sub-image, and different positive sample generation methods are adopted according to the relative size result.

[0030] Figure 2 It is a schematic diagram of collecting sub-images when the selection box is smaller than the sub-image in an embodiment of the present invention.

[0031] like Figure 2 As shown in the figure, the black irregular closed circle in the pathology image is a selection box, and the line segment with an arrow is used to indicate the corresponding area of ​​the intercepted sub-image in the pathology image. When the selection box is smaller than the sub-image, the sub-image contains all the images in the selection box, that is, the abnormal cells in the selection box are completely set in the sub-image.

[0032] Figure 3 It is a schematic diagram of collecting sub-images when the selection box is larger than the sub-image in an embodiment of the present invention.

[0033] like Figure 3 As shown in the figure, the black irregular closed circle in the pathology image is the selection box, and the line segment with an arrow is used to indicate the area corresponding to the intercepted sub-image in the pathology image. When the selection box is larger than the sub-image, a point on the edge of the selection box, i.e., the irregular closed circle, is randomly selected, and another point is set according to the shape and size of the sub-image, and the rectangular area formed by the two points as the diagonal is used as the sub-image.

[0034] The negative sample generating unit 103 is used to intercept a plurality of sub-images from the non-selected box area of ​​the pathology image, and set the corresponding pathology category to normal cells, so as to obtain a plurality of negative samples.

[0035] The training set construction unit 104 is used to construct a training data set according to all positive samples and negative samples.

[0036] The training set construction unit 104 performs data enhancement on the subgraphs in the positive sample and the negative sample to obtain data enhanced samples. The training data set includes data enhanced samples, positive samples and negative samples. Data enhancement includes flipping, rotation, scaling, cropping, shifting and Gaussian noise. Data enhancement is implemented by a deep learning library, and the deep learning library in this embodiment is keras.

[0037] The model training module 20 is used to construct a classification model and train the classification model through a training data set to obtain a trained classification model. Among them, the backbone network of the classification model is a residual network, and the initial classification model in this embodiment is a ResNet-50 model.

[0038]

[0039]

[0040] In the above table, the first column is the layer name, the second column is the specific layer corresponding to the layer name, that is, the type, and the third column is the output dimension corresponding to the layer. In this embodiment, after the image is input into the ResNet-50 model, the output of the model, that is, the output of the fully connected layer, is processed by the softmax activation function layer to obtain the corresponding predicted category probability. In this embodiment, the kernel_size of Conv1 is set to 7, the stride is set to 2, and the padding is set to 3, the kernel_size of MaxPool is set to 3 and the stride is set to 2, the kernel_size of Conv2_1, Conv2_3, Conv3_3, and Conv4_3 are all set to 1 and the stride is set to 1, the kernel_size of Conv2_2, Conv3_2, Conv4_2, and Conv5_2 are all set to 3, the stride is set to 1, and the padding is set to 1, the kernel_size of Conv3_1, Conv4_1, and Conv5_1 are all set to 1 and the stride is set to 2, the kernel_size of Average Pool is set to 7, N is the number of categories, and each residual block consists of 3 convolutional layers.

[0041] The training data set is divided into a training subset and a test subset. The classification model is trained with the training subset, and the test subset is used to determine whether the classification model is trained. If so, a trained classification model is obtained. If not, the classification model is retrained using the training data set.

[0042] The model training module 20 uses the training subset to calculate the loss function and trains the classification model using back propagation according to the calculation result. The expression of the loss function is:

[0043]

[0044] Where p i is the pathological category corresponding to sample i, is the predicted category probability of the classification model for sample i in the training subset, L i is the calculation result of sample i calculated by the loss function.

[0045] In the test phase using the test subset, the test images are extracted through a sliding window of a fixed size, and the moving distance of the sliding window is set to be smaller than the side length of the sliding window, so that there is an overlapping area between the test images. In this embodiment, the size of the sliding window is 320x320 pixels, the moving distance is 160 pixels, and the test images are extracted from left to right and from top to bottom.

[0046] For overlapping areas, the classification model uses the mean of the prediction values ​​corresponding to all test images containing the overlapping area as the prediction value of the overlapping area. For example, if the local images of an abnormal cell exist in four test images, the prediction values ​​of the corresponding local images in the four test images are averaged and the average value is used as the prediction value of the abnormal cell, thereby obtaining the pathological category recognition result of the abnormal cell.

[0047] In this embodiment, based on the existing pathological images, the trained classification model is subjected to an image-level performance test, that is, the consistency between the prediction result generated by the classification model for the entire image and the actual result is tested. The specific results are shown in the following table:

[0048] category AUC Sensitivity Specificity Small cell carcinoma 0.9307 0.8043 0.9065 Adenocarcinoma 0.9244 0.7625 0.9030 Squamous cell carcinoma 0.9827 0.8500 0.9822 Papillary thyroid carcinoma metastasis 0.9490 0.6667 0.9463

[0049] In the above table, the first column of the actual results contains abnormal cell categories, and the second to fourth columns are the values ​​of the prediction performance indicators for the abnormal cell categories, which are AUC, Sensitivity, and Specificity, respectively.

[0050] In this embodiment, based on the existing pathological images, the trained classification model is further tested for sampling performance, that is, the consistency between the prediction results generated by the classification model for each sampled image collected in the entire image and the actual results is tested. The specific results are shown in the following table:

[0051] category AUC Sensitivity Specificity Small cell carcinoma 0.8262 0.8553 0.6529 Adenocarcinoma 0.9361 0.6905 0.7903 Squamous cell carcinoma 0.9619 0.9200 0.9233 Papillary thyroid carcinoma metastasis 0.9619 0.7500 0.9734

[0052] In the above table, the first column of the actual results contains abnormal cell categories, and the second to fourth columns are the values ​​of the prediction performance indicators for the abnormal cell categories, which are AUC, Sensitivity, and Specificity, respectively.

[0053] This shows that the classification model can accurately predict and classify cells at both the image level and the sampling level.

[0054] The sampling module 30 is used to sample the pathological image to be identified and generate multiple sampled images. In this embodiment, the pathological image is evenly divided into multiple sampled images of fixed size according to the size of the pathological image, that is, the pathological image can be reconstructed through all the sampled images.

[0055] The recognition module 40 is used to input all sample images into the trained classification model in sequence to obtain the pathology category recognition result corresponding to each sample image.

[0056] The display module 50 is used to display the pathology image and the pathology category recognition result. In this embodiment, the display module 50 marks the range of the sampled image with abnormal cell pathology category recognition result on the pathology image in the form of a red frame, so that the user can clearly and explicitly focus on the abnormal part.

[0057] The control module 60 stores a control program for controlling the operation of each module.

[0058] The following describes the process of using the ROSE cell morphology-based pathology type identification device 100 to perform the ROSE cell morphology-based pathology type identification method in conjunction with the accompanying drawings.

[0059] Figure 4 4 is a flow chart of a method for identifying pathological types based on ROSE cell morphology in an embodiment of the present invention.

[0060] like Figure 4 As shown, the pathological type identification method based on ROSE cell morphology includes the following steps:

[0061] Step S1-1, using the marking unit 101 to mark the selection box and the corresponding pathological category of the abnormal cells in the pathological image.

[0062] In step S1-2, the positive sample generating unit 102 is used to capture multiple sub-images from the pathology image according to the selection box, and combined with the corresponding pathology categories to obtain multiple positive samples.

[0063] Step S1-3, using the negative sample generating unit 103 to intercept multiple sub-images from the non-selected box area of ​​the pathology image, and setting the corresponding pathology category to normal cells, to obtain multiple negative samples.

[0064] Step S1-4, using the training set construction unit 104 to construct a training data set based on all positive samples and negative samples.

[0065] Step S2, using the model training module 20 to build a classification model, and training the classification model through a training data set to obtain a trained classification model.

[0066] Step S3: using the sampling module 30 to sample the pathological image to be identified to generate a plurality of sampled images.

[0067] Step S4, using the recognition module 40 to sequentially input all the sampled images into the trained classification model to obtain the pathological category recognition results corresponding to each sampled image.

[0068] Step S5: using the display module 50 to display the pathology image and the pathology category recognition result.

[0069] Functions and Effects of the Embodiments

[0070] According to the pathological type identification method and device based on ROSE cell morphology involved in this embodiment, on the one hand, according to the size relationship between the selection box and the sub-image, a corresponding strategy is adopted to collect abnormal cells; on the other hand, when testing the trained classification model, the cell category in the overlapping area is predicted by using the mean of the predicted value through the overlapping area between adjacent test images, and then a classification model with good prediction performance is trained. In short, this method can interpret ROSE results more accurately.

[0071] Those skilled in the art should understand that the present invention is not limited to the above embodiments, and the above embodiments and descriptions are only for explaining the principles of the present invention. Without departing from the spirit and scope of the present invention, the present invention may have various changes and improvements, and these changes and improvements fall within the scope of the present invention to be protected. The scope of protection of the present invention is defined by the attached claims and their equivalents.

Claims

1. A pathological type identification method based on ROSE cell morphology, used for identifying the pathological type of pathological images collected by the rapid on-site evaluation ROSE method, characterized in that: The following steps are involved: Step S1, constructing a training data set based on the existing multiple pathological images; Step S2, constructing a classification model, and training the classification model using the training data set to obtain a trained classification model; Step S3, sampling the pathological image to be identified to generate a plurality of sampled images; Step S4, inputting all the sampled images into the trained classification model in sequence to obtain the pathological category recognition results corresponding to each of the sampled images, Wherein, the step S1 includes the following sub-steps: Step S1-1, marking a selection box and a corresponding pathological category for abnormal cells in the pathological image; Step S1-2, intercepting a plurality of sub-images from the pathology image according to the selection box, and combining the corresponding pathology categories to obtain a plurality of positive samples; Step S1-3, intercepting a plurality of sub-images from the area of ​​the pathology image that is not the selected box, and setting the corresponding pathology category to normal cells, to obtain a plurality of negative samples; Step S1-4, constructing the training data set according to all the positive samples and the negative samples.

2. The method for identifying pathological types based on ROSE cell morphology according to claim 1, characterized in that: in, In the step S1-2, when the selection box is smaller than the sub-image, the sub-image includes all images within the selection box; When the selection box is larger than the sub-image, a point on the edge of the selection box is randomly selected, and another point is set according to the shape and size of the sub-image, and a rectangular area formed by the two points as diagonals is used as the sub-image.

3. The method for identifying pathological types based on ROSE cell morphology according to claim 1, characterized in that: in, In step S2, the training data set is divided into a training subset and a test subset. The classification model is trained through the training subset, and the test subset is used to determine whether the classification model is trained. If so, the trained classification model is obtained. If not, the classification model is retrained using the training data set. During the testing phase using the test subset, test images are extracted through a sliding window of a fixed size, and the moving distance of the sliding window is set to be smaller than the side length of the sliding window, so that overlapping areas exist between the test images.

4. The method for identifying pathological types based on ROSE cell morphology according to claim 3, characterized in that: in, For the overlapping area, the classification model uses the average of the prediction values ​​corresponding to all the test images containing the overlapping area as the prediction value of the overlapping area.

5. The method for identifying pathological types based on ROSE cell morphology according to claim 1, characterized in that: in, In the step S1-4, data enhancement is performed on the sub-graphs in the positive sample and the negative sample to obtain data enhanced samples. The training data set includes the data enhancement samples, the positive samples and the negative samples, The data enhancement includes flipping, rotation, scaling, cropping, shifting and Gaussian noise. The data enhancement is achieved through a deep learning library.

6. The method for identifying pathological types based on ROSE cell morphology according to claim 1, characterized in that: in, The backbone network of the classification model is a residual network.

7. The method for identifying pathological types based on ROSE cell morphology according to claim 1, characterized in that: in, In step S2, the classification model is trained by calculating the loss function and using back propagation according to the calculation result. The expression of the loss function is: Where p i is the pathological category corresponding to sample i, is the predicted category probability of the classification model for sample i in the training data set, L i is the calculation result of sample i calculated by the loss function.

8. A pathology type identification device based on ROSE cell morphology, used for identifying the pathology type of pathology images collected by the rapid on-site evaluation ROSE method, characterized in that: include: A training data generation module, used to construct a training data set based on the existing multiple pathological images; A model training module is used to construct a classification model and train the classification model using the training data set to obtain a trained classification model; A sampling module, used for sampling the pathological image to be identified to generate a plurality of sampling images; The recognition module is used to input all the sampled images into the trained classification model in sequence to obtain the pathological category recognition results corresponding to each of the sampled images. Wherein, the training data generation module includes: A marking unit, used for marking a selection box and a corresponding pathological category for abnormal cells in the pathological image; A positive sample generating unit, used for intercepting a plurality of sub-images from the pathology image according to the selection box, and combining the corresponding pathology categories to obtain a plurality of positive samples; A negative sample generating unit, used for intercepting a plurality of sub-images from the area of ​​the pathology image that is not the selected box, and setting the corresponding pathology category to normal cells, to obtain a plurality of negative samples; The training set construction unit is used to construct the training data set according to all the positive samples and the negative samples.

9. The method for identifying pathological types based on ROSE cell morphology according to claim 8, characterized in that: Also includes: A display module is used to display the pathology image and the pathology category recognition result.

Images