Multi-disease cell detection method, device and equipment and storage medium

Abstract

The invention discloses a multi-disease cell detection method, device and equipment and a storage medium. The method comprises the following steps: acquiring a cell slide image collected by a medical microscope, and determining a disease cell category of the cell slide image through a disease classification model; determining a corresponding cell detection model according to the disease cell category of the cell slide image, and determining the position of a positive cell in the cell slide image through the cell detection model; and displaying the cell slide image and the corresponding disease cell category, and marking the positive cells in the correspondingly displayed cell slide image according to the positions of the positive cells for a doctor to check. Through the technical means, a doctor can judge the condition of a patient under the assistance of benign, malignant and category of disease cells in the cell slide image, so that the diagnosis efficiency and the accuracy of a diagnosis result are improved, and the problems of low diagnosis efficiency and low diagnosis accuracy caused by the fact that the doctor observes the cells with naked eyes in the prior art are solved.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to a method, device, equipment and storage medium for detecting multiple disease cells. Background Art

[0002] A medical microscope is an optical instrument composed of one or more lenses that can magnify tiny substances such as cells and viruses that cannot be distinguished by the naked eye into images that can be distinguished by the naked eye. It has become an indispensable microscopic device in daily medical diagnosis and medical microscopic research. Microscopes provide a critical microscopic perspective for cytological research. By observing the morphological characteristics of cells under a microscope, such as the shape, size, and chromatin distribution of the cell nucleus, it is possible to accurately judge whether the cells are normal or not and whether there are morphological changes. This is extremely important for revealing the structural characteristics of cells under physiological conditions and abnormal morphological changes under pathological conditions.

[0003] In the prior art, when doctors diagnose various diseases in patients, they can observe the morphological characteristics and degree of differentiation of cells in liquid-based cell preparations through a microscope to determine the benign or malignant nature and type of cells. However, when doctors rely on the naked eye to observe liquid-based cells, it often takes a long time to accurately identify the characteristics of each cell, which may affect the efficiency of diagnosis. In addition, long-term observation can easily lead to visual fatigue, which makes doctors at risk of missed detection and misanalysis during the judgment process, thereby affecting the accurate assessment of the patient's condition. Summary of the invention

[0004] The present application provides a method, device, equipment and storage medium for detecting multiple disease cells, which determine the disease cell category and positive cell location of a cell slide image through a disease classification model and a cell detection model, so that doctors can judge the patient's condition with the help of the benign or malignant nature and category of the disease cells in the cell slide image, thereby improving the diagnostic efficiency and the accuracy of the diagnostic results, and solving the problem of low diagnostic efficiency and diagnostic accuracy caused by doctors observing cells with the naked eye in the prior art.

[0005] In a first aspect, the present application provides a method for detecting multiple disease cells, comprising: Acquire a cell slide image collected by a medical microscope, and determine the disease cell category of the cell slide image through a disease classification model; Determine a corresponding cell detection model according to the disease cell category of the cell slide image, and determine the position of positive cells in the cell slide image by using the cell detection model; The cell slide image and the corresponding disease cell category are displayed, and the positive cells are marked in the corresponding displayed cell slide image according to the positions of the positive cells for the doctor to view.

[0006] In a second aspect, the present application provides a multi-disease cell detection device, comprising: A cell classification module is configured to obtain a cell slide image collected by a medical microscope, and determine the disease cell category of the cell slide image through a disease classification model; A cell detection module is configured to determine a corresponding cell detection model according to the disease cell category of the cell slide image, and determine the position of positive cells in the cell slide image by using the cell detection model; The detection result display module is configured to display the cell slide image and the corresponding disease cell category, and mark the positive cells in the corresponding displayed cell slide image according to the location of the positive cells for the doctor to view.

[0007] In a third aspect, the present application provides a multi-disease cell detection device, comprising: One or more processors; a storage device storing one or more programs, when the one or more programs are executed by the one or more processors, the one or more processors implement the multi-disease cell detection method as described in the first aspect.

[0008] In a fourth aspect, the present application provides a storage medium comprising computer executable instructions, which, when executed by a computer processor, are used to perform the multi-disease cell detection method as described in the first aspect.

[0009] In the present application, by acquiring a cell slide image collected by a medical microscope, the disease cell category of the cell slide image is determined by a disease classification model; the corresponding cell detection model is determined according to the disease cell category of the cell slide image, and the position of the positive cells in the cell slide image is determined by the cell detection model; the cell slide image and the corresponding disease cell category are displayed, and the positive cells are marked in the corresponding displayed cell slide image according to the position of the positive cells for the doctor to view. Through the above technical means, the disease classification model can be used to classify the cell slide image to obtain the category information of the disease cells in the cell slide image, and then the corresponding cell detection model is selected by the category of the disease cells to perform positive cell detection on the cell slide image to accurately locate the positive cells in the cell slide image, and the category information and positive cells of the cell slide image are displayed to the doctor through the screen, so that the doctor can quickly confirm the benign and malignant nature and category of the liquid-based cells by viewing the displayed cell slide image, thereby improving the doctor's diagnostic efficiency of the patient's condition and the accuracy of the diagnostic results. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] Figure 1 This is a flow chart of a multi-disease cell detection method provided in an embodiment of the present application; Figure 2This is a flow chart of a training disease classification model provided in an embodiment of the present application; Figure 3 It is a flow chart of positive cell detection on a cell slide image provided in an embodiment of the present application; Figure 4 is a flow chart of a training cell detection model provided in an embodiment of the present application; Figure 5 It is one of the schematic diagrams of the display interface of the computer screen provided in the embodiment of the present application; Figure 6 This is a flowchart of displaying similar pictures according to the reading time provided by an embodiment of the present application; Figure 7 This is a second schematic diagram of a display interface of a computer screen provided in an embodiment of the present application; Figure 8 It is a schematic diagram of the structure of a multi-disease cell detection device provided in an embodiment of the present application; Fig. 9 It is a schematic diagram of the structure of a multi-disease cell detection device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0011] In order to make the purpose, technical scheme and advantages of the present application clearer, the specific embodiments of the present application are further described in detail below in conjunction with the accompanying drawings. It is understood that the specific embodiments described herein are only used to explain the present application, rather than to limit the present application. It should also be noted that, for the convenience of description, only part of the present application is shown in the accompanying drawings, but not all of the content. Before discussing the exemplary embodiments in more detail, it should be mentioned that some exemplary embodiments are described as processes or methods depicted as flow charts. Although the flow chart describes each operation (or step) as a sequential process, many of the operations therein can be implemented in parallel, concurrently or simultaneously. In addition, the order of the operations can be rearranged. The process can be terminated when its operation is completed, but it can also have additional steps not included in the accompanying drawings. The process can correspond to a method, a function, a procedure, a subroutine, a subprogram, etc.

[0012] The terms "first", "second", etc. in the specification and claims of the present application are used to distinguish similar objects, and are not used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable under appropriate circumstances, so that the embodiments of the present application can be implemented in an order other than those illustrated or described here, and the objects distinguished by "first", "second", etc. are generally of one type, and the number of objects is not limited. For example, the first object can be one or more. In addition, "and / or" in the specification and claims represents at least one of the connected objects, and the character " / " generally indicates that the objects associated with each other are in an "or" relationship.

[0013] In a more common implementation method, when diagnosing various diseases in patients, doctors can observe the morphological characteristics and degree of differentiation of cells in liquid-based cell preparations through a microscope to determine the benign or malignant nature and classification of cells. However, when doctors rely on the naked eye to observe liquid-based cells, it often takes a long time to accurately identify the characteristics of each cell, which may affect the efficiency of diagnosis. In addition, long-term observation can easily lead to visual fatigue, which makes doctors at risk of missed detection and misanalysis during the judgment process, thereby affecting the accurate assessment of the patient's condition. Therefore, improving the observation and analysis efficiency of liquid-based cells is crucial to improving the accuracy and timeliness of overall disease diagnosis.

[0014] To solve the above problems, the present embodiment provides a multi-disease cell detection method, which determines the disease cell category and positive cell location of the cell slide image through a disease classification model and a cell detection model, so that doctors can judge the patient's condition with the help of the benign or malignant nature and category of the disease cells in the cell slide image, thereby improving the diagnostic efficiency and the accuracy of the diagnostic results.

[0015] The multi-disease cell detection method provided in this embodiment can be performed by a multi-disease cell detection device, which can be implemented by software and / or hardware. The multi-disease cell detection device can be composed of two or more physical entities, or can be composed of one physical entity. For example, the multi-disease cell detection device can be an intelligent terminal with strong processing capabilities such as a computer and a server. Among them, the server can be implemented as an independent server or a server cluster composed of multiple servers.

[0016] The multi-disease cell detection device is installed with at least one type of operating system. The multi-disease cell detection device can install at least one application based on the operating system. The application can be an application that comes with the operating system or an application downloaded from a third-party device or server. In this embodiment, the multi-disease cell detection device is installed with at least an application that can execute the multi-disease cell detection method.

[0017] For ease of understanding, this embodiment is described by taking a computer as an example of the main body for executing the multi-disease cell detection method.

[0018] Figure 1 A flowchart of a multi-disease cell detection method provided in an embodiment of the present application is given. Figure 1 The multi-disease cell detection method specifically includes: S110, obtaining a cell slide image collected by a medical microscope, and determining the disease cell category of the cell slide image through a disease classification model.

[0019] Exemplarily, a medical microscope is connected to a computer for communication, such as the medical microscope can be connected to a computer's peripheral interface to serve as an external device of the computer. When a cell slide is placed under a medical microscope, the medical microscope captures a cell slide image and transmits the cell slide image to the computer through the computer's peripheral interface. After the computer receives the cell slide image transmitted by the medical microscope, it calls a disease classification model to classify the cell slide image and determine the category of diseased cells in the cell slide image. The disease classification model is pre-trained using a cell slide sample image labeled with category information.

[0020] In this embodiment, the disease cell categories include cells from different parts such as cervical cells, breast cells, and urothelial cells. After the disease cell category of the cell slide image is identified by the disease classification model, the disease cell category can be output to the doctor to assist the doctor in diagnosing the patient's condition. Specifically, the cell slide image is input into the disease classification model, and the disease classification model extracts the cell features in the cell slide image, and determines the disease cell category of the cell slide image according to the cell features.

[0021] Optionally, the disease classification model can be a deep residual network or other deeply separable network. This embodiment takes the disease classification model as a deep residual network as an example to describe the training process of the disease classification model. Figure 2 This is a flow chart of the training disease classification model provided in the embodiment of the present application. Figure 2 As shown, the steps of training the disease classification model specifically include S210-S220: S210, obtaining a plurality of sample data, the plurality of sample data including microscope image samples and whole-slice image samples, each sample data being marked with a corresponding disease cell category.

[0022] For example, 20,000 samples of data corresponding to each disease cell category are collected, of which 80% are microscope image samples and 20% are whole slide (WSI) image samples. Microscope image samples are collected through optical microscopes, and whole slide image samples can be collected on open websites. Since the imaging size of an optical microscope is fixed, it will collect 3000*4000 microscope image samples. In order to unify the size of the sample data, the whole slide image samples can be reduced or enlarged to 3000*4000.

[0023] S220. Train a deep residual network using multiple sample data to obtain a disease classification model.

[0024] Exemplarily, a sample data is input into a deep residual network, and the deep residual network extracts the cell features in the sample data, and predicts the disease cell category of the sample data based on the cell features. According to the disease cell category predicted by the deep residual network and the disease cell category marked by the sample data, a loss value is calculated, and the network parameters of the deep residual network are reversely optimized through the loss value, thereby completing a training of the deep residual network. After that, new sample data is obtained, and the disease cell category of the new sample data is predicted by the deep residual network. The loss value is calculated based on the disease cell category of the new sample data, and the deep residual network is optimized again based on the loss value. After all sample data are used to train the deep residual network, the deep residual network is used as a disease classification model.

[0025] The sample data used for training the disease classification model in this embodiment includes microscope image samples and whole-slice image samples. The disease classification model can learn the different feature expressions of the same cell in the two samples, thereby improving the detection accuracy of the disease classification model.

[0026] S120, determining a corresponding cell detection model according to the disease cell category of the cell slide image, and determining the position of the positive cells in the cell slide image through the cell detection model.

[0027] Exemplarily, the computer is deployed with multiple cell detection models corresponding to disease cell categories, for example, cervical cells correspond to cell detection model A, and urothelial cells correspond to cell detection model B. If the disease classification model identifies that the disease cell category of the cell slide image is cervical cells, the position of the positive cells in the cell slide image is determined by cell detection model A; if the disease classification model identifies that the disease cell category of the cell slide image is urothelial cells, the position of the positive cells in the cell slide image is determined by cell detection model B.

[0028] The cell slide image is input into the cell detection model corresponding to the disease cell category. The cell detection model extracts the shape and texture features of each cell in the cell slide image. The shape and texture features of each cell determine the positive cells and negative cells. The positive cells in the cell slide image are marked by the detection frame and then output. The position of the detection frame is the position of the corresponding marked positive cell.

[0029] Generally speaking, the size of the input image of the cell detection model is 1024*1024, and the size of the cell slide image is 3000*4000. In order to meet the size requirements of the cell detection model, the cell slide image can be reduced to 1024*1024 and then input into the cell detection model. However, size reduction will reduce the resolution of the cell slide image, resulting in the loss of some features and thus affecting the accuracy of cell detection. To this end, the cell slide image can be divided into multiple 1024*1024 sub-images for positive cell detection, and finally the positive cells can be located in the cell slide image according to the position of the sub-image in the cell slide image. Exemplary, Figure 3 1 is a flow chart of positive cell detection on a cell slide image provided by an embodiment of the present application. Figure 3 As shown, the step of performing positive cell detection on the cell slide image specifically includes S1201-S1203: S1201, dividing the cell slide image into a plurality of sub-images of preset sizes, and determining the position of each sub-image in the cell slide image.

[0030] Exemplarily, a plurality of sub-images are cropped from the cell slide image through a 1024*1024 sliding window, and the pixel coordinates of each sub-image in the cell slide image are determined.

[0031] S1202, input the sub-image into a cell detection model corresponding to the disease cell category, and obtain the position of the positive cells in the sub-image output by the cell detection model.

[0032] Exemplarily, the sub-image is input into a cell detection model corresponding to the disease cell category of the corresponding cell slide image, and the cell detection model extracts the morphological features and texture features of each cell in the sub-image, determines the negative cells and positive cells based on the morphological features and texture features, and outputs the detection box of the positive cells and the pixel coordinates of the detection box in the sub-image.

[0033] S1203. Determine the position of the positive cells in the cell slide image according to the position of the positive cells in the sub-image and the position of the sub-image in the cell slide image.

[0034] Exemplarily, according to the pixel coordinates of the detection frame corresponding to the positive cell in the sub-image and the pixel coordinates of the sub-image in the cell slide image, the detection frame corresponding to the positive cell is mapped to the cell slide image, so as to synchronously mark the detection frame of the positive cell in the cell slide image and determine the pixel coordinates of the detection frame. It should be noted that the detection frame of the positive cell circles the corresponding positive cell, and the pixel coordinates of the detection frame can be regarded as the pixel coordinates of the positive cell.

[0035] Optionally, the cell detection model can be a single-stage detection model such as the YOLO (You Only Look Once) model, the SSD (Single Shot MultiBox Detector) model, and the RetinaNet model, or a two-stage detection model such as the Faster R-CNN and Cascade R-CNN. Considering that disease diagnosis is generally real-time diagnosis with high requirements for timeliness, the cell detection model has a high detection speed, so the YOLO model can be selected as the cell detection model.

[0036] This embodiment takes the YOLO model as an example to describe the training process of the cell detection model. Figure 4 : is a flow chart of the training cell detection model provided in the embodiment of the present application. Figure 4 As shown, the steps of training the cell detection model specifically include S310-S330: S310 , dividing the sample data of the same disease type cell category into the same training set according to the disease type cell categories marked corresponding to the plurality of sample data.

[0037] Exemplarily, the sample data used to train the disease classification model is also used to train the cell detection model to avoid recollecting sample images and improve the model training efficiency. Multiple sample data are divided according to the corresponding labeled disease cell categories to divide the sample data of the same disease cell category into the same training set. For example, the sample data corresponding to cervical cells are divided into training set A, and the sample data corresponding to urothelial cells are divided into training set B.

[0038] It should be noted that the sample data used to train the disease classification model have the same number of samples corresponding to different disease cell categories, which is 20,000 sample data. Moreover, 80% of the sample data corresponding to a certain disease cell category are microscope image samples and 20% are whole slice image samples. When the sample data corresponding to different disease cell categories are used to train the corresponding cell detection model, the model performance of each cell detection model can be guaranteed to be roughly the same.

[0039] S320, dividing each sample data in the training set into a plurality of sample sub-images of preset sizes, and marking the positive cells in the sample sub-images with detection frames.

[0040] For example, the size of the microscope image sample and the scaled whole-slice image sample is 3000*4000, and the microscope image sample or the scaled whole-slice image sample can be slidably cropped by a sliding window of size 1024*1024 to obtain multiple sub-images of size 1024*1024. The positive cells in the sample data are marked by a detection frame, and after the sample data is divided into multiple sub-images, the sub-images also have detection frames marking positive cells.

[0041] S330 , training the YOLO model through the sample sub-images of each sample data in the training set, and obtaining a cell detection model for the disease cell category corresponding to the training set.

[0042] Exemplarily, the training set A includes sample data corresponding to cervical cells, and a YOLO model is trained using sample sub-images of each sample data in the training set A to obtain a cell detection model for detecting cervical cells.

[0043] When using the sample sub-images of each sample data in the training set to train the YOLO model, the sample sub-image is input into the YOLO model, and the YOLO model outputs the detection box of the positive cells in the sample sub-image. The loss value is calculated by comparing the detection box output by the YOLO model with the detection box marked in the sample sub-image, and the model parameters of the YOLO model are reversely optimized according to the loss value, thereby completing one training of the YOLO model. After that, a new sample sub-image is obtained, and the YOLO model is trained again based on the new sample sub-image. When the sample sub-images of each sample data in the training set are used to train the YOLO model, the YOLO model training is completed, and the trained YOLO model is used as the cell detection model corresponding to the disease cell category of the sample data in the training set.

[0044] After using one training set to train a cell detection model for the corresponding disease cell category, another training set is used to train the untrained YOLO model to obtain a cell detection model for the corresponding disease cell category in another training set. By analogy, cell detection models corresponding to multiple disease cell categories can be trained.

[0045] S130, displaying the cell slide image and the corresponding disease cell category, marking the positive cells in the corresponding displayed cell slide image according to the location of the positive cells for the doctor to view.

[0046] Exemplarily, a cell slide image is displayed on a computer screen and the disease cell category of the cell slide image is highlighted. A positive cell detection frame is marked in the cell slide image displayed on the screen according to the pixel coordinates of the positive cell detection frame, so that the doctor can confirm the positive cells in the cell slide image according to the detection frame.

[0047] For example, Figure 5is one of the schematic diagrams of the display interface of the computer screen provided in the embodiment of the present application. Figure 5 As shown, a cell slide image 11 is displayed in the display interface 10, and the disease cell category of the cell slide image 11 is displayed as cervical cells in text on one side of the cell slide image 11. The positive cells 13 in the cell slide image 11 are marked by a detection frame 12, and the negative cells 14 are not marked with a detection frame. Optionally, the computer counts the number of positive cells as n based on the number of detection frames 12 in the cell slide image 11, and displays the number of positive cells in text on one side of the cell slide image 11. The cell detection model can not only output the detection frame 12 of the positive cells, but also output the shape features, texture features, confidence and other information of the positive cells in the detection frame 12. The upper left corner of the detection frame 12 in the cell slide image 11 is marked with a corresponding detection frame identifier, and the information of the corresponding positive cells is displayed in text on one side of the cell slide image 11 based on the detection frame identifier. Furthermore, the cell detection model can not only identify positive cells but also negative cells. The cell detection model can use two different detection frames to mark positive cells and negative cells respectively, but the computer only displays the detection frame of positive cells in the cell slide image 11 displayed on the screen. The computer can count the number of detection frames of negative cells output by the cell detection model, count the number of negative cells in the cell slide image as m, and display the number of negative cells in text on one side of the cell slide image 11.

[0048] In one embodiment, the computer provides drawing controls, deletion controls, and adjustment controls in a display interface that displays a cell slide image. The doctor can use the drawing controls, deletion controls, and adjustment controls to trigger drawing operations, deletion operations, and adjustment operations, respectively, to add, delete, and modify detection boxes in the cell slide image.

[0049] refer to Figure 5 , the display interface 10 displays a drawing control 17, a deletion control 16, and an adjustment control 15. When the doctor finds that there are unmarked positive cells in the cell slide image 11, he can click the drawing control 17 to draw a bounding box around the unmarked positive cells in the cell slide image 11 to trigger the drawing operation. The computer responds to the drawing operation by drawing a bounding box in the cell slide image, and the bounding box is used to mark the positive cells in the cell slide image that are not marked by the detection box. Among them, the bounding box can be regarded as a detection box of positive cells drawn by the user. Once the bounding box is formed, the computer will determine the bounding box as the detection box of the positive cells at the corresponding position.

[0050] When the doctor finds that the cells marked by the detection frame 12 in the cell slide image 11 are negative cells, the doctor can click the delete control 16 and select the detection frame 12 in the cell slide image 11 to trigger the delete operation. The computer deletes the corresponding detection frame 12 in the cell slide image in response to the delete operation. For example, the doctor clicks the delete control 16 and selects the 1# detection frame. At this time, the computer can pop up a query window to ask the doctor whether to delete the detection frame. If the doctor clicks yes, the 1# detection frame is deleted from the cell slide image 11.

[0051] When the doctor finds that the position or size of the detection frame in the cell slide image 11 does not match the position or size of the corresponding positive cells, the doctor can click the adjustment control 15 and select the inner area, border or vertex of the detection frame 12 in the cell slide image 11 to slide for adjustment operation. The computer adjusts the position and / or size of the corresponding detection frame in the cell slide image in response to the adjustment operation. For example, when the user selects the inner area of ​​the detection frame 12 to move, the entire detection frame 12 is moved accordingly to adjust the position of the detection frame 12, when the user selects the border of the detection frame 12 to move, the selected border is moved accordingly to adjust the size and position of the detection frame 12, and when the user selects the vertex of the detection frame 12 to move, the selected vertex is moved accordingly to adjust the size and position of the detection frame 12.

[0052] This embodiment provides doctors with an editing tool for editing the detection frame of positive cells in a cell slide image, so that the doctor can re-edit the detection frame in the cell slide image, which is beneficial to improving the user experience.

[0053] The computer also provides a function to correct the disease cell category of the cell slide image. If the doctor finds that the disease cell category of the cell slide image is incorrect, the disease cell category of the cell slide image can be modified. After the doctor corrects the disease cell category of the cell slide image, the computer optimizes the model parameters of the disease classification model based on the cell slide image and the corresponding modified disease cell category.

[0054] When the doctor re-edits the cell slide image, it indicates that the cell detection result predicted by the cell detection model is inaccurate. In order to further improve the detection accuracy of the cell detection model, the re-edited cell slide image can be used as error data, and the model parameters of the cell detection model can be optimized again based on the error data. Optionally, the loss value can be calculated based on the detection box and / or bounding box in the cell slide image before and after editing, and the model parameters of the cell detection model of the corresponding disease cell category can be fine-tuned according to the loss value. For example, after the doctor adds, deletes or adjusts the detection box in the cell slide image, an edited cell slide image is obtained. The detection box in the edited cell slide image can be regarded as the correct detection box of each positive cell, and the edited cell slide image can be divided into multiple sub-images. The corresponding loss value is calculated based on the sub-images at the same position in the cell slide image before and after editing, and the average value of the loss value of each sub-image is calculated. The model parameters of the cell detection model of the disease cell category of the cell slide image are reversely optimized based on the average value.

[0055] In addition, the edited cell slide image can also be stored as an optimized image. When the number of optimized images of a certain disease cell category reaches a preset number, the model parameters of the cell detection model of the corresponding disease cell category are optimized based on the saved cell slide image. For example, the edited cell slide image is stored as an optimized image in the optimized data set of the corresponding disease cell category. When the number of optimized images in the optimized data set of a certain disease cell category reaches a preset number, each optimized image in the optimized data set is divided into multiple sub-images, and the sub-images are input into the cell detection model of the corresponding disease cell category to obtain the detection frame predicted by the cell detection model, and the loss value is calculated based on the detection frame predicted by the cell detection model in the sub-image and the detection frame in the corresponding optimized image, and the model parameters of the cell detection model are reversely optimized based on the loss value. After the cell detection model is optimized using the sub-images of each optimized image in the optimized data set, the optimized cell detection model can be used to perform cell detection on the cell slide images collected in real time by an optical microscope.

[0056] This embodiment utilizes the function of doctors editing cell slide images to obtain image data that is incorrectly predicted by the cell detection model, and fine-tunes the model parameters of the corresponding category of cell detection model based on the incorrectly predicted image data and the correct detection frame marks after editing, so that the cell detection model can learn more features of the cell slide images to adapt to a variety of practical application scenarios and improve the accuracy and robustness of the model.

[0057] In one embodiment, the behavior data of the doctor when viewing the cell slide image can be obtained, the doctor's reading habits can be analyzed based on the doctor's behavior data, and corresponding reading assistance can be provided to the doctor based on the doctor's reading habits.

[0058] Optionally, the doctors' attention to the cell slide images can be analyzed based on the doctors' reading time, and for cell slide images with high attention, an auxiliary function for viewing similar images can be provided to the doctors. Figure 6 This is a flow chart of displaying similar pictures according to the reading time provided by the embodiment of the present application. Figure 6 As shown, the step of displaying similar pictures according to the film viewing time specifically includes S1401-S1402: S1401. Obtain the length of time the doctor spends viewing the cell slide image.

[0059] Exemplarily, the timing starts from when the cell slide image is displayed on the display interface, and the real-time accumulated time is used as the time the doctor views the cell slide image.

[0060] S1402. When the viewing time exceeds a preset time, similar images are determined from historical cell slide images of the same disease cell category according to the disease cell category of the cell slide image, and the similar images are displayed on a split screen with the cell slide image; wherein the historical cell slide image is a cell slide image of which positive cell positions have been detected.

[0061] For example, when the viewing time does not exceed the preset time, it indicates that the doctor's attention to the cell slide image is low, and similar images of the cell slide image can be shown to the doctor. When the viewing time exceeds the preset time, it indicates that the doctor's attention to the cell slide image is high, and the doctor may want to focus on the distribution of cells in the cell slide image, or the doctor may not be able to diagnose the patient's condition based on the distribution of cells in the cell slide image. At this time, historical cell slide images of the same category can be searched according to the disease cell category of the cell slide image, and the similarity between the cell slide image and the historical cell slide images of the same category can be calculated. Images with similarity higher than the preset similarity threshold are used as similar images of the cell slide image, and the similar images and the cell slide images are displayed in split screen on the display interface of the computer. Figure 7 This is a second schematic diagram of a display interface of a computer screen provided in an embodiment of the present application. Figure 7 As shown, the similar image 18 is displayed on one side of the cell slide image 11, and the view of the similar image 18 is smaller than the view of the cell slide image 11. When the user clicks on the similar image 18, the corresponding clicked similar image 18 is enlarged and the cell slide image 11 is reduced to make the view sizes of the two the same, and then the cell slide image 11 and the similar image 18 are displayed in a left and right split screen.

[0062] In this embodiment, the cell slide images whose viewing time exceeds the preset time can also be stored as optimized images, so as to be used together with the cell slide images with predicted errors to optimize the cell detection model of the corresponding disease cell category in the future.

[0063] In addition, the average viewing time of doctors for various disease cell categories can be determined based on the viewing time of cell slide images of various disease cell categories, and the disease cell category with the longest average viewing time can be determined as the disease cell category with the highest attention. Accordingly, when the computer determines that the disease cell category of the currently displayed cell slide image is the disease cell category with the highest attention, similar images are directly determined in the historical cell slide images of the same category, and the cell slide image and the similar images are displayed in split screen.

[0064] In summary, the multi-disease cell detection method provided in the embodiment of the present application obtains the cell slide image collected by the medical microscope, determines the disease cell category of the cell slide image through the disease classification model; determines the corresponding cell detection model according to the disease cell category of the cell slide image, and determines the position of the positive cells in the cell slide image through the cell detection model; displays the cell slide image and the corresponding disease cell category, and marks the positive cells in the corresponding displayed cell slide image according to the position of the positive cells for the doctor to view. Through the above-mentioned technical means, the disease classification model can be used to classify the cell slide image to obtain the category information of the disease cells in the cell slide image, and then the corresponding cell detection model is selected according to the category of the disease cells to perform positive cell detection on the cell slide image to accurately locate the positive cells in the cell slide image, and the category information and positive cells of the cell slide image are displayed to the doctor through the screen, so that the doctor can quickly confirm the benign and malignant nature and category of the liquid-based cells by viewing the displayed cell slide image, thereby improving the doctor's diagnostic efficiency of the patient's condition and the accuracy of the diagnostic results.

[0065] Based on the above embodiments, Figure 8 This is a schematic diagram of the structure of a multi-disease cell detection device provided in an embodiment of the present application. Figure 8 The multi-disease cell detection device provided in this embodiment specifically includes: a cell classification module 21, a cell detection module 22 and a detection result display module 23.

[0066] The cell classification module 21 is configured to obtain a cell slide image collected by a medical microscope and determine the disease cell category of the cell slide image through a disease classification model; The cell detection module 22 is configured to determine a corresponding cell detection model according to the disease cell category of the cell slide image, and determine the position of the positive cells in the cell slide image through the cell detection model; The test result display module 23 is configured to display the cell slide image and the corresponding disease cell category, and mark the positive cells in the corresponding displayed cell slide image according to the location of the positive cells for the doctor to view.

[0067] On the basis of the above-mentioned embodiment, the cell detection module 22 includes: an image division unit, configured to divide the cell slide image into a plurality of sub-images of preset sizes, and determine the position of each sub-image in the cell slide image; a sub-image detection unit, configured to input the sub-image into a cell detection model of the corresponding disease cell category, and obtain the position of the positive cells in the sub-image output by the cell detection model; a positive cell determination unit, configured to determine the position of the positive cells in the cell slide image according to the position of the positive cells in the sub-image and the position of the sub-image in the cell slide image.

[0068] Based on the above embodiments, the multi-disease cell detection device also includes a classification model training module, which includes: a sample data acquisition unit, configured to acquire multiple sample data, the multiple sample data include microscope image samples and whole slice image samples, and each sample data is marked with a corresponding disease cell category; a classification model training unit, configured to train a deep residual network through multiple sample data to obtain a disease classification model.

[0069] On the basis of the above embodiments, the multi-disease cell detection device also includes a detection model training module, which includes: a training set generation unit, configured to divide sample data of the same disease cell category into the same training set according to the disease cell categories marked corresponding to multiple sample data; a sample image division unit, configured to divide each sample data in the training set into multiple sample sub-images of preset sizes, and the positive cells in the sample sub-images are marked by detection frames; and a detection model training unit, configured to train the YOLO model through the sample sub-images of each sample data in the training set to obtain a cell detection model of the disease cell category corresponding to the training set.

[0070] On the basis of the above embodiments, the multi-disease cell detection device also includes an image editing module, which includes: a drawing unit, configured to mark the positive cells in the corresponding displayed cell slide image according to the position of the positive cells for the doctor to view, and then draw a bounding box in the cell slide image in response to the drawing operation, wherein the bounding box is used to mark the positive cells in the cell slide image that are not marked by the detection box; and / or, a deleting unit, configured to delete the corresponding detection box in the cell slide image in response to the deleting operation; and / or, an adjusting unit, configured to adjust the position and / or size of the corresponding detection box in the cell slide image in response to the adjustment operation.

[0071] On the basis of the above embodiments, the multi-disease cell detection device also includes a model optimization module, which includes: a first optimization unit, configured to mark the positive cells in the corresponding displayed cell slide image according to the position of the positive cells for the doctor to view, and then calculate the loss value based on the detection box and / or the bounding box in the cell slide image before and after editing, and fine-tune the model parameters of the cell detection model of the corresponding disease cell category according to the loss value; and / or, a second optimization unit, configured to store the edited cell slide image as an optimized image, and when the number of optimized images of a certain disease cell category reaches a preset number, optimize the model parameters of the cell detection model of the corresponding disease cell category based on the saved cell slide image.

[0072] On the basis of the above-mentioned embodiments, the multi-disease cell detection device also includes an auxiliary viewing module, which includes: a viewing time determination unit, which is configured to obtain the doctor's viewing time of the cell slide image after marking the positive cells in the corresponding displayed cell slide image according to the position of the positive cells for the doctor to view; a similar image display unit, which is configured to determine similar images in historical cell slide images of the same disease cell category according to the disease cell category of the cell slide image when the viewing time exceeds a preset time, and display the similar images and the cell slide image in split screen; wherein the historical cell slide image is a cell slide image in which the positive cell position has been detected.

[0073] As mentioned above, the multi-disease cell detection device provided in the embodiment of the present application obtains the cell slide image collected by the medical microscope, determines the disease cell category of the cell slide image through the disease classification model; determines the corresponding cell detection model according to the disease cell category of the cell slide image, and determines the position of the positive cells in the cell slide image through the cell detection model; displays the cell slide image and the corresponding disease cell category, and marks the positive cells in the corresponding displayed cell slide image according to the position of the positive cells for the doctor to view. Through the above technical means, the disease classification model can be used to classify the cell slide image to obtain the category information of the disease cells in the cell slide image, and then the corresponding cell detection model is selected according to the category of the disease cells to perform positive cell detection on the cell slide image to accurately locate the positive cells in the cell slide image, and the category information and positive cells of the cell slide image are displayed to the doctor through the screen, so that the doctor can quickly confirm the benign and malignant nature and category of the liquid-based cells by viewing the displayed cell slide image, thereby improving the doctor's diagnostic efficiency of the patient's condition and the accuracy of the diagnostic results.

[0074] The multi-disease cell detection device provided in the embodiment of the present application can be used to execute the multi-disease cell detection method provided in the above embodiment, and has corresponding functions and beneficial effects.

[0075] Fig. 9 This is a schematic diagram of the structure of a multi-disease cell detection device provided in the embodiment of the present application, with reference to Fig. 9 The multi-disease cell detection device includes: a processor 31, a memory 32, a communication device 33, an input device 34 and an output device 35. The number of processors 31 in the multi-disease cell detection device can be one or more, and the number of memories 32 in the multi-disease cell detection device can be one or more. The processor 31, memory 32, communication device 33, input device 34 and output device 35 of the multi-disease cell detection device can be connected through a bus or other methods.

[0076] The memory 32, as a computer-readable storage medium, can be used to store software programs, computer executable programs and modules, such as program instructions / modules corresponding to the multi-disease cell detection method of any embodiment of the present application (for example, the cell classification module 21, the cell detection module 22 and the detection result display module 23 in the multi-disease cell detection device). The memory 32 may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function; the data storage area may store data created according to the use of the device, etc. In addition, the memory 32 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other non-volatile solid-state storage device. In some instances, the memory may further include a memory remotely arranged relative to the processor, and these remote memories may be connected to the device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network and a combination thereof.

[0077] The communication device 33 is used for data transmission.

[0078] The processor 31 executes various functional applications and data processing of the device by running the software programs, instructions and modules stored in the memory 32, that is, realizes the above-mentioned multi-disease cell detection method.

[0079] The input device 34 may be used to receive input digital or character information and generate key signal input related to user settings and function control of the device. The output device 35 may include a display device such as a display screen.

[0080] The multi-disease cell detection device provided above can be used to execute the multi-disease cell detection method provided in the above embodiment, and has corresponding functions and beneficial effects.

[0081] An embodiment of the present application also provides a storage medium containing computer executable instructions, which, when executed by a computer processor, are used to execute a method for detecting cells for multiple diseases, the method comprising: acquiring a cell slide image acquired by a medical microscope, and determining the disease cell category of the cell slide image through a disease classification model; determining a corresponding cell detection model according to the disease cell category of the cell slide image, and determining the position of positive cells in the cell slide image through the cell detection model; displaying the cell slide image and the corresponding disease cell category, and marking the positive cells in the corresponding displayed cell slide image according to the position of the positive cells for the doctor to view.

[0082] Storage medium - any of various types of memory devices or storage devices. The term "storage medium" is intended to include: installation media, such as CD-ROM, floppy disk or tape device; computer system memory or random access memory, such as DRAM, DDR RAM, SRAM, EDO RAM, Rambus RAM, etc.; non-volatile memory, such as flash memory, magnetic media (such as hard disk or optical storage); registers or other similar types of memory elements, etc. Storage media may also include other types of memory or combinations thereof. In addition, the storage medium may be located in the first computer system in which the program is executed, or may be located in a different second computer system, which is connected to the first computer system via a network (such as the Internet). The second computer system can provide program instructions to the first computer for execution. The term "storage medium" may include two or more storage media residing in different locations (for example, in different computer systems connected by a network). The storage medium may store program instructions (for example, embodied as a computer program) that can be executed by one or more processors.

[0083] Of course, the storage medium containing computer executable instructions provided in the embodiments of the present application is not limited to the above-mentioned multi-disease cell detection method, and the computer executable instructions can also execute related operations in the multi-disease cell detection method provided in any embodiment of the present application.

[0084] The multi-disease cell detection apparatus, multi-disease cell detection system, storage medium and multi-disease cell detection equipment provided in the above embodiments can execute the multi-disease cell detection method provided in any embodiment of the present application. For technical details not described in detail in the above embodiments, please refer to the multi-disease cell detection method provided in any embodiment of the present application.

[0085] The above are only preferred embodiments of the present application and the technical principles used. The present application is not limited to the specific embodiments herein, and various obvious changes, readjustments and substitutions that can be made by those skilled in the art will not deviate from the protection scope of the present application. Therefore, although the present application is described in more detail through the above embodiments, the present application is not limited to the above embodiments, and may also include more other equivalent embodiments without departing from the concept of the present application, and the scope of the present application is determined by the scope of the claims.

Claims

1. A method for detecting multiple disease cells, characterized in that: include: Acquire a cell slide image collected by a medical microscope, and determine the disease cell category of the cell slide image through a disease classification model; Determine a corresponding cell detection model according to the disease cell category of the cell slide image, and determine the position of positive cells in the cell slide image by using the cell detection model; The cell slide image and the corresponding disease cell category are displayed, and the positive cells are marked in the corresponding displayed cell slide image according to the positions of the positive cells for the doctor to view.

2. The multi-disease cell detection method according to claim 1, characterized in that: Determining the position of positive cells in the cell slide image by the cell detection model includes: Dividing the cell slide image into a plurality of sub-images of preset sizes, and determining the position of each sub-image in the cell slide image; Inputting the sub-image into a cell detection model corresponding to the disease cell category to obtain the position of positive cells in the sub-image output by the cell detection model; The position of the positive cells in the cell slide image is determined according to the position of the positive cells in the sub-image and the position of the sub-image in the cell slide image.

3. The multi-disease cell detection method according to claim 1, characterized in that: The training steps of the disease classification model include: Acquire a plurality of sample data, wherein the plurality of sample data include microscope image samples and whole-slice image samples, and each sample data is marked with a corresponding disease cell category; The disease classification model is obtained by training a deep residual network with the multiple sample data.

4. The multi-disease cell detection method according to claim 3, characterized in that: The training steps of the cell detection model include: According to the disease cell categories marked corresponding to the plurality of sample data, the sample data of the same disease cell category are divided into the same training set; Dividing each sample data in the training set into a plurality of sample sub-images of preset sizes, and marking positive cells in the sample sub-images by detection frames; The YOLO model is trained by using sample sub-images of each sample data in the training set to obtain a cell detection model for the disease cell category corresponding to the training set.

5. The multi-disease cell detection method according to claim 1, characterized in that: After marking the positive cells in the corresponding displayed cell slide image according to the positions of the positive cells for the doctor to view, the method further includes: In response to the drawing operation, a bounding box is drawn in the cell slide image, wherein the bounding box is used to mark positive cells in the cell slide image that are not marked by the detection box; and / or, In response to the deletion operation, the corresponding detection frame in the cell slide image is deleted; and / or, In response to the adjustment operation, the position and / or size of the corresponding detection frame in the cell slide image is adjusted.

6. The multi-disease cell detection method according to claim 5, characterized in that: After marking the positive cells in the corresponding displayed cell slide image according to the positions of the positive cells for the doctor to view, the method further includes: Calculate loss values ​​based on detection boxes and / or bounding boxes in the cell slide images before and after editing, and fine-tune model parameters of the cell detection model corresponding to the disease cell category according to the loss values; and / or, The edited cell slide image is stored as an optimized image. When the number of optimized images of a certain disease cell category reaches a preset number, the model parameters of the cell detection model of the corresponding disease cell category are optimized based on the saved cell slide image.

7. The multi-disease cell detection method according to claim 1, characterized in that: After marking the positive cells in the corresponding displayed cell slide image according to the positions of the positive cells for the doctor to view, the method further includes: Obtaining the length of time the doctor spends viewing the cell slide image; When the viewing time exceeds the preset time, similar images are determined from historical cell slide images of the same disease cell category according to the disease cell category of the cell slide image, and the similar images are displayed in split screen with the cell slide image; wherein the historical cell slide image is a cell slide image in which the positive cell position has been detected.

8. A multi-disease cell detection device, characterized in that: include: A cell classification module is configured to obtain a cell slide image collected by a medical microscope, and determine the disease cell category of the cell slide image through a disease classification model; A cell detection module is configured to determine a corresponding cell detection model according to the disease cell category of the cell slide image, and determine the position of positive cells in the cell slide image by using the cell detection model; The detection result display module is configured to display the cell slide image and the corresponding disease cell category, and mark the positive cells in the corresponding displayed cell slide image according to the location of the positive cells for the doctor to view.

9. A multi-disease cell detection device, characterized in that: include: one or more processors; A storage device stores one or more programs, and when the one or more programs are executed by the one or more processors, the one or more processors implement the multi-disease cell detection method as described in any one of claims 1-7.

10. A storage medium containing computer executable instructions, characterized in that: The computer executable instructions are used to execute the multi-disease cell detection method as described in any one of claims 1-7 when executed by a computer processor.

Images