Biological cell image classification method and device, computer device and storage medium

Abstract

This application relates to a method, apparatus, computer device, storage medium, and computer program product for classifying biological cell images. The method includes: acquiring a first cell image using an optical microscope; inputting the first cell image into an optical microscope classification model for classification processing to obtain a first classification result; if the first classification result meets preset conditions, acquiring a second cell image using a transmission electron microscope; inputting the second cell image into an electron microscope classification model for classification processing to obtain a second classification result, and feeding the second classification result back to a client. This method can improve classification accuracy.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computers, and in particular to a biological cell classification method and device, a computer device, a storage medium, and a computer program product. BACKGROUND

[0002] With the development of computer technology, determining the category of biological cells through morphological examination technology plays an increasingly important role in medical examination. In traditional technology, biological cells are manually observed morphologically through an optical microscope to determine the category of the biological cells. Due to the low magnification of the optical microscope, the ultrastructure of the biological cells, such as the cell membrane structure, organelles, and inclusion bodies, cannot be observed, and manual observation is affected by the subjectivity of the observer, so the error rate of classifying biological cells is high. SUMMARY

[0003] Therefore, it is necessary to provide a biological cell classification method and device, a computer device, a computer readable storage medium, and a computer program product that can improve the classification accuracy.

[0004] In a first aspect, the present application provides a biological cell image classification method. The method comprises:

[0005] acquiring a first cell image collected by an optical microscope;

[0006] inputting the first cell image into an optical microscope classification model for classification processing to obtain a first classification result;

[0007] if the first classification result meets a preset condition, acquiring a second cell image collected by a transmission electron microscope;

[0008] inputting the second cell image into an electron microscope classification model for classification processing to obtain a second classification result, and feeding back the second classification result to a client.

[0009] In a second aspect, the present application further provides a biological cell image classification device. The device comprises:

[0010] an acquisition module configured to acquire a first cell image collected by an optical microscope;

[0011] a classification module configured to input the first cell image into an optical microscope classification model for classification processing to obtain a first classification result;

[0012] the acquisition module is further configured to acquire a second cell image collected by a transmission electron microscope if the first classification result meets a preset condition;

[0013] The feedback module is configured to input the second cell image into the electron microscope classification model to perform classification processing, obtain a second classification result, and feed back the second classification result to the client.

[0014] In a third aspect, the present application provides a computer device. The computer device comprises a memory and a processor. The memory stores a computer program. The processor implements the following steps when executing the computer program:

[0015] obtaining a first cell image collected by an optical microscope;

[0016] inputting the first cell image into an optical microscope classification model to perform classification processing, and obtaining a first classification result;

[0017] if the first classification result meets a preset condition, obtaining a second cell image collected by a transmission electron microscope;

[0018] inputting the second cell image into an electron microscope classification model to perform classification processing, obtaining a second classification result, and feeding back the second classification result to the client.

[0019] In a fourth aspect, the present application provides a computer readable storage medium. The computer readable storage medium stores a computer program. The computer program is executed by a processor to implement the following steps:

[0020] obtaining a first cell image collected by an optical microscope;

[0021] inputting the first cell image into an optical microscope classification model to perform classification processing, and obtaining a first classification result;

[0022] if the first classification result meets a preset condition, obtaining a second cell image collected by a transmission electron microscope;

[0023] inputting the second cell image into an electron microscope classification model to perform classification processing, obtaining a second classification result, and feeding back the second classification result to the client.

[0024] In a fifth aspect, the present application provides a computer program product. The computer program product comprises a computer program. The computer program is executed by a processor to implement the following steps:

[0025] obtaining a first cell image collected by an optical microscope;

[0026] inputting the first cell image into an optical microscope classification model to perform classification processing, and obtaining a first classification result;

[0027] If the first classification result meets a preset condition, a second cell image collected by a transmission electron microscope is obtained;

[0028] The second cell image is input into an electron microscope classification model for classification processing to obtain a second classification result, and the second classification result is fed back to the client.

[0029] The above biological cell image classification method, device, computer equipment, storage medium and computer program product, obtain a first cell image collected by an optical microscope, and input the first cell image into an optical microscope classification model for classification processing to obtain a first classification result. If the first classification result meets a preset condition, it means that there may be tumor cells in the cell, but because the magnification of the optical microscope is low, the ultrastructure of the cell cannot be observed according to the cell image collected by the optical microscope, so the category of the biological cell cannot be accurately determined. Therefore, a second cell image collected by a transmission electron microscope is obtained, and then the second cell image is input into an electron microscope classification model for classification processing to obtain a second classification result, and the second classification result is fed back to the client. Because the magnification of the transmission electron microscope is much higher than that of the optical microscope, the second cell image including the ultrastructure of the biological cell can be collected by the transmission electron microscope, so that the biological cell can be accurately classified according to the second cell image, and the classification accuracy is improved. In addition, the classification processing is performed by the classification model, which improves the classification efficiency and can avoid classification errors caused by insufficient experience when manually classifying cells, further improving the classification accuracy. BRIEF DESCRIPTION OF DRAWINGS

[0030] Figure 1 It is an application environment diagram of the biological cell image classification method in one embodiment;

[0031] Figure 2 It is a flowchart of the biological cell image classification method in one embodiment;

[0032] Figure 3 It is a schematic diagram of the first cell image in one embodiment;

[0033] Figure 4 It is a schematic diagram of the second cell image in one embodiment;

[0034] Figure 5 It is a schematic diagram of the principle of the biological cell image classification method in one embodiment;

[0035] Figure 6 It is a schematic diagram of the optical microscope auxiliary classification unit in one embodiment;

[0036] Figure 7 It is a transmission electron microscope auxiliary classification unit in one embodiment;

[0037] Figure 8 Flowchart of a biological cell image classification method according to another embodiment;

[0038] Figure 9 Flowchart of a biological cell image classification method according to another embodiment;

[0039] Figure 10 Block diagram of a biological cell image classification device according to an embodiment;

[0040] Figure 11 Block diagram of a biological cell image classification device according to another embodiment;

[0041] Figure 12 Internal structure diagram of a computer device according to an embodiment. DETAILED DESCRIPTION

[0042] In order to make the purposes, technical solutions and advantages of the present application clearer, the present application will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and not intended to limit the present application.

[0043] It should be noted that the terms "first", "second", and the like in the description and claims of the present application and the above-described accompanying drawings are used to distinguish similar objects, and do not necessarily have to describe a specific order or sequence. It should be understood that the data thus used can be interchanged under appropriate circumstances, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein.

[0044] The biological cell image classification method provided by the embodiments of the present application can be applied to, for example, Figure 1The application environment is shown. Among them, the data storage system can store the data required by the server 102 to process. The data storage system can be integrated on the server 102, or placed on the cloud or other network servers. The server 102 acquires a first cell image collected by an optical microscope; input the first cell image into an optical microscope classification model for classification processing to obtain a first classification result; if the first classification result meets the preset condition, a second cell image collected by a transmission electron microscope is acquired; input the second cell image into an electron microscope classification model for classification processing to obtain a second classification result, and feed back the second classification result to the client. Among them, the server 102 can be an independent physical server, or a server cluster composed of multiple physical servers, can be a cloud server providing cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content distribution network (Content Delivery Network, CDN), and big data and artificial intelligence platform Basic cloud computing services such as platform.

[0045] In one embodiment, as Figure 2 shown, a biological cell image classification method is provided, which is applied to the server in Figure 1 for example, including the following steps:

[0046] S202, acquiring a first cell image collected by an optical microscope.

[0047] Among them, the optical microscope is an optical instrument for magnifying and imaging objects by optical principles, which is composed of a mirror seat, an objective lens, an ocular lens, a carrier table and a mirror arm, etc. The maximum magnification is 1000 times. The first cell image is a cell image collected by an optical microscope, for example, as Figure 3 shown, the first cell image includes images of multiple nuclear heteromorphous mesothelial cells. The first cell image can be a cell image obtained by collecting a first biological specimen. The biological specimen can be a body fluid specimen or a tissue specimen, for example, the specimen of cerebrospinal fluid, serous cavity effusion (including pleural effusion, peritoneal effusion and pericardial effusion), joint cavity effusion, stool, semen, vaginal secretion, urine, alveolar lavage fluid, gastric juice and duodenal drainage fluid, prostatic fluid, etc.; The specimen of tissue such as epithelial tissue, muscle tissue or nerve tissue, etc. The server can label the first cell image, and the labeling content includes cell characteristics, whether there is bacteria and bacteria type, parasitic information and crystal information, etc.

[0048] In one embodiment, S202 specifically comprises: collecting the test smear by an optical microscope to obtain a first cell image. The test smear is a smear prepared according to a biological specimen collected from a body fluid or tissue. When the biological specimen is collected, the biological specimen is sequentially subjected to specimen classification, cell counting, centrifugation and staining treatment to obtain the test smear of the biological specimen.

[0049] S204, inputting the first cell image into the optical microscope classification model for classification processing to obtain a first classification result.

[0050] The optical microscope classification model is a machine learning model for classifying cells according to cell images collected by an optical microscope. It can be a machine learning model trained by a supervised learning algorithm. For example, the optical microscope classification model can be a support vector machine model, a logistic regression binary classification model, a neural network model or a decision tree model, etc. The first classification result is the classification result of the biological cells in the first cell image. The first classification result can be normal cells or abnormal cells; or it can also represent the specific category of biological cells, such as red blood cells, white blood cells, tumor cells, etc.; or the first classification result can also include the category of biological cells and the probability value corresponding to the category, etc.

[0051] In one embodiment, the optical microscope classification model is deployed in an optical microscope assisted classification unit. S202 further comprises: determining whether the first cell image is classified by the optical microscope assisted classification unit; if yes, inputting the first cell image into the optical microscope classification model for classification processing to obtain the first classification result. If no, classifying the first cell image by manual reading to obtain the first classification result.

[0052] In one embodiment, S204 further comprises: if the first classification result fails to pass the review, re-collecting the first cell image by the optical microscope, and inputting the re-collected first cell image into the optical microscope classification model for classification processing to obtain the first classification result.

[0053] S206, if the first classification result meets a preset condition, obtaining a second cell image collected by a transmission electron microscope.

[0054] The preset condition is a condition for determining whether to classify cells by a transmission electron microscope. For example, the preset condition can be that the first classification result contains tumor cells or special substances such as crystals, bacteria, etc.

[0055] Transmission electron microscopy (TEM) is an imaging instrument that uses an electron beam as a light source and an electromagnetic field as a lens. It consists of multiple parts, including an electron gun, objective lens, diffraction mirror, and projection mirror. TEM can achieve magnifications of several million times, allowing observation of ultrastructures as small as 0.2 nanometers. The second cell image is a cell image acquired using a TEM, for example... Figure 4 As shown, the second cell image is an image of a heterogeneous mesothelial cell, which clearly displays the ultrastructure of the cell, including the nucleus. The second cell image can be a cell image obtained from a second biological specimen. The second biological specimen and the first biological specimen can be the same biological specimen, or they can be specimens collected from the same organism. The biological specimen can be a body fluid specimen or a tissue specimen. Examples of body fluid specimens include cerebrospinal fluid, serous cavity effusion (including pleural effusion, ascites, and pericardial effusion), joint effusion, feces, semen, vaginal secretions, urine, bronchoalveolar lavage fluid, gastric juice and duodenal drainage fluid, and prostatic fluid. Examples of tissue specimens include epithelial tissue, muscle tissue, or nerve tissue. The server can annotate the second cell image, including ultrastructural or morphological information such as organelles, the nucleus, and the cell membrane.

[0056] S208: Input the second cell image into the electron microscope classification model for classification processing, obtain the second classification result, and feed the second classification result back to the client.

[0057] The electron microscope classification model is a machine learning model used to classify cells based on cell images acquired through a transmission electron microscope. It can be a machine learning model trained using a supervised learning algorithm. For example, the electron microscope classification model can be a support vector machine model, a neural network model, or a decision tree model. The second classification result is the classification of biological cells in the second cell image. The second classification result can be normal cells or abnormal cells; or it can include the category of the biological cell and the corresponding probability value.

[0058] In one embodiment, after S208, the method further includes: if the second classification result fails the review, then re-acquiring the second cell image using a transmission electron microscope, and inputting the re-acquiring second cell image into the electron microscope classification model for classification processing to obtain the second classification result.

[0059] In the above embodiment, the first cell image collected by the optical microscope is acquired, and the first cell image is input into the optical microscope classification model for classification processing to obtain a first classification result. If the first classification result meets the preset condition, it indicates that there may be tumor cells in the cells. However, because the magnification of the optical microscope is low, the ultrastructure of the cells cannot be observed according to the cell image collected by the optical microscope, so the category of the biological cells cannot be accurately determined. Therefore, the second cell image collected by the transmission electron microscope is acquired, and then the second cell image is input into the electron microscope classification model for classification processing to obtain a second classification result, and the second classification result is fed back to the client. Because the magnification of the transmission electron microscope is much higher than that of the optical microscope, the second cell image including the ultrastructure of the biological cells can be collected by the transmission electron microscope, so that the biological cells can be accurately classified according to the second cell image, and the classification accuracy is improved. In addition, the classification processing is performed by using the classification model, the classification efficiency is improved, and the classification error caused by insufficient experience when the cells are classified manually can be avoided, and the classification accuracy is further improved.

[0060] In one embodiment, S202 specifically includes: acquiring a first low-magnification cell image collected by an optical microscope at a low magnification; moving a lens of the optical microscope above a target region of the first cell image according to the first low-magnification cell image; acquiring a first high-magnification cell image of the target region collected by the optical microscope at a high magnification, and taking the first high-magnification cell image as the first cell image.

[0061] The target region can be an image region containing a specific type of biological cell. For example, the target region can be an image region containing suspected tumor cells in the first cell image; or the target region can also be an image region containing abnormal cells in the first cell image.

[0062] The server first acquires a first low-magnification cell image collected by an optical microscope at a low magnification, and searches for abnormal cells or suspected tumor cells in the first low-magnification cell image. When the abnormal cells or suspected tumor cells are found, the lens of the optical microscope is moved above the target region containing the abnormal cells or suspected tumor cells, and then the lens of the optical microscope is adjusted to a high-magnification lens. The abnormal cells or suspected tumor cells in the target region are observed by using the high-magnification lens, and the target region is collected at a high magnification to obtain a first high-magnification cell image.

[0063] In one embodiment, S202 specifically includes: driving the lens of the optical microscope to move through a digital driving interface; and acquiring an image of an observation region corresponding to the lens during the movement of the lens to obtain the first cell image.

[0064] The server drives the lens of the optical microscope to move through a digital driving interface, so that the observation area of the optical microscope covers all the test smears in turn. During the movement of the lens, the observation area corresponding to the lens is imaged by the optical microscope to obtain a first cell image.

[0065] In one embodiment, S202 further includes: if the first cell image is a color image, performing grayscale processing on the first cell image to convert the first cell image into a grayscale image. Then the grayscale first cell image is sequentially binarized to sharpen the edges of the first cell image. Then the binarized first cell image is edge detected to extract the cell contour in the first cell image. Then image features are extracted from the cell contour, and the image features are input into the optical microscope classification model for classification processing.

[0066] In the above embodiment, the server obtains a first low magnification cell image collected by the optical microscope at a low magnification, and finds a specific type of biological cell according to the first low magnification cell image. Since the magnification of the first low magnification cell image is low, the observation range of the lens is large, which can improve the search speed of the specific type of biological cell. Then the lens of the optical microscope is moved above the target area containing the specific biological cell. The first high magnification cell image obtained by the optical microscope at a high magnification is obtained, so that the high magnification cell image of the specific type of biological cell can be collected, the image clarity is improved, and the classification of the specific type of biological cell is facilitated, and the classification accuracy is improved.

[0067] In one embodiment, S206 specifically includes: if the first classification result meets a preset condition, obtaining a second low magnification cell image collected by the transmission electron microscope at a low magnification; moving the lens of the transmission electron microscope above a target area of the second cell image according to the second low magnification cell image; obtaining a second high magnification cell image obtained by the transmission electron microscope at a high magnification on the target area, and taking the second high magnification cell image as the second cell image.

[0068] The server first acquires a second low-magnification cell image collected by the transmission electron microscope at a low magnification (e.g., 5000 times), searches for a specific type of abnormal cell or suspected tumor cell in the second low-magnification cell image, when a specific type of biological cell is found, moves the lens of the transmission electron microscope above the target area containing the specific type of biological cell, and then adjusts the lens of the transmission electron microscope to a high magnification lens (e.g., 10000 times), observes the specific type of biological cell in the target area through the high magnification lens, and collects the target area at a high magnification to obtain a second high-magnification cell image.

[0069] In one embodiment, S206 specifically includes: driving the lens of the transmission electron microscope to move through the digital driving interface; and collecting an image of an observation area corresponding to the lens during the movement of the lens to obtain a second cell image.

[0070] The server drives the lens of the transmission electron microscope to move through the digital driving interface, so that the observation area of the transmission electron microscope sequentially covers all the sample sections. During the movement of the lens, the lens corresponding observation area is collected by the transmission electron microscope to obtain a second cell image.

[0071] In the above embodiment, the server acquires a second low-magnification cell image collected by the transmission electron microscope at a low magnification, and searches for a specific type of biological cell according to the second low-magnification cell image. Since the magnification of the second low-magnification cell image is low, the search speed for the specific type of biological cell can be improved. Then the lens of the transmission electron microscope is moved to above the target area containing the specific biological cell. The target area is collected by the transmission electron microscope at a high magnification to obtain a second high-magnification cell image, so that a high-magnification cell image of the specific type of biological cell can be collected, the image clarity is improved, and the classification of the specific type of biological cell is facilitated, and the classification accuracy is improved.

[0072] In one embodiment, S208 further includes: acquiring a cell image sample collected by the transmission electron microscope, and composing a training data set from the cell image sample; adjusting the parameters of the pre-trained learning model according to the training data set; and taking the learning model after parameter adjustment as the electron microscope classification model.

[0073] The cell image sample collected by the transmission electron microscope can be a cell image sample collected from a biological specimen. The biological specimen can be a body fluid specimen or a tissue specimen. The body fluid specimen can be, for example, cerebrospinal fluid, serous cavity effusion (including pleural effusion, peritoneal effusion, and pericardial effusion), joint cavity effusion, stool, semen, vaginal secretion, urine, alveolar lavage fluid, gastric juice and duodenal drainage, prostatic fluid, etc. The tissue specimen can be, for example, epithelial tissue, muscle tissue, or nerve tissue, etc. The server adjusts the parameters of the pre-trained learning model according to a training data set composed of the pre-collected cell image samples, and takes the learning model with the adjusted parameters as the transmission electron microscope classification model.

[0074] In one embodiment, the second cell image is added to the training data set to obtain a new training data set, and the pre-trained second learning model is adjusted according to the new training data set. Thus, the learning model can be continuously iteratively optimized, the training samples in the training data set are expanded as the second cell image collected by the transmission electron microscope increases, and the learning model can be continuously optimized, thereby improving the classification accuracy of the obtained classification model.

[0075] In one embodiment, S204 further includes: obtaining a cell image sample collected by an optical microscope, and composing a training data set from the collected cell image sample; adjusting the parameters of a pre-trained learning model according to the training data set; and taking the learning model with the adjusted parameters as an optical microscope classification model.

[0076] The cell image sample collected by the optical microscope can be a cell image sample collected from a biological specimen. The biological specimen can be a body fluid specimen or a tissue specimen. The body fluid specimen can be, for example, cerebrospinal fluid, serous cavity effusion (including pleural effusion, peritoneal effusion, and pericardial effusion), joint cavity effusion, stool, semen, vaginal secretion, urine, alveolar lavage fluid, gastric juice and duodenal drainage, prostatic fluid, etc. The tissue specimen can be, for example, epithelial tissue, muscle tissue, or nerve tissue, etc. The server adjusts the parameters of the pre-trained learning model according to a training data set composed of the pre-collected cell image samples, and takes the learning model with the adjusted parameters as the optical microscope classification model.

[0077] In one embodiment, the server adds the first cell image to the training data set to obtain a new training data set, and adjusts the parameters of the pre-trained learning model according to the new training data set. Thus, the learning model can be continuously iteratively optimized, the training samples in the training data set are expanded as the first cell image collected by the optical microscope increases, and the learning model can be continuously optimized, thereby improving the classification accuracy of the obtained classification model.

[0078] In the above embodiment, the server acquires the cell image sample collected by the optical microscope, and forms a training data set from the first cell image sample, then adjusts the parameters of the pre-trained learning model according to the training data set, and takes the learning model after parameter adjustment as the optical microscope classification model. Thus, the structure of the learning model can be optimized by adjusting the parameters of the learning model, and the classification accuracy of the obtained classification model is improved.

[0079] In one embodiment, S208 further includes: when the second classification result includes the ultrastructure of other cells other than the cell types in the classification list, inputting the second cell image into a clustering model for clustering processing to obtain a clustering result; and feeding back the clustering result to the client.

[0080] The classification list is a list storing known cell types. The clustering model is a machine learning model trained by an unsupervised learning algorithm. When the second classification result includes other cell types other than the cell types in the classification list, for example, when the second classification result is displayed as "other types", the second cell image is input into the clustering model for clustering processing to obtain a clustering result.

[0081] In the above embodiment, when the second classification result includes other cell types other than the cell types in the classification list, the second cell image is input into the clustering model for clustering processing to obtain a clustering result and feed back the clustering result to the client. Thus, the same type of images in the second cell image are found by clustering operation of the second cell image through the unsupervised learning algorithm, to assist in discovering unknown cell types.

[0082] In one embodiment, the second cell image is an image acquired from a slide prepared from the second biological specimen. First, the second biological specimen is centrifuged and concentrated. Then, the treated second biological specimen is placed in a 2.5% glutaraldehyde solution and fixed at 4 degrees Celsius for 12 hours. The supernatant (glutaraldehyde fixative) is then removed with a pipette, and the specimen is fixed with agarose to form a specimen block. Excess agarose is removed, and the specimen block is rinsed three times with 0.1M phosphate buffer (pH 7.0), each rinse lasting 15 minutes. After rinsing, the specimen is soaked in a 1% osmium tetroxide solution for 1–2 hours, and the osmium tetroxide waste is aspirated. The specimen block is then rinsed three more times with 0.1M phosphate buffer (pH 7.0), each rinse lasting 15 minutes. Finally, the sample is dehydrated with ethanol solutions of varying concentrations (30%, 50%, 70%, and 80%), each concentration for 15 minutes. Subsequently, the sample was treated with 90% and 95% acetone solutions for 15 minutes each. It was then treated twice with pure acetone, 20 minutes each time (the sample was shaken during the process to ensure thorough dehydration). After dehydration, the sample was treated with a mixture of low-viscosity resin embedding agent and acetone (V / V = 1:1) for 1 hour, followed by a mixture of low-viscosity resin embedding agent and acetone (V / V = 3:1) for 3 hours, and then treated with pure low-viscosity resin embedding agent at room temperature for at least 12 hours. After embedding, the embedded sample was kept at 70°C for 12 hours before being sectioned (70–90 nm thick). Finally, the sections were stained with lead citrate solution and 50% ethanol saturated solution of uranium acetate for 5–10 minutes.

[0083] In one embodiment, such as Figure 5 As shown, the biological cell image classification system includes a specimen processing unit, a smear preparation unit, a slide preparation unit, an optical microscope, a transmission electron microscope, an optical microscope-assisted classification unit, and a transmission electron microscope-assisted classification unit. The specimen processing unit processes biological specimens, including specimen classification, cell counting, and centrifugation. Then, the smear preparation unit prepares test smears from the processed biological specimens, and the slide preparation unit prepares sample slides from the processed biological specimens. The server acquires a first cell image obtained through an optical microscope and inputs it into the optical microscope-assisted classification unit. The optical microscope classification model deployed in the optical microscope-assisted classification unit classifies the first cell image to obtain a first classification result. If the first classification result meets preset conditions, a second cell image obtained through a transmission electron microscope is acquired. The second cell image is input into the transmission electron microscope-assisted classification unit, where the electron microscope classification model deployed in the transmission electron microscope-assisted classification unit classifies the second cell image to obtain a second classification result, which is then fed back to the client.

[0084] As shown in Figure 6 The optical microscope assisted classification unit includes a data interface subunit, an image processing subunit, a model training subunit, and a model classification subunit. The server obtains the first cell image sample collected by the optical microscope through the data interface subunit, and then performs image processing on the first cell image sample through the image processing subunit, to form a training data set composed of the first cell image sample after image processing. The model training subunit trains the pre-trained learning model according to the training data set, to obtain an optical microscope classification model. The model classification subunit classifies the first cell image through the optical microscope classification model, to obtain a first classification result.

[0085] As shown in Figure 7 The transmission electron microscope assisted classification unit includes a data interface subunit, an image processing subunit, a model training subunit, a model classification subunit, and a clustering unit. The server obtains the second cell image sample collected by the transmission electron microscope through the data interface subunit, and then performs image processing on the second cell image sample through the image processing subunit, to form a training data set composed of the second cell image sample after image processing. The model training subunit trains the pre-trained learning model according to the training data set, to obtain an electron microscope classification model. The model classification subunit classifies the second cell image through the electron microscope classification model, to obtain a second classification result. When the second classification result includes other cell types in addition to the cell types in the classification list, the second cell image is input into the clustering unit for clustering processing, to obtain a clustering result and feed the clustering result back to the client.

[0086] In one embodiment, as Figure 8As shown, the server acquires a first cell image collected by an optical microscope on a preprocessed specimen, and then determines whether to perform assisted classification. If assisted classification is performed, the first cell image is processed, and then the processed first cell image is input into an optical microscope classification model for classification processing to obtain a first classification result. The optical microscope classification model is a machine learning model obtained by training on first cell image samples. The server audits the first classification result. If the audit fails, the server reacquires the first cell image by the optical microscope, and obtains a first classification result according to the reacquired first cell image. After obtaining the first classification result, the server determines whether to observe the ultrastructure of the cell. If the determination result is yes, the server acquires a second cell image collected by a transmission electron microscope on a specimen slice prepared according to the biological specimen. The server determines whether to perform assisted classification. If assisted classification is performed, the second cell image is processed, and then the processed second cell image is input into an electron microscope classification model for classification processing to obtain a second classification result. The electron microscope classification model is a machine learning model obtained by training on second cell image samples. The server labels the classification result and the ultrastructure of the second cell image, and adds the labeled second cell image to a training data set.

[0087] In one embodiment, as shown in Figure 9 The biological cell image classification method includes the following steps:

[0088] S902, acquiring a first low magnification cell image collected by an optical microscope at a low magnification.

[0089] S904, moving a lens of the optical microscope to above a target region of the first cell image according to the first low magnification cell image.

[0090] S906, acquiring a first high magnification cell image collected by the optical microscope at a high magnification on the target region, and taking the first high magnification cell image as the first cell image.

[0091] S908, inputting the first cell image into an optical microscope classification model for classification processing to obtain a first classification result.

[0092] S910, if the first classification result meets a preset condition, acquiring a second low magnification cell image collected by a transmission electron microscope at a low magnification.

[0093] S912, moving a lens of the transmission electron microscope to above a target region of the second cell image according to the second low magnification cell image.

[0094] S914, acquire a second high-magnification cell image obtained by collecting the target region at a high magnification by a transmission electron microscope, and take the second high-magnification cell image as a second cell image.

[0095] S916, input the second cell image into an electron microscope classification model for classification processing to obtain a second classification result, and feed back the second classification result to the client.

[0096] S918, when the second classification result includes ultrastructures of other cells other than the cell types in the classification list, input the second cell image into a clustering model for clustering processing to obtain a clustering result, and feed back the clustering result to the client.

[0097] The specific contents of S902 to S918 above can refer to the specific implementation process described above.

[0098] It should be understood that although each step in the flowchart involved in each of the above embodiments is displayed in sequence according to the arrow, these steps are not necessarily executed in sequence according to the arrow. Unless otherwise stated herein, the execution of these steps is not strictly limited in sequence, and these steps can be executed in other sequences. Moreover, at least part of the steps in the flowchart involved in each of the above embodiments can include multiple steps or stages, which are not necessarily executed at the same time, but can be executed at different times, and the execution sequence of these steps or stages is not necessarily sequential, but can be executed alternately or alternately with at least part of other steps or steps or stages in other steps.

[0099] Based on the same inventive concept, the embodiments of the present application also provide a biological cell image classification device for implementing the above-mentioned biological cell image classification method. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme described in the above method, so the specific limitations in one or more biological cell image classification device embodiments provided below can refer to the limitations of the biological cell image classification method in the above, which will not be repeated here.

[0100] In one embodiment, as shown in Figure 10 , a biological cell image classification device is provided, comprising: an acquisition module, a classification module and a feedback module, wherein:

[0101] The acquisition module 1002 is configured to acquire a first cell image collected by an optical microscope;

[0102] The classification module 1004 is configured to input the first cell image into an optical microscope classification model for classification processing to obtain a first classification result;

[0103] The acquisition module 1002 is further configured to acquire a second cell image collected by a transmission electron microscope if the first classification result meets the preset condition.

[0104] The feedback module 1006 is configured to input the second cell image into an electron microscope classification model for classification processing to obtain a second classification result, and feed back the second classification result to the client.

[0105] In the above embodiment, the first cell image collected by the optical microscope is acquired, and the first cell image is input into the optical microscope classification model for classification processing to obtain the first classification result. If the first classification result meets the preset condition, it indicates that there may be tumor cells in the cell. However, due to the low magnification of the optical microscope, the ultrastructure of the cell cannot be observed according to the cell image collected by the optical microscope, so the category of the biological cell cannot be accurately determined. Therefore, the second cell image collected by the transmission electron microscope is acquired, and then the second cell image is input into the electron microscope classification model for classification processing to obtain the second classification result, and the second classification result is fed back to the client. Since the magnification of the transmission electron microscope is much higher than that of the optical microscope, the second cell image including the ultrastructure of the biological cell can be collected by the transmission electron microscope, so that the biological cell can be accurately classified according to the second cell image, and the classification accuracy is improved. In addition, the classification processing by the classification model improves the classification efficiency, and can avoid classification errors due to insufficient experience when manually classifying cells, further improving the classification accuracy.

[0106] In one embodiment, the acquisition module 1002 is further configured to:

[0107] acquire a first low-magnification cell image collected by the optical microscope at a low magnification;

[0108] According to the first low-magnification cell image, move the lens of the optical microscope above the target region of the first cell image;

[0109] acquire a first high-magnification cell image collected by the optical microscope at a high magnification on the target region, and take the first high-magnification cell image as the first cell image.

[0110] In one embodiment, the acquisition module 1002 is further configured to:

[0111] If the first classification result meets the preset condition, acquire a second low-magnification cell image collected by the transmission electron microscope at a low magnification;

[0112] According to the second low-magnification cell image, move the lens of the transmission electron microscope above the target region of the second cell image;

[0113] obtaining a second high-magnification cell image collected by a transmission electron microscope at a high magnification on the target region, and taking the second high-magnification cell image as a second cell image.

[0114] In one embodiment, as shown in Figure 11 The apparatus further includes:

[0115] The obtaining module 1002 is further configured to obtain a cell image sample collected by the transmission electron microscope, and to form the training data set from the cell image sample.

[0116] The adjusting module 1008 is configured to adjust parameters of the pre-trained learning model according to the training data set, and to take the learning model after the parameter adjustment as the electron microscope classification model.

[0117] In one embodiment, the apparatus further includes:

[0118] The adding module 1010 is configured to add the second cell image to the training data set to obtain a new training data set.

[0119] The adjusting module 1008 is further configured to adjust parameters of the second pre-trained learning model according to the new training data set.

[0120] In one embodiment, the apparatus further includes:

[0121] The clustering module 1012 is configured to input the second cell image into a clustering model for clustering processing when the second classification result includes ultrastructures of other cells other than the cell types in the classification list, to obtain a clustering result.

[0122] The feedback module 1006 is configured to feed back the clustering result to the client.

[0123] The above various modules in the biological cell image classification apparatus can be all or partially implemented by software, hardware, and combinations thereof. The above various modules can be embedded in or independent of a processor in a computer device in a hardware form, or can be stored in a memory in the computer device in a software form, so as to be called and executed by a processor to perform operations corresponding to the above various modules.

[0124] In one embodiment, a computer device is provided, which can be a server, and an internal structure diagram of the computer device can be as shown in Figure 12As shown in the figure. The computer device includes a processor, a memory, an input / output interface (I / O for short) and a communication interface. Among them, the processor, the memory and the input / output interface are connected through the system bus, and the communication interface is connected to the system bus through the input / output interface. Among them, the processor of the computer device is used to provide computing and control capability. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store biological cell image classification data. The input / output interface of the computer device is used to exchange information between the processor and external devices. The communication interface of the computer device is used to communicate with external terminals through network connection. The computer program is executed by the processor to realize a biological cell image classification method.

[0125] Those skilled in the art can understand that, Figure 12 The structure shown in the figure is only a block diagram of part of the structure related to the scheme of the present application, and does not constitute a limitation on the computer device to which the scheme of the present application is applied. The specific computer device can include more or fewer components than those shown in the figure, or combine certain components, or have a different component arrangement.

[0126] In one embodiment, a computer device is provided, comprising a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the following steps: acquiring a first cell image collected by an optical microscope; inputting the first cell image into an optical microscope classification model for classification processing to obtain a first classification result; if the first classification result meets a preset condition, acquiring a second cell image collected by a transmission electron microscope; inputting the second cell image into an electron microscope classification model for classification processing to obtain a second classification result, and feeding back the second classification result to a client.

[0127] In one embodiment, the processor executing the computer program further implements the following steps: acquiring a first low magnification cell image collected by the optical microscope at a low magnification; moving the lens of the optical microscope to above a target region of the first cell image according to the first low magnification cell image; acquiring a first high magnification cell image obtained by collecting the target region by the optical microscope at a high magnification, and taking the first high magnification cell image as the first cell image.

[0128] In one embodiment, the processor, when executing the computer program, further implements the following steps: if the first classification result meets a preset condition, obtaining a second low-magnification cell image collected by the transmission electron microscope at a low magnification; moving a lens of the transmission electron microscope to above a target region of the second cell image according to the second low-magnification cell image; obtaining a second high-magnification cell image of the target region collected by the transmission electron microscope at a high magnification, and taking the second high-magnification cell image as the second cell image.

[0129] In one embodiment, the processor, when executing the computer program, further implements the following steps: obtaining a cell image sample collected by the transmission electron microscope, and composing a training data set from the cell image sample; adjusting parameters of the pre-trained learning model according to the training data set; and taking the learning model after parameter adjustment as the electron microscope classification model.

[0130] In one embodiment, the processor, when executing the computer program, further implements the following steps: adding the second cell image to the training data set to obtain a new training data set; and adjusting parameters of the pre-trained second learning model according to the new training data set.

[0131] In one embodiment, the processor, when executing the computer program, further implements the following steps: when the second classification result includes ultrastructures of cells other than the cell types in the classification list, inputting the second cell image into the clustering model for clustering processing to obtain a clustering result; and feeding back the clustering result to the client.

[0132] In one embodiment, a computer readable storage medium is provided, and the computer readable storage medium stores a computer program. The computer program, when executed by a processor, implements the following steps: obtaining a first cell image collected by an optical microscope; inputting the first cell image into an optical microscope classification model for classification processing to obtain a first classification result; if the first classification result meets a preset condition, obtaining a second cell image collected by a transmission electron microscope; inputting the second cell image into an electron microscope classification model for classification processing to obtain a second classification result, and feeding back the second classification result to a client.

[0133] In one embodiment, the computer program, when executed by the processor, further implements the following steps: obtaining a first low-magnification cell image collected by the optical microscope at a low magnification; moving a lens of the optical microscope to above a target region of the first cell image according to the first low-magnification cell image; obtaining a first high-magnification cell image of the target region collected by the optical microscope at a high magnification, and taking the first high-magnification cell image as the first cell image.

[0134] In an embodiment, the computer program, when executed by the processor, further implements the following steps: if the first classification result meets a preset condition, obtaining a second low-magnification cell image collected by the transmission electron microscope at a low magnification; moving a lens of the transmission electron microscope to above a target region of the second cell image according to the second low-magnification cell image; obtaining a second high-magnification cell image of the target region collected by the transmission electron microscope at a high magnification, and taking the second high-magnification cell image as the second cell image.

[0135] In an embodiment, the computer program, when executed by the processor, further implements the following steps: obtaining a cell image sample collected by the transmission electron microscope, and composing a training data set from the cell image sample; adjusting parameters of the pre-trained learning model according to the training data set; and taking the learning model after parameter adjustment as the electron microscope classification model.

[0136] In an embodiment, the computer program, when executed by the processor, further implements the following steps: adding the second cell image to the training data set to obtain a new training data set; and adjusting parameters of the pre-trained second learning model according to the new training data set.

[0137] In an embodiment, the computer program, when executed by the processor, further implements the following steps: when the second classification result includes ultrastructures of cells other than the cell types in the classification list, inputting the second cell image into the clustering model for clustering processing to obtain a clustering result; and feeding back the clustering result to the client.

[0138] In an embodiment, a computer program product is provided, comprising a computer program which, when executed by a processor, implements the following steps: obtaining a first cell image collected by an optical microscope; inputting the first cell image into an optical microscope classification model for classification processing to obtain a first classification result; if the first classification result meets a preset condition, obtaining a second cell image collected by a transmission electron microscope; inputting the second cell image into an electron microscope classification model for classification processing to obtain a second classification result, and feeding back the second classification result to a client.

[0139] In an embodiment, the computer program, when executed by the processor, further implements the following steps: obtaining a first low-magnification cell image collected by the optical microscope at a low magnification; moving a lens of the optical microscope to above a target region of the first cell image according to the first low-magnification cell image; obtaining a first high-magnification cell image of the target region collected by the optical microscope at a high magnification, and taking the first high-magnification cell image as the first cell image.

[0140] In an embodiment, the computer program, when executed by the processor, further implements the following steps: if the first classification result meets a preset condition, obtaining a second low-magnification cell image collected by the transmission electron microscope at a low magnification; moving a lens of the transmission electron microscope to a target region of the second cell image according to the second low-magnification cell image; obtaining a second high-magnification cell image of the target region collected by the transmission electron microscope at a high magnification, and taking the second high-magnification cell image as the second cell image.

[0141] In an embodiment, the computer program, when executed by the processor, further implements the following steps: obtaining a cell image sample collected by the transmission electron microscope, and composing a training data set from the cell image sample; adjusting parameters of the pre-trained learning model according to the training data set; and taking the learning model after parameter adjustment as the electron microscope classification model.

[0142] In an embodiment, the computer program, when executed by the processor, further implements the following steps: adding the second cell image to the training data set to obtain a new training data set; and adjusting parameters of the pre-trained second learning model according to the new training data set.

[0143] In an embodiment, the computer program, when executed by the processor, further implements the following steps: when the second classification result includes ultrastructures of other cells other than the cell types in the classification list, inputting the second cell image into the clustering model for clustering processing to obtain a clustering result; and feeding back the clustering result to the client.

[0144] It should be noted that the user information (including but not limited to user equipment information, user personal information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) involved in the present application are all information and data authorized by the user or authorized by all parties, and the collection, use and processing of related data need to comply with relevant national and regional laws, regulations and standards.

[0145] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a non-volatile computer readable storage medium, and when the computer program is executed, the processes of the above-mentioned embodiments of the methods can be included. Any reference to memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (Read-Only Memory, ROM), magnetic tape, floppy disk, flash memory, optical storage, high-density embedded non-volatile memory, resistive memory (ReRAM), magnetoresistive random access memory (Magnetoresistive Random Access Memory, MRAM), ferroelectric memory (Ferroelectric Random Access Memory, FRAM), phase change memory (Phase Change Memory, PCM), graphene memory, etc. Volatile memory can include random access memory (Random Access Memory, RAM) or external cache memory, etc. As an illustration but not limitation, RAM can be in various forms, such as static random access memory (Static Random Access Memory, SRAM) or dynamic random access memory (Dynamic Random Access Memory, DRAM), etc. The database involved in the embodiments provided in the present application can include at least one of a relational database and a non-relational database. The non-relational database can include a distributed database based on a block chain, etc., without being limited thereto. The processor involved in the embodiments provided in the present application can be a general-purpose processor, a central processing unit, a graphics processing unit, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., without being limited thereto.

[0146] Any combination of the technical features of the above embodiments can be made. In order to make the description simple, all possible combinations of the technical features in the above embodiments are not described, however, as long as the combination of the technical features does not exist contradictory, it should be considered as the scope of the present application.

[0147] The above embodiments only express several implementation manners of the present application, and the description is more specific and detailed, but it should not be understood as a limitation on the scope of the patent of the present application. It should be pointed out that for ordinary skilled in the art, without departing from the concept of the present application, a number of modifications and improvements can be made, which are within the scope of protection of the present application. Therefore, the protection scope of the present application should be subject to the appended claims.

Claims

1. A biological cell image classification method, characterized by, The method comprises: acquiring a first cell image collected by an optical microscope on a test smear of a biological specimen, wherein the test smear is obtained by subjecting the biological specimen to specimen classification, cell counting, centrifugation and staining treatment; inputting the first cell image into an optical microscope classification model for classification processing to obtain a first classification result, the first classification result indicating the category of biological cells in the first cell image and the probability value corresponding to the category, the optical microscope classification model being a machine learning model trained by a supervised learning algorithm and used for cell classification according to a cell image collected by an optical microscope; if the first classification result meets the condition for cell classification by a transmission electron microscope, acquiring a second cell image collected by the transmission electron microscope on a sample slice of the biological specimen, wherein the first classification result meeting the condition for cell classification by the transmission electron microscope includes that the probability value corresponding to suspected tumor cells or abnormal cells in the first classification result exceeds a preset threshold, and the sample slice is obtained by subjecting the biological specimen to fixation, dehydration, embedding, slicing and staining treatment, wherein the fixation treatment includes using glutaraldehyde solution and phosphate buffer, the dehydration treatment includes using gradient concentration ethanol and acetone, and the embedding treatment includes using a low-viscosity resin embedding agent; acquiring a cell image sample collected by the transmission electron microscope on the biological specimen, and forming a training data set from the cell image sample, adding the second cell image to the training data set to obtain a new training data set, and adjusting the parameters of a pre-trained learning model according to the new training data set to obtain an electron microscope classification model; inputting the second cell image into the electron microscope classification model for classification processing to obtain a second classification result, and feeding back the second classification result to a client, the second classification result indicating the category of biological cells in the second cell image and the probability value corresponding to the category, the electron microscope classification model being a machine learning model trained by a supervised learning algorithm and used for cell classification according to a cell image collected by a transmission electron microscope; when the second classification result indicates that the second cell image includes ultrastructure of other types of cells other than known cell types in a classification list, inputting the second cell image into a clustering model for clustering processing to obtain a clustering result, the clustering model being a machine learning model trained by an unsupervised learning algorithm; feeding back the clustering result to the client.

2. The method of claim 1, wherein, The acquiring of the first cell image collected by the optical microscope on the test smear of the biological specimen comprises: acquiring a first low-magnification cell image collected by the optical microscope on the test smear of the biological specimen at a low magnification; moving the lens of the optical microscope to above a target area of the first cell image according to the first low-magnification cell image; acquire a first high-magnification cell image collected by the optical microscope at a high magnification on the target region, and take the first high-magnification cell image as the first cell image.

3. The method of claim 1, wherein, If the first classification result meets the condition for cell classification by the transmission electron microscope, the acquiring a second cell image collected by the transmission electron microscope on a sample slice of the biological specimen includes: If the first classification result meets the preset condition, acquiring a second low-magnification cell image collected by the transmission electron microscope at a low magnification on the sample slice of the biological specimen; According to the second low-magnification cell image, moving the lens of the transmission electron microscope above a target region of the second cell image; acquiring a second high-magnification cell image collected by the transmission electron microscope at a high magnification on the target region, and taking the second high-magnification cell image as the second cell image.

4. A biological cell image classification apparatus characterized by comprising: The device includes: An acquisition module is configured to acquire a first cell image collected by an optical microscope on a test smear of a biological specimen, wherein the test smear is obtained by subjecting the biological specimen to specimen classification, cell counting, centrifugation, and staining treatment. A classification module is configured to input the first cell image into an optical microscope classification model for classification processing to obtain a first classification result, wherein the first classification result indicates the category of biological cells in the first cell image and the probability value corresponding to the category, and the optical microscope classification model is a machine learning model trained by a supervised learning algorithm and is used for cell classification based on a cell image collected by an optical microscope. The acquisition module is further configured to acquire a second cell image collected by a transmission electron microscope on a sample slice of a biological specimen if the first classification result meets the condition for cell classification by the transmission electron microscope, wherein the condition for the first classification result to meet the cell classification by the transmission electron microscope includes that the probability value corresponding to suspected tumor cells or abnormal cells in the first classification result exceeds a preset threshold, and the sample slice is obtained by subjecting the biological specimen to fixation, dehydration, embedding, slicing, and staining treatment, wherein the fixation treatment includes using glutaraldehyde solution and phosphate buffer, the dehydration treatment includes using gradient concentration ethanol and acetone, and the embedding treatment includes using a low-viscosity resin embedding agent. A training module is configured to acquire a cell image sample collected by the transmission electron microscope on a biological specimen, and to form a training data set from the cell image sample, add the second cell image to the training data set to obtain a new training data set, and adjust the parameters of a pre-trained learning model according to the new training data set to obtain an electron microscope classification model. The feedback module is configured to input the second cell image into the electron microscope classification model for classification processing to obtain a second classification result, and feed back the second classification result to the client, the second classification result indicating a category of biological cells in the second cell image and a probability value corresponding to the category, the electron microscope classification model being a machine learning model trained by a supervised learning algorithm and used for cell classification according to a cell image collected by a transmission electron microscope. The feedback module is further configured to, when the second classification result indicates that the second cell image includes ultrastructures of other types of cells except for known cell types in a classification list, input the second cell image into a clustering model for clustering processing to obtain a clustering result, the clustering model being a machine learning model trained by an unsupervised learning algorithm, and feed back the clustering result to the client.

5. The apparatus of claim 4, wherein, The acquisition module is further configured to acquire a first low-magnification cell image of a test smear of a biological specimen collected by an optical microscope at a low magnification; move a lens of the optical microscope to above a target region of the first cell image according to the first low-magnification cell image; acquire a first high-magnification cell image of the target region collected by the optical microscope at a high magnification, and use the first high-magnification cell image as the first cell image.

6. The apparatus of claim 4, wherein, The acquisition module is further configured to, if the first classification result meets a preset condition, acquire a second low-magnification cell image of a sample slice of a biological specimen collected by a transmission electron microscope at a low magnification; move a lens of the transmission electron microscope to above a target region of the second cell image according to the second low-magnification cell image; acquire a second high-magnification cell image of the target region collected by the transmission electron microscope at a high magnification, and use the second high-magnification cell image as the second cell image. 7.A computer device, comprising a memory and a processor, wherein the memory stores a computer program, and the computer device is configured to perform the method according to any one of claims 1-6 when the computer program is executed by the processor. The processor implements the steps of the method of any one of claims 1 to 3 when executing the computer program.

8. A computer-readable storage medium having stored thereon a computer program, characterized in that, The computer program, when executed by the processor, implements the steps of the method of any one of claims 1 to 3.

9. A computer program product comprising a computer program, characterized in that, The computer program, when executed by the processor, implements the steps of the method of any one of claims 1 to 3.

Images