Method, device, and computer program for assisting in disease analysis, and method, device, and program for training a computer algorithm

Abstract

To allow for identifying diseases based on individual cell images.SOLUTION: In order to clear the above challenge, a method of assisting disease analysis is provided, comprising: classifying morphology of multiple analysis target cells using a captured image of the multiple analysis target cells contained in a sample collected from a subject; acquiring information on cell morphology classification corresponding to the sample based on the classification result; and analyzing a disease in the subject on the basis of the information on cell morphology classification using a computer algorithm.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to a method, an apparatus and a computer program for assisting disease analysis, and to a method, an apparatus and a program for training a computer algorithm for assisting disease analysis. [Background technology]

[0002] Patent Document 1 describes a method for differentiating pathological tissue into "scirrhous carcinoma pattern" and "intraductal fibrous tumor pattern" by inputting into a neural network the number, area, shape, circularity, color, and chromaticity of nuclear regions, the number, area, shape, circularity of void regions, the number, area, shape, circularity, color, chromaticity of interstitial regions, the number, area, shape, circularity, image texture, features calculated using wavelet transform values, the degree to which epithelial cells exhibit a two-layer structure accompanied by myoepithelial cells, the degree of fibrosis, the presence or absence of a papillary pattern, the presence or absence of a cribriform pattern, the presence or absence of necrotic material, the presence or absence of a solid pattern, and features calculated using the color or chromaticity of the image, all of which are extracted from a tissue image. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 10-197522 Summary of the Invention [Problem to be solved by the invention]

[0004] However, although Patent Document 1 discloses that a disease is identified based on an image of a tissue, it does not disclose that a disease is identified based on an image of an individual cell.

[0005] An object of the present invention is to accurately identify diseases from individual cell images. [Means for solving the problem]

[0006] The present invention relates to a method for supporting disease analysis, which classifies the morphology of a plurality of cells to be analyzed from images of the cells to be analyzed contained in a sample taken from a subject, obtains information on a cell morphology classification corresponding to the sample based on the classification result, and analyzes a disease possessed by the subject by a computer algorithm based on the information on the cell morphology classification. With these configurations, the disease can be identified based on the individual cell images.

[0007] Preferably, in the step of classifying the morphology of each of the analysis targets, the type of cell of each of the analysis targets is identified. More preferably, the information on the cell morphology classification is information on the number of cells of each cell type (64). According to these configurations, it is possible to identify diseases based on the type of each individual cell.

[0008] Preferably, in the step of classifying the morphology of each of the cells to be analyzed, the type of abnormality in each of the cells to be analyzed is identified. More preferably, the information on the cell morphology classification is information on the number of cells for each type of abnormality (63). According to these configurations, it is possible to identify a disease based on the type of abnormality in each cell.

[0009] In the step of classifying the morphology of each of the analysis target cells, the type of abnormal findings is identified for each of the analysis target cell types. , diseases can be identified.

[0010] In the step of classifying the morphology of the cells of each of the analysis targets, analysis data including information on the cells of each of the analysis targets is input to a deep learning algorithm having a neural network structure, and the deep learning algorithm classifies the morphology of the cells of each of the analysis targets. With these configurations, it is possible to more accurately identify diseases.

[0011] The computer algorithm is a machine learning algorithm, and the analysis of the disease of the subject is performed by inputting the information on the cell morphology classification as a feature into the machine learning algorithm (67). With these configurations, it is possible to more accurately identify diseases.

[0012] The machine learning algorithm (67) is preferably selected from trees, regression, neural networks, Bayesian, clustering, or ensemble learning. More preferably, the machine learning algorithm is gradient boosting trees. Using these machine learning algorithms, diseases can be identified more accurately.

[0013] In the step of acquiring information on the cell morphology classification, the probability that each of the cells to be analyzed belongs to each of a plurality of cell morphology classifications is calculated, the sum of the probabilities for each type of cell morphology classification is calculated, and the sum is acquired as information on the cell morphology classification. With these configurations, it is possible to identify diseases with higher accuracy.

[0014] Preferably, the sample is a blood sample, since cells in blood reflect the pathological conditions of various diseases, it is possible to identify diseases with higher accuracy.

[0015] Preferably, the disease is a hematopoietic disease. According to the present invention, hematopoietic diseases can be accurately identified.

[0016] According to the present invention, the hematopoietic disease is aplastic anemia or myelodysplastic syndrome.

[0017] Preferably, the abnormal findings are at least one selected from the group consisting of nuclear morphological abnormalities, granular abnormalities, cell size abnormalities, cell malformations, cell destruction, empty cells, immature cells, the presence of inclusion bodies, Dohle bodies, satellite phenomena, nuclear reticulum abnormalities, petal-like nuclei, large N / C ratios, bleb-like, smudge and hairy cell-like morphologies. By evaluating these abnormal findings in these cells, it is possible to identify diseases with higher accuracy.

[0018] Preferably, the nuclear morphological abnormality includes at least one selected from hyper-lobulation, hypo-lobulation, pseudo-Pelger's anomaly, ring-shaped nucleus, spherical nucleus, oval nucleus, apoptosis, multinucleation, karyorrhexis, enucleation, bare nucleus, irregular nuclear margin, nuclear fragmentation, internuclear bridge, multiple nuclei, cleaved nucleus, nuclear division, and nucleolar abnormality. The granular abnormality includes at least one selected from degranulation, granular distribution abnormality, toxic granules, Auer rods, Bassoon cells, and pseudo-Chediak-Higashi granule-like granules. The cell size abnormality includes giant platelets. By evaluating these abnormal findings, more accurate identification of diseases is possible.

[0019] Preferably, the types of cells include at least one selected from neutrophils, eosinophils, platelets, lymphocytes, monocytes, and basophils. By evaluating these types of cells, diseases can be identified with higher accuracy.

[0020] More preferably, the cell types are further selected from the group consisting of metamyelocytes, myelocytes, promyelocytes, blasts, and cytoplasmic cells. The types of cells included in the endothelial cell population include at least one selected from the group consisting of stromal cells, atypical lymphocytes, immature eosinophils, immature basophils, erythroblasts, and megakaryocytes. By evaluating the types of cells among these cells, it is possible to more accurately identify diseases.

[0021] The present invention relates to an apparatus (200) for supporting disease analysis. The apparatus (200) includes a processing unit (20), which classifies the morphology of a plurality of cells to be analyzed from images obtained from the cells to be analyzed contained in a sample taken from a subject, obtains information on a cell morphology classification corresponding to the sample based on the classification result, and analyzes a disease possessed by the subject by a computer algorithm based on the information on the cell morphology classification.

[0022] The present invention relates to a program for assisting disease analysis, which causes a computer to execute the steps of: classifying the morphology of multiple analysis target cells contained in a sample taken from a subject from images obtained from the cells, and obtaining information regarding a cell morphology classification corresponding to the sample based on the classification results; and analyzing a disease possessed by the subject using a computer algorithm based on the information regarding the cell morphology classification.

[0023] A device or program for assisting disease analysis enables highly accurate disease identification.

[0024] The present invention relates to a method for training a computer algorithm for assisting disease analysis, the method comprising: classifying the morphology of a plurality of analysis target cells contained in a sample taken from a subject from images of the cells, acquiring information on a cell morphology classification corresponding to the sample based on the classification result, and inputting the acquired information on the cell morphology classification as first training data and disease information of the subject as second training data into the computer algorithm.

[0025] The present invention relates to a training device (100) for a computer algorithm for assisting disease analysis. The training device (100) includes a processing unit (10), which classifies the morphology of a plurality of cells to be analyzed from an image obtained from the cells to be analyzed contained in a sample taken from a subject, obtains information on a cell morphology classification corresponding to the sample based on the classification result, and inputs the obtained information on the cell morphology classification as first training data and disease information (55) of the subject as second training data to the computer algorithm.

[0026] The present invention relates to a training program for a computer algorithm for supporting disease analysis, which causes a computer to execute the steps of: classifying the morphology of a plurality of analysis target cells contained in a sample taken from a subject from images obtained from the cells, acquiring information on the cell morphology classification corresponding to the sample based on the classification results, and inputting the information on the cell morphology classification as first training data and disease information (55) of the subject as second training data into a computer algorithm.

[0027] The training method, training device (100) or training program allows for accurate disease identification. Effect of the Invention

[0028] According to the present invention, diseases can be identified with high accuracy from individual cell images. [Brief description of the drawings]

[0029] [Figure 1] FIG. 1 is a diagram illustrating an overview of a support method using a classifier. [Diagram 2] FIG. 1 is a schematic diagram showing an example of a procedure for generating training data for deep learning and a procedure for training a first deep learning algorithm, a second deep learning algorithm, and a machine learning algorithm. [Diagram 3] Examples of label values ​​are as follows: [Figure 4] 1 shows an example of training data for machine learning. [Diagram 5] FIG. 1 is a schematic diagram showing an example of a procedure for generating analysis data and a procedure for disease analysis using a computer algorithm. [Figure 6] 1 is a diagram showing an outline of a configuration example of a disease analysis system 1. FIG. [Figure 7] 2 is a block diagram showing an example of a hardware configuration of a vendor-side device 100. FIG. [Figure 8] FIG. 2 is a block diagram showing an example of a hardware configuration of a user device 200. [Figure 9] FIG. 2 is a block diagram for explaining an example of the function of a training device 100A. [Figure 10] 1 is a flowchart showing an example of the flow of a deep learning process. [Figure 11] FIG. 1 is a schematic diagram for explaining a neural network. [Figure 12] FIG. 2 is a block diagram for explaining an example of the functions of the machine learning device 100A. [Figure 13] 11 is a flowchart showing an example of the flow of a machine learning process using first information. [Figure 14] 11 is a flowchart showing an example of the flow of a machine learning process using the first information and the second information. [Figure 15] FIG. 2 is a block diagram for explaining an example of the functions of a disease analysis apparatus 200A. [Figure 16] 13 is a flowchart showing an example of the flow of a disease analysis process using first information. [Figure 17] 13 is a flowchart showing an example of the flow of a disease analysis process using the first information and the second information. [Figure 18] 1 is a diagram showing an outline of a configuration example of a disease analysis system 2. FIG. [Figure 19] FIG. 2 is a block diagram for explaining an example of functions of an integrated disease-analysis apparatus 200B. [Figure 20] FIG. 2 is a diagram showing an outline of a configuration example of a disease analysis system 3. [Figure 21] FIG. 2 is a block diagram for explaining an example of the functions of an integrated image analyzing device 100B. [Figure 22] FIG. 2 is a diagram showing the structure of a classifier used in the examples. [Figure 23] This is a table showing the number of cells used as training data for the deep learning algorithm and the number of cells used for validation to evaluate the performance of the trained deep learning algorithm. [Figure 24] 13 is a table showing the results of evaluating the performance of the trained second deep learning algorithm. [Diagram 25] 1 is a table showing the results of evaluating the performance of the first trained deep learning algorithm. [Figure 26] This is a heat map of abnormal findings that contribute to disease analysis. [Figure 27] FIG. 1 is a diagram showing an ROC curve of disease analysis results. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0030] Hereinafter, the embodiments of the present invention will be described in detail with reference to the accompanying drawings. In the following description and drawings, the same reference numerals denote the same or similar components, and therefore, the description of the same or similar components will be omitted.

[0031] A method for assisting in the analysis of a disease possessed by a subject (hereinafter, sometimes simply referred to as the "assistance method") will be described. The assistance method includes classifying the morphology of each analysis target cell, and analyzing the disease possessed by the subject based on the classification result. From an image obtained from a plurality of analysis target cells contained in a sample taken from the subject, the morphology of each analysis target cell is classified, and information on the cell morphology classification corresponding to the sample is obtained based on the classification result. The assistance method includes analyzing the disease possessed by the subject based on information on the type of abnormal findings (hereinafter, sometimes referred to as "first information") as information on the cell morphology classification. The first information is information on the abnormal findings detected in each of the plurality of analysis target cells contained in the sample. The second information is information on the type of the abnormal finding corresponding to the sample, obtained based on the type of the cell. The abnormal finding is identified from an image of the cells to be analyzed. The support method also includes analyzing a disease possessed by the subject based on information on the type of cell (hereinafter sometimes referred to as "second information") as information on cell morphology classification. The second information is information on the type of the cell corresponding to the sample, obtained based on a plurality of types of cells to be analyzed contained in the sample. The type of cell is identified from an image of the cells to be analyzed.

[0032] The subject is not limited as long as it is an animal that can be the subject of disease analysis. Examples of the animal include humans, dogs, cats, rabbits, monkeys, etc. Preferably, the subject is a human.

[0033] The disease is not limited to those that the animal suffers from, and may include, for example, tumors in tissues other than the hematopoietic system, hematopoietic diseases, metabolic diseases, renal diseases, infectious diseases, allergic diseases, autoimmune diseases, trauma, and the like.

[0034] Tumors of tissues other than the hematopoietic system may include benign epithelial tumors, benign non-epithelial tumors, malignant epithelial tumors, and malignant non-epithelial tumors. Tumors of tissues other than the hematopoietic system preferably include malignant epithelial tumors and malignant non-epithelial tumors.

[0035] Examples of hematopoietic diseases include tumors, anemia, polycythemia, platelet abnormalities, myelofibrosis, etc. Examples of hematopoietic tumors include myelodysplastic syndrome, leukemia (acute myeloblastic leukemia, acute myeloblastic leukemia (with neutrophil differentiation), acute promyelocytic leukemia, acute myelomonocytic leukemia, acute monocytic leukemia, erythroleukemia, acute megakaryoblastic leukemia, acute myelogenous leukemia, acute lymphocytic leukemia, lymphoblastic leukemia, chronic myelogenous leukemia, chronic lymphocytic leukemia, etc.), malignant lymphoma (Hodgkin's lymphoma, non-Hodgkin's lymphoma, etc.), multiple myeloma, granuloma, etc. Examples of hematopoietic malignant tumors include myelodysplastic syndrome, leukemia, and multiple myeloma, and more preferably myelodysplastic syndrome.

[0036] Examples of anemia include aplastic anemia, iron deficiency anemia, megaloblastic anemia (including vitamin B12 deficiency, folic acid deficiency, etc.), hemorrhagic anemia, renal anemia, hemolytic anemia, thalassemia, sideroblastic anemia, atransferrinemia, etc. Examples of anemia include aplastic anemia, pernicious anemia, iron deficiency anemia, and sideroblastic anemia, and more preferably aplastic anemia.

[0037] Polycythemia may include polycythemia vera and secondary polycythemia. As polycythemia, polycythemia vera is preferable.

[0038] Platelet abnormalities may include thrombocytosis, thrombocytosis, and megakaryocytic abnormalities. Thrombocytosis may include disseminated intravascular coagulation, idiopathic thrombocytopenic purpura, MYH9 abnormalities, Bernard-Soulier syndrome, and the like. Thrombocytosis may include essential thrombocythemia. Megakaryocytic abnormalities may include micromegakaryocytes, multinucleated megakaryocytes, platelet hypoplasia, and the like.

[0039] Myelofibrosis can include primary myelofibrosis and secondary myelofibrosis.

[0040] The metabolic disease may include carbohydrate metabolism disorder, lipid metabolism disorder, electrolyte disorder, metal metabolism disorder, etc. The carbohydrate metabolism disorder may include mucopolysaccharidosis, diabetes, and glycogen storage disease. As the carbohydrate metabolism disorder, mucopolysaccharidosis and diabetes are preferable. The lipid metabolism disorder may include Gaucher disease, Niemann-Pick disease, hyperlipidemia, and arteriosclerotic disease. The arteriosclerotic disease may include arteriosclerosis, atherosclerosis, thrombosis, embolism, etc. The electrolyte disorder may include hyperkalemia, hypokalemia, hypernatremia, hyponatremia, etc. The metal metabolism disorder may include iron metabolism disorder, copper metabolism disorder, and the like. These may include metabolic disorders, calcium metabolic disorders, and inorganic phosphorus metabolic disorders.

[0041] Renal disorders may include nephrotic syndrome, decreased renal function, acute renal failure, chronic renal disease, renal failure, and the like.

[0042] Infectious diseases may include bacterial, viral, rickettsial, chlamydial, fungal, protozoal, and parasitic infections.

[0043] The causative bacteria of bacterial infections are not particularly limited, and examples of the causative bacteria include coliform bacteria, staphylococci, streptococci, Haemophilus, Neisseria, Moraxella, Listeria, Corynebacterium diphtheriae, Corynebacterium truncatum, Helicobacter, and Mycobacterium tuberculosis.

[0044] There is no limitation on the causative virus of the viral infection, and examples of the causative virus include influenza virus, measles virus, rubella virus, chickenpox virus, dengue fever virus, cytomegalovirus, EB virus, enterovirus, human immunodeficiency virus, HTLV-1 (human T-lymphotropic virus type-I), and rabies virus.

[0045] The causative fungus of the fungal infection is not particularly limited. The causative fungus may include yeast-like fungi, filamentous fungi, etc. The yeast-like fungus may include fungi of the genus Cryptococcus, fungi of the genus Candida, etc. The filamentous fungus may include fungi of the genus Aspergillus, etc.

[0046] The causative agent of the protozoan infection is not particularly limited, and may include malaria, kala azar, and the like.

[0047] Parasitic infections can include roundworms, nematodes, hookworms, and the like.

[0048] The infectious disease is preferably a bacterial infection, a viral infection, a protozoan infection, or a parasitic infection, more preferably a bacterial infection, and may include pneumonia, sepsis, meningitis, or a urinary tract infection.

[0049] The allergic disease may include allergic diseases belonging to type I, type II, type III, type IV, or type V. The allergic disease belonging to type I may include hay fever, anaphylactic shock, allergic rhinitis, conjunctivitis, bronchial asthma, urticaria, atopic dermatitis, etc. The allergic disease belonging to type II may include immune incompatible transfusion, autoimmune hemolytic anemia, autoimmune thrombocytopenia, autoimmune granulocytopenia, Hashimoto's disease, Good-Bascher's syndrome, etc. The allergic disease belonging to type III may include immune complex nephritis, Arthus reaction, serum sickness, etc. The allergic disease belonging to type IV may include tuberculosis, contact dermatitis, etc. The allergic disease belonging to type V may include Graves' disease, etc. The allergic disease is preferably type I, type II, type III, and type IV, more preferably type I, type II, and type III, and even more preferably type I. Allergic diseases belonging to types II, III, and V overlap in part with the autoimmune diseases described below.

[0050] The autoimmune disease may include systemic lupus erythematosus, rheumatoid arthritis, multiple sclerosis, Sjogren's syndrome, scleroderma, dermatomyositis, primary biliary cirrhosis, primary sclerosing cholangitis, ulcerative colitis, Crohn's disease, psoriasis, vitiligo vulgaris, bullous pemphigoid, alopecia areata, idiopathic dilated cardiomyopathy, type 1 diabetes, Graves' disease, Hashimoto's disease, myasthenia gravis, IgA nephropathy, membranous nephropathy, megaloblastic anemia, etc. Preferred examples of the autoimmune disease include systemic lupus erythematosus, rheumatoid arthritis, multiple sclerosis, Sjogren's syndrome, scleroderma, and dermatomyositis. In addition, the autoimmune disease may include inflammatory bowel disease, inflammatory bowel disease, and inflammatory bowel disease. The infectious disease is preferably an autoimmune disease in which antinuclear antibodies are detected.

[0051] Trauma can include bone fractures, burns, and the like.

[0052] The sample is not limited as long as it can be collected from a subject. Preferred samples are blood, bone marrow, urine, and body fluids. Examples of blood include peripheral blood, venous blood, and arterial blood. The blood is preferably peripheral blood. Examples of blood include peripheral blood collected using an anticoagulant such as ethylenediaminetetraacetate (sodium salt or potassium salt) and sodium heparin. The body fluid is other than blood and urine. Examples of such body fluids include ascites, pleural effusion, and cerebrospinal fluid.

[0053] The sample may be selected depending on the disease to be analyzed. In particular, in the above-mentioned diseases, cells in blood often have characteristics different from normal, such as the numerical distribution of cell types and / or the types of abnormal findings described below. Therefore, it is possible to analyze various diseases using blood samples. In addition, bone marrow can be used to analyze hematopoietic diseases in particular. Cells contained in ascites, pleural effusion, cerebrospinal fluid, etc. are particularly effective in diagnosing tumors in tissues other than the hematopoietic system, hematopoietic diseases, infectious diseases, etc. Urine can be used to analyze tumors in tissues other than the hematopoietic system, infectious diseases, etc.

[0054] The cells to be analyzed are not limited as long as they are contained in the sample. The cells to be analyzed are intended to be cells used to analyze a disease. The cells to be analyzed may include a plurality of cells. Here, "a plurality" may include a case where the number of cells of one type is a plurality, or a case where there are a plurality of cell types. In a normal case, the sample may include a plurality of types of cells that are morphologically classified by histological or cytological microscopic observation. The morphological classification of cells (also called "cell morphological classification") includes the classification of cell types and the classification of types of abnormal findings of cells. Preferably, the cells to be analyzed are a group of cells belonging to a specific cell lineage. The specific cell lineage is a group of cells belonging to the same lineage that are differentiated from a certain type of tissue stem cell. The specific cell lineage is preferably the hematopoietic system, and more preferably, cells in the blood (also called "blood cells").

[0055] In the conventional method, hematopoietic cells are morphologically classified by a human observing a specimen stained for bright field in the bright field of a microscope. The stain is preferably selected from Wright's stain, Giemsa stain, Wright-Giemsa stain, and May-Giemsa stain. May-Giemsa stain is more preferable. The specimen is not limited as long as the morphology of each cell belonging to a predetermined cell group can be individually observed. Examples include smears and imprint specimens. A smear of peripheral blood or bone marrow is preferable, and a smear of peripheral blood is more preferable.

[0056] In terms of morphological classification, blood cells include cell types such as neutrophils, including segmented and band-shaped neutrophils, metamyelocytes, myelocytes, promyelocytes, blasts, lymphocytes, plasma cells, atypical lymphocytes, monocytes, eosinophils, basophils, erythroblasts (nucleated red blood cells, including proerythroblasts, basophilic erythroblasts, polychromatic erythroblasts, normochromatic erythroblasts, promegaloblasts, basophilic megaloblasts, polychromatic megaloblasts, and normochromatic megaloblasts), platelets, platelet aggregates, and megakaryocytes (nucleated megakaryocytes, including micromegakaryocytes).

[0057] Furthermore, the predetermined cell group may include abnormal cells exhibiting morphological abnormalities in addition to normal cells. The abnormality appears as a characteristic of cells classified morphologically. Examples of abnormal cells are cells that appear when a patient is affected with a predetermined disease, such as tumor cells. In the case of the hematopoietic system, the predetermined disease includes myelodysplastic syndrome, leukemia (including acute myeloblastic leukemia, acute myeloblastic leukemia, acute promyelocytic leukemia, acute myelomonocytic leukemia (with neutrophil differentiation), acute monocytic leukemia, erythroleukemia, acute megakaryoblastic leukemia, acute myeloid leukemia, acute lymphocytic leukemia, lymphoblastic leukemia, chronic myeloid leukemia, and chronic lymphocytic leukemia, etc.), The abnormality is a disease selected from the group consisting of malignant lymphoma (Hodgkin's lymphoma, non-Hodgkin's lymphoma, etc.) and multiple myeloma. In the case of the hematopoietic system, the abnormal finding is a cell having at least one morphological characteristic selected from the group consisting of abnormal nuclear morphology, presence of vacuoles, abnormal granule morphology, abnormal granule distribution, presence of abnormal granules, abnormal cell size, presence of inclusion bodies, and bare nuclei.

[0058] Nuclear morphological abnormalities include small nuclei, large nuclei, hypersegmented nuclei, nuclei that are not segmented when they should normally be segmented (including pseudo-Pelger's nuclear abnormality), vacuoles, enlarged nucleoli, cleaved nuclei, and abnormalities in which there is two nuclei in a cell when there should normally be one nucleus in a cell.

[0059] Abnormalities in the overall morphology of the cells include those with vacuoles in the cytoplasm (also known as vacuolar degeneration), those with morphological abnormalities in granules such as giant platelets, azurophilic granules, neutrophilic granules, acidophilic granules, basophilic granules, etc., those with abnormal distribution of the above granules (excess, or reduction or disappearance), those with abnormal granules (e.g., toxic granules, etc.), those with abnormal cell size (larger or smaller than normal), those with inclusion bodies (Dahley bodies, Auer bodies, etc.), and those with bare nuclei.

[0060] Preferably, the abnormal finding is at least one selected from the group consisting of nuclear morphological abnormalities, granular abnormalities, abnormal cell size, cell malformation, cell destruction, empty cells, immature cells, presence of inclusion bodies, Dohle bodies, satellite phenomenon, nuclear reticulum abnormalities, petal-like nuclei, large N / C ratio, bleb-like, smudge and hairy cell-like morphology.

[0061] Preferably, the nuclear morphological abnormality includes at least one selected from hyper-lobulation, hypo-lobulation, pseudo-Pelger's anomaly, ring-shaped nucleus, spherical nucleus, oval nucleus, apoptosis, multinucleation, karyorrhexis, enucleation, bare nucleus, irregular nuclear margin, nuclear fragmentation, internuclear bridge, multiple nuclei, cleaved nucleus, nuclear division, and nucleolar abnormality. The granular abnormality includes at least one selected from degranulation, granular distribution abnormality, toxic granules, Auer rods, Bassoon cells, and pseudo-Chediak-Higashi granule-like granules. Granular abnormalities in eosinophils and basophils include phenomena such as abnormal granules in which granules are distributed unevenly within the cells. The cell size abnormality includes giant platelets.

[0062] Preferably, the cell types include at least one selected from neutrophils, eosinophils, platelets, lymphocytes, monocytes, and basophils.

[0063] More preferably, the types of cells further include at least one selected from metamyelocytes, myelocytes, promyelocytes, blasts, plasma cells, atypical lymphocytes, immature eosinophils, immature basophils, erythroblasts, and megakaryocytes.

[0064] More preferably, the hematopoietic disease is aplastic anemia or myelodysplastic syndrome, and when the type of the cells is neutrophils, the abnormal finding is at least one selected from dysgranularity and hypersegmentation, or when the type of the cells is eosinophils, the abnormal finding is abnormal granules. Also, the abnormal finding of cells includes giant platelets. By evaluating these findings, aplastic anemia and myelodysplastic syndrome can be distinguished.

[0065] <Summary of support method> In the assistance method, there is no limitation on identifying abnormal findings and / or identifying cell types as long as the abnormal findings and / or cell types can be identified from the image. The identification may be performed by an examiner or by using a classifier described below.

[0066] The outline of the support method using a classifier will be described with reference to FIG. The classifier includes a computer algorithm. Preferably, the computer algorithm includes a first computer algorithm and a second computer algorithm. More preferably, the first computer algorithm includes a plurality of deep learning algorithms having a neural network structure. The second computer algorithm includes a machine learning algorithm. Preferably, the deep learning algorithm includes a first neural network 50 for extracting a feature amount quantitatively representing a morphological feature of a cell, a second neural network 51 for identifying a type of abnormal finding in a cell, and / or a second neural network 52 for identifying a type of cell. The first neural network 50 extracts a feature amount of a cell, and the second neural networks 51 and 52 are downstream of the first neural network and identify an abnormal finding in a cell or a type of a cell based on the feature amount extracted by the first neural network 50. More preferably, the second neural networks 51 and 52 may include a neural network trained for the purpose of identifying a type of cell, and a plurality of types of neural networks trained for each abnormal finding in a cell corresponding to each abnormal finding. For example, in FIG. 1, the first deep learning algorithm is a deep learning algorithm for identifying a first abnormal finding (e.g., granular abnormality) and includes a first neural network 50 and a second neural network 51 trained to identify the first abnormal finding. The first' deep learning algorithm is a deep learning algorithm for detecting a second abnormal finding (e.g., hypersegmentation) and includes a first neural network 50 and a second neural network 51 trained to identify the second abnormal finding. The second deep learning algorithm is a deep learning algorithm for identifying cell types and includes a first neural network 50 and a second neural network 52 trained to identify cell types.

[0067] The machine learning algorithm analyzes the disease carried by the subject from whom the sample was collected based on the features output from the deep learning algorithm for each sample, and outputs the disease name or a label indicating the disease name as the analysis result.

[0068] Next, a method for generating deep learning training data 75, machine learning training data, and a method for analyzing a disease will be described using the examples shown in Figures 2 to 4. For convenience of explanation, the following description will be given using a first neural network, a second neural network, and a gradient boosting tree, which is a machine learning algorithm.

[0069] <Generating training data for deep learning> The training images 70 used to train the deep learning algorithm are images of cells to be analyzed contained in a sample taken from a subject whose disease has already been named. A plurality of training images 70 are taken for one sample. The cells to be analyzed contained in each image are linked to the type of cells based on morphological classification and the results of abnormal findings identified by the examiner. It is preferable that the specimen for capturing the training images 70 is prepared from a specimen containing the same type of cells as the cells to be analyzed, using the same specimen preparation method and staining method as the specimen containing the cells to be analyzed. In addition, it is preferable that the training images 70 are captured under the same conditions as the imaging conditions for the cells to be analyzed.

[0070] The training image 70 can be acquired in advance for each cell using an imaging device such as a known optical microscope or a virtual slide scanner. In the example shown in FIG. 2, an automatic blood image analyzer DI-60 (manufactured by Sysmex Corporation) is used to capture an image of 360 pixels x 365 pixels. The training image 70 is generated by reducing the raw image obtained by the above process to 255 pixels by 255 pixels, but this reduction is not essential. The number of pixels in the training image 70 is not limited as long as analysis is possible, but it is preferable that each side has more than 100 pixels. In the example shown in FIG. 2, red blood cells are present around a neutrophil, but the image may be cropped so that only the target cell is included. At least one cell to be trained should be included in one image. (Red blood cells and normal-sized platelets may be included), and an image can be used as a training image 70 if the pixels corresponding to the cells to be trained are present in approximately 1 / 9 or more of the total pixels of the image.

[0071] For example, the imaging device preferably captures images in RGB color and CMY color, etc. The color image preferably represents the shade or brightness of each primary color, such as red, green, and blue, or cyan, magenta, and yellow, as a 24-bit value (8 bits x 3 colors). The training image 70 may include at least one hue and the shade or brightness of that hue, but more preferably includes at least two hues and the shade or brightness of each hue. Information including the hue and the shade or brightness of that hue is also called color tone.

[0072] Next, the color tone information of each pixel is converted, for example, from RGB color to a format including brightness information and hue information. As a format including brightness information and hue information, YUV (YCbCr, YPbPr, YIQ, etc.) can be given. Here, conversion to YCbCr format is explained as an example. Here, since the training image is RGB color, it is converted to brightness 72Y, a first hue (for example, blue-based) 72Cb, and a second hue (for example, red-based) 72Cr. Conversion from RGB to YCbCr can be performed by a known method. For example, conversion from RGB to YCbCr can be performed according to the international standard ITU-R BT.601. The converted brightness 72Y, first hue 72Cb, and second hue 72Cr can be expressed as a matrix of gradation values ​​as shown in FIG. 2 (hereinafter, also referred to as color tone matrix 72y, 72cb, 72cr). The luminance 72Y, the first hue 72Cb, and the second hue 72Cr are each expressed in 256 gradations ranging from 0 to 255. Here, instead of the luminance 72Y, the first hue 72Cb, and the second hue 72Cr, the training images may be converted using the three primary colors of red R, green G, and blue B, or the three primary colors of cyan C, magenta M, and yellow Y.

[0073] Next, based on the color tone matrices 72y, 72cb, 72cr, color tone vector data 74 is generated for each pixel by combining three gradation values, luminance 72y, first hue 72cb, and second hue 72cr.

[0074] Next, for example, since the training image 70 in Fig. 2 is a segmented neutrophil, a label value 77 of "1" indicating that the training image 70 is a segmented neutrophil is assigned to each piece of color tone vector data 74 generated from the training image 70 in Fig. 2, and the data becomes deep learning training data 75. In Fig. 2, the deep learning training data 75 is shown as 3 pixels x 3 pixels for convenience, but in reality, there is as much color tone vector data as there are pixels when the training image 70 is captured.

[0075] 3 shows an example of the label value 77. Different label values ​​77 are assigned depending on the type of cell and the presence or absence of abnormal findings in each cell.

[0076] <Overview of classifier generation> An overview of a method for generating a classifier will be described using Fig. 2 as an example. Generating a classifier may include training a deep learning algorithm and training a machine learning algorithm.

[0077] Training deep learning algorithms The first deep learning algorithm comprises a first neural network 50 and a second neural network 51 for generating first information 53, which is information about the type of abnormal finding. The second deep learning algorithm comprises a first neural network 50 and a second neural network 52 for generating second information 54, which is information about the type of cell.

[0078] The number of nodes in the input layer 50a of the first neural network 50 corresponds to the product of the number of pixels in the input deep learning training data 75 and the number of luminances and hues contained in the image (for example, in the above example, there are three: luminance 72y, first hue 72cb, and second hue 72cr). The color tone vector data 74 is input as a set 76 to the input layer 50a of the first neural network 50. The label values ​​77 of each pixel in the deep learning training data 75 are input to the output layer 50b of the first neural network to train the first neural network 50.

[0079] The first neural network 50 extracts features of the cell reflecting the above-mentioned morphological cell type and abnormal findings based on the deep learning training data 75. The output layer 50b of the first neural network outputs the results reflecting these features. The input layer 51a of the second neural network 51 and the input layer 52a of the second neural network 52 receive the results output from the softmax function of the output layer 50b of the first neural network 50. In addition, since cells belonging to a certain cell lineage have similar cell morphology, the second neural networks 51 and 52 are trained to further specialize in identifying cell features reflecting a specific morphological cell type or a specific abnormal finding. Therefore, the label values ​​77 of the deep learning training data 75 are also input to the output layers 51b and 52b of the second neural network. References 50c, 51c, and 52c in FIG. 2 indicate intermediate layers. Also, one second neural network 51 can be trained for one abnormal finding. In other words, the number of second neural networks 51 corresponding to the number of types of abnormal findings to be analyzed can be trained. There is one type of second neural network 52 for identifying the type of cell.

[0080] Preferably, the first neural network 50 is a convolution-connect neural network, and the second neural networks 51 and 52 are full-connect neural networks.

[0081] Thus, a first deep learning algorithm having the trained first neural network 60 and second neural network 61, and a second deep learning algorithm having the trained first neural network 61 and second neural network 62 are generated (see Figure 5).

[0082] For example, the second neural network 61 for identifying abnormal findings outputs the probability of whether or not an abnormal finding is present as an abnormal finding identification result. The probability may be the name of the abnormal finding or a label value corresponding to the abnormal finding. The second neural network 62 for identifying cell types outputs the probability that each analysis target cell belongs to each of a plurality of cell types input as training data as an identification result. The probability may be the name of the cell type or a label value corresponding to the cell type.

[0083] Training machine learning algorithms As training data for training the machine learning algorithm 57, machine learning training data 90 shown in FIG. 4 is used. The machine learning training data 90 includes feature values ​​and disease information 55. For each sample, the abnormal findings and / or the probability of the cell type output from the deep learning algorithm or a value obtained by converting the probability into the number of cells can be used as the feature value to be learned. In the machine learning training data 90, the feature values ​​are linked to disease information represented by the disease name, disease label value, etc., possessed by the subject from whom each sample was collected.

[0084] The feature quantity input to the machine learning algorithm 57 is at least one of information on the type of abnormal finding and information on the type of cell. It is preferable to use information on the type of abnormal finding and information on the cell as the feature quantity. The abnormal findings used may be of one type or of multiple types. The types of cells used as the feature quantities may be of one type or of multiple types.

[0085] The training images 70 captured from each sample used to train the deep learning algorithm are analyzed using the trained first deep learning algorithm and / or second deep learning algorithm to identify abnormal findings and / or cell types for cells in each training image 70. The second neural network 61 outputs the probability of each abnormal finding for each cell and a label value indicating the abnormal finding. The probability of each abnormal finding and the label value indicating the abnormal finding are the identification results of the type of abnormal finding. The second neural network 62 outputs the probability of each cell type and a label value indicating the cell type. The probability of each cell type and the label value indicating the cell type are the identification results of the cell type. Based on this information, a feature to be input to the machine learning algorithm 57 is generated.

[0086] An example of machine learning training data 90 is shown in FIG. 4. For convenience of explanation, FIG. 4 shows three cell counts (cell No. 1 to 3), five abnormal findings, namely, degranulation, Auer rods, spheroidal nuclei, hypersegmentation, and giant platelets, and eight cell types, namely, segmented neutrophils, band neutrophils, lymphocytes, monocytes, eosinophils, basophils, blasts, and platelets. In FIG. 4, A to F at the top of the table indicate the numbers of each column of the table. 1 to 19 at the left of the table indicate row numbers.

[0087] For each sample, the first neural network 50 and the second neural network 51 calculate the probability of each abnormal finding being present for each cell to be analyzed, and the second neural network 51 outputs the calculated probability. In FIG. 4, the probability of having an abnormal finding is expressed as a number between 0 and 1. For example, if an abnormal finding is present, it may be expressed as a value close to "1", and if no abnormal finding is present, it may be expressed as a value close to "0". In FIG. 4, the values ​​written in the cells of rows 1 to 5 of columns A to E are the values ​​output by the second neural network 51. Next, for each sample, the total probability of each type of abnormal finding analyzed is calculated. In FIG. 4, the values ​​written in the cells of rows 1 to 5 of column F are the total of each abnormal finding. For one sample, a group of data linking a label indicating the abnormal finding name with the total probability of having each abnormal finding per sample is called "information on the type of abnormal finding". The information on the type of abnormal finding is the first information 53 in FIG. 2. 4, a group of data linking cells B1 and F1, cells B2 and F2, cells B3 and F3, cells B4 and F4, and cells B5 and F5 is "information on the type of abnormal finding" and becomes first information 53. The first information 53 is linked to disease information 55 represented by a disease name or a label value indicating a disease name, and becomes machine learning training data 90. In FIG. 4, row 19 indicates disease information 55.

[0088] 4, a probability less than "1" may be indicated for one type of abnormal finding. In this case, for example, a predetermined cutoff value may be set and all types of abnormal findings showing values ​​lower than the cutoff value may be regarded as having a probability of "0." Also, a predetermined cutoff value may be set and all types of abnormal findings showing values ​​higher than the cutoff value may be regarded as having a probability of "1."

[0089] Here, the probability of each type of abnormal finding may be expressed as the number of cells of each type of abnormal finding.

[0090] As for the cell type, the first neural network 50 and the second neural network 52 calculate the probability that each analysis target cell belongs to each cell type, and the second neural network 52 outputs the calculated probability. The probability that each cell belongs to each cell type is calculated for all cell types that are analysis targets of the first neural network 50 and the second neural network 52. In the example of FIG. 4, the probability that one analysis target cell belongs to segmented neutrophils, band neutrophils, lymphocytes, monocytes, eosinophils, basophils, blasts, and The probability of each cell type being included in all items of the cell types, including platelets, is calculated. The values ​​written in cells in rows 6 to 13 of columns A to E are the values ​​output by the second neural network 52. Next, the total probability of each analyzed cell type is calculated for each sample. In FIG. 4, the values ​​written in cells in rows 6 to 13 of column F are the total for each cell type. A group of data linking the label indicating the cell type name with the total probability of each cell type being included per sample is called "information on cell type". The information on cell type is the second information 54 in FIG. 2. In FIG. 4, a group of data linking cells B6 and F6, cells B7 and F7, cells B8 and F8, cells B9 and F9, cells B10 and F10, cells B11 and F11, cells B12 and F12, and cells B13 and F13 is the "information on cell type", which is the second information 54. The second information 54 is linked to disease information 55 represented by a disease name or a label value indicating a disease name, and becomes machine learning training data 90. In FIG.

[0091] Here, the probability of each cell type may be expressed by the number of cells of each cell type. Also, as shown in FIG. 4, a probability higher than 0 may be indicated for multiple cell type items for one cell to be analyzed. In this case, for example, a predetermined cutoff value may be set and all cell types showing values ​​lower than the cutoff value may be regarded as having a probability of "0". Also, a predetermined cutoff value may be set and all cell types showing values ​​higher than the cutoff value may be regarded as having a probability of "1".

[0092] Further, a preferred feature is information on the type of abnormal findings acquired for each type of cell. Explaining with reference to FIG. 4, as shown in rows 14 to 18, when generating training data 75 for deep learning, a specific cell type and a specific type of abnormal findings are linked for training, such as degranulation of neutrophils (cell B14), Auer rods of blasts (cell B15), spheroidal nuclei of neutrophils (cell B16), hypersegmentation of neutrophils (cell B17), and giant platelets (cell B18). The generation of features is the same as when a type of abnormal findings not linked to a cell type is used. Information on the type of abnormal findings acquired for each type of cell is called third information. The third information is linked to disease information 55 represented by a disease name or a label value indicating a disease name, to become training data 90 for machine learning.

[0093] Machine learning training data 90 is input to the machine learning algorithm 57 to train the machine learning algorithm 57, thereby generating a trained machine learning algorithm 67 (see FIG. 5).

[0094] As a method for training the machine learning algorithm 57, preferably, at least one of the machine learning training data 90 in which the first information and the disease information 55 are linked, the machine learning training data 90 in which the second information and the disease information 55 are linked, and the machine learning training data 90 in which the third information and the disease information 55 are linked is used. More preferably, the machine learning training data 90 in which the first information and the disease information 55 are linked and the machine learning training data 90 in which the second information and the disease information 55 are linked is used, or the machine learning training data 90 in which the third information and the disease information 55 are linked and the machine learning training data 90 in which the second information and the disease information 55 are linked is used to train the machine learning algorithm 57. Most preferably, the machine learning training data 90 in which the second information 54 is linked to the disease information 55 represented by a disease name or a label value indicating the disease name, and the machine learning training data 90 in which the third information is linked to the disease information 55 represented by a disease name or a label value indicating the disease name are both input as training data to the machine learning algorithm 57. In this case, the cell type of the second information 54 and the cell type linked to the third information may be the same or different.

[0095] The machine learning algorithm is not limited as long as it can analyze diseases based on the above features. For example, the model may be selected from regression, trees, neural networks, Bayesian, time series models, clustering, and ensemble learning.

[0096] The regression may include linear regression, logistic regression, support vector machines, etc. The trees may include gradient boosted trees, decision trees, regression trees, random forests, etc. The neural networks may include perceptrons, convolutional neural networks, recurrent neural networks, residual networks, etc. The time series models may include moving averages, autoregression, autoregressive moving averages, autoregressive integrated moving averages, etc. The clustering may include k-nearest neighbors. The ensemble learning may include boosting, bagging, etc. Preferably, gradient boosted trees.

[0097] <How to support disease analysis> FIG. 5 shows an example of a method for supporting disease analysis. In the method, analysis data 81 is generated from an analysis image 78 obtained by capturing an image of cells to be analyzed. The analysis image 78 is an image of cells to be analyzed contained in a sample collected from a subject. The analysis image 78 can be obtained by using an imaging device such as a known optical microscope or a virtual slide scanner. In the example shown in FIG. 5, an automatic blood image analyzer DI-60 (manufactured by Sysmex Corporation) is used. The analysis image 78 is generated by reducing a raw image captured at 360 pixels x 365 pixels to 255 pixels x 255 pixels, as in the training image 70, using the 360 ​​pixels x 365 pixels, but this reduction is not essential. The number of pixels in the analysis image 78 is not limited as long as the analysis is possible, but it is preferable that one side exceeds 100 pixels. In the example shown in FIG. 5, red blood cells are present around a segmented neutrophil, but the image may be cropped so that only the target cell is included. If at least one cell to be analyzed is included in one image (red blood cells and normal-sized platelets may be included) and the pixels corresponding to the cell to be analyzed are present in about 1 / 9 or more of the total pixels of the image, it can be used as the analysis image 78.

[0098] For example, the imaging device preferably captures images in RGB color and CMY color, etc. The color image preferably represents the shade or brightness of each primary color, such as red, green, and blue, or cyan, magenta, and yellow, as a 24-bit value (8 bits x 3 colors). The analysis image 78 may include at least one hue and the shade or brightness of that hue, but more preferably includes at least two hues and the shade or brightness of each hue. Information including the hue and the shade or brightness of that hue is also called color tone.

[0099] For example, the RGB color is converted into a format including luminance information and hue information. As a format including luminance information and hue information, YUV (YCbCr, YPbPr, YIQ, etc.) can be given. Here, conversion to YCbCr format is taken as an example. Here, since the analysis image is RGB color, it is converted into luminance 79Y, a first hue (e.g., blue-based) 79Cb, and a second hue (e.g., red-based) 79Cr. Conversion from RGB to YCbCr can be performed by a known method. For example, conversion from RGB to YCbCr can be performed according to the international standard ITU-R BT.601. The converted luminance 79Y, first hue 79Cb, and second hue 79Cr can be expressed as a matrix of gradation values ​​as shown in FIG. 5 (hereinafter, also referred to as color tone matrix 79y, 79cb, 79cr). The luminance 72Y, the first hue 72Cb, and the second hue 72Cr are each expressed in 256 gradations ranging from 0 to 255. Here, instead of the luminance 79Y, the first hue 79Cb, and the second hue 79Cr, the training images may be converted using the three primary colors of red R, green G, and blue B, or the three primary colors of cyan C, magenta M, and yellow Y.

[0100] Next, a color tone vector data is generated for each pixel by combining three gradation values ​​of a luminance 79y, a first hue 79cb, and a second hue 79cr based on the color tone matrices 79y, 79cb, and 79cr. A set of color tone vector data 80 generated from one analysis image 78 is generated as analysis data 81.

[0101] It is preferable that the generation of the analysis data 81 and the generation of the deep learning training data 75 be the same at least in terms of the imaging conditions and the conditions for generating the vector data to be input from each image to the neural network.

[0102] The first deep learning algorithm comprises a first neural network 60 and a second neural network 62 to generate first information 63, which is information about the type of abnormal finding. The second deep learning algorithm comprises a first neural network 60 and a second neural network 62 to generate second information 64, which is information about the type of cell.

[0103] Analysis data 81 is input to the input layer 60a of the trained first neural network 60. The first neural network 60 extracts cellular features from the analysis data 81 and outputs the results from the output layer 60b of the first neural network 60. The input layer 61a of the second neural network 61 and the input layer 62a of the second neural network 62 receive the results output from the softmax function of the output layer 60b of the first neural network 60.

[0104] Next, the result output from the output layer 60b is input to the input layer 61a of the trained second neural network 61. For example, the second neural network 61 for identifying abnormal findings outputs the probability of whether or not there is an abnormal finding as an identification result of the abnormal finding from the output layer 61b based on the input feature amount.

[0105] The results output from the output layer 60b are input to the input layer 62a of the trained second neural network 62. Based on the input feature amount, the second neural network 62 outputs the probability that the cell to be analyzed included in the analysis image belongs to each of the cell types input as training data from the output layer 62b. In Fig. 5, reference characters 60c, 61c, and 62c indicate intermediate layers.

[0106] Next, based on the abnormal finding identification results, information on the type of abnormal finding corresponding to each sample (first information 63 in FIG. 5) is obtained for each sample. The first information 63 is, for example, the total probability of each type of analyzed abnormal finding output from the output layer 61b of the second neural network 61. The method of generating the first information 63 is similar to the method of generating training data for machine learning.

[0107] Furthermore, based on the cell type identification results, information on the cell type corresponding to each sample (second information 64 in FIG. 5) is obtained for each sample. The second information 64 is, for example, the sum of the probability for each analyzed cell type output from the output layer 62b of the second neural network 62. The method of generating the second information 64 is the same as the method of generating the machine learning training data 90.

[0108] The generated first information 63 and second information 64 are input to a trained machine learning algorithm 67, and an analysis result 83 is generated by the machine learning algorithm 67. The analysis result 83 may be a disease name or a label value representing the disease name.

[0109] As data to be input to the machine learning algorithm 67, preferably, at least one of the first information 63, the second information 64, and the third information can be used. More preferably, the first information 63 and the second information 64 are used, or the third information and the second information 64 are used. It is most preferable to use both the second information 64 and the third information as the analysis data 81. In this case, the cell type of the second information 64 and the cell type linked to the third information may be the same or different. The third information is information generated by linking a specific cell type with a specific type of abnormal finding when generating the analysis data 81, and the generation method thereof is the same as the method described in the generation method of the machine learning training data 90.

[0110] [Disease analysis support system 1] <Configuration of disease analysis support system 1> A disease analysis support system 1 will be described. Referring to FIG. 6, the disease analysis support system 1 includes a training device 100A and a disease analysis device 200A. The vendor side device 100 operates as the training device 100A, and the user side device 200 operates as the disease analysis device 200A. The training device 100A generates a classifier using deep learning training data 75 and machine learning training data 90, and provides the classifier to the user. The classifier is provided from the training device 100A to the disease analysis device 200A via a recording medium 98 or a network 99. The disease analysis device 200A analyzes an image of a cell to be analyzed using the classifier provided from the training device 100A.

[0111] The training device 100A is, for example, a general-purpose computer, and performs deep learning processing based on a flowchart described later. The disease analysis device 200A is, for example, a general-purpose computer, and performs disease analysis processing based on a flowchart described later. The recording medium 98 is, for example, a computer-readable, non-transitory tangible recording medium, such as a DVD-ROM or a USB memory.

[0112] The training apparatus 100A is connected to an imaging device 300. The imaging device 300 includes an image sensor 301 and a fluorescent microscope 302, and captures a bright-field image of a training specimen 308 set on a stage 309. The training specimen 308 is stained as described above. The training apparatus 100A acquires a training image 70 captured by the imaging device 300.

[0113] The disease analysis apparatus 200A is connected to an imaging device 400. The imaging device 400 includes an image sensor 401 and a fluorescence microscope 402, and captures a bright field image of a specimen 408 to be analyzed that is set on a stage 409. The specimen 408 to be analyzed has been stained in advance as described above. The disease analysis apparatus 200A acquires an analysis object image 78 captured by the imaging device 400.

[0114] The imaging devices 300 and 400 may be a known optical microscope or a virtual slide scanner having a function of capturing an image of a sample.

[0115] <Hardware configuration of the training device> 7, the vendor side device 100 (training device 100A) includes a processing unit 10 (10A), an input unit 16, and an output unit 17.

[0116] The processing unit 10 includes a CPU (Central Processing Unit) 11 that performs data processing, which will be described later. The processor 10 comprises a memory 12 used as a working area for data processing, a storage unit 13 for recording programs and processing data described below, a bus 14 for transmitting data between each unit, an interface unit 15 for inputting and outputting data to and from external devices, and a GPU (Graphics Processing Unit) 19. An input unit 16 and an output unit 17 are connected to the processor 10. Illustratively, the input unit 16 is an input device such as a keyboard or a mouse, and the output unit 17 is a display device such as a liquid crystal display. The GPU 19 functions as an accelerator that assists the arithmetic processing (for example, parallel arithmetic processing) performed by the CPU 11. That is, in the following description, the processing performed by the CPU 11 includes processing performed by the CPU 11 using the GPU 19 as an accelerator. means.

[0117] In addition, in order to perform the processing of each step described below in Fig. 10, Fig. 13, and Fig. 14, the processing unit 10 pre-records the program and the classifier according to the present invention in, for example, an executable format in the storage unit 13. The executable format is, for example, a format generated by conversion from a programming language by a compiler. The processing unit 10 uses the program recorded in the storage unit 13 to perform training processing of the first neural network 50, the second neural network 51, the second neural network 52, and the machine learning algorithm 57.

[0118] In the following description, unless otherwise specified, the processing performed by processing unit 10 refers to processing performed by CPU 11 based on the programs stored in storage unit 13 or memory 12, as well as first neural network 50, second neural network 51, second neural network 52, and machine learning algorithm 57. CPU 11 uses memory 12 as a working area to temporarily store necessary data (intermediate data during processing, etc.) in a volatile manner, and appropriately stores data to be stored for a long time, such as calculation results, in storage unit 13 in a non-volatile manner.

[0119] <Hardware configuration of disease analysis device> 8, the user side device 200 (disease analysis device 200A, disease analysis device 200B, disease analysis device 200C) includes a processing unit 20 (20A, 20B, 20C), an input unit 26, and an output unit 27.

[0120] The processing unit 20 includes a CPU (Central Processing Unit) 21 that performs data processing, which will be described later. The processor 20 includes a memory 22 used as a working area for data processing, a storage unit 23 for storing a program and processing data described later, a bus 24 for transmitting data between each unit, an interface unit 25 for inputting and outputting data to and from an external device, and a GPU (Graphics Processing Unit) 29. An input unit 26 and an output unit 27 are connected to the processor 20. Illustratively, the input unit 26 is an input device such as a keyboard or a mouse, and the output unit 27 is a display device such as a liquid crystal display. The GPU 29 functions as an accelerator that assists the arithmetic processing (for example, parallel arithmetic processing) performed by the CPU 21. That is, in the following description, the processing performed by the CPU 21 includes the processing performed by the CPU 21 using the GPU 29 as an accelerator.

[0121] Furthermore, in order to perform the processing of each step described in the disease analysis processing below, the processing unit 20 pre-records the program and classifier according to the present invention in, for example, an executable format in the storage unit 23. The executable format is, for example, a format generated by conversion from a programming language by a compiler. The processing unit 20 performs processing using the program and classifier recorded in the storage unit 23.

[0122] In the following description, unless otherwise specified, the processing performed by the processing unit 20 means processing actually performed by the CPU 21 of the processing unit 20 based on the program and deep learning algorithm 60 stored in the storage unit 23 or memory 22. The CPU 21 uses the memory 22 as a working area to temporarily store necessary data (such as intermediate data during processing) in a volatile manner, and appropriately records data to be stored for a long time, such as calculation results, in the storage unit 23 in a non-volatile manner.

[0123] <Function blocks and processing procedures> (Deep learning processing) 9, the processing unit 10A of the training device 100A includes a deep learning training data generation unit 101, a deep learning training data input unit 102, and a deep learning algorithm update unit 103. These functional blocks install a program for causing a computer to execute deep learning processing in the storage unit 13 or memory 12 of the processing unit 10A, and execute the program. This is realized by execution by the CPU 11. A deep learning training data database (DB) 104 and a deep learning algorithm database (DB) 105 are recorded in the storage unit 13 or the memory 12 of the processing unit 10A.

[0124] The training images 70 are captured in advance by the imaging device 300, and are stored in advance in the storage unit 13 or memory 12 of the processing unit 10A in association with, for example, the morphological cell type to which the cells to be analyzed belong or abnormal findings. The untrained first neural network 50 and the second neural network 52 are stored in advance in the deep learning algorithm database 105. The first neural network 50 and the second neural network 52, which have been trained once and are available for updating, are stored in advance in the deep learning algorithm database 105.

[0125] The processing unit 10A of the training device 100A performs the processes shown in Fig. 10. Explaining this using the functional blocks shown in Fig. 9, the processes of steps S11, S12, S16, and S17 are performed by the deep learning training data generation unit 101. The process of step S13 is performed by the deep learning training data input unit 102. The process of step S14 is performed by the deep learning algorithm update unit 103.

[0126] An example of deep learning processing performed by the processing unit 10A will be described with reference to FIG. 10. First, the processing unit 10A acquires training images 70. The training images 70 are acquired by the operator through the I / F unit 15 via the I / F unit 15, or through the imaging device 300 or the recording medium 98, or through the network. When acquiring the training images 70, information on which of the types of cells and / or abnormal findings classified morphologically the training images 70 indicate is also acquired. The information on which of the types of cells and / or abnormal findings classified morphologically the training images 70 indicate may be linked to the training images 70, or may be input by the operator through the input unit 16.

[0127] In step S11, the processing unit 10A converts the acquired training image 70 into a luminance Y, a first hue Cb, and a The hue is converted to the second hue Cr, and the color tone vector data 74 is generated according to the procedure explained in the method for generating training data above.

[0128] In step S12, the processing unit 10A assigns a label value corresponding to the color tone vector data 74 based on information indicating which of a morphologically classified cell type and / or abnormal finding associated with the training image 70 and the label value associated with the morphologically classified cell type or abnormal finding stored in the memory 12 or the storage unit 13. In this way, the processing unit 10A generates training data 75 for deep learning.

[0129] 10, the processing unit 10A trains the first neural network 50 and the second neural network 51 using deep learning training data 75. The training results of the first neural network 50 and the second neural network 51 are accumulated every time training is performed using a plurality of deep learning training data 75.

[0130] Next, in step S14, the processing unit 10A judges whether or not training results for a predetermined number of trials have been accumulated. If training results for the predetermined number of trials have been accumulated (YES), the processing unit 10A proceeds to processing in step S15, and if training results for the predetermined number of trials have not been accumulated (NO), the processing unit 10A proceeds to processing in step S16.

[0131] Next, when the training results have been accumulated for a predetermined number of trials, in step S15, The management unit 10A updates the connection weights w of the first neural network 50 and the second neural network 51, or the first neural network 50 and the second neural network 52, using the training results accumulated in step S13. Since the disease analysis method uses the stochastic gradient descent method, the connection weights w of the first neural network 50 and the second neural network 51, or the first neural network 50 and the second neural network 52 are updated at a stage where learning results for a predetermined number of trials are accumulated. The process of updating the connection weights w is specifically a process of performing calculations by the gradient descent method shown in (Equation 11) and (Equation 12) described later.

[0132] In step S16, the processing unit 10A determines whether the first neural network 50 and the second neural network 51, or the first neural network 50 and the second neural network 52 have been trained with a prescribed number of training data 75. If training has been performed with a prescribed number of training data 75 (YES), the deep learning process is terminated.

[0133] If the first neural network 50 and the second neural network 51, or one neural network 50 and the second neural network 52 have not been trained with the specified number of training data 75 (NO), the processing unit 10A proceeds from step S16 to step S17, and performs processing from step S11 to step S16 on the next training image 70.

[0134] According to the above-described process, the first neural network 50 and the second neural network 51, or the first neural network 50 and the second neural network 52 are trained to obtain the second deep learning algorithm and the second deep learning algorithm.

[0135] (Neural network structure) 11(a) illustrates the structures of the first neural network 50 and the second neural network 51, 52. The first neural network 50 and the second neural network 51 include input layers 50a, 51a, 52a, output layers 50b, 51b, 52b, and intermediate layers 50c, 51c, 52c between the input layers 50a, 51a, 52a and the output layers 50b, 51b, 52b, and the intermediate layers 50c, 51c, 52c are configured with a plurality of layers. The number of layers configuring the intermediate layers 50c, 51c, 52c can be, for example, five or more layers.

[0136] In the first neural network 50 and the second neural network 51, or in the first neural network 50 and the second neural network 52, a plurality of nodes 89 arranged in layers are connected between the layers, so that information is propagated in only one direction, as indicated by the arrow D in the figure, from the layers 50a, 51a, and 52a on the input side to the layers 50b, 51b, and 52b on the output side.

[0137] (Operation at each node) Fig. 11(b) is a schematic diagram showing the calculations at each node. Each node 89 receives multiple inputs and calculates one output (z). In the example shown in Fig. 11(b), node 89 receives four inputs. The total input (u) received by node 89 is expressed by the following (Equation 1).

[0138]

number

[0139] Each input is multiplied by a different weight. In (Equation 1), b is a value called the bias. The output (z) of the node is the output of a given function f for the total input (u) expressed in (Equation 1), and is expressed by the following (Equation 2). The function f is called the activation function.

[0140]

number

[0141] FIG. 11(c) is a schematic diagram showing the operation between nodes. In the first neural network 50 and the second neural networks 51 and 52, nodes that output a result (z) expressed by (Equation 2) for a total input (u) expressed by (Equation 1) are arranged in layers. The output of the node in the previous layer becomes the input of the node in the next layer. In the example shown in FIG. 11(c), the output of the node 89a in the left layer becomes the input of the node 89b in the right layer. Each node 89b in the right layer receives the output from the node 89a in the left layer. A different weight is applied to each connection between each node 89a in the left layer and each node 89b in the right layer. If the outputs of each of the multiple nodes 89a in the left layer are x1 to x4, the inputs to each of the three nodes 89b in the right layer are expressed by the following (Equations 3-1) to (Equations 3-3).

[0142]

number

[0143]

number

[0144]

number

[0145] (Activation function) In the disease analysis method, a rectified linear unit function is used as the activation function. The rectified linear unit function is expressed by the following (Equation 5).

[0146]

number

[0147]

number

[0148] (Neural network training) If the function represented using a neural network is denoted as y(x:w), the function y(x:w) will change when the parameter w of the neural network is changed. Adjusting the function y(x:w) so that the neural network selects the parameter w that is more suitable for the input x is called neural network learning. Suppose multiple pairs of input and output of the function represented using a neural network are given. If the desired output for a certain input x is d, the input / output pair is {(x1,d1),(x2,d2),...,(x n ,d n )}. A set of pairs represented by (x, d) is called training data. Specifically, the set of pairs of color density values ​​and true image labels for each pixel in the single-color images of Y, Cb, and Cr shown in Fig. 2(b) is the training data shown in Fig. 2(a).

[0149] The learning of a neural network is based on what kind of input / output pairs (x n ,d n ), input x n Given the neural network output y(x n :w) but the output is d n This means adjusting the weights w so that the weights are as close as possible to the error function. The error function is the closeness of the function expressed by the neural network to the training data.

[0150]

number

[0151]

number

[0152]

number

[0153]

number

[0154] The target output d by the softmax function of (Eq. 7) n Let d be 1 only if the output is the correct class, and 0 otherwise. Let d be the target output. n =[d n1 , ,d nK ], for example, input x n If the correct class of is C3, the target output d n3 Only the input is 1, and the other target outputs are 0. When encoded in this way, the posterior distribution is expressed by the following (Equation 9).

[0155]

number

[0156]

number

[0157] Minimizing the error function E(w) with respect to the parameter w is the same as finding a local minimum of the function E(w). The parameter w is the weight of the connection between nodes. The minimum of the weight w is found by iterative calculations that start with an arbitrary initial value and repeatedly update the parameter w. One example of such calculations is the gradient descent method.

[0158] In the gradient descent method, a vector expressed by the following (Equation 11) is used.

number

[0159]

number

number

[0160] The calculation by (Equation 12) may be performed on all the training data (n = 1, , N) or only on a part of the training data. The gradient descent method performed on only a part of the training data is called the stochastic gradient descent method. The disease analysis method uses stochastic gradient descent.

[0161] (Machine learning processing 1) The first machine learning process trains a machine learning algorithm 57 from the first information or the second information. 12, the processing unit 10A of the training device 100A trains the machine learning algorithm 57 based on the first information 53 and the disease information 55. The processing unit 10A includes a machine learning training data generating unit 101a, a machine learning training data input unit 102a, and a machine learning algorithm updating unit 103a. These functional blocks are realized by installing a program for causing a computer to execute machine learning processing in the storage unit 13 or memory 12 of the processing unit 10A and executing this program by the CPU 11. A machine learning training data database (DB) 104a and a machine learning algorithm database (DB) 105a are recorded in the storage unit 13 or memory 12 of the processing unit 10A.

[0162] The first information or the second information is generated by the processing unit 10A, and is stored in advance in the storage unit 13 or the memory 12 of the processing unit 10A in association with, for example, the morphological cell type to which the cell to be analyzed belongs or an abnormal finding. The untrained first neural network 50, the second neural network 51, and the second neural network 52 are stored in advance in the machine learning algorithm database 105. The first neural network 50, the second neural network 51, and the second neural network 52, which have been trained once and are available for update, are stored in advance in the machine learning algorithm database 105a.

[0163] The processing unit 10A of the training device 100A performs the processes shown in Fig. 13. To explain using the functional blocks shown in Fig. 12, the processes of steps S111, S112, S114, and S115 are performed by the machine learning training data generation unit 101a. The process of step S113 is performed by the machine learning training data input unit 102a.

[0164] An example of the first machine learning process performed by the processing unit 10A will be described with reference to FIG. In step S111, the processing unit 10A of the training device 100A generates the first information or the second information according to the method described in the section on training the machine learning algorithm. Specifically, the processing unit 10A identifies the type of abnormal findings for the cells of each training image 70 from the first deep learning algorithm or the second deep learning algorithm trained in steps S11 to S16, and obtains an identification result. The processing unit 10A outputs the identification result of the type of abnormal findings for each cell from the second neural network 61. In step S111, the processing unit 10A generates the first information for each sample from which the training image 70 was acquired, based on the identification result of the type of abnormal findings. In addition, the processing unit 10A identifies the type of cell for each training image 70 from the second neural network 62, and obtains an identification result. The processing unit 10A generates the second information for each sample from which the training image 70 was acquired, based on the identification result of the type of cell.

[0165] Next, in step S112, the processing unit 10A generates machine learning training data 90 from the first information and the disease information 55 linked to the training image 75. Alternatively, the processing unit 10A generates machine learning training data 90 from the second information and the disease information 55 linked to the training image 75.

[0166] Next, in step S113, the processing unit 10A inputs the machine learning training data 90 to the machine learning algorithm, and trains the machine learning algorithm.

[0167] Next, in step S114, the processing unit 10A judges whether the processing has been performed for all the training samples. If the processing has been performed for all the training samples, the processing ends. If the processing has not been performed for all the training samples, the processing proceeds to step S115, where the identification result of the type of abnormal finding of another sample or the identification result of the type of cell is obtained, and the processing returns to step S111 to repeat the training of the machine learning algorithm.

[0168] (Machine learning processing 2) The second machine learning process trains a machine learning algorithm 57 from the first information and the second information. 12, the processing unit 10A of the training device 100A trains the machine learning algorithm 57 based on the first information 53 and the disease information 55. The processing unit 10A includes a machine learning training data generating unit 101b, a machine learning training data input unit 102b, and a machine learning algorithm updating unit 103b. These functional blocks are realized by installing a program for causing a computer to execute machine learning processing in the storage unit 13 or memory 12 of the processing unit 10A and executing this program by the CPU 11. A machine learning training data database (DB) 104b and a machine learning algorithm database (DB) 105b are recorded in the storage unit 13 or memory 12 of the processing unit 10A.

[0169] The first information or the second information is generated by the processing unit 10A, and is stored in advance in the storage unit 13 or the memory 12 of the processing unit 10A in association with, for example, the morphological cell type to which the cell to be analyzed belongs or an abnormal finding. The untrained first neural network 50, the second neural network 51, and the second neural network 52 are stored in advance in the machine learning algorithm database 105. The first neural network 50, the second neural network 51, and the second neural network 52, which have been trained once and are available for update, are stored in advance in the machine learning algorithm database 105b.

[0170] The processing unit 10A of the training device 100A performs the processes shown in Fig. 14. To explain using the functional blocks shown in Fig. 12, the processes of steps S1111, S1112, S1114, and S1115 are performed by the machine learning training data generation unit 101b. The process of step S1113 is performed by the machine learning training data input unit 102.

[0171] An example of the second machine learning process performed by the processing unit 10A will be described with reference to FIG. 14. In step S1111, the processing unit 10A of the training device 100A generates the first information and the second information according to the method described in the section on training the machine learning algorithm. Specifically, the processing unit 10A identifies the type of abnormal findings for the cells of each training image 70 from the first deep learning algorithm and the second deep learning algorithm trained in steps S11 to S16, and obtains an identification result. The processing unit 10A outputs the identification result of the type of abnormal findings for each cell from the second neural network 61. In step S1111, the processing unit 10A generates first information for each sample from which the training image 70 was obtained, based on the identification result of the type of abnormal findings. In addition, the processing unit 10A identifies the type of cell for each training image 70 from the second neural network 62, and obtains an identification result. The processing unit 10A outputs the type of cell from the second neural network 62. Based on the class identification results, second information is generated for each sample from which the training images 70 were acquired.

[0172] Next, in step S1112, the processing unit 10A generates machine learning training data 90 from the first information, the second information, and the disease information 55 linked to the training image 75.

[0173] Next, in step S1113, the processing unit 10A inputs the machine learning training data 90 to the machine learning algorithm, and trains the machine learning algorithm.

[0174] Next, in step S1114, the processing unit 10A judges whether the processing has been performed for all the training samples. If the processing has been performed for all the training samples, the processing ends. If the processing has not been performed for all the training samples, the processing proceeds to step S1115, where the identification result of the type of abnormal finding of another sample or the identification result of the type of cell is obtained, and the processing returns to step S1111 to repeat the training of the machine learning algorithm.

[0175] The outline of the machine learning algorithm used in steps S113 and S1113 is as follows.

[0176] As a machine learning algorithm, we use ensemble such as Gradient Boosting. Ensemble learning (a classifier composed of multiple classifiers) can be used. Examples of ensemble learning include Extreme Gradient Boosting (EGB) and Stochastic Gradient Boosting. Gradient Boosting is a type of boosting algorithm, This is a method of constructing multiple weak learners. For example, regression trees can be used as weak learners.

[0177] For example, a regression tree is a set of input vectors x and labels y, and the entire learning

number

number

[0178]

number

[0179] (Disease analysis processing) 15 shows a functional block diagram of a disease analysis device 200A that performs disease analysis processing from an analysis target image 78 to generating an analysis result 83. The processing unit 20A of the disease analysis device 200A includes an analysis data generating unit 201, an analysis data input unit 202, and an analysis unit 203. These functional blocks are realized by installing a program for causing a computer according to the present invention to execute disease analysis processing in the storage unit 23 or memory 22 of the processing unit 20A and executing this program by the CPU 21. A deep learning training data database (DB) 104 and a deep learning algorithm database (DB) 105 are provided from the training device 100A via a recording medium 98 or a network 99, and are recorded in the storage unit 23 or memory 22 of the processing unit 20A. The machine learning training data database (DB) 104a, b and the deep learning algorithm database (DB) 105a, b are provided from the training device 100A via a recording medium 98 or a network 99, and are recorded in the memory unit 23 or memory 22 of the processing unit 20A.

[0180] The image 78 to be analyzed is captured by the imaging device 400 and stored in the storage unit 23 or memory 22 of the processing unit 20A. The first neural network 60 and the second neural networks 61, 62 including the trained connection weights w are stored in the deep learning algorithm database 105 in association with the type of cell or the type of abnormal finding based on the morphological classification to which the cells to be analyzed belong, and function as a program module that is a part of a program that causes a computer to execute disease analysis processing. That is, the first neural network 60 and the second neural networks 61, 62 are used in a computer equipped with a CPU and memory, and output the determination result of the type of abnormal finding or the identification result of the type of cell. The CPU 21 of the processing unit 20A causes the computer to function to execute calculations or processing of specific information according to the purpose of use. In addition, the trained machine learning algorithm 67 is stored in the machine learning algorithm databases 105a, b, and is stored in the deep learning algorithm databases 105a, b. The machine learning algorithm 67 functions as a program module that is a part of a program that causes the computer to execute a disease analysis process. That is, the machine learning algorithm 67 is used in a computer having a CPU and a memory, and outputs a disease analysis result.

[0181] Specifically, the CPU 21 of the processing unit 20A generates a discrimination result of the type of abnormal findings in the analysis data generation unit 201 using a first deep learning algorithm stored in the storage unit 23 or the memory 22. The processing unit 20A generates first information 63 based on the discrimination result of the type of abnormal findings in the analysis data generation unit 201. The generated first information 63 is input to the analysis data input unit 202 and stored in the machine learning training data DB 104a. The processing unit 20A performs disease analysis in the analysis unit 203 and outputs the analysis result 83 to the output unit 27. Alternatively, the CPU 21 of the processing unit 20A generates a discrimination result of the type of abnormal findings in the analysis data generation unit 201 using a second deep learning algorithm stored in the storage unit 23 or the memory 22. The processing unit 20A generates second information 64 based on the discrimination result of the type of abnormal findings in the analysis data generation unit 201. The generated second information 64 is input to the analysis data input unit 202 and stored in the machine learning training data DB 104b. The processing unit 20A performs disease analysis in the analysis unit 203 and outputs the analysis result 83 to the output unit 27.

[0182] 16, the processes of steps S21 and S22 are performed by the analysis data generation unit 201. The processes of steps S23, S24, S25, and S27 are performed by the analysis data input unit 202. The process of step S26 is performed by the analysis unit 203.

[0183] (Disease analysis processing 1) 16, an example of the first disease analysis process performed by the processing unit 20A from the analysis target image 78 to outputting the analysis result 83 will be described. The first disease analysis process outputs the analysis result 83 from the first information or the second information.

[0184] First, the processing unit 20A acquires the analysis image 78. The analysis image 78 is acquired by a user's operation, by being imported from the imaging device 400, by being imported from the recording medium 98, or by being acquired via the I / F unit 25 over a network.

[0185] In step S21, similar to step S11 shown in FIG. 10, the acquired analysis image 78 is converted into luminance Y, first hue Cb, and second hue Cr, and color tone vector data 80 is generated according to the procedure described in the above-mentioned method for generating analysis data.

[0186] Next, in step S22, the processing unit 20A generates analysis data 81 from the color tone vector data 80 according to the procedure explained in the above-mentioned method for generating analysis data.

[0187] Next, in step S23, the processing unit 20A acquires the first deep learning algorithm or the second deep learning algorithm stored in the algorithm database 105.

[0188] Next, in step S24, the processing unit 20A inputs the analysis data 81 to the first neural network 60 constituting the first deep learning algorithm. The processing unit 20A inputs the feature amount output from the first neural network 60 to the second neural network 61 according to the procedure described in the above disease analysis method, and outputs the identification result of the type of abnormal finding from the second neural network 61. The processing unit 20A stores this identification result in the memory 22 or the storage unit 23. Alternatively, in step S24, the processing unit 20A inputs the analysis data 81 to the first neural network 60 constituting the second deep learning algorithm. The processing unit 20A inputs the feature amount output from the first neural network 60 to the second neural network 61 according to the procedure described in the above disease analysis method, and outputs the identification result of the type of abnormal finding from the second neural network 61. The feature amount output from the neural network 60 is input to the second neural network 62, and the cell type identification result is output from the second neural network 62. The processing unit 20A stores this identification result in the memory 22 or the storage unit 23.

[0189] In step S25, the processing unit 20A judges whether all of the initially acquired analysis images 78 have been classified. If the classification of all of the analysis images 78 has been completed (YES), the processing unit 20A proceeds to step S26, where the first information 63 is generated from the result of the classification of the cell type, or the second information is generated from the result of the classification of the cell type. If the classification of all of the analysis images 78 has not been completed (NO), the processing unit 20A proceeds to step S27, where the processing of steps S21 to S25 is performed on the analysis images 78 that have not yet been classified.

[0190] Next, in step S28, the processing unit 20A acquires the machine learning algorithm 67. Subsequently, in step S29, the processing unit 20A inputs the first information or the second information to the machine learning algorithm 67.

[0191] Finally, in step S30, the processing unit 20A outputs the analysis result 83 to the output unit 27 as a disease name or a label value linked to the disease name.

[0192] (Disease analysis processing 2) 17, an example of the second disease analysis process performed by the processing unit 20A from the analysis target image 78 to outputting the analysis result 83 will be described. The second disease analysis process outputs the analysis result 83 from the first information and the second information.

[0193] First, the processing unit 20A acquires the analysis image 78. The analysis image 78 is acquired by a user's operation, by being imported from the imaging device 400, by being imported from the recording medium 98, or by being acquired via the I / F unit 25 over a network.

[0194] In step S121, similar to step S11 shown in FIG. 10, the acquired analysis image 78 is converted into luminance Y, first hue Cb, and second hue Cr, and color tone vector data 80 is generated according to the procedure described in the above-mentioned method for generating analysis data.

[0195] Next, in step S122, the processing unit 20A generates analysis data 81 from the color tone vector data 80 according to the procedure explained in the above-mentioned method for generating analysis data.

[0196] Next, in step S123, the processing unit 20A acquires the first deep learning algorithm and the second deep learning algorithm stored in the algorithm database 105.

[0197] Next, in step S124, the processing unit 20A inputs the analysis data 81 to the first neural network 60 constituting the first deep learning algorithm. The processing unit 20A inputs the feature amount output from the first neural network 60 to the second neural network 61 according to the procedure described in the above disease analysis method, and outputs the identification result of the type of abnormal findings from the second neural network 61. The processing unit 20A stores this identification result in the memory 22 or the storage unit 23. Also, in step S124, the processing unit 20A inputs the analysis data 81 to the first neural network 60 constituting the second deep learning algorithm. The processing unit 20A inputs the feature amount output from the first neural network 60 to the second neural network 62 according to the procedure described in the above disease analysis method, and outputs the identification result of the type of cells from the second neural network 62. The processing unit 20A stores this identification result in the memory 22 or the storage unit 23.

[0198] In step S125, the processing unit 20A judges whether all of the initially acquired analysis images 78 have been classified. If the classification of all of the analysis images 78 has been completed (YES), the processing unit 20A proceeds to step S126, where the first information 63 is generated from the results of the classification of the cell types, and the second information is generated from the results of the classification of the cell types. If the classification of all of the analysis images 78 has not been completed (NO), the processing unit 20A proceeds to step S127, where the processing of steps S121 to S125 is performed on the analysis images 78 that have not yet been classified.

[0199] Next, in step S128, the processing unit 20A acquires the machine learning algorithm 67. Subsequently, in step S129, the processing unit 20A inputs the first information and the second information to the machine learning algorithm 67.

[0200] Finally, in step S130, the processing unit 20A outputs the analysis result 83 to the output unit 27 as a disease name or a label value linked to the disease name.

[0201] <Computer Program> A computer program for assisting disease analysis that causes a computer to execute the processes of steps S21 to S30 or steps S121 to S130 will be described. The computer program may include a program for training a machine learning algorithm that causes a computer to execute the processes of steps S11 to S17 and steps S111 to S115, or a program for training a machine learning algorithm that causes a computer to execute the processes of steps S11 to S17 and steps S1111 to S1115.

[0202] Further, a program product such as a storage medium storing the computer program will be described. The computer program is stored in a storage medium such as a hard disk, a semiconductor memory element such as a flash memory, an optical disk, etc. The format of the program stored in the storage medium is not limited as long as the processing unit can read the program. It is preferable that the storage in the storage medium is non-volatile.

[0203] [Disease Analysis System 2] <Disease Analysis System 2 Configuration> Another aspect of the disease analysis system will be described. FIG. 18 shows a configuration example of the second disease analysis system 2. The disease analysis system 2 includes a user-side device 200, which operates as an integrated disease analysis device 200B. The disease analysis device 200B is, for example, configured with a general-purpose computer, and performs both the deep learning process and the disease analysis process described in the disease analysis system 1. In other words, the disease analysis system 2 is a stand-alone system that performs deep learning and disease analysis on the user side. In the second disease analysis system, the integrated disease analysis device 200B installed on the user side serves the functions of both the training device 100A and the disease analysis device 200A.

[0204] 18, the disease analysis apparatus 200B is connected to an imaging device 400. The imaging device 400 captures a training image 70 during deep learning processing, and captures an analysis target image 78 during disease analysis processing.

[0205] <Hardware configuration> The hardware configuration of the disease analysis apparatus 200B is similar to the hardware configuration of the user side apparatus 200 shown in FIG.

[0206] <Function blocks and processing procedures> 19 shows a functional block diagram of a disease analysis device 200B. A processing unit 20B of the disease analysis device 200B includes a deep learning training data generation unit 101 and a deep learning training data input unit 10. 2, a deep learning algorithm update unit 103, machine learning training data generation units 101a, b, machine learning training data input units 102a, b, machine learning algorithm update units 103a, b, an analysis data generation unit 201, an analysis data input unit 202, and an analysis unit 203.

[0207] The processing unit 20B of the disease analysis device 200B performs the process shown in FIG. 9 during deep learning processing, performs the process shown in FIG. 13 or FIG. 14 during machine learning, and performs the process shown in FIG. 16 or FIG. 17 during disease analysis processing. Explained using the functional blocks shown in FIG. 19, during deep learning processing, the processes of steps S11, S12, S16, and S17 in FIG. 9 are performed by the deep learning training data generation unit 101. The process of step S13 in FIG. 9 is performed by the deep learning training data input unit 102. The process of step S14 in FIG. 9 is performed by the deep learning algorithm update unit 103. During machine learning processing, the processes of steps S111, S112, S114, and S115 in FIG. 13 are performed by the machine learning training data generation unit 101a. The process of step S113 is performed by the machine learning training data input unit 102a. Alternatively, during machine learning, the processes of steps S1111, S1112, S1114, and S1115 in FIG. 14 are performed by the machine learning training data generation unit 101b. The process of step S1113 in FIG. 14 is performed by the machine learning training data input unit 102. During disease analysis processing, the processes of steps S21 and S22 in FIG. 16 are performed by the analysis data generation unit 201. The processes of steps S21 and S22 in FIG. 16 are performed by the analysis data generation unit 201. The processes of steps S23, S24, S25, and S27 in FIG. 16 are performed by the analysis data input unit 202. The process of step S26 in FIG. 16 is performed by the analysis unit 203. Alternatively, the processes of steps S121 and S122 in FIG. 17 are performed by the analysis data generation unit 201. 17 are performed by the analysis data input unit 202. The process of step S126 in FIG.

[0208] The procedure of the deep learning process and the procedure of the disease analysis process performed by the disease analysis device 200B are similar to those performed by the training device 100A and the disease analysis device 200A, respectively. However, the disease analysis device 200B acquires the training image 70 from the imaging device 400.

[0209] In the disease analysis device 200B, the user can check the classification accuracy of the classifier. If the classification result of the classifier differs from the classification result based on the user's image observation, the first deep learning algorithm and the second deep learning algorithm can be trained and corrected using the analysis data 81 as training data 78 and the classification result based on the user's image observation as the label value 77. In this way, the training efficiency of the first neural network 50 and the first neural network 51 can be further improved.

[0210] [Disease Analysis System 3] <Disease analysis system 3 configuration> Another aspect of the disease analysis system will be described. FIG. 20 shows a configuration example of a third disease analysis system 3. The disease analysis system 3 includes a vendor-side device 100 and a user-side device 200. The vendor-side device 100 includes a processing unit 10 (10B), an input unit 16, and an output unit 17. The vendor-side device 100 operates as an integrated disease analysis device 100B, and the user-side device 200 operates as a terminal device 200C. The disease analysis device 100B is, for example, a general-purpose computer, and is a cloud server-side device that performs both the deep learning process and the disease analysis process described in the disease analysis system 1. The terminal device 200C is, for example, a general-purpose computer, and transmits an image to be analyzed to the disease analysis device 100B through a network 99, and receives an image of the analysis result from the disease analysis device 100B through the network 99.

[0211] The disease analysis system 3 is an integrated disease analysis device 100B installed on the vendor side. The third disease analysis system has the functions of both the device 100A and the disease analysis device 200A. On the other hand, the third disease analysis system includes a terminal device 200C, and provides an input interface for an analysis image 78 and an output interface for an analysis result image to the user-side terminal device 200C. In other words, the third disease analysis system is a cloud service-type system in which a vendor side that performs deep learning processing and disease analysis processing provides an input interface for providing an analysis image 78 to the user side and an output interface for providing an analysis result 83 to the user side. The input interface and the output interface may be integrated.

[0212] The disease analysis apparatus 100B is connected to an imaging apparatus 300, and acquires training images 70 captured by the imaging apparatus 300.

[0213] The terminal device 200C is connected to the imaging device 400, and acquires an analysis target image 78 captured by the imaging device 400.

[0214] <Hardware configuration> The hardware configuration of the disease analysis apparatus 100B is similar to that of the vendor-side apparatus 100 shown in Fig. 7. The hardware configuration of the terminal apparatus 200C is similar to that of the user-side apparatus 200 shown in Fig. 8.

[0215] <Function blocks and processing procedures> 21 shows a functional block diagram of the disease analysis device 100B. The processing unit 10B of the disease analysis device 100B includes a deep learning training data generation unit 101, a deep learning training data input unit 102, a deep learning algorithm update unit 103, machine learning training data generation units 101a and 101b, machine learning training data input units 102a and 102b, machine learning algorithm update units 103a and 103b, an analysis data generation unit 201, an analysis data input unit 202, and an analysis unit 203.

[0216] The processing unit 20B of the disease analysis device 200B performs the process shown in FIG. 9 during deep learning processing, performs the process shown in FIG. 13 or FIG. 14 during machine learning, and performs the process shown in FIG. 16 or FIG. 17 during disease analysis processing. Explained using the functional blocks shown in FIG. 19, during deep learning processing, the processes of steps S11, S12, S16, and S17 in FIG. 9 are performed by the deep learning training data generation unit 101. The process of step S13 in FIG. 9 is performed by the deep learning training data input unit 102. The process of step S14 in FIG. 9 is performed by the deep learning algorithm update unit 103. During machine learning processing, the processes of steps S111, S112, S114, and S115 in FIG. 13 are performed by the machine learning training data generation unit 101a. The process of step S113 is performed by the machine learning training data input unit 102a. Alternatively, during machine learning, the processes of steps S1111, S1112, S1114, and S1115 in FIG. 14 are performed by the machine learning training data generation unit 101b. The process of step S1113 in FIG. 14 is performed by the machine learning training data input unit 102. During disease analysis processing, the processes of steps S21 and S22 in FIG. 16 are performed by the analysis data generation unit 201. The processes of steps S21 and S22 in FIG. 16 are performed by the analysis data generation unit 201. The processes of steps S23, S24, S25, and S27 in FIG. 16 are performed by the analysis data input unit 202. The process of step S26 in FIG. 16 is performed by the analysis unit 203. Alternatively, the processes of steps S121 and S122 in FIG. 17 are performed by the analysis data generation unit 201. 17 are performed by the analysis data input unit 202. The process of step S126 in FIG.

[0217] The deep learning processing procedure and the disease analysis processing procedure performed by the disease analysis device 100B are similar to the procedures performed by the training device 100A and the disease analysis device 200A, respectively.

[0218] The processing unit 10B receives the analysis target image 78 from the user's terminal device 200C, and generates deep learning training data 75 according to steps S11 to S17 shown in FIG.

[0219] 12, the processing unit 10B transmits the analysis results including the analysis result 83 to the user-side terminal device 200C. In the user-side terminal device 200C, the processing unit 20C outputs the received analysis results to the output unit 27.

[0220] As described above, the user of the terminal device 200C can obtain the analysis result 83 by transmitting the analysis target image 78 to the disease analysis device 100B.

[0221] According to the disease analysis device 100B, a user can use the classifier without acquiring the training data database 104 and the algorithm database 105 from the training device 100A. This makes it possible to provide a service for identifying cell types and cell features based on morphological classification as a cloud service.

[0222] [Other formats] The present invention is not limited to the above embodiment.

[0223] In the above embodiment, a method of converting the color tone into luminance Y, a first hue Cb, and a second hue Cr to generate the deep learning training data 75 is illustrated, but the conversion of the color tone is not limited to this. For example, the three primary colors of red (R), green (G), and blue (B) may be used as they are without converting the color tone. Alternatively, two primary colors may be used by removing one hue from the primary colors. Alternatively, one primary color may be only one of the three primary colors of red (R), green (G), and blue (B) (for example, green (G)). The primary colors may be converted into three primary colors of cyan (C), magenta (M), and yellow (Y). For example, the analysis target image 78 is not limited to a color image of the three primary colors of red (R), green (G), and blue (B), but may be a color image of two primary colors, or may be an image including one or more primary colors.

[0224] In the above-mentioned training data generating method and analysis data generating method, in step S11, the processing units 10A, 20B, 10B generate the color tone matrices 72y, 72cb, 72cr from the training image 70, but the training image 70 may be an image method converted into luminance Y, a first hue Cb, and a second hue Cr. That is, the processing units 10A, 20B, 10B may directly acquire the luminance Y, the first hue Cb, and the second hue Cr from, for example, a virtual slide scanner from the beginning. Similarly, in step S21, the processing units 20A, 20B, 10B generate the color tone matrices 72y, 72cb, 72cr from the analysis target image 78, but the processing units 20A, 20B, 10B may directly acquire the luminance Y, the first hue Cb, and the second hue Cr from, for example, a virtual slide scanner from the beginning.

[0225] In addition to RGB and CMY, YUV and CIE L * a * b * Various color spaces, such as RGB, ...

[0226] In the color tone vector data 74 and the color tone vector data 80, the color tone information is stored for each pixel in the order of brightness Y, first hue Cb, and second hue Cr, but the order in which the color tone information is stored and handled is not limited to this. However, it is preferable that the order of the color tone information in the color tone vector data 74 and the order of the color tone information in the color tone vector data 80 are the same.

[0227] In each field image analysis system, the processing units 10A and 10B are realized as an integrated device. However, the processing units 10A and 10B do not need to be integrated devices. The CPU 11, memory 12, storage unit 13, GPU 19, etc. may be located in separate locations and connected to each other via a network. The processing units 10A and 10B, the input unit 16, and the output unit 17 do not necessarily have to be located in one place, and may be located in separate places and connected to each other so as to be able to communicate with each other via a network. The processing units 20A, 20B, and 20C are similar to the processing units 10A and 10B.

[0228] In the above disease analysis support system, each functional block of the deep learning training data generation unit 101, the machine learning training data generation unit 101a, 101b, the deep learning training data input unit 102, the machine learning training data input unit 102a, 102b, the deep learning algorithm update unit 103, the machine learning algorithm update unit 103a, 103b, the analysis data generation unit 201, the analysis data input unit 202, and the analysis unit 203 is executed by a single CPU 11 or a single CPU 21, but each of these functional blocks does not necessarily have to be executed by a single CPU, and may be executed in a distributed manner by multiple CPUs. In addition, each of these functional blocks may be executed in a distributed manner by multiple GPUs, or may be executed in a distributed manner by multiple CPUs and multiple GPUs.

[0229] In the disease analysis support system, a program for performing the processing of each step described in Fig. 9 and Fig. 12 is pre-recorded in the storage units 13, 23. Alternatively, the program may be installed in the processing units 10B, 20B from a computer-readable, non-transitory, tangible recording medium 98, such as a DVD-ROM or a USB memory. Alternatively, the processing units 10B, 20B may be connected to a network 99, and the program may be downloaded from, for example, an external server (not shown) via the network 99 and installed.

[0230] In each disease analysis system, the input units 16, 26 are input devices such as a keyboard or a mouse, and the output units 17, 27 are realized as display devices such as a liquid crystal display. Alternatively, the input units 16, 26 and the output units 17, 27 may be integrated into a touch panel display device. Alternatively, the output units 17, 27 may be configured as a printer or the like.

[0231] In each of the disease analysis systems described above, the imaging device 300 is directly connected to the training device 100A or the disease analysis device 100B, but the imaging device 300 may be connected to the training device 100A or the disease analysis device 100B via a network 99. Similarly, the imaging device 400 is directly connected to the disease analysis device 200A or the disease analysis device 200B, but the imaging device 400 may be connected to the disease analysis device 200A or the disease analysis device 200B via a network 99.

[0232] [Effect of classifier] <Training deep learning and machine learning algorithms> Between 2017 and 2018, 1,165 peripheral blood (PB) smears from subjects with hematological disorders (myelodysplastic syndrome (n = 94), myeloproliferative disorders (mPD) and hematopoietic disorders (n = 106)) were obtained at Juntendo University Hospital. malignancies (n = 127), acute myeloid leukemia (n = 38), and acute lymphoblastic leukemia (n = 27). A total of 3,261 peripheral blood (PB) smears were evaluated, including pulmonary smears (n = 324), malignant lymphoma (n = 324), multiple myeloma (n = 82), and non-neoplastic hematological disorders (n = 473). The PB smear slides were stained with May Grunwald-Giemsa using a smear preparation device SP-10 (Sysmex Corporation). A total of 703,970 digitized cell images were obtained from the PB smear slides using an automatic blood image analyzer DI-60 (Sysmex Corporation). Training data 75 for deep learning was generated according to the training data generation method of the present invention.

[0233] A deep learning algorithm was used as the first computer algorithm. The deep learning algorithm used a Convolutional Neural Network (CNN) as the first neural network and a Fully Connected Neural Network (FCNN) as the second neural network to identify the cell type and the type of abnormal findings.

[0234] As the second computer algorithm, we used the machine learning algorithm Extreme Gradient Boosting (EGB) to build an automatic disease analysis support system.

[0235] The configuration of the classifier used in the examples is shown in Figure 22. In order to simultaneously detect the type of cell and the type of abnormal findings, a deep learning algorithm was systematized.

[0236] This deep learning algorithm has two main modules: the "CNN module" and the "FCNN module". The CNN module extracts features represented by color vector data from the images captured by the DI-60. The FCNN module extracts the features extracted by the CNN module. The system analyzes the data and classifies cell images into 17 cell types along with 97 abnormal findings, such as cell and nuclear size and shape, and cytoplasmic image patterns.

[0237] The "CNN module" consists of two submodules. The first (upstream) submodule has three congruence blocks, each of which consists of several convolutional networks. The stack of layers is divided into two parallel paths: the input image data and the output Optimize the extraction of features from the parameters to the next block. The model has eight consecutive blocks. Each block has two parallel paths, one with a sequence of convolutional layers and one path without convolutional components. This is called a Residual Network (ResNet). This ResNet acts as a buffer to prevent the system from saturating.

[0238] Separable convolution, exception-based convolution layer (Conv 2D), batch normalization layer (BN), and Each layer of the activation layer (ACT) has a different role. Separable convolutions are used in Xception. Conv 2D is a modified convolution called "feature extraction" that extracts features and processes the image to generate "feature It is the main component of neural networks that optimizes parameters in forming a "map". The ACT following the two layers of Conv 2D and BN is the Rectified Linear Unit (ReLU). The first submodule is connected to a second submodule consisting of eight consecutive similar blocks to create a feature map. Conv 2D is bypassed in the second module to avoid unexpected saturation of the deep layers, which leads to the effective computation of weights by backpropagation. The above deep convolutional neural network architecture is implemented with Keras and Tensorflow backends.

[0239] The results of identifying the 17 cell types and the types of abnormal findings for each of the 97 cell types, which were identified by the first computer algorithm, were used to train the machine learning algorithm. When identifying the types of abnormal findings for each cell type, neutrophils were linked to abnormal findings without distinguishing between segmented neutrophils and band neutrophils. In addition, among the abnormal findings shown in Figure 3, the items "Other abnormalities," "Pseudo-Chediak-Higashi granule-like," and "Other abnormalities (including aggregation)" for platelets were not analyzed. For XGBoost, first information was generated from the results of classifying the types of abnormal findings for each cell type, and second information was generated from the results of classifying the cell type, and these were input.

[0240] 703,970 digitized cells to train deep learning algorithms We split the images into 695,030 images for the training dataset and 8,940 images for the validation dataset.

[0241] To build the system, peripheral blood cells from 89 myelodysplastic syndrome (MDS) cases and 43 aplastic anemia (AA) cases were used for training. Next, PB smear images obtained from 26 MDS patients and 11 AA patients were used to verify the EGB-based automated disease analysis support system. did.

[0242] The cells used in the training were identified according to the morphological classification of the Clinical Laboratory Standards Institute (CLSI) H20-A2 guideline. The study was conducted by two board-certified hematology technicians and one senior hematology pathologist according to the WHO Classification of Myeloid Neoplasms and Acute Leukemias (2016 revised) and the WHO Classification of Myeloid Neoplasms and Acute Leukemias (2016 revised). Cell types and 97 abnormal findings were classified.

[0243] FIG. 23 shows the types and numbers of cell images for training and testing.

[0244] After training, the performance of the first computer algorithm was evaluated using the evaluation data set. Figure 24 shows the accuracy of the cell type discrimination results by the trained first computer algorithm. The sensitivity and specificity calculated by the ROC curve were good.

[0245] 25 shows the accuracy of the discrimination results of the types of abnormal findings by the first trained computer algorithm. The sensitivity, specificity, and AUC calculated by the ROC curve were good.

[0246] Thus, the discrimination accuracy of the first computer algorithm trained was shown to be good.

[0247] Next, a discriminator was used to distinguish between MDS and AA. Figure 26 shows the abnormalities of each cell type. The contribution of the types of findings is shown as a heat map of SHAP values. Each column of the heat map shown in Figure 26 corresponds to a sample from one patient, and each row corresponds to an abnormal finding for each cell type. The patients from the 1st column to the 26th column on the left side correspond to MDS patients, and the patients from the 27th column to the 37th column on the right side correspond to MDS patients. Patients up to the first eye correspond to AA patients. The detection rate is expressed by the shade of the heat map. From Figure 26, the detection rate of abnormal degranulation in neutrophils and abnormal granules in eosinophils in MDS patients, Furthermore, the detection rate of giant platelets was significantly higher in patients with AA than in those with AA.

[0248] Figure 27 shows the results of evaluating the accuracy of the classifier in the differential diagnosis of MDS and AA. The evaluation was performed using the sensitivity, specificity, and AUC calculated using the ROC curve. The sensitivity and specificity of the classifier were 96.2% and 100%, respectively, and the AUC of the ROC curve was 0.990. , and showed high accuracy in the differential diagnosis of MDS and AA.

[0249] The classifiers described above have been shown to be useful in aiding in the analysis of diseases. [Explanation of symbols]

[0250] 200 Disease analysis support device 20 Processing section 60 The First Neural Network 61 The Second Neural Network 62 The Second Neural Network 67 Machine Learning Algorithms 55 Disease Information 53 Information on types of abnormal findings 54 Cell Type Information 81 Analysis Data

Claims

1. classifying the morphology of each of a plurality of analysis target cells contained in a sample taken from a subject from images obtained from the cells, and acquiring information on a cell morphology classification corresponding to the sample based on the classification results; Analyzing the disease possessed by the subject by a computer algorithm based on the information on the cell morphology classification.

23. A method for assisting in disease analysis, comprising:

2. In the step of classifying the morphology of each of the analysis target cells, the type of each of the analysis target cells is identified. The method of claim 1.

3. The method according to claim 2 , wherein the information relating to the cell morphology classification is information relating to the number of cells of each cell type.

4. In the step of classifying the morphology of each of the cells to be analyzed, a type of abnormality in each of the cells to be analyzed is identified. The method according to claim 1 or 2.

5. The method according to claim 4 , wherein the information relating to the cell morphology classification is information relating to the number of cells for each type of abnormal finding.

6. The method according to claim 1 or 2, wherein in the step of classifying the morphology of each of the analyzed cells, a type of abnormal finding is identified for each type of the analyzed cells.

7. In the step of classifying the morphology of each of the analysis target cells, analysis data including information about each of the analysis target cells is input to a deep learning algorithm having a neural network structure, and the morphology of each of the analysis target cells is classified by the deep learning algorithm.

7. The method according to any one of claims 1 to 6.

8. the computer algorithm is a machine learning algorithm; The method according to claim 1 , wherein the analysis of the disease suffered by the subject is performed by inputting information about the cell morphology classification as features into the machine learning algorithm.

9. The method of claim 8 , wherein the machine learning algorithm is selected from trees, regression, neural networks, Bayesian, clustering, or ensemble learning.

10. The method of claim 9 , wherein the machine learning algorithm is gradient boosting trees.

11. 11. A method according to claim 1, wherein the step of acquiring information regarding the cell morphology classification comprises determining the probability that each analyzed cell belongs to each of a plurality of cell morphology classifications, calculating the sum of the probabilities for each type of cell morphology classification, and acquiring the sum as information regarding the cell morphology classification.

12. The method of any one of claims 1 to 11, wherein the sample is a blood sample.

13. The method of any one of claims 1 to 12, wherein the disease is a hematopoietic disease.

14. The hematopoietic disease is aplastic anemia or myelodysplastic syndrome. The method of claim 13.

15. The method according to any one of claims 4 to 6, wherein the abnormal findings include at least one selected from the group consisting of nuclear morphological abnormalities, granular abnormalities, abnormal cell size, cell malformations, cell destruction, empty cells, immature cells, the presence of inclusion bodies, Dohle bodies, satellite phenomena, nuclear reticulum abnormalities, petal-like nuclei, large N / C ratios, bleb-like, smudge and hairy cell-like morphology.

16. The nuclear morphological abnormality includes at least one selected from hyper-lobulation, hypo-lobulation, pseudo-Pelger nuclear anomaly, ring-shaped nucleus, spherical nucleus, oval nucleus, apoptosis, multinucleation, karyorrhexis, enucleation, bare nucleus, irregular nuclear margin, nuclear fragmentation, internuclear bridge, multiple nuclei, cleaved nucleus, nuclear division, and nucleolar abnormality; The granular abnormality includes at least one selected from degranulation, granule distribution abnormality, toxic granules, Auer rods, Bassoon cells, and pseudo-Chediak-Higashi granule-like granules; The cell size abnormality includes giant platelets. The method of claim 15.

17. The method of claim 2 or 3, wherein the cell types include at least one selected from neutrophils, eosinophils, platelets, lymphocytes, monocytes, and basophils.

18. The method of claim 17, wherein the cell types further include at least one selected from metamyelocytes, myelocytes, promyelocytes, blasts, plasma cells, atypical lymphocytes, immature eosinophils, immature basophils, erythroblasts, and megakaryocytes.

19. An apparatus for assisting disease analysis, comprising: A processing unit is provided, The processing unit classifies the morphology of each of a plurality of analysis target cells contained in a sample collected from a subject from an image obtained from the cells, and acquires information on a cell morphology classification corresponding to the sample based on the classification result. Analyzing the disease possessed by the subject by a computer algorithm based on the information on the cell morphology classification. Device.

20. A step of classifying the morphology of each of a plurality of analysis target cells contained in a sample taken from a subject from images obtained from the cells, and acquiring information on the cell morphology classification corresponding to the sample based on the classification result; analyzing the disease possessed by the subject by a computer algorithm based on the information on the cell morphology classification; A program to assist in disease analysis by having a computer execute the following:

21. 1. A method for training a computer algorithm to assist in disease analysis, comprising: classifying the morphology of each of a plurality of analysis target cells contained in a sample taken from a subject from images obtained from the cells, and acquiring information on a cell morphology classification corresponding to the sample based on the classification results; A training method comprising: inputting the acquired information on the cell morphology classification as first training data and disease information on the subject as second training data into the computer algorithm.

22. 1. A training device for a computer algorithm for assisting in disease analysis, comprising: A processing unit is provided, The processing unit classifies the morphology of each of a plurality of cells to be analyzed contained in a sample taken from a subject from images obtained from the cells to be analyzed, obtains information regarding a cell morphology classification corresponding to the sample based on the classification results, and inputs the obtained information regarding the cell morphology classification as first training data and disease information of the subject as second training data into the computer algorithm.

23. 1. A program for training a computer algorithm for assisting in disease analysis, comprising: A step of classifying the morphology of each of a plurality of analysis target cells contained in a sample taken from a subject from images obtained from the cells, and acquiring information on the cell morphology classification corresponding to the sample based on the classification result; and inputting information relating to the cell morphology classification as first training data and disease information of the subject as second training data into a computer algorithm.

Images