Cellular analysis method, apparatus, system and program

Abstract

This invention aims to simplify the analysis of multiple analytical projects. The problem is solved by a cell analysis method, which includes: generating analytical data of cells contained in a sample; selecting an artificial intelligence algorithm from multiple artificial intelligence algorithms, wherein the artificial intelligence algorithm receives the generated analytical data as input; and generating data displaying the characteristics of the cells based on the analytical data.

Description

[Technical Field]

[0001] This invention relates to cell analysis methods, cell analysis apparatus, cell analysis systems, and cell analysis procedures for analyzing cells. [Background Technology]

[0002] Patent Document 1 discloses a method for applying filtered microscope images to a trained machine learning model, determining the center and boundary of a specific type of cell, counting the determined cells, and outputting an image of the cell.

[0003] [Existing Technical Documents]

[0004] [Patent Documents]

[0005] [Patent Document 1] International Publication No. 2015 / 065697

[0006] [Summary of the Invention]

[0007] [The problem the invention aims to solve]

[0008] During the inspection, multiple parsing items are parsed. However, as the number of parsing items increases, it becomes necessary to increase the number of training iterations of the machine learning model or increase the variety of parameters input to the machine learning model in order to reduce judgment errors. As a result, training becomes more time-consuming and the machine learning model becomes larger.

[0009] The present invention aims to provide a cell analysis method, cell analysis apparatus, cell analysis system, and cell analysis program that facilitate the analysis of multiple analysis items.

[0010] [Methods used to solve problems]

[0011] One embodiment of the present invention relates to a cell analysis method for analyzing cells. The cell analysis method generates analysis data (82, 87, 585) of cells contained in a sample, selects an artificial intelligence algorithm (60, 63, 560) from a plurality of artificial intelligence algorithms (60, 63, 560), the artificial intelligence algorithm accepts the generated analysis data (82, 87, 585) as input, and generates data (84, 88, 582) showing the characteristics of the cells based on the analysis data (82, 87, 585) using the selected artificial intelligence algorithm (60, 63, 560, 563).

[0012] One embodiment of the present invention relates to a cell analysis apparatus (400A, 200T) for analyzing cells. The cell analysis apparatus (400A, 200T) includes a control unit (40A, 20T) configured to select an artificial intelligence algorithm (60, 63, 560) from a plurality of artificial intelligence algorithms (60, 63, 560). The artificial intelligence algorithm receives input data (82, 87, 585) for analyzing cells contained in a sample, and generates data (84, 88, 582) displaying the characteristics of the cells based on the analysis data (82, 87, 585).

[0013] One embodiment of the present invention relates to a cell analysis system (1000). The cell analysis system (1000) includes: a flow cell (110) through which a sample containing cells flows, a light source (120, 121, 122, 123) for irradiating the sample flowing through the flow cell (110) with light, an imaging unit (160) for capturing images of cells in the sample irradiated with the light, and a control unit (40A). The control unit (40A) is configured such that: a sample containing cells flows through a flow path (111), and an artificial intelligence algorithm (60, 63) is selected from a plurality of artificial intelligence algorithms (60, 63). The artificial intelligence algorithm accepts input of analytical data (82, 87) generated from analytical object images (80, 85) of cells passing through the flow path (111). The selected artificial intelligence algorithm (60, 63) generates data (84, 88) based on the analytical data (82, 87, 85) that displays the characteristics of the cells contained in the analytical object images (80, 85).

[0014] In one embodiment of the present invention, the cell analysis system (5000) includes: a flow path (4113a) through which a cell-containing sample flows, a signal acquisition unit (610) for acquiring signals from cells in the sample flowing through the flow path, and a control unit (20T). The control unit (20T) is configured to: allow the cell-containing sample to flow through the flow path (4113a), acquire the signal intensity of each cell passing through the flow path (4113a), generate analysis data (585) from the acquired signal intensity, select an artificial intelligence algorithm (560, 563) from a plurality of artificial intelligence algorithms (560, 563), wherein the artificial intelligence algorithm receives input of the analysis data (585), and the selected artificial intelligence algorithm (560, 563) generates data (582) displaying the characteristics of the cells based on the analysis data (585).

[0015] One embodiment of the present invention relates to a cell analysis system (1000). The cell analysis system (1000) includes: a microscope 700 with a stage on which a slide containing a cell-coated sample is placed; an imaging unit (710d) that captures images of cells in the sample magnified by the microscope; and a control unit (40A). The control unit (40A) selects an artificial intelligence algorithm (60, 63) from a plurality of artificial intelligence algorithms (60, 63). The artificial intelligence algorithm receives input analysis data (82, 87) generated from analysis object images (80, 85) of cells contained in the sample coated on the slide, magnified by the microscope (700). The selected artificial intelligence algorithm (60, 63) generates data (84, 88) based on the analysis data (82, 87) displaying the characteristics of the cells contained in the analysis object images (80, 85).

[0016] One embodiment of the present invention relates to a cell analysis program for analyzing cells. The cell analysis program causes a computer to perform a process comprising the following steps: selecting an artificial intelligence algorithm (60, 63, 560) from a plurality of artificial intelligence algorithms (60, 63, 560), wherein the artificial intelligence algorithm accepts input of analysis data (82, 87, 585) of cells contained in a sample; and generating data (84, 88, 582) showing the characteristics of the cells based on the analysis data (82, 87, 585) by the selected artificial intelligence algorithm (60, 63, 560).

[0017] [The effects of the invention]

[0018] According to the present invention, in cell analysis, the analysis of multiple analysis items is made easier.

[0019] [Brief explanation of the attached image]

[0020]

【 Figure 1 This section presents a summary of the present invention.

[0021]

【 Figure 2 [This section shows the method for generating training data for the first artificial intelligence algorithm 50 used to analyze chromosomal abnormalities. (A) Shows the method for generating positive training data. (B) Shows the method for generating negative training data.]

[0022]

【 Figure 3 This describes the method for generating training data for the first artificial intelligence algorithm 50 used to analyze chromosomal abnormalities.

[0023]

【 Figure 4 The diagram shows the method for generating data for analyzing chromosomal abnormalities and the method for analyzing cells trained by the first artificial intelligence algorithm 60.

[0024]

【 Figure 5 The image shows the staining pattern of PML-RARA chimeric gene-positive cells obtained using imaging flow cytometry. (A) shows the image of channel 2 on the left and right. (B) is a different cell from (A), with the image of channel 2 shown on both the left and right sides.

[0025]

【 Figure 6 [Image showing a fluorescent label]

[0026]

【 Figure 7 [Image showing a fluorescent label]

[0027]

【 Figure 8 [Image showing a fluorescent label]

[0028]

【 Figure 9 This section describes the method for generating training data for the second artificial intelligence algorithm 53 used to analyze peripheral circulating tumor cells.

[0029]

【 Figure 10 [This section shows the method for generating training data for the second artificial intelligence algorithm 53 used to analyze peripheral circulating tumor cells. (A) Shows the method for generating positive training data. (B) Shows the method for generating negative training data.]

[0030]

【 Figure 11 This section describes the method for generating training data for the second artificial intelligence algorithm 53 used to analyze peripheral circulating tumor cells.

[0031]

【 Figure 12 The diagram shows the method for generating data for analyzing peripheral circulating tumor cells and the method for analyzing cells trained by the second artificial intelligence algorithm 63.

[0032]

【 Figure 13 This shows the hardware configuration of the Cell Analysis System 1000.

[0033]

【 Figure 14 The hardware configuration of the training device 200A is shown.

[0034]

【 Figure 15 [Displays the function blocks of the training device 200A.]

[0035]

【 Figure 16 The flowchart shows the training process of the first artificial intelligence algorithm.

[0036]

【 Figure 17 The hardware configuration of the cell imaging device 100A and the cell analysis device 400A is shown.

[0037]

【 Figure 18 [This displays the functional blocks of the Cell Analysis Device 400A.]

[0038]

【 Figure 19 The flowchart for cell analysis and processing is displayed.

[0039]

【 Figure 20 This section shows an example of an algorithm database that contains the artificial intelligence algorithms described in the first embodiment.

[0040]

【 Figure 21 The cell imaging device is a hardware component of a microscope.

[0041]

【 Figure 22 This shows the structure of the optical system of the microscope.

[0042]

【 Figure 23 This section shows an example of how the training data was generated.

[0043]

【 Figure 24 [Example of displaying label values]

[0044]

【 Figure 25 This section shows an example of how the data is generated for parsing.

[0045]

【 Figure 26 Example showing the appearance of the Cell Analysis System 5000.

[0046]

【 Figure 27 [Example of the functional configuration of the measurement unit 600]

[0047]

【 Figure 28 [A schematic example of the optical system of the nucleated cell detection unit 611 of the display measurement unit 600.]

[0048]

【 Figure 29 This is a schematic example of the sample modulation section 640 of the measurement unit 600.

[0049]

【 Figure 30 This section shows an example of the hardware configuration of the training device.

[0050]

【 Figure 31 This section shows an example of the hardware configuration of a cell analysis device.

[0051]

【 Figure 32 [Example of the functional configuration of the training device]

[0052]

【 Figure 33 This section shows an example of the functional configuration of a cell analysis device.

[0053]

【 Figure 34 The flowchart shows the training process of the 3rd and 4th artificial intelligence algorithms.

[0054]

【 Figure 35 The flowchart for cell analysis and processing is displayed.

[0055]

【 Figure 36 The flowchart for cell analysis and processing is displayed.

[0056]

【 Figure 37 This shows an example of receiving a screen in parsing mode.

Detailed Implementation Methods

[0057]

I. Summary of Embodiments of the Invention

[0058] use Figure 1 The following is a summary description of the embodiments of the present invention.

[0059] Embodiments of the present invention relate to a cell analysis method for analyzing cells. For example... Figure 1 As shown, the cell analysis method is characterized by generating analytical data 82, 87, or 585 of the cells contained in the sample, and selecting an artificial intelligence algorithm from multiple artificial intelligence algorithms 405(a), 405(b), T205(a), and T205(b) corresponding to the examination item, wherein the artificial intelligence algorithm accepts the generated analytical data as input. The selected artificial intelligence algorithm generates data from the input analytical data that displays the characteristics of the cells of the analytical object.

[0060] like Figure 1 As shown, when analyzing cells, flow cytometry or microscopy can be used to obtain images of the cells or waveform data based on the cell signal intensity. The specific data collected from the cells corresponds to examination or analysis items such as chromosomal abnormality testing, peripheral circulating tumor cell testing, peripheral blood testing, and urine testing, which are predetermined. Therefore, in this invention, an artificial intelligence algorithm suitable for each analysis item is selected corresponding to the examination or analysis item.

[0061]

II. First Implementation

[0062] The first embodiment relates to a method for analyzing cells from images of cells using artificial intelligence algorithms.

[0063] [1. Overview of Cell Analysis Methods]

[0064] This embodiment relates to a cell analysis method that uses an artificial intelligence algorithm to analyze cells. In this cell analysis method, a cell-containing sample is flowed through a flow path to capture an image of the cells to be analyzed, and the cells passing through the flow path are captured. Analysis data, used as input to the artificial intelligence algorithm, is generated from the acquired image of the cell to be analyzed. When analysis data is input to the artificial intelligence algorithm, the algorithm generates data displaying the characteristics of the cells contained in the image of the cell to be analyzed. Preferably, the image of the cell to be analyzed is captured one by one as the cells pass through the flow path.

[0065] In this embodiment, the sample may be a sample prepared from a test body collected from a subject. The test body may contain, for example, blood test bodies such as peripheral blood, venous blood, and arterial blood, urine test bodies, and body fluid test bodies other than blood and urine. As body fluids other than blood and urine, they may include bone marrow, ascites, pleural effusion, and medullary fluid. Sometimes, body fluids other than blood and urine are simply referred to as "body fluids". The blood is preferably peripheral blood. For example, blood may be peripheral blood collected using an anticoagulant such as sodium or potassium ethylenediaminetetraacetate (EDTA) or sodium heparin.

[0066] The preparation of the sample from the test subject can be performed according to known methods. For example, the examiner can recover nucleated cells by centrifugation using a cell separation medium such as Ficoll on a blood sample collected from the subject. Alternatively, instead of centrifuging, a hemolytic agent can be used to lyse red blood cells to retain the nucleated cells. The target sites of the recovered nucleated cells are labeled using at least one of the following methods: fluorescence in situ hybridization (FISH), immunostaining, and organelle staining, preferably fluorescently labeled. The suspension of the labeled cells is then used as a sample for imaging, such as with an imaging flow cytometer, to obtain images of the cells.

[0067] The sample may contain multiple cells. There is no particular limitation on the number of cells in the sample, but it must be at least 10. 2 More than 10, preferably 10 3 More than one, preferably 10 4 More than 10, then select the best 10. 5 More than one, or more preferably 10 6 More than one. In addition, multiple cells may contain different types of cells.

[0068] In this embodiment, the cells that will also be the subject of analysis are referred to as the target cells. The target cells can be cells contained in a test subject collected from the subject. Preferably, these cells can be nucleated cells. The cells may contain both normal and abnormal cells.

[0069] Normal cells refer to cells that should normally be present in the test subject at the site of collection. Abnormal cells refer to cells other than normal cells. Abnormal cells may include cells with chromosomal abnormalities and / or tumor cells. Preferably, the tumor cells are peripheral circulating tumor cells. More preferably, peripheral circulating tumor cells do not refer to hematopoietic tumor cells that are present in the blood under normal pathological conditions, but rather to tumor cells originating from cell lines other than the hematopoietic cell system circulating in the blood. In this specification, tumor cells circulating in the peripheral circulation are also referred to as circulating tumor cells (CTCs).

[0070] The target site for detecting chromosomal abnormalities is the nucleus of the target cell. Examples of chromosomal abnormalities include transposition, deletion, inversion, and duplication. Cells exhibiting such chromosomal abnormalities include, for example, cells found in diseases such as myelodysplastic syndromes, acute myeloid leukemia, acute promyelocytic leukemia, acute myelomonocytic leukemia, acute monocytic leukemia, erythroleukemia, acute megakaryoblastic leukemia, acute myeloid leukemia, acute lymphoblastic leukemia, lymphoblastic leukemia, chronic myeloid leukemia, or chronic lymphocytic leukemia, as well as malignant lymphomas such as Hodgkin's lymphoma, non-Hodgkin's lymphoma, and multiple myeloma.

[0071] Chromosomal abnormalities can be detected by known methods such as FISH. Generally, the tests used to detect chromosomal abnormalities are set according to the type of abnormal cell to be detected. The genes or loci to be analyzed are set according to the specific tests performed on the subject. In the detection of chromosomal abnormalities by FISH, abnormalities in the position or number of chromosomes can be detected by hybridizing probes that specifically bind to genes present in the nucleus of the cell to be analyzed or to loci. The probes are labeled with a labeling substance. The labeling substance is preferably a fluorescent dye. Depending on the probe, when the labeling substance is a fluorescent dye, combinations of fluorescent dyes with different fluorescence wavelength regions can be used to detect multiple genes or loci in a single cell.

[0072] Abnormal cells are cells that appear when a specific disease is present, and may include, for example, tumor cells such as cancer cells, leukemia cells, etc. In the case of hematopoietic organs, the specified diseases may be diseases selected from myelodysplastic syndromes, acute myeloid leukemia, acute promyelocytic leukemia, acute myeloid monocytic leukemia, acute monocytic leukemia, erythroleukemia, acute megakaryoblastic leukemia, acute myeloid leukemia, acute lymphoblastic leukemia, lymphoblastic leukemia, chronic myeloid leukemia, or chronic lymphocytic leukemia, malignant lymphomas such as Hodgkin's lymphoma, non-Hodgkin's lymphoma, and multiple myeloma. In addition, in cases involving organs other than the hematopoietic system, the specified diseases may include malignant tumors of the digestive tract originating from the upper pharynx, esophagus, stomach, duodenum, jejunum, ileum, cecum, appendix, ascending colon, transverse colon, descending colon, S-shaped colon, rectum, or anus; liver cancer; gallbladder cancer and bile duct cancer; pancreatic cancer; pancreatic duct cancer; malignant tumors of the urinary tract originating from the bladder, ureter, or kidneys; malignant tumors of the female reproductive system originating from the ovaries, fallopian tubes, and uterus; breast cancer; prostate cancer; skin cancer; malignant tumors of the endocrine system, including the hypothalamus, pituitary gland, thyroid gland, parathyroid gland, adrenal gland, and pancreas; malignant tumors of the central nervous system; and solid tumors of the bone and soft tissues.

[0073] Abnormal cells can be detected using at least one of the following: bright-field images, immunostaining images for various antigens, and organelle staining images specifically stained for organelles.

[0074] Bright-field images are obtained by illuminating cells and capturing either transmitted or reflected light from the cells. Preferably, bright-field images are images of the phase difference of cells captured using transmitted light.

[0075] Immunostaining images are obtained by imaging cells that have undergone immunostaining using antibodies that bind to antigens present at at least one intracellular or cellular site selected from the nucleus, cytoplasm, and cell surface. The labeling material is preferably a fluorescent dye, similar to that used in FISH methods. Depending on the antigen, when the labeling material is a fluorescent dye, combinations of fluorescent dyes with different fluorescence wavelength regions can be used to detect multiple antigens in a single cell.

[0076] Organelle staining images can be obtained by photographing cells stained with dyes that selectively bind to proteins, glycans, lipids, or nucleic acids present in at least one intracellular or cell membrane site selected from the nucleus, cytoplasm, and cell membrane. Examples of nuclear-specific staining dyes include DNA-binding dyes such as Hoechst 33342, Hoechst 33258, 4',6-diamidino-2-phenylindole (DAPI), Propidium Iodide (PI), and ReadyProbes nuclear staining reagent, as well as histone-binding reagents such as CellLight. Examples of RNA-specific staining reagents include SYTO RNASelect, which binds specifically to RNA. Examples of cytoskeleton-specific staining reagents include fluorescently labeled phalloidin. For staining other organelles such as lysosomes, vesicles, Golgi apparatus, and mitochondria, dyes such as the CytoPainter series from Abcam (Abcam plc, Cambridge, UK) can be used. These staining dyes or reagents are fluorescent dyes or reagents containing fluorescent dyes, and different fluorescence wavelength regions can be selected corresponding to the fluorescence wavelength regions of the fluorescent dyes used in other staining of organelles or cells.

[0077] When detecting abnormal cells, the testing items are set according to the type of abnormal cell being detected. The testing items may include analytical items necessary for detecting abnormal cells. These analytical items can be set to correspond to the bright-field image, each antigen, and each organelle mentioned above. Fluorescent dyes with different fluorescence wavelength regions correspond to the analytical items other than the bright-field image, allowing different analytical items to be detected in a single cell.

[0078] The data used as input to the artificial intelligence algorithm for analysis is obtained by the method described later. The data generated by the artificial intelligence algorithm, showing the characteristics of cells contained in the image being analyzed, indicates, for example, whether the cells are normal or abnormal. More specifically, the data showing the characteristics of cells contained in the image being analyzed indicates whether the cells are cells with chromosomal abnormalities or peripheral circulating tumor cells.

[0079] In this specification, for the convenience of description, "analyzed object image" is sometimes referred to as "analyzed image," "analyzed data" as "analyzed data," "training image" as "training image," and "training data" as "training data." Additionally, "fluorescent image" refers to either a training image of a fluorescent label or an analyzed image of a fluorescent label.

[0080] [2. Cell analysis method using the first artificial intelligence algorithm]

[0081] The training methods for the first artificial intelligence algorithm 50 and the second artificial intelligence algorithm 53, and the analysis methods for the cells trained by the first artificial intelligence algorithm 60 and the second artificial intelligence algorithm 63, are used. Figures 2 to 12 The following explanation is provided. The first and second artificial intelligence algorithms, 60 and 63, can be deep learning algorithms with neural network structures. The aforementioned neural network structures can be selected from fully connected deep neural networks (FC-DNN), folded neural networks (CNN), autoregressive neural networks (RNN), and combinations thereof. Folded neural networks are preferred.

[0082] Artificial intelligence algorithms can be used, for example, from Python companies.

[0083] [2-1. Artificial Intelligence Algorithms for Detecting Chromosomal Abnormalities]

[0084] This embodiment relates to a training method for a first artificial intelligence algorithm 50 for detecting chromosomal abnormalities and a method for parsing cells using the trained first artificial intelligence algorithm 60 for detecting chromosomal abnormalities. The term "training" or "training" is sometimes replaced with "generation" or "generation".

[0085] (1) Generation of training data

[0086] use Figure 2 and Figure 3 The training method for the first artificial intelligence algorithm 50 used to detect chromosomal abnormalities is explained. Figure 2 An example of FISH staining of the PML-RARA chimera gene, formed by transposition of the PML gene (a transcriptional control factor located on the long arm of chromosome 15 (15q24.1)) and the retinoic acid receptor α (RARA) gene located on the long arm of chromosome 17 (17q21.2), is shown.

[0087] like Figure 2 As shown, positive training image 70P, representing cells with positive chromosome abnormalities (hereinafter referred to as "positive control cell 1"), and negative training image 70N, representing cells with negative chromosome abnormalities (hereinafter referred to as "negative control cell 1"), are used to generate labeled positive integrated training data 73P and labeled negative integrated training data 73N, respectively. Sometimes, positive training image 70P and negative training image 70N are referred to together as training image 70. Additionally, sometimes labeled positive integrated training data 73P and labeled negative integrated training data 73N are referred to together as training data 73.

[0088] In the example of detecting the PML-RARA chimeric gene, a first fluorescent dye is shown that causes the probe detecting the PML locus to bind to a green wavelength region of fluorescence, while a second fluorescent dye is shown that causes the probe detecting the RARA locus to bind to a red wavelength region of fluorescence, different from the first fluorescent dye. The nuclei of a first positive control cell and a first negative control cell can be labeled with the first and second fluorescent dyes respectively by FISH using probes that bind the first and second fluorescent dyes. Sometimes, labeling of the target site with the first fluorescent dye is referred to as first fluorescent labeling, and labeling of the target site with the second fluorescent dye is referred to as second fluorescent labeling.

[0089] A sample containing cells labeled with a first fluorescent marker and a second fluorescent marker can be analyzed in a cell imaging device such as an imaging flow cytometer to capture images of the cells. Multiple images of the cells can be captured for the same field of view of the same cells. Because the first and second fluorescent markers have different wavelength regions of fluorescence due to their respective fluorescent dyes, the first filter for transmitting light emitted from the first fluorescent dye and the second filter for transmitting light emitted from the second fluorescent dye are different. Therefore, the light transmitted through the first filter and the light transmitted through the second filter are received by the imaging unit 160 (described later) via their respective first and second channels, and captured as separate images of the same field of view of the same cells. That is, in the imaging unit 160, multiple images corresponding to the number of labeled substances on the labeled cells are acquired for the same field of view of the same cells.

[0090] Therefore, in Figure 2 In the example, such as Figure 2 As shown in (A), the positive training image 70P may contain a first positive training image 70PA (green fluorescent label) captured via the first channel and a second positive training image 70PB (red fluorescent label) captured via the second channel for the first positive control cell. The first positive training image 70PA and the second positive training image 70PB correspond to images of the same field of view for the same cells. Next, the brightness of the captured light in each pixel of the image is transformed into numerical training data 71PA and 71PB for the first and second positive cells, respectively.

[0091] Using the first positive training image 70PA, the method for generating the first positive numerical training data 71PA will be explained. Each image captured by the camera unit 160 is adjusted to, for example, 100 pixels vertically × 100 pixels horizontally, to generate the training image 70. At this time, the image is adjusted so that the images obtained from each channel have the same field of view for each cell. The first positive training image 70PA is represented as, for example, a 16-bit grayscale image. Therefore, the brightness of each pixel can be represented by a brightness value of 65,536 color levels from 1 to 65,536. Figure 2 As shown in (A), the value of the brightness level in each pixel of the first positive training image 70PA is the first positive numerical training data 71PA, represented by rows and columns corresponding to the numbers of each pixel.

[0092] Similar to the first positive numerical training data 71PA, the second positive numerical training data 71PB, which represents the brightness of the captured light in each pixel of the image, can be generated from the second positive training image 70PB.

[0093] Next, for each pixel, the first positive value training data 71PA and the second positive value training data 71PB are integrated to generate positive integrated training data 72P. For example... Figure 2 As shown in (A), the positive integrated training data 72P becomes the row and column data in which the values ​​of each pixel of the first positive numerical training data 71PA and the values ​​of each corresponding pixel of the second positive numerical training data 71PB are displayed side by side.

[0094] Next, the label value 74P, derived from the label of the first positive control cell, is shown attached to the positive integration training data 72P. Labels are then generated along with the positive integration training data 73P. As a label indicating that it is the first positive control cell, in... Figure 2 (A) Appendix “2”.

[0095] The negative training image 70N is generated from the negative integrated training data 73N in the same way as when generating the labeled integrated training data 73P with positive data.

[0096] like Figure 2As shown in (B), the negative training image 70N contains a first negative training image 70NA, captured via the first channel, showing a green first fluorescent label, and a second negative training image 70NB, captured via the second channel, showing a red second fluorescent label, for the first negative control cells. The imaging and retouching, and the numerical representation of the light intensity in each pixel, are performed in the same manner as when obtaining the first positive numerical training data 71PA from the first positive training image 70PA. The first negative numerical training data 71NA, which numerically represents the light intensity in each pixel of the image, can be generated from the first negative training image 70NA using the same method as the first positive numerical training data 71PA.

[0097] Similarly, second negative numerical training data 71NB, which represents the brightness of the captured light in each pixel of the image, can be generated from the second negative training image 70NB.

[0098] like Figure 2 As shown in (B), based on the method for generating positive integrated training data 72P, the first negative numerical training data 71NA and the second negative numerical training data 71NB are integrated for each pixel to generate negative integrated training data 72N. Figure 2 As shown in (B), the negative integrated training data 72N becomes row and column data in which the values ​​in each pixel of the first negative numerical training data 71NA and the values ​​in each corresponding pixel of the second negative numerical training data 71NB are displayed side by side.

[0099] Next, the label value 74N, derived from the label value 72N of the first negative control cell, is shown attached to the negative integration training data 72N, and a label is generated along with the negative integration training data 73N. As a label indicating that it is the first negative control cell, in... Figure 2 (B) Appendix “1”.

[0100] exist Figure 3The method of inputting label-attached positive integrated training data 73P and label-attached negative integrated training data 73N into the first artificial intelligence algorithm 50 is shown. The number of nodes in the input layer 50a of the first artificial intelligence algorithm 50, which has a neural network structure, corresponds to the product of the number of pixels in the training image 70 (100 × 100 = 10,000 in the example above) and the number of channels per cell (two channels, green and red, in the example above). Data corresponding to the label-attached positive integrated training data 72P is input to the input layer 50a of the neural network. The label value 74P corresponding to the data input to the input layer 50a is input to the output layer 50b of the neural network. Additionally, data corresponding to the label-attached negative integrated training data 72N is input to the input layer 50a of the neural network. The label value 74N corresponding to the data input to the input layer 50a is input to the output layer 50b of the neural network. The weights in the intermediate layer 50c of the neural network are calculated from these inputs, the first artificial intelligence algorithm 50 is trained, and the trained first artificial intelligence algorithm 60 is generated.

[0101] (2) Data generation and cell analysis

[0102] use Figure 4 The method for generating integrated parsed data 72 from the parsed image 80 and the cell parsing method using the trained first artificial intelligence algorithm 60 will be described. The parsed image 80 can be captured in the same way as the training image 70.

[0103] like Figure 4 As shown, the resolved image 80 includes a first resolved image 80A (captured via the first channel with a green first fluorescent label) and a second resolved image 80B (captured via the second channel with a red second fluorescent label) for the target cell. The imaging and retouching, and the numerical representation of the light intensity in each pixel, are performed in the same manner as when obtaining the first positive numerical training data 71PA from the first positive training image 70PA. The first numerical resolved data 81A, which represents the light intensity captured in each pixel of the image, can be generated from the first resolved image 80A using the same method as the first positive numerical training data 71PA.

[0104] Similarly, second numerical analysis data 81B, which represents the brightness of the captured light in each pixel of the image, can be generated from the second analyzed image 80B.

[0105] like Figure 4 As shown, based on the method for generating positive integrated training data 72P, integrated parsing data 82 is generated by integrating the first numerical parsing data 81A and the second numerical parsing data 81B for each pixel. Figure 4As shown, the integrated parsing data 82 is displayed side-by-side with the values ​​of each pixel in the first numerical parsing data 81A and the corresponding values ​​in each pixel in the second numerical parsing data 81B.

[0106] like Figure 4 As shown, the generated integrated parsing data 82 is input into the input layer 60a of the neural network within the first trained artificial intelligence algorithm 60. The values ​​contained in the input integrated parsing data 82 are processed through the intermediate layer 60c of the neural network and output from the output layer 60b of the neural network a label value 84 indicating whether the parsed cell has a chromosomal abnormality. Figure 4 In the example shown, when the parsed cell is determined to have no chromosomal abnormalities, the output is "1" as the label value; when it is determined to have chromosomal abnormalities, the output is "2" as the label value. Alternatively, labels such as "none," "present," "normal," or "abnormal" can be output instead of the label value.

[0107] (3) Other components

[0108] i. In this embodiment, sometimes in imaging flow cytometry, an Extended Depth of Field (EDF) filter, used to magnify the depth of field when photographing cells, is used to restore the depth of focus of the image after imaging, thus providing the examiner with a cell image. However, in this embodiment, the training image 70 and the resolved image 80 are preferably images that have not undergone restoration processing, taken using an EDF filter. Examples of images that have not undergone restoration processing are shown in... Figure 5 . Figure 5 Cells showing positive PML-RARA chimera genes. (A) and (B) are images of different cells. Figure 5 (A) and Figure 5 (B) The image on the left shows the image of the first fluorescent label. Figure 5 (A) and Figure 5 (B) The image on the right is the same cell as the one on the left, showing the image of the second fluorescently labeled cell in the same field of view as the image on the left.

[0109] ii. From the training image 70 and the parsed image 80 during the shooting process, images with out-of-focus conditions can be excluded. Whether an image is in focus or not can be determined by calculating the difference in brightness between each pixel and its neighboring pixels. If there are no extreme parts of the gradient of the difference in the overall image, the image can be judged as having out-of-focus conditions.

[0110] iii. In this embodiment, the training image 70 and the parsed image 80 are exemplarily modified so that the number of pixels is 100 pixels vertically × 100 pixels horizontally, but the image size is not limited to this. The number of pixels can be appropriately set between 50 and 500 pixels vertically and between 50 and 500 pixels horizontally. It is not necessary for the number of pixels in the vertical direction and the number of pixels in the horizontal direction of the image to be the same. However, it is preferable that the training image 70 used to train the first artificial intelligence algorithm 50 and the parsed image 80 used to generate the integrated parsed data 82 input to the first artificial intelligence algorithm 60 trained using the training image 70 have the same number of pixels, and the number of pixels in the vertical and horizontal directions are also the same.

[0111] iv. In this embodiment, the training image 70 and the analyzed image 80 are 16-bit grayscale images. However, the brightness levels can be 8-bit, 32-bit, or other methods besides 16-bit. Furthermore, in this embodiment, the brightness values ​​represented in 16 bits (65,536 levels) are directly used for each numerical training data 71PA, 71PB, 71NA, and 71NB. However, it is also possible to perform a low-dimensionality processing by combining these values ​​with a certain range of levels, and use the low-dimensional values ​​as the numerical training data 71PA, 71PB, 71NA, and 71NB. In this case, it is preferable to perform the same processing on the training image 70 and the analyzed image 80.

[0112] v. In this embodiment, the detectable chromosomal abnormalities are not limited to the PML-RARA chimera gene. For example, the following can be detected: BCR / ABL fusion gene, AML1 / ETO(MTG8) fusion gene (t(8; (21)), PML / RARα fusion gene (t(15; (17)), AML1(21q22) transposition, MLL(11q23) transposition, TEL(12p13) transposition, TEL / AML1 fusion gene (t(12; (21)), IgH(14q32) transposition, CCND1(BCL1) / IgH fusion gene (t(11; (14)), BCL2(18q21) transposition, IgH / MAF fusion gene (t(14; (16)), IgH / BCL2 fusion gene (t(14; (18)), c-myc / IgH Fusion genes (t(8;(14)), FGFR3 / IgH fusion gene (t(4;(14)), BCL6(3q27) transposition, c-myc(8q24) transposition, MALT1(18q21) transposition, API2 / MALT1 fusion gene (t(11;(18) transposition), TCF3 / PB×1 fusion gene (t(1;(19) transposition), EWSR1(22q12) transposition, PDGFRβ(5q32) transposition, IGH-CCND1 gene [(IGH-BCL1)(t(11;(14) transposition)], IGH-FGFR3 gene (t(4;(14) transposition), IgH-MAF gene (t(14;(16) transposition), etc.

[0113] Furthermore, transposition can contain a wide variety of variations. Figure 6 and Figure 7 Examples of fluorescent markers displaying a typical positive pattern (main pattern) of the BCR / ABL fusion gene. With the first and second fluorescent marker images overlapping, when using the ES probe, the negative example has 2 first fluorescent markers, 2 second fluorescent markers, and 0 fusion fluorescent markers. The typical positive pattern using the ES probe has 1 first fluorescent marker, 2 second fluorescent markers, and 1 fusion fluorescent marker. When using the DF probe, with the first and second fluorescent marker images overlapping, the negative pattern has 2 first fluorescent markers, 2 second fluorescent markers, and 0 fusion fluorescent markers. The typical positive pattern using the DF probe has 1 first fluorescent marker, 1 second fluorescent marker, and 2 fusion fluorescent markers.

[0114] Figure 7Examples of fluorescent markers for atypical positive patterns of the BCR / ABL fusion gene. One example of an atypical positive pattern was a minor BCR / ABL pattern; because the BCR gene cutoff site was relatively upstream of the BCR gene, three first fluorescent markers were also detected using the ES probe. Other examples of atypical positive patterns involved a partial deletion of the binding region of a probe targeting the ABL gene on chromosome 9; when using the DF probe in reliance on this, only one of the two fusion fluorescent markers that should have been detected was detected. Additionally, other examples of atypical positive patterns involved the deletion of both a portion of the binding region of a probe targeting the ABL gene on chromosome 9 and a portion of the binding region of a probe targeting the BCR gene on chromosome 22. When using the DF probe in reliance on this, only one of the two fusion fluorescent markers that should have been detected was detected.

[0115] exist Figure 8 This example shows a reference pattern for negative and positive patterns when detecting chromosomal abnormalities associated with the ALK locus. In the negative pattern, since the ALK gene is not cleaved, there are two fusion fluorescent markers. On the other hand, in the positive pattern, since the ALK gene is cleaved, the fusion fluorescent marker becomes only one (when only one allele is cleaved), or the fusion fluorescent marker becomes undetectable (when both alleles are cleaved). This negative and positive pattern is identical for the ALK gene, ROS1 gene, and RET gene.

[0116] Furthermore, in Figure 8 This is an example of a reference pattern showing a chromosomal abnormality involving the deletion of the long arm (5q) of chromosome 5. For example, it is designed such that a first fluorescently labeled probe binds to the long arm of chromosome 5, and a second fluorescently labeled probe binds to the centromere of chromosome 5. In the negative pattern, since the number of centromeres and long arms of chromosome 5 are the same, there are two of each of the first and second fluorescent labels, reflecting the same number of chromosomes. In the positive pattern, where the long arm is deleted on one or both sides of chromosome 5, the number of first fluorescent labels becomes only one or zero. The deletion of short or long arms of other chromosomes is also the same in this negative and positive pattern. Examples of long arm deletions on other chromosomes include chromosome 7 and chromosome 20. In addition, examples of the same positive and negative patterns include 7q31 (deleted), p16 (9p21 deleted), IRF-1 (5q31) deleted, D20S108 (20q12) deleted, D13S319 (13q14) deleted, 4q12 deleted, ATM (11q22.3) deleted, and p53 (17p13.1) deleted.

[0117] Furthermore, in Figure 8An example showing trisomy of chromosome 8. The first fluorescently labeled probe binds to, for example, the centromere of chromosome 8. In a positive pattern, the first fluorescent label becomes three times. In a negative pattern, the first fluorescent label becomes two times. The same applies to trisomy of chromosome 12 with similar fluorescently labeled patterns. Furthermore, in monosomy of chromosome 7, when using a first fluorescently labeled probe that binds to, for example, the centromere of chromosome 7, the first fluorescent label becomes one time in a positive pattern. In a negative pattern, the first fluorescent label becomes two times.

[0118] [2-2. Artificial Intelligence Algorithm for Detecting Peripheral Circulating Tumor Cells]

[0119] This embodiment relates to a training method for a second artificial intelligence algorithm 53 for detecting peripheral circulating tumor cells and a cell parsing method using the trained second artificial intelligence algorithm 63 for detecting peripheral circulating tumor cells. The term "training" or "generating" is sometimes replaced with "generating" or "generating".

[0120] (1) Generation of training data

[0121] use Figures 9 to 11 The training method for the second artificial intelligence algorithm 53 used to detect peripheral circulating tumor cells is explained.

[0122] Figure 9 This shows the preprocessing method for images captured by camera unit 160. Figure 9 (A) Displays the captured image before preprocessing. Preprocessing is a trimming process used to make the training image 75 and the parsed image 85 the same size, and can be performed on all images used as either training image 75 or parsed image 85. Figure 9 In (A), (a) and (b) are images of the same cell, but the imaging channels are different. Figure 9 In (A), (c) is an image of a cell different from (a). (c) and (d) are images of the same cell, but taken using different channels. Figure 9 As shown in (a) and (c) of (A), the image size of the cells sometimes differs. Furthermore, the size of the cells themselves also varies depending on the cell size. Therefore, it is preferable to resize the acquired image in a way that reflects the cell size and results in a consistent image size. Figure 9 In the example shown, the centroid of the cell nucleus within the image is used as the center, and 16 pixels are positioned vertically and horizontally from this center as trimming positions. The trimmed image is shown below. Figure 9 (B) Figure 9 (B)(a) is from Figure 9 (A)(a) The cut-out image, Figure 9 (B)(b) is from Figure 9 (A)(b) The cut-out images Figure 9 (B)(c) is from Figure 9 (A)(c) The cut-out images Figure 9 (B)(d) is from Figure 9 (A)(d) The cut-out image. Figure 9 (B) The images are 32 pixels vertically × 32 pixels horizontally. The centroid of the nucleus can be determined, for example, using the analysis software (IDEAS) attached to the imaging flow cytometer (ImageStream Mark II, Luminex).

[0123] exist Figure 10 and Figure 11 This shows the training method for the second artificial intelligence algorithm 53.

[0124] like Figure 10 As shown, positive training image 75P, which captures peripheral circulating tumor cells (hereinafter referred to as "second positive control cells"), and negative training image 75N, which captures cells other than peripheral circulating tumor cells (hereinafter referred to as "second negative control cells"), are used to generate positive integrated training data 78P and negative integrated training data 78N, respectively. Sometimes, positive training image 75P and negative training image 75N are referred to together as training image 75. Additionally, sometimes positive integrated training data 78P and negative integrated training data 78N are referred to together as training data 78.

[0125] When detecting peripheral circulating tumor cells, the image captured by the imaging unit 160 may include both bright-field and fluorescence images. The bright-field image captures the phase difference of the cells. This imaging may be performed, for example, by the first channel. The fluorescence image captures fluorescently labeled targets within the cell by immunostaining or organelle staining. Each antigen and / or each organelle is fluorescently labeled using a fluorescent dye with a different wavelength range of fluorescence.

[0126] For example, when binding the first antigen to a first fluorescent dye that fluoresces in the first green wavelength region, the first antigen can be labeled with the first fluorescent dye by binding an antibody that binds directly or indirectly to the first antigen to the first fluorescent dye.

[0127] When an antibody that binds to the second antigen binds to a second fluorescent dye that emits fluorescence in a reddish wavelength region different from that of the first fluorescent dye, the second antigen can be labeled with the second fluorescent dye by binding the second fluorescent dye to an antibody that binds directly or indirectly to the second antigen.

[0128] When an antibody that binds to the third antigen binds to a third fluorescent dye that has a yellowish wavelength region different from the first and second fluorescent dyes, the third antigen can be labeled with the third fluorescent dye by binding the antibody that binds directly or indirectly to the third antigen.

[0129] In this way, by labeling each antigen and / or each organelle with a fluorescent dye having a different wavelength region of fluorescence, the first to the Xth fluorescent labels can be labeled with fluorescent dyes having different wavelength regions of fluorescence.

[0130] A sample containing cells with fluorescent labels of the first to the Xth generation can be captured by imaging in a cell imaging device such as an imaging flow cytometer, and images of the cells can be obtained. Multiple images of the cells can be obtained for the same field of view of the same cells. Since the first to the Xth fluorescent labels have different wavelength regions of fluorescence due to the different fluorescent dyes, the filters used to transmit light emitted from each fluorescent dye are different for each dye. Furthermore, it is necessary to use a different filter for bright-field images than the filter that transmits light from the fluorescent dye. Therefore, the light transmitted through each filter is received by the imaging unit 160 (described later) through its respective channel, and is captured as separate images of the same field of view of the same cells. That is, in the imaging unit 160, multiple images corresponding to the number of labeled cells plus the number of bright-field images are obtained for the same field of view of the same cells.

[0131] exist Figure 10 In the example shown, in Figure 10 In (A) and (B), channel 1 (Ch1) displays a bright-field camera image. Figure 10 In (A) and (B), the second channel (Ch2), the third channel (Ch3), ... the Xth channel (ChX) refer to the channels for capturing images of different labeled substances.

[0132] exist Figure 10 In the example, such as Figure 10 As shown in (A), the positive training image 75P may contain a first positive training image 75P1 captured via the first channel for the second positive control cell, a second positive training image 75P2 captured via the second channel for the first fluorescent label, a third positive training image 75P3 captured via the third channel for the second fluorescent label, and an Xth positive training image 75Px captured via the Xth channel for each fluorescent label. The first positive training images 75P1 to the Xth positive training images 75Px correspond to images of the same field of view for the same cells. Next, the brightness of the captured light in each pixel of the image is transformed into numerical values ​​representing the first positive numerical training data 76P1 to the Xth positive numerical training data 76Px.

[0133] Using the first positive training image 75P1, the method for generating the first positive numerical training data 76P1 will be explained. Each image captured by the camera unit 160 is preprocessed as described above, for example, trimmed to a pixel count of 32 pixels vertically × 32 pixels horizontally, to become a training image 75. The first positive training image 75P1 is represented as, for example, a 16-bit grayscale image. Therefore, the brightness of each pixel can be represented by a brightness value of 65,536 color levels from 1 to 65,536. Figure 10 As shown in (A), the value of the brightness level in each pixel of the first positive training image 75P1 is the first positive numerical training data 76P1, represented by rows and columns corresponding to the numbers of each pixel.

[0134] Similar to the first positive numerical training data 76P1, positive numerical training data 76P2 to Xth positive numerical training data 76Px, which represent the brightness of the captured light in each pixel of the image, can be generated from the second positive training image 75P2 to the Xth positive training image 75Px.

[0135] Next, for each pixel, the training data from the first positive value training data 76P1 to the Xth positive value training data 76Px are integrated to generate the integrated positive value training data 77P. For example... Figure 10 As shown in (A), the positive integrated training data 77P becomes the values ​​in each pixel of the first positive numerical training data 76PA and the values ​​in each pixel of the second positive numerical training data 76P2 to the Xth positive numerical training data 76Px, which are displayed side by side as row and column data.

[0136] Next, the label value 79P, derived from the label value of the second positive control cell, is shown attached to the positive integration training data 77P. Labels are generated along with the positive integration training data 78P. As a label indicating that it is the second positive control cell, in... Figure 10 (A) Appendix “2”.

[0137] The negative training image 75N is generated from the negative integrated training data 78N in the same way as the generated labeled integrated training data 78P with positive data.

[0138] like Figure 10As shown in (B), the negative training image 75N contains, similarly to the positive training image 75P, the first negative training images 75N1 to Xth negative training images 75Nx obtained from the images through the first channel to the Xth. The numerical representation of the light intensity in each pixel is the same as when obtaining the first positive numerical training data 76P1 to Xth positive numerical training data 76Px from the first positive training images 75PA to the Xth positive training images 75Px. The first negative numerical training data 76N1, which represents the light intensity captured in each pixel (pixel) of the image, can be generated from the first negative training image 75N1 using the same method as the first positive numerical training data 76P1.

[0139] Similarly, negative numerical training data 76N2 to Xth can be generated from the second negative training image 75N2 to the Xth negative training image 75Nx, which express the brightness of the captured light in each pixel (pixel) in the image in numerical terms.

[0140] like Figure 10 As shown in (B), based on the method for generating positive integrated training data 77P, negative integrated training data 77N is generated by integrating the first negative numerical training data 76N1 to the Xth negative numerical training data 76Nx for each pixel. Figure 10 As shown in (B), the negative integrated training data 77N becomes the row and column data in which the values ​​in each pixel of the first negative numerical training data 76N1 are displayed side by side with the values ​​in each pixel of the second negative numerical training data 76N2 to the Xth negative numerical training data 76Nx.

[0141] Next, the label value 79N, derived from the label value 77N of the second negative control cell, is shown attached to the negative integration training data 77N, and a label is generated along with the negative integration training data 78N. This serves as a label indicating that it is the second negative control cell. Figure 10 (B) Appendix “1”.

[0142] exist Figure 11The method of inputting label-attached positive integration training data 78P and label-attached negative integration training data 78N into the second artificial intelligence algorithm 53 is shown. The number of nodes in the input layer 53a of the first artificial intelligence algorithm 53, which has a neural network structure, corresponds to the product of the number of pixels in the training image 75 (32×32 = 1024 in the example above) and the number of channels for one cell (X channels from 1 to X in the example above). Data corresponding to the positive integration training data 77P of the label-attached positive integration training data 78P is input to the input layer 53a of the neural network. The label value 79P corresponding to the data input to the input layer 53a is input to the output layer 53b of the neural network. Additionally, data corresponding to the negative integration training data 77N of the label-attached negative integration training data 78N is input to the input layer 53a of the neural network. The label value 79N corresponding to the data input to the input layer 53a is input to the output layer 53b of the neural network. The weights in the intermediate layer 53c of the neural network are calculated from these inputs, the second artificial intelligence algorithm 53 is trained, and the trained second artificial intelligence algorithm 63 is generated.

[0143] (2) Generation of data for parsing

[0144] use Figure 12 The method for generating integrated parsed data 72 from the parsed image 85 and the cell parsing method using the trained second artificial intelligence algorithm 63 will be described. The parsed image 85 can be captured and preprocessed in the same way as the training image 75.

[0145] like Figure 12 As shown, the resolved image 85 includes a first resolved image 85T1, taken via the first channel as a bright-field image for the target cell, and second resolved images 85T2 to Xth resolved images 85Tx, taken via the second to Xth channels for fluorescent labels. The imaging and preprocessing, and the numerical representation of the light intensity in each pixel, are the same as when obtaining the first positive numerical training data 76P1 from the first positive training image 75P1. The first numerical resolved data 86T1, which represents the light intensity of each pixel in the image in numerical terms, can be generated from the first resolved image 85T1 using the same method as the first positive numerical training data 76P1.

[0146] Similarly, from the second resolved image 85T2 to the Xth resolved image 85Tx, the second numerical resolved data 86T2 to the Xth numerical resolved data 86Tx, which represent the brightness of the captured light in each pixel (pixel) within the image, can be generated.

[0147] like Figure 12As shown, based on the method for generating positive integrated training data 77P, integrated analytical data 87 is generated by integrating the first numerical analytical data 86T1 to the Xth numerical analytical data 86Tx for each pixel. Figure 12 As shown, the integrated parsing data 87 is displayed side-by-side with the values ​​of each pixel in the first numerical parsing data 86T1 and the corresponding values ​​of each pixel in the second numerical parsing data 86T2 to the Xth numerical parsing data 86Tx.

[0148] like Figure 12 As shown, the generated integrated analytical data 87 is input into the input layer 63a of the neural network within the trained second artificial intelligence algorithm 63. The values ​​contained in the input integrated analytical data 87 are processed through the intermediate layer 63c of the neural network and output from the output layer 63b of the neural network a label value 89 indicating whether the analyzed target cell is a peripheral circulating tumor cell. Figure 12 In the example shown, when the target cell is determined not to be a peripheral circulating tumor cell, "1" is output as the label value; when it is determined to be a peripheral circulating tumor cell, "2" is output as the label value. Alternatively, labels such as "None," "Yes," "Normal," or "Abnormal" can be output instead of the label value.

[0149] (3) Other components

[0150] i. In this embodiment, the training image 75 and the analytical image 85 are preferably images taken using an EDF filter that have not undergone restoration processing.

[0151] ii. During the shooting process, images that are out of focus can be excluded from the training image 75 and the parsed image 85.

[0152] iii. In this embodiment, the training image 75 and the parsed image 85 are exemplarily modified to have a pixel count of 32 pixels vertically × 32 pixels horizontally, but this is not limited as long as the image size is such that the entire cell is contained within the image. The pixel count can be appropriately set between 30 and 50 pixels vertically and between 30 and 50 pixels horizontally. It is not necessary for the vertical and horizontal pixel counts of the image to be the same. However, the training image 75 used to train the first artificial intelligence algorithm 53 and the parsed image 85 used to generate the integrated parsed data 87 input to the first artificial intelligence algorithm 63 trained using the training image 75 are preferably to have the same pixel count, and the vertical and horizontal pixel counts are also the same.

[0153] iv. In this embodiment, the training image 70 and the analyzed image 80 are 16-bit grayscale images. However, the brightness levels can be 8-bit, 32-bit, or other methods besides 16-bit. Furthermore, in this embodiment, the brightness values ​​represented in 16 bits (65,536 levels) are directly used for each of the numerical training data 76P1 to 76Px and 76N1 to 76Nx. However, it is also possible to perform a low-dimensionality processing by combining these values ​​with a certain range of levels, and use the low-dimensional values ​​as the values ​​for each of the numerical training data 76P1 to 76Px and 76N1 to 76Nx. In this case, it is preferable to perform the same processing on the training image 70 and the analyzed image 80.

[0154] 4. Cell Analysis System

[0155] The following uses Figures 13 to 21 The cell analysis systems 1000, 2000, and 3000 described in the first to third embodiments will be explained.

[0156] [4-1. First Embodiment of the Cell Analysis System]

[0157] exist Figure 13 This illustrates the hardware configuration of the cell analysis system 1000 according to the first embodiment. The cell analysis system 1000 may include a training device 200A for training artificial intelligence algorithms, a cell imaging device 100A, and a cell analysis device 400A. The cell imaging device 100A and the cell analysis device 400A can be communicatively connected. Furthermore, the training device 200A and the cell analysis device 400A can be connected via a wired or wireless network.

[0158] 【4-1-1. Training Device】

[0159] (1) Hardware composition

[0160] use Figure 14 The hardware configuration of the training device 200A will be described below. The training device 200A includes a control unit 20A, an input unit 26, and an output unit 27. Furthermore, the training device 200A can be connected to a network 99.

[0161] The control unit 20A includes: a CPU (Central Processing Unit) 21 for data processing (described later), a memory 22 used in the data processing work area, a storage unit 23 for recording the program and processed data (described later), a bus 24 for transmitting data between the units, an interface (I / F) unit 25 for inputting and outputting data to and from external machines, and a GPU (Graphics Processing Unit) 29. An input unit 26 and an output unit 27 are connected to the control unit 20A via the I / F unit 25. For example, the input unit 26 is an input device such as a keyboard or mouse, and the output unit 27 is a display device such as a liquid crystal display. The GPU 29 functions as an accelerator to assist the computational processing (e.g., parallel computation processing) performed by the CPU 21. In the following description, the processing performed by the CPU 21 also includes processing performed by the CPU 21 using the GPU 29 as an accelerator. Alternatively, a chip preferred for neural network computation may be installed instead of the GPU 29. Examples of such chips include FPGA (Field-Programmable Gate Array), ASIC (Application-Specific Integrated Circuit), and MyriadX (Intel).

[0162] In addition, the control unit 20A is for the following purposes: Figure 16 The processing steps described herein involve pre-recording the training program used to train the artificial intelligence algorithm and the artificial intelligence algorithm before training in an executable form in the storage unit 23, for example. The executable form is, for example, a form generated by a compiler from a programming language. The control unit 20A coordinates the operating system and the training program recorded in the storage unit 23 to perform the training processing of the artificial intelligence algorithm before training.

[0163] In the following description, unless otherwise specified, the processing performed by the control unit 20A refers to the processing performed by the CPU 21 or CPU 21 and GPU 29 based on the program and artificial intelligence algorithm stored in the storage unit 23 or memory 22. The CPU 21 uses the memory 22 as its working area to temporarily store necessary data (intermediate data during processing, etc.), and appropriately records long-term data such as calculation results in the storage unit 23.

[0164] (2) Functional Composition of Training Device

[0165] exist Figure 15 The training device 200A is shown to have the following functional configuration: a training data generation unit 201, a training data input unit 202, an algorithm update unit 203, a training data database (DB) 204, and algorithm databases (DB) 205(a) and 205(b). Figure 16The step S11 shown corresponds to the training data generation unit 201. Figure 16 The step S112 shown corresponds to the training data input unit 202. Figure 16 The step S14 shown corresponds to the algorithm update unit 203.

[0166] Training images 70PA, 70PB, 70NA, 70NB, 75P1-75Px, and 75N1-75Nx are pre-acquired from the cell imaging device 100A by the cell analysis device 400A and pre-stored in the storage unit 23 or memory 22 of the control unit 20A of the training device 200A. The training device 200A may acquire the training images 70PA, 70PB, 70NA, 70NB, 75P1-75Px, and 75N1-75Nx from the cell analysis device 400A via a network, or via the media driver D98. The training data database (DB) 204 stores the generated training data 73 and 78. The artificial intelligence algorithm before training is pre-stored in the algorithm databases 205(a) and 205(b). The trained first artificial intelligence algorithm 60 is recorded in the algorithm database 205(a) corresponding to the examination items and analysis items used to check for chromosomal abnormalities. The trained second artificial intelligence algorithm 63 is recorded in the algorithm database 205(b) corresponding to the examination items and analysis items used to examine peripheral circulation tumor cells.

[0167] (3) Training Processing

[0168] The control unit 20A of the training device 200A performs... Figure 16 The training process is shown in the figure.

[0169] First, based on the user's request to begin processing, the CPU 21 of the control unit 20A acquires training images 70PA, 70PB, 70NA, and 70NB stored in the storage unit 23 or memory 22; and training images 75P1 to 75Px and 75N1 to 75Nx. Training images 70PA, 70PB, 70NA, and 70NB can be used to train the first artificial intelligence algorithm 50, while training images 75P1 to 75Px and 75N1 to 75Nx can be used to train the second artificial intelligence algorithm 53.

[0170] [i. Training and processing of the first artificial intelligence algorithm 50]

[0171] Control unit 20A in Figure 16In step S11, positive integrated training data 72P is generated from positive training images 70PA and 70PB, and negative integrated training data 72N is generated from negative training images 70NA and 70NB. The control unit 20A attaches corresponding label values ​​74P or 74N to the positive integrated training data 72P and the negative integrated training data 72N, generating labeled positive integrated training data 73P or labeled negative integrated training data 73N. The labeled positive integrated training data 73P or labeled negative integrated training data 73N is recorded as training data 73 in the storage unit 23. The method for generating labeled positive integrated training data 73P and labeled negative integrated training data 73N is as described in section 2-1 above.

[0172] Next, control unit 20A in Figure 16 In step S12, the generated label-attached positive integrated training data 73P and label-attached negative integrated training data 73N are input into the first artificial intelligence algorithm 50 to train the first artificial intelligence algorithm 50. The training results of the first artificial intelligence algorithm 50 are accumulated to the training degree using multiple label-attached positive integrated training data 73P and label-attached negative integrated training data 73N.

[0173] Next, control unit 20A in Figure 16 In step S13, it is determined whether the training result has been accumulated for a predetermined number of trials. If the training result has been accumulated for the specified number of trials ("YES"), the control unit 20A proceeds to step S14; if the training result has not been accumulated for the specified number of trials ("NO"), the control unit 20A proceeds to step S15.

[0174] When the training results are accumulated for a specified number of trials, in step S14, the control unit 20A updates the weight w of the first artificial intelligence algorithm 50 (combined with the weight w) using the training results accumulated in step S12.

[0175] Next, in step S15, the control unit 20A determines whether to train the first artificial intelligence algorithm 50 using a predetermined number of label-attached positive integration training data 73P and label-attached negative integration training data 73N. If training is performed using the predetermined number of label-attached positive integration training data 73P and label-attached negative integration training data 73N (in the "YES" case), the training process ends. The control unit 20A stores the trained first artificial intelligence algorithm 60 in the storage unit 23.

[0176] When the first artificial intelligence algorithm 50 is not trained with a specified number of label-attached positive integrated training data 73P and label-attached negative integrated training data 73N (in the case of "NO"), the control unit 20A proceeds to steps S15 to S16 and performs the processing of steps S11 to S15 for the following positive training images 70PA, 70PB and negative training images 70NA, 70NB.

[0177]

ii. Training and processing of the second artificial intelligence algorithm 53

[0178] Control unit 20A in Figure 16 In step S11, positive integrated training data 77P is generated from positive training images 75P1 to 75Px, and negative integrated training data 77N is generated from negative training images 75N1 to 75Nx. The control unit 20A assigns corresponding label values ​​79P or 79N to the positive integrated training data 77P and the negative integrated training data 77N, generating labeled positive integrated training data 78P or labeled negative integrated training data 78N. The labeled positive integrated training data 78P or labeled negative integrated training data 78N is recorded as training data 78 in the storage unit 23. The method for generating labeled positive integrated training data 78P and labeled negative integrated training data 78N is described in section 2-2 above.

[0179] Next, control unit 20A in Figure 16 In step S12, the generated label-attached positive integrated training data 78P and label-attached negative integrated training data 78N are input into the second artificial intelligence algorithm 53 to train the second artificial intelligence algorithm 53. The training results of the second artificial intelligence algorithm 53 are accumulated to the training degree using multiple label-attached positive integrated training data 78P and label-attached negative integrated training data 78N.

[0180] Next, control unit 20A in Figure 16 In step S13, it is determined whether the training result has been accumulated for a predetermined number of trials. If the training result has been accumulated for the specified number of trials ("YES"), the control unit 20A proceeds to step S14. If the training result has not been accumulated for the specified number of trials ("NO"), the control unit 20A proceeds to step S15.

[0181] When the training results are accumulated for a specified number of trials, in step S14, the control unit 20A updates the weight w of the second artificial intelligence algorithm 53 (combined with the weight w) using the training results accumulated in step S12.

[0182] Next, in step S15, the control unit 20A determines whether to train the second artificial intelligence algorithm 53 using a predetermined number of tag-attached positive integration training data 78P and tag-attached negative integration training data 78N. If the training is performed using the predetermined number of tag-attached positive integration training data 78P and tag-attached negative integration training data 78N (in the "YES" case), the training process ends. The control unit 20A stores the trained second artificial intelligence algorithm 63 in the storage unit 23.

[0183] When the second artificial intelligence algorithm 53 is not trained with a specified number of label-attached positive integrated training data 78P and label-attached negative integrated training data 78N (in the case of "NO"), the control unit 20A proceeds to steps S15 to S16 and performs the processing of steps S11 to S15 for the following positive training images 75P1 to 75Px and negative training images 75N1 to 75Nx.

[0184] (4) Training Procedure

[0185] This embodiment includes a computer program for training an artificial intelligence algorithm, which causes the computer to perform the processes of steps S11 to S16.

[0186] Furthermore, one embodiment of this invention relates to a program article, such as a storage medium, that stores the aforementioned computer program. That is, the computer program can be stored in a semiconductor memory element such as a hard disk or flash memory, or a storage medium such as an optical disc. The form in which the program is recorded to the storage medium is not limited as long as the training device 200A can read the program. The recording to the storage medium is preferably non-volatile.

[0187] Among them, "program" refers not only to programs that can be directly executed by the CPU, but also includes the concepts of source programs, compressed programs, encrypted programs, etc.

[0188] 【4-1-2. Cellular Imaging Device】

[0189] The configuration of the cell imaging device 100A for capturing training images 70, 75 and / or resolving images 80, 85 is shown in the figure. Figure 17 . Figure 17 The cell imaging device 100A shown illustrates an imaging flow cytometer. The operation of the imaging device 100A is controlled by the cell analysis device 400A.

[0190] As described above, chromosomal abnormalities or peripheral circulating tumor cells are detected using one or more fluorescent dyes to target sites. Preferably, the FISH method uses two or more fluorescent dyes to detect target sites on chromosome 1 and chromosome 2 (the "first" and "second" in "modified chromosome" do not refer to chromosome number, but are a concept encompassing sex). For example, a probe hybridizing to the PML locus is a nucleic acid with a sequence complementary to the base sequence of the PML locus, labeled with a first fluorescent dye that produces first fluorescence at wavelength λ21 when illuminated with light at wavelength λ11. By using this probe, the PML locus is labeled with the first fluorescent dye. A probe hybridizing to the RARA locus is a nucleic acid with a sequence complementary to the base sequence of the RARA locus, labeled with a second fluorescent dye that produces second fluorescence at wavelength λ22 when illuminated with light at wavelength λ12. By using this probe, the RARA locus is labeled with the second fluorescent dye. The nucleus is stained with a nuclear staining dye that produces third fluorescence at wavelength λ23 when illuminated with light at wavelength λ13. Wavelengths λ11, λ12, and λ13 are the so-called excitation light. In addition, wavelength λ114 is light emitted from a halogen lamp or similar source used for bright-field observation.

[0191] The cell imaging device 100A includes: a flow cell 110, a light source 120-123, a condenser lens 130-133, a dichroic mirror 140-141, a condenser lens 150, an optical unit 151, a condenser lens 152, and an imaging unit 160. A flow path 111 allows the sample 10 to flow through the flow cell 110.

[0192] Light sources 120-123 irradiate the sample 10 as it flows upward through the flow cell 110. Light sources 120-123 are, for example, semiconductor laser sources. Each of the light sources 120-123 emits light with wavelengths λ11-λ14.

[0193] Concentrating lenses 130-133 focus light of wavelengths λ11-λ14 emitted from light sources 120-123. Dichroic mirror 140 allows light of wavelength λ11 to pass through and refracts light of wavelength λ12. Dichroic mirror 141 allows light of wavelengths λ11 and λ12 to pass through and refracts light of wavelength λ13. Thus, light of wavelengths λ11-λ14 is irradiated onto the sample 10 flowing through the flow path 111 of the flow cell 110. Furthermore, the number of semiconductor laser sources provided by the cell imaging device 100A is not limited as long as there is one or more. The number of semiconductor laser sources can be selected from, for example, 1, 2, 3, 4, 5, or 6.

[0194] When light of wavelengths λ11 to λ13 is irradiated onto the sample 10 flowing through flow cell 110, fluorescence occurs from the fluorescent dye labeled on the cells flowing through flow path 111. Specifically, when light of wavelength λ11 is irradiated onto the first fluorescent dye labeled the PML locus, a first fluorescence of wavelength λ21 occurs from the first fluorescent dye. When light of wavelength λ12 is irradiated onto the second fluorescent dye labeled the RARA locus, a second fluorescence of wavelength λ22 occurs from the second fluorescent dye. When light of wavelength λ13 is irradiated onto a nuclear staining dye used to stain the nucleus, a third fluorescence of wavelength λ23 occurs from the nuclear staining dye. When light of wavelength λ14 is irradiated onto the sample 10 flowing through flow cell 110, this light passes through the cells. The transmitted light of wavelength λ14 that passes through the cells is used in the generation of bright-field images. For example, in this embodiment, the first fluorescence is in the wavelength region of green light, the second fluorescence is in the wavelength region of red light, and the third fluorescence is in the wavelength region of blue light.

[0195] The condenser lens 150 focuses the first to third fluorescence generated by the sample 10 flowing through the flow path 111 of the flow cell 110 and the transmitted light passing through the flow path 111 of the flow cell 110. The optical unit 151 has a configuration of four dichroic mirrors. The four dichroic mirrors of the optical unit 151 reflect the first to third fluorescence and the transmitted light at slightly different angles, separating them at the light receiving surface of the imaging unit 160. The condenser lens 152 focuses the first to third fluorescence and the transmitted light.

[0196] The camera unit 160 is composed of a TDI (Time Delay Integration) camera. The camera unit 160 captures the first to third fluorescence and transmitted light, and outputs the fluorescence images corresponding to the first to third fluorescence and the bright-field image corresponding to the transmitted light as imaging signals to the cell analysis device 400A. The captured images can be color images or grayscale images.

[0197] In addition, the cell imaging device 100A may also be equipped with a preprocessing device 300 as needed.

[0198] The pretreatment device 300 samples a portion of the test subject and performs FISH, immunostaining, or organelle staining on the cells contained in the test subject, and prepares sample 10.

[0199] 【4-1-3. Cell Analysis Device】

[0200] (1) Hardware composition

[0201] use Figure 17The hardware configuration of the cell analysis device 400A will be described. The cell analysis device 400A can be communicatively connected to the cell imaging device 100A. The cell analysis device 400A includes a control unit 40A, an input unit 46, an output unit 47, and a media driver 98. Furthermore, the cell analysis device 400A can be connected to a network 99.

[0202] The configuration of the control unit 40A is the same as that of the control unit 20A of the training device 200A. Specifically, the CPU 21, memory 22, storage unit 23, bus 24, I / F unit 25, and GPU 29 in the control unit 20A of the training device 200A are respectively read as CPU 41, memory 42, storage unit 43, bus 44, I / F unit 45, and GPU 49. However, the storage unit 43 stores the artificial intelligence algorithms 60 and 63 generated by the training device 200A and trained by the CPU 41 from the I / F unit 45 via the network 99 or the media driver 98.

[0203] The resolved images 80 and 85 can be acquired by the cell imaging device 100A and stored in the storage unit 43 or memory 42 of the control unit 40A of the cell analysis device 400A.

[0204] (2) Functional composition of the cell analysis device

[0205] exist Figure 18 The functional configuration of the cell analysis device 400A is shown. The cell analysis device 400A includes: a data generation unit 401, a data input unit 402, an analysis unit 403, a data database (DB) 404, and algorithm databases (DB) 405(a) and 405(b). Figure 19 The step S21 shown corresponds to the parsing data generation unit 401. Figure 19 The step S22 shown corresponds to the parsing data input unit 402. Figure 19 Step S23 shown corresponds to the parsing unit 403. The parsing data database 404 stores parsing data 82 and 88.

[0206] The first trained artificial intelligence algorithm 60 is recorded in algorithm database 405(a) corresponding to the examination and analysis items used to check for chromosomal abnormalities. The second trained artificial intelligence algorithm 63 is recorded in algorithm database 405(b) corresponding to the examination and analysis items used to check for peripheral circulating tumor cells.

[0207] (3) Cell analysis processing

[0208] The control unit 40A of the cell analysis device 400A performs... Figure 19 The cell analysis process shown is made easy to perform with high precision and high speed by this embodiment.

[0209] Based on the user's request to begin processing, or triggered by the start of parsing by the cell imaging device 100A, the CPU 41 of the control unit 40A begins cell parsing processing.

[0210] Control unit 40A in Figure 19 In step S20, the inspection item is received by the input unit 46. Specifically, the inspection item is received by reading information about the inspection item from the barcode attached to each sample using a barcode reader, which is an example of the input unit 46. In step S21, corresponding to the inspection item received in step S20, the control unit 40A selects the channel used in generating the parsing data from the cell images of each channel output from the cell imaging device 100A, obtains the cell image corresponding to the selected channel, and generates integrated parsing data 82 or integrated parsing data 87. The method for generating integrated parsing data 82 is described in section 2-1 above. The method for generating integrated parsing data 87 is described in section 2-2 above. The control unit 40A stores the generated integrated parsing data 82 or integrated parsing data 87 in the storage unit 43 or the memory 42.

[0211] In step S22, control unit 40A selects either the first artificial intelligence algorithm 60 or the second artificial intelligence algorithm 63, corresponding to the inspection item accepted in step S20. The association between the accepted inspection item and the artificial intelligence algorithm is determined by... Figure 20 The check items – algorithm table shown is used. The check items – algorithm table is stored in storage unit 43. Figure 20 The examination item-algorithm table shown is categorized as "examination items," exemplifying "chromosomal abnormalities" and "peripheral circulating tumor cells." As analysis items, "BCR-ABL," "PML-RARA," and "IGH-CCND1, IGH-FGFR3, IGH-MAF" correspond to the examination item "chromosomal abnormalities," while "CTC" corresponds to the examination item "peripheral circulating tumor cells." Furthermore, each examination item is labeled with "green," "yellow," "cyan," and "red" to indicate the wavelength region of fluorescence at the target site, and "bright field" to indicate bright field imaging. Additionally, each fluorescence wavelength region and bright field imaging is labeled with "ch1," "ch2," "ch3," "ch4," and "bright field" as the names of the corresponding imaging channels. The first trained artificial intelligence algorithm 60 is then associated with the analysis items "BCR-ABL," "PML-RARA," and "IGH-CCND1, IGH-FGFR3, IGH-MAF." The second trained artificial intelligence algorithm 63 is associated with the analysis item "CTC."

[0212] In step S22, if the inspection item is "chromosomal abnormality", the control unit 40A selects the first artificial intelligence algorithm 60 corresponding to the label of the analysis item; if the inspection item is "peripheral circulating tumor cells", the control unit 40A selects the second artificial intelligence algorithm 63 corresponding to the label of the analysis item.

[0213] In step S23, the control unit 40A uses either the selected first artificial intelligence algorithm 60 or the second artificial intelligence algorithm 63 to determine the characteristics of the target cells in the analyzed images 80A and 80B, and stores the label value 84 of the determination result in the storage unit 43 or the memory 42. The determination method is described in sections 2-1 and 2-2 above.

[0214] In step S24, the control unit 40A determines whether all parsed images 80A and 80B have been evaluated. If all parsed images 80A and 80B have been evaluated (in the case of "YES"), the process proceeds to step S25, where the evaluation result corresponding to the tag value 84 is stored in the storage unit 43, and the evaluation result is output to the output unit. If not all parsed images 80A and 80B have been evaluated in step S24 (in the case of "NO"), the control unit 40A updates the parsed images 80A and 80B in step S26, and repeats steps S21 to S24 until all parsed images 80A and 80B are evaluated. The evaluation result can be the tag value itself, or a tag corresponding to each tag value such as "present," "absent," "abnormal," or "normal."

[0215] (4) Cell analysis procedure

[0216] This embodiment includes a computer program for performing cell analysis, which causes the computer to execute the processes of steps S20 to S26 and steps S221 to S222.

[0217] Furthermore, one embodiment of this invention relates to a program article, such as a storage medium, that stores the aforementioned computer program. That is, the computer program can be stored in a semiconductor memory element such as a hard disk or flash memory, or a storage medium such as an optical disc. The form in which the program is recorded to the storage medium is not limited as long as the training device 200A can read the program. The recording to the storage medium is preferably non-volatile.

[0218] Among them, "program" refers not only to programs that can be directly executed by the CPU, but also includes the concepts of source programs, compressed programs, encrypted programs, etc.

[0219] 【5. Other Implementation Methods】

[0220] (1) Modification of the analytical apparatus

[0221] In the first embodiment, an example will be described where the control unit 40A selects an artificial intelligence algorithm based on the inspection items received in S20. However, it is also possible to change the algorithm to be based on the inspection items. Figure 36 and Figure 37 The parsing mode shown.

[0222] Control unit 40A in Figure 36 In step S200 shown, the parsing mode is received by the input unit 46. Specifically, the mode is displayed on a touch panel display that combines the functions of the input unit 46 and the output unit 47. Figure 37 The resolution mode receiving screen shown is accessed by accepting the chromosome abnormality judgment mode button 801 or the CTC judgment mode 802. In step S201, corresponding to the resolution mode accepted in step S200, the control unit 40A selects the channel used in generating the resolution data from the cell images of each channel output from the cell imaging device 100A, obtains the cell image corresponding to the selected channel, and generates integrated resolution data 82 or integrated resolution data 87. The method for generating integrated resolution data 82 is described in section 2-1 above. The method for generating integrated resolution data 87 is described in section 2-2 above. The control unit 40A stores the generated integrated resolution data 82 or integrated resolution data 87 in the storage unit 43 or the memory 42.

[0223] In step S202, if the accepted parsing mode is "chromosome abnormality judgment mode", the control unit 40A selects the first artificial intelligence algorithm 60, and if the accepted parsing mode is "CTC judgment mode", it selects the second artificial intelligence algorithm 63.

[0224] In step S203, the control unit 40A uses either the selected first artificial intelligence algorithm 60 or the second artificial intelligence algorithm 63 to determine the characteristics of the target cells in the analyzed images 80A and 80B, and stores the label value 84 of the determination result in the storage unit 43 or the memory 42. The determination method is described in sections 2-1 and 2-2 above.

[0225] Due to the processing of steps S204 to S206 by the control unit 40A and Figure 19 Steps S24 to S26 described herein are the same and are omitted here.

[0226] (2) Modification of the camera unit

[0227] In the first embodiment, examples of imaging flow cytometers equipped with imaging units 160 are described in sections 1 to 4 above. However, it is also possible to use an imaging flow cytometer. Figure 21 and Figure 22 The microscope 700 shown is an example. Among them, Figure 21The microscope shown is disclosed in U.S. Patent No. 2018-0074308, which is incorporated herein by reference.

[0228] like Figure 21 As shown, the microscope apparatus 700 includes a housing portion 710 and a moving portion 720. The microscope apparatus 700 includes an imaging portion 710d and a slide mounting portion 711. The imaging portion 710d includes an objective lens 712, a light source 713, and an imaging element 714. The slide mounting portion 711 is located on the upper surface (Z1 direction side) of the housing portion 710. The objective lens 712, light source 713, and imaging element 714 are located inside the housing portion 710. The microscope apparatus 700 includes a display portion 721. The display portion 721 is located in front of the moving portion 720 (Y1 direction side). The display surface 721a of the display portion 721 is disposed in front of the moving portion 720. The microscope apparatus 700 includes a drive portion 710a for moving the moving portion 720 relative to the housing portion 710.

[0229] The slide mounting section 711 includes a stage 711a. The stage 711a is movable in the horizontal direction (X and Y directions) and the vertical direction (Z direction). The stage 711a can move independently in the X, Y, and Z directions. Therefore, since the slide can be moved relative to the objective lens 712, the desired position of the slide can be magnified for observation.

[0230] Objective lens 712 is positioned close to the stage 711a of slide mounting portion 711. Objective lens 712 is positioned below (in the Z2 direction) the stage 711a of slide mounting portion 711. Objective lens 712 is positioned facing slide mounting portion 711 in the vertical direction (Z direction). Objective lens 712 is positioned such that its optical axis is substantially perpendicular to the slide mounting surface on which the slide is mounted in slide mounting portion 711. Objective lens 712 is positioned upwards. Objective lens 712 can move relative to slide mounting portion 711 in the vertical direction (Z direction). Objective lens 712 is positioned such that its long side is in the vertical direction. That is, objective lens 712 is positioned such that its optical axis is in the substantially vertical direction. Objective lens 712 includes multiple lenses.

[0231] Light source 713 illuminates a glass slide coated with a sample. Light source 713 illuminates the glass slide through objective lens 712. Light source 713 illuminates the glass slide from the same side as imaging element 714. Light source 713 can output light of a specified wavelength. Light source 713 can output light of multiple different wavelengths. That is, light source 713 can output different types of light. Light source 713 includes a light-emitting element. The light-emitting element includes, for example, an LED element or a laser element.

[0232] exist Figure 22This describes an example of the optical system configuration of the microscope apparatus 700. The microscope apparatus 700, as its optical system configuration, includes an objective lens 712, a light source 713, an imaging element 714, a first optical element 715, filters 716a, second optical elements 716b, 716c, 716f, and 716g, lenses 716d, 716e, and 716h, reflecting parts 717a, 717b, and 717d, and lens 717c. The objective lens 712, light source 713, imaging element 714, first optical element 715, filters 716a, second optical elements 716b, 716c, 716f, and 716g, lenses 716d, 716e, and 716h, reflecting parts 717a, 717b, and 717d, and lens 717c are disposed inside the housing portion 710.

[0233] The first optical element 715 is configured such that light irradiated from the light source 713 is reflected in the direction of the optical axis of the objective lens 712, thereby allowing light from the slide to pass through. The first optical element 715 includes, for example, a dichroic mirror. That is, the first optical element 715 is configured such that it reflects light of the wavelength irradiated from the light source 713, thereby allowing light of the wavelength irradiated from the slide to pass through.

[0234] The filter 716a is configured to allow light of a specified wavelength to pass through while blocking light of other wavelengths, or to block light of a specified wavelength while allowing light of other wavelengths to pass through. That is, light of the desired wavelength passes through the filter 716a and reaches the imaging element 714.

[0235] The second optical elements 716b, 716c, 716f, and 716g are configured to reflect light from their own glass slides toward the imaging element 714. The second optical elements 716b, 716c, 716f, and 716g include reflective portions. The second optical elements 716b, 716c, 716f, and 716g include, for example, mirrors.

[0236] The reflecting portions 717a, 717b, and 717d are configured to reflect light from the light source 713 toward the objective lens 712. The reflecting portions 717a, 717b, and 717d include, for example, mirrors.

[0237] Light emitted from light source 713 is reflected by reflector 717a and enters reflector 17b. Light entering reflector 717b is reflected and enters reflector 717d via lens 717c. Light entering reflector 717d is reflected and enters first optical element 715. Light entering first optical element 715 is reflected and enters slide mounting section 11 via objective lens 712, illuminating the slide.

[0238] Light emitted from the slide based on light source 713 passes through objective lens 712 and enters first optical element 715. Light incident on first optical element 715 passes through filter 716a and enters second optical element 716b. Light incident on second optical element 716b is reflected and enters second optical element 716c. Light incident on second optical element 716c is reflected and passes through lenses 716d and 716e to enter second optical element 716f. Light incident on second optical element 716f is reflected and enters second optical element 716g. Light incident on second optical element 716g is reflected and passes through lens 16h to reach imaging element 714. Imaging element 714 captures a magnified image of the slide based on the arriving light.

[0239] The captured images were sent from microscope 700 to Figure 21 The computer 800 shown is equivalent to a generation device (200A) and / or a cell analysis device (400A).

[0240]

III. Second Implementation

[0241] The second embodiment relates to a method for analyzing cells using artificial intelligence algorithms from waveform data based on the signal strength of the cells themselves.

[0242] 【1. Cell analysis methods】

[0243] This embodiment relates to a cell analysis method for analyzing cells contained in a biological sample. The analysis method inputs numerical data corresponding to the signal intensity of each cell into a third artificial intelligence algorithm 560 or a fourth artificial intelligence algorithm 563 having a neural network structure. Then, based on the results output from the third artificial intelligence algorithm 560 or the fourth artificial intelligence algorithm 563, the category of the cell from which the signal intensity was obtained is determined for each cell.

[0244] In one embodiment of this invention, the cell type to be determined is based on a morphological classification of cells, which varies depending on the type of biological sample. When the biological sample is blood, and when the blood is collected from a healthy person, the cell types to be determined in this embodiment include nucleated cells such as red blood cells and white blood cells, and platelets. Among the nucleated cells, there are neutrophils, lymphocytes, monocytes, eosinophils, and basophils. Among the neutrophils, there are segmented neutrophils and band neutrophils. On the other hand, when the blood is collected from an unhealthy person, the nucleated cells may sometimes contain at least one type selected from immature granulocytes and abnormal cells. Such cells are also included in the cell types to be determined in this embodiment. Among the immature granulocytes, there may be cells such as postmyelocytes, myelocytes, premyelocytes, and myeloblasts.

[0245] use Figures 23-25 The example shown illustrates the method for generating training data 575 and the method for analyzing waveform data. The term "training" is sometimes replaced with "generation." For ease of description, "analyzed object image" is sometimes referred to as "analyzed image," "analyzed data" as "analyzed data," "training image" as "training image," and "training data" as "training data." Additionally, "fluorescent image" refers to either a training image of a fluorescent marker or an analyzed image of a fluorescent marker.

[0246] (1) Generation of training data

[0247] Figure 23 The example shown is one example of a method for generating training waveform data used to train a third artificial intelligence algorithm for classifying white blood cells, immature granulocytes, and abnormal cells. Waveform data 570a of forward scattering, 570b of side scattering, and 570c of lateral fluorescence, which are used as training waveform data, are associated with the cells of the training object. The waveform data 570a, 570b, and 570c of forward scattering, side scattering, and lateral fluorescence are also collectively referred to as training waveform data 570. The training waveform data 570a, 570b, and 570c obtained from the cells of the training object can also be waveform data obtained by flow cytometry from cells whose cell types are known based on morphological classification. Alternatively, waveform data from cells whose cell types have already been determined from scatter plots of healthy individuals can also be used. Furthermore, as waveform data for classifying cells in healthy individuals, a pool of waveform data from cells obtained from multiple individuals can also be used. The subjects used to obtain training waveform data 570a, 570b, and 570c are preferably samples containing cells of the same type as those used in the training subjects, and are treated using the same subject treatment method as those used in samples containing cells used in the training subjects. Furthermore, the training waveform data 570a, 570b, and 570c are preferably obtained under the same conditions as those used to obtain the cells for analysis. The training waveform data 570a, 570b, and 570c can be obtained in advance for each cell, for example, using known methods such as flow cytometry or sheath current resistance measurement. When the cells used in the training subjects are erythrocytes or platelets, the training data may sometimes be waveform data obtained by sheath current resistance measurement, and the waveform data may sometimes be derived from electrical signal intensity.

[0248] exist Figure 23In the example shown, training waveform data 570a, 570b, and 570c are obtained by flow cytometry using a Sysmex XN-1000. Training waveform data 570a, 570b, and 570c are, for example, obtained by acquiring the signal intensity of the frontal scattered light, the signal intensity of the side scattered light, and the signal intensity of the side fluorescence after the frontal scattered light reaches a specified threshold, and acquiring waveform data at multiple time points at fixed intervals for a single training subject between a specified time and the end of acquisition. Examples of acquiring waveform data at multiple time points at fixed intervals include, for example, 1024 minutes at 10 nanosecond intervals, 128 minutes at 80 nanosecond intervals, or 64 minutes at 160 nanosecond intervals. Each waveform data is obtained by passing cells contained in a biological sample through a cell detection flow path within a measurement section of a flow cytometer or a sheath resistance measurement device, which allows for individual cell detection, and for each cell passing through the flow path. Specifically, for each signal, a data set is acquired at multiple time points between designated locations within the flow path of a training subject, using the value of the time at which the signal strength was acquired and the value of the signal strength at that time point as elements. This data is used as training waveform data 570a, 570b, and 570c. The information regarding the time points is not limited as long as it can be stored in a manner that allows the control units 10T and 20T (described later) to determine how long has elapsed since the acquisition of the signal strength began. For example, the time point information can be the start time of the measurement or any other aspect. The signal strength is preferably stored together with the information about the time points at which its signal strength was acquired in the storage units 13 and 23 or the memories 12 and 22 (described later).

[0249] Figure 23The training waveform data 570a, 570b, and 570c, if displayed as their original values, would be, for example, the sequence data 572a for frontal scattering, the sequence data 572b for lateral scattering, and the sequence data 572c for lateral fluorescence. The sequence data 572a, 572b, and 572c are the time points at which the signal intensity is synchronously acquired for each training subject's cells, becoming the sequence data 576a for frontal scattering, the sequence data 576b for lateral scattering, and the sequence data 576c for lateral fluorescence. That is, the signal intensity measured at time t=0, starting from the second value from the left of 576a, is 10. Similarly, the signal intensity measured at time t=0, starting from the second value from the left of 576b and 576c, are 50 and 100, respectively. Furthermore, the signal intensity is collected in adjacent cells within each of 576a, 576b, and 576c at 10-nanosecond intervals. The sequence data 576a, 576b, and 576c are combined with the label value 577 representing the cell category of the training object, and grouped as training data 575 using three signal intensities at the same time point (signal intensity of frontal scattering, signal intensity of lateral scattering, and signal intensity of lateral fluorescence). For example, when the training object cell is a neutrophil, the sequence data 576a, 576b, and 576c, as the label value 577 representing a neutrophil, are assigned "1", generating training data 575. Figure 24 Example of label value 577 shown. Training data 575 is generated for each cell category, and label values ​​577 are assigned to different cell types. The synchronization of signal intensity acquisition time points refers to aligning the measurement points by combining, for example, the front-scattered light data series 572a, the side-scattered light data series 572b, and the side-fluorescence data series 572c at the same time point from the start of the measurement. In other words, it means adjusting the signal intensity acquired by one cell in the flow cell at the same time point for each of the front-scattered light data series 572a, the side-scattered light data series 572b, and the side-fluorescence data series 572c. The measurement start time can be a specified threshold such as the front-scattered light signal intensity exceeding a threshold, or a threshold for the signal intensity using other scattered light or fluorescence. Alternatively, a threshold can be set for each data series.

[0250] The data sequences 576a, 576b, and 576c can be the directly obtained signal strength values, or they can be processed as needed, such as noise removal, baseline correction, and standardization. In this specification, the "numerical data corresponding to signal strength" may include the obtained signal strength values ​​themselves, as well as the values ​​after noise removal, baseline correction, and standardization as needed.

[0251] by Figure 23As an example, an overview of the training of the third artificial intelligence algorithm 550 and the fourth artificial intelligence algorithm 553, which have neural network structures, will be explained. The third artificial intelligence algorithm 550 is an algorithm for classifying neutrophils, lymphocytes, monocytes, eosinophils, basophils, and immature granulocytes, and the fourth artificial intelligence algorithm 553 is an algorithm for classifying abnormal cells. The third artificial intelligence algorithm 550 and the fourth artificial intelligence algorithm 553 are preferably folded neural networks. The number of nodes in the input layer 550a of the third artificial intelligence algorithm 550 corresponds to the number of sequences contained in the waveform data of the input training data 575. The training data 575 is combined in such a way that the time points at which the signal strength of the sequence data 576a, 576b, and 576c are obtained are the same time points, and this is used as the first training data and input to the input layer 550a of the third artificial intelligence algorithm 550. The label values ​​577 of each waveform data of the training data 575 are input into the output layer 550b of the third artificial intelligence algorithm 550 as the second training data to train the third artificial intelligence algorithm 550. Figure 23 The symbol 550c represents the intermediate layer. The fourth artificial intelligence algorithm 553 also has the same structure.

[0252] (2) Methods for generating analytical data and analyzing cells

[0253] exist Figure 25 This example illustrates a method for analyzing waveform data of cells used for analysis. In this cell analysis method using waveform data, analysis data 585 is generated from waveform data 580a of forward scattered light, waveform data 580b of side scattered light, and waveform data 580c of lateral fluorescence obtained from the cells being analyzed. Waveform data 580a, waveform data 580b, and waveform data 580c are collectively referred to as analysis waveform data 580. The analysis waveform data 580a, 580b, and 580c can be obtained using, for example, a known flow cytometry technique. Figure 25 In the example shown, the analytical waveform data 580a, 580b, and 580c are obtained using a Sysmex XN-1000 in the same way as the training waveform data 570a, 570b, and 570c. If the analytical waveform data 580a, 580b, and 580c are displayed as their original data values, they would, for example, become the sequence data 582a for frontal scattered light, the sequence data 582b for side scattered light, and the sequence data 582c for side fluorescence.

[0254] Regarding the generation of analytical data 585 and training data 575, it is preferable to at least make the conditions for generating the data input to the neural network from the acquisition conditions and waveform data the same. The sequence data 582a, 582b, and 582c are the time points at which the signal intensity is simultaneously acquired for each training object's cells, becoming sequence data 586a (forward scattered light), sequence data 586b (side scattered light), and sequence data 586c (side fluorescence). The sequence data 586a, 586b, and 586c are combined in a group manner with three signal intensities (signal intensity of forward scattered light, signal intensity of side scattered light, and signal intensity of side fluorescence) at the same time point and used as analytical data 585 as input to the third artificial intelligence algorithm 560 or the fourth artificial intelligence algorithm 563.

[0255] When parsing data 585 is input to the input layer 560a of the third artificial intelligence algorithm 560 constituting training or the input layer 563a of the fourth artificial intelligence algorithm 563, the probability of the cell to which the parsing object belongs is obtained from the respective output of the cell category input as training data from the output layer 560b or the input layer 563a. Figure 25 The symbols 560c and 563c represent intermediate layers. Furthermore, it's also possible to determine if the cell in the parsed data 585 belongs to the highest-valued category based on this probability, and output a label value 582 associated with that cell's category, etc. Regarding the output cell parsing result 583, besides the label value itself, it can also be data where the label value is replaced with information displaying the cell's category (e.g., terminology). Figure 25 The output shows the label value "1" with the highest probability of the cell belonging to the parsed object obtained by the third artificial intelligence algorithm 560 or the fourth artificial intelligence algorithm 563 based on the parsed data 585. Furthermore, the text data corresponding to this label value, called "neutrophil", is output as an example of the parsing result 583 about the cell. The label value can be output by the third artificial intelligence algorithm 560 or the fourth artificial intelligence algorithm 563, or by other computer programs outputting the optimal label value based on the probability calculated by the third artificial intelligence algorithm 560 or the fourth artificial intelligence algorithm 563.

[0256] 【2. Cell Analysis Device System】

[0257] The waveform data in this embodiment can be obtained in the cell analysis system 5000. Figure 26The appearance of the cell analysis system 5000 is shown. The cell analysis system 5000 includes a measurement unit (also called a measurement section) 600 and processing units 100T and 200T for setting or measuring the measurement conditions of the sample in the measurement unit 600. The measurement unit 600 and the processing units 100T and 200T can be connected to each other via wired or wireless means for communication. Next, examples of the configuration of the measurement unit 600 will be shown. In this embodiment, the explanation is not limited to the following examples. The processing unit 100T or the processing unit 200T can be shared with the training device 100T or the cell analysis device 200T described later. Here, an example will be described where the training device 100T or the cell analysis device 200T is used as the processing unit 100T or the processing unit 200T respectively.

[0258] 【2-1. Cell Analysis System 5000】

[0259] (1) Composition of the first measuring unit

[0260] use Figures 26 to 28 Here is an example of the configuration of the measuring unit 600 when it is a flow cytometer used to detect nucleated cells in a blood sample.

[0261] Figure 27 This figure shows an example of the functional configuration of the measurement unit 600. As shown, the measurement unit 600 includes a detection unit 610 for detecting blood cells, an analog processing unit 620 for the output of the detection unit 610, a measurement unit control unit 680, a display-operation unit 650, a sample modulation unit 640, and a device mechanism unit 630. The analog processing unit 620 performs noise removal processing on the electrical signal, which is an analog signal input from the detection unit, and outputs the processing result as an electrical signal to the A / D conversion unit 682.

[0262] The detection unit 610 functions as a signal acquisition unit and includes at least a nucleated cell detection unit 611 for detecting nucleated cells such as white blood cells, a red blood cell / platelet detection unit 612 for measuring red blood cell count and platelet count, and a hemoglobin detection unit 613 for measuring the amount of blood dye in the blood as needed. Furthermore, the nucleated cell detection unit 611 is composed of an optical detection unit, and more specifically, it is configured for performing flow cytometry detection.

[0263] like Figure 27 As shown, the measurement unit control unit 680 includes: an A / D conversion unit 682, a digital value calculation unit 683, and an interface unit 689 connected to the training device 100T or the cell analysis device 200T. Furthermore, the measurement unit control unit 680 includes an interface unit 486 located between the display and operation units 650 and an interface unit 688 located between the device mechanism units 630.

[0264] Furthermore, the digital value calculation unit 683 is connected to the interface unit 689 via the interface unit 684 and the bus 685. In addition, the interface unit 689 is connected to the display-operation unit 650 via the bus 685 and the interface unit 486, and is connected to the detection unit 610, the device mechanism unit 630 and the sample modulation unit 640 via the bus 685 and the interface unit 688.

[0265] The A / D conversion unit 682 converts the light-receiving signal, which is an analog signal, output from the analog processing unit 620 into a digital signal and outputs it to the digital value calculation unit 683. The digital value calculation unit 683 performs specified calculation processing on the digital signal output from the A / D conversion unit 682. These specified calculation processes include, for example, acquiring the signal intensity of the forward scattered light, the signal intensity of the side scattered light, and the signal intensity of the side fluorescence after the forward scattered light reaches a specified threshold; acquiring waveform data at multiple time points at certain intervals for a single training cell between specified time intervals and the end of acquisition; and extracting the peak values ​​of the waveform data, etc., and are not limited to these. Furthermore, the digital value calculation unit 683 outputs the calculation results (measurement results) to the training device 100T or the cell analysis device 200T via the interface unit 684, the bus 685, and the interface unit 689.

[0266] The training device 100T or the cell analysis device 200T is connected to the digital value calculation unit 683 via the interface unit 684, the bus 685, and the interface unit 689. The calculation results output from the digital value calculation unit 683 can be received by the training device 100T or the cell analysis device 200T. In addition, the training device 100T or the cell analysis device 200T performs control of the device mechanism unit 630, which consists of an automatic sample supply container sampler (not shown), a flow system for sample modulation and measurement, and other controls.

[0267] The nucleated cell detection unit 611 allows a sample containing cells to flow through a cell detection flow path, irradiates the cells flowing through the flow path with light, and measures the scattered light and fluorescence emitted from the cells. The red blood cell / platelet detection unit 612 allows a sample containing cells to flow through a cell detection flow path, measures the electrical resistance of the cells flowing through the flow path, and detects the cell volume.

[0268] In this embodiment, the measurement unit 600 preferably includes a flow cytometer and / or a sheath current resistance detection unit. Figure 27 In this section, the nucleated cell detection unit 611 can be a flow cytometer. Figure 27 In this design, the red blood cell / platelet detection unit 612 can be a sheath current resistance detection unit. Nucleated cells can be measured using the red blood cell / platelet detection unit 612, or red blood cells and platelets can be measured using the nucleated cell detection unit 611.

[0269] (2) Flow cytometer

[0270] like Figure 28 As shown, in the determination using a flow cytometer, when the cells contained in the test sample pass through the flow cell (sheath flow cell) 4113 in the flow cytometer, the light source 4111 illuminates the flow cell 4113, and the scattered light and fluorescence emitted by the cells in the flow cell 4113 from this light are detected.

[0271] In this embodiment, the scattered light is not particularly limited as long as it is scattered light that can be measured by a commonly used flow cytometer. For example, frontal scattered light (e.g., light receiving angle around 0 to 20 orders) and side scattered light (light receiving angle around 90 orders) can be considered as scattered light. It is known that side scattered light reflects internal information of the cell, such as the nucleus or granules, while frontal scattered light reflects information about the cell size. In this embodiment, it is preferable to measure the intensity of both frontal scattered light and side scattered light as the scattered light intensity.

[0272] Fluorescence is the light emitted from a fluorescent dye that binds to nucleic acids or other substances within cells when excited by an appropriate wavelength of light. The excitation wavelength and the light-receiving wavelength correspond to the type of fluorescent dye used.

[0273] Figure 28 This figure shows an example of the optical system configuration of the nucleated cell detection unit 611. In this figure, light emitted from a laser diode, which serves as a light source 4111, is irradiated by an irradiation lens system 4112 onto cells passing through a flow cell 6113.

[0274] In this embodiment, the light source 4111 of the flow cytometer is not particularly limited, and a light source 4111 with a wavelength suitable for the excitation of the fluorescent dye is selected. As such a light source 4111, a semiconductor laser containing a red semiconductor laser and / or a blue semiconductor laser, a gas laser such as an argon laser, a helium-neon laser, or a mercury arc lamp is used. In particular, semiconductor lasers are very inexpensive compared to gas lasers, and are therefore suitable.

[0275] like Figure 28As shown, forward-scattered light emitted from particles passing through flow cell 4113 is received by forward-scattered light receiving element 4116 via condenser lens 4114 and pinhole portion 4115. Forward-scattered light receiving element 4116 can be a photodiode, etc. Side-scattered light is received by side-scattered light receiving element 4121 via condenser lens 4117, dichroic mirror 4118, bandpass filter 4119, and pinhole portion 4120. Side-scattered light receiving element 4121 can be a photodiode, photomultiplier tube, etc. Side fluorescence is received by side-fluorescence light receiving element 4122 via condenser lens 4117 and dichroic mirror 4118. Side-fluorescence light receiving element 4122 can be an avalanche photodiode, photomultiplier tube, etc.

[0276] The optical received signals output from each of the optical receiving elements 4116, 4121, and 4122 are each amplified by amplifiers 4151, 4152, and 4153. Figure 27 The analog processing unit 620 shown performs analog processing such as amplification and waveform processing, and transmits the data to the measurement unit control unit 680.

[0277] Back Figure 27 The measuring unit 600 may also include a sample modulation unit 640 for modulating the measured sample. The sample modulation unit 640 is controlled by the measuring unit information control unit 481 via the interface unit 688 and the bus 685. Figure 29 In the sample preparation unit 640 provided in the measurement unit 600, a blood sample, staining reagent and hemolysis reagent are mixed to prepare a measurement sample, and the pattern of the obtained measurement sample is displayed for measurement by the nucleated cell detection unit.

[0278] exist Figure 29 In the reaction chamber 602, a blood sample is aspirated from sample container 00a using pipette 601. A measured amount of the blood sample is mixed with a specified amount of diluent using pipette 601 and transported to reaction chamber 602. A specified amount of hemolyzing reagent is added to reaction chamber 602. A specified amount of staining reagent is supplied to reaction chamber 602 and mixed with the above mixture. By allowing the blood sample and the mixture of staining reagent and hemolyzing reagent to react in reaction chamber 602 for a specified time, the red blood cells in the blood sample are hemolyzed, resulting in an assay sample in which nucleated cells are stained with a fluorescent dye.

[0279] The obtained test sample, together with the sheath fluid (e.g., CELLPACK(II), manufactured by Sysmex Co., Ltd.), is transferred to the flow cell 4113 in the nucleated cell detection unit 611, where it is measured by flow cytometry.

[0280] (1) Hardware configuration of the training device

[0281] Figure 30The hardware configuration of the training device 100T is illustrated below. The training device 100T includes a control unit 10T, an input unit 16, and an output unit 17. Furthermore, the training device 100T can be connected to a network 99.

[0282] The configuration of the control unit 10T is the same as that of the control unit 20A of the training device 200A. However, the CPU 21, memory 22, storage unit 23, bus 24, I / F unit 25, and GPU 29 in the control unit 20A of the training device 200A are respectively read as CPU 11, memory 12, storage unit 13, bus 14, I / F unit 15, and GPU 19. However, the storage unit 13 houses the third artificial intelligence algorithm 550 and the fourth artificial intelligence algorithm 560.

[0283] The training waveform data 570 can be obtained by the measurement unit 600 and stored in the storage unit 13 or memory 12 of the control unit 10T of the training device 100T.

[0284] (2) Hardware configuration of the analysis device

[0285] Reference Figure 31 The cell analysis device 200T includes a control unit 20, an input unit 26, an output unit 27, and a media driver D98. Furthermore, the cell analysis device 200T can be connected to a network 99.

[0286] The configuration of the control unit 20T is the same as that of the control unit 40A of the cell analysis device 400A. Specifically, the CPU 41, memory 42, storage unit 43, bus 44, I / F unit 45, and GPU 49 in the control unit 40A of the cell analysis device 400A are respectively read as CPU 21, memory 22, storage unit 23, bus 24, I / F unit 25, and GPU 29. However, the storage unit 23 uses multiple trained third artificial intelligence algorithms 560 as described later. Figure 32 The database shown is stored in the image.

[0287] The waveform data 580 used for analysis can be obtained by the measurement unit 600 and stored in the storage unit 23 or memory 22 of the control unit 20T of the cell analysis device 200T.

[0288] (3) Functional composition of the training device

[0289] Reference Figure 32 The control unit 10T of the training device 100T includes a training data generation unit T101, a training data input unit T102, and an algorithm update unit T103. Figure 34 The step S1001 shown corresponds to the training data generation unit T101. Figure 34 The step S1002 shown corresponds to the training data input unit T102. Figure 34The step S1004 shown corresponds to the algorithm update unit T103. The training data database (DB) T104 and the algorithm database (DB) T105(a) and T105(b) can be recorded in the storage unit 13 of the control unit 10T.

[0290] The training waveform data 570a, 570b, and 570c are acquired in advance by the measurement unit 600 and stored in the training data database T104(a) of the control unit 10T. The third artificial intelligence algorithm 550 is stored in the algorithm database T105(b).

[0291] (4) Functional composition of cell analysis device

[0292] exist Figure 33 The functional configuration of the cell analysis device 200T is shown. The cell analysis device 200T includes a data generation unit T201, a data input unit T202, an analysis unit T203, a data database (DB) T204, and algorithm databases (DB) T205(a) and T205(b). Figure 35 The step S2001 shown corresponds to the parsing data generation unit T201. Figure 35 The step S2002 shown corresponds to the parsing data input unit T202. Figure 35 Step S2003 shown corresponds to the analysis unit T403. Waveform data 580 for analysis is acquired by the measurement unit 600 and stored in the analysis data database T204. Multiple trained third artificial intelligence algorithms 560 are stored in the algorithm database T205(a). A fourth artificial intelligence algorithm 563 is stored in the algorithm database T205(b).

[0293] (5) Training Processing

[0294] exist Figure 34 This shows an example of processing performed by the control unit 10T of the training device 100T.

[0295] First, the control unit 10T acquires training waveform data 570a, 570b, and 570c. Training waveform data 570a is the waveform data of forward scattered light, training waveform data 570b is the waveform data of side scattered light, and training waveform data 570c is the waveform data of side fluorescence. The acquisition of training waveform data 570a, 570b, and 570c is performed by the operator, received from the measurement unit 600, or received from the media driver D98, and transmitted via the network through the I / F unit 15. When acquiring training waveform data 570a, 570b, and 570c, information on whether the training waveform data 570a, 570b, and 570c represents any cell type is also acquired. Information indicating the cell type is associated with the training waveform data 570a, 570b, and 570c, and can also be input by the operator from the input unit 16.

[0296] In step S1001, the control unit 10T assigns information associated with the training waveform data 570a, 570b, and 570c, indicating whether it is any of the cell types, to the cell type-associated tag values ​​stored in the memory 12 or storage unit 13, and to the tag values ​​577 corresponding to the sequence data 572a, 572b, and 572c, which synchronize the waveform data of frontal scattering, side scattering, and lateral fluorescence at the time of acquiring the waveform data. Thus, the control unit 10T generates training data 575.

[0297] In step S1002, the control unit 10T uses training data 575 to train either the third artificial intelligence algorithm 550 or the fourth artificial intelligence algorithm 553. The training results of the third artificial intelligence algorithm 550 and the fourth artificial intelligence algorithm 553 are accumulated to the training degree using multiple training data 575. When the label value 577 of the training data 575 shows neutrophils, lymphocytes, monocytes, eosinophils, basophils, and immature granulocytes, the third artificial intelligence algorithm 550 is trained; when the label value 577 of the training data 575 shows abnormal cells, the fourth artificial intelligence algorithm 553 is trained.

[0298] In the cell category parsing method of this embodiment, in order to use a folded neural network, a probability gradient descent method is used. In step S1003, the control unit 10T determines whether to accumulate the training result of a predetermined number of trials. If the training result has accumulated the specified number of trials ("YES"), the control unit 10T proceeds to step S1004; if the training result has not accumulated the specified number of trials ("NO"), the control unit 10T proceeds to step S15.

[0299] Next, when the training results have been accumulated for a specified number of trials, in step S1004, the control unit 10T updates the weights w of the third artificial intelligence algorithm 550 or the fourth artificial intelligence algorithm 553 using the training results accumulated in step S1002. In the cell category parsing method according to this embodiment, in order to use the probability gradient descent method, the weights w of the third artificial intelligence algorithm 550 or the fourth artificial intelligence algorithm 553 are updated during the stage of accumulating training results for a specified number of trials.

[0300] In step S1005, the control unit 10T determines whether to train the third artificial intelligence algorithm 550 or the fourth artificial intelligence algorithm 553 with a predetermined number of training data 575. If training is performed with the predetermined number of training data 575 (in the case of "YES"), the training process ends.

[0301] When the control unit 10T determines that the third artificial intelligence algorithm 550 or the fourth artificial intelligence algorithm 553 has not been trained with a specified number of training data 575 in step S1005 (the case of "NO"), it proceeds to steps S1005 to S1006 and performs the processing of steps S1001 to S1005 on the following training waveform data 570.

[0302] Based on the processing described above, the control unit 10T trains the third artificial intelligence algorithm 550 and the fourth artificial intelligence algorithm 553, generating the third artificial intelligence algorithm 560 and the fourth artificial intelligence algorithm 563. The third artificial intelligence algorithm 560 and the fourth artificial intelligence algorithm 563 can be recorded on a computer.

[0303] (6) Cell analysis and processing

[0304] First, the control unit 20T acquires waveform data 580a, 580b, and 580c for analysis. The acquisition of waveform data 580a, 580b, and 580c is performed by user operation, or automatically from the measurement unit 600, or from the recording medium 98, via the network through the I / F unit 25.

[0305] Control unit 20T Figure 35 In step S2000, the input unit 26 accepts the inspection items. Specifically, the inspection items are accepted by reading information about the inspection items from the barcodes attached to each sample using a barcode reader, which is an example of the input unit 46. The inspection items are selected from "blood cell classification test" and "abnormal cell test". In step S2001, the control unit 20T generates analysis data 585 about the cells from the sequence data 582a, 582b, and 582c in the order described in the cell analysis method above. The control unit 20T stores the generated analysis data 585 in the storage unit 23 or the memory 22.

[0306] In step S2002, the control unit 20T selects either the third artificial intelligence algorithm 560 or the fourth artificial intelligence algorithm 563, corresponding to the inspection item accepted in step S2000. When the inspection item is "blood cell classification examination," the control unit 20T selects the third artificial intelligence algorithm 560; when the inspection item is "abnormal cell examination," it selects the fourth artificial intelligence algorithm 563.

[0307] In step S2003, the control unit 20T uses the selected third artificial intelligence algorithm 560 or the fourth artificial intelligence algorithm 563 to classify the target cells and stores the label value 577 of the judgment result in the storage unit 43 or the memory 42.

[0308] In step S2004, the control unit 20T determines whether all target cells have been analyzed. If all have been analyzed ("YES"), the process proceeds to step S2005, where the analysis result is stored in the storage unit 43 and output to the output unit. If not all have been analyzed in step S2004 ("NO"), the control unit 20T updates the target cells in step S2006 and repeats steps S2001 to S2004 until all have been analyzed.

[0309] The above-described embodiments enable the identification of cell types regardless of the examiner's skill level. Furthermore, in cell analysis, the analysis of multiple analytical items becomes easier.

[0310] (7) Various procedures

[0311] The second embodiment includes a computer program for training an artificial intelligence algorithm, which causes a computer to perform the processes of steps S1001 to S1006.

[0312] The second embodiment includes a computer program for analyzing cells, which causes a computer to perform the processes of steps S2000 to S2006 and S22001 to S22002.

[0313] Furthermore, one embodiment of this invention relates to a program article, such as a storage medium, that stores the aforementioned computer program. That is, the computer program can be stored in a semiconductor memory element such as a hard disk or flash memory, or a storage medium such as an optical disc. The form in which the program is recorded to the storage medium is not limited as long as the training device 200A can read the program. The recording to the storage medium is preferably non-volatile.

[0314] Among them, "program" refers not only to programs that can be directly executed by the CPU, but also includes the concepts of source programs, compressed programs, encrypted programs, etc.

[0315] [IV. Other]

[0316] This invention is not to be interpreted in a manner limited to the embodiments described above. For example, in the embodiments described above, the algorithm used to select the accepted inspection item may also be used to select the parsing item, and the algorithm used to select the parsing item may also be used to select the accepted parsing item.

[0317] [Explanation of Symbols]

[0318] 82, 87, 585: Data for parsing

[0319] 60, 63, 560, 563: Artificial Intelligence Algorithms

[0320] 84, 88, 582: Data showing cell traits

[0321] 400A, 200B, 200T: Cell Analysis Device

[0322] 1000, 5000: Cell Analysis System

[0323] 110: Flow pool

[0324] 120, 121, 122, 123: Light source

[0325] 160: Camera Department

Claims

1. A cell analysis method for analyzing cells, comprising: Accept the inspection item or the parsing item, or accept a parsing mode from multiple parsing modes. The sample containing cells labeled at the target site flows through the flow path. Multiple analytical object images of the same cell are generated by capturing images of the cells passing through the flow path in different wavelength regions within the same field of view. One or more of the parsed object images are selected from a plurality of parsed object images based on the accepted inspection items, parsing items, or parsing modes. Data for analyzing the cells contained in the sample is generated based on one or more selected images of the target object. An artificial intelligence algorithm is selected from multiple artificial intelligence algorithms pre-stored in the storage unit. This artificial intelligence algorithm accepts the generated parsing data as input. The selected artificial intelligence algorithm generates data displaying the characteristics of the cells based on the parsed data. The artificial intelligence algorithm mentioned above is selected based on the accepted inspection items, parsing items, or parsing patterns, and The data used for analysis includes row and column data showing the brightness of each pixel in one or more of the selected images of the analysis objects.

2. The cell analysis method according to claim 1, wherein the plurality of analysis object images include: Capture the first fluorescent image of the first fluorescent label present in the nucleus, and Image the second fluorescent marker present in the nucleus.

3. The cell analysis method according to claim 1, wherein the plurality of analysis object images include: The bright-field image of the cell, and Take fluorescent images of the fluorescently labeled cells.

4. The cell analysis method of claim 1, wherein the analysis mode is received via a mode receiving screen for receiving the analysis mode.

5. The cell analysis method according to claim 1, wherein the artificial intelligence algorithm is a deep learning algorithm with a neural network structure.

6. The cell analysis method of claim 1, wherein the plurality of artificial intelligence algorithms includes at least one of the following algorithms: Algorithms for generating data that indicate whether cells have chromosomal abnormalities, and An algorithm that generates data indicating whether a cell is a peripheral circulating tumor cell.

7. The cell analysis method of claim 1, wherein the plurality of artificial intelligence algorithms are stored in the same computer.

8. A cell analysis system, which includes: The flow cell through which the sample containing cells labeled at the target site flows. A light source used to illuminate the sample flowing through the flow cell. An imaging unit that captures images of cells in the sample irradiated with said light, and Control Department The camera unit captures images of the cells passing through the flow path in different wavelength regions within the same field of view, thereby generating multiple analytical object images of the same cell. The control unit is configured as follows: Accept the inspection item or the parsing item, or accept a parsing mode from multiple parsing modes. One or more of the parsed object images are selected from a plurality of parsed object images based on the accepted inspection items, parsing items, or parsing modes. Data for analyzing the cells contained in the sample is generated based on one or more selected images of the target object. An artificial intelligence algorithm is selected from multiple AI algorithms pre-stored in the storage unit, wherein the AI ​​algorithm accepts the generated parsing data as input. The selected artificial intelligence algorithm generates data based on the parsed data, displaying the characteristics of the cells contained in the parsed object image. The artificial intelligence algorithm mentioned above is selected based on the accepted inspection items, parsing items, or parsing patterns, and The data used for analysis includes row and column data showing the brightness of each pixel in one or more of the selected images of the analysis objects.

Images