Cell image analysis method

Abstract

This cell image analysis method comprises: a step for acquiring a cell image (10) in which cells (90) appear; a step for inputting the cell image to a learned model (6) having been trained to classify cells into two or more groups; a step for acquiring an index value (20) that shows a certainty of into which of the two or more groups the cells appearing in the cell image are classified, on the basis of the analysis results of each pixel of the cell image outputted by the learned model; and a step for displaying the index value thus acquired.

Description

[Technical Field]

[0001] The present invention relates to a cell image analysis method, and more particularly to a cell analysis method for analyzing cells using a trained model. [Background technology]

[0002] Conventionally, a cell analysis method for analyzing cells using a trained model has been known. Such a cell analysis method is disclosed, for example, in International Publication No. 2019 / 171546.

[0003] WO 2019 / 171546 discloses a cell image analysis method for analyzing cell images captured by an imaging device. Specifically, WO 2019 / 171546 discloses a configuration for acquiring cell images by photographing cells cultured in a culture plate with an imaging device such as a microscope. The cell image analysis method disclosed in WO 2019 / 171546 uses the analysis results of a trained model to classify cells depicted in the cell image as normal or abnormal. WO 2019 / 171546 also discloses a configuration for classifying cells by a segmentation process that determines which category each pixel belongs to for each pixel in the cell image. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] International Publication No. 2019 / 171546 Summary of the Invention [Problem to be solved by the invention]

[0005] Although not disclosed in International Publication No. 2019 / 171546, when classifying cells in a cell image using the analysis results of a trained model, classification is performed for each pixel using the highest value of the analysis results for each pixel. However, when classifying cells using the highest value, the same classification result will be obtained regardless of whether the difference between the highest value and the next highest value is small or not. In other words, when classifying cells in a cell image, the same classification result will be obtained as long as the analysis result has the largest value, regardless of whether the classification accuracy is high or not. Therefore, it may be difficult for an operator to grasp the classification accuracy of cells in a cell image at a glance. Therefore, a cell image analysis method that allows the operator to easily grasp the classification accuracy of cells in a cell image is desired.

[0006] The present invention has been made to solve the above-mentioned problems, and one object of the present invention is to provide a cell image analysis method that makes it possible to easily grasp the accuracy of the classification of cells depicted in a cell image. [Means for solving the problem]

[0007] In order to achieve the above object, a cell image analysis method according to one aspect of the present invention includes the steps of: acquiring a cell image in which a cell is captured; inputting the cell image into a trained model that has been trained to classify cells into two or more types; acquiring an index value representing a probability that a cell captured in the cell image belongs to one of the two or more types based on an analysis result of each pixel of the cell image output by the trained model; and displaying the acquired index value. The trained model is trained to output a probability value that is an estimate of the classification of each pixel of the cell image as an analysis result, and in the step of acquiring an index value, one representative value that is one of the mean, median, maximum, minimum, and mode of the probability values ​​for each pixel of the cell image output by the trained model is acquired as one index value that indicates the accuracy of classification for one cell image. . [Effects of the Invention]

[0008] The cell image analysis method according to the above aspect includes the steps of: acquiring an index value representing the accuracy of which of two or more classifications the cell image belongs to, based on the analysis results of each pixel of the cell image output by the trained model; and displaying the acquired index value. This allows the operator to easily grasp the accuracy of the classification of the cells in the cell image by checking the index value, since the index value representing the accuracy of which of the two or more classifications the cells in the cell image belong to is displayed. As a result, a cell image analysis method that allows the operator to easily grasp the accuracy of the classification of the cells in the cell image can be provided. [Brief explanation of the drawings]

[0009] [Figure 1] 1 is a schematic diagram showing the overall configuration of a cell image analyzer according to an embodiment. [Figure 2] FIG. 1 is a schematic diagram for explaining a cell image. [Figure 3] FIG. 2 is a schematic diagram illustrating cells cultured in a culture vessel. [Figure 4] A schematic diagram for explaining a method for learning a learning model according to one embodiment and a method for analyzing cell images using the learned model. [Figure 5] 1A to 1C are schematic diagrams illustrating differences in cell images due to different imaging conditions. [Figure 6] FIG. 2 is a schematic diagram for explaining a configuration in which an image processing unit according to an embodiment generates a probability distribution image. [Figure 7] 1A to 1C are schematic diagrams for explaining a superimposed cell image generated by a superimposed cell image generating unit according to one embodiment. [Figure 8] FIG. 10 is a schematic diagram for explaining differences in numerical data of representative values ​​of probability values ​​due to differences in focus of a cell image. [Figure 9] FIG. 10 is a schematic diagram for explaining a configuration for displaying a frequency distribution of probability values ​​when a cell image is in focus. [Figure 10]FIG. 10 is a schematic diagram for explaining a configuration for displaying a frequency distribution of probability values ​​when a cell image is out of focus. [Figure 11] 1 is a schematic diagram for explaining a configuration in which a cell analysis device according to one embodiment displays a superimposed cell image, a representative value of probability values, and a frequency distribution. [Figure 12] 1A and 1B are schematic diagrams illustrating the difference in superimposed cell images due to differences in the coating agent of the culture vessel in which the cells are cultured. [Figure 13] FIG. 10 is a schematic diagram for explaining differences in numerical data of representative values ​​of probability values ​​due to differences in coating agents. [Figure 14] 1A to 1D are schematic diagrams illustrating the difference in superimposed cell images due to the difference in the number of days of cell culture. [Figure 15] FIG. 10 is a schematic diagram for explaining differences in the numerical data of representative values ​​of probability values ​​due to differences in the number of days of cell culture. [Figure 16] 10 is a flowchart for explaining a process in which a cell image analyzing device according to one embodiment displays an index value, a frequency distribution, and a superimposed cell image. [Figure 17] 10 is a flowchart illustrating a process for generating a trained model by a cell image analysis apparatus according to an embodiment. [Figure 18] 10 is a flowchart for classifying cell images by a cell image analysis apparatus according to one embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0010] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, specific embodiments of the present invention will be described with reference to the accompanying drawings.

[0011] The configuration of a cell-image analyzing apparatus 100 according to one embodiment will be described with reference to FIG.

[0012] (Configuration of cell image analyzer) As shown in FIG. 1, the cell image analyzer 100 includes an image acquisition unit 1, a processor 2, a storage unit 3, a display unit 4, and an input reception unit 5.

[0013] The image acquisition unit 1 is configured to acquire a cell image 10. The cell image 10 is an image of a cell 90 (see FIG. 2). Specifically, the cell image 10 is an image of a cultured cell 90 cultured in a culture solution 81 (see FIG. 3) filled in a culture vessel 80 (see FIG. 3). In this embodiment, the image acquisition unit 1 is configured to acquire the cell image 10 from a device that captures the cell image 10, such as a microscope 8 equipped with an imaging device. The image acquisition unit 1 includes, for example, an input / output interface.

[0014] The processor 2 is configured to analyze the acquired cell image 10. The processor 2 includes a central processing unit (CPU), read-only memory (ROM), random access memory (RAM), a graphics processing unit (GPU), or a field-programmable gate array (FPGA) configured for image processing. The processor 2, which is comprised of hardware such as a CPU, also includes software (program) functional blocks, including a control unit 2a, an image analysis unit 2b, an image processing unit 2c, and a superimposed cell image generation unit 2d. The processor 2 executes programs stored in the storage unit 3 to function as the control unit 2a, the image analysis unit 2b, the image processing unit 2c, and the superimposed cell image generation unit 2d. The control unit 2a, the image analysis unit 2b, the image processing unit 2c, and the superimposed cell image generation unit 2d may be configured individually by hardware using dedicated processors (processing circuits).

[0015] The control unit 2a is configured to control the cell image analyzer 100. The control unit 2a is also configured to acquire an index value 20 that indicates the probability that a cell 90 depicted in a cell image 10 is classified into one of two or more types. Specifically, the control unit 2a is configured to acquire, as the index value 20, a representative value 20a of the probability values ​​21 acquired based on the probability values ​​21 (see FIG. 4) output by the trained model 6. The index value 20 is a real value that indicates the probability that a cell 90 depicted in the cell image 10 is classified into one of two or more types. In this embodiment, the index value 20 is a numerical value in the range of 0 to 100. In this embodiment, the control unit 2a outputs one index value 20 for one cell image 10.

[0016] More specifically, the control unit 2a is configured to acquire at least one of the index values ​​20 indicating whether the cell image 10 was in focus when captured, whether the coating agent on the culture vessel 80 (see FIG. 3) is appropriate, and whether the number of days of culture is appropriate. The probability value 21 is an estimated value of classification output by the trained model 6 as an analysis result. The trained model 6 outputs the probability value 21 for each pixel of the cell image 10 as an analysis result.

[0017] The control unit 2a is also configured to control the display of the superimposed cell image 50 on the display unit 4. The configuration by which the control unit 2a acquires the index value 20 and the details of the superimposed cell image 50 will be described later.

[0018] In this embodiment, the image analysis unit 2b classifies cells 90 (see FIG. 2) into two or more types. Specifically, the image analysis unit 2b is configured to classify cells 90 shown in the cell image 10 into two or more types using a trained model 6 that has learned to classify cells 90 into two or more types. The trained model 6 also includes a first trained model 6a, a second trained model 6b, and a third trained model 6c that perform classification according to the imaging conditions and culture conditions described below. Details of normal cells, abnormal cells, the first trained model 6a, the second trained model 6b, and the third trained model 6c will be described later.

[0019] The image processing unit 2c is configured to generate a probability distribution image 12 (see FIG. 6) described later. The image processing unit 2c is also configured to acquire a cell region, which is a region of a cell 90 depicted in the cell image 10, based on the probability distribution image 12. Details of the configuration of the image processing unit 2c for generating the probability distribution image 12 and the configuration for acquiring the cell region will be described later.

[0020] The superimposed cell image generating unit 2d is configured to generate a superimposed cell image 50 in which the distribution of probability values ​​21 (see FIG. 4) is superimposed on the cell image 10. Details of the configuration in which the superimposed cell image generating unit 2d generates the superimposed cell image 50 will be described later.

[0021] The storage unit 3 is configured to store a cell image 10, a first trained model 6a, a second trained model 6b, and a third trained model 6c. The storage unit 3 is also configured to store various programs executed by the processor 2. The storage unit 3 includes a storage device such as a hard disk drive (HDD) or a solid state drive (SSD).

[0022] The display unit 4 is configured to display the superimposed cell image 50 generated by the superimposed cell image generating unit 2d, the index value 20, and the frequency distribution 22. The display unit 4 includes, for example, a display device such as a liquid crystal monitor.

[0023] The input receiving unit 5 is configured to be able to receive operation inputs from an operator. The input receiving unit 5 includes input devices such as a mouse and a keyboard.

[0024] (Cell image) The cell image 10 will be described with reference to FIG. 2. The cell image 10 is an image of a cultured cell 90. In this embodiment, the cell image 10 is a microscopic image captured by a microscope 8 equipped with an imaging device. The cell image 10 is an image of a cell 90 having differentiation potential as the cultured cell 90. For example, the cell 90 includes an iPS cell (induced pluripotent stem cell), an ES cell (embryonic stem cell), etc. An undifferentiated cell is a cell that has differentiation potential. An undifferentiated deviation cell is a cell that has begun to differentiate into a specific cell but no longer has differentiation potential. In this embodiment, the undifferentiated cell is considered to be a normal cell. An undifferentiated deviation cell is considered to be an abnormal cell.

[0025] (Cultured cells cultivated in a culture vessel) Next, with reference to FIG. 3, the cultured cells 90 cultured in the culture vessel 80 will be described.

[0026] As shown in Fig. 3, cells 90 are cultured cells cultured in a culture solution 81 filled in a culture vessel 80. In this embodiment, cell image 10 is an image including cultured cells 90 cultured in the culture vessel 80. A coating agent for culturing cells 90 is applied to a bottom surface 80a of the culture vessel 80. The coating agent contains proteins and the like that are necessary for the cells 90 to settle in the culture vessel 80.

[0027] (Image analysis method) Next, with reference to FIG. 4, a method for analyzing a cell image 10 using a cell image analysis method according to this embodiment will be described. In this embodiment, a configuration will be described in which a cell image analysis device 100 (see FIG. 1) analyzes the cell image 10 to classify cells 90 depicted in the cell image 10 into two or more types. In this embodiment, the cell image analysis device 100 analyzes the cell image 10 using a trained model 6 (see FIG. 1) to determine which of the two or more types the cell 90 depicted in the cell image 10 belongs to. When the cell image 10 is input, the trained model 6 outputs a probability value 21 for each pixel of the cell image 10. The probability value 21 is an estimated value of the classification.

[0028] Fig. 4 is a block diagram showing the flow of image processing according to this embodiment. As shown in Fig. 4, in this embodiment, the cell image analysis method is broadly divided into an image analysis method 101 and a method 102 for generating a trained model 6 (see Fig. 1).

[0029] (Generating a learning model) A method 102 for generating a trained model 6 according to this embodiment generates a trained model 6 by training a training model 7 using a cell image 10. Specifically, the trained model 6 is generated by training the training model 7 to output a probability value 21 for each pixel of the cell image 10 as an analysis result. As shown in FIG. 4, the method 102 for generating a trained model 6 includes a step 102a of inputting a teacher cell image 30 into the training model 7, and a step 102b of training the training model 7 to output a teacher correct answer image 31. The trained model 6 is, for example, a convolutional neural network (CNN) shown in FIG. 4, or includes a convolutional neural network as part of the CNN. The trained model 6 generated by training the training model 7 is stored in the memory unit 3 (FIG. 1) of the cell image analysis device 100.

[0030] In this embodiment, the trained model 6 is created by learning to classify at least one of whether the cell image 10 was in focus when captured, whether the coating agent on the culture vessel 80 is appropriate, and whether the number of days of culture is appropriate. A method 102 for generating the trained model 6 generates the trained model 6 using a teacher cell image 30, which is the cell image 10, and a teacher correct answer image 31 to which the cell image 10 is assigned label values ​​for at least two types of capturing conditions corresponding to the classification, or to which the cell image 10 is assigned label values ​​for at least two types of culture conditions corresponding to the classification.

[0031] In this embodiment, the trained model 6 includes a first trained model 6a, a second trained model 6b, and a third trained model 6c. The first trained model 6a is a trained model that has learned to classify, based on a cell image 10, under which type of imaging condition a cell 90 depicted in the cell image 10 was captured, among images captured under two or more types of imaging conditions. In other words, the teacher cell images 30 used to generate the first trained model 6a are cell images 10 captured under different imaging conditions.

[0032] Furthermore, the teacher supervised image 31 is an image in which different label values ​​are assigned to each pixel depending on the imaging conditions. Specifically, the teacher supervised image 31 is an image in which label values ​​of two or more types of imaging conditions are assigned to each pixel. The imaging conditions are whether the cell image 10 (teacher cell image 30) is in focus or not. Therefore, the teacher supervised image 31 is an image in which a label value indicating that the cell image 10 is in focus and a label value indicating that the cell image 10 is out of focus are assigned to each pixel. In other words, the teacher supervised image 31 is an image divided into two classes: an in-focus class and an out-of-focus class. In this way, the learning model 7 is trained to classify each pixel of an input image into one of two or more types under the imaging conditions, thereby generating the first trained model 6a.

[0033] The second trained model 6b and the third trained model 6c are trained models that have learned to classify, based on the cell image 10, whether the cell 90 depicted in the cell image 10 is an image of a cell 90 cultured under two or more types of culture conditions. Specifically, when generating the second trained model 6b and the third trained model 6c, cell images 10 with different culture conditions are used as the teacher cell images 30. Furthermore, an image in which different label values ​​are assigned to each pixel depending on the difference in culture conditions is used as the teacher correct answer image 31. Specifically, the teacher correct answer image 31 is an image in which label values ​​for two or more types of culture conditions are assigned to each pixel. Note that the culture conditions include differences in the coating agent of the culture vessel 80 (see FIG. 3) in which the cells 90 are cultured and differences in the number of days the cells 90 have been cultured.

[0034] That is, when generating the second trained model 6b, a teacher supervised image 31 to which at least two types of label values ​​related to the coating agent of the culture vessel 80 in which the cells 90 are cultured are assigned is used. Specifically, when generating the second trained model 6b, an image in which each pixel is assigned a label value indicating that the coating agent of the culture vessel 80 in which the cells 90 are cultured is coating agent A and a label value indicating that the coating agent is not coating agent A is used as the teacher supervised image 31. That is, the teacher supervised image 31 is an image divided into two classes: a class in which the coating agent is A and a class in which the coating agent is B.

[0035] Furthermore, when generating the third trained model 6c, a supervising teacher image 31 to which at least two types of label values ​​related to the number of days for which the cells 90 have been cultured are assigned is used. Specifically, when generating the third trained model 6c, an image in which a label value indicating that the number of days for which the cells 90 have been cultured is a predetermined number of days and a label value indicating that the number of days for which the cells 90 have been cultured is not a predetermined number of days is assigned to each pixel is used as the supervising teacher image 31. In this embodiment, the predetermined number of days for which the cells 90 have been cultured is, for example, five days. That is, the supervising teacher images 31 are images divided into two classes: a class for which the number of days for which the cells have been cultured is five days and a class for which the number of days for which the cells have been cultured is not five days.

[0036] In this embodiment, the trained model 6 is created by learning to classify whether cells 90 are suitable for analysis as normal cells or abnormal cells by classifying the cells 90 into two or more types under the imaging conditions or culture conditions. Also, in this embodiment, the trained model 6 is created by learning to classify cells 90 of the same type as to whether they are suitable for analysis as normal cells or abnormal cells.

[0037] (Image analysis method) An image analysis method 101 according to this embodiment is an image analysis method for classifying a cell 90 depicted in a cell image 10 acquired by an image acquisition unit 1 from a microscope 8 (see FIG. 1) or the like as one of two or more types. The image analysis method 101 according to this embodiment includes the steps of acquiring a cell image 10 depicting a cell 90 (see FIG. 2), inputting the cell image 10 to a trained model 6, acquiring an index value 20 representing the likelihood that the cell 90 depicted in the cell image 10 is classified into one of two or more types based on the analysis results of each pixel of the cell image 10 output by the trained model 6, and displaying the acquired index value 20. Detailed processing of each step of the image analysis method 101 will be described later.

[0038] In this embodiment, the step of acquiring a cell image 10 is performed by the image acquisition unit 1. The image acquisition unit 1 acquires the cell image 10 from an image capturing device such as a microscope 8 (see FIG. 1). The image acquisition unit 1 also outputs the acquired cell image 10 to the image analysis unit 2b. The image acquisition unit 1 also outputs the acquired cell image 10 to the superimposed cell image generation unit 2d.

[0039] In this embodiment, the step of analyzing the cell image 10 is performed by the image analysis unit 2b. The image analysis unit 2b inputs the cell image 10 to the trained model 6 to acquire the index value 20. Specifically, the image analysis unit 2b inputs the cell image 10 to any one of the first trained model 6a, the second trained model 6b, and the third trained model 6c to acquire the index value 20. The control unit 2a determines whether the image analysis unit 2b will perform the analysis using the first trained model 6a, the second trained model 6b, or the third trained model 6c. The image analysis unit 2b outputs the acquired index value 20 to the control unit 2a and the superimposed cell image generation unit 2d. Specifically, the image analysis unit 2b outputs a probability value 21 as the index value 20 to the control unit 2a and the superimposed cell image generation unit 2d.

[0040] The control unit 2a determines, based on an operation input from an operator, whether to perform analysis using the first trained model 6a, the second trained model 6b, or the third trained model 6c. Specifically, the control unit 2a determines, based on an operation input indicating under what conditions the cell image 10 will be analyzed, whether to perform analysis using the first trained model 6a, the second trained model 6b, or the third trained model 6c.

[0041] Furthermore, the control unit 2a acquires a representative value 20a of the probability values ​​21 based on the probability values ​​21. In this embodiment, the control unit 2a acquires one representative value 20a for one cell image 10 based on the probability values ​​21 acquired for each pixel of the cell image 10. In this embodiment, the control unit 2a is configured to acquire the average value of the probability values ​​21 as the representative value 20a.

[0042] Furthermore, the control unit 2a acquires a frequency distribution 22 of the probability values ​​21 based on the probability values ​​21. The control unit 2a also causes the acquired representative values ​​20a and frequency distribution 22 to be displayed on the display unit 4. Details of the configuration by which the control unit 2a acquires the representative values ​​20a and frequency distribution 22 will be described later.

[0043] The superimposed cell image generating unit 2d generates a superimposed cell image 50 based on the cell image 10 and the index value 20. The superimposed cell image generating unit 2d also causes the display unit 4 to display the generated superimposed cell image 50.

[0044] (Differences in cell images due to differences in imaging conditions) Next, differences in the cell image 10 due to differences in imaging conditions will be described with reference to Fig. 5. In this embodiment, the difference in imaging conditions is whether or not the cell image 10 is in focus when it is captured. Note that cell images 10a to 10c shown in Fig. 5(A) to Fig. 5(C) were captured at the same location in the culture vessel 80 (see Fig. 3) with the focal position changed.

[0045] The cell image 10a shown in FIG. 5(A) is an in-focus cell image 10. That is, the cell image 10a is an image in which the contrast of the cells 90 is high. In other words, the cell image 10a is an image in which the contours of the cells 90 are clear. Note that an in-focus image does not mean that all of the cells 90 depicted in the cell image 10 are in focus, but rather that the center of the cell image 10 is in focus. That is, the focus of the cells 90 depicted in the cell image 10a is not uniform, and the degree of defocus increases with distance from the center of the image. That is, there may be out-of-focus cells 90 in the cell image 10a.

[0046] The cell image 10b shown in FIG. 5(B) is an out-of-focus cell image 10. That is, the cell image 10b shown in FIG. 5(B) is an image in which the contrast of the cell 90 is low. In other words, the cell image 10b is an image in which the outline of the cell 90 is unclear. Furthermore, the cell image 10b is an image in which the degree of out-of-focus (degree of defocus) is smaller than that of the cell image 10c shown in FIG. 5(C). Note that the "defocus -1" label in FIG. 5(B) indicates that the degree of defocus of the cell image 10b is smaller than that of the cell image 10c shown in FIG. 5(C). Furthermore, in the cell image 10b shown in FIG. 5(B), the outline of the cell 90 is shown with a dashed line to indicate that it is out of focus. Furthermore, in the cell image 10b shown in FIG. 5(B), the focus of the cell 90 in the image is not uniform, and the degree of defocus increases as the image moves away from the center of the image.

[0047] The cell image 10c shown in FIG. 5(C) is an out-of-focus cell image 10. That is, the cell image 10c shown in FIG. 5(C) is an image in which the contrast of the cell 90 is low. In other words, the cell image 10c is an image in which the outline of the cell 90 is unclear. Furthermore, the cell image 10c is an image in which the degree of out-of-focus (degree of defocus) is greater than that of the cell image 10b. That is, the cell image 10c is an image in which the outline of the cell 90 is less clear. Note that the "defocus -2" label in FIG. 5(C) indicates that the degree of defocus of the cell image 10c is greater than that of the cell image 10b shown in FIG. 5(B). Furthermore, the cell image 10c shown in FIG. 5(C) does not show the outline of the cell 90, thereby indicating that it is more out-of-focus than the cell image 10b. Also in the cell image 10b shown in FIG. 5(B), the cells 90 in the image are not uniformly focused, and the degree of defocus increases as the image moves away from the center.

[0048] (Probability distribution image) In this embodiment, the image analysis unit 2b uses the first trained model 6a to classify the cells 90 shown in the cell images 10a to 10c into two or more types.

[0049] Specifically, the image analysis unit 2b inputs cell images 10a to 10c into the first trained model 6a, and generates a probability distribution image 12 based on the probability value 21 output from the first trained model 6a.

[0050] In the example shown in FIG. 6, the image analysis unit 2b acquires a probability value 21 by inputting an in-focus cell image 10a to the first trained model 6a. That is, in the example shown in FIG. 6, the image analysis unit 2b acquires a probability value 21 for each pixel of the cell image 10a. Furthermore, the image analysis unit 2b outputs the acquired probability value 21 to the image processing unit 2c. In the example shown in FIG. 6, the image analysis unit 2b acquires, as the probability value 21, an estimated value of the in-focus class of each pixel of the cell image 10.

[0051] As shown in FIG. 6, the image processing unit 2c generates a probability distribution image 12, which is an image showing the distribution of probability values ​​21. The probability distribution image 12 is an image in which probability values ​​21, which are estimated values ​​of classification, are distributed as pixel values. The probability distribution image 12 shown in FIG. 6 is an image showing the distribution of probability values ​​21, which are estimated values ​​of the class in which each pixel of the cell image 10 is in focus. In the example shown in FIG. 6, differences in the probability values ​​21 are represented by different hatching. As shown in legend 8, the probability value 21 increases in the order of black, dark hatching, and light hatching. Furthermore, as shown in legend 8, one hatching does not represent one probability value 21, but one hatching is added for each probability value 21 in a predetermined range.

[0052] Although not shown in Figure 6, the image analysis unit 2b also obtains a probability distribution image 12 showing the distribution of probability values ​​21, which are estimated values ​​of the out-of-focus class, by inputting a cell image 10 to the first trained model 6a.

[0053] In addition, the image processing unit 2c similarly acquires a probability distribution image 12 of the in-focus class and a probability distribution image 12 of the out-of-focus class for the out-of-focus cell image 10b (see Figure 5) and the cell image 10c (see Figure 5).

[0054] (Overlapping cell images and differences in overlapping cell images due to differences in imaging conditions) Next, referring to Figure 7, the superimposed cell image 50 (see Figure 1) and the differences in the superimposed cell image 50 due to differences in imaging conditions will be described. The superimposed cell image generating unit 2d generates the superimposed cell image 50 based on the cell image 10 and the probability distribution image 12. Specifically, the superimposed cell image generating unit 2d generates the superimposed cell image 50 using the cell image 10 and the probability distribution image 12 acquired for at least two types of label values.

[0055] Specifically, the superimposed cell image generating unit 2d generates a superimposed cell image 50 by superimposing, on the cell image 10, markers that allow for identification of differences in probability values ​​21 based on the probability distribution image 12. In this embodiment, the superimposed cell image generating unit 2d superimposes markers that allow for identification of probability values ​​21 of label values ​​of two or more types of classifications. Specifically, the superimposed cell image generating unit 2d superimposes markers that allow for identification of probability values ​​21 of label values ​​of two or more types of imaging conditions. More specifically, the superimposed cell image generating unit 2d superimposes markers that allow for identification of probability values ​​21 of in-focus label values ​​and probability values ​​21 of out-of-focus label values ​​on the cell image 10. For example, the superimposed cell image generating unit 2d superimposes a blue marker 51 on the probability value 21 of in-focus label values. Furthermore, the superimposed cell image generating unit 2d superimposes a red marker 52 on the probability value 21 of out-of-focus label values. In the example shown in Fig. 7, the blue marker 51 is represented by hatching with the narrowest spacing, as shown in legend 9. Also, in the example shown in Fig. 7, the red marker 52 is represented by hatching with the widest spacing, as shown in legend 9.

[0056] The superimposed cell image 50a shown in FIG. 7(A) is an image in which the distribution of probability values ​​21 obtained by inputting the cell image 10a (see FIG. 5(A)) into the first trained model 6a is superimposed on the in-focus cell image 10a. The superimposed cell image 50b shown in FIG. 7(B) is an image in which the distribution of probability values ​​21 obtained by inputting the cell image 10b (see FIG. 5(B)) into the first trained model 6a is superimposed on the out-of-focus cell image 10b. The superimposed cell image 50c shown in FIG. 7(C) is an image in which the distribution of probability values ​​21 obtained by inputting the cell image 10c into the first trained model 6a is superimposed on the out-of-focus cell image 10c (see FIG. 5(C)). In the example shown in FIG. 7, a blue marker 51 is superimposed on the in-focus probability values ​​21. A red marker 52 is superimposed on the out-of-focus probability values ​​21. Therefore, in the example shown in Fig. 7, in an area where the in-focus probability value 21 and the out-of-focus probability value 21 coexist, a blue and red gradation mark 53 is displayed as if it were superimposed. Note that in the example shown in Fig. 7, as shown in legend 9, the blue and red gradation mark 53 is represented by hatching with medium intervals.

[0057] In the superimposed cell image 50a, which is in focus, there are many areas where blue markers 51, which indicate an in-focus probability value 21, are superimposed. In the superimposed cell image 50c, which is the most out-of-focus, there are many areas where red markers 52, which indicate a high out-of-focus probability value 21, are superimposed. In the superimposed cell image 50b, which is less out-of-focus than the superimposed cell image 50c, there are the most areas where blue markers 51 are superimposed, followed by areas where blue-red gradation markers 53 are superimposed. In the superimposed cell image 50b, there are also areas where red markers 52 are superimposed.

[0058] (Typical values ​​and differences due to differences in shooting conditions) In this embodiment, as shown in FIG. 8 , the control unit 2a is configured to acquire a representative value 20a of the probability values ​​21. Specifically, the control unit 2a is configured to acquire numerical data of the representative value 20a of the probability values ​​21. That is, in this embodiment, the control unit 2a is configured to acquire one representative value 20a from the probability values ​​21 acquired for each pixel of the cell image 10a. Furthermore, in this embodiment, the control unit 2a is configured to acquire a representative value 20a of the probability values ​​21 in a cellular region as the representative value 20a of the probability values ​​21. The cellular region is acquired by the image processing unit 2c. Specifically, the image processing unit 2c adds together probability distribution images 12 of at least two types of label values, and acquires, as the cellular region, a region in the probability distribution image 12 after addition that has a predetermined probability value 21 or more.

[0059] In this embodiment, the control unit 2a acquires the representative value 20a based on the probability value 21 of the label value of one of the classifications of two or more types of imaging conditions. Specifically, the control unit 2a acquires the representative value 20a based on the probability value 21 of the label value that is in focus. In other words, the control unit 2a acquires the representative value 20a based on the probability value 21 of the label value that is suitable for analyzing whether the cell is normal or abnormal.

[0060] Furthermore, in this embodiment, the control unit 2a obtains a graph that collectively displays the numerical data of multiple representative values ​​20a, as shown in graph 40a. In graph 40a, the horizontal axis represents the defocus of each cell image 10, and the vertical axis represents the representative value 20a. That is, the horizontal axis "0" in graph 40a represents the in-focus cell image 10a. The horizontal axis "-1" in graph 40a represents the out-of-focus cell image 10b. The horizontal axis "-2" in graph 40a represents the out-of-focus cell image 10c. As shown in graph 40a, the representative value 20a decreases as the focus increases.

[0061] (Frequency distribution and differences in frequency distribution due to differences in shooting conditions) Next, with reference to FIGS. 9 and 10, the frequency distribution 22 (see FIG. 4) acquired by the control unit 2a and the difference in the frequency distribution 22 (see FIG. 4) due to differences in the shooting conditions will be described.

[0062] The frequency distribution 22a shown in FIG. 9 is a frequency distribution acquired based on the probability values ​​21 of the in-focus cell image 10a. In the frequency distribution 22a, the horizontal axis represents the probability value 21, and the vertical axis represents the frequency. That is, the frequency distribution 22a is a graph of the frequency of the probability values ​​21 for each pixel of the cell image 10a (see FIG. 5). In addition, in the frequency distribution 22a, the probability values ​​21 of a first type of label value among two or more types are hatched. That is, in the frequency distribution 22a, the probability values ​​21 of the in-focus class are hatched. In addition, in the frequency distribution 22a, the probability values ​​21 of a second type of label value different from the first type among the two or more types are not hatched and are shown in white. That is, in the frequency distribution 22a, the probability values ​​21 of the out-of-focus class are shown in white.

[0063] 9, the frequency distribution 22a of the in-focus cell image 10a has a high frequency of pixels with a high probability value 21 of the in-focus class, so many pixels are distributed on the right side of the frequency distribution 22a.Furthermore, the frequency distribution 22a of the in-focus cell image 10a also has a high frequency of pixels with a low probability value 21 of the out-of-focus class, so many pixels are also distributed on the left side of the frequency distribution 22a.

[0064] The frequency distribution 22b shown in Figure 10 is a frequency distribution obtained based on the probability values ​​21 of the out-of-focus cell image 10b. In the frequency distribution 22b, the horizontal axis represents the probability value 21 and the vertical axis represents the frequency. In other words, the frequency distribution 22b is a graph of the frequency of the probability value 21 for each pixel of the cell image 10b (see Figure 5). Also in the frequency distribution 22b, the probability values ​​21 of the in-focus class are hatched, and the probability values ​​21 of the out-of-focus class are not hatched and are shown in white.

[0065] As shown in FIG. 10, the frequency distribution 22b of the out-of-focus cell image 10b is different from the frequency distribution 22a of the in-focus cell image 10a (see FIG. 9) in that the frequency of pixels with a high probability value 21 of the in-focus class is lower and the frequency of pixels with a low probability value 21 of the in-focus class is higher. Therefore, the distribution of pixels with a high probability value 21 of the in-focus class is distributed over almost the entire area, rather than being biased to the right side of the frequency distribution 22a. Furthermore, the frequency distribution 22a of the in-focus cell image 10a is different from the frequency distribution 22a in that the frequency of pixels with a low probability value 21 of the out-of-focus class is lower and the frequency of pixels with a low probability value 21 of the out-of-focus class is higher. Therefore, the distribution of pixels with a high probability value 21 of the in-focus class is distributed over almost the entire area, rather than being biased to the left side of the frequency distribution 22a. In other words, by simply looking at the shape of the frequency distribution 22, it is easy to determine which of two or more types of cells 90 appear in the cell image 10 under the imaging conditions belongs to.

[0066] (Display of superimposed cell images, representative values, and frequency distribution) In this embodiment, the control unit 2a (see FIG. 1) displays, on the display unit 4, numerical data of the representative values ​​20a (see FIG. 4) of the probability values ​​21 (see FIG. 4), and a superimposed cell image 50 in which the distribution of the probability values ​​21 is superimposed on the cell image 10, as shown in FIG. 11. In this embodiment, the control unit 2a displays, on the display unit 4, the frequency distribution 22 of the probability values ​​21 (see FIG. 4), together with the numerical data of the representative values ​​20a of the probability values ​​21 and the superimposed cell image 50. In the example shown in FIG. 11, the control unit 2a displays superimposed cell images 50a to 50c as the superimposed cell image 50. In addition, in the example shown in FIG. 11, the control unit 2a displays a graph 40a as the numerical data of the representative values ​​20a. In addition, in the example shown in FIG. 11, the control unit 2a displays a frequency distribution 22a as the frequency distribution 22.

[0067] (Differences in superimposed cell images due to differences in coating agents) Next, with reference to FIG. 12, differences in the superimposed cell image 50 (see FIG. 1) due to differences in coating agents will be described. The superimposed cell image 50d shown in FIG. 12(A) is an image generated based on the cell image 10 and a probability distribution image 12 generated based on the index value 20 obtained by analyzing the cell image 10 using the second trained model 6b. Specifically, the superimposed cell image 50d is an image generated based on the cell image 10, which is an image of a cell 90 cultured in a culture vessel 80 having a bottom surface 80a of the culture vessel 80 coated with coating agent A. Note that the configuration for generating the superimposed cell image 50d is similar to the configuration for generating the superimposed cell images 50a to 50c using the first trained model 6a, except that the second trained model 6b is used instead of the first trained model 6a, and therefore a detailed description thereof will be omitted.

[0068] 12(B) is an image generated based on the cell image 10 and a probability distribution image 12 generated based on index values ​​20 obtained by analyzing the cell image 10 using the second trained model 6b. Specifically, the superimposed cell image 50e is an image generated based on a cell image 10 obtained by capturing a cell 90 cultured in a culture vessel 80 in which a coating agent B is applied to the bottom surface 80a of the culture vessel 80. In this embodiment, the second trained model 6b is generated by training the model 6b to output the probability that the coating agent applied to the bottom surface 80a of the culture vessel 80 is coating agent A as a probability value 21. Therefore, the superimposed cell image 50d shown in FIG. 12(A) has many regions in which blue markers 51 are superimposed (regions with narrowest hatching). The superimposed cell image 50e shown in FIG. 12(B) has many regions in which red markers 52 are superimposed (regions with widest hatching).

[0069] (Representative values ​​vary depending on the coating material) The control unit 2a acquires a representative value 20a for each cell image 10 based on the probability value 21 output by the second trained model 6b. The control unit 2a also acquires a graph that collectively displays multiple representative values ​​20a, such as graph 40b shown in FIG. 13. Graph 40b is a graph showing differences in representative values ​​20a due to different coating agents. In graph 40b, the horizontal axis represents the type of coating agent, and the vertical axis represents the representative value 20a. Note that the configuration by which the control unit 2a generates graph 40b is similar to the configuration by which the control unit 2a generates graph 40a, except that the probability value 21 output by the second trained model 6b is used instead of the probability value 21 output by the first trained model 6a, and therefore detailed description thereof will be omitted.

[0070] As shown in graph 40b, the representative value 20a of the cell image 10 taken of the cell 90 cultured in the culture vessel 80 coated with coating agent A is larger than the representative value 20a of the cell image 10 taken of the cell 90 cultured in the culture vessel 80 coated with coating agent B.

[0071] (Differences in superimposed cell images due to differences in culture days) Next, with reference to FIG. 14, differences in the superimposed cell image 50 (see FIG. 1) due to differences in the number of days of culture will be described. The superimposed cell image 50f shown in FIG. 14(A) is an image generated based on a cell image 10 obtained by photographing a cell 90 that has been cultured for five days. Specifically, the superimposed cell image 50f is an image generated based on the cell image 10 and a probability distribution image 12 generated based on an index value 20 obtained by analyzing the cell image 10a using the third trained model 6c. Note that the configuration for generating the superimposed cell image 50f is similar to the configuration for generating the superimposed cell images 50a to 50c using the first trained model 6a, except that the third trained model 6c is used instead of the first trained model 6a, and therefore a detailed description thereof will be omitted.

[0072] Moreover, the superimposed cell image 50g shown in FIG. 14(B) is an image generated based on the cell image 10 obtained by photographing the cell 90 after four days of culture. Moreover, the superimposed cell image 50h shown in FIG. 14(C) is an image generated based on the cell image 10 obtained by photographing the cell 90 after six days of culture. Moreover, the superimposed cell image 50i shown in FIG. 14(D) is an image generated based on the cell image 10 obtained by photographing the cell 90 after seven days of culture.

[0073] The third trained model 6c is generated by training the model to output a probability value 21 indicating whether the number of culture days is 5 days or not. Therefore, in the superimposed cell image 50f shown in FIG. 14(A), there are many areas where blue markers 51 are superimposed (areas with narrowest hatching). Furthermore, in the superimposed cell image 50g shown in FIG. 14(B), compared to the superimposed cell image 50g shown in FIG. 14(A), there are more areas where red markers 52 are superimposed (areas with widest hatching). Furthermore, as shown in FIGS. 14(C) and 14(D), as the number of culture days increases, the area where red markers 52 are superimposed increases, and the area where blue-to-red gradation markers 53 are superimposed (areas with medium-spaced hatching) also increases.

[0074] (Representative values ​​vary depending on the number of days cultured) The control unit 2a acquires a representative value 20a for each cell image 10 based on the probability value 21 output by the third trained model 6c. The control unit 2a also acquires a graph that collectively displays multiple representative values ​​20a, such as graph 40c shown in FIG. 15. Graph 40c is a graph that shows differences in representative values ​​20a due to differences in the number of culture days. In graph 40c, the horizontal axis represents the number of culture days, and the vertical axis represents the representative value 20a. Note that the configuration by which the control unit 2a generates graph 40c is similar to the configuration by which the control unit 2a generates graph 40a, except that the probability value 21 output by the third trained model 6c is used instead of the probability value 21 output by the first trained model 6a, and therefore detailed description thereof will be omitted.

[0075] As shown in graph 40c, the representative value 20a of the cell image 10 obtained by photographing the cells 90 that had been in culture for five days is the highest. It can also be seen that the representative value 20a of the cell image 10 obtained by photographing the cells 90 that had been in culture for other than five days is smaller than the representative value 20a of the cell image 10 obtained by photographing the cells 90 that had been in culture for five days. It can also be seen that when the representative value 20a of the cell image 10 obtained by photographing the cells 90 that had been in culture for four days is compared with the representative value 20a of the cell image 10 obtained by photographing the cells 90 that had been in culture for six days, the representative value 20a of the cell image 10 obtained by photographing the cells 90 that had been in culture for six days is higher.

[0076] (Thresholding of representative values) In this embodiment, as shown in FIG. 11 , the display unit 4 displays multiple superimposed cell images 50, numerical data (graph) of the representative values ​​20a, and a frequency distribution 22a. Therefore, based on the multiple superimposed cell images 50, the numerical data (graph) of the representative values ​​20a, and the frequency distribution 22a displayed on the display unit 4, the operator can determine which of two or more types of classifications the cell 90 depicted in the cell image 10 belongs to. That is, the operator can determine the classification of the cell 90 depicted in the cell image 10 that is suitable for analyzing whether the cell 90 depicted in the cell image 10 is a normal cell or an abnormal cell under two or more types of imaging conditions or two or more types of culture conditions. Therefore, the operator can determine whether the cell image 10 is suitable for analyzing whether the cell 90 depicted in the cell image 10 is a normal cell or an abnormal cell. In this case, based on the numerical data of the representative values ​​20a, it is possible to set a threshold for determining whether the cell image 10 is suitable for analyzing whether the cell 90 depicted in the cell image 10 is a normal cell or an abnormal cell. Therefore, in this embodiment, the control unit 2a is configured to determine whether the index value 20 is greater than a threshold value. Specifically, the control unit 2a determines whether the cell image 10 is suitable for analyzing whether the cell 90 depicted in the cell image 10 is a normal cell or an abnormal cell by determining whether the representative value 20a of the probability values ​​21 is greater than a threshold value set by the operator. For example, if the representative value 20a is greater than 50%, the control unit 2a determines that the cell image 10 is suitable for analyzing whether the cell 90 depicted in the cell image 10 is a normal cell or an abnormal cell. Furthermore, for example, if the representative value 20a is less than 50%, the control unit 2a determines that the cell image 10 is not suitable for analyzing whether the cell 90 depicted in the cell image 10 is a normal cell or an abnormal cell.

[0077] (Display processing of superimposed cell images, representative values, and frequency distribution) Next, with reference to FIG. 16, a process in which the cell-image analyzing device 100 displays the superimposed cell image 50, the representative value 20a, and the frequency distribution 22 will be described.

[0078] In step 200, the image acquisition unit 1 acquires a cell image 10 in which a cell 90 is captured.

[0079] In step 201, the image analysis unit 2b inputs the cell image 10 to the trained model 6 that has been trained to classify cells 90 into two or more types.

[0080] In step 202, the image processing unit 2c acquires a cell region, which is the region of the cell 90 depicted in the cell image 10. In this embodiment, the image processing unit 2c acquires the cell region based on the probability distribution image 12 (see FIG. 6).

[0081] In step 203, the control unit 2a acquires an index value 20 representing the degree of certainty that the cell 90 depicted in the cell image 10 is classified into one of two or more types, based on the analysis results of each pixel of the cell image 10 output by the trained model 6. In the present embodiment, in the processing of step 203, the control unit 2a acquires, as the index value 20, a representative value 20a of the probability values ​​21 acquired based on the probability values ​​21 output by the trained model 6. Specifically, the control unit 2a acquires at least one index value 20 representing whether the cell image 10 was in focus when captured, whether the coating agent on the culture vessel 80 is appropriate, and whether the number of days of culture is appropriate. Furthermore, in the processing of step 203, the control unit 2a acquires, as the index value 20, a value representing the degree of certainty that the cell image 10 is suitable for analyzing whether the cell 90 depicted in the cell image 10 is a normal cell or an abnormal cell, based on the probability value 21.

[0082] In this embodiment, in the process of step 203, the control unit 2a acquires a representative value 20a of the probability values ​​21 in the cellular region as the representative value 20a of the probability values ​​21. Specifically, the control unit 2a acquires the average value of the probability values ​​21 as the representative value 20a.

[0083] In step 204, the control unit 2a acquires the frequency distribution 22. Specifically, the control unit 2a acquires the frequency distribution 22 based on the probability value 21 output by the trained model 6.

[0084] In step 205, the superimposed cell image generating unit 2d generates the superimposed cell image 50. Specifically, the superimposed cell image generating unit 2d generates the superimposed cell image 50 based on the cell image 10 and the probability distribution image 12 (see FIG. 6) acquired based on the probability value 21.

[0085] In step 206, the control unit 2a displays the acquired index value 20. In this embodiment, in the processing of step 206, the control unit 2a displays the numerical data of the representative value 20a of the probability values ​​21 and a superimposed cell image 50 in which the distribution of the probability values ​​21 is superimposed on the cell image 10. In this embodiment, the control unit 2a displays the frequency distribution 22 of the probability values ​​21 together with the numerical data of the representative value 20a of the probability values ​​21 and the superimposed cell image 50. Thereafter, the processing ends.

[0086] It should be noted that either the process in step 204 or the process in step 205 may be performed first.

[0087] (Generation process of trained model) Next, the generation process of the trained model 6 will be described with reference to FIG.

[0088] In step 300, the image acquisition unit 1 acquires a teacher cell image 30. The teacher cell image 30 is a cell image 10.

[0089] In step 301, the image acquisition unit 1 acquires a teacher's correct answer image 31. The teacher's correct answer image 31 is a labeled image to which the cell image 10 is assigned label values ​​relating to at least two types of photographing conditions corresponding to the classification, or to which the cell image 10 is assigned label values ​​relating to at least two types of culture conditions corresponding to the classification.

[0090] In this embodiment, when generating the first trained model 6a, in the processing of step 301, a cell image 10 to which label values ​​related to at least two types of imaging conditions corresponding to the classification are assigned is acquired as the teacher supervised image 31. Specifically, a cell image 10 to which, as label values ​​related to the imaging conditions, a focused label value and an out-of-focus label value when the cell image 10 is captured are assigned for each pixel is acquired as the teacher supervised image 31. Note that the out-of-focus label value includes multiple label values ​​depending on the degree of out-of-focus. In this embodiment, the out-of-focus label value includes two label values.

[0091] Furthermore, when generating the second trained model 6b, in the process of step 301, the image acquisition unit 1 acquires cell images 10 to which label values ​​related to at least two types of culture conditions corresponding to the classification are assigned as the teacher supervised image 31. Specifically, the image acquisition unit 1 acquires cell images 10 to which at least two types of label values ​​related to the coating agent of the culture vessel 80 in which the cells 90 are cultured are assigned as the teacher supervised image 31. In this embodiment, the image acquisition unit 1 acquires the teacher supervised image 31 to which two label values ​​related to the coating agent, a label value of coating agent A and a label value of coating agent B, are assigned to each pixel.

[0092] Furthermore, when generating the third trained model 6c, in the process of step 301, the image acquisition unit 1 acquires cell images 10 to which label values ​​related to at least two types of culture conditions corresponding to the classification are assigned as the teacher supervised image 31. Specifically, the image acquisition unit 1 acquires cell images 10 to which at least two types of label values ​​related to the number of days of culture of the cells 90 are assigned as the teacher supervised image 31. In this embodiment, the image acquisition unit 1 acquires the teacher supervised image 31 in which, as label values ​​related to the number of days of culture, a label value indicating that the number of days of culture is 5 days and a label value indicating that the number of days of culture is other than 5 days are assigned to each pixel.

[0093] In step 302, the image processing unit 2c creates a trained model 6 using a teacher cell image 30, which is a cell image 10, and a teacher correct answer image 31 to which the cell image 10 is assigned label values ​​for at least two types of shooting conditions corresponding to the classification, or to which label values ​​for at least two types of culture conditions corresponding to the classification have been assigned.

[0094] In this embodiment, in the processing of step 302, the image processing unit 2c creates a trained model 6 using a training answer image 31 to which at least two types of label values ​​have been assigned, as label values ​​related to the imaging conditions, either related to whether the cell image 10 was in focus when it was captured or related to the coating agent of the culture vessel 80 in which the cells 90 are cultured and the number of days of culture, as label values ​​related to the culture conditions. Then, the processing ends.

[0095] (Classification of cell images) Next, the process of classifying the cell image 10 by the cell-image analyzing device 100 will be described with reference to FIG.

[0096] In step 400, the control unit 2a acquires the index value 20. In this embodiment, the control unit 2a acquires the index value 20 that the image analysis unit 2b acquires using the cell image 10 and the trained model 6.

[0097] In step 401, the control unit 2a acquires a threshold value. Specifically, the control unit 2a acquires a threshold value that has been set in advance by an operator and stored in the storage unit 3.

[0098] In step 402, the control unit 2a determines whether the index value 20 is greater than the threshold value. That is, the control unit 2a determines whether the index value 20 is greater than the threshold value for each cell image 10. If the index value 20 is greater than the threshold value, the process proceeds to step 403. If the index value 20 is less than the threshold value, the process proceeds to step 404.

[0099] In step 403, the control unit 2a classifies the cell image 10 into an image suitable for analyzing whether the cell 90 is a normal cell or an abnormal cell. The control unit 2a also stores the cell image 10 classified into an image suitable for analyzing whether the cell 90 is a normal cell or an abnormal cell in the storage unit 3. Thereafter, the processing ends.

[0100] Furthermore, if the process proceeds from step 402 to step 404, in step 404, the control unit 2a classifies the cell image 10 as an image that is not suitable for analyzing whether the cell 90 is a normal cell or an abnormal cell. In this case, the control unit 2a does not store the cell image 10 in the storage unit 3. Thereafter, the process ends.

[0101] (Effects of this embodiment) In this embodiment, the following effects can be obtained.

[0102] In this embodiment, as described above, the cell image analysis method includes the steps of acquiring a cell image 10 containing a cell 90, inputting the cell image 10 into a trained model 6 that has been trained to classify the cell 90 into two or more types, acquiring an index value 20 that indicates the likelihood that the cell 90 shown in the cell image 10 is one of the two or more types based on the analysis results of each pixel of the cell image 10 output by the trained model 6, and displaying the acquired index value 20.

[0103] This displays an index value 20 that indicates the accuracy of which of two or more classifications the cell 90 depicted in the cell image 10 is classified into, and the operator can easily grasp the accuracy of the classification of the cell 90 depicted in the cell image 10 by checking the index value 20. As a result, it is possible to provide a cell image analysis method that makes it possible to easily grasp the accuracy of the classification of the cell 90 depicted in the cell image 10.

[0104] Furthermore, in the above embodiment, the following additional effects can be obtained by configuring as follows.

[0105] That is, in this embodiment, as described above, the trained model 6 is trained to output probability values ​​21, which are estimated values ​​of classification, as analysis results, and in the step of acquiring index values ​​20, a representative value 20a of the probability values ​​21 acquired based on the probability values ​​21 output by the trained model 6 is acquired as the index value 20. This makes it possible to easily grasp the accuracy of classification of the cell 90 depicted in the cell image 10 for each cell image 10 by the representative value 20a of the probability values ​​21, unlike a configuration in which the probability value 21 is displayed for each pixel of the cell image 10.

[0106] Furthermore, in this embodiment, as described above, the cell image 10 is an image containing cultured cells 90 cultured in a culture vessel 80, and the trained model 6 is created by learning to classify at least one of whether the cell image 10 was in focus when captured, whether the coating agent on the culture vessel 80 is appropriate, and whether the number of days of culture is appropriate. In the step of acquiring the index value 20, at least one of the index values ​​20 is acquired, whether the cell image 10 was in focus when captured, whether the coating agent on the culture vessel 80 is appropriate, and whether the number of days of culture is appropriate. This allows the operator to easily grasp at least one of whether the cell image 10 was in focus when captured, whether the coating agent on the culture vessel 80 is appropriate, and whether the number of days of culture is appropriate, by checking the index value 20.

[0107] Furthermore, in this embodiment, as described above, the trained model 6 is created by learning to classify whether a cell 90 is suitable for analyzing whether it is a normal cell or an abnormal cell. In the step of acquiring the index value 20, a value representing the likelihood that the cell image 10 is suitable for analyzing whether the cell 90 depicted in the cell image 10 is a normal cell or an abnormal cell is acquired as the index value 20 based on the probability value 21. This allows the operator to easily determine whether the cell image 10 is suitable for analyzing whether the cell 90 depicted in the cell image 10 is a normal cell or an abnormal cell by checking the index value 20. As a result, a cell image analysis method can be provided that allows the operator to easily determine whether the cell image 10 is suitable for analyzing whether the cell 90 depicted in the cell image 10 is a normal cell or an abnormal cell.

[0108] Furthermore, in this embodiment, as described above, the trained model 6 is created by learning to classify cells 90 of the same type as to whether they are suitable for analysis as normal cells or abnormal cells. As a result, by analyzing cell images 10 using the trained model 6, it is possible to classify images of cells 90 of the same type as to whether they are suitable for analysis as normal cells or abnormal cells.

[0109] Furthermore, as described above, this embodiment further includes a step of acquiring a cell region, which is the region of the cell 90 depicted in the cell image 10, and in the step of acquiring the representative value 20a of the probability values ​​21, the representative value 20a of the probability values ​​21 within the cell region is acquired as the representative value 20a of the probability values ​​21. This makes it possible to suppress an increase in processing load compared to a configuration in which the representative value 20a is acquired based on the probability values ​​21 of all pixels in the cell image 10.

[0110] Furthermore, in this embodiment, as described above, in the step of displaying the representative values ​​20a of the probability values ​​21, the numerical data of the representative values ​​20a of the probability values ​​21 and the superimposed cell image 50 in which the distribution of the probability values ​​21 is superimposed on the cell image 10 are displayed. As a result, the representative values ​​20a of the probability values ​​21 are displayed, and therefore the numerical data of the representative values ​​20a of the probability values ​​21 makes it possible to easily grasp the accuracy of classification of the cells 90 depicted in the cell image 10 for each cell image 10. Furthermore, since the superimposed cell image 50 is displayed, the accuracy of classification of each of the cells 90 depicted in the cell image 10 can be grasped from the superimposed cell image 50.

[0111] Furthermore, in this embodiment, as described above, in the step of displaying the representative value 20a of the probability value 21, the frequency distribution 22 of the probability value 21 is displayed together with the numerical data of the representative value 20a of the probability value 21 and the superimposed cell image 50. As a result, by checking the frequency distribution 22 together with the numerical data of the representative value 20a of the probability value 21 and the superimposed cell image 50, the accuracy of classification of the cell 90 depicted in the cell image 10 can be grasped from multiple perspectives for each cell image 10.

[0112] Furthermore, in this embodiment, as described above, in the step of acquiring the representative value 20a of the probability values ​​21, the average value of the probability values ​​21 is acquired as the representative value 20a. Here, compared to a configuration in which the median of the probability values ​​21 is acquired as the representative value 20a, for example, when the cell image 10 includes an area that is small in area but has a very high probability (probability value 21) of being classified as a first type out of two or more types, the representative value 20a becomes the value of the first type classification. In this case, even if the cells 90 depicted in the cell image 10 are classified as a second type that is different from the first type as a whole, the cells 90 depicted in the cell image 10 are classified as a first type out of two or more types due to the probability values ​​21 of some of the cell images 10. Therefore, by obtaining the average value of the probability values ​​21 as the representative value 20a as described above, when classifying the cell images 10 by classifying the cells 90 depicted in the cell images 10, it is possible to prevent the cell images 10 from being classified as one of two or more first types due to the probability values ​​21 of a portion of the cell images 10. As a result, by classifying the cells 90 depicted in the cell images 10, it is possible to prevent a decrease in classification accuracy when classifying the cell images 10.

[0113] Furthermore, as described above, this embodiment further includes a step of creating a trained model 6 using a teacher cell image 30, which is the cell image 10, and a teacher correct answer image 31 to which the cell image 10 is assigned label values ​​for at least two types of imaging conditions corresponding to the classification, or to which the cell image 10 is assigned label values ​​for at least two types of culture conditions corresponding to the classification. By using the teacher correct answer image 31 to which the cell image 10 is assigned label values ​​for at least two types of imaging conditions corresponding to the classification, a trained model 6 can be generated that can be used to classify the cell image 10 as to under which of two or more imaging conditions a cell 90 depicted in the cell image 10 was photographed. Furthermore, by using the teacher correct answer image 31 to which the label values ​​for at least two types of culture conditions corresponding to the classification, a trained model 6 can be generated that can be used to classify the cell image 10 as to under which of two or more culture conditions a cell 90 depicted in the cell image 10 was cultured.

[0114] Furthermore, in this embodiment, as described above, in the step of creating the trained model 6, the trained model 6 is created using a training correct answer image 31 to which, as label values ​​related to the imaging conditions, two types of label values ​​are assigned, namely, whether the cell image 10 was in focus when it was captured, or at least two types of label values ​​related to the coating agent of the culture vessel 80 in which the cells 90 are cultured and the number of culture days are assigned, as label values ​​related to the culture conditions. This makes it possible to generate a trained model 6 that can be used to classify images into two or more types of classifications based on the conditions of whether the cell image 10 is in focus, whether the coating agent of the culture vessel 80 is appropriate, and whether the number of culture days is appropriate.

[0115] Furthermore, as described above, this embodiment further includes a step of determining whether the index value 20 is greater than a threshold value. This allows a cell image 10 whose index value 20 is greater than the threshold value to be classified as an image suitable for analyzing whether a cell 90 depicted in the cell image 10 is a normal cell or an abnormal cell. Furthermore, a cell image 10 whose index value 20 is equal to or less than the threshold value to be classified as an image unsuitable for analyzing whether a cell 90 depicted in the cell image 10 is a normal cell or an abnormal cell. Therefore, when analyzing whether a cell 90 depicted in the cell image 10 is a normal cell or an abnormal cell using multiple cell images 10, the analysis can be performed using only the cell images 10 suitable for analyzing whether the cell 90 depicted in the cell image 10 is a normal cell or an abnormal cell. As a result, a decrease in the accuracy of the analysis of whether a cell 90 depicted in the cell image 10 is a normal cell or an abnormal cell can be suppressed.

[0116] [Variations] The embodiments disclosed herein should be considered to be illustrative and not restrictive in all respects. The scope of the present invention is defined by the claims, not by the description of the above embodiments, and further includes all modifications (variations) within the meaning and scope of the claims.

[0117] For example, in the above embodiment, the control unit 2a acquires the average value of the probability values ​​21 as the representative value 20a, but the present invention is not limited to this. For example, the control unit 2a may be configured to acquire any one of the median, maximum, minimum, and mode of the probability values ​​21 as the representative value 20a.

[0118] Furthermore, in the above embodiment, an example of a configuration was described in which the trained model 6 was created by learning to classify, as an imaging condition, whether or not the image is in focus, or whether or not the coating agent of the culture vessel 80 is appropriate, and / or whether or not the number of days the cells 90 have been cultured is appropriate. However, the present invention is not limited to this. For example, the trained model 6 may be created by learning to classify, as an imaging condition, conditions other than whether or not the image is in focus. For example, the trained model 6 may be created by learning to classify, as an imaging condition, whether or not the type of imaging device is appropriate. Furthermore, the trained model 6 may be created by learning to classify based on culture conditions other than whether or not the coating agent of the culture vessel 80 is appropriate or whether or not the number of days the cells 90 have been cultured is appropriate. For example ... whether or not the type of culture device is appropriate. The imaging conditions and culture conditions classified by the trained model 6 may be any.

[0119] Furthermore, in the above embodiment, an example of a configuration in which the control unit 2a acquires a representative value 20a of the probability values ​​21 in a cellular region has been described, but the present invention is not limited to this. For example, the control unit 2a may be configured to acquire the representative value 20a based on the probability values ​​21 of all pixels included in the cell image 10. However, if the control unit 2a is configured to acquire a representative value 20a of the probability values ​​21 of all pixels included in the cell image 10, the processing load for acquiring the representative value 20a increases. Therefore, it is preferable that the control unit 2a be configured to acquire a representative value 20a of the probability values ​​21 in a cellular region.

[0120] In the above embodiment, the control unit 2a displays the superimposed cell image 50, the numerical data of the representative values ​​20a, and the frequency distribution 22 on the display unit 4, but the present invention is not limited to this. For example, the control unit 2a may be configured to display only the numerical data of the representative values ​​20a on the display unit 4. Furthermore, the control unit 2a may be configured to display the numerical data of the representative values ​​20a and the superimposed cell image 50 on the display unit 4.

[0121] In the above embodiment, the control unit 2a displays any one of graphs 40a, 40b, and 40c, which collectively display the numerical data of the representative values ​​20a, when displaying the numerical data of the representative values ​​20a on the display unit 4. However, the present invention is not limited to this. For example, when displaying the numerical data of the representative values ​​20a, the control unit 2a may be configured to display the numerical values ​​of the representative values ​​20a themselves, rather than displaying them as a graph.

[0122] Furthermore, in the above embodiment, an example of a configuration in which the cell-image analyzing device 100 generates a trained model 6 has been described, but the present invention is not limited to this. For example, the cell-image analyzing device 100 may be configured to use a trained model 6 generated by an image analyzing device other than the cell-image analyzing device 100.

[0123] Furthermore, in the above embodiment, an example of a configuration was shown in which the superimposed cell image generating unit 2d generates a superimposed cell image 50 in which a blue mark 51 is superimposed on the probability value 21 of the label value of a first type of classification among two or more types, and a red mark 52 is superimposed on the probability value 21 of the label value of a second type of classification different from the first type, but the present invention is not limited to this. As long as the difference in the probability values ​​21 can be distinguished, the superimposed cell image generating unit 2d may superimpose marks of any color on the probability value 21 of each label value of two or more types of classification.

[0124] In the above embodiment, the image processing unit 2c generates the first trained model 6a by using the supervised training image 31 to which two types of label values, that is, whether or not the image is in focus, are assigned. However, the present invention is not limited to this. For example, the image processing unit 2c may be configured to train the learning model 7 using the supervised training image to which three or more types of label values ​​are assigned depending on the degree of focus.

[0125] In the above embodiment, the image processing unit 2c generates the second trained model 6b by using the supervised training image 31 to which two label values, indicating whether the type of coating agent is coating agent A or not, are assigned. However, the present invention is not limited to this. For example, the image processing unit 2c may be configured to train the learning model 7 using supervised training images to which three or more label values ​​are assigned depending on the type of coating agent.

[0126] In the above embodiment, the image processing unit 2c generates the third trained model 6c by using the supervised training images 31 to which two label values, i.e., whether the number of culture days is 5 days or not, are assigned. However, the present invention is not limited to this. For example, the image processing unit 2c may be configured to train the learning model 7 using the supervised training images to which three or more label values ​​are assigned depending on the number of culture days.

[0127] In the above embodiment, the image acquisition unit 1 acquires the cell image 10 as the process of step 201, but the present invention is not limited to this. For example, the image processing unit 2c may be configured to acquire the cell image 10 that has been acquired in advance by the image acquisition unit 1 and stored in the storage unit 3.

[0128] In the above embodiment, the control unit 2a performs a process of determining whether the index value 20 is greater than a threshold value, but the present invention is not limited to this. For example, the control unit 2a does not need to perform a process of determining whether the index value 20 is greater than a threshold value.

[0129] [Aspect] It will be appreciated by those skilled in the art that the exemplary embodiments described above are examples of the following aspects.

[0130] (Item 1) acquiring a cell image showing the cell; inputting the cell image into a trained model that has been trained to classify the cells into two or more types; acquiring an index value representing the likelihood that the cell depicted in the cell image is classified into one of two or more types based on the analysis results of each pixel of the cell image output by the trained model; and displaying the acquired index value.

[0131] (Item 2) the trained model is trained to output, as the analysis result, a probability value that is an estimate of the classification; 2. The cell image analysis method according to item 1, wherein in the step of acquiring the index value, a representative value of the probability values ​​acquired based on the probability values ​​output by the trained model is acquired as the index value.

[0132] (Item 3) the cell image is an image including cultured cells cultured in a culture vessel, The trained model is created by learning to classify at least one of whether the cell image is in focus when captured, whether the coating agent of the culture vessel is appropriate, and whether the number of days of culture is appropriate; 3. The cell image analysis method according to item 2, wherein in the step of acquiring the index value, at least one of the index values ​​acquired is whether the cell image was in focus when captured, whether the coating agent on the culture vessel is appropriate, and whether the number of days of culture is appropriate.

[0133] (Item 4) the trained model is created by learning to classify whether the cell is suitable for analysis as a normal cell or an abnormal cell; 4. The image analysis method according to item 2 or 3, wherein in the step of acquiring the index value, a value representing the degree of certainty that the cell image is suitable for analyzing whether the cell depicted in the cell image is the normal cell or the abnormal cell is acquired as the index value based on the probability value.

[0134] (Item 5) 5. The cell image analysis method according to Item 4, wherein the trained model is created by training the model to classify cells of the same type as to whether they are suitable for analysis as normal cells or abnormal cells.

[0135] (Item 6) further comprising a step of acquiring a cell area that is an area of ​​the cell that appears in the cell image, 6. The cell image analysis method according to any one of items 2 to 5, wherein in the step of acquiring the representative value of the probability values, the representative value of the probability values ​​within the cell region is acquired as the representative value of the probability values.

[0136] (Item 7) 7. The cell image analysis method according to any one of items 2 to 6, wherein in the step of displaying the representative value of the probability values, numerical data of the representative value of the probability values ​​and a superimposed cell image in which the distribution of the probability values ​​is superimposed on the cell image are displayed.

[0137] (Item 8) 8. The cell image analysis method according to item 7, wherein in the step of displaying the representative value of the probability values, a frequency distribution of the probability values ​​is displayed together with the numerical data of the representative value of the probability values ​​and the superimposed cell image.

[0138] (Item 9) 9. The cell image analyzing method according to any one of items 2 to 8, wherein in the step of acquiring the representative value of the probability values, an average value of the probability values ​​is acquired as the representative value.

[0139] (Item 10) 10. The cell image analysis method according to any one of items 1 to 9, further comprising a step of creating the trained model using a teacher cell image, which is the cell image, and a teacher correct answer image to which label values ​​for at least two types of imaging conditions corresponding to the classification are assigned, or to which label values ​​for at least two types of culture conditions corresponding to the classification are assigned.

[0140] (Item 11) Item 11. The cell image analysis method according to Item 10, wherein in the step of creating the trained model, two types of label values, whether or not the cell image was in focus when captured, are assigned as the label values ​​related to the imaging conditions, or the trained model is created using the correct training image to which at least two types of label values, either related to a coating agent for a culture vessel in which the cells are cultured or related to the number of days of culture, are assigned as the label values ​​related to the culture conditions.

[0141] (Item 12) 12. The cell image analyzing method according to any one of items 1 to 11, further comprising a step of determining whether the index value is greater than a threshold value. [Explanation of symbols]

[0142] 6 Pre-trained models 10 Cell images 20 Index Values 20a Representative value (average value) 21 Probability Values 22, 22a, 22b Frequency distribution 80 Culture vessels 81 Culture solution 90 cells (cultured cells)

Claims

1. acquiring a cell image showing the cell; inputting the cell image into a trained model that has been trained to classify the cells into two or more types; acquiring an index value representing the likelihood that the cell depicted in the cell image belongs to one of two or more types of classifications based on the analysis results of each pixel of the cell image output by the trained model; and displaying the acquired index value; the trained model is trained to output, as the analysis result, a probability value that is an estimate of the classification of each pixel of the cell image; A cell image analysis method, in which, in the step of acquiring the index value, one representative value, which is any of the mean, median, maximum, minimum, and mode of the probability values ​​for each pixel of the cell image output by the trained model, is acquired as one index value representing the accuracy of the classification for one cell image.

2. the cell image is an image including cultured cells cultured in a culture vessel, The trained model is created by learning to classify at least one of whether the cell image is in focus when captured, whether the coating agent of the culture vessel is appropriate, and whether the number of days of culture is appropriate; 2. The cell image analysis method according to claim 1, wherein in the step of acquiring the index value, at least one of the index values ​​acquired is whether the cell image was in focus when captured, whether the coating agent on the culture vessel is appropriate, and whether the number of days of culture is appropriate.

3. the trained model is created by learning to classify whether the cell is suitable for analysis as an undifferentiated cell or an undifferentiated deviation cell; The cell image analysis method according to claim 1, wherein in the step of acquiring the index value, a value representing the degree to which the cell image is suitable for analyzing whether the cell depicted in the cell image is the undifferentiated cell or the undifferentiated deviation cell is acquired as the index value based on the probability value.

4. The cell image analysis method of claim 3, wherein the trained model is created by training the model to classify cells of the same type as to whether they are suitable for analysis as undifferentiated cells or undifferentiated deviation cells.

5. further comprising a step of acquiring a cell area that is an area of ​​the cell that appears in the cell image, The cell image analysis method according to claim 1 , wherein in the step of acquiring the representative value of the probability values, the representative value of the probability values ​​within the cell region is acquired as the representative value of the probability values.

6. The cell image analysis method of claim 1, wherein in the step of displaying the representative value of the probability values, the numerical data of the representative value of the probability values ​​and a superimposed cell image in which the distribution of the probability values ​​is superimposed on the cell image are displayed.

7. The cell image analysis method according to claim 6, wherein in the step of displaying the representative value of the probability values, a frequency distribution of the probability values ​​is displayed together with the numerical data of the representative value of the probability values ​​and the superimposed cell image.

8. The cell image analysis method according to claim 1 , wherein in the step of acquiring the representative value of the probability values, an average value of the probability values ​​is acquired as the representative value.

9. The cell image analysis method of claim 1, further comprising a step of creating the trained model using a teacher cell image that is the cell image and a teacher correct answer image to which label values ​​for at least two types of shooting conditions corresponding to the classification are assigned, or to which label values ​​for at least two types of culture conditions corresponding to the classification are assigned.

10. 10. The cell image analysis method according to claim 9, wherein in the step of creating the trained model, two types of label values, one relating to whether the cell image was in focus when it was captured or the other relating to the culture conditions, are assigned as the label values ​​related to the imaging conditions, and the trained model is created using the correct training image to which at least two types of label values, one relating to a coating agent for a culture vessel in which the cells are cultured and the number of days of culture, are assigned as the label values ​​related to the culture conditions.

11. The cell image analyzing method according to claim 1 , further comprising the step of determining whether the index value is greater than a threshold value.

Images