Cell Image Analysis Method

Abstract

To provide a cell image analysis method capable of suppressing analysis of a cell image in a state in which, a cell image which is not suitable for analysis is included, for suppressing deterioration of analysis accuracy of the cell image.SOLUTION: A cell image analysis method comprises: a step for acquiring a cell image 10; a step for determining whether, the cell image 10 includes at least one of, blur of the cell 90, reflected glare of a foreign matter 80, reflected glare of stray light, image quality deterioration of the cell image 10 caused by a medium, a region 83 in which a pixel value is saturated, and a cell 90a in which a boundary 84 is not clear; a step for, when the cell image 10 does not include, the blur of the cell 90, the reflected glare of the foreign matter 80, the reflected glare of the stray light, the image quality deterioration of the cell image 10 caused by the medium, the region 83 in which the pixel value is saturated, and the cell 90a in which the boundary 84 is not clear, classifying the cell image 10 into a proper cell image 11, and when the cell image includes at least one of them, classifying the cell image 10 to an improper cell image 12.SELECTED DRAWING: Figure 5

Description

Technical Field

[0001] This invention relates to a method for analyzing cell images.

Background Art

[0002] Conventionally, a method for analyzing cell images has been known (see, for example, Patent Document 1).

[0003] Patent Document 1 discloses a method for analyzing cell images by analyzing an image of cells captured by an imaging device. Specifically, Patent Document 1 discloses obtaining a first cell image and a second cell image, and obtaining a correlation value or difference in feature amounts of the cells in the first cell image and the second cell image. The configuration disclosed in Patent Document 1 analyzes the cell image based on the temporal change in the correlation value or difference in the feature amounts of the cells in the first cell image and the second cell image.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] Here, although not disclosed in Patent Document 1, depending on the imaging conditions when capturing a cell image, for example, a cell image that is not suitable for analysis may be obtained, such as when the cells appear blurred. Also, when analyzing cell images, multiple cell images may be obtained, such as by photographing multiple locations inside the container in which the cells are cultured, and the multiple obtained cell images may be analyzed together by batch processing. In this way, when obtaining multiple cell images, there is a disadvantage that it is difficult for the user to notice even if an image that is not suitable for analysis is included. In this case, since the cell image analysis is performed with a cell image that is not suitable for analysis included, there is a problem that the accuracy of the analysis result decreases.

[0006] The present invention has been made to solve the above-described problems, and one object of the present invention is to suppress a decrease in the analysis accuracy of cell images by suppressing the analysis of cell images in a state including cell images unsuitable for analysis. It is to provide a cell image analysis method capable of

Means for Solving the Problems

[0007] To achieve the above object, a cell image analysis method according to one aspect of the present invention includes a step of acquiring a cell image, and the cell image includes blurring of cells shown in the cell image, reflection of foreign matter in the cell image, reflection of stray light in the cell image, a step of determining whether or not the cell image includes at least one of a decrease in the image quality of the cell image due to the medium when culturing the cells, a region where the pixel value in the cell image is saturated, and a cell in which the boundary between the cell region and the region other than the cell is not clear, and when the cell image does not include blurring of cells, reflection of foreign matter, reflection of stray light, a decrease in the image quality of the cell image due to the medium, a region where the pixel value is saturated, and a cell in which the boundary is not clear, classifying the cell image as an appropriate cell image, and when the cell image includes at least one of blurring of cells, reflection of foreign matter, reflection of stray light, a decrease in the image quality of the cell image due to the medium, a region where the pixel value is saturated, and a cell in which the boundary is not clear, classifying the cell image as an inappropriate cell image generating an inappropriate cell image by performing image processing that degrades the image quality of an appropriate cell image; creating a first learned model that is learned to determine whether a cell image is an appropriate cell image or an inappropriate cell image by learning the appropriate cell image and the inappropriate cell image generated from the appropriate cell image; and comprising In the determining step, using the first learned model, determine whether the cell image is an appropriate cell image or an inappropriate cell image. 。

Effects of the Invention

[0008] In the cell image analysis method in the above-described one aspect, as described above, it is determined whether the cell image includes at least one of the blurring of the cells shown in the cell image, the reflection of foreign matter in the cell image, the reflection of stray light in the cell image, the degradation of the image quality of the cell image due to the culture medium when culturing the cells, the region where the pixel values in the cell image are saturated, and the cells where the boundary between the cell region and the region outside the cell is not clear; and when the cell image does not include the blurring of the cells, the reflection of foreign matter, the reflection of stray light, the degradation of the image quality of the cell image due to the culture medium, the region where the pixel values are saturated, and the cells where the boundary is not clear, the cell image is classified as an appropriate cell image, and when it includes at least one of the blurring of the cells, the reflection of foreign matter, the reflection of stray light, the degradation of the image quality of the cell image due to the culture medium, the region where the pixel values are saturated, and the cells where the boundary is not clear, the cell image is classified as an inappropriate cell image. generating an inappropriate cell image by performing image processing that degrades the image quality of an appropriate cell image; creating a first learned model that is learned to determine whether a cell image is an appropriate cell image or an inappropriate cell image by learning the appropriate cell image and the inappropriate cell image generated from the appropriate cell image; comprising In the determining step, using the first learned model, determine whether the cell image is an appropriate cell image or an inappropriate cell image. . As a result, only appropriate cell images that do not include the blurring of the cells, the reflection of foreign matter, the reflection of stray light, the degradation of the image quality of the cell image due to the culture medium, the region where the pixel values are saturated, and the cells where the boundary is not clear can be used for analysis. Consequently, it is possible to suppress the occurrence of analysis using cell images that include inappropriate cell images, and thus it is possible to suppress a decrease in the accuracy of cell image analysis.

Brief Description of the Drawings

[0009]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Figure 13

Figure 14

Embodiments for Carrying Out the Invention

[0010] Hereinafter, embodiments embodying the present invention will be described with reference to the drawings.

[0011] With reference to FIG. 1, the configuration of a cell image analysis apparatus 100 according to an embodiment will be described.

[0012] (Configuration of the Cell Image Analysis Device) As shown in FIG. 1, the cell image analysis device 100 includes an image acquisition unit 1, a processor 2, a storage unit 3, a display unit 4, and an input reception unit 5.

[0013] The image acquisition unit 1 is configured to acquire a cell image 10. The cell image 10 is an image of cultured cells 90 (see FIG. 2(A)) cultured in a container. In this embodiment, the image acquisition unit 1 is configured to acquire the cell image 10 from a device that captures the cell image 10, such as a microscope 20 to which an imaging device is attached. The image acquisition unit 1 includes, for example, an input / output interface.

[0014] The processor 2 is configured to analyze the acquired cell image 10. The processor 2 includes a CPU (Central Processing Unit), a ROM (Read Only Memory), a RAM (Random Access Memory), a GPU (Graphics Processing Unit), or an FPGA (Field-Programmable Gate Array) configured for image processing. Further, the processor 2 composed of a CPU etc. as hardware functions as a control unit 2a, an image determination unit 2b, an image analysis unit 2c, a learned model generation unit 2d, and an inappropriate cell image generation unit 2e as functional blocks of software (program). The control unit 2a, the image determination unit 2b, the image analysis unit 2c, the learned model generation unit 2d, and the inappropriate cell image generation unit 2e may be individually configured by hardware by providing dedicated processors (processing circuits).

[0015] The control unit 2a is configured to control the cell image analysis device 100. Further, the control unit 2a is configured to perform control to display the analysis result 50 on the display unit 4.

[0016] The image determination unit 2b is configured to determine whether the image quality of the cell image 10 meets the determination criteria. In the present embodiment, the image determination unit 2b is configured to determine whether the image quality of the cell image 10 meets the determination criteria by using a determination by the first learned model 6 and a determination by image processing (rule-based processing) in combination. The determination criteria are for at least either the determination of whether the image quality of the cell image 10 is suitable for the analysis of the cell image 10 when classifying the cell image 10 for analyzing the cell image 10, and the determination of whether the image quality of the cell image 10 is suitable for the learning of the analysis for the second learning model 7a (see FIG. 9) when classifying the cell image 10 used for the learning of the second learning model 7a for analyzing the cell image 10. Specifically, the determination criteria include the presence or absence of blur of the cell 90 (see FIG. 2(A)) in the cell image 10, the presence or absence of reflection of the foreign matter 80 (see FIG. 2(C)) in the cell image 10, the presence or absence of reflection of stray light, the presence or absence of a decrease in the image quality of the cell image 10 due to the culture medium, the presence or absence of a region 83 (see FIG. 2(F)) where the pixel values in the cell image 10 are saturated, and the presence or absence of the cell 90a (see FIG. 2(G)) where the boundary 84 between the region of the cell 90 and the region 91 (see FIG. 2(G)) other than the cell 90 is not clear. In the present embodiment, the image determination unit 2b determines whether the cell image 10 includes at least one of the blur of the cell 90 shown in the cell image 10, the reflection of the foreign matter 80 in the cell image 10, the reflection of stray light in the cell image 10, the decrease in the image quality of the cell image 10 due to the culture medium when culturing the cell 90, the region 83 where the pixel values in the cell image 10 are saturated, and the cell 90a where the boundary 84 between the region of the cell 90 and the region 91 other than the cell 90 is not clear. Note that in the present embodiment, the cell image 10 is an image of the cell 90 having differentiation ability. For example, the cell 90 includes iPS cells (induced Pluripotent Stem Cells), ES cells (Embryonic Stem Cells), and the like.

[0017] The image analysis unit 2c is configured to analyze the cell image 10. In the present embodiment, the image analysis unit 2c is configured to analyze, for example, whether the cell 90 (see FIG. 2) shown in the cell image 10 is an undifferentiated cell or an undifferentiated deviated cell. Note that an undifferentiated cell is a cell having the ability to differentiate. Also, an undifferentiated deviated cell is a cell that does not have the ability to differentiate.

[0018] The learned model generation unit 2d is configured to generate the first learned model 6 by training the first learning model 6a (see FIG. 5). Also, the learned model generation unit 2d is configured to generate the second learned model 7 by training the second learning model 7a (see FIG. 9). Details of the configuration in which the learned model generation unit 2d generates the first learned model 6 and the second learned model 7 will be described later.

[0019] The inappropriate cell image generation unit 2e is configured to generate an inappropriate cell image 12 from the appropriate cell image 11. Details of the configuration in which the inappropriate cell image generation unit 2e generates the inappropriate cell image 12 will be described later.

[0020] The storage unit 3 is configured to store the cell image 10, the first learned model 6, and the second learned model 7. Also, the storage unit 3 is configured to store various programs executed by the processor 2. The storage unit 3 includes, for example, a storage device such as an HDD (Hard Disk Drive) or an SSD (Solid State Drive).

[0021] The display unit 4 is configured to display the analysis result 50 analyzed by the image analysis unit 2c and the like. The display unit 4 includes, for example, a display device such as a liquid crystal monitor.

[0022] The input reception unit 5 is configured to be able to receive an operation input by an operator. The input reception unit 5 includes, for example, an input device such as a mouse or a keyboard.

[0023] (Appropriate cell image and inappropriate cell image) With reference to FIGS. 2(A) to 2(G), the appropriate cell image 11 and the inappropriate cell image 12 will be described.

[0024] FIG. 2(A) is a schematic diagram of the appropriate cell image 11. The appropriate cell image 11 is an image that meets the judgment criteria. Specifically, the appropriate cell image 11 has no blur of the cell 90, no reflection of the foreign matter 80 (see FIG. 2(C)), no reflection of stray light, no deterioration in image quality due to the culture medium, no region 83 (see FIG. 2(F)) where the pixel value is saturated, and no cell 90a with an unclear boundary 84 (see FIG. 2(G)) between the region of the cell 90 and the region 91 outside the cell 90 (see FIG. 2(G)).

[0025] FIG. 2(B) is a schematic diagram of the inappropriate cell image 12a. In the inappropriate cell image 12a shown in FIG. 2(B), the cell 90 appears blurred. In other words, the inappropriate cell image 12a is an image that is out of focus when photographing the cell image 10, and the inside of the cell 90 is unclear. That is, the inappropriate cell image 12a shown in FIG. 2(B) is an image that does not meet the judgment criteria for the presence or absence of blur of the cell 90 shown in the cell image 10. In the example shown in FIG. 2(B), for the sake of convenience, the boundary line between the cell 90 and the background is shown as a dashed line to represent the blurred state of the cell 90. Also, in the example shown in FIG. 2(B), for the sake of convenience, the cell 90 is given a different hatching from the cell 90 shown in FIG. 2(A) to represent that the inside of the cell 90 is unclear.

[0026] FIG. 2(C) is a schematic diagram of the inappropriate cell image 12b. The inappropriate cell image 12b shown in FIG. 2(C) is an image in which the foreign matter 80 is reflected. That is, the inappropriate cell image 12 shown in FIG. 2(C) is an image that does not meet the judgment criteria for the presence or absence of reflection of the foreign matter 80 in the cell image 10. The foreign matter 80 is something other than the cell 90 and includes objects mixed in from the outside and insoluble substances contained in the culture medium.

[0027] FIG. 2(D) is a schematic diagram of an inappropriate cell image 12c. The inappropriate cell image 12c shown in FIG. 2(D) is an image in which stray light is reflected. That is, the inappropriate cell image 12c shown in FIG. 2(D) is an image that does not meet the criteria for determining the presence or absence of stray light reflection in the cell image 10. Note that the inappropriate cell image 12c shown in FIG. 2(D) is an image in which stray light reflected by a container or the like is reflected as a bright line 81.

[0028] FIG. 2(E) is a schematic diagram of an inappropriate cell image 12d. The inappropriate cell image 12d shown in FIG. 2(E) is an image in which the image quality of the cell image 10 has deteriorated due to the culture medium when culturing the cells 90. That is, the inappropriate cell image 12d shown in FIG. 2(E) is an image that does not meet the criteria for determining the presence or absence of deterioration in the image quality of the cell image 10 due to the culture medium. Here, at the position where the inner peripheral surface of the container for culturing the cells 90 contacts the culture medium, the liquid surface of the culture medium bends due to surface tension. This is called a meniscus. The thickness of the culture medium in the optical axis direction of the optical member of the microscope 20 differs between the part where the meniscus occurs and the part where the meniscus does not occur. In the case of a meniscus when the liquid surface of the culture medium is concave, the thickness of the culture medium in the meniscus part becomes large. Therefore, when a concave meniscus occurs, the meniscus part becomes dark. Also, in the case of a meniscus when the liquid surface of the culture medium is convex, the thickness of the culture medium in the meniscus part becomes small. Therefore, when a convex meniscus occurs, the meniscus part becomes bright. The example shown in FIG. 2(E) is a schematic diagram when a concave meniscus occurs. That is, the deterioration in image quality due to the culture medium when culturing the cells 90 means that there are regions with low luminance values or high luminance values in the cell image 10 due to the meniscus.

[0029] FIG. 2(F) is a schematic diagram of an inappropriate cell image 12e. The inappropriate cell image 12e shown in FIG. 2(F) is an image including a region 83 where pixel values in the cell image 10 are saturated. That is, the inappropriate cell image 12e shown in FIG. 2(F) is an image that does not meet the determination criteria for the presence or absence of the region 83 where pixel values in the cell image 10 are saturated. Note that the inappropriate cell image 12e shown in FIG. 2(F) is an image in which the region 83 where pixel values are saturated has occurred in the cell image 10 due to stray light or the like.

[0030] FIG. 2(G) is a schematic diagram of an inappropriate cell image 12f. The inappropriate cell image 12f shown in FIG. 2(G) is an image including a cell 90a in which a boundary 84 between a region of the cell 90 and a region 91 outside the cell 90 is not clear. In other words, the inappropriate cell image 12f has proper focus when the cell image 10 is taken, and the inside of the cell 90a is clearly shown, but due to stray light or the like, the boundary 84 has become unclear. Note that the region 91 outside the cell 90 includes, for example, the background portion of the cell image 10. In the example shown in FIG. 2(G), for the sake of convenience, the fact that the boundary 84 is not clear is represented by illustrating the boundary 84 as a broken line. In the example shown in FIG. 2(G), for the sake of convenience, the fact that the inside of the cell 90a is clearly shown is represented by attaching the same hatching as the cell 90 shown in FIG. 2(A) to the cell 90a.

[0031] (Image analysis method) Next, with reference to FIG. 3, a method for analyzing the cell image 10 by the cell image analysis method according to the present embodiment will be described. In the present embodiment, a configuration will be described in which the cell image analysis apparatus 100 classifies the cell image 10 into an appropriate cell image 11 and an inappropriate cell image 12 by analyzing the cell image 10. In the present embodiment, the cell image analysis apparatus 100 determines whether the cell image 10 is an appropriate cell image 11 or an inappropriate cell image 12 by analyzing the cell image 10 using the first learned model 6. When the cell image 10 is input, the first learned model 6 outputs a cell image classification label 13. The cell image classification label 13 includes 0 and 1. When the cell image classification label 13 is 1, it means that the cell image 10 has been classified as an appropriate cell image 11. When the cell image classification label 13 is 0, it means that the cell image 10 has been classified as an inappropriate cell image 12.

[0032] Further, the cell image analysis apparatus 100 determines whether the cell 90 shown in the appropriate cell image 11 is an undifferentiated cell or a cell deviating from undifferentiation by analyzing the appropriate cell image 11 using the second learned model 7.

[0033] FIG. 3 is a block diagram showing the flow of image processing according to the present embodiment. As shown in FIG. 3, in the present embodiment, the cell image analysis method broadly includes an image analysis method 101 and a method 102 for generating the first learned model 6.

[0034] (First learned model generation) The method 102 for generating the first trained model 6 according to this embodiment generates the first trained model 6 by causing the first learning model 6a to learn to classify the cell image 10 into an appropriate cell image 11 and an inappropriate cell image 12 using the teacher cell image 30 and the teacher cell image classification label 31. That is, by using the appropriate cell image 11 as the teacher cell image 30 as input data and 1 as the teacher cell image classification label 31 as output data, the first learning model 6a learns that the input image is the appropriate cell image 11. Also, by using the inappropriate cell image 12 as the teacher cell image 30 as input data and 0 as the teacher cell image classification label 31 as output data, the first learning model 6a learns that the input image is the inappropriate cell image 12. The first trained model 6 is, for example, a convolutional neural network (CNN) shown in FIG. 3, or includes a convolutional neural network in part. The first trained model 6 generated by causing the first learning model 6a to learn is stored in the storage unit 3 (FIG. 1) of the cell image analysis device 100.

[0035] (Image analysis method) The image analysis method 101 according to this embodiment is an image analysis method for classifying the cell image 10 acquired by the image acquisition unit 1 from a microscope 20 or the like into an appropriate cell image 11 and an inappropriate cell image 12. The image analysis method 101 according to this embodiment includes a step of acquiring the cell image 10, a step of determining whether the image quality of the cell image 10 satisfies a determination criterion, and a step of classifying the cell image 10 into an appropriate cell image 11 and an inappropriate cell image 12. Details of the processing of each step of the image analysis method 101 will be described later.

[0036] In this embodiment, as shown in FIG. 3, the step of acquiring the cell image 10 is performed by the image acquisition unit 1. The image acquisition unit 1 acquires the cell image 10 from an image capturing device such as the microscope 20. Also, the image acquisition unit 1 outputs the acquired cell image 10 to the image determination unit 2b.

[0037] Also, in the present embodiment, as shown in FIG. 3, the step of classifying the cell image 10 is performed by the image determination unit 2b. The image determination unit 2b outputs a cell image classification label 13 for the input cell image 10 by inputting the cell image 10 into the first pre-trained model 6. That is, the image determination unit 2b classifies the cell image 10 into an appropriate cell image 11 and an inappropriate cell image 12.

[0038] (First pre-trained model generation process) Next, with reference to FIG. 4, the process by which the pre-trained model generation unit 2d generates the first pre-trained model 6 will be described.

[0039] In step 200, the image acquisition unit 1 acquires the cell image 10.

[0040] In step 201, the inappropriate cell image generation unit 2e generates the inappropriate cell image 12 based on the cell image 10. Specifically, the inappropriate cell image generation unit 2e generates the inappropriate cell image 12 by performing image processing to degrade the image quality of the appropriate cell image 11. The appropriate cell image 11 used in step 201 is a cell image 10 that has been previously classified as an appropriate cell image 11 by the operator.

[0041] The inappropriate cell image generation unit 2e changes the parameters of the image processing that degrades the image quality of the appropriate cell image 11, and creates an image that falls within the allowable range as the appropriate cell image 11 and an image that does not fall within the allowable range. In the present embodiment, in the process of step 201, the inappropriate cell image generation unit 2e performs a filtering process (for example, a process using a Gaussian filter or the like) on the appropriate cell image 11, thereby reducing the contrast of the appropriate cell image 11 and making the inappropriate cell image 12a (see FIG. 2(B)) in which the cell 90 appears blurred, or an inappropriate cell image 12f (see FIG. 2(G)) including the cell 90a in which the boundary 84 between the region of the cell 90 and the region 91 other than the cell 90 is not clear. Further, the inappropriate cell image generation unit 2e sets the pixel values of some pixels of the appropriate cell image 11 to be equal to or less than a threshold value, thereby generating an inappropriate cell image 12b (see FIG. 2(C)) in which the foreign matter 80 is reflected, or an inappropriate cell image 12d (see FIG. 2(E)) in which the image quality degradation due to the medium occurs. Further, the inappropriate cell image generation unit 2e sets the pixel values of some pixels (several pixels to several tens of pixels) of the appropriate cell image 11 to be equal to or greater than a threshold value, thereby generating an inappropriate cell image 12c (see FIG. 2(D)) in which stray light is reflected, or an inappropriate cell image 12 in which the image quality degradation due to the medium occurs. Further, the inappropriate cell image generation unit 2e saturates the pixel values of some pixels (several pixels to several tens of pixels) of the appropriate cell image 11, thereby generating an inappropriate cell image 12e (see FIG. 2(F)) including the saturated region 83. The allowable range is the range of image quality set by the operator.

[0042] In step 202, the learned model generation unit 2d creates a first learned model 6 that has learned to determine whether the cell image 10 satisfies the determination criteria by learning the appropriate cell image 11 and the inappropriate cell image 12. Specifically, in the step of creating the first learned model 6, the learned model generation unit 2d creates the first learned model 6 based on the appropriate cell image 11 and the inappropriate cell image 12 generated from the appropriate cell image 11.

[0043] (Image analysis process) Next, referring to FIG. 5, the process by which the processor 2 analyzes the cell image 10 will be described. The cell image analysis method according to the present embodiment includes a step 300 of acquiring the cell image 10, a step 301 of determining whether the image quality of the cell image 10 meets the determination criteria, a step 302 of classifying the cell image 10 as an appropriate cell image 11 when the image quality of the cell image 10 meets the determination criteria, and a step 303 of classifying the cell image 10 as an inappropriate cell image 12 when the determination criteria are not met.

[0044] In step 300, the image acquisition unit 1 acquires the cell image 10.

[0045] In step 301, the image determination unit 2b determines whether the cell image 10 includes at least one of the reflection of the foreign matter 80, the reflection of stray light, the degradation of the image quality of the cell image 10 due to the culture medium, the region 83 where the pixel value is saturated, and the cell 90a with unclear boundary 84. In other words, in step 301, when classifying the cell image 10 for analyzing the cell image 10, the image determination unit 2b determines whether the image quality of the cell image 10 is suitable for analyzing the cell image 10, and when classifying the cell image 10 used for training the second learning model 7a for analyzing the cell image 10, the image determination unit 2b determines whether the image quality of the cell image 10 meets the determination criteria for at least one of the determinations of whether the image quality of the cell image 10 is suitable for the learning of the analysis for the second learning model 7a. If the cell image 10 does not include the reflection of the foreign matter 80, the reflection of stray light, the image quality of the cell image 10 due to the culture medium, the region 83 where the pixel value is saturated, and the cell 90a with unclear boundary 84, the process proceeds to step 302. If the cell image 10 includes at least one of the reflection of the foreign matter 80, the reflection of stray light, the degradation of the image quality of the cell image 10 due to the culture medium, the region 83 where the pixel value is saturated, and the cell 90a with unclear boundary 84, the process proceeds to step 303.

[0046] In step 302, the image determination unit 2b classifies the cell image 10 into an appropriate cell image 11. Specifically, the image determination unit 2b classifies the cell image 10 into an appropriate cell image 11 when the cell image 10 does not include the reflection of the foreign matter 80, the reflection of stray light, the degradation of the image quality of the cell image 10 due to the culture medium, the region 83 where the pixel value is saturated, and the cell 90a with unclear boundary 84. Further, the image determination unit 2b stores the appropriate cell image 11 in the storage unit 3. Then, the process ends.

[0047] Also, when the process proceeds from step 301 to step 303, in step 303, the image determination unit 2b classifies the cell image 10 into an inappropriate cell image 12. Specifically, the image determination unit 2b classifies the cell image 10 into an inappropriate cell image 12 when it includes at least one of the reflection of the foreign matter 80, the reflection of stray light, the degradation of the image quality of the cell image 10 due to the culture medium, the region 83 where the pixel value is saturated, and the cell 90a with unclear boundary 84. Then, the process ends.

[0048] In this embodiment, as the process of step 301, the image determination unit 2b determines whether the cell image 10 includes at least one of the blurring of the cell 90 shown in the cell image 10, the reflection of the foreign matter 80 in the cell image 10, the reflection of stray light in the cell image 10, the degradation of the image quality of the cell image 10 due to the culture medium when culturing the cell 90, the region 83 where the pixel value is saturated, and the unclear boundary 84 between the region of the cell 90 and the region 91 other than the cell 90. Specifically, in the process of step 301, the image determination unit 2b uses the first pre-trained model 6 (see FIG. 1) to determine whether the cell image 10 is an appropriate cell image 11 or an inappropriate cell image 12. Further, in the process of step 301, the image determination unit 2b performs image processing on the cell image 10 to determine whether the cell image 10 includes at least one of the reflection of the foreign matter 80, the reflection of stray light, the degradation of the image quality of the cell image 10 due to the culture medium, the region 83 where the pixel value is saturated, and the unclear boundary 84 of the cell 90a. That is, the image determination unit 2b uses both the determination by the pre-trained model 6 and the determination by the image processing. Note that when the image determination unit 2b determines the cell image 10 by image processing in the process of step 301, it determines the presence or absence of the blurring of the cell 90 and the presence or absence of the cell 90a with an unclear boundary 84 based on the contrast of the cell image 10, and based on the distribution of the luminance values of the cell image 10, it determines the presence or absence of the reflection of the foreign matter 80, the presence or absence of the reflection of stray light, the presence or absence of the degradation of the image quality of the cell image 10 due to the culture medium, and the presence or absence of the region 83 where the pixel value is saturated.

[0049] When determining based on the contrast of the cell image 10, the image determination unit 2b obtains the difference between the pixel value of a predetermined pixel in the cell image 10 and the pixel values of the pixels adjacent to the predetermined pixel. Also, the image determination unit 2b obtains the difference between the pixel value of each pixel and the pixel values of the adjacent pixels in the horizontal direction of the cell image 10, and integrates the obtained differences in the horizontal direction of the cell image 10. Further, the image determination unit 2b numerically quantifies the contrast of the entire cell image 10 by integrating the integrated differences of the pixel values in the horizontal direction in the vertical direction. Then, the image determination unit 2b determines whether there is blur in the cell 90 of the cell image 10 based on whether the numerically quantified contrast is equal to or greater than a threshold value. That is, when the numerically quantified contrast is equal to or greater than the threshold value, the image determination unit 2b determines that there is no blur in the cell 90 shown in the cell image 10 and that the cell image 10 meets the determination criteria. Also, when the numerically quantified contrast is smaller than the threshold value, the image determination unit 2b determines that the cell 90 shown in the cell image 10 is blurred or includes a cell 90a with an unclear boundary 84, and that the cell image 10 does not meet the determination criteria.

[0050] Also, when determining based on the distribution of the luminance values of the cell image 10, the image determination unit 2b numerically quantifies the ratio of the pixels having a luminance value equal to or lower than a certain level in the cell image 10 to the whole, and determines whether there is reflection of the foreign matter 80 or degradation of the image quality of the cell image 10 due to the culture medium based on whether the numerically quantified ratio is equal to or greater than a threshold value. That is, when the ratio of the pixels having the minimum luminance value is equal to or greater than the threshold value, the image determination unit 2b determines that there is reflection of the foreign matter 80 and that the cell image 10 does not meet the determination criteria. Also, when the ratio of the pixels having a luminance value equal to or lower than a certain level and not being the minimum value is equal to or greater than the threshold value, the image determination unit 2b determines that there is degradation of the image quality of the cell image 10 due to the culture medium and that the cell image 10 does not meet the determination criteria. Also, when the ratio of the pixels having a luminance value equal to or lower than a certain level is smaller than the threshold value, the image determination unit 2b determines that there is no reflection of the foreign matter 80 or degradation of the image quality of the cell image 10 due to the culture medium and that the cell image 10 meets the determination criteria.

[0051] In addition, the image determination unit 2b quantifies the ratio of the pixels having a luminance value equal to or higher than a certain level in the cell image 10 to the whole, and determines whether there is reflection of stray light, whether there is deterioration of the image quality of the cell image 10 due to the culture medium, or whether there is a region 83 where the pixel values are saturated, based on whether the quantified ratio is equal to or higher than a threshold value. That is, when the ratio of the pixels having the maximum luminance value (the luminance value is saturated) is equal to or higher than the threshold value, the image determination unit 2b determines that there is reflection of stray light or a region 83 where the pixel values are saturated, and determines that the cell image 10 does not satisfy the determination criteria. In addition, when the ratio of the pixels having a luminance value equal to or higher than a certain level and not saturated is equal to or higher than the threshold value, the image determination unit 2b determines that there is deterioration of the image quality of the cell image 10 due to the culture medium, and determines that the cell image 10 does not satisfy the determination criteria. Further, when the ratio of the pixels having a luminance value equal to or higher than a certain level is smaller than the threshold value, the image determination unit 2b determines that there is no reflection of stray light, no deterioration of the image quality of the cell image 10 due to the culture medium, or no region 83 where the pixel values are saturated, and determines that the cell image 10 satisfies the determination criteria.

[0052] In the present embodiment, when the analysis result 50 by the first learned model 6 is different from the analysis result 50 by image processing, the image determination unit 2b causes the confirmation dialog 44 (see FIG. 7) described later to be displayed on the display unit 4, or classifies it as an inappropriate cell image 12.

[0053] Note that the analysis of the cell image 10 using the first learned model 6 is mainly performed at the time of shooting the cell image 10, at the time of generating the second learned model 7, and at the time of analyzing the cell image 10 using the second learned model 7.

[0054] (Analysis processing at the time of shooting a cell image) Next, with reference to FIGS. 6 and 7, a configuration in which the cell image analysis apparatus 100 analyzes the cell image 10 when shooting the cell image 10 will be described.

[0055] FIG. 6 shows a screen displayed on the display unit 4 when photographing the cell image 10. As shown in FIG. 6, on the screen displayed on the display unit 4 when photographing the cell image 10, an image of the cell 90 being observed with the microscope 20 (see FIG. 1) is displayed in the first display area 40. In this state, when the operator presses the photographing button 41, the photographing of the cell image 10 is started. That is, when the photographing button 41 is pressed, a plurality of cell images 10 are photographed at a preset photographing position. As shown in FIG. 6, when a check is entered in the automatic image quality check column 42, the image determination unit 2b analyzes the cell image 10. When no check is entered in the automatic image quality check column 42, the cell image 10 is not analyzed by the image determination unit 2b. As shown in FIG. 6, on the display unit 4, an image of the photographed cell image 10 is displayed in the second display area 43.

[0056] If the photographed cell image 10 includes an image that does not meet the criteria, as shown in FIG. 7, a confirmation dialog 44 for whether to register the cell image 10 together with the image that does not meet the criteria is displayed on the display unit 4. The confirmation dialog 44 displays a discard button 45 and a registration button 46. When the discard button 45 is pressed, the control unit 2a discards the displayed cell image 10. When the registration button 46 is pressed, the control unit 2a registers the cell image 10. That is, even if the cell image 10 is determined to be an inappropriate cell image 12, when the operator presses the registration button 46, the cell image 10 determined to be an inappropriate cell image 12 is stored in the storage unit 3 as an appropriate cell image 11.

[0057] Next, with reference to FIG. 8, the image analysis process at the time of photographing the cell image 10 will be described.

[0058] In step 400, the image acquisition unit 1 acquires the cell image 10.

[0059] In step 401, the image determination unit 2b determines whether the image quality of the cell image 10 meets the determination criteria. If the image quality of the cell image 10 meets the determination criteria, the process proceeds to step 402. If the image quality of the cell image 10 does not meet the determination criteria, the process proceeds to step 404. Note that since the process of step 401 is the same as the process of step 301, a detailed description is omitted.

[0060] In step 402, the image determination unit 2b classifies the cell image 10 into an appropriate cell image 11.

[0061] In step 403, the image determination unit 2b stores the appropriate cell image 11 in the storage unit 3. Then, the process ends.

[0062] When the process proceeds from step 401 to step 404, in step 404, the control unit 2a notifies that the cell image 10 does not meet the determination criteria. Specifically, when the cell image 10 is classified as an inappropriate cell image 12, the control unit 2a, as shown in FIG. 7, displays a confirmation dialog 44 on the display unit 4, so that when analyzing the cell image 10, it notifies that the cell image 10 is not suitable for analysis, or when learning the second learning model 7a for analyzing the cell image 10, it notifies that it is not suitable for learning the analysis for the second learning model 7a.

[0063] In step 405, the image determination unit 2b determines whether an operation input for storing the cell image 10 classified as an inappropriate cell image 12 has been received. If an operation input for storing the cell image 10 classified as an inappropriate cell image 12 has been received, the process proceeds to step 402. If an operation input for storing the cell image 10 classified as an inappropriate cell image 12 has not been received, the process proceeds to step 406.

[0064] In step 406, the image determination unit 2b determines whether there is an operation input to discard the cell image 10. If there is an operation input to discard the cell image 10, the process proceeds to step 407. If there is no operation input to discard the cell image 10, the process returns to step 405.

[0065] In step 407, the image determination unit 2b classifies the cell image 10 as an inappropriate cell image 12. Then, the process ends.

[0066] (Analysis processing during generation of the second learned model) Next, with reference to FIGS. 9 and 10, a configuration for analyzing the cell image 10 when the cell image analysis apparatus 100 generates the second learned model 7 by training the second learning model 7a will be described. Note that the second learning model 7a is, for example, a convolutional neural network or includes a convolutional neural network in part.

[0067] As shown in FIG. 9, the learned model generation unit 2d according to the present embodiment is configured to generate the second learned model 7 by training the second learning model 7a to analyze the cell image 10 and output the analysis result 50. Specifically, the image acquisition unit 1 acquires the cell image 10 and outputs the acquired cell image 10 to the image determination unit 2b. The analysis result 50 includes a label indicating what kind of cell the cell 90 shown in the cell image 10 is. That is, the second learned model 7 is trained with the cell image 10 as input data and the label indicating whether the cell 90 shown in the input cell image 10 is an undifferentiated cell or an undifferentiated deviating cell as output data.

[0068] The image determination unit 2b, to which the cell image 10 is input, determines whether the cell image 10 is an appropriate cell image 11 or an inappropriate cell image 12. In the present embodiment, the image determination unit 2b classifies the cell image 10 by using the first learned model 6 or by image processing.

[0069] The learned model generation unit 2d learns to cause the second learning model 7a to output the analysis result 50 using the appropriate cell image 11 classified by the image determination unit 2b. In the present embodiment, the learned model generation unit 2d causes the second learning model 7a to learn to output, as the analysis result 50, whether the cell 90 shown in the appropriate cell image 11 is an undifferentiated cell or an undifferentiated deviated cell. Further, the generated second learned model 7 is stored in the storage unit 3.

[0070] FIG. 10 is a schematic diagram of a screen displayed on the display unit 4 when the learning of the second learning model 7a is completed and the generation of the second learned model 7 is completed.

[0071] The control unit 2a causes the display unit 4 to display a message 51 indicating that the generation of the second learned model 7 is completed. Further, the control unit 2a causes the display unit 4 to display the learning result list 52.

[0072] In the present embodiment, the control unit 2a causes the display unit 4 to display, as the learning result list 52, the performance 52a of the learning result, the list 52b of the learning target images, and the list 52c of the images excluded from the learning. In the example shown in FIG. 10, the control unit 2a displays Accuracy (accuracy), Precision (precision), Recall (recall rate), and IoU (Intersection over Union) as the performance 52a of the learning result.

[0073] Next, with reference to FIG. 11, the generation process of the second learned model 7 will be described.

[0074] In step 500, the image determination unit 2b acquires a cell image 10 that is a candidate for learning.

[0075] In step 501, the image determination unit 2b determines whether the image quality of the cell image 10 meets the determination criteria. If the image quality of the cell image 10 meets the determination criteria, the process proceeds to step 502. If the image quality of the cell image 10 does not meet the determination criteria, the process proceeds to step 503. When performing the process of step 501, if the cell image classification label 13 is attached to the cell image 10, the determination of the cell image 10 is performed preferentially with the cell image classification label 13. That is, if 0 of the cell image classification label 13 is attached in advance, even if it does not meet the determination criteria, it is classified as the appropriate cell image 11.

[0076] In step 502, the image determination unit 2b classifies the cell image 10 as the appropriate cell image 11.

[0077] When the process proceeds from step 501 to step 503, in step 503, the image determination unit 2b classifies the cell image 10 as the inappropriate cell image 12.

[0078] In step 504, the image determination unit 2b excludes the inappropriate cell image 12 from the learning. That is, when the cell image 10 is classified as the inappropriate cell image 12, the image determination unit 2b excludes the cell image 10 from the learning of the analysis of the cell image 10 with respect to the second learning model 7a.

[0079] In step 505, the image determination unit 2b determines whether it has confirmed whether all the images that are candidates for learning targets meet the determination criteria. If the image determination unit 2b has confirmed whether all the images that are candidates for learning targets meet the determination criteria, the process proceeds to step 506. If the image determination unit 2b has not confirmed whether all the images that are candidates for learning targets meet the determination criteria, the process proceeds to step 500.

[0080] In step 506, the trained model generation unit 2d trains the second learning model 7a. In the present embodiment, the trained model generation unit 2d generates the second trained model 7 by training the second learning model 7a to perform analysis using the cell image 10 classified as the appropriate cell image 11.

[0081] In step 507, the control unit 2a causes the display unit 4 to display the list 52b of images to be learned and the list 52c of images excluded from learning. In the present embodiment, the control unit 2a also causes the display unit 4 to display the message 51 indicating that the generation of the second trained model 7 has been completed and the performance 52a of the learning result. Then, the process ends.

[0082] (Analysis process of cell images) Next, with reference to FIGS. 12 and 13, the configuration in which the image analysis unit 2c analyzes the cell image 10 will be described.

[0083] As shown in FIG. 12, the image analysis unit 2c according to the present embodiment is configured to output the analysis result 50 by analyzing the cell image 10. Specifically, the image acquisition unit 1 acquires the cell image 10 and outputs the acquired cell image 10 to the image determination unit 2b.

[0084] The image determination unit 2b determines whether the cell image 10 is an appropriate cell image 11 or an inappropriate cell image 12. In the present embodiment, the image determination unit 2b classifies the cell image 10 by using the first trained model 6 and image processing.

[0085] The image analysis unit 2c performs image analysis using the appropriate cell image 11. The image analysis unit 2c outputs the analysis result 50 obtained by analyzing the appropriate cell image 11. In the present embodiment, the control unit 2a causes the display unit 4 to display the analysis result 50.

[0086] FIG. 13 is a schematic diagram when the analysis of the cell image 10 is completed and the analysis result 50 is displayed on the display unit 4.

[0087] The control unit 2a causes the display unit 4 to display a message 60 indicating that the analysis of the cell image 10 has been completed. Further, the control unit 2a causes the display unit 4 to display a list of the analysis results 50.

[0088] In the present embodiment, the control unit 2a causes the display unit 4 to display, as a list of the analysis results 50, an analysis item 50a, a list 50b of images to be analyzed, and a list 50c of images excluded from the analysis. In the example shown in FIG. 13, the control unit 2a causes the display unit 4 to display, as the analysis item 50a, a label indicating what kind of cell the cell shown in the cell image 10 is. That is, the control unit 2a causes the display unit 4 to display, as the analysis item 50a, the probability that the cell 90 shown in the cell image 10 is A, the probability that the cell 90 is B, and the probability that the cell 90 is C.

[0089] Next, with reference to FIG. 14, the analysis process of the cell image 10 will be described.

[0090] In step 600, the image determination unit 2b acquires a cell image 10 that is a candidate for analysis.

[0091] In step 601, the image determination unit 2b determines whether or not the image quality of the cell image 10 satisfies a determination criterion. If the image quality of the cell image 10 satisfies the determination criterion, the process proceeds to step 602. If the image quality of the cell image 10 does not satisfy the determination criterion, the process proceeds to step 603. When performing the process of step 601, if the cell image classification label 13 is attached to the cell image 10, the determination of the cell image 10 is performed with priority given to the cell image classification label 13. That is, if 0 of the cell image classification label 13 is attached in advance, even if the determination criterion is not satisfied, the cell image 10 is classified as an appropriate cell image 11.

[0092] In step 602, the image determination unit 2b classifies the cell image 10 as an appropriate cell image 11.

[0093] When the process proceeds from step 601 to step 603, in step 603, the image determination unit 2b classifies the cell image 10 as an inappropriate cell image 12.

[0094] In step 604, the image determination unit 2b excludes the inappropriate cell image 12 from the analysis. That is, when the cell image 10 is classified as the inappropriate cell image 12, the image determination unit 2b excludes the cell image 10 from the analysis.

[0095] In step 605, the image determination unit 2b determines whether it has confirmed whether all the images that are candidates for analysis meet the determination criteria. When the image determination unit 2b has confirmed whether all the cell images 10 that are candidates for analysis meet the determination criteria, the process proceeds to step 606. When the image determination unit 2b has not confirmed whether all the cell images 10 that are candidates for analysis meet the determination criteria, the process proceeds to step 600.

[0096] In step 606, the image analysis unit 2c analyzes the cell image 10. In the present embodiment, the image analysis unit 2c performs the analysis using the cell image 10 classified as the appropriate cell image 11.

[0097] In step 607, the control unit 2a causes the display unit 4 to display the list 61b of the analysis target images and the list 50c of the images excluded from the analysis. In the present embodiment, the control unit 2a also causes the display unit 4 to display a message 60 indicating that the analysis of the cell image 10 has been completed, and the analysis items 50a. Then, the process ends.

[0098] (Effect of the present embodiment) In the present embodiment, the following effects can be obtained.

[0099] In this embodiment, as described above, the cell image analysis method includes a step of acquiring a cell image 10, and determining whether the cell image 10 includes at least one of the following: blurring of the cells 90 shown in the cell image 10, reflection of foreign matter 80 in the cell image 10, reflection of stray light in the cell image 10, degradation of the image quality of the cell image 10 due to the culture medium when culturing the cells 90, a region 83 where the pixel values in the cell image 10 are saturated, and a cell 90a where the boundary 84 between the region of the cells 90 and the region 91 outside the cells 90 is not clear. When the cell image 10 does not include blurring of the cells 90, reflection of foreign matter 80, reflection of stray light, degradation of the image quality of the cell image 10 due to the culture medium, a region 83 where the pixel values are saturated, and a cell 90a where the boundary 84 is not clear, the cell image 10 is classified as an appropriate cell image 11. When the cell image 10 includes at least one of blurring of the cells 90, reflection of foreign matter 80, reflection of stray light, degradation of the image quality of the cell image 10 due to the culture medium, a region 83 where the pixel values are saturated, and a cell 90a where the boundary 84 is not clear, the cell image 10 is classified as an inappropriate cell image 12. Thereby, only the appropriate cell image 11 that does not include blurring of the cells 90, reflection of foreign matter 80, reflection of stray light, degradation of the image quality of the cell image 10 due to the culture medium, a region 83 where the pixel values are saturated, and a cell 90a where the boundary 84 is not clear can be used for analysis. As a result, it is possible to suppress the occurrence of analysis using the cell image 10 including the inappropriate cell image 12, so that it is possible to suppress a decrease in the accuracy of the analysis of the cell image 10.

[0100] Also, in the above embodiment, by configuring as follows, the following further effects can be obtained.

[0101] That is, in the present embodiment, as described above, the cell image 10 is an image of cultured cells 90 cultured in a container, and the determination step is to determine whether the image quality of the cell image 10 is suitable for the analysis of the cell image 10 when classifying the cell image 10 for analyzing the cell image 10, and when classifying the cell image 10 used for training the second learning model 7a for analyzing the cell image 10, it is determined whether the image quality of the cell image 10 satisfies at least one of the determination criteria for determining whether it is suitable for the training of the analysis for the second learning model 7a. Thereby, by determining the cell image 10 according to the above determination criteria, it is possible to easily determine whether the cell image 10 is suitable for analysis and whether it is suitable for the training of the analysis for the second learning model 7a.

[0102] Also, in the present embodiment, as described above, the determination criteria include the presence or absence of blur of the cells 90, the presence or absence of foreign matter 80 being reflected, the presence or absence of stray light being reflected, the presence or absence of a decrease in the image quality of the cell image 10 due to the culture medium, the presence or absence of a region 83 where the pixel values are saturated, and the presence or absence of cells 90a with unclear boundaries 84. Thereby, a cell image 10 in which the cells 90 appear blurred, a cell image 10 in which foreign matter 80 is reflected, a cell image 10 in which stray light is reflected, a cell image 10 in which the image quality has deteriorated due to the culture medium, a cell image 10 including a region 83 where the pixel values are saturated, and a cell image 10 including cells 90a with unclear boundaries 84 can be classified as inappropriate cell images 12. As a result, due to a cell image 10 in which the cells 90 appear blurred, a cell image 10 in which foreign matter 80 is reflected, a cell image 10 in which stray light is reflected, a cell image 10 in which the image quality has deteriorated due to the culture medium, a cell image 10 including a region 83 where the pixel values are saturated, and a cell image 10 including cells 90a with unclear boundaries 84, it is possible to suppress a decrease in the accuracy of the analysis of the cell image 10 or the training of the second learning model 7a.

[0103] In addition, in the present embodiment, as described above, by training the appropriate cell image 11 and the inappropriate cell image 12, a first trained model 6 is further created that is trained to determine whether the cell image 10 meets the determination criteria. In the determination step, using the first trained model 6, it is determined whether the cell image 10 is an appropriate cell image 11 or an inappropriate cell image 12. Thereby, even when it is difficult to determine by image processing, it is possible to determine whether the cell image 10 is an appropriate cell image 11 or an inappropriate cell image 12.

[0104] In addition, in the present embodiment, as described above, the present embodiment further includes a step of generating an inappropriate cell image 12 by performing image processing that degrades the image quality of the appropriate cell image 11. In the step of creating the first trained model 6, the first trained model 6 is created based on the appropriate cell image 11 and the inappropriate cell image 12 generated from the appropriate cell image 11. Thereby, when creating the first trained model 6, even when it is difficult to obtain a sufficient number of inappropriate cell images 12 by shooting or the like, by generating inappropriate cell images 12 from appropriate cell images 11, a sufficient number of inappropriate cell images 12 can be obtained. As a result, since it is possible to use a sufficient number of appropriate cell images 11 and inappropriate cell images 12 when generating the first trained model 6, the learning accuracy of the first trained model 6 can be improved.

[0105] In addition, in the present embodiment, as described above, in the step of generating the inappropriate cell image 12, the parameters of the image processing for degrading the image quality of the appropriate cell image 11 are changed to create an image that falls within the allowable range as the appropriate cell image 11 and an image that does not fall within the allowable range. Thereby, when generating an inappropriate cell image 12 from an appropriate cell image 11, by generating an image that falls within the allowable range of the appropriate cell image 11, not only the inappropriate cell image 12 but also the number of appropriate cell images 11 can be increased. As a result, since it is possible to further increase the number of images used for generating the first trained model 6, the learning accuracy of the first trained model 6 can be further improved.

[0106] Also, in the present embodiment, as described above, in the determination step, by performing image processing on the cell image 10, it is determined whether the cell image 10 includes at least one of out-of-focus of the cell 90, reflection of the foreign matter 80, reflection of stray light, degradation of the image quality of the cell image 10 due to the culture medium, the region 83 where the pixel value is saturated, and the cell 90a whose boundary 84 is not clear. Thereby, when it is possible to easily determine by image processing such as the reflection of the foreign matter 80, it is possible to determine whether the cell image 10 is an appropriate cell image 11 or an inappropriate cell image 12 without using a learned model.

[0107] Also, in the present embodiment, as described above, in the determination step, the presence or absence of out-of-focus of the cell 90 is determined based on the contrast of the cell image 10, and based on the distribution of the luminance values of the cell image 10, the presence or absence of reflection of the foreign matter 80, the presence or absence of reflection of stray light, the presence or absence of degradation of the image quality of the cell image 10 due to the culture medium, the presence or absence of the region 83 where the pixel value is saturated, and the presence or absence of the cell 90a whose boundary 84 is not clear are determined. Thereby, by obtaining the contrast of the cell image 10 and the distribution of the luminance values, it is possible to easily determine whether the cell image 10 is an appropriate cell image 11 or an inappropriate cell image 12.

[0108] Also, in the present embodiment, as described above, when the cell image 10 is classified as an inappropriate cell image 12, the method further includes a step of excluding the cell image 10 from the analysis, and a step of performing the analysis using the cell image 10 classified as an appropriate cell image 11. Thereby, it is possible to suppress the inclusion of inappropriate cell images 12 in the analysis of the cell image 10. As a result, the accuracy of the analysis of the cell image 10 can be improved.

[0109] In addition, in the present embodiment, as described above, when the cell image 10 is classified as the inappropriate cell image 12, when analyzing the cell image 10, it is notified that the cell image 10 is not suitable for analysis, or when training the second learning model 7a for analyzing the cell image 10, it is notified that it is not suitable for the training of the analysis for the second learning model 7a. This enables the user to grasp whether the captured image is an appropriate cell image 11 or an inappropriate cell image 12 when, for example, capturing the cell image 10. As a result, the convenience of the user can be improved. In addition, since it is possible to make the user aware that the inappropriate cell image 12 is included at the time of capturing the cell image 10, it is possible to suppress the re-capture of the cell image 10. As a result, the burden on the operator due to re-capturing can be reduced.

[0110] In addition, in the present embodiment, as described above, when the cell image 10 is classified as the inappropriate cell image 12, the cell image 10 is excluded from the training of the analysis of the cell image 10 for the second learning model 7a, and the cell image 10 classified as the appropriate cell image 11 is used to train the second learning model 7a for analysis. This can suppress training using the inappropriate cell image 12 when training the second learning model 7a to analyze the cell image 10. As a result, it is possible to improve the learning accuracy of the second learning model 7a, and thus improve the analysis system of the second trained model 7.

[0111] [Modification Example] The embodiment disclosed this time should be considered as illustrative in all respects and not restrictive. The scope of the present invention is shown not by the description of the above embodiment but by the claims, and further includes all changes (modification examples) within the meaning and scope equivalent to the claims.

[0112] For example, in the above-described embodiment, an example of a configuration in which the image determination unit 2b determines whether the cell image 10 is an appropriate cell image 11 or an inappropriate cell image 12 by using the first pre-trained model 6 and image processing was shown, but the present invention is not limited thereto. For example, the image determination unit 2b may be configured to classify the cell image 10 by either the first pre-trained model 6 or image processing.

[0113] Further, in the above-described embodiment, an example of a configuration in which the image determination unit 2b classifies the inappropriate cell image 12 as an appropriate cell image 11 based on an operation input by an operator was shown, but the present invention is not limited thereto. For example, when the image determination unit 2b classifies it as an inappropriate cell image 12, it may be configured not to classify it as an appropriate cell image 11 regardless of the operation input by the operator. That is, when the image determination unit 2b once classifies it as an inappropriate cell image 12, it may be configured not to classify it as an appropriate cell image 11 thereafter.

[0114] Also, in the above-described embodiment, an example of a configuration in which the pre-trained model generation unit 2d generates the first pre-trained model 6 and the second pre-trained model 7 was shown, but the present invention is not limited thereto. For example, the first pre-trained model 6 and the second pre-trained model 7a may be generated by a device different from the cell image analysis device 100.

[0115] Further, in the above-described embodiment, an example of a configuration in which the inappropriate cell image generation unit 2e generates the inappropriate cell image 12 by performing image processing on the appropriate cell image 11 was shown, but the present invention is not limited thereto. For example, the cell image analysis device 100 may be configured to acquire the inappropriate cell image 12 generated by a device different from the cell image analysis device 100.

[0116] Also, in the above-described embodiment, an example of a configuration in which the inappropriate cell image generation unit 2e generates an image within the allowable range of the appropriate cell image 11 was shown, but the present invention is not limited thereto. For example, the inappropriate cell image generation unit 2e may be configured to generate only the inappropriate cell image 12.

[0117] In the above-described embodiment, an example of the configuration that is used in all of the cell image capturing of the cell image 10, the generation of the first learned model 6, and the analysis of the cell image 10 has been shown. However, the present invention is not limited to this. For example, the cell image analysis apparatus 100 may be used in any one of the cell image capturing of the cell image 10, the generation of the first learned model 6, and the analysis of the cell image 10. Further, the cell image analysis apparatus 100 may be used in any two of the cell image capturing of the cell image 10, the generation of the first learned model 6, and the analysis of the cell image 10.

[0118] [Aspect] Those skilled in the art will understand that the above-described exemplary embodiments are specific examples of the following aspects.

[0119] (Item 1) A step of acquiring a cell image, and a step of determining whether the cell image includes at least one of blurring of cells shown in the cell image, reflection of foreign matter in the cell image, reflection of stray light in the cell image, deterioration of the image quality of the cell image due to a culture medium when culturing the cells, a region where pixel values in the cell image are saturated, and cells in which a boundary between the cells and regions other than the cells is not clear; and a step of classifying the cell image as an appropriate cell image when the cell image does not include the blurring of the cells, the reflection of the foreign matter, the reflection of the stray light, the deterioration of the image quality of the cell image due to the culture medium, the region where the pixel values are saturated, and the cells in which the boundary is not clear, and classifying the cell image as an inappropriate cell image when the cell image includes at least one of the blurring of the cells, the reflection of the foreign matter, the reflection of the stray light, the deterioration of the image quality of the cell image due to the culture medium, the region where the pixel values are saturated, and the cells in which the boundary is not clear. A cell image analysis method comprising the steps of.

[0120] (Item 2) The cell image is an image obtained by photographing cultured cells cultured in a container, In the step of making the determination, when classifying the cell image for analyzing the cell image, it is determined whether the image quality of the cell image meets at least one of the determination criteria for determining whether the image quality of the cell image is suitable for the analysis of the cell image and for determining whether the image quality of the cell image is suitable for the learning of the analysis for the learning model used for analyzing the cell image. The cell image analysis method according to item 1.

[0121] (Item 3) The determination criteria include the presence or absence of blurring of the cell, the presence or absence of foreign matter reflection, the presence or absence of stray light reflection, the presence or absence of degradation of the image quality of the cell image due to the medium, the presence or absence of an area where the pixel values are saturated, and the presence or absence of cells with unclear boundaries. The cell image analysis method according to item 2.

[0122] (Item 4) The method further includes a step of creating a first learned model that has been learned to determine whether the cell image meets the determination criteria by training the appropriate cell image and the inappropriate cell image. In the step of making the determination, the first learned model is used to determine whether the cell image is an appropriate cell image or an inappropriate cell image. The cell image analysis method according to item 2 or 3.

[0123] (Item 5) The method further includes a step of generating an inappropriate cell image by performing image processing to degrade the image quality of the appropriate cell image. In the step of creating the first learned model, the first learned model is created based on the appropriate cell image and the inappropriate cell image generated from the appropriate cell image. The cell image analysis method according to item 4.

[0124] (Item 6) In the step of generating the inappropriate cell image, parameters of image processing for deteriorating the image quality of the appropriate cell image are changed to create an image that falls within the allowable range as the appropriate cell image and an image that does not fall within the allowable range, according to the cell image analysis method of item 5.

[0125] (Item 7) In the step of determining, by performing image processing on the cell image, it is determined whether the cell image includes at least one of the defocusing of the cell, the reflection of foreign matter, the reflection of stray light, the deterioration of the image quality of the cell image due to the culture medium, the region where the pixel value is saturated, and the cell with unclear boundaries, according to the cell image analysis method of any one of items 1 to 6.

[0126] (Item 8) In the step of determining, based on the contrast of the cell image, the presence or absence of defocusing of the cell is determined, and based on the distribution of the luminance values of the cell image, the presence or absence of the reflection of foreign matter, the presence or absence of the reflection of stray light, the presence or absence of the deterioration of the image quality of the cell image due to the culture medium, the presence or absence of the region where the pixel value is saturated, and the presence or absence of the cell with unclear boundaries are determined, according to the cell image analysis method of item 7.

[0127] (Item 9) When the cell image is classified as the inappropriate cell image, the step of excluding the cell image from the analysis, and The step of performing the analysis using the cell image classified as the appropriate cell image, according to the cell image analysis method of any one of items 1 to 8.

[0128] (Item 10) When the cell image is classified as the inappropriate cell image, in the step of performing the analysis of the cell image, a step of notifying that the cell image is not suitable for analysis or that the cell image is not suitable for learning the analysis of the learning model when learning the learning model for analyzing the cell image, according to the cell image analysis method of any one of items 1 to 8.

[0129] (Item 11) When the cell image is classified as the inappropriate cell image, excluding the cell image from the learning of the analysis of the cell image with respect to the learning model; Using the cell image classified as the appropriate cell image to train the learning model to perform the analysis, the cell image analysis method according to any one of Items 1 to 8, further comprising.

Explanation of Reference Numerals

[0130] 6 First trained model 7 Second trained model 7a Second learning model (learning model) 10 Cell image 11 Appropriate cell image 12, 12a, 12b, 12c, 12d Inappropriate cell image 80 Foreign matter 83 Region where pixel value is saturated 84 Boundary (boundary between cell region and non-cell region) 90 Cell (cultured cell) 90a Cell (cell with unclear boundary)

Claims

1. A step of acquiring a cell image; A step of determining whether the cell image includes at least one of blurring of cells shown in the cell image, reflection of foreign matter in the cell image, reflection of stray light in the cell image, degradation of the image quality of the cell image due to a culture medium when culturing the cells, a region where pixel values in the cell image are saturated, and a cell whose boundary between the cell and a region other than the cell is not clear; When the cell image does not include blurring of the cells, reflection of the foreign matter, reflection of the stray light, degradation of the image quality of the cell image due to the culture medium, a region where the pixel values are saturated, and a cell whose boundary is not clear, classifying the cell image as an appropriate cell image, and when the cell image includes at least one of blurring of the cells, reflection of the foreign matter, reflection of the stray light, degradation of the image quality of the cell image due to the culture medium, a region where the pixel values are saturated, and a cell whose boundary is not clear, classifying the cell image as an inappropriate cell image; A step of generating the inappropriate cell image by performing image processing to degrade the image quality of the appropriate cell image; A step of creating a first learned model that has learned to determine whether the cell image is an appropriate cell image or an inappropriate cell image by learning the appropriate cell image and the inappropriate cell image generated from the appropriate cell image; A cell image analysis method, wherein in the determining step, the first learned model is used to determine whether the cell image is an appropriate cell image or an inappropriate cell image.

2. The cell image is an image obtained by photographing cultured cells cultured in a container, In the determining step, when classifying the cell image for analyzing the cell image, determining whether the image quality of the cell image meets at least one of the determination criteria for determining whether the image quality of the cell image is suitable for the analysis of the cell image and for determining whether the image quality of the cell image is suitable for the learning of the analysis for the learning model used for learning the learning model for analyzing the cell image. The cell image analysis method according to claim 1.

3. The determination criteria include the presence or absence of blur in the cells, the presence or absence of reflection of foreign matter, the presence or absence of reflection of stray light, the presence or absence of deterioration in the image quality of the cell image due to the medium, the presence or absence of a region where the pixel value is saturated, and the presence or absence of cells with unclear boundaries. The cell image analysis method according to claim 2.

4. In the step of creating the first learned model, by training the appropriate cell image and the inappropriate cell image, a first learned model for determining whether the cell image satisfies the determination criteria is created. The cell image analysis method according to claim 2 or 3.

5. In the step of generating the inappropriate cell image, by changing the parameters of the image processing for deteriorating the image quality of the appropriate cell image, an image within the allowable range as the appropriate cell image and an image not within the allowable range are created. The cell image analysis method according to claim 1.

6. In the step of determining, by performing image processing on the cell image, it is determined whether the cell image includes at least one of the blur of the cells, the reflection of foreign matter, the reflection of stray light, the deterioration in the image quality of the cell image due to the medium, the region where the pixel value is saturated, and the cells with unclear boundaries. The cell image analysis method according to any one of claims 1 to 5.

7. In the step of determining, based on the contrast of the cell image, the presence or absence of blur in the cells is determined, and based on the distribution of the luminance values of the cell image, the presence or absence of reflection of foreign matter, the presence or absence of reflection of stray light, the presence or absence of deterioration in the image quality of the cell image due to the medium, the presence or absence of a region where the pixel value is saturated, and the presence or absence of cells with unclear boundaries are determined. The cell image analysis method according to claim 6.

8. When the cell image is classified as the inappropriate cell image, the step of excluding the cell image from the analysis; The step of performing the analysis using the cell image classified as the appropriate cell image. The cell image analysis method according to any one of claims 1 to 7.

9. When the cell image is classified as the inappropriate cell image, when performing the analysis of the cell image, notifying that the cell image is not suitable for analysis, or when training a learning model for analyzing the cell image, notifying that it is not suitable for training the analysis for the learning model; The cell image analysis method according to any one of claims 1 to 7, further comprising the step of

10. When the cell image is classified as the inappropriate cell image, excluding the cell image from the training of the analysis of the cell image for the learning model; Using the cell image classified as the appropriate cell image to train the learning model to perform the analysis; The cell image analysis method according to any one of claims 1 to 7, further comprising the step of

Images