Cell image analysis method and cell image analysis device

Abstract

The cell image analysis method includes the following steps: implementing a segmentation process to determine the region (12) of cells that have begun to differentiate and the region (11) of undifferentiated cells in a first image (21) to transform it into a label image (21A); acquiring a shape feature value (25) from the label image; and determining whether a candidate colony region to be a search object is included based on the shape feature value (25) and a determination criterion (30).

Description

Technical Field

[0001] The invention relates to a cell image analysis method and a cell image analysis device. Background Art

[0002] Conventionally, there is known a technique for culturing pluripotent cells such as iPS cells (induced pluripotent stem cells).

[0003] During cell culture, subculturing is performed, which means extracting the cells in culture from the culture container and transferring them to another culture container to set up the next generation of cell lines. The extraction of cells is called picking. In the culture container, there are cells that have begun to differentiate (cells that have lost their pluripotency) and other cells that are not suitable for subculturing, so during subculturing, the operator (user) selects cells that are suitable for subculturing.

[0004] Specifically, the operator (user) searches for cells suitable for passage in the culture container by confirming the cell colonies in the culture container one by one with a microscope. Then, the operator (user) marks the position where the cells suitable for passage exist, and aspirates the cell colonies at the marked position with a pipette or the like. In addition, in the early stage of culture, the operation of extracting and removing cells that have begun to differentiate (cells that have lost their pluripotency), dead cells, etc. from the culture container is also performed. Similar to passage, such removal operation is also performed by searching for removal objects using a microscope and selecting the removal objects found.

[0005] Such cell search and selection operations are delicate and burdensome to the user, so a device that automates part of the operation has been proposed (for example, see Non-Patent Document 1). Non-Patent Document 1 discloses a device that takes a microscope image of cells in a culture container, and after the user determines the selection object from the image, automatically performs the cell selection operation using a pipette by operating a controller.

[0006] Prior art literature

[0007] Non-patent document 1: "Cell culture support device CELL PICKER", [online], [searched on June 19, 2020], Internate<URL:https: / / www.an.shimadzu.co.jp / bio / cell / cellpicker / index.htm> (Cell culture support device, [Online], [Retrieved on June 19, 2020], Internet,<URL:https: / / www.an.shimadzu.co.jp / bio / cell / cellpicker / index.htm> ) Summary of the invention

[0008] Problem that the invention aims to solve

[0009] According to the device disclosed in the above-mentioned Non-Patent Document 1, the user himself / herself is also required to perform the cell search operation, and therefore it is desired to reduce the user's workload of performing the cell search operation.

[0010] However, the cells to be selected also vary depending on the purpose of cell culture to be performed by the user, the type of cells to be cultured, etc. Therefore, the user needs to actively determine which cell region meets the selection target from among the multiple cell regions observed from the microscope image. As a result, it has been difficult to reduce the burden on the user caused by the search operation accompanying the selection target when culturing pluripotent cells.

[0011] The present invention has been made to solve the above-mentioned problems, and one object of the present invention is to provide a cell image analysis method and a cell image analysis device that can effectively reduce the burden on the user caused by the search operation for cells to be selected in cell culture.

[0012] Solutions for solving problems

[0013] In order to achieve the above-mentioned purpose, the cell image analysis method of the first aspect of the present invention includes the following steps: obtaining a first image obtained by photographing a cell colony containing cells with differentiation ability; implementing a segmentation process for determining the colony area of ​​cells that have begun to differentiate and the colony area of ​​undifferentiated cells in the cell colony in the first image to transform the first image into a label image; obtaining shape feature values ​​of the cell colony from the label image; using a computer to accept input related to the colony area as a search object from a user; setting a judgment criterion for the shape feature value based on the user's input; and judging whether each cell colony contains a candidate colony area that is a search object based on the shape feature value and the judgment criterion.

[0014] The cell image analysis device of the second aspect of the present invention comprises: a storage unit, which is input with a microscope image obtained by photographing a cell colony containing cells with differentiation ability; a segmentation processing unit, which implements segmentation processing for determining the colony area of ​​cells that have begun to differentiate and the colony area of ​​undifferentiated cells in the cell colony in the microscope image to convert the microscope image into a label image; an input unit, which receives input related to the colony area as a search object; and a judgment processing unit, which judges whether each cell colony included in the microscope image contains a candidate colony area that becomes the search object, wherein the judgment processing unit is configured to: obtain a shape feature value of the cell colony from the label image, and judge the colony area based on the shape feature value and a judgment criterion of the shape feature value set based on the user's input.

[0015] The cell image analysis method of the third aspect of the present invention includes the following steps: preparing a learned model for determination obtained by machine learning; obtaining a first image obtained by photographing a cell colony containing cells with differentiation ability; implementing segmentation processing for determining the colony area of ​​cells that have begun to differentiate and the colony area of ​​undifferentiated cells in the cell colony in the first image to transform the first image into a label image; and determining whether each cell colony contains a candidate colony area that becomes a search object by inputting the label image of the first image into the learned model, wherein the step of preparing the learned model includes the following steps: receiving input of selection information of whether the cell colony in the second image acquired in advance contains the desired colony area; and preparing the learned model by machine learning using the label image obtained by segmenting the second image as input data and the selection information as teacher data.

[0016] Effects of the Invention

[0017] According to the first to third aspects of the present invention, as described above, by segmentation processing, it is possible to determine from an image the colony region of cells that have begun to differentiate and the colony region of undifferentiated cells that can be selected according to the purpose of culture, etc. Moreover, in the first and second aspects, it is possible to obtain a determination result of whether a candidate colony region that is a search object is included for each cell colony in the image based on the shape feature value and the determination criterion of the shape feature value set based on the user's input. In addition, in the third aspect, by using the learning model obtained by machine learning using the selection information of whether the desired colony region is included, it is possible to obtain a determination result of whether a candidate colony region that is a search object is included for each cell colony in the image. As a result, according to the first to third aspects of the present invention, it is possible to identify and show the cell colony that contains the colony region that is highly likely to be selected by the user, so the user does not need to observe multiple cell regions one by one to make a judgment. As a result, it is possible to effectively reduce the burden on the user caused by the search operation of cells that are selected in cell culture. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 This is a diagram showing an overview of a cell image analysis method according to one embodiment.

[0019] Figure 2 This is a flowchart for explaining the cell image analysis method.

[0020] Figure 3 This is a block diagram showing an overview of a cell image analysis device according to one embodiment.

[0021] Figure 4 It is a diagram for explaining the first learned model.

[0022] Figure 5 This is a flowchart for explaining the process of creating the first learned model.

[0023] Figure 6 It is a diagram for explaining a method of inputting selection information.

[0024] Figure 7 : is a diagram showing an example of shape feature amount.

[0025] Figure 8 Schematic diagram of a label image for explaining shape feature quantities.

[0026] Fig. 9 It is a diagram for explaining the second learned model.

[0027] Fig.10 The diagram shows an example of an input image, a teacher label image, and a label image of a segmentation result in machine learning.

[0028] Fig.11 This is a diagram for explaining an example of a selection system including a cell image analysis device.

[0029] Fig.12 It is a stereogram used to explain the selection system.

[0030] Fig.13 This is a flowchart for explaining the cell image analysis process and the selection process.

[0031] Fig.14 It is a diagram showing an example of a judgment criterion for selecting an object.

[0032] Fig.15 It is a diagram for explaining a learned model for determination according to a modification example.

[0033] Fig.16 This is a flowchart for explaining a cell image analysis method according to a modified example.

[0034] Fig.17 It is a schematic diagram showing a modified example of performing the division process and the determination process on the server side.

[0035] Fig.18 It is a schematic diagram showing a modified example of the division processing performed on the server side. DETAILED DESCRIPTION

[0036] Hereinafter, embodiments of the present invention will be described with reference to the drawings.

[0037] Reference Figure 1 to Figure 14 The cell image analysis method and the cell image analysis device 100 according to the present embodiment will be described.

[0038] (Cell Image Analysis Method)

[0039] Figure 1 The cell image analysis method shown is a method in which, in the cell selection operation in cell culture, a microscope image of a cell colony 10 is acquired, and it is determined whether the cell colony 10 captured in the microscope image includes a candidate colony region to be searched.

[0040] The cell colony 10 is a cell cluster (aggregate of multiple cells) derived from a single cell. The colony region is a region consisting of specific cells in the cell colony.

[0041] The cell picking refers to extracting cells or cell clusters to be picked from the cell culture container 90. For example, the cell picking is performed by sucking the picked target using a pipette or other instrument.

[0042] The cells that become the object of selection in the present embodiment are cells with differentiation ability. Cells with differentiation ability are, for example, iPS cells, ES cells (embryonic stem cells), etc. These cells have the differentiation pluripotency (differentiation ability) of differentiating into cells constituting various tissues and organs. In the culture of such cells, cells that maintain pluripotency, i.e., "undifferentiated cells", and cells that have separated from the undifferentiated state and have begun to differentiate, i.e., "undifferentiated detached cells", are produced. Therefore, colony regions of undifferentiated cells and colony regions of undifferentiated detached cells are formed in cell colonies. Regarding cell colonies, there is a situation that only the colony regions of undifferentiated cells and the colony regions of undifferentiated detached cells are composed of one of them, and there is also a situation that the colony regions of undifferentiated cells and the colony regions of undifferentiated detached cells are included.

[0043] In order to proliferate undifferentiated cells that maintain pluripotency, an operation called passaging is performed, which is an operation of picking up undifferentiated cells and transferring them to another culture container to culture them as the next generation of cell lines, and removing them from the culture container 90.

[0044] For example, cells used for passage may not be any cells as long as they are undifferentiated cells. When selecting, the user who performs the culturing operation searches for colony regions of undifferentiated cells that match the user's purpose and selects them as objects for selection.

[0045] In the cell image analysis method of this embodiment, when culturing such cells with differentiation ability, the colony region 11 of undifferentiated cells and the colony region 12 of undifferentiated detached cells are distinguished. In addition, in the cell image analysis method, the cell colonies 10 including the candidate colony regions to be searched for selection are identified according to the purpose of the user performing the culturing operation, thereby assisting the search operation for the cell colonies 10 to be selected. The colony region 12 of undifferentiated detached cells is an example of "colony region of cells that have started differentiation" in the claims.

[0046] like Figure 1 and Figure 2 As shown, the cell image analysis method of this embodiment includes at least the following steps 71 to 76.

[0047] (Step 71 ) A first image 21 is acquired by photographing a cell colony 10 including cells having differentiation ability.

[0048] (Step 72) Segmentation processing is performed to identify the colony region 12 of cells that have started differentiation (undifferentiated detached cells) and the colony region 11 of undifferentiated cells in the cell colony 10 in the first image 21, thereby converting the first image 21 into the label image 21A.

[0049] (Step 73) The shape feature value 25 of the cell colony 10 is acquired from the label image 21A.

[0050] (Step 74) An input related to a colony area to be searched is received from a user using a computer.

[0051] (Step 75 ) The determination criterion 30 of the shape feature amount 25 is set based on the user's input.

[0052] (Step 76 ) Based on the shape feature 25 and the determination criterion 30 , it is determined for each cell colony 10 whether or not a candidate colony region to be a search target is included.

[0053] In step 71, the first image 21 may be a microscope image obtained by photographing the cell colony 10 containing cells having differentiation ability. The microscope image is, for example, an image obtained by an optical microscope such as a transmission observation microscope or a phase contrast microscope. The first image 21 may be a holographic microscope image. The first image 21 is not particularly limited as long as it can be nondestructively photographed at a magnification that allows the shape of the cell colony 10 to be observed.

[0054] The first image 21 is an image obtained by photographing at least one cell colony 10. Figure 1 As shown, the first image 21 is acquired by photographing the cell colony 10 in the culture container 90 using the imaging device 200. The culture container 90 is a transparent flat dish such as a so-called petri dish, a microplate (well plate) having a plurality of wells, or the like.

[0055] exist Figure 2 In step 72, the acquired first image 21 is segmented. In this specification, "segmentation" is a process of segmenting an image into a plurality of regions, and is a process of dividing an input image into a plurality of label regions by assigning a label representing the detection object to the region in which the detection object is captured. The label region is a region (a part of the image) composed of a group of pixels in the image that are assigned a common label. The segmentation process is implemented by image processing using a computer.

[0056] A label is information indicating the meaning of the image portion that constitutes the label area. Segmentation is performed by assigning a label to each pixel in the image. Alternatively, a label may be assigned to a set of multiple pixels (pixel group). The type of label is called a category.

[0057] like Figure 1 As shown, by performing segmentation processing on the first image 21, a label image 21A is generated by segmenting the first image 21 into a plurality of label areas.

[0058] The segmentation process in step 72 is a process of segmenting the first image 21 into at least a label region of the colony region 12 of cells that have begun to differentiate (undifferentiated detached cells) and a label region of the colony region 11 of undifferentiated cells. In other words, classification into at least two categories is performed. Thus, the generated label image 21A includes at least two label regions, namely, the colony region 12 of undifferentiated detached cells and the colony region 11 of undifferentiated cells. The label image 21A, for example, includes three (three categories of) label regions, namely, the colony region 12 of undifferentiated detached cells, the colony region 11 of undifferentiated cells, and the background region 13 outside the colony region.

[0059] In the label image 21A, each pixel in the same label region is expressed with the same pixel value or the same color, and different label regions are expressed with different pixel values ​​or different colors. Figure 1 The label image 21A in the example is a three-valued image obtained by distinguishing each pixel in the first image 21 according to one of the three pixel values ​​corresponding to the three categories of labels (colony region of undifferentiated detached cells, colony region of undifferentiated cells, and background region). As a result, image information irrelevant to the shape of the cell colony, such as the pattern of the colony region and the gradient of light and dark, appearing in the first image 21 is removed, and only the shape of each label region can be accurately extracted.

[0060] In step 73, the shape feature 25 of the cell colony 10 is acquired from the label image 21A. The shape feature 25 is information that expresses the shape of the cell colony 10 contained in the label image 21A by numerical values. The shape feature 25 can be a quantity (numerical value) indicating the size, outline shape, width, length, etc. of the label area showing the cell colony 10. The shape feature 25 of the cell colony 10 can be a shape feature related to the entire area of ​​the cell colony 10, or a shape feature related to the colony area 11 of undifferentiated cells or the colony area 12 of undifferentiated detached cells that are part of the cell colony 10. The specific content of the shape feature 25 will be described later.

[0061] In step 74, input regarding the colony region to be searched is received via the input device provided in the computer. Then, in step 75, the computer sets the determination criterion 30 of the shape feature 25 based on the input information.

[0062] The information related to the colony region as the search object is information for setting the judgment criterion 30 of the shape feature 25. The information related to the colony region as the search object may also be the judgment criterion 30 itself. The information related to the colony region as the search object may also be information indicating the user's preference of what kind of colony region the user determines as the search object. In simpler terms, the information related to the colony region as the search object may be information indicating the user's preference for the cell colony 10, such as what kind of colony region the user desires. Therefore, based on the input related to the colony region as the search object, the judgment criterion 30 of the shape feature 25 that meets the user's own judgment criterion is set.

[0063] The judgment criterion 30 of the shape feature 25 set based on the user's input can be, for example, a threshold value of the shape feature 25. When a plurality of shape feature 25 are used for judgment, the judgment criterion 30 can include a weight for each shape feature 25 used for judgment. For example, the user inputs and sets the threshold value or weight of the shape feature 25 according to the characteristics of the cell colony 10 to be selected as the object of selection. The judgment criterion 30 can also be set by preparing a microscope image as a sample in advance, and selecting and inputting (pointing) whether a colony region that is a candidate for the search object is included for each cell colony 10 captured in the microscope image.

[0064] In step 76, for each cell colony 10 captured in the image, it is determined whether each cell colony 10 in the first image 21 (label image 21A) includes a colony region that is a candidate for the search target. The determination process is realized by image processing using a computer.

[0065] A determination process is performed on whether or not a colony region including a candidate for the search target is included, based on whether or not the shape feature 25 acquired for the cell colony 10 to be searched matches a determination criterion 30 for the shape feature 25 .

[0066] As a result of step 76, a determination result 35 is generated for each cell colony 10 in the first image 21 (label image 21A) to determine whether or not a candidate colony region to be the search target is included. The determination result 35 can be binary information of "a colony region to be the search target candidate (positive example)" or "a colony region to be the search target candidate (negative example)". As described later, the determination result 35 can be information indicating the degree of possibility of "a colony region to be the search target candidate is included".

[0067] In step 77, the determination result 35 is output. Whether to set each cell colony 10 in the first image 21 (label image 21A) as a selection object can be determined by the user himself based on the determination result 35 or automatically. For example, the user considers the determination result 35 of each cell colony 10 and selects a cell colony 10 that meets the purpose from the cell colonies 10 determined as positive examples as a selection object. In addition, for example, through automatic processing such as threshold processing, a certain cell colony 10 is selected from the cell colonies 10 determined as positive examples as a selection object. Therefore, the user can determine the selection object without performing exploration operations such as detailed confirmation of the morphology of each cell colony 10 in the culture container 90 in the selection operation.

[0068] (Cell Image Analysis Device)

[0069] Reference Figure 3 The outline of the cell image analysis device 100 of this embodiment is described. The cell image analysis device 100 is a device that performs Figure 1 and Figure 2 A device showing the cell image analysis method.

[0070] Figure 3 The cell image analysis device 100 shown includes a storage unit 110, a segmentation processing unit 121, a determination processing unit 122, and an input unit 140. The cell image analysis device 100 can acquire a microscope image from the cell imaging device 200 by inputting and outputting signals, data communication, or data transfer via a recording medium.

[0071] A microscope image obtained by photographing a cell colony 10 including cells having differentiation ability is input to the storage unit 110. The microscope image is Figure 1 The cell image analysis device 100 is configured to store the image data of the first image 21 acquired from the cell imaging device 200 in the storage unit 110. Figure 2 Step 71.

[0072] The segmentation processing unit 121 performs segmentation processing for determining the colony region 12 of cells that have started differentiation and the colony region 11 of undifferentiated cells in the cell colony 10 in the microscope image (first image 21). Figure 2 Step 72. The segmentation processing unit 121 transforms the first image 21 into the label image 21A (see Figure 1 ).

[0073] The determination processing unit 122 determines whether or not a candidate colony region to be a search target is included for each cell colony 10 included in the microscope image.

[0074] The determination processing unit 122 is configured to obtain the shape feature value 25 (see Figure 1 ), the colony region is determined based on the shape feature 25 and the determination criterion 30 of the shape feature 25. That is, the determination processing unit 122 executes Figure 2 In steps 74 and 75 before the determination process (step 76), the determination criterion 30 of the shape feature value 25 is set for the cell image analysis device 100 based on the input by the user using the input unit 140 and stored in the storage unit 110.

[0075] The determination processing unit 122 Figure 2 The determination result 35 is outputted in step 77. Whether to set each cell colony 10 in the first image 21 (label image 21A) as a selection target can be determined by the user based on the determination result 35 or automatically.

[0076] exist Figure 3 In the example shown, the cell image analysis device 100 is composed of a personal computer (PC), which includes a processor 120 such as a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), and an FPGA (field-programmable gate array) for performing calculations, a storage unit 110 for storing data, and an input unit 140.

[0077] exist Figure 3 In the example of , the processor 120 functions as the segmentation processing unit 121 and the determination processing unit 122 by executing the program 111 stored in the storage unit 110. Figure 3 In the example of FIG. 1 , the division processing unit 121 and the determination processing unit 122 are implemented as functional blocks of the processor 120. The division processing unit 121 and the determination processing unit 122 may be configured as independent hardware.

[0078] The independent hardware includes the following structures: the segmentation processing unit 121 and the determination processing unit 122 are composed of different processors. The independent hardware includes the following structures: the cell image analysis device 100 includes a plurality of computers (PC), and a computer (PC) for segmentation processing and a computer (PC) for determination processing are respectively provided.

[0079] The storage unit 110 can be composed of a volatile and / or non-volatile storage device. For example, the storage unit 110 includes a hard disk drive or a solid state drive. The storage unit 110 stores a program 111 of the cell image analysis device 100. The storage unit 110 stores various data such as the acquired microscope image (first image 21).

[0080] The input unit 140 is a device for receiving operation input from the user. The input unit 140 includes input devices such as a mouse and a keyboard. Thus, the input unit 140 is configured to receive input related to the colony area as the search object. In addition, the cell image analysis device 100 is connected to the display unit 130 for displaying images. The display unit 130 includes a monitor of a liquid crystal type, an organic / inorganic EL type, etc. The input unit 140 can also be a touch panel integrated with the display unit 130.

[0081] The cell image analysis device 100 is configured to be able to set the determination criterion 30 by receiving an operation input from the user via the input unit 140. The cell image analysis device 100 can display the microscope image (first image 21), the label image 21A, the determination result 35, etc. on the display unit 130. For example, the cell image analysis device 100 receives an input of a selection operation performed on the label image 21A displayed on the display unit 130. Thus, the cell image analysis device 100 can determine the selection object (the cell colony 10 including the colony region that is a candidate for the search object) by the user.

[0082] (Setting of Judgment Criteria)

[0083] Next, an example of a method for setting the criterion 30 will be described. By accepting an operation input from the user, the criterion 30 can be automatically set based on the input information.

[0084] For example, the step 74 of accepting input related to the colony region to be searched includes the step of accepting input of selection information 40 from the user regarding whether the desired colony region is included in the cell colonies 10 in the second image 22 acquired in advance. Then, in the step 75 of setting the judgment criterion 30, the judgment criterion 30 is set based on the received selection information 40.

[0085] The second image 22 is a microscope image acquired in advance as a sample for setting the judgment standard 30. Figure 1 ) Similarly, the second image 22 is an image obtained by photographing the cell colony 10 containing cells with differentiation ability.

[0086] The selection information 40 is information indicating whether the user has selected whether the cell colony 10 in the second image 22 is a positive example (including the desired colony region) or a negative example (not including the desired colony region). In other words, the selection information 40 is information that is associated with a cell colony 10 in the second image 22 one-to-one and indicates whether the associated cell colony 10 is a positive example or a negative example. The selection information 40 is binary information indicating whether it is a positive example (e.g., "1") or a negative example (e.g., "0").

[0087] The selection information 40 is acquired by accepting a selection made by the user on the image displayed on the display unit 130, for example, via the input unit 140. In the present embodiment, the step of accepting the input of the selection information 40 includes the step of enabling the user to specify a colony region in the second image 22 or the label image 22A of the second image 22, or enabling the user to select a colony region in the second image 22 or the label image 22A of the second image 22. Thus, the selection information 40 can be input by the user actually specifying a colony region or by the user actually selecting a colony region.

[0088] Figure 6 FIG. 2 shows an example of operation input performed on the label image 22A of the second image 22. Figure 6 In the figure, the label image 22A and the schematic diagram for explanation are shown side by side. The user enters the second image 22 or the label image 22A of the second image 22 via the input unit 140 (refer to Figure 3 ) is inputted 41 to select the cell colony 10 as a positive example. Alternatively, the user actually selects the cell colony 10 as a positive example while the second image 22 or the label image 22A of the second image 22 is displayed. As a result of the selection, by removing the cell colony 10 from the image, selection information 40 indicating that the removed cell colony 10 is a positive example can be obtained. The cell colony 10 for which no input is made is given selection information 40 indicating that it is a negative example. The cell colony 10 determined by the user as a negative example may also be inputted to indicate that it is a negative example.

[0089] The selection information 40 is generated based on the user's operation input for the appropriate number of cell colonies 10. The user's tendency (i.e., preference information) of which cell colonies 10 are determined as positive examples can be understood based on the generated plurality of selection information 40. As a result, the determination criterion 30 is set based on the received selection information 40.

[0090] <First learning completion model>

[0091] In this embodiment, the determination criterion 30 can be obtained by machine learning using the selection information 40. Figure 4As shown, the step 75 of setting the judgment criterion 30 includes the following steps: by using the shape feature 25 obtained from the label image 22A of the second image 22 as input data and the selection information 40 as teacher data for machine learning, a first learned model 50 is created in which the judgment criterion 30 for the shape feature 25 is obtained. In addition, the step 76 of determining whether a candidate colony region to be a search target is included (see Figure 2 ) includes the following steps: inputting the shape feature 25 acquired from the label image 21A of the first image 21 into the first learned model 50 to generate the judgment result 35.

[0092] and Figure 1 Similarly to the label image 21A of the first image 21 shown, the label image 22A of the second image 22 is obtained by segmenting the second image 22. The label image 22A of the second image 22 is segmented into a plurality of regions similarly to the label image 21A of the first image 21. That is, the label image 22A of the second image 22 (see Figure 6 ) at least includes a colony area 12 of undifferentiated detached cells and a colony area 11 of undifferentiated cells.

[0093] The shape feature 25 is calculated for each cell colony 10 based on the label image 22A of the second image 22. The shape feature 25 of the cell colony 10 and the selection information 40 about the cell colony 10 are acquired for each cell colony 10 captured in the second image 22. By machine learning in which the shape feature 25 is used as input data and the selection information 40 is used as teacher data, the following is learned: when the shape feature 25 of the cell colony 10 of interest is provided, it is determined whether the cell colony 10 is a positive example (including a colony region that is a candidate for the search object). That is, the first learned model 50 in which the judgment criterion 30 of the shape feature 25 is obtained is generated by machine learning.

[0094] The first learned model 50 (learning model for determination processing) is, for example, an SVM (support vector machine). Preferably, the first learned model 50 is a nonlinear SVM. The first learned model 50 takes the shape feature 25 related to the unknown cell colony 10 captured in the label image 21A of the first image 21 as input to generate a determination result 35 of whether the cell colony 10 corresponds to a positive example. The first learned model 50 is configured, for example, to generate a degree of probability, i.e., a confidence level, of the cell colony 10 of interest that corresponds to a positive example within a range of 0(%) to 100(%) as the determination result 35.

[0095] exist Figure 5 The process of creating the first learned model 50 is shown in FIG. Figure 3The first learned model 50 is created using the cell image analysis device 100 shown.

[0096] In step 81 , the second image 22 is acquired from the cell imaging device 200 and input into the storage unit 110 .

[0097] In step 82, the segmentation processing unit 121 performs segmentation processing on the second image 22. The label image 22A of the second image 22 is generated by the segmentation processing.

[0098] In step 83, the input of the selection information 40 for the cell colony 10 in the label image 22A is received by the input unit 140. This step 83 is an example of "the step of receiving the input of the selection information from the user" in the cell image analysis method of this embodiment. In addition, in step 84, the shape feature 25 of the cell colony 10 to which the selection information 40 is input is calculated by the determination processing unit 122.

[0099] In step 85, the determination processing unit 122 performs machine learning using the shape feature 25 as input data and the selection information 40 as training data. Step 85 is an example of "a step of setting a determination criterion" in the cell image analysis method of the present embodiment.

[0100] In step 86, it is determined whether the learning of the prescribed number of times has been completed. If the learning of the prescribed number of times has not been completed, the process returns to step 81, and the next cell colony 10 is learned by the determination processing unit 122. If the learning of the prescribed number of times has been completed, the machine learning is completed. In step 87, the first learned model 50 produced is stored in the storage unit 110.

[0101] Through the above steps, the first learning model 50 is created by obtaining the judgment criterion 30 of the shape feature quantity 25. Figure 4 As shown, in the determination process ( Figure 2 In step 76), the determination processing unit 122 calculates the shape feature 25 based on the label image 21A of the first image 21 and inputs it to the first learned model 50. As a result, the determination processing unit 122 generates a determination result 35 through the first learned model 50. The machine learning and determination processing using such shape feature 25, for example, does not matter if the input data amount is small compared to the case where the entire image is set as input data, so the processing burden is small and high-speed processing can be performed. Therefore, the machine learning of the first learned model 50 in which the user participates can be completed in a short time, and the processing until the determination result 35 is output can be performed quickly when performing the selection operation.

[0102] (Shape feature)

[0103] Next, the shape feature amount 25 will be described.

[0104] like Figure 7 As shown, the shape feature 25 includes shape feature related to at least one of the following areas: i) the entire area of ​​the cell colony 10 included in the label image 21A; ii) the colony area 11 of undifferentiated cells included in the cell colony 10; and iii) the colony area 12 of cells that have begun to differentiate (undifferentiated detached cells) included in the cell colony 10. Figure 7 For convenience, the shape characteristic value 25A of the entire area of ​​the cell colony 10 is recorded as the "colony characteristic value", the shape characteristic value 25B of the colony area 11 of the undifferentiated cells is recorded as the "undifferentiated area characteristic value", and the shape characteristic value 25C of the colony area 12 of the undifferentiated detached cells is recorded as the "detached area characteristic value".

[0105] In this embodiment, the first image 21 (second image 22) is segmented into three categories of labels: the colony region 11 of undifferentiated cells, the colony region 12 of undifferentiated detached cells, and the background region 13. Figure 8 As shown, the overall area of ​​the cell colony 10 is the area obtained by adding the colony area 11 of the undifferentiated cells and the colony area 12 of the undifferentiated detached cells. Figure 8 Although a cell colony 10 including a colony region 11 and a colony region 12 is shown, there are also cell colonies 10 including a colony region 11 of undifferentiated cells and a cell colony 10 including a colony region 12 of undifferentiated detached cells.

[0106] like Figure 7 As shown, the shape feature 25 specifically includes at least one of i) the area of ​​the region, ii) the contour length of the region, iii) the circularity of the region, iv) the aspect ratio of the minimum circumscribed rectangle of the region, and v) the area ratio of the colony region 11 to the entire region of the cell colony 10. The "region" mentioned here is any one of the entire region of the cell colony 10, the colony region 11, and the colony region 12.

[0107] The “area of ​​the region” is equivalent to the number of pixels included in the region of interest. The “contour length of the region” is equivalent to the number of pixels constituting the contour 91 of the region of interest. The “circularity of the region” is a feature quantity whose value is closer to 1 as the contour 91 of the region of interest is closer to a circle. If the area of ​​the region of interest is S and the contour length of the region of interest is C, the circularity R of the region is expressed by R=4π×(S / C 2 )express.

[0108] The "aspect ratio of the minimum circumscribed rectangle of the region" is represented by the (short side / long side) of the minimum circumscribed rectangle 92 of the region of interest. The minimum circumscribed rectangle 92 is the rectangle with the smallest area among the rectangles surrounding the region of interest. The "area ratio of the colony region 11 to the entire area of ​​the cell colony 10" is the ratio of the colony region 11 of undifferentiated cells to the cell colony 10. The area ratio is represented by (area of ​​the colony region 11 of undifferentiated cells / area of ​​the entire cell colony 10).

[0109] Figure 7 An example of a shape feature 25 that can be used when selecting the colony region 11 of undifferentiated cells during subculturing is shown. As the shape feature 25A (colony feature) related to the entire region of the cell colony 10, the area of ​​the entire region of the cell colony 10, the outline length of the entire region of the cell colony 10, the circularity of the entire region of the cell colony 10, and the aspect ratio of the entire region of the cell colony 10 can be used.

[0110] As the shape feature 25B (undifferentiated region feature) related to the undifferentiated cell colony region 11, the area of ​​the undifferentiated cell colony region 11, the contour length of the undifferentiated cell colony region 11, the circularity of the undifferentiated cell colony region 11, and the aspect ratio of the undifferentiated cell colony region 11 can be used.

[0111] As the shape feature 25C (detached region feature) related to the colony region 12 of undifferentiated detached cells, the area of ​​the colony region 12 of undifferentiated detached cells and the area ratio of the colony region 11 to the overall area of ​​the cell colony 10 can be used. In addition, the larger the value of the area ratio, the smaller the colony region 12 of undifferentiated detached cells is. When the colony region 11 of undifferentiated cells is selected, the area ratio is a measure of the small proportion of undifferentiated detached cells.

[0112] The shape feature 25 used in the judgment process includes one or more of the above feature quantities. The feature points of the cell colony 10, which is the judgment material for the user to actually judge whether to set it as a selection object, are not single, so it is preferable to use multiple shape feature quantities 25 in the judgment process. For example, in the judgment process, at least one of the shape feature 25A related to the entire area of ​​the cell colony 10, the shape feature 25B related to the colony area 11 of undifferentiated cells, and the shape feature 25C related to the colony area 12 of undifferentiated detached cells is used one by one. The greater the number of shape feature quantities 25, the more difficult it is to set an appropriate judgment criterion 30. By using Figure 4The first learned model 50 obtained by machine learning is preferred because it can obtain a comprehensive determination criterion 30 without directly finding an appropriate threshold value related to each shape feature 25 even when a plurality of shape feature values ​​25 are used.

[0113] (Second learning completed model)

[0114] In this embodiment, Figure 2 The segmentation process shown in step 72 can be performed by performing machine learning on the segmentation process to obtain the second learned model 60 (see Fig. 9 That is, the step 72 of transforming the first image 21 into the label image 21A is as follows: Fig. 9 As shown, the method includes a step of generating a label image 21A by using a second learned model 60 that is a learned model that uses the first image 21 as input data and assigns a label of a segmentation result to the colony region 11 .

[0115] The second completed learning model 60 performs segmentation processing on the input image (first image 21, second image 22), and outputs a label image (label image 21A, label image 22A) segmented into multiple label areas. As a machine learning method, any method such as a fully convolutional neural network (FCN), a neural network, a support vector machine (SVM), or boosting can be used. From the perspective of the recognition performance of the label area, in the second completed learning model 60 of the present embodiment, it is preferred to use a convolutional neural network that is frequently used in semantic segmentation, and it is more preferred to use a fully convolutional neural network. Such a second completed learning model 60 is constructed to include an input layer, a convolutional layer, and an output layer of the input image.

[0116] In order to create the second learned model 60, machine learning is performed using a learning data set including a plurality of learning data 61. Fig.10 , an input image 23, a teacher label image 24 as teacher data, and a label image 23A as a result of segmenting the input image 23 using the second learning model 60 after machine learning are shown. The teacher label image 24 and the label image 23A are given labels of three categories: the colony area 11 of undifferentiated cells, the colony area 12 of undifferentiated detached cells, and the background area 13. Each image is actually a color image, and the label image is an image divided into three colors. However, for convenience, grayscale is performed, so for illustration, a schematic diagram representing the cell colony 10 is shown side by side in each image.

[0117] The learning data 61 used in the machine learning includes an input image 23 and a teacher label image 24 related to the same cell colony 10. The input image 23 is an original image before segmentation processing, and is an image of the cell colony 10 as in the first image 21 and the second image 22. The teacher label image 24 is produced as a correct image that should be generated as a result of the segmentation processing of the input image 23. In other words, the teacher label image 24 is a label image obtained by segmenting the input image 23 into a plurality of label regions.

[0118] The teacher label image 24 is produced by the creator of the learning image for executing machine learning. For example, a cell membrane staining image obtained by staining the cell region with a staining agent and a nuclear staining image obtained by staining the nuclear staining region of undifferentiated cells with an undifferentiated marker are obtained for the cell colony 10, and the difference between the two images is obtained after the cell membrane staining image and the nuclear staining image are binarized by threshold processing, thereby producing the teacher label image 24.

[0119] like Fig. 9 and Fig.10 As shown, when creating the second learned model 60, the learning model for segmentation processing learns the transformation processing (segmentation processing) of transforming the input image 23 into the correct teacher label image 24. By inputting the first image 21 and the second image 22 to be processed into the second learned model 60 created as a result of machine learning, the label image 21A and the label image 22A obtained by the segmentation processing are generated.

[0120] The first learned model 50 for determination processing and the second learned model 60 for segmentation processing as described above are, for example, Figure 3 A portion of the program 111 executed by the processor 120 is stored in the storage unit 110. When the processor 120 functions as the determination processing unit 122, the determination processing is performed using the first learned model 50. When the processor 120 functions as the segmentation processing unit 121, the segmentation processing is performed using the second learned model 60.

[0121] (Cell Selection System)

[0122] For a more specific structural example, refer to Fig.11 A cell picking system 500 including the cell image analysis device 100 , the cell imaging device 200 , and the cell picking device 300 according to the present embodiment will be described.

[0123] The cell picking system 500 is configured to be able to use the cell imaging device 200 to photograph the cell colony 10 in the culture container 90 (see Figure 1), the obtained first image 21 is segmented and determined, and the cell selection operation is automatically or semi-automatically performed based on the determination result 35. The cell selection system 500 includes a control unit 400 for controlling the selection operation. Fig.12 2 shows a configuration example in which the cell imaging device 200, the cell picking device 300, and the control unit 400 in the cell picking system 500 are combined.

[0124] like Fig.11 and Fig.12 As shown, the cell imaging device 200 captures the cell colony 10 in the culture container 90 to obtain a microscope image (a first image 21 and a second image 22). The cell imaging device 200 is configured to output the microscope image to the control unit 400. The cell imaging device 200 includes an illumination unit 210, an optical system 220, and an imaging unit 230.

[0125] The lighting unit 210 irradiates the optical system 220 with illumination light through the culture container 90 placed on the mounting table 450. The lighting unit 210 includes a light source such as an LED for generating illumination light in the visible wavelength region. The optical system 220 includes a lens group such as an objective lens, which sends the light incident through the culture container 90 to the camera unit 230 to form an image of a desired magnification. The camera unit 230 includes an image sensor, which converts the light received via the optical system 220 into an electrical signal to generate a microscope image. As an image sensor, there are, for example, a CCD (Charge-Coupled Device) image sensor, a CMOS (Complementary MOS: Complementary Metal Oxide Semiconductor) image sensor, and the like.

[0126] Fig.12 The cell imaging device 200 is shown as a configuration example in which an imaging unit 230 is attached to an inverted optical microscope (phase contrast microscope). The user can check the cell colony 10 with the naked eye through the eyepiece 250 of the optical microscope without viewing an image.

[0127] The control unit 400 is a computer having a processor and a storage unit (not shown). The processor functions as a control unit by executing a program stored in the storage unit. The control unit 400 is connected to the cell image analysis device 100, the cell imaging device 200, and the cell selection device 300 in a manner that allows communication. Fig.12 In the example of FIG. 4 , the control unit 400 is a tablet terminal, and includes a display unit 410 and a touch panel type input unit 420. The input unit 420 can be used for the user to input the selection information 40.

[0128] The control unit 400 acquires the captured microscope image (first image 21, second image 22) from the cell imaging device 200 and displays it on the display unit 410. The control unit 400 receives the input operation performed by the user on the displayed image via the input unit 420. The control unit 400 sends the acquired microscope image (first image 21, second image 22) to the cell image analysis device 100, and sends the selection information 40 received from the user via the input unit 420 to the cell image analysis device 100. The control unit 400 controls the selection action of the cell selection device 300 by sending coordinate information to the cell selection device 300. In addition, the control unit 400 may not be provided, and the cell image analysis device 100 may function as the control unit 400.

[0129] The cell selection device 300 is configured to select cells in the culture container 90 placed on the mounting table 450. Fig.12 In the example of FIG. 3 , the cell picking device 300 includes a nozzle unit 310 , a suction mechanism 320 , and a nozzle moving mechanism 330 .

[0130] The nozzle part 310 is configured to enter the culture container 90 to aspirate cells. The nozzle part 310 is configured to be able to attach and detach, for example, a disposable pipette tip 340. The nozzle part 310 aspirates cells from the front end of the installed pipette tip 340. The suction mechanism 320 is configured to be fluidically connected to the nozzle part 310 to apply a suction force to the nozzle part 310. The nozzle moving mechanism 330 is configured to move the nozzle part 310. The nozzle moving mechanism 330 is a robot mechanism having a driving source such as a motor, and moves the front end of the nozzle part 310 to the selection coordinates in the culture container 90 during selection, and retracts the front end of the nozzle part 310 to the outside of the culture container 90 after suction. The cell selection device 300 is controlled by the control unit 400 to start and end the selection operation, and receives the input of the selection coordinates from the control unit 400.

[0131] The stage 450 is, for example, an electric stage. In this case, the control unit 400 can control the stage 450 so as to sequentially acquire microscope images of a predetermined range in the culture container 90. The stage 450 may not be an electric stage. In this case, in the selection operation, the user can, for example, hold the culture container 90 on the stage 450 and move it, thereby sequentially acquiring a plurality of microscope images at each imaging position in the culture container 90.

[0132] exist Fig.11In the example of , the processor 120 of the cell image analysis device 100 includes a segmentation processing unit 121, a determination processing unit 122, and a selection processing unit 123 as functional blocks. The segmentation processing unit 121 uses the second learned model 60 stored in the storage unit 110 to perform segmentation processing on the microscope image (first image 21, second image 22). The determination processing unit 122 is configured to obtain the second image 22 and the selection information 40 via the control unit 400, create the first learned model 50 and store it in the storage unit 110. The determination processing unit 122 uses the first learned model 50 stored in the storage unit 110 to perform determination processing on the cell colony 10 in the first image 21. The selection processing unit 123 is configured to: determine whether the cell colony 10 meets the selection object, and set the selection coordinates 26 (refer to Figure 8 ).

[0133] The storage unit 110 stores the second learned model 60 in advance. The segmentation process is not affected by the user's preference (the tendency to determine which cell colony 10 is to be selected), so the second learned model 60 created by performing machine learning in advance can be stored in advance. Image data including the first image 21 and the second image 22, the selection information 40, etc. are input and stored in the storage unit 110.

[0134] In the cell image analysis device 100, as a preparatory operation before acquiring the first image 21 and executing the determination process and the selection process, Figure 5 The first learned model 50 is created in advance based on the second image 22 as a sample and is stored in the storage unit 110 .

[0135] First, the user prepares a culture container 90 containing cultured cells as a sample and places it on the mounting table 450. The second image 22 is captured by the cell imaging device 200 and sent to the cell image analysis device 100 ( Figure 5 Step 81).

[0136] The segmentation processing unit 121 performs segmentation processing using the second learned model 60 to generate a label image 22A ( Figure 5 Step 82). The control unit 400 may Figure 6 As shown in FIG. 1 , the second image 22 or the generated label image 22A is displayed on the display unit 410, and the input of the selection information 40 from the user is received via the input unit 420 ( Figure 5 In addition, the display unit 130 and the input unit 140 (see step 83) provided in the cell image analysis device 100 can also be used. Fig.11 ) for image display and input acceptance.

[0137] When inputting selection information 40, Figure 6 As shown in the figure, when there is a cell colony 10 including a desired colony region in the displayed image, the user inputs the selection information 40 of the positive example by clicking the image of the cell colony 10 on the screen. When the next image is displayed without clicking the image, the selection information 40 of the negative example is annotated to the cell colony 10. The control unit 400 transmits the selection information 40 to the cell imaging device 200.

[0138] When the selection information 40 is acquired, the determination processing unit 122 calculates the shape feature value 25 ( Figure 5 Step 84). The determination processing unit 122 determines from Figure 7 One or more shape feature quantities 25 set in advance are calculated from among the various shape feature quantities shown.

[0139] The determination processing unit 122 performs machine learning using the shape feature 25 of the cell colony 10 and the selection information 40 labeled with the cell colony 10 ( Figure 5 The user repeats the above operation to perform the specified number of machine learning operations required to create the first learned model 50 ( Figure 5 Step 86). Thus, the preparation of the first learned model 50 is completed. The prepared first learned model 50 is stored in the storage unit 110 ( Figure 5 Step 87).

[0140] In addition, in the process of creating the first learned model 50, the selection operation may be actually performed. That is, when the user clicks on the cell colony 10 in the displayed image, the control unit 400 obtains the clicked position as the selection coordinate 26 (see Figure 6 ). The control unit 400 sends the acquired selection coordinates 26 and a start command of the selection operation to the cell selection device 300. Thus, the cell selection device 300 selects the colony area 11 of the designated selection coordinates.

[0141] According to this structure, the user can perform the preparation process (machine learning) of the first learned model 50 by only manually inputting to actually perform the selection operation such as subculturing. For example, when there are multiple culture containers 90 to be selected, the user manually inputs to make the cell selection system 500 perform the selection operation on the first culture container 90. Since the culture container 90 contains multiple cell colonies 10 to be determined, for example, if the selection operation for the first culture container 90 is completed, the preparation process of the first learned model 50 based on machine learning is also completed. Then, as described later, the selection operation for the second and subsequent culture containers 90 can be automatically or semi-automatically performed using the first learned model 50 that has been produced.

[0142] (Image analysis and selection)

[0143] Next, refer to Fig.13 The image analysis process and the selection process performed by the cell selection system 500 using the first learned model 50 created in advance will be described.

[0144] also, Fig.13 Step 151 corresponds to Figure 2 Step 71 of the cell image analysis method of this embodiment is shown. Fig.13 Step 152 corresponds to Figure 2 Step 72 is shown. Fig.13 Step 153 in corresponds to Figure 2 Step 73 is shown. Fig.13 Step 154 ​​corresponds to Figure 2 Step 76 is shown. In addition, Fig.13 In the example of FIG. 1 , a cell removal method using a cell image analysis method is shown. That is, the cell removal method of this embodiment has Figure 2 In steps 71 to 76, a determination result 35 of whether a candidate colony region to be a search object is included is obtained. In addition, the cell removal method further includes steps 156 and 157. In step 156, based on the determination result 35 of whether a candidate colony region to be a search object is included, the selection coordinates 26 of the colony region 11 determined to be desired by the user are set. In step 157, the cell at the selection coordinates 26 is selected from the culture container 90.

[0145] In step 151 , the cell imaging device 200 captures the cell colonies 10 in the culture container 90 on the stage 450 and generates a first image 21 that captures the cell colonies 10 in the imaging field. The segmentation processing unit 121 of the cell image analysis device 100 acquires the first image 21 via the control unit 400 .

[0146] In step 152 , the segmentation processing unit 121 performs segmentation processing on the acquired first image 21 . That is, the segmentation processing unit 121 generates the label image 21A of the first image 21 by inputting the first image 21 to the second learned model 60 .

[0147] In step 153, the determination processing unit 122 acquires the shape feature value 25 of the cell colony 10 from the generated label image 21A. Figure 7 In the illustrated example, one or more predetermined shape feature quantities 25 are calculated.

[0148] In step 154, the determination processing unit 122 determines whether each cell colony 10 contains a candidate colony region to be searched. That is, the determination processing unit 122 outputs a determination result 35 of whether each cell colony 10 is a positive example (containing a candidate colony region to be searched) by inputting the shape feature 25 of each cell colony 10 into the first learned model 50. The determination result 35 is output in the form of a confidence level (numerical value) of 0 (%) to 100 (%).

[0149] In step 155, the selection processing unit 123 of the cell image analysis device 100 determines whether the cell colony 10 outputting the determination result 35 is a target for selection. If the selection processing unit 123 determines that the cell colony 10 outputting the determination result 35 is not a target for selection, the processing proceeds to step 158.

[0150] When the selection processing unit 123 determines that the cell colony 10 for which the determination result 35 is outputted corresponds to the selection target, the selection processing unit 123 sets the selection coordinates 26 of the colony region 11 determined to be desired by the user in step 156 .

[0151] In the example of performing subculturing, in the cell colony 10 determined to be suitable for selection, the position coordinates of the undifferentiated cells in the colony area 11 are set as the selection coordinates 26. For example, the selection processing unit 123 Figure 8 As shown in the example, the geometric center of gravity in the undifferentiated cell colony region 11 is calculated, and the calculated center of gravity position is set as the selection coordinate 26. In addition, the selection processing unit 123 obtains the contour shape of the area actually selected by the cell selection device 300 in advance, and sets the selection coordinate 26 to the position where the selected area converges in the undifferentiated cell colony region 11. The selection processing unit 123 outputs the set selection coordinate 26 to the control unit 400.

[0152] In step 157, the cell picking device 300 picks up the cells at the picking coordinates 26 from the culture container 90 under the control of the control unit 400. The cell picking device 300 moves the tip of the nozzle unit 310 at the picking coordinates 26 sent from the control unit 400 and performs suction, thereby picking up the cells (colony region) at the picking coordinates 26 into the pipette tip. After the tip of the nozzle unit 310 is retracted to the outside of the culture container 90, the cell picking device 300 discharges the picked up cells into a cell container at a predetermined position. When the picking is completed, the process proceeds to step 158.

[0153] In step 158, the control unit 400 determines whether to end the picking operation. If the control unit 400 determines that the picking operation is not to be ended, the control unit 400 returns the process to step 151 to acquire the next first image 21. For example, the control unit 400 determines that the picking operation is to be ended when an operation input indicating that the picking operation is to be ended is received from the user, a predetermined number of cell colonies 10 are picked, or the above-mentioned process is performed on all cell colonies 10.

[0154] Through the above steps, the image analysis process and selection process using the first learned model 50 are completed.

[0155] exist Fig.13 , an example of performing determination processing (hereinafter referred to as sequential processing) each time the first image 21 is acquired at each imaging position in the culture container 90 is shown. In the present embodiment, unlike this example, a plurality of first images 21 at each imaging position in the culture container 90 may be captured in advance, and determination processing (hereinafter referred to as batch processing) may be performed on each cell colony 10 captured in each captured first image 21. In the case of batch processing, after obtaining the determination result 35 for each cell colony 10 captured in advance, it is determined whether each cell colony 10 is set as a selection object based on each determination result 35.

[0156] Fig.14 An example of the judgment criterion for judging whether the cell colony 10 meets the selection object in step 155 is shown. As a first judgment criterion, the selection processing unit 123 judges that it meets the selection object when the value (confidence) of the judgment result 35 is greater than a preset threshold value. The threshold value is input in advance by the user. When the user wants to strictly select the cell colonies 10 that become the selection object, the user sets a relatively high threshold value, and when the user wants to select as many cell colonies 10 as possible, the user sets a relatively low threshold value. The first judgment criterion can also be applied to any of the cases of sequential processing and batch processing.

[0157] As a second judgment criterion, the selection processing unit 123 determines that the plurality of cell colonies 10 are eligible for selection when the ranking of the values ​​(confidence) of the judgment results 35 is higher than the preset ranking threshold. The second judgment criterion can be applied to batch processing. That is, after the judgment processing is performed on the plurality of (all) cell colonies 10 captured in the plurality of first images 21 through batch processing, the cell colonies 10 are sorted according to the judgment results 35, and the first N cell colonies 10 with the highest values ​​of the judgment results 35 are set as the selection objects. The ranking threshold N is input in advance by the user.

[0158] In the first judgment criterion and the second judgment criterion, by presetting the threshold value, the selection process can be fully automatically performed. In addition, as a third judgment criterion, the selection processing unit 123 determines that it meets the selection object when receiving an input operation that means to set it as a selection object. For example, the value (confidence) of the judgment result 35 is displayed on the display unit 410 together with the image of the cell colony 10, so that the user inputs whether to select the displayed cell colony 10 as the selection object through the input unit 420. The user can use the numerical value of the judgment result 35 as a reference and decide whether to set it as a selection object after personally confirming the cell colony 10.

[0159] (Effects of this embodiment)

[0160] In this embodiment, the following effects can be obtained.

[0161] The cell image analysis method of the present embodiment includes the following steps as described above: step 71, obtaining a first image 21 obtained by photographing a cell colony 10 containing cells with differentiation ability; step 72, implementing a segmentation process for determining the colony area 12 of cells that have begun to differentiate (undifferentiated detached cells) and the colony area 11 of undifferentiated cells in the cell colony 10 in the first image 21, so as to transform the first image 21 into a label image 21A; step 73, obtaining the shape feature 25 of the cell colony 10 from the label image 21A; step 74, using a computer to receive input related to the colony area as a search object from the user; step 75, setting the judgment criterion 30 of the shape feature 25 based on the user's input; and step 76, based on the shape feature 25 and the judgment criterion 30, determining whether each cell colony 10 contains a candidate colony area that is a search object.

[0162] In addition, as described above, the cell image analysis device 100 of the present embodiment includes: a storage unit 110, which is input with a microscope image (first image 21) obtained by photographing a cell colony 10 containing cells with differentiation ability; a segmentation processing unit 121, which performs segmentation processing for determining the colony area 12 of cells that have started to differentiate and the colony area 11 of undifferentiated cells in the cell colony 10 in the microscope image to transform the microscope image (first image 21) into a label image 21A; an input unit 140, which receives input related to the colony area as a search object; and a determination processing unit 122, which determines whether each cell colony 10 included in the microscope image contains a candidate colony area that becomes a search object, wherein the determination processing unit 122 is configured to: obtain the shape feature 25 of the cell colony 10 from the label image 21A, and determine the colony area based on the shape feature 25 and a determination criterion 30 of the shape feature 25 set based on the user's input.

[0163] According to the above configuration, by segmentation processing, it is possible to identify the colony region 12 of cells that have begun to differentiate and the colony region 11 of undifferentiated cells that can be the target of selection according to the purpose of culture, etc. from the image. Moreover, it is possible to obtain a determination result 35 of whether a candidate colony region that is a search target is included for each cell colony 10 in the image based on the shape feature 25 and the determination criterion 30 of the shape feature 25 set based on the user's input. As a result, in this embodiment, it is possible to determine and show the cell colony 10 that contains the colony region that the user determines to be a target of selection with a high probability, so the user does not need to observe multiple cell regions in the microscope image one by one to make a determination. As a result, it is possible to effectively reduce the burden on the user caused by the search operation for cells that are the target of selection in cell culture.

[0164] In addition, in the example of the above-mentioned embodiment, further effects can be obtained by configuring as follows.

[0165] That is, the cell image analysis method of the above embodiment includes step 83, in which the input of selection information 40 of whether the desired colony region is included in the cell colonies 10 in the second image 22 acquired in advance is received from the user, and step 85, in which the judgment criterion 30 is set based on the received selection information 40. If configured in this way, the judgment criterion 30 of whether the "candidate colony region to be the search target" is included can be set by the user inputting the judgment result obtained by the user himself / herself on the cell colonies 10 in the second image 22 as a sample. In this case, the user himself / herself does not need to find the optimal value of the threshold value of the shape feature 25, the weight of a plurality of shape feature 25, etc., which becomes the judgment criterion 30, and thus the user's workload for setting the judgment criterion 30 can be reduced.

[0166] In the cell image analysis method of the above-mentioned embodiment, the step 85 of setting the judgment criterion 30 includes the following steps: by using the shape feature 25 acquired from the label image 21A of the second image 22 as input data and the selection information 40 as teacher data for machine learning, a first learned model 50 is generated that has obtained the judgment criterion 30 for the shape feature 25. Furthermore, the step 76 of determining whether or not a candidate colony region to be a search target is included includes the following steps: inputting the shape feature 25 acquired from the label image 21A of the first image 21 into the first learned model 50 to generate a judgment result 35. If configured in this way, the user can obtain the first learned model 50 that has obtained the judgment criterion 30 for the shape feature 25 by machine learning by only inputting the selection information 40. As a result, the judgment result 35 generated by the first learned model 50 can be provided to the user. Here, the colony region captured in the first image 21 has various forms, and the user's preferences (selection tendency of the selected object) are also various. Therefore, it is actually difficult to construct the determination criterion 30 for generating the determination result 35 that meets the user's preference by a rule-based method that does not rely on machine learning. In contrast, if a machine learning method is used, it is easier to construct the determination criterion 30 composed of a combination of a plurality of shape feature quantities 25 than by a rule-based method. By learning using the selection information 40 that is the result of the user's own determination, it is easy to provide a highly accurate determination result 35 that meets the user's preference.

[0167] In the cell image analysis method of the above-mentioned embodiment, the step 83 of accepting the input of the selection information 40 includes the step of enabling the user to specify the colony region in the second image 22 or the label image 21A of the second image 22, or enabling the user to select the colony region in the second image 22 or the label image 21A of the second image 22. With such a configuration, the user can input the selection information 40 as a sample for setting the judgment criterion 30, simply by specifying the colony region in order to determine the object to be selected for the cell selection device 300. Therefore, there is no need to perform a special input operation for inputting the selection information 40, and thus the user's workload for setting the judgment criterion 30 can be reduced.

[0168] In the cell image analysis method of the above-mentioned embodiment, the shape feature 25 includes a shape feature 25 related to at least one of the following areas: i) the entire area of ​​the cell colony 10 included in the label image 21A; ii) the colony area 11 of undifferentiated cells included in the cell colony 10; and iii) the colony area 12 of undifferentiated detached cells that have begun to differentiate included in the cell colony 10. Here, the inventors of the present application conducted an in-depth study and came to the following conclusion: when the user himself determines the selection object based on the microscope image, the overall shape of the cell colony 10, the shape of the colony area 11 of undifferentiated cells, and the shape of the colony area 12 of undifferentiated detached cells will also greatly affect the user's judgment. Therefore, by using such a shape feature 25, it is possible to easily and accurately determine whether a colony area that is a candidate for a search object is included.

[0169] In the cell image analysis method of the above-mentioned embodiment, the shape feature 25 includes at least one of i) the area of ​​the region, ii) the contour length of the region, iii) the circularity of the region, iv) the aspect ratio of the minimum circumscribed rectangle of the region, and v) the area ratio of the colony region 11 to the entire region of the cell colony 10. If it is configured in this way, in order to determine whether a candidate colony region to be a search target is included, it is possible to obtain a useful feature that gives characteristics to the entire cell colony 10 and the colony region.

[0170] In addition, in the cell image analysis method of the above-mentioned embodiment, the step 72 of converting the first image 21 into the label image 21A includes the step of generating the label image 21A by the second learned model 60, which is a learned model that uses the microscope image (input image 23) obtained by photographing the cell colony 10 as input data to assign a label of the segmentation result to the colony region (11, 12). If it is constructed in this way, it is possible to perform high-precision segmentation processing using the second learned model 60 that has learned the morphology of various cell colonies 10. As described above, the colony region 11 photographed in the microscope image has a variety of morphologies, and it is difficult to implement the segmentation processing that can accurately determine the colony region 11 (undifferentiated region, detached region) of various morphologies by a rule-based method that does not rely on machine learning. In contrast, if a machine learning method is used, even morphological features that are difficult to regularize can be learned by the learning model, and high-precision segmentation processing can be achieved.

[0171] In addition, the cell removal method of the above embodiment further includes steps 156 and 157. In step 156, based on the determination result 35 of whether or not a candidate colony region to be a search target is included obtained in steps 71 to 76, the selection coordinates 26 of the colony region 11 determined to be desired by the user are set. In step 157, the cells at the selection coordinates 26 are selected from the culture container 90. According to this configuration, not only the determination result 35 of whether or not a candidate colony region to be a search target is included can be provided to the user, but also the colony region determined to be a selection target can be automatically selected based on the determination result 35. As a result, not only the search operation of the selection target can be realized, but also the automation or semi-automation until the selection operation after the search can be realized, so that the user's work burden in the entire selection operation related to cell culture can be effectively reduced.

[0172] (Variation Example)

[0173] Reference Fig.15 and Fig.16 A modified example of the cell image analysis method will be described.

[0174] In the above embodiment, an example is shown in which the shape feature 25 of the cell colony 10 is acquired from the label image 21A obtained by segmentation processing and the determination processing is performed based on the shape feature 25. However, in the embodiment of FIG. Fig.15 and Fig.16 The modification shown here shows an example in which the determination process is performed without using the shape feature 25 of the cell colony 10. In addition, since the cell image analysis method of the modification can be implemented by the same hardware configuration as the above-mentioned embodiment, the description of the hardware is omitted.

[0175] The cell image analysis method of this variant includes the following steps: step 161, preparing a learned model 55 for determination obtained by machine learning; step 162, obtaining a first image 21 obtained by photographing a cell colony 10 containing cells with differentiation ability; step 163, implementing a segmentation process for determining the colony area 12 of cells that have begun to differentiate and the colony area 11 of undifferentiated cells in the cell colony 10 in the first image 21, so as to transform the first image 21 into a label image 21A; and step 164, by inputting the label image 21A of the first image 21 into the learned model 55, determining for each cell colony 10 whether it contains a candidate colony area that is a search object.

[0176] Step 161 of producing a completed learning model 55 for determination includes step 167 and step 168. In step 167, input of selection information 40 is received as to whether the cell colony 10 in the second image 22 acquired in advance contains the desired colony area. In step 168, the completed learning model 55 is produced by performing machine learning using the label image 22A obtained by segmenting the second image 22 as input data and the selection information 40 as teacher data.

[0177] In this way, the second image 22 is used as a sample in the creation of the learned model 55 for determination. First, in step 165, the second image 22 is acquired. In step 166, the input of the selection information 40 is received. In addition, in step 167, the segmentation process is performed on the second image 22. Steps 165 to 167 are similar to Figure 5 Steps 81 to 83 are the same.

[0178] In a modified example, Figure 5 Unlike the example, the shape feature 25 is not calculated. In the modified example, Fig.15 As shown, the label image 22A of the second image 22 is set as input data instead of the shape feature 25. The selection information 40 is set as teacher data. Therefore, in step 168, the learning model learns the following by machine learning using the label image 21A as input data and the selection information 40 as teacher data: when the label image 21A of the cell colony 10 of interest is provided, it is determined (inferred) whether the cell colony 10 is a positive example (whether it contains a candidate colony area that becomes a search object).

[0179] As a machine learning method, any method such as convolutional neural network, neural network, SVM, boosting, etc. can be used. From the viewpoint of the recognition performance of the label region, it is preferable to use a convolutional neural network in the learned model 55 for determination in the modified example.

[0180] In the case where it is desired to determine whether a cell colony 10 contains a candidate colony region that is a search object based on a cell image, the information contained in the image becomes too diverse, and the determination result 35 may be difficult to converge. However, if the label image 22A obtained by the segmentation process is set as an input image as in this modification, the input image becomes an image that is distinguished by label regions of three categories (at least two categories), and information such as brightness deviation in the image caused by the texture (pattern) of the cell surface, illumination light, and external light is removed. In other words, the label image 22A can be said to be an image obtained by selectively extracting only information on the shape of the cell colony 10. As a result, in machine learning with the label image 22A as the input image, the following can be effectively learned: similarly to the shape feature 25 in the above-mentioned embodiment, based on the shape of the cell colony 10 in the image, it is determined whether the cell colony 10 is a positive example.

[0181] Once the learned model 55 for determination is created by machine learning using the second image 22 , preparations for cell image analysis on the first image 21 are complete.

[0182] like Fig.16 As shown, in the cell image analysis using the learned model 55, first, in step 162, the first image 21 is acquired, and in step 163, the label image 21A of the first image 21 is generated by segmenting the first image 21. Steps 162 and 163 are similar to Figure 2 Step 71 and step 72 are shown to be the same.

[0183] In step 164, the generated label image 21A of the first image 21 is input to the learned model 55 for determination, thereby generating a determination result 35 for each cell colony 10 as to whether or not a candidate colony region to be searched is included. In step 169, the generated determination result 35 is output.

[0184] (Effects of the Cell Image Analysis Method of Modification Example)

[0185] According to this modification, by segmentation processing, it is possible to identify the colony regions 12 of cells that have begun to differentiate and the colony regions 11 of undifferentiated cells that can be selected according to the purpose of culture, etc. from the image. Then, by using the selected information 40 of whether the desired colony region is included, the learning model 55 obtained by machine learning can be used to obtain the judgment result 35 of whether a candidate colony region that becomes the search object is included for each cell colony 10 in the image. As a result, according to the modification, the cell colony 10 containing the colony region that is judged by the user to be a high possibility of the selection object can be judged and displayed to the user, so the user does not need to observe multiple cell regions in the microscope image one by one to make a judgment. As a result, the burden on the user caused by the search operation of the cell that becomes the selection object in the cell culture can be effectively reduced.

[0186] (Other Modifications)

[0187] In addition, the embodiments disclosed this time should be considered to be illustrative in all aspects and not restrictive. The scope of the present invention is not shown by the description of the above embodiments, but by the claims, and also includes all changes (modifications) within the meaning and scope equivalent to the claims.

[0188] For example, the following example is shown in the above-mentioned embodiment: in order to perform the selection operation of the passage in the cell culture, the cell colony 10 containing the colony area 11 of undifferentiated cells is determined as the selection object, but the present invention is not limited to this. As described above, the cell colony 10 containing the colony area 12 of undifferentiated detached cells can also be determined as the selection object. That is, the colony area 12 of undifferentiated detached cells can also be set as a candidate colony area to be the search object for determination processing. The determination criteria 30 are different in the colony area 11 of undifferentiated cells and the colony area 12 of undifferentiated detached cells, so the determination criteria 30 are set separately according to the purpose of the selection. Since the selection information 40 input as the teacher data is different from the situation of the colony area 11 of undifferentiated cells, the first learned models 50 (learned models 55 for determination) are prepared separately. The first learned model 50 for passage and the first learned model 50 for removal of undifferentiated detached cells can also be pre-stored in Figure 3 The storage unit 110 shown switches the learned model to be used according to the purpose of the user.

[0189] In addition, in the above-mentioned embodiment, an example of segmentation processing of three categories of undifferentiated cells, undifferentiated detached cells and background is shown, but the present invention is not limited to this. In the present invention, segmentation processing of two categories such as live cells and dead cells can also be further performed. In addition, segmentation processing of two categories such as cells adhering to the hole and peeled cells can also be further performed. Thus, for example, cell colonies 10 containing colony areas that are candidates for search objects can be determined from cell colonies 10 containing colony areas of "live cells" and "undifferentiated cells", or cell colonies 10 containing colony areas that are candidates for search objects can be determined from cell colonies 10 containing colony areas of "adherence" and "undifferentiated cells". As another example, the category of "foreign matter" other than cells can also be added to find foreign matter mixed in the culture container 90 and remove it.

[0190] In addition, in the above embodiment, the cell image analysis device 100 and the cell imaging device 200 are connected to each other in a communicable manner, but the present invention is not limited thereto. In the present invention, part or all of the processing performed by the cell image analysis device 100 may be performed via a network.

[0191] For example, Fig.17 In the illustrated variation, the cell image analysis device 100 located at a remote location is configured as a server device connected to the cell imaging device 200 via a network 600. The user uses the cell imaging device 200 to capture the first image 21 and the second image 22, and inputs the selection information 40. The cell imaging device 200 transmits the first image 21, the second image 22, and the selection information 40 to the cell image analysis device 100 via the network 600. The cell image analysis device 100 performs segmentation processing to generate label images 21A and 22A. The cell image analysis device 100 uses the generated label image 22A and the received selection information 40 to create a first learned model 50. The cell image analysis device 100 inputs the label image 21A of the first image 21 to the first learned model 50 to perform determination processing, and transmits the determination result 35 to the cell imaging device 200. Thus, the user can obtain the determination result 35 of each cell colony 10 included in the first image 21, and perform a selection operation using the determination result 35.

[0192] In the above embodiment, the cell image analysis device 100 performs segmentation processing, creation of the first learned model 50, and determination processing, but the present invention is not limited thereto. In the present invention, part of the processing performed by the cell image analysis device 100 may be performed by other devices.

[0193] For example, Fig.18In the illustrated variation, a data processing device 610, which is a server device located at a remote location, is connected to a determination processing device 620 via a network 600 in a communicable manner. The data processing device 610 receives a microscope image (a first image 21 or a second image 22) from a PC used by a user, namely the determination processing device 620, and performs segmentation processing. The data processing device 610 performs segmentation processing using, for example, the second learned model 60, and sends the generated label image (21A, 22A) to the determination processing device 620. The determination processing device 620 creates a first learned model 50 by machine learning using the label image 22A of the second image 22 received from the data processing device 610 and the selection information 40 received from the user. The determination processing device 620 calculates the shape feature 25 of the cell colony 10 based on the label image 21A of the first image 21 received from the data processing device 610, and performs determination processing using the first learned model 50. The determination processing device 620 obtains the determination result 35 by inputting the shape feature 25 to the first learned model 50. The user can obtain the determination result 35 of each cell colony 10 included in the first image 21 and perform a selection operation using the determination result 35 .

[0194] In this way, the cell image analysis method of the present invention may be implemented through the cooperation of a plurality of computers connected to a network in a so-called cloud service or the like.

[0195] In addition, in the above embodiment, the area of ​​the region of interest, the contour length, the circularity, the aspect ratio, the area ratio relative to the entire cell colony, etc. are used as the shape feature 25, but the present invention is not limited to this. In the present invention, quantities other than the above may also be used as the shape feature 25.

[0196] In addition, in the above embodiment, an example of using the first learned model 50 to perform the determination process is shown, but the present invention is not limited to this. As described above, the determination process may be performed using a rule-based method based on the determination criteria 30 such as thresholds and weights set according to the user's preferences instead of using the machine learning method.

[0197] In addition, in the above embodiment, an example of segmentation processing using the second learned model 60 is shown, but the present invention is not limited to this. In the present invention, for example, threshold processing, feature extraction processing, etc. for images can also be used to perform segmentation processing using a rule-based method instead of a machine learning method.

[0198] In addition, Fig.12The example shown is an example of a structure in which the cell imaging device 200 is set as an optical microscope and the imaging unit 230 is provided, but the present invention is not limited to this. In the present invention, the cell imaging device 200 may be a device that is dedicated to taking microscope images of cells, or may not have the function of an optical microscope. Fig.12 In the example shown, the cell imaging device 200 functions as an optical microscope, and therefore an example in which the stage 450 is a motorized stage is shown, but the cell imaging device 200 may include an imaging unit 230 that is movable relative to a fixed stage 450 .

[0199] [Way]

[0200] It will be understood by those skilled in the art that the above-described exemplary embodiments are specific examples of the following aspects.

[0201] (Item 1)

[0202] A cell image analysis method comprises the following steps:

[0203] Acquiring a first image obtained by photographing a cell colony containing cells having differentiation ability;

[0204] Performing a segmentation process for determining a colony region of cells that have started to differentiate and a colony region of undifferentiated cells in the cell colonies in the first image to transform the first image into a label image;

[0205] Acquire the shape feature of the cell colony from the label image;

[0206] accepting input from a user regarding a colony region to be searched using a computer;

[0207] setting a determination criterion for the shape feature amount based on the user's input; and

[0208] Based on the shape feature value and the determination criterion, it is determined for each of the cell colonies whether or not the colony region that is a candidate for the search target is included.

[0209] (Item 2)

[0210] The cell image analysis method according to item 1, wherein:

[0211] The step of accepting input related to the colony region as a search target includes the steps of: accepting input from a user of selection information on whether the cell colonies in the second image acquired in advance include the desired colony region;

[0212] In the step of setting the determination criterion, the determination criterion is set based on the received selection information.

[0213] (Item 3)

[0214] The cell image analysis method according to item 2, wherein:

[0215] The step of setting the judgment criterion includes the following steps: by using the shape feature quantity obtained from the label image of the second image as input data and the selection information as teacher data for machine learning, a first learning model is produced that obtains the judgment criterion for the shape feature quantity,

[0216] The step of determining whether the colony region that is a candidate for the search target is included includes the step of inputting the shape feature acquired from the label image of the first image into the first learned model to generate a determination result.

[0217] (Item 4)

[0218] The cell image analysis method according to item 2, wherein:

[0219] The step of accepting the input of the selection information includes the following steps: enabling the user to specify the colony area in the second image or the label image of the second image, or enabling the user to select the colony area in the second image or the label image of the second image.

[0220] (Item 5)

[0221] The cell image analysis method according to item 1, wherein:

[0222] The shape feature includes shape feature related to at least one of the following areas: i) the entire area of ​​the cell colony contained in the label image; ii) the colony area of ​​the undifferentiated cells contained in the cell colony; and iii) the colony area of ​​the cells that have begun the differentiation contained in the cell colony.

[0223] (Item 6)

[0224] The cell image analysis method according to item 5, wherein:

[0225] The shape feature includes at least one of i) the area of ​​the region, ii) the contour length of the region, iii) the circularity of the region, iv) the aspect ratio of the minimum circumscribed rectangle of the region, and v) the area ratio of the colony region to the overall area of ​​the cell colony.

[0226] (Item 7)

[0227] The cell image analysis method according to item 1, wherein:

[0228] The step of converting the first image into a label image includes the step of generating the label image by a second learned model, wherein the second learned model is a learned model that uses a microscope image obtained by photographing the cell colony as input data to assign a label of a segmentation result to the colony area.

[0229] (Item 8)

[0230] A cell removal method using the cell image analysis method according to item 1,

[0231] The cell removal method comprises the following steps:

[0232] setting the selection coordinates of the colony area determined to be desired by the user based on the determination result of whether the colony area that is a candidate for the search object is included; and

[0233] The cells at the selected coordinates are selected from the culture container.

[0234] (Item 9)

[0235] A cell image analysis device, comprising:

[0236] a storage unit to which a microscope image obtained by photographing a cell colony including cells having differentiation ability is input;

[0237] a segmentation processing unit that performs segmentation processing for determining a colony region of cells that have started to differentiate and a colony region of undifferentiated cells in the cell colony in the microscope image, thereby converting the microscope image into a label image;

[0238] An input unit that receives an input related to a colony area to be searched; and

[0239] a determination processing unit for determining, for each of the cell colonies included in the microscope image, whether or not the colony region that is a candidate for the search target is included;

[0240] The determination processing unit is configured to acquire a shape feature value of the cell colony from the label image, and determine the colony region based on the shape feature value and a determination criterion of the shape feature value set based on the input of the user.

[0241] (Item 10)

[0242] A cell image analysis method comprises the following steps:

[0243] Creating a learning model for judgment obtained through machine learning;

[0244] Acquiring a first image obtained by photographing a cell colony containing cells having differentiation ability;

[0245] performing a segmentation process for determining a colony region of cells that have started to differentiate and a colony region of undifferentiated cells in the cell colonies in the first image to transform the first image into a label image; and

[0246] By inputting the label image of the first image into the learned model, it is determined for each of the cell colonies whether the colony region that is a candidate for the search target is included.

[0247] The step of making the learning model comprises the following steps:

[0248] receiving input of selection information on whether the cell colonies in the second image acquired in advance include the desired colony region; and

[0249] The learned model is created by performing machine learning using a label image obtained by segmenting the second image as input data and the selection information as teacher data.

[0250] Description of Reference Numerals

[0251] 10: cell colony; 11: colony area (colony area of ​​undifferentiated cells); 12: colony area (colony area of ​​cells that have begun to differentiate); 21: first image; 21A: label image of the first image; 22: second image; 22A: label image of the second image; 25: shape feature; 26: selected coordinates; 35: judgment result; 40: selection information; 50: first learned model; 55: learned model; 60: second learned model; 90: culture container; 100: cell image analysis device; 110: storage unit; 121: segmentation processing unit; 122: judgment processing unit.

Claims

1. A cell image analysis method, comprising the following steps: Acquiring a first image obtained by photographing a cell colony containing cells having differentiation ability; Performing a segmentation process for determining a colony region of cells that have started to differentiate and a colony region of undifferentiated cells in the cell colonies in the first image to transform the first image into a label image; Acquire the shape feature of the cell colony from the label image; accepting input from a user regarding a colony region to be searched using a computer; setting a determination criterion for the shape feature amount based on the user's input; as well as Based on the shape feature and the determination criterion, determining for each of the cell colonies whether or not the colony region that is a candidate for the search target is included; in, The step of accepting input related to the colony region as a search target includes the steps of: accepting input from a user of selection information on whether the cell colonies in the second image acquired in advance include the desired colony region; In the step of setting the determination criterion, the determination criterion is set by machine learning using the selection information.

2. The cell image analysis method according to claim 1, characterized in that: The step of setting the judgment criterion includes the following steps: by using the shape feature quantity obtained from the label image of the second image as input data and the selection information as teacher data for machine learning, a first learning model is produced that obtains the judgment criterion for the shape feature quantity, The step of determining whether the colony region that is a candidate for the search target is included includes the step of inputting the shape feature acquired from the label image of the first image into the first learned model to generate a determination result.

3. The cell image analysis method according to claim 1, characterized in that: The step of accepting the input of the selection information includes the following steps: enabling the user to specify the colony area in the second image or the label image of the second image, or enabling the user to select the colony area in the second image or the label image of the second image.

4. The cell image analysis method according to claim 1, characterized in that: The shape feature includes the shape feature related to at least one of the following areas: i) the entire area of ​​the cell colony contained in the label image; ii) the colony area of ​​the undifferentiated cells contained in the cell colony; and iii) the colony area of ​​the cells that have begun the differentiation contained in the cell colony.

5. The cell image analysis method according to claim 4, characterized in that: The shape feature includes at least one of i) the area of ​​the region, ii) the contour length of the region, iii) the circularity of the region, iv) the aspect ratio of the minimum circumscribed rectangle of the region, and v) the area ratio of the colony region to the overall area of ​​the cell colony.

6. The cell image analysis method according to claim 1, characterized in that: The step of converting the first image into a label image includes the step of generating the label image by a second learned model, wherein the second learned model is a learned model that uses a microscope image obtained by photographing the cell colony as input data to assign a label of a segmentation result to the colony area.

7. A cell removal method using the cell image analysis method according to claim 1, the cell removal method comprising the following steps: setting the selection coordinates of the colony area determined to be desired by the user based on the determination result of whether the colony area that is a candidate for the search object is included; as well as The cells at the selected coordinates are selected from the culture container.

8. A cell image analysis device comprising: a storage unit to which a microscope image obtained by photographing a cell colony including cells having differentiation ability is input; a segmentation processing unit that performs segmentation processing for determining a colony region of cells that have started to differentiate and a colony region of undifferentiated cells in the cell colony in the microscope image, thereby converting the microscope image into a label image; An input unit that receives input related to a colony area to be searched; as well as a determination processing unit for determining, for each of the cell colonies included in the microscope image, whether or not the colony region that is a candidate for the search target is included; The determination processing unit is configured to obtain a shape feature value of the cell colony from the label image, and to determine the colony area based on the shape feature value and a determination criterion of the shape feature value set based on the input of the user. The input unit is configured to receive input from a user of selection information on whether the cell colonies in the second image acquired in advance include a desired colony region, The determination processing unit is configured to set the determination criterion through machine learning using the selection information.

9. A cell image analysis method comprising the following steps: Creating a learning model for judgment obtained through machine learning; Acquiring a first image obtained by photographing a cell colony containing cells having differentiation ability; Performing a segmentation process for determining a colony region of cells that have started to differentiate and a colony region of undifferentiated cells in the cell colonies in the first image to transform the first image into a label image; as well as By inputting the label image of the first image into the learned model, it is determined whether the colony region that is a candidate for a search target is included for each of the cell colonies. The step of making the learning model comprises the following steps: receiving input of selection information on whether the cell colonies in the second image acquired in advance include the desired colony region; and The learned model is created by performing machine learning using a label image obtained by segmenting the second image as input data and the selection information as teacher data.

Images