Image analysis-based cell evaluation system, program, and cell evaluation method

Abstract

This invention provides a cell evaluation system and method using image analysis. [Solution] The cell evaluation system 10, which is executed by a computer program, includes an evaluation unit that inputs a cell image including a plurality of cells and a container in which the cells are arranged into an evaluation model and evaluates the state of the cells included in the cell image, and a division unit that divides an image taken of a plurality of cells arranged in a container to generate a plurality of cell images.

Description

[Technical Field]

[0001] This invention relates to a cell evaluation system and a cell evaluation method using image analysis. [Background technology]

[0002] Experiments to visually classify cells for the presence or absence of cytopathic effects (CPEs) are commonly performed. Patent Document 1 discloses a cell transport device comprising: a dish having a plurality of holding sections for holding cells to be transported; a microplate having a plurality of wells for receiving the cells, the wells being divided into a plurality of groups; a transport unit for transporting the cells from the dish to the microplate; an imaging unit for capturing an image of the dish with the cells held in the holding sections; an evaluation unit that, based on the image of the dish, selects cells that meet predetermined criteria from among the objects held in the dish as usable cells, and assigns an evaluation level to each of the usable cells by referring to a predetermined plurality of evaluation levels; a distribution processing unit that sets the destination for each cell on the dish so that cells with the same evaluation level are evenly distributed to each of the groups in the microplate; and a transport control unit that controls the transport unit to transport the cells held in the holding sections of the dish to each well in each group of the microplate according to the destination set by the distribution processing unit. [Prior art documents] [Patent Documents]

[0003] [Patent Document 1] Patent No. 6710772 [Overview of the project]

[0004] In a first aspect of the present invention, a cell evaluation system is provided. The cell evaluation system may include an evaluation unit that inputs a cell image including a plurality of cells and a container in which the cells are arranged into an evaluation model to evaluate the state of the cells included in the cell image. The cell evaluation system may include a division unit that divides a captured image of a plurality of cells arranged in a container to generate a plurality of cell images.

[0005] In the above, the cell evaluation system may further include a photographing unit that acquires the captured image.

[0006] In the above, the division unit may generate cell images such that the number of cells included in the cell images is a predetermined number.

[0007] In the above, the division unit may divide the captured image by a first mode in which no overlapping region occurs in adjacent cell images or a second mode in which a partial overlapping region occurs in adjacent cell images.

[0008] In the above, the cell evaluation system may further include a learning unit that learns an evaluation model based on learning data including a plurality of combinations of cell images and the states of cells given in advance.

[0009] In the above, the learning unit may learn a neural network as the evaluation model.

[0010] In the above, the neural network may include a pre-trained fixed layer and a learning layer to be learned. In the above, the learning unit may perform learning only on the learning layer.

[0011] In the above, the learning unit may learn different evaluation models according to the number of days of cell culture. In the above, the evaluation unit may evaluate the state of the cells using an evaluation model corresponding to the number of days of cell culture included in the cell image.

[0012] In the above, when the edge of the container is included in the cell image of the training data, the cell evaluation system may include an extraction unit that extracts only the edge from the cell image of the training data.

[0013] In the above, the evaluation unit may classify the cell image into a plurality of states.

[0014] In the above, the plurality of states may include a normal state, a cell death state, and a cell death omen state.

[0015] In the above, the normal state may include an initial culture normal state and a late culture normal state.

[0016] In the above, the plurality of states may include an edge state in which the edge of the container is included in the cell image.

[0017] In the above, the evaluation unit may have a calculation unit that calculates a classification attribution probability that is the probability that the cell image belongs to each of the plurality of states. In the above, the evaluation unit may have a classification unit that classifies the captured image that is the source of the cell image based on the classification attribution probability for each cell image.

[0018] In the above, the evaluation unit may input the cell image into a plurality of different evaluation models for evaluating cells infected with a plurality of different infectors, and evaluate the state of the cells included in the cell image for each infector.

[0019] In the above, the cell evaluation system may further include a display unit that represents, as contour lines, the probability that each region in the captured image belongs to a specific state based on the classification attribution probability for each cell image, and displays it overlaid on the captured image.

[0020] In the above, the cell evaluation system may further include a display unit that highlights a region in the captured image where the probability of belonging to a specific state is equal to or greater than a threshold value based on the classification attribution probability for each cell image.

[0021] In a second aspect of the present invention, a program is provided. In the above, a program may be provided that is executed by a computer and causes the computer to function as the cell evaluation system described above.

[0022] A third embodiment of the present invention provides a cell evaluation method. The cell evaluation method may include an evaluation step in which a cell image including a plurality of cells and a container in which the cells are arranged is input to an evaluation model and the state of the cells contained in the cell image is evaluated. The cell evaluation method may also include a division step in which an image captured of a plurality of cells arranged in a container is divided to generate a plurality of cell images.

[0023] It should be noted that the above summary of the invention does not enumerate all the necessary features of the present invention. Furthermore, subcombinations of these features may also constitute an invention. [Brief explanation of the drawing]

[0024] [Figure 1] An example of the configuration of the cell evaluation system 10 according to this embodiment is shown. [Figure 2] An example of the flow chart for the cell evaluation method according to this embodiment is shown. [Figure 3] An example of the subflow S100 in the flow shown in Figure 2 is presented. [Figure 4A] An example of the learning data (initial culture normal state) for S110 in the flow chart shown in Figure 3 is presented. [Figure 4B] An example of the learning data (late culture normal state) for S110 in the flow chart shown in Figure 3 is presented. [Figure 4C] An example of the training data (precursor state of cell death) for S110 in the flow chart shown in Figure 3 is presented. [Figure 4D] An example of the training data (cell death state) for S110 in the flow chart shown in Figure 3 is presented. [Figure 4E] An example of the training data (edge ​​state) for S110 in the flow shown in Figure 3 is presented. [Figure 5A] Figure 3 shows an example of extracting the edges of cell images in step S120 of the flowchart. [Figure 5B]Figure 3 shows an example of extracting the edges of cell images in step S120 of the flowchart. [Figure 5C] Figure 3 shows an example of extracting the edges of cell images in step S120 of the flowchart. [Figure 5D] Figure 3 shows an example of extracting the edges of cell images in step S120 of the flowchart. [Figure 6] Figure 3 shows an example of the layer configuration of the neural network in step S140 of the flow chart. [Figure 7] An example of an image captured at S220 in the flow chart shown in Figure 2 is presented. [Figure 8A] Figure 2 shows an example of the image segmentation (first mode) of the captured image at S300 in the flow chart. [Figure 8B] Figure 2 shows an example of the image segmentation (second mode) of the captured image at S300 in the flow chart. [Figure 9] An example of the subflow of S400 in the flow shown in Figure 2 is presented. [Figure 10] An example of calculating the classification probability in S410 of the flow chart shown in Figure 9 is presented. [Figure 11A] An example of the output (contour lines) of S500 in the flow shown in Figure 2 is presented. [Figure 11B] An example of the output (contour lines) of S500 in the flow shown in Figure 2 is presented. [Figure 12] An example of the output (highlighted) of S500 in the flow shown in Figure 2 is presented. [Figure 13] This presents a second modified concept for evaluating what type of infectious agent a cell has infected. [Figure 14] This section presents a fourth modified concept for evaluating the state of cells using an evaluation model based on the number of days the cells have been cultured. [Figure 15] An example of a computer hardware configuration is shown. [Modes for carrying out the invention]

[0025] The present invention will be described below through embodiments, but these embodiments are not intended to limit the invention as defined in the claims. Furthermore, not all combinations of features described in the embodiments are necessarily essential to the solution of the invention.

[0026] Figure 1 shows an example of the cell evaluation system 10 of this embodiment. The cell evaluation system 10 inputs a cell image, including multiple cells and a container in which the cells are arranged, into an evaluation model and evaluates the state of the cells contained in the cell image. For example, the cell evaluation system 10 determines whether the cells are undergoing cell death or showing signs of cell death. For example, the cell evaluation system 10 determines whether the cells are infected with an infectious agent (e.g., a virus, bacteria, etc.).

[0027] The cell evaluation system 10 may be a computer such as a PC (personal computer), tablet computer, smartphone, workstation, server computer, or general-purpose computer, and may also be a computer system in which multiple computers are connected.

[0028] Alternatively, the cell evaluation system 10 may be a dedicated computer designed for cell evaluation, or dedicated hardware implemented by a dedicated circuit. The cell evaluation system 10 may be implemented by a single device (computer), or by multiple devices with assigned roles. The cell evaluation system 10 may comprise all or part of the extraction unit 120, imaging unit 200, segmentation unit 300, preprocessing unit 330, learning unit 340, evaluation unit 400, and display unit 500. Although not specifically described below, the cell evaluation system 10 is equipped with memory / hard disk, etc., and the information necessary for processing is stored as appropriate, and information is transmitted between each processing module such as the extraction unit 120 and the preprocessing unit 330.

[0029] The extraction unit 120 extracts a portion from the cell images (sometimes called "small regions") of the training data used to train the evaluation model. For example, if the cell images of the training data used to train the evaluation model include the edge of the container (sometimes called an "edge region"), the extraction unit 120 extracts only the edge from the cell images of the training data. For example, the extraction unit 120 determines whether the cell image includes the edge of the container based on the brightness of the cell image and extracts only the edge from the cell image.

[0030] The training data may include multiple combinations of cell images and cell states. For example, the training data may consist of cell images, each pre-assigned with a cell state, and then combined. The cell images may include multiple cells and containers in which the cells are arranged.

[0031] The cell state may include, for example, a normal state, a dead state, and a state indicating impending cell death. The cell state may also include a state in which the cell image includes the edge of the container (sometimes called the "edge state").

[0032] Details of how the extraction unit 120 extracts only the margins from the cell images of the training data will be described later. The extraction unit 120 may output to the preprocessing unit 330 a set of training data including cell images from which only the margins have been extracted and / or a set of training data including cell images other than those from which only the margins have been extracted.

[0033] The imaging unit 200 acquires images of multiple cells arranged in a container. The imaging unit 200 may capture transmission images of cells and generate images. The imaging unit 200 may be a camera that captures transmission images of cells (e.g., a cooled camera). The imaging unit 200 may magnify and photograph cells using an objective lens placed in the optical path of a microscope. The imaging unit 200 may output the captured images to the division unit 300 and / or the display unit 500.

[0034] The division unit 300 divides the captured image acquired from the imaging unit 200 to generate multiple cell images. The division unit 300 may divide the captured image acquired by the imaging unit 200, or it may retrieve and divide cell images stored in the memory / hard disk or an information transmission medium such as a CD provided in the cell evaluation system 10. Details of how the division unit 300 divides the captured image will be described later.

[0035] The division unit 300 may output the cell images obtained by dividing the captured image to the preprocessing unit 330. If preprocessing of the cell images is not required, the division unit 300 may directly output the divided cell images to the evaluation unit 400.

[0036] The preprocessing unit 330 performs preprocessing on the cell images. For example, the preprocessing unit 330 performs preprocessing on cell images from which only the edges have been extracted by the extraction unit 120, and / or on cell images other than those from which only the edges have been extracted. Preprocessing may include resizing, rotating, and / or cropping of the images. By preprocessing the cell images, the preprocessing unit 330 can expand the training data. The preprocessing unit 330 may output the preprocessed cell images to the training unit 340.

[0037] The preprocessing unit 330 may perform preprocessing on the cell images output by the division unit 300. As preprocessing of the cell images, the preprocessing unit 330 may appropriately perform the above-mentioned image resizing, rotation, and / or cropping. The preprocessing unit 330 may output the preprocessed cell images to the evaluation unit 400. Details of how the preprocessing unit 330 preprocesses the cell images will be described later.

[0038] The learning unit 340 learns an evaluation model. For example, the learning unit 340 learns an evaluation model based on training data that includes multiple combinations of cell images and cell states preprocessed by the preprocessing unit 330. The learning unit 340 may learn an existing model, such as a neural network, as the evaluation model. Details of how the learning unit 340 learns the evaluation model will be described later. The learning unit 340 may output the learned evaluation model to the evaluation unit 400.

[0039] The evaluation unit 400 evaluates the cell images. For example, the evaluation unit 400 takes the cell images acquired from the segmentation unit 300 and / or the preprocessing unit 330 as input and inputs them to the evaluation model acquired from the learning unit 340 to evaluate the state of the cells contained in the cell images. The evaluation unit 400 may perform the evaluation by classifying the cell images into a plurality of states (for example, normal state, cell death state, and cell death precursor state, etc.) and outputting them. The evaluation unit 400 may have a calculation unit 410 and a classification unit 430.

[0040] The calculation unit 410 calculates classification assignment probabilities, which are the probabilities that a cell image belongs to each of several states of the cell. For example, the calculation unit 410 uses an evaluation model to calculate classification assignment probabilities for a cell image (small region) acquired from the segmentation unit 300 and / or the preprocessing unit 330, such as "normal state probability 90%, pre-death state probability 10%, and death state probability 0%."

[0041] The calculation unit 410 may determine the state in which the classification probability is maximized as the state of the cell image (small region). Details of how the calculation unit 410 calculates the classification probability will be described later. The calculation unit 410 may output the calculated classification probability and / or the cell image used in the calculation to the classification unit 430.

[0042] The classification unit 430 classifies the captured images. For example, the classification unit 430 classifies the captured images that are the source of the cell images divided by the division unit 300, based on the classification assignment probability for each cell image calculated by the calculation unit 410. First, the classification unit 430 may aggregate the classification assignment probability and / or the state of the cells of each subregion included in the captured image that is the source of the cell images. For example, the classification unit 430 aggregates the number of all subregions (cell images after division) in the captured image to be 1000, the number of subregions defined as "dead cells" to be 600, and the number of subregions defined as "edge regions" to be 200.

[0043] In this case, the classification unit 430 calculates the probability P of the cell death state in the captured image. CPE of, P CPE =600 / (1000-200)=0.75=75% Based on this calculation, it can be determined that 75% of the cells in the captured image are in a "dead cell state". Details of how the classification unit 430 classifies the captured image will be described later. The classification unit 430 calculates the probability of the captured image being in a dead cell state P CPE The classification results of the cell state in the captured image and / or the cell images used for classification may be output to the display unit 500.

[0044] The display unit 500 displays the processing results of the cell evaluation system 10. For example, the display unit 500 displays at least one of the following: cell images acquired from the imaging unit 200 and / or evaluation unit 400, the classification probability calculated by the calculation unit 410, the state of the cell, and the classification result of the state of the cell in the imaging image classified by the classification unit 430. The display unit 500 may, but is not limited to, an output device such as a monitor connected to the evaluation unit 400.

[0045] Thus, the cell evaluation system 10 of this embodiment can easily classify the state of cells by receiving input of cell images including multiple cells and containers in which the cells are arranged. Experimental procedures to detect cell death require human resources, computational resources, memory resources, and / or time, but the cell evaluation system 10 of this embodiment can detect the state of cells (such as cell death) in a short time, with few resources, and / or with high accuracy. The cell evaluation system 10 of this embodiment can identify and visualize the state of cells in each small region of an captured image. The cell evaluation system 10 of this embodiment can be used as a training tool for researchers involved in cell death experiments.

[0046] Figure 2 shows an example of a flow chart of the cell evaluation method according to this embodiment. The cell evaluation system 10 of this embodiment can evaluate the state of cells contained in a cell image by performing the processes S100 to S600 in Figure 2. For the sake of explanation, the processes S100 to S600 will be described in order, but at least a portion of these processes may be executed in parallel, or some steps may be omitted and / or the steps may be rearranged without departing from the spirit of the present invention.

[0047] First, as shown in S100 of Figure 2, the cell evaluation system 10 learns an evaluation model.

[0048] Figure 3 shows the subflows S110 to S140 included in S100. The cell evaluation system 10 may execute S100 by executing the subflows S110 to S140. Some parts of S110 to S140 may be omitted, and / or their order may be changed. The cell evaluation system 10 may perform additional processing in addition to S110 to S140.

[0049] First, as shown in S110, the extraction unit 120 acquires training data to be used for training the evaluation model. The extraction unit 120 may acquire the training data from a memory / hard disk or an information transmission medium such as a CD connected to the extraction unit 120.

[0050] Figures 4A to 4E show an example of training data. The training data may include multiple sets of pairs (cell image, state of the cell contained in the cell image). The state of the cell may be pre-assigned to each cell image. For example, the extraction unit 120 pre-assigns "(initial culture) normal state" to each of the three cell images in Figure 4A, and uses these three cell images as a set of training data.

[0051] A cell image is an image containing a predetermined number (one or more) of cells. The predetermined number of cells may be 1, approximately 5, approximately 10, approximately 15, approximately 20, approximately 30, approximately 40, approximately 50, etc., but is not limited to these, and may be appropriately changed depending on the size of the cells and / or the processing capacity of the learning unit 340, etc. The cell image may be an image of cells magnified to approximately 1000 times, 2000 times, approximately 3000 times, approximately 4000 times, approximately 5000 times, or approximately 10,000 times, etc., but is not limited to these.

[0052] The cell image may be an image of multiple cells aggregated together, or an image of multiple cells dispersed together. Preferably, the cell image has a resolution clear enough to distinguish the organelles inside the cell. In the cell image, the cells may be labeled with a fluorescent dye (e.g., DAPI), express recombinant proteins (e.g., proteins labeled with GFP), or contain infectious agents other than cells (e.g., mycoplasma).

[0053] Cell images may be square, rectangular, or other shapes, but are not limited to these. Cell images may or may not include the edges of the container. Cell images may include the bottom of the container and / or the culture medium.

[0054] The cell states included in the training data set may be increased or decreased as appropriate depending on the cell culture period or the purpose of the experiment. The cell states may include some or all of the normal state, the cell death state, the pre-cell death state, and the edge state. The cell states may also include other states.

[0055] For example, when the cell culture period is relatively short (e.g., 5 to 10 days), the cell state falls into three categories: normal, dead (Figure 4D), and edge state (a state including an edge region: Figure 4E). For example, when the cell culture period is relatively long (e.g., 5 to 10 days or longer), the normal state can be divided into early culture normal state and late culture normal state. When the culture period is relatively long, cells may become confluent or be in the G0 / G1 phase, causing even normal cells to exhibit morphologies similar to those of dead cells. However, by learning cell images classified as late culture normal state, the cell evaluation system 10 is less likely to mistakenly identify cells in the (late culture) normal state as being in the dead state.

[0056] The initial culture normal state (Figure 4A) refers to a state in which cells remain normal without dying or becoming infected by pathogens when the cell culture period is relatively short. The late culture normal state (Figure 4B) refers to a state in which cells remain normal without dying or becoming infected by pathogens when the cell culture period is relatively long.

[0057] A pre-cell death state (Figure 4C) is a state in which cells are expected to enter a state of cell death if culture is continued for a certain period (e.g., 2-5 days). Before cell death occurs, cells are known to undergo morphological changes such as an increase and / or decrease in cell volume, leakage of intracellular organelles into the extracellular space, and condensation and fragmentation of the nucleus. Using such morphological changes as indicators, the extraction unit 120 may pre-apply the pre-cell death state to cell images. Alternatively, the observer may pre-extract cell images in which cell death is expected and apply the pre-cell death state to them.

[0058] When the cell culture period is relatively long, the cell state may be categorized into five categories: early normal culture, late normal culture, cell death, pre-cell death, and edge state. When the cell culture period is relatively long, additional categories may be added to / instead of these. For example, the cell culture period may be divided into three stages, with a mid-stage normal culture (intermediate culture period) added between the early normal culture (short culture period) and the late normal culture (long culture period), but this is not limited to this. An example of adding categories will be explained in detail in the second modification.

[0059] Next, as shown in S120, if the cell image of the acquired training data includes the edge of the container, the extraction unit 120 extracts only the edge from the cell image of the training data. The extraction unit 120 may determine whether or not the cell image includes the edge of the container based on the brightness of the cell image.

[0060] Figures 5A to 5D show an example of how the extraction unit 120 extracts the container edge from a cell image. As shown in Figure 5A, the cell image contains a mixture of cells (normal cells NC) and the container edge (E). In the cell image, the cells and the container edge have different brightness levels.

[0061] Next, as shown in Figure 5B, the extraction unit 120 receives input from the observer and places several points (points 121, 122, and 123 in the figure) in the region to be considered the edge of the container. The extraction unit 120 places similar points in each cell image that includes the edge of the container. Using these cell images as input, the learning unit 340 learns, for example, a Segment Anything Model (SAM). The SAM model may learn the relationship between cell images (input) and edge regions (output) and extract edge regions from cell images. The learning unit 340 may learn other image segmentation models instead of the SAM model.

[0062] Next, as shown in Figure 5C, the extraction unit 120 generates a mask in the training data that covers everything except the edges of the cell images with edge states (i.e., cell images including the edges of the container) based on the model (SAM model) for evaluating edge regions that the learning unit 340 has trained. By generating a similar mask for each cell image including the edges of the container, the extraction unit 120 can obtain a set of training data that associates the cell images with masks with the edge states, as shown in Figure 5D. After the extraction unit 120 outputs the set of training data including edge states and / or the set of training data without edge states to the preprocessing unit 330, the extraction unit 120 may proceed to processing S130.

[0063] Next, in S130, the preprocessing unit 330 performs preprocessing on the cell images of the training data acquired from the extraction unit 120. The preprocessing may be a process that modifies the cell images. For example, the preprocessing may be a process that enlarges and / or reduces the cell images. For example, the preprocessing may be a process that rotates the cell images at an arbitrary position. As an example, the preprocessing may be a process that rotates the cell images 180 degrees around the intersection of their diagonals (the center of the cell images).

[0064] For example, the preprocessing can be cropping. One example is preprocessing which involves randomly cropping an n'xm' pixel region from an nxm pixel cell image (where n, m, n', and m' are natural numbers satisfying n>n' and m>m'). Another example is preprocessing which involves randomly cropping a 100x100 pixel region from a 1000x1000 pixel cell image.

[0065] For example, the preprocessing may be a process to adjust the brightness of the cell images. By performing preprocessing on the cell images of the training data, the preprocessing unit 330 can expand the training data that the training unit 340 learns. After the preprocessing unit 330 outputs the set of training data that it has preprocessed to the training unit 340, the preprocessing unit 330 may proceed to S140.

[0066] Next, in S140, the learning unit 340 learns an evaluation model based on a set of learning data that includes combinations of cell images preprocessed by the preprocessing unit 330 and pre-assigned cell states. The evaluation model learned by the learning unit 340 may take a cell image as input and output the cell state and / or a classification assignment probability to which the cell belongs.

[0067] The evaluation model that the learning unit 340 learns may be a neural network. For example, the neural network may be a convolutional neural network (CNN). As an example, the convolutional neural network may be a 16-layer VGG-16 model learned on a set of image data, but it is not limited to this.

[0068] Figure 6 shows an example of the layer configuration of a neural network. The neural network 145 may have a fixed layer 146 and a learning layer 147.

[0069] The learning unit 340 may employ transfer learning using a neural network when training the evaluation model. The learning unit 340 may use fixed layers 146 that are pre-trained except for the final layer of the neural network 145 (for example, the VGG-16 model). The learning unit 340 may use the final layer of the neural network 145 as the learning layer 147. In this case, the learning unit 340 performs training only on the learning layer 147. By employing transfer learning, the learning unit 340 can reduce the workload and time required to create the evaluation model. Alternatively, the learning unit 340 may perform training on some or all of the pre-trained fixed layers 146 without creating any new layers.

[0070] The learning unit 340 may further provide an attention layer 148 in addition to the final layer (learning layer 147) of the neural network 145 when learning the evaluation model. For example, the attention layer 148 is a channel attention layer and / or a spatial attention layer. The channel attention layer is a layer that learns the shape pattern of an object (e.g., a cell included in a cell image). The spatial attention layer is a layer that learns the position pattern of an object. The attention layer 148 may be provided immediately after the learning layer 147. When the attention layer 148 provides both a channel attention layer and a spatial attention layer, the channel attention layer may be provided before the spatial attention layer, or the spatial attention layer may be provided before the channel attention layer.

[0071] During the training of the evaluation model, cells at the edges of the cell image may appear incomplete because the cells are cut off at the edges. By adding an attention layer 148 to the neural network 145, the learning unit 340 can focus more on training the central part of the cell image than the edges. This reduces the noise that the learning unit 340 encounters when training the evaluation model. After the learning unit 340 outputs the trained evaluation model to the evaluation unit 400, the learning unit 340 may proceed to processing S160.

[0072] Next, in S160, the cell evaluation system 10, in response to the observer's operation, cultures the cells to be photographed by the imaging unit 200. The cells may be seeded in a transparent container and cultured in a culture medium. The container may be a petri dish, flask, 96-well plate, etc., but is not limited to these.

[0073] The cell culture vessel may be provided in a CO2 incubator with controlled temperature, humidity, and carbon dioxide concentration. The culture medium may be complete medium, basic medium, or buffer. The culture may be carried out according to a known cell culture protocol. The cells may be cultured until the day they are photographed by the imaging unit 200.

[0074] Next, in S200, the cell evaluation system 10 is equipped with an imaging unit 200 that photographs cells in response to an operation by the observer. The imaging unit 200 may photograph cells using an objective lens placed in the optical path of the microscope. The connected microscope may be an inverted microscope or an upright microscope, but is not limited to these.

[0075] Next, in S220, the imaging unit 200 photographs multiple cells placed in a container and acquires images of the cells. For example, the imaging unit 200 photographs the cells using the objective lens of a microscope and generates images. The imaging unit 200 may photograph the cells according to a known cell biology protocol. The imaging unit 200 (or the memory / hard disk connected to it) may further acquire information regarding the imaging conditions of the captured images (magnification, etc.), the cell type of the photographed cells, and / or the number of days the cells have been cultured.

[0076] The captured images may be magnified images of cells at approximately 1000x, 2000x, 3000x, 4000x, 5000x, or 10,000x magnification, depending on the size of the cells. The captured images may be phase-contrast images and / or fluorescence images of cells. The captured images may be grayscale or RGB color. The captured images may be images taken at a specific time during time-lapse photography.

[0077] The captured image may be an image of multiple cells aggregated together, or an image of multiple cells dispersed together. Preferably, the captured image has a resolution clear enough to allow for the identification of organelles inside the cells. In the captured image, the cells may be labeled with a fluorescent dye (e.g., DAPI), express recombinant proteins (e.g., GFP-labeled proteins), or contain infectious agents other than cells (e.g., mycoplasma).

[0078] The captured image may be square, rectangular, or other shapes, but is not limited to these. The captured image may include the rim of the container, the bottom of the container, and / or the culture medium.

[0079] Figure 7 is an example of an image captured by the imaging unit 200. As shown in Figure 7, each grain represents a cell. The imaging unit 200 may output the generated image to the division unit 300 and / or the display unit 500. The imaging unit 200 may output information about the captured cells to the evaluation unit 400 and / or the display unit 500. After that, the imaging unit 200 may proceed to processing S300.

[0080] Next, in S300 (sometimes called the "division stage"), the division unit 300 divides the captured image acquired from the imaging unit 200 to generate multiple cell images. The division unit 300 may generate cell images such that the number of cells included in each cell image is a predetermined number. For example, the division unit 300 divides one captured image into approximately 10 to 30,000 cell images, preferably approximately 100 to 10,000, and more preferably approximately 300 to 5,000. For example, the division unit 300 divides each cell image so that it contains 1 to approximately 50 cells, preferably approximately 5 to 30, and more preferably approximately 10 to 20 cells.

[0081] For example, the division unit 300 may divide an image of approximately 1000 pixels to 2000 pixels x 2000 pixels to 3000 pixels into cell images (small regions) of approximately 50 pixels to 200 pixels x 50 pixels to 200 pixels. Preferably, the division unit 300 divides an image of approximately 1000 pixels to 2000 pixels x 2000 pixels to 3000 pixels into cell images (small regions) of approximately 50 pixels to 150 pixels x 50 pixels to 150 pixels, and more preferably into cell images (small regions) of approximately 80 pixels to 120 pixels x 80 pixels to 120 pixels. As an example, the division unit 300 divides the 1542-pixel x 2080-pixel image acquired from the imaging unit 200 into 100-pixel x 100-pixel cell images (small regions) that contain approximately 10 cells.

[0082] Figures 8A and 8B show an example of how the division unit 300 divides the image captured in Figure 7. The division unit 300 may divide the image in such a way that there is no overlapping region in adjacent cell images (sometimes referred to as the "first mode"), or it may divide the image in such a way that there is an overlapping region (sometimes referred to as the "second mode").

[0083] Figure 8A shows an example where the division unit 300 performs division using the first mode. If the captured image 301 is 1542 pixels × 2080 pixels, the division unit 300 may define small regions as 100 pixels × 100 pixels and divide them into small regions 302, 303, 304, etc., so that the small regions do not overlap. In this case, the division unit 300 divides the captured image 301 into 15 × 20 = 300 small regions, thereby obtaining 300 cell images.

[0084] Figure 8B shows an example of the division unit 300 performing division in the second mode. The division unit 300 provides one cell image (small region) of approximately (50 pixels to approximately 200 pixels) x (50 pixels to approximately 200 pixels) within an captured image of approximately (1000 pixels to approximately 2000 pixels) x (2000 pixels to approximately 3000 pixels). The division unit 300 may sequentially move this small region in any direction by approximately 10 pixels to approximately 100 pixels, preferably approximately 20 pixels to approximately 50 pixels, and more preferably approximately 25 pixels to approximately 40 pixels, thereby dividing one captured image into multiple cell images (small regions).

[0085] If the captured image 306 is 1542 pixels × 2080 pixels, the division unit 300 may define a small region as 100 pixels × 100 pixels and move it 25 pixels in the vertical and horizontal directions to divide it into small regions 307, 308, 309, etc. In this case, the division unit 300 divides the captured image 306 into 58 × 80 = 4640 small regions, thereby obtaining 4640 cell images.

[0086] Upon receiving input from the observer, the division unit 300 may select whether to divide the captured image using the first mode or the second mode. By performing division using the first mode, the cell evaluation system 10 can conserve the amount of data obtained from the cell images and obtain cell images quickly. By performing division using the second mode, the cell evaluation system 10 can obtain more information from the captured images and classify the state of the cells more accurately.

[0087] If preprocessing of the cell images is required, the division unit 300 outputs the divided cell images to the preprocessing unit 330. After the division unit 300 outputs the cell images, the division unit 300 may proceed to processing S330. If preprocessing of the cell images is not required, the division unit 300 outputs the divided cell images directly to the evaluation unit 400. After the division unit 300 outputs the cell images, the division unit 300 may proceed to processing S400.

[0088] Next, in S330, the preprocessing unit 330 performs preprocessing on the cell images output by the division unit 300, if necessary. For cell image preprocessing, the image resizing, rotation, cropping, and / or brightness adjustment described in S130 may be performed as appropriate. After the preprocessing unit 330 outputs the cell images to the evaluation unit 400, the preprocessing unit 330 may proceed to processing in S400.

[0089] Next, in S400 (sometimes referred to as the "evaluation stage"), the evaluation unit 400 inputs the cell images acquired from the division unit 300 and / or the preprocessing unit 330 into the evaluation model acquired from the learning unit 340 to evaluate the state of the cells contained in the cell images. The evaluation unit 400 may input information about the captured cells (such as the number of days of culture) into the evaluation model. S400 may include the subflows S410 to S430 shown below.

[0090] Figure 9 shows the subflows S410 to S430 included in S400. The cell evaluation system 10 may execute S400 by executing the subflows S410 to S430. Some parts of S410 to S430 may be omitted, and / or their order may be changed. The cell evaluation system 10 may perform additional processing in addition to S410 to S430.

[0091] First, in S410, the calculation unit 410 uses an evaluation model to calculate the classification assignment probability, which is the probability that a cell image belongs to each of several cell states. For example, the classification assignment probability is the probability that a cell state belongs to one of several categories: normal state, cell death state, or edge state. The calculation unit 410 may use a softmax function to convert the classification assignment probability into a numerical range of 0 to 1 and normalize it so that the sum of the classification assignment probabilities for each cell state is 1 for a single cell image.

[0092] For example, the calculation unit 410 calculates the probability that each of the subregions obtained in the division stage of S300 belongs to each cell state using an evaluation model. For example, the calculation unit 410 calculates the probability that each of the subregions belongs to one of three categories: normal state, cell death state, and edge state. Alternatively, for example, the calculation unit 410 may calculate the probability that each of the subregions belongs to one of five categories: early culture normal state, late culture normal state, cell death state, cell death precursor state, and edge state.

[0093] Figure 10 is a diagram illustrating the flow of S410. The calculation unit 410 may use the evaluation model obtained from the learning unit 340 to calculate the classification assignment probability for the subregion 411 as (probability of normal initial culture, probability of normal late culture, probability of pre-cell death state, probability of cell death state, probability of edge state) = (0.9, 0, 0, 0, 0.1). In this case, the calculation unit 410 calculates that for the subregion 411, "the probability of normal initial culture is 90%, and the probability of edge state is 10%."

[0094] Next, the calculation unit 410 determines the state of the cells in the subregion. The calculation unit 410 may determine that the state of cells that gives the maximum classification assignment probability is the state of cells in that subregion. For example, the calculation unit 410 determines that the state that gives the maximum classification assignment probability for subregion 411 is the initial culture normal state (90%), and therefore determines that subregion 411 is in the "initial culture normal state". The calculation unit 410 may perform the same procedure to calculate the classification assignment probability and determine the state of cells for subregions 412 and 413.

[0095] Alternatively, the calculation unit 410 may use the calculation result itself as the determination result for the sub-region 411, which is "90% probability of initial culture normal state, 10% probability of cell death state." After the calculation unit 410 outputs the classification assignment probability and the determination result of the cell state for each sub-region to the classification unit 430, the calculation unit 410 may proceed to processing S420.

[0096] Next, in S420, the classification unit 430 classifies the state of cells in the subregions. For example, the classification unit 430 aggregates the classification assignment probability and the cell state determination result for each subregion output from the calculation unit 410. As an example, the classification unit 430 aggregates the cell state determination results for subregions 411, 412, and 413, etc., across the entire captured image 420 from which they are divided. After the classification unit 430 aggregates the cell state determination results, the classification unit 430 may proceed to processing in S430.

[0097] Next, in S430, the classification unit 430 classifies the captured images that will be used to divide the cell images based on the aggregated results. For example, in the above example, the classification unit 430 classifies the state of the cells in the captured images 420 that will be used to divide the subregions 411, 412, 413, etc.

[0098] For example, the classification unit 430 calculates the total probability of a cell death state, the total probability of a cell death precursor state, and the probability of a normal state for each of the subregions 411, 412, 413, etc., based on the determined state of the cells, using the following [Equations 1], [2], and [3]. [Formula 1] Total cell death state probability P CPE ={N CPE / (N all -N edge )}×100(%)。 [Formula 2] Total cell death precursor state probability P preCPE ={N preCPE / (N all -N edge )}×100(%)。 [Formula 3] Total normal state probability P normal =1-(P CPE +P preCPE )(%)。 Here, N all is the number of all small regions of the captured image 420, N CPE is the number of small regions determined to be in the cell death state in the captured image 420, N preCPE is the number of small regions determined to be in the cell death precursor state in the captured image 420, N edge is the number of small regions determined to be in the edge state in the captured image 420, Each represents. Since it is often unclear whether the cells included in the cell image in the edge state are alive or dead, the classification unit 430 subtracts the number of small regions determined to be in the edge state from the number of all small regions of the captured image 420.

[0099] For example, if the classification unit 430 totals that the number of small regions determined as N all is 1000, the number of small regions determined as N CPE is 600, and the number of small regions determined as N edge is 200 in the captured image 420, the cell death state probability P CPE of the captured image is P CPE =600 / (1000 - 200)=0.75 = 75% It can be calculated as such and classified that 75% of the cells included in the captured image are in the 'cell death state'. <00, Alternatively, the classification unit 430 may classify the cells in a container as "dead cells" or "indicative of cell death (will be dead in the future)" if the cells in an area exceeding a predetermined percentage (for example, 50% or more, 60% or more, 70% or more, etc.) of the total area of ​​the container are in a state of cell death or a state of pre-cell death.

[0101] For example, if the imaging unit 200 captures the entire well of a 96-well plate as a single image, and the division unit 300 divides the entire image into 500 cell images (small regions) using the first mode, and the calculation unit 410 determines that 250 or more (more than 50% of the total) of the small regions are in a state of cell death, then the classification unit 430 classifies the cells in that well as being in a state of cell death.

[0102] The classification unit 430 receives the cell image, classification probability, classification result of the cell state, and / or calculated cell death state probability P. CPE After outputting the results to the display unit 500, the classification unit 430 may proceed to processing S500.

[0103] Next, in S500, the display unit 500 may display the processing results of the evaluation unit 400. For example, the display unit 500 may display at least one of the cell image acquired from the imaging unit 200 and / or the evaluation unit 400, the classification probability, and the classification result of the cell state. The display unit 500 may project the cell image. The display unit 500 may display the classification probability and the classification result in text format, or it may display the classification probability and the classification result in tabular format.

[0104] The display unit 500 may display the probability of each region in the captured image belonging to a specific state as contour lines, based on the classification assignment probability for each cell image. The display unit 500 may also display the contour lines superimposed on the captured image.

[0105] Figures 11A and 11B show an example of the display unit 500 displaying classification probability using contour lines. As shown in Figure 11A, the calculation unit 410 or the classification unit 430 provides a mesh grid shape (grid points) corresponding to a small region on the coordinate plane corresponding to the captured image. Each grid point in Figure 11A may be the center of each small region (the intersection of the diagonals, the centroid). Next, the calculation unit 410 or the classification unit 430 places the calculated classification probability result (for example, a normal state probability of 90%) onto each grid point of the mesh grid.

[0106] As shown in Figure 11B, the calculation unit 410 or the classification unit 430 passes the classification probability of each grid point to the Contour function to calculate the classification probability as a contour line. The display unit 500 outputs the obtained contour line. By using a finer mesh grid, the calculation unit 410 or the classification unit 430 can display the contour line more smoothly.

[0107] In this way, by displaying the classification probability as contour lines, the cell evaluation system 10 can provide the observer with a visually easy-to-understand representation of which areas of the captured image contain a high concentration of normal cells and / or cell death.

[0108] The display unit 500 may highlight areas in the captured image where the probability of belonging to a specific state is above a threshold, based on the classification assignment probability for each cell image. For example, the highlighting may be, but is not limited to, coloring, adding contrast, and / or circling.

[0109] Figure 12 shows an example of how the display unit 500 highlights classification assignment probabilities by color. The display unit 500 may display small areas with a cell death probability of 75% or higher in red. In Figure 12, the small area with a cell death probability of 75% is displayed in dark gray. The display unit 500 may use different colors for thresholds such as cell death probability and / or for each cell state.

[0110] In this way, by highlighting the display unit 500 with color, the cell evaluation system 10 can provide the observer with a visually easy-to-understand representation of which areas of the captured image contain a high concentration of normal cells and / or cell death.

[0111] Next, in S600, the cell evaluation system 10 provides the observer with the cell image and / or contour lines displayed on the display unit 500. Based on the observer's confirmation, the observer may decide whether to continue or discontinue cell culture. For example, if the display unit 500 displays that 75% of the cells in the captured image are in a "pre-cell death state" as the state of cells on day 5 of culture, the observer may discontinue cell culture and discard the cells. By displaying a result indicating a "pre-cell death state," the cell evaluation system 10 can provide the observer with a prediction of whether the cells will undergo cell death in the future, relatively early in the culture process.

[0112] Next, a modified version of this embodiment is shown. Multiple modifications shown below can be combined to evaluate the state of cells included in a cell image.

[0113] [First variation] In this embodiment, the classification unit 430 calculated the total normal state probability of the captured image 420 to be divided using [Equation 3]. In the first modified example, instead of using [Equation 3], the learning unit 340 learns an evaluation model using a set of training data to which only normal states have been pre-assigned, and the evaluation unit 400 calculates the total normal state probability using that evaluation model.

[0114] In the first modified example, steps S110 to S130 may be carried out in the same manner as in this embodiment.

[0115] In S140, the learning unit 340 trains an evaluation model using cell images that have been pre-assigned only to the normal state as the training data set. The normal state may include the normal state in the initial culture and the normal state in the later culture. The evaluation model that the learning unit 340 trains is a model that takes cell images that have been assigned only to the normal state as input and outputs the normal state and / or the classification assignment probability to belong to the normal state as the state of the cell.

[0116] The learning unit 340 may employ a diffusion model and / or a GAN (Generative Adversarial Network) as the evaluation model. The learning unit 340 outputs the learned evaluation model to the evaluation unit 400. Steps S160 to S330 may be carried out in the same manner as in this embodiment.

[0117] In S410, the calculation unit 410 calculates the probability that the cell image belongs to a normal state using the evaluation model acquired above. S410 and S420 may be carried out in the same manner as in this embodiment.

[0118] In S430, the classification unit 430 calculates the total normal state probability P of the captured image 420 that will be divided. normal This may be calculated using the following equations [Equation 4], [Equation 5], and [Equation 6]. [Formula 4] Total initial culture normality probability P YoungNormal ={N YoungNormal / (N all -N edge )} × 100 (%). [Formula 5] Total late-stage culture normality probability P OldNormal ={N OldNormal / (N all -N edge )} × 100 (%). [Formula 6] Total probability of normal state P normal =P YoungNormal +P OldNormal (%). Here, N YoungNormal This refers to the number of small regions in the captured image 420 that were determined to be in a normal state in the initial culture, N OldNormal This refers to the number of small regions in the captured image 420 that were determined to be in a normal state in the late culture stage. These represent the respective values. Steps S500 and S600 may be carried out in the same manner as in this embodiment.

[0119] In the first modification, by employing an evaluation model that assesses the normal state of cells, the cell evaluation system 10 can more accurately assess the normal state of cells. This further reduces the risk that the evaluation unit 400 may mistakenly identify normal cells as being in a pre-death state or in a state of cell death.

[0120] [Second variation] In this embodiment, the cell evaluation system 10 evaluated the state of cells by classifying them into three and / or five categories. In a second modification, the cell evaluation system 10 inputs cell images into multiple different evaluation models for evaluating cells infected with multiple different infectious agents, and evaluates the state of cells contained in the cell images for each infectious agent. In other words, in the second modification, the cell evaluation system 10 can evaluate what infectious agent a cell has infected. In this case, the state of the cell may include the state of being infected with an infectious agent.

[0121] In S110, the extraction unit 120 acquires training data to be used for training the evaluation model. The training data may include multiple sets of pairs of (cell image, type of infected organism that infected the cells contained in the cell image, and whether or not the cells were infected). For example, the training data may be (cell image, infected with infected organism A), (cell image, not infected with infected organism A), (cell image, infected with infected organism B), etc. S120 and S130 may be carried out in the same manner as in this embodiment.

[0122] In S140, the learning unit 340 may learn different evaluation models for each set of training data based on the type of infected organism and whether or not infection has occurred. An evaluation model is a model that takes a cell image as input and outputs whether or not the cell is infected with an infected organism as the state of the cell. For example, the learning unit 340 learns evaluation model P using the set of training data (cell image, infected organism A infected). For example, the learning unit 340 learns evaluation model Q using the set of training data (cell image, infected organism A not infected). For example, the learning unit 340 learns evaluation model R using the set of training data (cell image, infected organism B infected). The learning unit 340 may output the learned evaluation models P, Q, and R to the evaluation unit 400. The steps from S160 to S330 may be carried out in the same manner as in this embodiment.

[0123] Figure 13 shows the concept of S400 in a second modified example. In S400, the evaluation unit 400 inputs the cell images acquired from the division unit 300 or the preprocessing unit 330 into the acquired evaluation models P, Q, R, etc., to determine which infectious agent the cells are infected with. For example, in S410, the calculation unit 410 uses the evaluation models P, Q, R, etc., acquired from the learning unit 340 to calculate the classification assignment probability for a small region as (infected with infectious agent A, not infected with infectious agent A, infected with infectious agent B...) = (0.70, 0, 0,...) using the same processing as in this embodiment. In this case, the calculation unit 410 may calculate that the probability of the small region being infected with infectious agent A is 70%.

[0124] Next, the calculation unit 410 may determine that the cell state that gives the maximum classification assignment probability is the cell state of the sub-region. For example, the calculation unit 410 determines that the state that gives the maximum classification assignment probability for the sub-region is "infected with infectious agent A" (70%), and therefore determines that the sub-region is in a "state infected with infectious agent A". The steps from S420 onward may proceed in the same manner as in this embodiment.

[0125] Cells can infect not only a single infected organism but also multiple infected organisms. In this case, the calculation unit 410 may use the classification probability of the small region itself as the determination result. For example, the calculation unit 410 may enumerate some or all of the infected organisms that show a classification probability above a predetermined threshold for each infected organism, and output these as classification probabilities to the display unit 500. In this case, the display unit 500 may display "Probability of infection with infected organism A: 70%, Probability of infection with infected organism C: 25%" as the processing result of the cell evaluation system 10.

[0126] In the second modification, the cell evaluation system 10 can evaluate what kind of infectious agent the cells are infected with (or not infected with) and whether the cells are contaminated with bacteria (or not).

[0127] Some viruses, instead of altering cell morphology or lifespan upon infection, cause abnormal cell proliferation (for example, cancer viruses). In such cases, the cell evaluation system 10 can evaluate the probability of infection by the infected organism and the state of the cells using abnormal cell proliferation as an indicator, following the same procedure as described above. In this case, the state of the cells may include a state in which cells are abnormally proliferating.

[0128] [Third variation] In the second modification described above, the cell evaluation system 10 evaluated what type of infectious agent the cells were infected with. In the third modification, the cell evaluation system 10 measures the titer (TCID50 value) of the virus solution.

[0129] In the third modified example, the steps in S100 may be carried out in the same manner as in this embodiment.

[0130] In S160, the cell evaluation system 10, in response to the observer's operation, cultures the cells to be photographed by the imaging unit 200 in a 96-well plate. Next, the observer prepares a series of diluted virus solutions, and the cells are infected with each diluted virus solution. For example, the concentration of the diluted virus solution is 10 4 Dilute twice, 10 5 Dilute twice, 106 Dilute twice, 10 7 Dilute twice, 10 8 Dilute twice, 10 9 Dilute twice, 10 10 Dilution twice and 10 11 There are eight series of 2:1 dilutions. In this case, for each dilution series, 10 wells of cells may be infected with the diluted virus solution. Steps S200 to S400 may be carried out in the same manner as in this embodiment or the second modified example.

[0131] In S500, the display unit 500 displays a cell image and the classification result of the cell state (whether it is in a state of cell death or infected) for each well of the 96-well plate. In the example above, since 10 wells of cells are infected in each dilution series, the concentration of that dilution series becomes the TCID50 value of the virus solution when exactly 5 wells of cells are in a state of cell death or infected.

[0132] If the culture period is relatively short and no wells are classified as being in a state of cell death, then if 10 wells of cells are infected in each dilution series and exactly 5 wells of cells are in a state of pre-cell death, the concentration of that dilution series can be estimated to be the TCID50 value of the virus solution. This allows the cell evaluation system 10 to estimate the TCID50 value of the virus solution in a short time when it is necessary to shorten the experimental time. In the example above, cells are shown being cultured in a 96-well plate, but the culture vessel can be selected arbitrarily.

[0133] If it is necessary to calculate the TCID50 value in more detail, the known Spearman-Carver method using a Poisson distribution may be used. The calculation of the TCID50 value using the Spearman-Carver method may be performed by a computer other than the cell evaluation system 10, or by the evaluation unit 400. When the evaluation unit 400 performs the calculation of the TCID50 value using the Spearman-Carver method, the evaluation unit 400 may output the result of the calculation of the TCID50 value to the display unit 500, and the display unit 500 may display the TCID50 value. In this third modification, the cell evaluation system 10 can measure or estimate the TCID50 value of the virus solution with less workload and working time.

[0134] [Fourth variation] In this embodiment, the learning unit 340 learned an evaluation model regardless of the number of days the cells were cultured. In the fourth modified example, the learning unit 340 learns different evaluation models according to the number of days the cells were cultured, and the evaluation unit 400 evaluates the state of the cells using the evaluation model corresponding to the number of days the cells were cultured, as contained in the cell image.

[0135] In the fourth modified example, steps S100 to S130 may be carried out in the same manner as in this embodiment.

[0136] In S140, the learning unit 340 may learn different evaluation models depending on the number of days the cells have been cultured. The evaluation model may be a model that takes cell images as input and outputs an initial normal culture state and / or a late normal culture state as the state of the cells. For example, the learning unit 340 learns evaluation model S using a set of learning data (cell image, initial normal culture state) of cells photographed before the 5th day of culture. For example, the learning unit 340 learns evaluation model T using a set of learning data (cell image, late normal culture state) of cells photographed from the 6th day of culture onwards. The learning unit 340 may output the learned evaluation model S and evaluation model T, etc., to the evaluation unit 400. The steps from S160 to S330 may be carried out in the same manner as in this embodiment.

[0137] Figure 14 shows the concept of S400 in the fourth modified example. In S400, the evaluation unit 400 inputs cell images from the division unit 300 or pre-processing unit 330 that were obtained before the 5th day of culture to the acquired evaluation model S. Similarly, the evaluation unit 400 inputs cell images from the division unit 300 or pre-processing unit 330 that were obtained after the 6th day of culture to the acquired evaluation model T. Except for adopting different evaluation models depending on the number of culture days, the evaluation unit 400 may proceed with processing from S400 onward in the same manner as in this embodiment.

[0138] By employing different evaluation models depending on the number of days the cells have been cultured, the cell evaluation system 10 can more accurately determine the state of the cells. When the culture period is relatively long, cells may become confluent or be in the G0 / G1 phase, and even normal cells may exhibit morphologies similar to those of cell death. However, by employing different evaluation models depending on the number of days the cells have been cultured, the cell evaluation system 10 is less likely to mistakenly identify cells in a (late culture) normal state as being in a cell death state. In the example above, the cell evaluation system 10 set the boundary between the initial culture normal state and the late culture normal state at 5-6 days after the start of culture, but the number of days for the boundary can be set arbitrarily.

[0139] [Fifth variation] In this embodiment, the learning unit 340 acquires pre-prepared training data and trains an evaluation model. In the fifth modified example, the learning unit 340 acquires further training data from captured images or cell images of cells evaluated by the evaluation unit 400 and trains an evaluation model.

[0140] For example, the learning unit 340 acquires a cell image that the calculation unit 410 has determined to be in a "dead cell state" as new training data and trains an evaluation model. The evaluation model takes a cell image in a dead cell state as input and outputs a dead cell state and / or a pre-death state as the cell state. In this case, the cell image may be a small cell region image obtained by dividing an image captured by the imaging unit 200 of cultured cells over time (for example, every other day in time lapse).

[0141] For example, if cell image K, included in an image of cells cultured for 10 days, and cell image L, included in an image of cells cultured for 5 days, both contain the same small region, and the calculation unit 410 determines that cell image K is in a "dead cell state," the learning unit 340 considers cell image K as a "dead cell state" and / or cell image L as a "precursor to cell death state," adds these to the training data, and trains the evaluation model.

[0142] For example, if the calculation unit 410 determines that cell image K, which is included in an image of cells that have been cultured for 10 days, is in a "dead cell state," the learning unit 340 considers cell image M (small region M), which is adjacent to cell image K (small region K), as being in a "dead cell state" or a "precursor to cell death state," and adds these to the training data to train the evaluation model.

[0143] In the fifth modification, the cell evaluation system 10 can evaluate the state of cells more accurately by adding cell images evaluated by the evaluation unit 400 to the training data and training the evaluation model in the learning unit 340. In the fifth modification, for small regions judged to be in a state of cell death, the cell evaluation system 10 can classify the state of cell death more accurately by adding cell images containing the same small region from a day before the judgment of cell death to the training data as a premonitory state of cell death.

[0144] Various embodiments of the present invention may be described with reference to flowcharts and block diagrams, where a block may represent (1) a stage in a process in which an operation is performed or (2) a section of a device having the role of performing the operation. Specific stages and sections may be implemented by dedicated circuits, programmable circuits supplied with computer-readable instructions stored on a computer-readable medium, and / or processors supplied with computer-readable instructions stored on a computer-readable medium. Dedicated circuits may include digital and / or analog hardware circuits, and may include integrated circuits (ICs) and / or discrete circuits. Programmable circuits may include reconfigurable hardware circuits, including logical AND, logical OR, logical XOR, logical NAND, logical NOR, and other logic operations, flip-flops, registers, memory elements such as field-programmable gate arrays (FPGAs), programmable logic arrays (PLAs), etc.

[0145] Computer-readable media may include any tangible device capable of storing instructions to be executed by a suitable device, and as a result, computer-readable media having instructions stored therein will comprise a product containing instructions that can be executed to create means for performing operations specified in a flowchart or block diagram. Examples of computer-readable media may include electronic storage media, magnetic storage media, optical storage media, electromagnetic storage media, semiconductor storage media, etc. More specific examples of computer-readable media may include floppy disks, diskettes, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), electrically erasable programmable read-only memory (EEPROM), static random access memory (SRAM), compact disk read-only memory (CD-ROM), digital multipurpose disc (DVD), Blu-ray® disc, memory stick, integrated circuit card, etc.

[0146] Computer-readable instructions may include assembler instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Smalltalk®, Java®, C++, and traditional procedural programming languages ​​such as the C programming language or similar programming languages.

[0147] Computer-readable instructions may be provided locally or via a wide area network (WAN) such as a local area network (LAN) or the internet to the processor or programmable circuit of a programmable data processing device such as a general-purpose computer, a special-purpose computer, or another computer, and the computer-readable instructions may be executed to create means for performing operations specified in a flowchart or block diagram. Examples of processors include computer processors, processing units, microprocessors, digital signal processors, controllers, microcontrollers, etc.

[0148] Figure 15 shows an example of a computer 2200 in which multiple aspects of the present invention may be embodied in whole or in part. A program installed on the computer 2200 can cause the computer 2200 to function as an operation or one or more sections of the cell evaluation system 10 according to an embodiment of the present invention, or to execute such operation or one or more sections, and / or to cause the computer 2200 to execute a process or a stage of such process according to an embodiment of the present invention. Such a program may be executed by the CPU 2212 to cause the computer 2200 to perform a particular operation associated with some or all of the blocks of the flowcharts and block diagrams described herein.

[0149] The computer 2200 according to this embodiment includes a CPU 2212, RAM 2214, a graphics controller 2216, and a display device 2218, which are interconnected by a host controller 2210. The computer 2200 also includes input / output units such as a communication interface 2222, a hard disk drive 2224, a DVD-ROM drive 2226, and an IC card drive, which are connected to the host controller 2210 via an input / output controller 2220. The computer also includes legacy input / output units such as a ROM 2230 and a keyboard 2242, which are connected to the input / output controller 2220 via an input / output chip 2240.

[0150] The CPU 2212 operates according to programs stored in the ROM 2230 and RAM 2214, thereby controlling each unit. The graphics controller 2216 acquires image data generated by the CPU 2212 from the frame buffer provided in RAM 2214 or from itself, and displays the image data on the display device 2218.

[0151] The communication interface 2222 communicates with other electronic devices via a network. The hard disk drive 2224 stores programs and data used by the CPU 2212 in the computer 2200. The DVD-ROM drive 2226 reads programs or data from the DVD-ROM 2201 and provides them to the hard disk drive 2224 via the RAM 2214. The IC card drive reads programs and data from the IC card and / or writes programs and data to the IC card.

[0152] The ROM 2230 stores boot programs and / or hardware-dependent programs of the computer 2200, which are executed by the computer 2200 upon activation. The input / output chip 2240 may also connect various input / output units to the input / output controller 2220 via parallel ports, serial ports, keyboard ports, mouse ports, etc.

[0153] The program is provided on a computer-readable medium such as a DVD-ROM 2201 or an IC card. The program is read from the computer-readable medium and installed on a hard disk drive 2224, RAM 2214, or ROM 2230, which are examples of computer-readable mediums, and executed by the CPU 2212. The information processing described within these programs is read by the computer 2200, resulting in coordination between the program and the various types of hardware resources described above. The apparatus or method may be configured to realize the manipulation or processing of information in accordance with the use of the computer 2200.

[0154] For example, when communication is performed between a computer 2200 and an external device, the CPU 2212 may execute a communication program loaded into RAM 2214 and, based on the processing described in the communication program, instruct the communication interface 2222 to perform communication processing. Under the control of the CPU 2212, the communication interface 2222 reads transmission data stored in a transmission buffer processing area provided in a recording medium such as RAM 2214, a hard disk drive 2224, a DVD-ROM 2201, or an IC card, transmits the read transmission data to the network, or writes received data received from the network to a reception buffer processing area provided on the recording medium.

[0155] The CPU 2212 reads all or necessary parts of a file or database stored on an external storage medium such as a hard disk drive 2224, a DVD-ROM drive 2226 (DVD-ROM 2201), or an IC card into the RAM 2214, and may perform various types of processing on the data in the RAM 2214. The CPU 2212 then writes the processed data back to the external storage medium.

[0156] Various types of information, such as various types of programs, data, tables, and databases, may be stored on the recording medium and subjected to information processing. The CPU 2212 may perform various types of processing on the data read from RAM 2214, including various types of operations, information processing, conditional judgments, conditional branching, unconditional branching, information retrieval / replacement, etc., as described throughout this disclosure and specified by the program instruction sequence, and write the results back to RAM 2214. The CPU 2212 may also retrieve information in files, databases, etc., within the recording medium. For example, if multiple entries are stored in the recording medium, each having an attribute value of a first attribute associated with an attribute value of a second attribute, the CPU 2212 may search among the multiple entries for an entry that matches the condition for which the attribute value of the first attribute is specified, read the attribute value of the second attribute stored in that entry, and thereby obtain the attribute value of the second attribute associated with the first attribute that satisfies a predetermined condition.

[0157] The programs or software modules described above may be stored on or near computer 2200 on a computer-readable medium. Alternatively, recording media such as hard disks or RAM provided within a server system connected to a dedicated communication network or the Internet can be used as computer-readable media, thereby providing programs to computer 2200 via the network.

[0158] Although the present invention has been described above using embodiments, the technical scope of the present invention is not limited to the scope described in the above embodiments. It will be apparent to those skilled in the art that various modifications or improvements can be made to the above embodiments. It will be clear from the claims that such modified or improved forms may also be included in the technical scope of the present invention.

[0159] It should be noted that the execution order of operations, procedures, steps, and stages in the devices, systems, programs, and methods shown in the claims, specifications, and drawings is not explicitly stated as "before," "prior to," etc., and can be implemented in any order unless the output of a previous process is used in a later process. Even if the operation flow in the claims, specifications, and drawings is described using phrases such as "first," "next," etc. for convenience, it does not mean that it is essential to perform them in that order. The notation "A and / or B" may mean "A, B, or A and C." The notation "A, B and / or C" may mean "any one of A, B, and C, or any combination of two or more of these." [Explanation of Symbols]

[0160] 10 Cell evaluation system 120 Extraction part 121 points 122 points 123 points 145 Neural Networks 146 Fixed layer 147 training layers 148 Attention Layer 200 Photography Department 300 division part 301 Photographed image 302 small area 303 small area 304 small area 306 Photographed Images 307 small area 308 small area 309 small area 330 Pre-processing section 340 Learning Department 400 Evaluation Department 410 Calculation Unit 411 Small area 412 small area 413 small area 420 captured images 430 Classification Department 500 Display 2200 Computers 2201 DVD-ROM 2210 Host Controller 2212 CPU 2214 RAM 2216 Graphics Controller 2218 Display Devices 2220 Input / Output Controller 2222 Communication Interface 2224 Hard Disk Drive 2226 DVD-ROM drive 2230 ROM 2240 Input / Output Chip 2242 keyboard

Claims

1. An evaluation unit inputs a cell image including multiple cells and a container in which the cells are arranged into an evaluation model and evaluates the state of the cells included in the cell image. A division unit that divides an image captured of multiple cells arranged in the container to generate multiple cell images, Equipped with, Cell evaluation system.

2. The system further includes a shooting unit that acquires the aforementioned captured image. The cell evaluation system according to claim 1.

3. The division unit generates the cell image such that the number of cells included in the cell image is a predetermined number. The cell evaluation system according to claim 1.

4. The aforementioned divided portion is A first mode in which the captured image is divided so that no overlapping regions occur in adjacent cell images, or In a second mode in which the captured image is divided such that a partial overlapping region occurs in adjacent cell images, The aforementioned captured image is divided into sections. The cell evaluation system according to claim 1.

5. The system further includes a learning unit that learns the evaluation model based on learning data that includes multiple combinations of the cell image and the pre-assigned state of the cell. The cell evaluation system according to claim 1.

6. The learning unit learns a neural network as the evaluation model. The cell evaluation system according to claim 5.

7. The aforementioned neural network includes a pre-trained fixed layer and a learning layer to be trained. The learning unit performs learning only on the learning layer. The cell evaluation system according to claim 6.

8. The learning unit learns different evaluation models depending on the number of days the cells have been cultured. The evaluation unit evaluates the state of the cells using the evaluation model corresponding to the number of days the cells have been cultured, as contained in the cell image. The cell evaluation system according to claim 5.

9. If the cell image in the training data includes the edge of the container, the system includes an extraction unit that extracts only the edge from the cell image in the training data. The cell evaluation system according to claim 5.

10. The evaluation unit classifies the cell image into multiple states. The cell evaluation system according to claim 1.

11. The aforementioned multiple states include a normal state, a cell death state, and a cell death precursor state. The cell evaluation system according to claim 10.

12. The aforementioned normal state includes the initial culture normal state and the late culture normal state. The cell evaluation system according to claim 11.

13. The aforementioned plurality of states include an edge state in which the cell image includes the edge of the container, The cell evaluation system according to claim 10.

14. The evaluation unit, A calculation unit that calculates a classification assignment probability, which is the probability that the cell image belongs to each of the multiple states, A classification unit that classifies the captured images that are the source of the cell image division based on the classification probability for each cell image, Having, The cell evaluation system according to claim 10.

15. The evaluation unit inputs the cell images into a plurality of different evaluation models for evaluating the cells infected with a plurality of different infectious agents. For each of the infected organisms, the state of the cells included in the cell image is evaluated. The cell evaluation system according to claim 1.

16. The system further includes a display unit that, based on the classification assignment probability for each cell image, represents the probability of each region in the captured image belonging to a specific state as contour lines and displays them superimposed on the captured image. The cell evaluation system according to claim 14.

17. The system further includes a display unit that highlights areas in the captured image where the probability of belonging to a specific state is above a threshold, based on the classification assignment probability for each cell image. The cell evaluation system according to claim 14.

18. It is executed by a computer, and the computer, To function as a cell evaluation system according to any one of claims 1 to 17, program.

19. An evaluation step involves inputting a cell image, including multiple cells and a container in which the cells are arranged, into an evaluation model to evaluate the state of the cells contained in the cell image. A division step involves dividing an image of multiple cells arranged in the container to generate multiple cell images, Equipped with, Cell evaluation methods.

Images