Image-based system for monitoring cell quality in culture

Abstract

Described herein are techniques for adjusting the treatment of a culture of cells having cells corresponding to different cell categories. Some techniques may be used in conjunction with a cell imaging and incubation system including an imaging sensor configured to acquire images of the culture and an incubator configured to incubate the culture. Adjusting the treatment of the culture may be based on processing an image of the culture acquired by the imaging sensor of the system. The processing may include segmenting the image of the culture into multiple image segments by assigning individual pixels of the image to corresponding cell categories. According to some embodiments, the techniques include identifying an amount of the culture corresponding to a particular cell category based on the image segments. The amount of the culture corresponding to a particular cell category may inform adjustment of the treatment of the culture.

Description

[Technical Field]

[0001] Related Applications This application claims priority under 35 U.S.C. § 119(e) to U.S. Provisional Patent Application No. 63 / 392,422, entitled "IMAGING-BASED SYSTEM FOR MONITORING QUALITY OF CELLS IN CULTURE," filed July 26, 2022, the entire contents of which are incorporated herein by reference. [Background technology]

[0002] Cell culture is the growth of cells in a controlled environment that can be used for many purposes, such as studying the effects of drugs, modeling disease, and studying genetic mutations. Cell culture management involves making decisions about how to treat the culture to produce high-quality cells. Summary of the Invention [Means for solving the problem]

[0003] Some embodiments provide a method for adjusting a treatment of a culture, the culture comprising a plurality of cells, the cells in the plurality of cells having one or more respective cell categories selected from a plurality of cell categories, the plurality of cell categories including a first cell category and a second cell category, the method comprising processing an image of the plurality of cells of the culture, thereby identifying one or more cell categories of cells shown in the image from among the plurality of cell categories, the processing comprising segmenting the image into a plurality of image segments by assigning individual pixels in the image to a corresponding cell category among the plurality of cell categories, the assigning comprising determining, for each individual pixel, a respective plurality of values corresponding to a respective one of the plurality of cell categories, each of the plurality of values indicating a likelihood that the pixel corresponds to a cell of a respective cell category among the plurality of cell categories, the plurality of image segments including a first image segment including pixels associated with cells of the first cell category and a second image segment including pixels associated with cells of the second cell category; and identifying a quantity of cells in the culture corresponding to the first cell category based on the plurality of image segments into which the image is segmented; and adjusting a treatment of the culture based on the quantity.

[0004] Some embodiments provide at least one non-transitory computer-readable storage medium encoding executable instructions that, when executed by at least one processor, cause the at least one processor to perform a method of regulating processing of a culture, the culture comprising a plurality of cells, the cells in the plurality of cells having one or more respective cell categories selected from a plurality of cell categories, the plurality of cell categories including a first cell category and a second cell category, the method comprising processing an image of the plurality of cells of the culture, whereby identifying one or more cell categories of cells shown in the image from among the plurality of cell categories, and processing includes assigning individual pixels in the image to a corresponding cell category from the plurality of cell categories. and processing the culture includes segmenting the image into a plurality of image segments, wherein assigning includes determining, for each individual pixel, a respective plurality of values corresponding to a respective plurality of cell categories, each of the plurality of values indicating a likelihood that the pixel corresponds to a cell of a respective cell category among the plurality of cell categories, the plurality of image segments including a first image segment including pixels associated with cells of the first cell category and a second image segment including pixels associated with cells of the second cell category; and identifying an amount of cells in the culture corresponding to the first cell category based on the plurality of image segments into which the image is segmented; and adjusting a treatment of the culture based on the amount.

[0005] In some embodiments, assigning individual pixels in the image to corresponding cell categories of the plurality of cell categories comprises classifying the individual pixels according to a plurality of classes, a first class of the plurality of classes corresponding to the first cell category and a second class of the plurality of classes corresponding to the second cell category, and classifying the individual pixels comprises selecting, for each of the individual pixels, a class to classify the individual pixel into based on the determined respective plurality of values.

[0006] In some embodiments, the assigning is performed using a trained machine learning model, and the assigning includes processing the image using the trained machine learning model to obtain, for each individual pixel, a plurality of respective values corresponding to a plurality of respective categories.

[0007] In some embodiments, the trained machine learning model comprises a deep neural network model that includes one or more convolutional layers.

[0008] In some embodiments, the deep neural network model includes a cascade of deep neural network blocks, each of the deep neural network blocks including a respective deep convolutional neural network (CNN), and the trained machine learning model performs computations at least in part using atlas spatial pyramid pooling.

[0009] In some embodiments, the deep neural network model includes a U-net architecture.

[0010] In some embodiments, the deep neural network model includes at least 1 million, at least 5 million, at least 10 million, at least 50 million, at least 100 million, at least 500 million, or at least 1 billion parameters whose values are used as part of processing the image using the deep neural network model.

[0011] Some embodiments further include processing the image of the plurality of cells in culture to estimate culture information about the culture, the culture information about the culture indicating a density of cells in the culture of the plurality of cells, and segmenting the image includes segmenting the image based on the culture information.

[0012] In some embodiments, the culture information is used to identify cellular coordinates of a plurality of cells, and segmenting the image based on the culture information includes providing the image and the coordinates as inputs to a trained machine learning model to obtain an output indicating the respective likelihood that each individual pixel corresponds to one of a plurality of cell categories.

[0013] Some embodiments further include processing the image of the plurality of cells of the culture to estimate culture information about the culture, the culture information about the culture indicating a density of cells in the culture of the plurality of cells, and adjusting the treatment of the culture includes adjusting the treatment of the culture based on the culture information and the amount of the culture corresponding to the first cell category.

[0014] In some embodiments, the culture information is used to identify coordinates of cells shown in the image, and adjusting the processing of the culture based on the culture information includes using the coordinates to identify a location of one or more cells of the plurality of cells and removing the cells from the identified location.

[0015] In some embodiments, processing images of the plurality of cells of the culture to estimate culture information includes estimating the number of the plurality of cells, the position of at least one cell of the plurality of cells, and / or the internuclear distance between at least two cells of the plurality of cells.

[0016] In some embodiments, adjusting the treatment of the culture based on the culture information and the amount of cells in the culture corresponding to the first cell category includes outputting a recommendation indicating a time to passage cells of the plurality of cells in the culture and / or a recommended number of new cultures to split the culture into.

[0017] In some embodiments, adjusting the treatment of the culture includes outputting a recommendation to modify how one or more substances are added to the culture to affect growth of the culture based on the amount of cells in the culture corresponding to the first cell category.

[0018] In some embodiments, adjusting the treatment of the culture comprises altering the way one or more substances are added to the culture to affect the growth of the culture.

[0019] In some embodiments, adjusting the treatment of the culture includes outputting a recommendation to passage cells of the plurality of cells of the culture based on the amount of cells in the culture corresponding to the first cell category.

[0020] In some embodiments, modulating the treatment of the culture comprises passaging cells of the plurality of cells in culture.

[0021] In some embodiments, adjusting the treatment of the culture includes outputting a recommendation to discard cells of the plurality of cells of the culture.

[0022] In some embodiments, adjusting the treatment of the culture comprises discarding cells of the plurality of cells of the culture.

[0023] Some embodiments further include comparing the amount of cells in the culture corresponding to the first cell category to a predetermined amount, and adjusting a treatment of the second culture based on the comparison to culture the second culture to have the predetermined amount of the first cell category.

[0024] In some embodiments, the image of the plurality of cells in culture comprises a bright field image.

[0025] Some embodiments further include acquiring images of the plurality of cells in culture with an imaging sensor of the cell imaging and incubation system.

[0026] In some embodiments, the first cell category corresponds to induced pluripotent stem cells (iPSCs) and the second cell category corresponds to non-iPSCs.

[0027] In some embodiments, the plurality of cell categories includes a third cell category, the third cell category corresponding to background, and the plurality of image segments further includes a third image segment that includes pixels associated with cells of the third cell category.

[0028] Some embodiments provide a cell imaging and incubation system, the cell imaging and incubation system including: an imaging sensor configured to acquire images of a plurality of cells in culture; an incubator configured to incubate the culture; at least one processor; and at least one non-transitory computer-readable storage medium encoding executable instructions that, when executed by the at least one processor, cause the at least one processor to perform a method of regulating processing of a culture comprising a plurality of cells, wherein cells in the plurality of cells have one or more respective cell categories selected from a plurality of cell categories, the plurality of cell categories including a first cell category and a second cell category, the method comprising processing images of the plurality of cells in culture, thereby identifying one or more cell categories of cells shown in the images from among the plurality of cell categories. The identifying and processing includes segmenting the image into a plurality of image segments by assigning individual pixels in the image to corresponding cell categories among a plurality of cell categories, wherein the assigning includes determining, for each individual pixel, a respective plurality of values corresponding to a respective one of the plurality of cell categories, each of the plurality of values indicating a likelihood that the pixel corresponds to a cell of a respective one of the plurality of cell categories, wherein the plurality of image segments includes a first image segment including pixels associated with cells of the first cell category and a second image segment including pixels associated with cells of the second cell category; and the processing includes identifying an amount of cells in the culture corresponding to the first cell category based on the plurality of image segments into which the image is segmented, and adjusting a treatment of the culture based on the amount.

[0029] Some embodiments further include a robotic system configured to transfer cultures in the cell imaging and incubation system between being cultured in the incubator and being imaged by the imaging sensor.

[0030] In some embodiments, the robotic system is configured to transfer the culture between incubating and imaging when a timing condition is met.

[0031] In some embodiments, the method further includes, when a timing condition is met, operating the robotic system to move the culture to an imaging sensor, and operating the imaging sensor to acquire images of a plurality of cells in the culture.

[0032] Some embodiments further comprise an imaging device, the imaging device comprising an imaging sensor and a chamber configured to receive a well plate, the well plate configured to hold a culture.

[0033] In some embodiments, the method further comprises processing an image of the plurality of cells in culture to estimate culture information of the culture, the culture information of the culture indicating a density of cells in the culture of the plurality of cells, and segmenting the image comprises segmenting the image based on the culture information.

[0034] In some embodiments, the method further includes processing the image of the plurality of cells in the culture to estimate culture information of the culture, the culture information of the culture indicating a density of cells in the culture of the plurality of cells, and adjusting a treatment of the culture includes adjusting a treatment of the culture based on the culture information and the amount of cells in the culture corresponding to the first cell category.

[0035] Some embodiments further include using the culture information to estimate a number of the plurality of cells, a position of at least one cell of the plurality of cells, and / or an internuclear distance between at least two cells of the plurality of cells.

[0036] In some embodiments, adjusting the treatment of the culture includes outputting a recommendation to modify how one or more substances are added to the culture to affect growth of the culture based on the amount of cells in the culture corresponding to the first cell category.

[0037] In some embodiments, adjusting the treatment of the culture includes outputting a recommendation to passage cells of the plurality of cells of the culture based on the amount of cells in the culture corresponding to the first cell category.

[0038] Some embodiments provide a method for adjusting treatment of a culture, the culture comprising a plurality of cells, the cells in the plurality of cells having one or more respective cell categories selected from a plurality of cell categories, the plurality of cell categories including a first cell category and a second cell category, the method comprising processing an image of the plurality of cells of the culture, thereby identifying one or more cell categories of the cells shown in the image from among the plurality of cell categories, the processing comprising segmenting the image into a plurality of image segments by assigning regions of the image to a corresponding cell category among the plurality of cell categories, each of the regions comprising two or more individual pixels in the image, the assigning comprising determining, for each of the regions, a respective plurality of values corresponding to a respective one of the plurality of cell categories, each of the plurality of values indicating a likelihood that the region corresponds to a cell of a respective cell category among the plurality of cell categories, the plurality of image segments including a first image segment including regions associated with cells of the first cell category and a second image segment including regions associated with cells of the second cell category; identifying a quantity of cells in the culture corresponding to the first cell category based on the plurality of image segments into which the image is segmented; and adjusting treatment of the culture based on the quantity.

[0039] The accompanying drawings are not intended to be drawn to scale. In the drawings, each identical or nearly identical component shown in various figures is represented by a like numeral. For clarity, it is not possible to label every component in every drawing. The drawings are as follows: [Brief explanation of the drawings]

[0040] [Figure 1A] FIG. 1 illustrates an exemplary technique 100 for regulating the treatment of a cell culture, according to some embodiments of the techniques described herein. [Figure 1B] FIG. 1 is a block diagram of an exemplary system 150 for regulating the processing of cell cultures, according to some embodiments of the technology described herein. [Figure 2A] 2 is a flowchart of an exemplary process 200 for regulating the treatment of a cell culture, according to some embodiments of the technology described herein. [Figure 2B] 2 is a flowchart of an example process 250 for segmenting an image into multiple image segments, according to some embodiments of the techniques described herein. [Figure 2C] 10A-10C illustrate examples of adjusting the treatment of a culture based on identified image segments, according to some embodiments of the technology described herein. [Figure 3A] 1 shows an exemplary segmented image of a cell culture, according to some embodiments of the techniques described herein. [Figure 3B] 3B shows an example of a segmented image of one cell category of the cell culture shown in FIG. 3A according to some embodiments of the technology described herein. [Figure 4A] 1 shows an example of a bright field image of a cell culture, according to some embodiments of the technology described herein. [Figure 4B] 4B shows an example of a density map generated for the cell culture shown in FIG. 4A according to some embodiments of the techniques described herein. [Figure 5A]1 illustrates an example of a cell imaging and incubation system according to some embodiments of the technology described herein. [Figure 5B] 5 is a flowchart of an exemplary process 550 for operating a cell imaging and incubation system, according to some embodiments of the technology described herein. [Figure 6] 1 illustrates an example process for generating and using induced pluripotent stem cells (iPSCs) according to some embodiments of the technology described herein. [Figure 7A] 1 shows the similarity between a gene expression dendrogram and a dendrogram generated from a confusion matrix when evaluating the performance of a machine learning model for clone identification, according to some embodiments of the technology described herein. [Figure 7B] 1 shows the similarity between a gene expression dendrogram and a dendrogram generated from a confusion matrix when evaluating the performance of a machine learning model for clone identification, according to some embodiments of the technology described herein. [Figure 7C] We show that segmentation techniques according to embodiments of the technology described herein can be used to distinguish between pixels associated with iPSCs, non-iPSCs, and background. [Figure 7D] We show that segmentation techniques according to embodiments of the technology described herein can be used to accurately predict the frequency of pixels associated with iPSCs. [Figure 8A] We show that density estimation techniques according to some embodiments of the techniques described herein can be used to generate density maps of bright-field images. [Figure 8B] We show that density estimation techniques according to some embodiments of the techniques described herein can be used to generate density maps of bright-field images. [Figure 8C] We show that density estimation techniques according to some embodiments of the techniques described herein can be used to generate density maps of bright-field images. [Figure 9]1 illustrates an example process for training a machine learning model to generate a density map of a cell culture, according to some embodiments of the technology described herein. [Figure 10A] We show that density estimation techniques according to some embodiments of the techniques described herein can be used to estimate the number of cells shown in a bright field image. [Figure 10B] We show that density estimation techniques according to some embodiments of the techniques described herein can be used to estimate the number of cells shown in a bright field image. [Figure 11A] We show that density estimation techniques according to some embodiments of the techniques described herein can be used to accurately estimate the number of cells shown in a bright field image when compared to ground truth. [Figure 11B] We show that density estimation techniques according to some embodiments of the techniques described herein can be used to accurately estimate the number of cells shown in a bright field image when compared to ground truth. [Figure 11C] We show that density estimation techniques according to some embodiments of the techniques described herein can be used to accurately estimate the number of cells shown in a bright field image when compared to ground truth. [Figure 12A] 1 shows that density estimation techniques according to some embodiments of the technology described herein can be used to monitor the number of cells in a well of a culture plate over time. [Figure 12B] 1 shows that density estimation techniques according to some embodiments of the technology described herein can be used to monitor the number of cells in a well of a culture plate over time. [Figure 13] 1 is a plot showing the number of cells in a culture corresponding to culture treatment adjustment determinations for the culture, according to some embodiments of the technology described herein. [Figure 14] We demonstrate that culture information prediction techniques, according to some embodiments of the technology described herein, can be used to accurately predict the number of cells in a culture. [Figure 15] FIG. 1 is a schematic diagram of an exemplary computing device capable of implementing aspects described herein. DETAILED DESCRIPTION OF THE INVENTION

[0041] Described herein are techniques for adjusting the treatment of a culture of cells by imaging the cell culture and using machine learning techniques to associate individual pixels in the image with respective cell categories. In some embodiments, the cell categories correspond to cell types or other characteristics of the cells or cell populations. In some embodiments, the techniques can be used in conjunction with a cell imaging and incubation system, which can include an imaging sensor configured to acquire images of the culture and an incubator configured to incubate the culture. In some such embodiments, adjusting the treatment of the culture can be based on processing an image of the culture acquired by the system's imaging sensor. The processing can include segmenting the image of the culture into multiple image segments, which can be done by assigning individual pixels of the image to corresponding cell categories based on pixel-level evaluation. According to some embodiments, the techniques include identifying an amount of the culture corresponding to a particular cell category based on the image segments. The amount of the culture corresponding to a particular cell category can inform adjustment of the treatment of the culture. For example, such an amount can inform decision-making throughout the cell culture expansion process.

[0042] The techniques described herein may improve automated cell culture management by providing more reliable and accurate identification of the composition of a cell culture than is available through conventional techniques for culture management, including either conventional manual or conventional automated culture management. In some embodiments, the results of the composition determination are used to identify the quality of the cell culture. For example, the results may be used to identify metrics such as culture area, size (e.g., minimum and maximum dimensions, aspect ratio), shape (e.g., circularity), texture, and roughness. Such metrics can indicate whether the cells in the culture are healthy and / or whether the cells are differentiated, factors that can contribute to the overall quality of the culture. Because the metrics are based on more accurate determination of the culture composition, the quality of the resulting culture can be identified with greater accuracy compared to conventional approaches. In some embodiments described herein, the results of the culture composition and quality identification may be used to adjust the processing of the cell culture over time. By basing processing on more accurate identification of the composition and quality of the cell culture, processing decisions can be more accurate and timely relative to the current culture state, thereby reducing errors and mitigating inefficiencies introduced by less accurate conventional techniques.

[0043] For example, some of the techniques described herein may be used to generate cell culture treatment recommendations that promote culture growth, prevent waste, and improve the overall efficiency of cell culture maintenance. Such recommendations may include recommendations to modify how substances (e.g., growth factors) are added to or removed from the culture (e.g., the amount of substance to add and / or remove from the culture and when to do so), recommendations indicating when to passage the cells in the culture, recommendations indicating the number of new cultures to split the culture into, recommendations to discard the cells in the culture, or other appropriate culture treatment steps. In some embodiments, such recommendations may be output to a user, who may implement the recommendations manually or by controlling a semi-automated system, or to an automated system configured to implement the recommendations. In embodiments where a semi-automated or automated system is available to partially or fully implement the recommendations, the system may do so upon receiving the recommendations.

[0044] Cell culturing is the process of growing cells in a controlled environment for a variety of purposes, such as studying cell morphology, physiology, and biochemistry, monitoring cellular responses to drugs, disease modeling, evaluating genetic variants, etc. Growing a cell culture can involve isolating cells from tissue, providing the tissue with appropriate conditions (e.g., temperature, medium, etc.), growing the cells until they occupy a certain percentage of the available substrate (e.g., reaching a confluence threshold), and then passage the cells by splitting the cells and transferring them to a new vessel to continue growing the culture.

[0045] Maintaining and adjusting cell culture processing can be a manual, laborious, and subjective process, resulting in inefficiencies and inconsistent results. Traditionally, to produce high-quality cells of the correct cell type, cultures are frequently evaluated to inform decisions regarding culture adjustments. Such decisions are often made based on manual visual inspection. These manual evaluations and decisions are made by highly skilled laboratory technicians, but the decisions are inherently subjective and subject to human error. Subjective evaluations can be inaccurate and can, and all too often, lead to decisions that can adversely affect a single culture (e.g., not feeding or dividing at the same time) or the entire culture process (e.g., investing resources in cultures that can be discarded). This is a universal problem in cell culture that has been widely experienced for decades and leads to well-known problems of lost time and physical resources.

[0046] One type of cell that is often cultured is induced pluripotent stem cells (iPSCs), which are stem cells that can be cultured into various cell types. In some cases, stem cell cultures can be maintained in an undifferentiated state, and each cell in the culture can be a stem cell that has not yet developed or has not yet developed into a cell with a specific function or structure. In such cases, the culture can be monitored for whether the cells have differentiated, and if not, the culture can be split into multiple cultures (once conditions such as culture time or culture size are met) to continue culturing a certain amount of undifferentiated stem cells. In such a scenario, if the culture appears to begin to differentiate, the quality of the culture can be related to how much of the culture has differentiated, and cultures composed primarily of differentiated cells can be discarded.

[0047] Therefore, in some situations, cell cultures may be frequently evaluated to ensure the growth of cells with the correct cell type. While cultures can be evaluated using fluorescent imaging, sequencing, and similar techniques, these techniques are time-consuming, invasive, and potentially damaging to the cells. Therefore, it is beneficial to use faster, non-invasive techniques, such as bright-field or phase microscopy, to evaluate cultures. However, accurately quantifying and distinguishing differentiated and undifferentiated cells using such non-invasive techniques can be difficult due to their high visual similarity. As a result, it is also difficult to determine how to modify the culture's treatment to prevent further cell differentiation. Often, cultures are discarded due to an inability to effectively treat the culture. While every discard of a culture represents a loss of resources and time, if a differentiating culture is not identified early enough, time and resources may be spent culturing cells that will ultimately be discarded, leading to further inefficiencies in resources and time.

[0048] Even when the cells have the correct cell type, the culture is often evaluated to estimate the state or health of the cells in the culture. Based on this estimation, a decision is made as to when to modify the way the cells are treated. As a non-limiting example, this may include determining when to change the culture medium, the rate at which to add specific growth factors, and their amounts. Because it is difficult to identify subtle changes that may occur in the culture, it is also difficult to determine when and how to modify the way the culture is treated.

[0049] Additionally or alternatively, cell cultures are frequently assessed to estimate the number of cells in the culture. This estimate is used to inform when to passage the cells in the culture and to determine the number of new cultures into which the culture can be split. However, even a slight miscalculation of cell number can result in cells being passaged at an incorrect time (e.g., too early or too late) and the culture being split into an incorrect number of new cultures. This can result in cell loss via apoptosis, affect how the cells differentiate, and ultimately result in the culture being discarded.

[0050] In many situations, the conventional approach is to have skilled laboratory personnel perform assessments of differentiation, cell state, or cell number in culture. As noted above, despite a high level of skill, these assessments are still inherently subjective and subject to inter-individual variability, even within the same tissue. This can lead to a trial-and-error approach to culture maintenance, resulting in cell loss and / or inconsistency between different cultures.

[0051] Various techniques have been used to attempt to automate cell culture management. However, these techniques have limitations and do not address the above-identified problems related to culture management. For example, one conventional technique involves attempting to evaluate cell cultures using image tile-based classification. Such techniques rely on capturing images of differentiation, dividing the images into multiple tiles, and then analyzing each tile to identify what is depicted in that tile. A tile refers to an image that shows a portion of a larger image, such as an image of a cell culture. When an image is divided into multiple tiles, the multiple tiles may contain images that show overlapping portions of the larger image. This technique is inherently limited in accuracy, as its accuracy is related to the quality of the tiling process and the size / accuracy of the tile. Furthermore, because tile analysis involves comparing tiles to previously viewed known tiles to identify the closest match (thereby identifying a tile as containing the same content as the previous tile it matched), reliable analysis of each tile depends on having a sufficient number of training tiles of each type that can be viewed by the system. Compiling training data represents a high administrative burden for system implementation. Increasing the tile size allows for a reduction in the number of tiles, reducing the burden of data collection, but increasing the tile size leads to a corresponding decrease in accuracy. These limitations in accuracy mean that conventional automated analytical techniques cannot reliably outperform manual, subjective interpretation by skilled laboratory technicians, leading to the continued use of manual processes despite the limitations described above. When conventional automated analytical techniques are used, their accuracy limitations continue to result in errors in regulating cell culture processing.

[0052] Conventional automated techniques for cell analysis are further limited in that they cannot identify cells or cell cultures that are in intermediate states of the cycle (e.g., the cell life cycle or differentiation cycle) rather than purely beginning or ending states. Conventional techniques are limited to making binary decisions about what a tile of an image depicts. For example, conventional techniques can be used to determine whether a tile contains cells of a particular cell type, which may be undifferentiated or differentiated, or a cell type that is at a steady state in the cell life cycle. However, such techniques will not capture the transition of cells or cell cultures between two different states, such as the transition of a culture from one cell type to another. Therefore, using conventional techniques, treatment decisions are based solely on whether the tile is primarily one particular cell type or another. These decisions do not consider the presence in a culture of cells that are transitioning between cell states, or in cultures that contain cells at various stages of the cycle and may use different types or levels of treatment than a single, non-intermediate culture. These limitations mean that culture composition cannot be accurately identified, which can contribute to errors in adjusting the treatment of cell cultures.

[0053] Regarding estimating the number of cells in a culture, conventional automated techniques rely on a measure of the confluence of the culture to estimate the number of cells. Confluence refers to the amount of culture substrate occupied by the cells of the culture. Despite the widespread use of confluence in both automated and manual analysis, confluence-based cell counting techniques are inadequate for accurately estimating the number of cells in a culture. Confluence-based techniques do not take into account cell compaction and area extent. As mentioned above, such miscalculation of the number of cells in a culture contributes to errors in regulating cell culture processing.

[0054] The inventors have recognized and appreciated that such challenges and inefficiencies can be alleviated by improved automated systems for analyzing images of cells in culture. Some techniques described herein include more reliable and / or more accurate approaches for quantifying cell number in a cell culture, estimating cell classification of cells in a cell culture based on pixel-level image analysis and segmentation, and / or adjusting treatment of the culture based on the results of such techniques.

[0055] The present inventors have developed systems and methods for regulating the processing of cell cultures. In some embodiments, the techniques involve estimating culture information of a cell culture, such as information indicative of density, cell number, and / or cell location. For example, this may involve using a machine learning model. The machine learning model may be implemented using one or more convolutional neural networks (CNNs) and adapted to identify the number of cells in a cell culture. In some embodiments, the model may generate information indicative of the density of cells in different regions of the culture, such as a density map. The information may also include identification of the location of individual cells in the culture, such as in the coordinate system of the culture or an image of the culture. The technique using machine learning models improves upon conventional techniques, mitigating the above-mentioned shortcomings associated with cell culture management by taking cell compaction and area into account, leading to more accurate estimation of cell number. This may lead to improved regulation of the processing of the culture, resulting in higher quality cells and consistency between independent cultures.

[0056] In some embodiments, techniques developed by the inventors and described herein include machine learning techniques for estimating the composition of a cell culture. This technique involves segmenting an image of a cell culture by evaluating individual pixels of the image of cells in the culture and identifying a corresponding cell category associated with each pixel. As discussed in more detail below, the cell category may, in some cases, be related to a cell type. The image may be segmented into different segments, each corresponding to a cell category, based on identifying a respective cell category for each pixel and identifying segments containing pixels associated with a common cell category. In some embodiments, culture information, such as may be generated using the cell counting techniques described above, may be used to inform the segmentation, leading to improved efficiency and accuracy of the resulting pixel assignments. Some techniques described herein may result in more accurate estimation of the cellular composition of a culture by achieving pixel-level resolution of the composition of the cell culture. By reducing the inaccuracies of prior techniques, the present technique improves the automated regulation of culture processing, enabling the production of consistent, high-quality cell cultures while also reducing waste and inefficiencies.

[0057] A cell category may relate to one or more characteristics of a cell or group of cells, such as one or more morphological and / or functional characteristics of the cell. Such morphological characteristics may include the shape of the cell or the shape of the group of cells. In some embodiments, such characteristics may be observable characteristics of the cell or group of cells, such as characteristics that can be determined from a depiction (e.g., an image) of the cells. Such observable characteristics may be markers that enable the cell or group of cells to exhibit a particular morphological structure (e.g., cell clustering), etc. Based on the presence or absence of one or more markers, a cell or group of cells may be identified as being in a certain category.

[0058] In some embodiments, cell categories may relate to the differentiation status of cells. Such differentiation status may relate to a particular cell, for example, whether a particular cell is an undifferentiated stem cell or a cell that has differentiated or initiated differentiation and is therefore not undifferentiated, or may relate to a cell population, for example, whether the population includes all undifferentiated cells or a certain amount of non-undifferentiated cells. Cell categories may also relate to cell types, for example, whether a cell type is an undifferentiated stem cell, or a particular type of cell that has initiated or is differentiating, such as a cell located in a particular anatomical structure (e.g., an organ) or that performs a particular anatomical function. The location of particular cellular organelles within a cell may also be a factor underlying the cell category in some embodiments. Where cell categories may relate to cell types, such cell types may include intermediate cell types that may be precursors to other cell types during the cell's developmental cycle. In some embodiments where cell categories relate to cell types, the cell category may relate to a cell population and to one or more cell types present in the cell population.

[0059] In some embodiments, a cell category may be related to a stage of a cell cycle, where a stage may be a process that a cell undergoes during its lifespan. Such a cell cycle may be, for example, mitosis, and a stage of the cell cycle may be a stage of mitosis. A cell category may be related to the vital status of a cell, such as whether the cell is dead or alive. A cell category may also be related to a level of proliferation, such as cell growth rate.

[0060] In some embodiments, the cell category may relate to the experimental population to which the cell or group of cells belongs, for example, whether the cells belong to a control population or to a population that is the subject of an experiment, such as by having been subjected to a perturbation, genetic mutation, or other experiment.

[0061] In some embodiments, a cell category may be associated with a combination of the aforementioned factors, or the system may operate with multiple cell categories, each associated with a different one or combination of the aforementioned factors. Embodiments are not limited to operating with any of the specific examples of cell categories described above.

[0062] In some embodiments, an image segmentation process according to the techniques described herein can analyze an image (e.g., pixel by pixel, as described herein) and identify a category into which each pixel should be classified. In some such image segmentation processes, the process can estimate a value indicative of the likelihood that the pixel is associated with a respective cell category. In doing so, in some cases, the image segmentation process can identify multiple categories for a pixel into which to classify the pixel, each of which can be associated with a respective (and potentially different) likelihood value. A cell category for the pixel can then be selected by review of the likelihood values, such as by selecting the category with the highest likelihood measure or other evaluation (examples of which are provided below).

[0063] In some embodiments, the image segmentation process may analyze the image using a machine learning model trained to identify a category into which each pixel should be classified. In some embodiments, the image includes a two-dimensional (2D) matrix of data points, with each pixel in the image corresponding to a single data point in the 2D matrix. Thus, the 2D matrix may be provided as input to the machine learning model to obtain, for a data point in the 2D matrix, an output identifying multiple categories into which the data point should be classified. In some other embodiments, the image includes a 2D matrix of data points, with each pixel in the image corresponding to multiple data points in the 2D matrix. Thus, the 2D matrix may be provided as input to the machine learning model to obtain, for multiple data points in the 2D matrix, an output identifying multiple categories into which the multiple data points should be classified.

[0064] Additionally or alternatively, in some embodiments, the image segmentation process may analyze an image and identify a category into which a region of the image should be classified. For example, a region of the image may include two or more individual pixels in the image. In some embodiments, the two or more pixels are neighboring pixels. For example, in a 2D matrix of data points, neighboring pixels include entries in the 2D matrix that are orthogonally or diagonally adjacent.

[0065] Various examples of how these techniques and systems may be implemented are described below, but it should be understood that embodiments are not limited to operating according to these examples. Other embodiments are possible.

[0066] FIG. 1A illustrates an exemplary technique 100 for regulating the treatment of a culture 102 of cells by processing an image 106 of the culture 102 using one or more machine learning models 108 to generate an output 110 that includes image segments 110-1 and / or culture information 110-2.

[0067] In some embodiments, culture 102 includes any suitable type of cells. For example, culture 102 can include cells of the same type and / or multiple (e.g., two or more) different types of cells. As a non-limiting example, in some embodiments, culture 102 includes pluripotent stem cells. Pluripotent stem cells are undifferentiated or partially differentiated cells that have the ability to self-renew and differentiate into various cell types. As another example, culture 102 can include induced pluripotent stem cells (iPSCs), a type of pluripotent stem cell derived from adult somatic cells that have been genetically reprogrammed to an embryonic stem cell-like state. Additionally or alternatively, culture 102 can include cell types into which pluripotent stem cells and / or iPSCs have differentiated. However, it should be understood that culture 102 includes any suitable type of cells, as aspects of the technology described herein are not limited in this respect.

[0068] In some embodiments, culture 102 is grown in any suitable type of vessel. For example, the type of vessel may depend on the type of experiment being performed on culture 102 and / or the type of imaging sensor used to capture images of culture 102. For example, culture 102 may be grown on a cover slip, in a Petri dish, in a sample well, in a multi-well plate (e.g., a microplate), in a culture flask, on an OptoSelect™ chip, or using any other suitable type of vessel.

[0069] In some embodiments, imaging sensor 104 is used to capture image 106 of culture 102. Imaging sensor 104 may include any suitable type of imaging sensor, such as, for example, an imaging sensor capable of capturing bright field, phase contrast, and / or fluorescence images. For example, imaging sensor 104 may include a microscope imaging system having one or more cameras, such as a Celigo® imaging cytometer, a Beacon® Optofluidic system, an Incucyte® live cell analysis system, and / or an Opera Phenix® high content screening system.

[0070] In some embodiments, the imaging sensor 104 automatically captures the image 106. For example, the imaging sensor 104 may automatically capture images at specified time intervals. Additionally or alternatively, the imaging sensor 104 may automatically capture the image 106 after the culture 102 is detected within the field of view of the imaging sensor 104 and / or at a particular position relative to the imaging sensor 104. In some embodiments, the imaging sensor 104 captures the image 106 in response to receiving user input indicating when the image 106 should be captured.

[0071] In some embodiments, image 106 includes an image of all or a portion of culture 102. For example, image 106 may show one, some, or all of the wells of a multi-well plate. Additionally or alternatively, image 106 may show one, some, or all of the cells in culture 102. In some embodiments, image 106 is an image captured using a bright field image, a phase contrast image, a fluorescent image, and / or any other suitable imaging modality. In some embodiments, any suitable image processing technique can be used to process image 106 captured using imaging sensor 104, and aspects of the technology described herein are not limited in this respect.

[0072] In some embodiments, image 106 is processed using one or more trained machine learning models 108 to obtain image segment 110-1 and / or culture information 110-2. In some embodiments, different machine learning models are used to obtain image segment 110-1 and culture information 110-2. For example, in the embodiment of FIG. 1A, machine learning model 108-1 is used to obtain image segment 110-1, and machine learning model 108-2 is used to obtain culture information 110-2.

[0073] In some embodiments, image 106 comprises a two-dimensional (2D) matrix of data points. In some embodiments, individual pixels in image 106 comprise (e.g., consist of) individual data points in the 2D matrix. In some embodiments, a region of image 106 comprises two or more individual pixels in the image. Thus, a region may comprise a group of two or more data points in the 2D matrix. In some embodiments, the two or more pixels are neighboring pixels, meaning they comprise two or more data points that are orthogonally or diagonally adjacent to each other in the 2D matrix.

[0074] In some embodiments, the image 106 (e.g., a 2D matrix) is provided to the machine learning model 108-1 to obtain the image segment 110-1. The machine learning model 108-1 can be of any suitable type. For example, the machine learning model can be a neural network, such as a deep neural network model. The deep neural network model can have any of a number of types of architecture and can include any suitable type of layer. For example, the deep neural network model can include a convolutional neural network (CNN), a U-Net network, a DeepLab network, or versions thereof (e.g., DeepLabv1, DeepLabv2, DeepLabv3, and DeepLabv3+), or any other suitable deep learning network architecture. The neural network can have one or more convolutional layers. In some embodiments, the final layer of the deep neural network can be modified to take into account a set of labels corresponding to a set of cell categories (e.g., iPSC, non-iPSC, and background).

[0075] The architecture of the deep neural network model may include one or more base blocks implemented using ResNet, MobileNet, Xception, or any other suitable deep learning network, or any variant of such a deep learning network. In some embodiments, the deep learning network may have any appropriate depth. The depth, in some embodiments, is the maximum number of consecutive convolutional or fully connected layers on the path from the input layer to the output layer. For example, the deep learning network may be 18 layers deep (e.g., ResNet-18), 50 layers deep (e.g., ResNet-50), 53 layers deep (e.g., MobileNet-v2), 71 layers deep (e.g., Xception), or any other appropriate depth, and aspects of the present technology are not limited in this respect. In some embodiments, the deep learning network has any appropriate number of layers, and aspects of the present technology are not limited in this respect. For example, the deep learning network may have 177 total layers and 54 convolutional layers (e.g., ResNet-50). However, it should be understood that the deep learning network may have more or fewer layers. In some embodiments, the deep neural network takes images of any suitable resolution, such as, for example, image input sizes of 224x224, 299x299, 1080x1080, or 1958x1958.Aspects of DeepLab are described in Chen, Liang-Chieh et al., "DeepLab: Semantic Image Segmentation with Convolutional Nets, Atrous Convolution, and Fully Connected CRFs," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 40, no. 4, pp. 834-848 (2017) and Chen, Liang-Chieh et al., "Encoder-decoder with atrous separable convolution for semantic image segmentation," in Ferrari, V., Hebert, M., Sminchisescu, C., Weiss, Y. (eds.) ECCV 2018, LNCS, vol. 11211, pp. 833-851, Springer, Cham (2018), each of which is incorporated herein by reference in its entirety. Aspects of ResNet are described in detail in He, K. et al., "Deep Residual Learning for Image Recognition," CVPR (2016), which is incorporated herein by reference in its entirety.

[0076] Regardless of the particular type of machine learning model used as part of the exemplary technique 100, the machine learning model 108-1 outputs an image segment 110-1 corresponding to the image 106. The image segment may include one or more pixels associated with a cell category. For example, as shown in the embodiment of FIG. 1A, the image segment 110-1 includes pixels associated with three different cell categories. For example, some image segments may include pixels associated with iPSCs, some image segments may include pixels associated with non-iPSCs, and some image segments may include pixels associated with a non-cell category. However, it should be understood that the machine learning model 108-1 may output an image segment including pixels associated with any suitable number or types of cell categories, as aspects of the technique described herein are not limited in this respect.

[0077] In some embodiments, the image 106 is optionally provided to a machine learning model 108-2 to obtain culture information 110-2. The machine learning model 108-2 can be of any suitable type. For example, the machine learning model 108-2 can be a neural network, such as a convolutional neural network (CNN) model. In some embodiments, the neural network includes two portions, each configured to process an input using a convolutional layer. In some embodiments, the first portion is configured to approximate the number of cells and classify the image into a threshold number of classes. The threshold number can be any suitable number of classes, such as, for example, 10 classes, as aspects of the technology described herein are not limited in this respect. In some embodiments, the first portion is configured to classify the image into classes based on the approximate number of cells in the image. Information generated by the first portion of the network can be used by the second portion of the network to generate a density map. In some embodiments, the first and second portions each include any suitable number of convolutional layers, as aspects of the technology described herein are not limited in this respect. For example, the CNN may use the architecture described in Sindagi, VAamd Patel, VMJ, "CNN-based Cascaded Multi-task Learning of High-level Prior and Density Estimation for Crowd Counting," arXiv:1707.09605 (2017), the entire contents of which are incorporated herein by reference.

[0078] Regardless of the particular type of machine learning model used as part of the exemplary technique 100, the machine learning model 108-2 outputs culture information 110-2 corresponding to the image 106. In some embodiments, the culture information includes a cell count indicating the number of cells shown in the image 106. In some embodiments, the culture information 110-2 indicates the physical location of the cells in the image 106. For example, the culture information 110-2 may include a three-dimensional density map in which density is distributed along the x and y coordinates of the image 106.

[0079] 1A , culture information 110-2 is optionally used to obtain image segment 110-1. For example, culture information 110-2 may be provided as input to machine learning model 108-1, which is used to predict image segment 110-1. Machine learning model 108-1 may process culture information 110-2 to estimate the probability that a cell or a particular cell type is present at a location associated with a pixel or group of pixels.

[0080] Additionally or alternatively, culture information 110-2 may be optionally compared with image segment 110-1 to determine whether there is a correlation therebetween. For example, culture information 110-2 may indicate whether cells are present at locations corresponding to particular pixels, while image segment 110-1 may indicate whether those pixels are associated with a particular cell category. This correlation may be used to identify cells associated with a given cell category and to isolate those cells from their identified locations. The given cell category may include any type of cell category. In some embodiments, the given cell category may include one or more cell categories. For example, the given cell category may include a first given cell category representing a first cell type (e.g., iPSC) or a second given cell category representing a second cell type (e.g., non-iPSC). In another example, the predetermined cell categories may include a first predetermined cell category representing a first cell type (e.g., a neuronal cell type), a second predetermined cell category representing a second cell type (e.g., a mesenchymal cell type), or a third predetermined cell category representing a background portion (e.g., a region of the image that does not contain cells). In another example, the predetermined cell categories may include a first predetermined cell category representing a first differentiation status of a cell (e.g., a differentiated cell), a second predetermined cell category representing a second differentiation status of a cell (e.g., an undifferentiated cell), and a third predetermined cell category representing a third differentiation status of a cell (e.g., a partially differentiated cell). In another example, the predetermined cell categories may include a first predetermined cell category representing a first phase of the cell cycle (e.g., G1 phase), a second predetermined cell category representing a second phase of the cell cycle (e.g., S phase), a third predetermined cell category representing a third phase of the cell cycle (e.g., G2 phase), a fourth predetermined cell category representing a fourth phase of the cell cycle (e.g., M phase), and a fifth predetermined cell category representing a fifth phase of the cell cycle (e.g., G0 phase). In another example, the predetermined cell categories may include a first predetermined cell category representing a first experimental group (e.g., control group) and a second predetermined cell category representing a second experimental group (e.g., treatment group).

[0081] In some embodiments, image segment 110-1 and (optionally) culture information 110-2 are used to adjust the treatment 112 of culture 102. For example, image segment 110-1 and / or culture information 110-2 may indicate the type, health, differentiation state, and / or quantity of cells in the culture. This information can then be used to inform decisions regarding the treatment of culture 102 and / or other cultures. For example, image segment 110-1 and / or culture information 110-2 may be used to inform decisions related to cell passaging, such as when to passage cells, how many new cultures to split the culture into, and whether to discard cells in the culture. Image segment 110-1 and / or culture information 110-2 may be used to inform decisions regarding how to treat the culture or future cultures, such as when to feed the culture, what to feed the culture with, and how much. Examples of culture treatment recommendations are further described herein with respect to at least FIG. 2C .

[0082] In some embodiments, one or more users adjust the culture treatment 112 manually or semi-automatically. For example, the exemplary technique 100 may include outputting a recommendation to adjust the culture treatment 112 to one or more users (e.g., via a user interface), who may implement the adjustment and / or provide user input to the system that causes the system to implement the treatment adjustment. In some embodiments, the adjustment of the culture treatment 112 is performed automatically. For example, a processor may generate a recommendation to adjust the culture treatment 112 and cause the system to implement the adjustment without user intervention.

[0083] 1B is a block diagram of an exemplary system 150 for regulating a treatment of a cell culture, according to some embodiments of the technology described herein. System 150 includes a computing device 180 configured to execute software 182 to perform various functions associated with evaluating and regulating a treatment of a cell culture.

[0084] Computing device 180 may be one or more computing devices of any suitable type. For example, computing device 180 may be a portable computing device (e.g., laptop, smartphone) or a fixed computing device (e.g., desktop computer, server). If computing device 180 includes multiple computing devices, the devices may be physically co-located (e.g., in a single room) or distributed across multiple physical locations. In some embodiments, computing device 180 may be part of a cloud computing infrastructure.

[0085] In some embodiments, computing device 180 may be operated by one or more users 160, such as one or more researchers and / or other individuals. For example, user 160 may provide user input indicating that imaging sensor 192 may capture images of the cultures, input specifying processing or other methods to be performed on the captured images, and / or input specifying how the processing of the cultures may be adjusted.

[0086] As shown in the embodiment of FIG. 1B, software 182 includes multiple modules. Each module may include processor-executable instructions that, when executed by at least one computer hardware processor, cause the at least one computer hardware processor to perform the functionality of the module. Such modules are sometimes referred to herein as “software modules.” The software modules shown in FIG. 1B include processor-executable instructions that, when executed by a computing device, cause the computing device to perform one or more processes, such as those described herein with respect to at least FIGS. 2A-2B and 5B. It should be understood that the modules shown in FIG. 1B are exemplary, and that in other embodiments, software 182 may be implemented using one or more other software modules in addition to or instead of the modules shown in FIG. 1B. In other words, software 182 may be organized differently than that shown in FIG. 1B.

[0087] 1B, software 182 includes multiple software modules for evaluating and regulating the processing of a culture of cells, such as image segmentation module 166, culture information prediction module 168, culture regulation module 172, and system automation module 162. In the embodiment of FIG. 1B, software 182 additionally includes machine learning model training module 164 for training one or more machine learning models and user interface module 170 for obtaining user input.

[0088] In some embodiments, the image segmentation module 166 acquires an image from the imaging sensor 192 or the image data store 152, acquires a trained machine learning model from the machine learning model data store 154, and processes the acquired image using the acquired machine learning model to segment the image. For example, the image segmentation module 166 may process the image using the machine learning model to predict, for pixels in the image, respective cell categories associated with the pixels. Techniques for segmenting images are described herein with respect to at least process 250 of FIG. 2B .

[0089] In some embodiments, culture information prediction module 168 acquires an image from imaging sensor 192 or image data store 152, acquires a trained machine learning model from machine learning model data store 154, and uses the acquired machine learning model to process the acquired image to acquire culture information corresponding to the image. For example, culture information prediction module 168 may process the image to estimate the number of cells in the image and / or generate a density map indicating the location of cells in the image. Techniques for acquiring culture information are described herein with respect to at least operation 204 of process 200 shown in FIG. 2A .

[0090] In some embodiments, culture adjustment module 172 obtains image segments from image segmentation module 166 and / or obtains culture information from culture information prediction module 168 and uses the obtained image segments and / or culture information to generate recommendations for adjusting the processing of the cell culture. For example, culture adjustment module 172 may determine when to passage cells in the culture, how many cultures to split the culture into, whether to discard the culture, whether to feed the culture, what to feed the culture, and / or how much. Example recommendations for adjusting the processing of the cell culture are described herein with respect to at least FIG. 2C.

[0091] In some embodiments, culture treatment adjustment recommendations may be output by culture adjustment module 172. For example, the recommendations may be output to user 160 via user interface module 170. Additionally or alternatively, the recommendations may be output to system automation module 162.

[0092] User interface module 170 may be a graphical user interface (GUI), a text-based user interface, and / or any other suitable type of interface through which a user can provide input or display information generated by software 182. For example, in some embodiments, the user interface may be a web page or web application accessible through an internet browser. In some embodiments, the user interface may be the graphical user interface (GUI) of an app running on a user's mobile device. In some embodiments, the user interface may include a number of selectable elements with which a user can interact. For example, the user interface may include drop-down lists, check boxes, text fields, or other suitable elements.

[0093] System automation module 162 is configured to control one or more of imaging sensor 192, incubator 194, and / or robotic system 196. For example, system automation module 162 may cause robotic system 196 to manipulate cells in culture (e.g., detach, spit, or discard cells), manipulate substances with respect to the culture (e.g., culture medium, growth factors) to modify the culture medium, and / or move the culture between imaging sensor 192 and incubator 194. Additionally or alternatively, system automation module 162 may cause imaging sensor 192 to capture images of the cell culture. Additionally or alternatively, system automation module 162 may cause incubator 194 to adjust its temperature or other settings. Examples of system automation software include Overlord™ scheduling software. Examples of automation systems and techniques for using such systems are described herein with respect to at least FIGS. 5A and 5B.

[0094] In some embodiments, system automation module 162 is configured to control imaging sensor 192, incubator 194, and / or robotic system 196 in response to obtaining a culture treatment adjustment recommendation from culture adjustment module 172. Additionally or alternatively, system automation module 162 is configured to control imaging sensor 192, incubator 194, and / or robotic system 196 independently of receiving an output from culture adjustment module 172. For example, system automation module 162 may control imaging sensor 192, incubator 194, and / or robotic system 196 to perform periodically in response to user input by user 160 and / or in response to receiving an output from another software module.

[0095] Imaging sensor 192 includes any suitable type of imaging sensor. In some embodiments, imaging sensor 192 is configured for bright-field imaging, phase-contrast imaging, and / or fluorescence imaging. For example, imaging sensor 192 may include a microscope imaging system having one or more cameras. Examples of imaging sensors are described herein with respect to at least FIG. 1A.

[0096] The incubator 194 includes any suitable type of culture incubation system. The incubator 194 may be configured to regulate the temperature, humidity, and / or CO2 levels in an environment in which a culture of cells can be stored. For example, the incubator 194 may include an input for CO2, a water tank for humidity, and / or a thermoregulation device for temperature control. The incubator 194 may be automatic or semi-automatic, meaning that the incubator self-regulates or regulates environmental conditions (e.g., temperature, humidity, and / or CO2 levels) in response to user input. Additionally or alternatively, the incubator 194 may be manual, meaning that a user regulates the environmental conditions. Examples of incubators include the Cytomat™ automated incubator and the LiCONic STX500 automated incubator.

[0097] Robotic system 196 includes any suitable robotic system configured to handle and / or manipulate cultures of cells. The robotic system may include a liquid handler configured to add or remove materials from the culture. The robotic system may include a robotic arm configured to handle the culture (e.g., move the culture between an incubator and an imaging platform). Examples of robotic systems include the Hamilton Microlab STAR liquid handling system and the PF3400 SCARA robot.

[0098] 1B , example system 150 also includes image data store 152 and machine learning model data store 154. In some embodiments, software 182 obtains data from image data store 152, machine learning model data store 154, and / or user 160 (e.g., by uploading the data). In some embodiments, the software also includes machine learning model training module 164 for training one or more machine learning models (e.g., stored in machine learning model data store 154).

[0099] In some embodiments, the images are obtained from image data store 152. Image data store 152 may be of any suitable type (e.g., a database system, a multi-file, a flat file, etc.) and may store image data in any suitable manner and in any suitable format, as aspects of the technology described herein are not limited in this respect. Image data store 152 may be part of computing device 180 or may be external to it.

[0100] 1A 。 In some embodiments, the image data store 152 includes image data acquired for a culture of cells, at least as described herein with respect to FIG. 1A . In some embodiments, the stored image data may be captured using the imaging sensor 192, previously uploaded by a user (e.g., user 160), and / or from one or more public data stores. In some embodiments, a portion of the image data may be processed by the image segmentation module 166 to obtain image segments. In some embodiments, a portion of the image data may be processed by the culture information prediction module 168 to obtain culture information. In some embodiments, a portion of the image data may be used to train one or more machine learning models (e.g., using the machine learning model training module 164).

[0101] In some embodiments, image segmentation module 166 and / or culture information prediction module 168 obtain (either retrieve or be provided with) a trained machine learning model from machine learning model data store 154. The machine learning model may be provided over a communications network (not shown), such as the Internet or other suitable network, and aspects of the technology described herein are not limited to any particular communications network.

[0102] In some embodiments, machine learning model datastore 154 stores one or more machine learning models used to segment images of cultures and / or obtain culture information corresponding to images of cultures. Machine learning model datastore 154 may be of any suitable type (e.g., a database system, a multi-file, a flat file, etc.) and may store trained machine learning models in any suitable manner and in any suitable format, as aspects of the technology described herein are not limited in this respect. Machine learning model datastore 154 may be part of or external to computing device 180.

[0103] In some embodiments, machine learning model training module 164, referred to herein as training module 164, may be configured to train one or more machine learning models to segment images of cultures of cells and / or to obtain culture information for images of cultures of cells. In some embodiments, training module 164 trains the machine learning models using a training set of image data. For example, training module 164 may obtain training data from image data store 152. In some embodiments, training module 164 may provide the trained machine learning models to machine learning model data store 154.

[0104] 2A is a flowchart of an exemplary process 200 for adjusting the treatment of a cell culture, according to some embodiments of the techniques described herein. One or more operations of process 200 may be performed automatically by any suitable computing device. For example, the operations may be performed by a laptop computer, a desktop computer, one or more servers in a cloud computing environment, computing device 180 described herein with respect to FIG. 1B, computer system 1500 described herein with respect to FIG. 15, and / or any other suitable manner. For example, in some embodiments, operation 202 may be performed automatically by any suitable computing device. As another example, operation 204 may be performed automatically by any suitable computing device.

[0105] Process 200 begins at operation 202, where an image of the cells in culture is obtained. In some embodiments, the image is acquired using one or more image sensors. In some embodiments, the image is acquired from a data store that stores images previously acquired using one or more image sensors. The one or more image sensors may include any suitable type of image sensor, as aspects of the present technology are not limited to any particular type of imaging sensor. For example, the one or more image sensors may include a camera configured to capture brightfield, phase contrast, and / or fluorescent images of the cells in culture. The camera may be included in a microscope imaging system. Examples of imaging systems are described herein with respect to at least FIGS. 1A and 1B.

[0106] In some embodiments, the obtained image shows a portion (e.g., some or all) of a culture of cells. For example, the image may show one, some, or all of the cells of the culture. FIG. 4A shows an example bright field image of cells of the culture that may be obtained in operation 202. However, it should be understood that the image may be captured using any suitable imaging modality, such as, for example, bright field, phase contrast, and / or fluorescence imaging.

[0107] Process 200 then proceeds to (optional) operation 204, where the acquired image is processed to estimate growth information for the culture. In some embodiments, processing the image includes providing the image as an input to a trained machine learning model. The machine learning model may be trained for one or more tasks. For example, the machine learning model may be trained to estimate the number of cells in the image. Additionally or alternatively, the machine learning model may be trained to estimate a density map in which density is distributed along the x and y coordinates of the image. In some embodiments, the predicted cell number is used to inform the density map estimation. FIG. 4B shows an example density map estimated for the brightfield image shown in FIG. 4A.

[0108] The machine learning model can be of any suitable type. For example, the machine learning model can be a neural network model, such as a convolutional neural network (CNN) model. The CNN model can have one or more convolutional layers. For example, the CNN can use the architecture described in Sindagi, V A and Patel, V M J, "CNN-based Cascaded Multi-task Learning of High-level Prior and Density Estimation for Crowd Counting," arXiv:1707.09605 (2017), the entire contents of which are incorporated herein by reference.

[0109] In some embodiments, the culture information includes the output of a machine learning model. For example, the culture information may include a predicted cell count. Additionally or alternatively, the culture information may indicate the physical location of cells shown in the image. For example, the culture information may include a density map estimated by the machine learning model. Additionally or alternatively, the culture information may indicate the coordinates of cells within the image.

[0110] Process 200 then proceeds to operation 206, where the image is processed to identify one or more cell categories of cells depicted in the image. In some embodiments, processing the image includes segmenting the image into one or more image segments at operation 206a. This may include, for example, providing the image as input to a trained machine learning model.

[0111] In some embodiments, a machine learning model is trained to predict, for each of a plurality (e.g., some or all) of pixels in an image, a value corresponding to a respective cell category. For example, the value corresponding to each cell category may indicate the likelihood that the pixel corresponds to a cell of the respective cell category. As non-limiting examples, the machine learning model may be trained to predict values indicating the likelihood that the pixel is associated with a cell of a particular cell type, the likelihood that the pixel is associated with a cell or group of cells having a particular characteristic, the likelihood that the pixel is associated with a cell having a particular differentiation status, the likelihood that the pixel is associated with a cell at a particular stage of the cell cycle, and / or the likelihood that the pixel is associated with a cell or group of cells belonging to a particular experimental group. Further examples of cell categories are described above.

[0112] In some embodiments, the culture information obtained in operation 204 is optionally provided as input to a machine learning model in operation 206a. Coordinate information from the estimated density map can be used to indicate predicted cell locations within the image. For example, a matrix of x and y coordinates can be provided along with the image as input to the trained machine learning model. Additionally or alternatively, the coordinates can be used to directly label the image before providing it to the machine learning model. In some embodiments, the machine learning model is trained to predict, for each of a plurality of pixels in the image, a value indicative of the likelihood that a cell is present at the location associated with the pixel.

[0113] The machine learning model can be of any suitable type. For example, the machine learning model can be a neural network, such as a deep neural network model. The deep neural network can have any of a number of types of architecture and can include any suitable type of layer. For example, the deep neural network model can include a convolutional neural network (CNN), U-Net, DeepLab, and versions thereof (e.g., DeepLabv1, DeepLabv2, DeepLabv3, and DeepLabv3+), or any other suitable deep learning network. The neural network can have one or more convolutional layers. The architecture of the deep neural network can include one or more base blocks implemented using ResNet, MobileNet, Xception, or any other suitable deep learning network can be used. Aspects of DeepLab are described in Chen, Liang-Chieh et al., "DeepLab: Semantic Image Segmentation with Convolutional Nets, Atrous Convolution, and Fully Connected CRFs," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 40, no. 4, pp. 834-848 (2017) and Chen, Liang-Chieh et al., "Encoder-decoder with atrous separable convolution for semantic image segmentation," In: Ferrari, V., Hebert, M., Sminchisescu, C., Weiss, Y. (eds.) ECCV 2018, LNCS, vol. 11211, pp. 833-851, Springer, Cham (2018), each of which is incorporated herein by reference in its entirety.Aspects of ResNet are described in He, K. et al., "Deep Residual Learning for Image Recognition," CVPR (2016), which is incorporated herein by reference in its entirety.

[0114] Figure 3A shows an example of image segmentation output from a machine learning model trained for image segmentation (e.g., model 108-1 in Figure 1A). The image is segmented into segments containing pixels associated with three cell categories: iPSCs, non-IPSCs, and background cells. Figure 3B shows the same image showing the likelihood that a pixel is associated with a cell in the iPSC cell category.

[0115] Process 200 then proceeds to operation 208, which uses the image segments to identify a quantity of culture corresponding to each of one or more cell categories. For a particular cell category, in some embodiments, this involves determining the number of pixels associated with that cell category compared to the total number of pixels and / or the number of pixels associated with one or more particular cell categories. For example, to identify a quantity of culture corresponding to iPSCs, the number of pixels associated with iPSCs may be compared to the sum of the number of pixels associated with iPSCs and the number of pixels associated with non-iPSCs.

[0116] Process 200 then proceeds to operation 210, where the treatment of the culture is adjusted based on the amount of culture corresponding to each of the one or more cell categories. In some embodiments, this includes outputting a recommendation to adjust the treatment of the culture. Non-limiting examples of recommendations include modifying the method of adding one or more substances to the culture, discarding the culture, passaging the cells of the culture, and harvesting the cells for downstream applications. Examples of recommendations to adjust the treatment of the culture are described herein with respect to at least FIG. 2C.

[0117] In some embodiments, adjusting the treatment of the culture is performed by one or more users. For example, recommendations to adjust the treatment of the culture may be output to a user via a user interface, such as user interface module 170 described at least herein with respect to FIG. 1B.

[0118] In some embodiments, adjusting the processing of the culture is performed by an automated or semi-automated system. The system may include one or more components for performing the adjustment, such as a robotic system (e.g., robotic system 196 shown in FIG. 1B), an incubator (e.g., incubator 194 shown in FIG. 1B), and / or one or more imaging sensors (e.g., imaging sensor 192 shown in FIG. 1B). In some embodiments, the system performs the adjustment in response to identifying an amount of culture corresponding to each of one or more cell categories, after a specified time has elapsed since the identification, and / or in response to receiving user input.

[0119] In some embodiments, adjusting the treatment of the culture is further based on the culture information estimated in operation 204. For example, the image segment may show one or more cells or groups of cells associated with a particular cell category. The manner in which the treatment of cells associated with that particular cell category is adjusted may differ from the manner in which the treatment of cells associated with other categories is adjusted. In some embodiments, the culture information may be used to precisely locate and treat cells corresponding to pixels associated with a particular cell category. For example, when removing differentiated cells from a culture, the image segment, along with the culture information, may be used to locate and remove cells corresponding to pixels associated with the differentiated cell category.

[0120] Process 200 then proceeds to (optional) operation 212, where the treatment of the second culture is adjusted. In some embodiments, the second culture is grown in parallel with the culture undergoing analysis (referred to herein as the first culture). For example, cells from both cultures may be seeded at the same time or within the same time frame. In some embodiments, the cells of the second culture may be seeded at a later time than the cells of the first culture, or after analysis of the first culture.

[0121] In some embodiments, the treatment of the second culture is adjusted in operation 212, culturing the second culture to have a predetermined amount of each of one or more cell categories. This may include comparing the amounts of the first culture corresponding to the particular cell categories (identified in operation 208) to the respective predetermined amounts. If the identified amounts of the first culture equal the predetermined amount or are within a threshold percentage (e.g., 1%, 2%, 5%, 10%, etc.) of the predetermined amount, the treatment of the second culture may be adjusted in the same or similar manner as the first culture. If the identified amount of the first culture does not equal the predetermined amount or is not within a threshold percentage of the predetermined amount, the treatment of the second culture may be adjusted differently than the first culture. The predetermined amount may be any number of cells set by one or more users, by an automated system, and / or by a semi-automated system. In some embodiments, the predetermined amount can be at least 1 cell, 10 cells, 100 cells, 1000 cells, 10,000 cells, 100,000 cells, or more cells. In some embodiments, the predetermined amount can be at most 100,000 cells, 10,000 cells, 1000 cells, 10 cells, or fewer cells. In some other embodiments, the predetermined amount can be between 17,000 and 23,000 cells, 17,500 and 22,500 cells, 18,000 and 22,000 cells, 18,500 and 21,500 cells, 19,000 and 21,000 cells, 19,500 and 20,500 cells, 19,700 and 20,300 cells, 19,800 and 20,200 cells, 19,900 and 20,100 cells, 19,950 and 20,050 cells, or any other suitable cell number range. Additionally or alternatively, in some embodiments, the predetermined amount can be any percentage of cells set by one or more users, by an automated system, and / or by a semi-automated system.In some embodiments, the predetermined amount can be at least 10% of the cells in the culture, at least 20% of the cells in the culture, at least 25% of the cells in the culture, at least 30% of the cells in the culture, at least 40% of the cells in the culture, at least 50% of the cells in the culture, at least 60% of the cells in the culture, at least 70% of the cells in the culture, at least 75% of the cells in the culture, at least 80% of the cells in the culture, at least 90% of the cells in the culture, at least 95% of the cells in the culture, between 10% and 100% of the cells in the culture, between 50% and 100% of the cells in the culture, between 75% and 95% of the cells in the culture, or any other suitable range. Non-limiting examples of recommendations for adjusting the processing of the second culture include modifying the method of adding one or more substances to the culture, discarding the culture, passaging the cells in the culture, and harvesting the cells for downstream applications. Examples of recommendations for adjusting the processing of the culture are described herein with respect to at least FIG. 2C.

[0122] Regulating the processing of the second culture, including with respect to at least operation 210, as described herein, may be performed by one or more users, by an automated system, and / or by a semi-automated system.

[0123] In operation 214, process 200 includes determining whether there are additional images of cells from the culture being processed. If so, operations 202-212 are repeated for the additional images.

[0124] It should be understood that any combination of operations may be performed as part of process 200. Process 200 may include additional or fewer operations than those shown in Figure 2A. For example, process 200 may include operations 202-214, operations 206-210, operations 202 and 206-210, operations 204-210, operations 202-210, etc.

[0125] 2B is a flowchart of an exemplary process 250 for segmenting an image into multiple image segments, in accordance with some embodiments of the techniques described herein. In some embodiments, operation 206a of process 200 may be implemented using process 250. Process 250 may be performed by any suitable computing device (e.g., computing device 180 described herein with respect to FIG. 1B and / or computer system 1500 described herein with respect to FIG. 15).

[0126] Process 250 begins at operation 252 where pixels in the image are assigned to corresponding cell categories. By way of non-limiting example, this may include assigning pixels to cell categories that represent a type of cell or group of cells, assigning pixels to cell categories that represent a characteristic of a cell or group of cells, assigning pixels to cell categories that represent the differentiation status of a cell, assigning pixels to cell categories that represent a particular stage of the cell cycle, and / or assigning pixels to cell categories that represent the experimental group to which the cell or group of cells belongs. Further examples of cell categories are described above.

[0127] In some embodiments, operation 252 includes assigning pixels to a plurality of corresponding cell categories. This may include assigning pixels to any corresponding combination of cell categories. For example, this may include assigning pixels to a cell category representing a particular cell type and a cell category representing a characteristic of the cell. As another example, this may include assigning a particular cell type to a cell category representing a differentiation status, a cell category representing a cell cycle stage, and a cell category representing a characteristic of a cell population.

[0128] Assigning individual pixels in the image to corresponding cell categories in operation 252 includes, in operation 252a, identifying a value for the pixel that corresponds to each cell category. In some embodiments, the value indicates the likelihood that the pixel corresponds to a cell of the respective cell category. For example, this may include identifying a first value for the pixel that indicates the likelihood that the pixel is associated with a cell of a first cell type (e.g., iPSC) and a second value that indicates the likelihood that the pixel is associated with a cell of a second type (e.g., non-iPSC). However, it should be understood that any suitable number of values may be identified for the pixel (e.g., 2 values, 3 values, 4 values, 5 values, 10 values, 20 values, etc.).

[0129] In some embodiments, operation 252a is performed using a machine learning model, such as the machine learning models described herein with respect to at least operation 206 of process 200. For example, the determined value may be obtained as an output from a machine learning model.

[0130] Process 250 then proceeds to operation 252b, where the pixel is assigned to a corresponding cell category based on the determined value. In some embodiments, this involves assigning the pixel to the cell category that corresponds to the value that indicates the greatest likelihood that the pixel is associated with a cell of that category. For example, if the value indicates a 40% likelihood that the pixel is associated with a cell of type A and a 60% likelihood that the pixel is associated with a cell of type B, type B may be assigned to the pixel because it corresponds to the value that indicates the greatest likelihood.

[0131] In some embodiments, a value corresponding to each cell category may be output regardless of the cell category assigned to the pixel in operation 252b. For example, for a particular pixel, the output may indicate that a value indicating the likelihood that the pixel is associated with a cell of type A is 40%, and a value indicating the likelihood that the pixel is associated with a cell of type B is 60%, even though the pixel may be assigned to type B. In this manner, the technique may be used to monitor cells in culture as they transition between cell categories. For example, the technique may be used to determine whether cells are transitioning between an undifferentiated state and a differentiated state. Pixels corresponding to those cells may be classified as "undifferentiated," but the value indicating the likelihood that the pixel is associated with an undifferentiated cell may be relatively low compared to the differentiated category (e.g., 51%, 55%, 60%, etc.), indicating that the cells are transitioning to a differentiated state. Therefore, it may be possible to adjust the treatment of the culture to support and / or prevent this transition. An example of adjusting the treatment of the culture is described herein with respect to at least FIG. 2C.

[0132] In operation 254, the process 250 includes determining whether there is another pixel. If so, operations 252-254 are repeated for the other pixel.

[0133] Figure 2C shows an example 280 of adjusting the treatment of a culture based on the identified image segments. It should be understood that Figure 2C shows an example, and that embodiments of the technology described herein are not limited to any particular manner of adjusting the treatment of a culture.

[0134] Example 280-1 involves modifying the method by which one or more substances are added to a culture to affect the growth of the culture.

[0135] In some embodiments, modifying the manner in which one or more substances are added to the culture includes modifying the type of substance added to the culture. For example, modifying the type of substance added to the culture can include adding serum, inorganic salts, buffers, carbohydrates, amino acids, vitamins, proteins, peptides, fatty acids, lipids, trace elements, antibiotics, and / or growth factors. Examples of substances are described at least in Chen, G. et al., "Chemically defined conditions for human iPSC derivation and culture," Nat. Methods. 2011;8(5):424-429, the entire contents of which are incorporated herein by reference. In some embodiments, the substances differ based on the type of cells cultured and / or the cell fate of the undifferentiated stem cells in the culture.

[0136] In some embodiments, modifying the way one or more substances are added to the culture comprises modifying the rate at which the substances are added, which may include increasing or decreasing the rate at which a particular substance is added and / or calculating the rate at which the substance can be added.

[0137] In some embodiments, modifying the manner in which one or more substances are added to the culture comprises modifying the amount of the substance added, which can include, for example, increasing or decreasing the amount of one or more substances added to the culture and / or quantifying the amount of the substance that can be added.

[0138] As a non-limiting example, maintaining iPSCs in an undifferentiated state involves adding substances, such as growth factors, to the culture at specified time intervals. If the iPSCs begin to differentiate, the type, amount, or rate of the substances added can be altered. In some embodiments, the techniques described herein with respect to at least processes 200 and 250 can be used to identify the presence and / or relative amount of a culture in a differentiated state. The presence and / or amount of differentiation can be used to inform whether to add different substances, the amounts of substances that can be added, and the rate at which they can be added to prevent further differentiation and maintain the iPSCs in an undifferentiated state. For example, if the amount of the culture associated with the non-iPSC cell category (compared to the amount associated with the iPSC cell category) is greater than 5%, greater than 10%, greater than 15%, greater than 20%, greater than 25%, or greater than 30%, the method of adding a substance can be adjusted. For example, adjusting the method by which a substance is added can include increasing the amount of fibroblast growth factor 2 (FGF2) in the culture.

[0139] As another example, iPSCs can be differentiated into different cell types by monitoring the cell culture and controlling the cell culture environment. For example, iPSCs can be differentiated toward the forebrain. Forkhead box G1 (FOXG1) is a forebrain marker, and inhibition of WNT signaling has been shown to improve FOXG1 induction. In some embodiments, the techniques described herein with respect to at least processes 200 and 250 can be used to determine the amount of culture corresponding to the FOXG1 marker cell category. For example, the techniques can be used to process an image after FOXG1 staining to determine, for pixels in the image, the likelihood that the pixel is associated with the FOXG1 marker cell category. If the amount of culture corresponding to the FOXG1 marker cell category is below a threshold amount, a WNT inhibitor, XAV939, can be added to the culture to improve FOXG1 induction. An example of neural differentiation is described in Maroof, et al., "Directed differentiation and functional maturation of cortical interneurons from human embryonic stem cells," Cell Stem Cell. 2013;12(5):559-572, the entire contents of which are incorporated herein by reference.

[0140] Example 280-2 includes discarding cells of a culture. In some embodiments, discarding cells of a culture includes discarding one, some, or all of the cells of a culture. In some embodiments, cells may be discarded if they are associated with one or more predetermined cell categories or if a certain amount of a culture is associated with one or more predetermined cell categories. For example, if the amount of a culture associated with a predetermined cell category exceeds a threshold, the culture may be discarded. Additionally or alternatively, cells corresponding to pixels associated with a predetermined cell category may be discarded. As a non-limiting example, the predetermined cell categories may include a non-iPSC call category, and a culture may be discarded if the amount of non-iPSC cell categories (compared to the amount associated with the iPSC cell categories) is greater than 5%, greater than 10%, greater than 15%, greater than 20%, greater than 25%, or greater than 30%. Additionally or alternatively, non-iPSCs may be identified and discarded.

[0141] In some embodiments, discarding cells of a culture includes discarding a culture if it is of lower quality than other cultures. For example, a well plate may have several wells (e.g., two or more), each containing a culture. In some embodiments, a user and / or an automated system may rank the cultures (e.g., using the techniques described herein). If the culture is not ranked high enough, the culture may be discarded. For example, on a plate with 96 wells, 10 of the cultures may be selected for further expansion, while the remaining cultures may be discarded.

[0142] Example 280-3 involves passaging cells of a culture. Passaging cells is the procedure of dividing (or splitting) cells of a culture into new cultures to promote further growth.

[0143] In some embodiments, regulating the passaging of a culture includes determining when to split the culture, determining the number of new cultures to split the culture into, and / or determining whether the culture has overgrown and can be discarded. In some embodiments, these determinations may depend on one or more factors, such as, for example, the number of cells in the culture, the conditions used to maintain the cell type, the differentiation protocol, the number and type of downstream assays, etc. The number of cells in the culture may be determined, for example, using the techniques described herein with respect to at least operation 204 of process 200.

[0144] In some embodiments, the culture is ready to split when the number of cells is equal to or approximately equal to (e.g., within 1%, 2%, 3%, 5%, 7%, 10%, etc.) a predetermined number. The predetermined number can be any number of cells set by one or more users, by an automated system, and / or by a semi-automated system. For example, the techniques may include determining that a culture is ready to divide when the number of cells in the culture is within the range of 17,000-23,000 cells, 17,500-22,500 cells, 18,000-22,000 cells, 18,500-21,500 cells, 19,000-21,000 cells, 19,500-20,500 cells, 19,700-20,300 cells, 19,800-20,200 cells, 19,900-20,100 cells, 19,950-20,050 cells, or any other suitable cell number range.

[0145] In some embodiments, a culture may be discarded when it has overgrown and the number of cells exceeds a threshold amount. For example, the techniques may include determining that a culture is overgrown and may be discarded when the number of cells in the culture exceeds 23,000 cells, 23,500 cells, 24,000 cells, 24,500 cells, 25,000 cells, 25,500 cells, 26,000 cells, 26,500 cells, 27,000 cells, or any other suitable number of cells.

[0146] In some embodiments, this technology can be used to determine the number of new cultures into which cells are split based on the proliferation rate of the cells in the culture. For example, cells with faster proliferation rates can be split into more new cultures. By monitoring the number of cells in a culture over time (e.g., by analyzing images captured over time), the proliferation rate can be quantified and used to inform the split ratio.

[0147] Example 280-4 includes recovering the cells for downstream applications. In some embodiments, recovering the cells includes monitoring and / or inducing cell differentiation. For example, this can include determining when the cells should be differentiated and / or whether differentiation was successful.

[0148] As a first example, the cardiomyocytes have a morphology indicative of mature, healthy cardiomyocytes. In particular, organized cardiomyocyte sarcomeres are indicative of mature, healthy cardiomyocytes, while disorganized cardiomyocyte sarcomeres are indicative of immature, unhealthy cardiomyocytes. In some embodiments, image segmentation techniques described herein with respect to at least FIGS. 2A and 2B can be used to identify sarcomere-associated pixels and non-sarcomere-associated pixels. The resulting image segments can be used to determine (e.g., based on user observation and / or using one or more computing devices) whether the sarcomeres are organized. If the sarcomeres are determined to be organized, the culture can be identified as having differentiated successfully and can be used for downstream experiments. If the sarcomeres are determined to be disorganized, the culture can be identified as having differentiated unsuccessfully and cannot be used for downstream experiments.

[0149] As another example, cell clustering can be used to determine when cells are capable of differentiation. As referred to herein, a cluster of cells can include a group of two or more cells that are in contact with each other. In some embodiments, cell clustering is determined using techniques described herein with respect to at least operations 204 and 206 of process 200. For example, an estimated density map can be used to predict the location of cells within a dish, which can then be used to determine whether the cells have formed clusters. Additionally or alternatively, image segmentation can be used to identify clusters by identifying pixels associated with cells versus pixels associated with non-cells.

[0150] In some embodiments, determining that cells are ready for differentiation includes determining whether the amount of culture corresponding to cell clusters compared to the amount of culture corresponding to non-clusters exceeds a threshold. For example, this may include determining whether the amount of culture corresponding to clusters exceeds at least 40%, at least 45%, at least 50%, at least 55%, at least 60%, at least 65%, at least 70%, or any other suitable threshold proportion (e.g., proportion or percentage) of the culture. If the amount exceeds the threshold, the cells may be identified as ready for differentiation. If the amount does not exceed the threshold, the cells may be identified as not ready for differentiation.

[0151] As yet another example, adjusting the treatment of a culture can include selecting cells and / or cultures for downstream analysis, which in some embodiments includes selecting cultures of cells with a particular proliferation rate, cells in the absence of one or more markers, cells in the presence of one or more markers, cells in the absence of debris, cells in the absence of necrosis, and / or cells with a particular level of target expression.

[0152] Determining whether cells have a particular proliferation rate can include, for example, monitoring the number of cells in the culture over time. In some embodiments, determining the number of cells in the culture includes using at least the techniques described herein with respect to operation 204 of process 200. In some embodiments, a culture is selected for downstream analysis if the number of cells in the culture is equal to or greater than the number of cells in another culture over a particular time interval.

[0153] Determining whether cells of the culture contain a particular marker includes identifying pixels associated with the marker in an image of the culture. In some embodiments, identifying pixels associated with the marker includes using image segmentation techniques described herein with respect to at least act 206 and process 250 of process 200. For example, the image of the culture may be provided to a machine learning model trained to predict, for each of a plurality of pixels in the image, the likelihood that the pixel is associated with a marker. In some embodiments, the culture is selected for downstream analysis if one or more particular markers are present or absent in the sample. In some embodiments, the culture is selected for downstream analysis if the amount of a particular marker in the culture is above or below a threshold amount.

[0154] For example, SSEA-3 and SSEA-4 are markers associated with pluripotency. Thus, to maintain iPSCs in culture, cultures can be monitored for SSEA-3 and SSEA-4. For example, in some embodiments, images of cultures stained for SSEA-3 and SSEA-4 can be obtained. The images can be processed using techniques described herein with respect to at least processes 200 and 250 to determine, for pixels in the image, the likelihood that each pixel is associated with the SSEA-3 and / or SSEA-4 marker cell category. If the amount of culture associated with the SSEA-3 and / or SSEA-4 marker cell category is below a threshold amount or a predetermined amount, FGF can be added to the cell culture to maintain the cells in an undifferentiated state and to increase SSEA-3 and SSEA-4.

[0155] Similarly, in some embodiments, determining the cell viability of the culture (e.g., whether the culture contains debris and / or necrosis) includes identifying pixels associated with debris and / or necrosis in an image of the culture. In some embodiments, identifying pixels associated with debris and / or necrosis includes using image segmentation techniques described herein with respect to at least operation 206 and process 250 of process 200. For example, the image of the culture may be provided to a machine learning model trained to predict, for each of a plurality of pixels in the image, the likelihood that the pixel is associated with debris and / or the likelihood that the pixel is associated with necrosis. In some embodiments, if the culture does not contain debris and / or necrosis, the culture is selected for downstream analysis. In some embodiments, the culture is selected for downstream analysis if the amount of culture associated with debris and / or necrosis is below a threshold amount.

[0156] In some embodiments, the target expression level of cells of the culture is measured by fluorescence. For example, this may include determining an amount of culture associated with fluorescence. In some embodiments, determining the amount of culture associated with fluorescence includes identifying pixels in a fluorescent image of the culture associated with fluorescence using techniques described herein with respect to at least operation 206 and process 250 of process 200. For example, the fluorescent image of the culture may be provided as input to a machine learning model trained to predict, for each of a plurality of pixels in the image, the likelihood that the pixel is associated with fluorescence. In some embodiments, if the amount of culture associated with fluorescence exceeds a certain threshold, the culture may be selected for downstream analysis.

[0157] Example 280-5 includes adjusting the processing of a second culture. For example, the second culture can include a future culture or a culture growing in parallel with the first culture. In some embodiments, adjusting the processing of the second culture includes modifying how one or more substances are added to the second culture, discarding the second culture, passaging the cells of the second culture, and / or making a decision about the second culture regarding harvesting the cells for a downstream application.

[0158] As a first example, the techniques described herein may be used to monitor the growth rate of a cell culture. For example, process 200 described herein with respect to FIG. 2A may be used to identify the amount of culture corresponding to cells and the amount of culture corresponding to background (e.g., non-cells). The identified amounts may be used to determine percent confluency. For example, if 80% of the culture is identified as corresponding to cells, the culture may be considered 80% confluent.

[0159] In some embodiments, cell cultures may grow rapidly and become overconfluent, which can result in cell damage (e.g., cells may begin to die). For example, if a cell culture exceeds a threshold percent confluency (e.g., 75%, 80%, 85%, 90%, etc.), cells may be damaged. Thus, in some embodiments, percent confluency can inform decisions regarding the treatment of the cell culture and / or the treatment of future cultures. For example, if the percent confluency exceeds a threshold percentage and the cells are not recoverable, the cell culture may be discarded. However, if the cells are recoverable, percent confluency can be used to inform decisions related to splitting the cells into one or more future cultures. For example, if the percent confluency exceeds a threshold percentage, this may indicate that the cells are growing rapidly and that the cells may be split into a relatively large number of new cultures to accommodate this growth. Additionally or alternatively, in some embodiments, the rapid growth rate of the cells may indicate that future cell cultures may be split at an earlier time point to prevent them from becoming overconfluent.

[0160] As another example, a cell culture may have one or more predetermined cells and / or conditions. In some embodiments, the presence of one or more predetermined cells and / or conditions is used to inform decisions regarding the treatment of future cultures. The predetermined cells and / or conditions may include any cells or conditions associated with any given cell culture. For example, if a given cell culture includes a differentiated culture and an undifferentiated culture, the predetermined cells may include differentiated cells and undifferentiated cells. In some embodiments, future cultures may be treated to avoid one or more predetermined cells and / or conditions (e.g., the cell culture being overconfluent).

[0161] For example, during hematopoietic cell proliferation, the cells become migratory and prone to dissociation. In some embodiments, the techniques described herein may be used to monitor a culture and identify the amount of the culture corresponding to dissociated cells. For example, process 200 described herein with respect to FIG. 2A may be used to identify the amount of the culture corresponding to the dissociated cell category, the amount of the culture corresponding to the adherent cell category, and any other suitable category (e.g., background cell category). The percentage of dissociated cells in a culture may be identified by comparing the amount of the culture corresponding to the detached cell category with the amount of the culture corresponding to the adherent cell category.

[0162] To prevent such dissociation, cell culture dishes can be coated with an adhesion matrix. However, if the amount of adhesion matrix is insufficient, the cells may still dissociate. In some embodiments, if a specified percentage of detached cells exceeds a threshold, the culture can be discarded. For example, if at least 45%, 50%, 55%, 60%, 70%, or any other suitable percentage of the culture corresponds to the dissociated cell category, the culture can be discarded. This may indicate that the amount of adhesion matrix coating can be increased for future cultures of hematopoietic cells to prevent such dissociation.

[0163] As another example, during neural differentiation, the culture may be monitored for the presence of mesenchymal cells and / or neurons. In some embodiments, the techniques described herein may be used to monitor the culture and identify the amount of the culture corresponding to mesenchymal cells and / or neurons. For example, process 200 described with respect to FIG. 2A may be used to identify the amount of the culture corresponding to the mesenchymal cell category, the amount of the culture corresponding to the neuronal cell category, and any other suitable category (e.g., background cell category). The proportion of mesenchymal cells in the culture may be identified by comparing the amount of the culture corresponding to the mesenchymal cell category with the amount of the culture corresponding to the neuronal cell category.

[0164] If the specified proportion of mesenchymal cells increases, this indicates that neural differentiation was not successful. In particular, in some embodiments, neural differentiation may fail if the concentration of SMAD inhibitors and / or fibroblast growth factor (FGF) is inappropriate. Therefore, the conditions of future cell cultures can be modified (e.g., by adjusting the concentration of SMAD inhibitors and / or FGF) to enable successful neural differentiation.

[0165] Additionally or alternatively, in some embodiments, the relationship between the condition of one or more cultures and the quality of those cultures can be monitored to select conditions for future cultures. For example, neural differentiation may result in a high percentage (e.g., 90%, 95%, 97%, 98%, 99%, 100%, etc.) of neurons and a low percentage (e.g., 0%, 1%, 2%, 3%, 5%, 10%, etc.) of mesenchymal cells. In some embodiments, neural differentiation is attempted in cell cultures with different cell culture conditions (e.g., concentrations of SMAD inhibitors and / or FGFs). Thus, different cultures may be grown with different ratios of neurons and mesenchymal cells. The relationship between culture conditions and the corresponding percentages of neurons and mesenchymal cells can be used to select cell culture conditions for future cell cultures that result in a high percentage of neurons and a low percentage of mesenchymal cells. For example, the cell culture conditions and the percentages of neurons and mesenchymal cells can be provided as input to a machine learning model trained to extrapolate the relationship between the two.

[0166] 5A shows an example of a cell imaging and incubation system according to some embodiments of the technology described herein. In some embodiments, the example cell imaging and incubation system 500 includes an incubator 502, a robotic system 504, and / or an imaging system 506.

[0167] In some embodiments, the incubator 502 is configured to hold one or more cultures of cells 520. The incubator 502 may automatically, semi-automatically, or manually regulate the environment in which the one or more cultures 520 are held. For example, the incubator 502 may include at least the incubator 194 described herein with respect to FIG. 1B.

[0168] In some embodiments, the robotic system 504 is configured to manipulate one or more cultures 520 and / or other materials used to regulate the processing of one or more cultures 520. As shown in Figure 5A, the robotic system 504 can be configured to move one or more cultures 520 between an incubator 502 and an imaging system 506. For example, the robotic system 504 can include at least the robotic system 196 described herein with respect to Figure 1B.

[0169] In some embodiments, imaging system 506 may include one or more imaging sensors configured to capture images of cells in one or more cultures 520. For example, the imaging sensors may be configured to capture any suitable type of image of the cells, such as bright field images, phase contrast images, and / or fluorescent images. For example, the imaging sensors may include at least imaging sensor 192 described herein with respect to FIG. 1B.

[0170] In some embodiments, a computing device (not shown) of the imaging and incubation system 500 is configured to process images obtained using the imaging system 506 to obtain image segments 508 and / or culture information 510. For example, the computing device may process the acquired images according to at least the techniques described herein with respect to Figures 2A and 2B.

[0171] In some embodiments, imaging and incubation system 500 is a fully or semi-automated system, meaning that it can automatically or semi-automatically (e.g., with user input) monitor the cell culture and adjust the processing of the cell culture to achieve a predetermined result. Techniques for operating example imaging and incubation system 500 are described herein with respect to at least FIG. 5B.

[0172] 5B is a flowchart of an exemplary process 550 for operating a cell imaging and incubation system in accordance with some embodiments of the techniques described herein. One or more operations of process 550 may be performed automatically by any suitable computing device. For example, the operations may be performed by a laptop computer, a desktop computer, one or more servers in a cloud computing environment, a computing device 180 described herein with respect to FIG. 1B, a computer system 1500 described herein with respect to FIG. 15, and / or in any other suitable manner.

[0173] In operation 552, process 550 includes determining whether a timing condition has been met. In some embodiments, determining whether a timing condition has been met includes determining whether a particular amount of time has elapsed since an initial time (e.g., since the cells were seeded into culture). For example, this may include determining whether 6 hours, 7 hours, 8 hours, 9 hours, 10 hours, 11 hours, 12 hours, 13 hours, 14 hours, 16 hours, 17 hours, 18 hours, or any other suitable amount of time has elapsed since the cells were seeded into culture; aspects of the technology described herein are not limited in this respect.

[0174] In some embodiments, determining whether a timing condition is met includes determining whether a particular amount of time has passed since the previous image was acquired (e.g., using imaging system 506 shown in FIG. 5A). For example, this may include determining whether 6, 7, 8, 9, 10, 11, 12, 13, 14, 16, 17, 18 hours, or any other suitable amount of time has passed since the cells were seeded into culture; aspects of the technology described herein are not limited in this respect.

[0175] If the timing condition is met at operation 552, process 550 proceeds to operation 554, where a robotic system is activated to move the culture to an imaging sensor. For example, the culture may be stored in an incubator, and the robotic system may be activated to move the culture from the incubator to an imaging system having one or more imaging sensors. In some embodiments, software running on a computing device (e.g., computing device 180 of FIG. 1B) operates the robotic system. For example, system automation module 162 of FIG. 1B may be configured to operate the robotic system.

[0176] Process 550 then proceeds to operation 556, where one or more imaging sensors are activated to acquire images of the plurality of cells of the culture. In some embodiments, the imaging sensors are activated in response to the placement of the culture within the imaging system. For example, the imaging system may include one or more sensors (e.g., presence detection sensors) configured to detect the presence of the culture within the imaging system. Additionally or alternatively, feedback from the robotic system may indicate that the robotic system has completed the task of moving the culture to the imaging sensor.

[0177] In some embodiments, the imaging sensor is activated after a timing condition is met. For example, determining whether the timing condition is met may include determining an amount of time that has elapsed since operation 554 began. In some embodiments, the timing condition depends on the configuration of the imaging and incubation system and / or the amount of time it takes for the robotic system to move the culture from the incubator to the imaging sensor.

[0178] In some embodiments, the imaging sensor operates according to user input, for example, a user may specify the time at which an image is captured.

[0179] In some embodiments, images acquired using the imaging sensor may be processed according to the techniques described herein with respect to at least FIGS. 2A and 2B.

[0180] In some embodiments, process 550 includes additional or alternative operations to those shown in FIG. 5B. For example, process 550 may further include, after operation 556, operating a robotic system to move the culture away from the imaging sensor. For example, the robotic system may operate to regulate processing of the culture of cells according to at least the techniques described herein with respect to FIGS. 2A-2C. Additionally or alternatively, the robotic system may be operated to return the culture to the incubator.

[0181] example Applications of iPSCs 6 illustrates an exemplary process for generating and using induced pluripotent stem cells (iPSCs) according to some embodiments of the technology described herein. As described herein, iPSCs 604 are a type of pluripotent stem cell derived from adult somatic cells (e.g., patient cells 602). iPSCs 604 have the ability to self-renew and differentiate into (i.e., give rise to) the many different cell types (e.g., cell type 606) that make up the adult human body. Thus, iPSCs 604 have many useful applications, such as, for example, human cell and developmental modeling 608, disease modeling 610, transplantation 612, drug and genetic screening 614, cell replacement therapy 616, drug selection 618, cell-based assays, biochemical assays, target validation and deisolation, drug response prediction, molecular refinement, and quality assurance.

[0182] Automated Platform Example An automated platform for monitoring iPSC proliferation was assembled by integrating a robotic arm, an automated incubator, an imaging cytometer, and automation and control software. Specifically, the automated platform included at least the following components: a PF3400 SCARA robot by Precise Automation, a Cytomat™ automated incubator by Thermo Scientific™, a Celigo image cytometer by Nexcelom Bioscience, and Overlord™ laboratory automation software by Peak Analysis and Automation.

[0183] Examples of culture quality evaluation technologies Experiments were conducted to assess whether brightfield data could be used to determine culture quality, and more specifically, to detect differences in cell morphology due to differentiation. Six iPSC lines (or clones) were generated using the same methods described herein, including at least those associated with the "Experimental Methods" section. Clones designated C2 through C7 were selected because of the varying degrees of clearly differentiated cells observed with continued passaging. PluriTest (Muller et al., 2011), an unbiased bioinformatics method for accurately identifying the pluripotency of human stem cells (Initiative, 2018), was used to establish quality scores and rankings for individual clones. Bulk mRNA sequencing was performed on all seven clones, and a PluriTest ranking from best to worst was obtained: LT, C7, C3, C4, C5, C6, and C2. Aspects of PluriTest are described in Muller et al., "A bioinformatic assay for pluripotency in human cells," Nat. Methods. 8, 315-317 (2011), the entire contents of which are incorporated herein by reference.

[0184] Different machine learning models were evaluated and selected to be most accurate in performing three artificial intelligence (AI) tasks for identifying clone quality: distinguishing between clones, classifying images as undifferentiated or differentiated, and semantic segmentation. The machine learning models evaluated included Xception, Resnet101, Inceptionv3, Densenet201, and Mobilenetv2.

[0185] Identification of clones Four models were evaluated for their ability to distinguish clones from a single image. The highest accuracy of this method among the four models tested was 79.36%, lower than expected if the AI could perfectly distinguish clones. However, the resulting confusion matrix could be used to generate a dendrogram based on the distances between clones (Figure 7B), which appears very similar to the gene expression dendrogram of the clones (Figure 7A), supporting the notion that simple bright-field images contain sufficient information to evaluate human pluripotent stem cells (hPSCs). Table 1 shows the performance of each model in distinguishing clones.

[0186] [Table 1]

[0187] The model was trained using MATLAB 2020b running on an AWS EC2 p3.2xlarge instance. To create the training set for clone identification, we selected 1,000 random images from each clone, for a total of 7,000. Of these, 60% were used for training, 20% for validation during training, and 20% for testing and evaluation of the trained model. The pre-trained model used for clone identification with transfer learning was Densenet201. The final model had a validation accuracy of 79.36%. Table 2 shows example hyperparameters and their corresponding values used to support transfer learning.

[0188] [Table 2]

[0189] Image Classification For image classification (i.e., identification of clone quality), training data was generated by classifying individual images of each clone into either pluripotent or differentiated categories until a balanced dataset of similarly sized classes was created for each clone. A model was then trained to distinguish between images that displayed undifferentiated hPSCs and images that contained differentiated cell types, achieving 95.87% accuracy.

[0190] The model was trained using MATLAB 2020b running on an AWS EC2 p3.2xlarge instance. To create a training set for image classification (i.e., identifying clone quality), images from several hPSC clones and an hPSC line from Life Technologies were used and separated into hPSC and non-hPSC classes. The original images were acquired in a 6-well plate, measuring 1958 × 1958, and tiled into four 979 × 979 images for training. A total of 2400 images were selected for each class. Of these, 60% were used for training, 20% for validation during training, and 20% for testing and evaluation of the trained model. The pre-trained model used to identify clone quality with transfer learning was Resnet101. The final model had a validation accuracy of 95.87%. Table 3 shows example hyperparameters and their corresponding values used to support transfer learning.

[0191] [Table 3]

[0192] Semantic Segmentation Thirty-two random images were selected for each clone, and the pixels were user-painted according to three classes: undifferentiated hPSCs, differentiated cells, and background. After training, the final model had an accuracy of 95.99%. Tables 4-1, 4-2, and 4-3 show the performance of each model in performing semantic segmentation on different image datasets to train and test the final semantic segmentation model.

[0193] [Table 4]

[0194] [Table 5]

[0195] [Table 6]

[0196] Results can be visualized as likelihood percentages for each class or as a combined pixel-paint image. Figure 7C shows pixel likelihood images for hPSC classes (middle row) and the corresponding images output from the semantic segmentation model (bottom row). The frequency of hPSC pixels relative to all cell-containing pixels was calculated to score and rank clones according to their pluripotency. The bar graph in Figure 7C shows the ratio of iPSCs (e.g., undifferentiated) to non-iPSCs. As shown in Figure 7D, the frequency of undifferentiated hPSC pixels strongly correlates with the percentage of triple-positive cells measured by flow cytometry, indicating that semantic segmentation successfully estimates the cellular composition of each hPSC clone and reports a quantitative score that can be used to rank clones, as shown in Table 5.

[0197] Because spatial information is also captured, semantic segmentation is a superior method for hPSC quality assessment compared to other models. While there are examples of image-based classification of hPSCs using U-Net to assess the presence of differentiated cells, these examples cannot assign classification at single-pixel resolution, a clear advantage of using semantic segmentation.

[0198] [Table 7]

[0199] The model was trained using MATLAB 2020b running on an AWS EC2 p3.2xlarge instance. To create a training set for segmenting hPSCs, non-hPSCs, and background in images, 32 random images of each hPSC clone were selected, for a total of 224 images. Of these, 60% were used for training, 20% for validation during training, and 20% for testing and evaluation of the trained model. Pixel labels were created by labeling pixels as hPSCs, non-hPSCs, or background using MATLAB Image Labeler. The semantic segmentation network used to train this model was Deeplab v3+ and above, and the base pre-trained network was Resnet50. The final model had a validation accuracy of 95.99%, a weighted intersection over union (IoU) score of 0.94, and an average marginal F1 (BF) score of 0.792. IoU and BF scores were calculated on the training dataset. IoU is the ratio of correctly classified pixels to the total number of ground truth and predicted pixels in that class. The BF score indicates how well the predicted boundaries for each class align with the true boundaries. Table 6 shows examples of hyperparameters and their corresponding values used in transfer learning.

[0200] [Table 8]

[0201] Examples of density estimation techniques Experiments were conducted to evaluate machine learning techniques for predicting the number of hPSCs within a colony. A convolutional neural network (CNN) was used to generate a three-dimensional density map in which density is distributed along the x and y coordinates of the image, thereby generating both an accurate count and physical location of cells. Aspects of the CNN are described in Sindagi, V.A. and Patel, V.M.J., "CNN-based Cascaded Multi-task Learning of High-level Prior and Density Estimation for Crowd Counting," arXiv:1707.09605 (2017), which is incorporated herein by reference in its entirety.

[0202] To test whether this CNN could be trained to detect hPSCs at single-cell resolution, we used an automated microscope, such as the one described in the "Example of an Automated Platform" section, to obtain aligned bright-field and Hoechst-stained images of hPSC colonies plated in a 96-well format. Examples of aligned bright-field and Hoechst-stained images are shown in Figures 8A and 8B, respectively. Center points were identified using the Hoechst-stained images and converted to density maps using Image-J / FIJI to serve as ground truth for training the model. The corresponding density map is shown in Figure 8C.

[0203] As shown in Figure 9, the image-based CNN consists of two parallel processes filtered through a convolutional layer 904. One half (top) classifies the image 902 with a 10-count classifier, with the intention of approximating the count and classifying the image based on the approximate number of cells in the field of view. This information is used to inform the second half (bottom), which generates a density map 910 with local densities maxima representing individual nuclei. Through repeated training rounds (epochs), connections between the CNN's individual layers are strengthened or weakened based on the similarity between the model-generated density map and a ground-truth density map 908 derived from the Hoechst-stained image 906. Images are passed through intermediate models captured at even epochs, and the results are output as density maps to capture the CNN training process. The model underwent a trial-and-error phase from epochs 8 to 48, including an inverted density map in which densities were assigned to empty portions of the dish before finding the correct approximate distribution and identifying the appropriate density localization. By epoch 60, the morphology and location of the total colony were correctly identified and further refined through iterations. This learning process corresponded to minimizing the training loss, mean error, and mean squared error. Training was stopped, and the model from epoch 680 was selected as the model with the smallest mean average error and mean squared error. The resulting model was found to ignore microscopic artifacts, including particulates at the bottom of the microwell plate, well edges, air bubbles, and focal plane variations, and was used in subsequent experiments (referred to as the optimized model) without augmenting the original brightfield image (e.g., as shown in Figure 8A).

[0204] The optimized model is data-rich and can identify the relative positions of cells within a dish, generate counts within a specific field of view, and subdivide them to summarize larger areas by calculating the area under the curve. To illustrate this point, the image can be sliced at a given horizontal coordinate and the gray values plotted. Figures 10A, 10B, 11A, and 11B show a comparison of feature size and sharpness between the original low-contrast image and the density map revealed by the model, demonstrating the information transformation performed by the trained model. As shown in Figure 11C, the optimized model was evaluated using newly captured data, correlating the ground truth provided by fluorescence-based object detection with the model results, which showed an R-squared value of 0.994.

[0205] Beyond the basic tasks of counting cells and detecting differentiated contaminants, the generated density map contains information that enables additional analyses. For example, it can be used to map cell location, detect confluency, and measure internuclear distance. This method can be easily incorporated into an automated process that can be scaled and scaled to meet the demands of automated hPSC culture. The cell counting and quality assessment method can be adapted to various hPSC lines and microscopes through training new models or through transfer learning using as few as one 96-well plate of hPSCs. This method can be used as a means to establish standards for training individuals to perform hPSC tissue culture work. This method provides a rapid quality control assessment of cells being cultured for use as cell replacement therapy and enhances existing validation methods such as gene expression profiling, flow cytometry, and immunocytochemical analysis.

[0206] Training data was acquired using the automated system described herein, at least in connection with the "Example of an Automated Platform" section. Full-well images from 27 barcoded 96-well tissue culture plates were recorded every 12 hours. Images were automatically uploaded to the cloud, and cell counts and heat maps were calculated to monitor cell growth over time and demonstrate the in-line performance of the automated system over time for hPSC expansion. Figure 12A shows cell counts over time. Figure 12B shows a heat map of the culture plate showing the relative number of cells in each well over time. Split decision training was performed by classifying images of hPSCs that could be fed, could be split, or were considered overconfluent. The classifications are shown in Figure 13. Limitations of the model were identified by either reducing the image area of the dish or reducing the resolution by merging pixels.

[0207] The optimized model was used to generate density maps of seven hPSC lines, demonstrating its accuracy and broad applicability to additional hPSC lines compared with Hoechst-stained images, as shown in Figure 14. A normal qq plot of cell counts for the hPSC lines was used to determine how closely the sampling of individual fields fit a normal distribution. As an example, when cell differentiation leads to confluency across all fields, as seen for C2, the result was a "moderate effect," where images with median counts were observed more frequently, seen as a downward concave curve. In contrast, the upward concave curve depicted for C6 represents a bimodal distribution compared to a normal distribution.

[0208] The model was trained using Amazon SageMaker. An ml.p3.8xlarge instance was used for training. The custom model and training script from Sendagi et al. (Sindagi and Patel, 2017) were packaged into a Docker image according to the SageMaker specification. All hyperparameters used during training were kept the same as those from Sendagi, et al. (Sindagi and Patel, 2017). The training time was approximately 4 hours.

[0209] A training dataset was assembled by randomly selecting 3,000 1,958 x 1,958 images from a larger dataset of 4,608 images acquired from three 96-well plates. Each image was then reduced in size to 256 x 256 by taking a random crop from the image. The 3,000 images were then manually sorted to remove images that were out of focus or had defects that prevented the nucleus segmentation algorithm from working properly. After manually sorting the 3,000 images, 2,375 were selected for training. The fluorescence channels from each image were run through a segmentation algorithm to find the nucleus center point. Aspects of the segmentation algorithm are described in Wang, Y. et al., "Segmentation of the Clustered Cells with Optimized Boundary Detection in Negative Phase Contrast Images," PloS One, 10 (2015), the entire text of which is incorporated herein by reference. These center points were then used to create a ground truth density map as described in (Sindagi and Patel, 2017). The training dataset was then further split into training and validation datasets, with 80% of the data used for training and 20% of the data used for validation during training.

[0210] Experimental Method tissue culture hPSC lines were obtained from Life Technologies (Thermo Fisher Scientific) and maintained between passages 25 and 45. The cell lines are episomal reprogrammed lines derived from CD34+ hematopoietic somatic cells. hPSCs were fed daily using mTeSR1, passaged using ReLeSR, and attached to 10 cm and 96-well tissue culture plates coated with hESC-qualified Matrigel (Corning). hPSCs were assessed for pluripotency by flow cytometry, karyotypic abnormalities, and mycoplasma to control the quality of the culture.

[0211] Reprogrammed hPSC clones were derived from CD34+ umbilical cord blood cells (STEMCELL Technologies). Reprogramming was performed using the CytoTune-IPS2.0 Sendai Reprogramming Kit (Invitrogen) according to the manufacturer's instructions. Once reprogramming was complete, clones were fed daily using mTeSR1, passaged using ReLeSR, and allowed to attach to 6-well tissue culture plates coated with hESC-qualified Matrigel (Corning).

[0212] Cell plating and staining for training dataset hPSC lines were dissociated from 10 cm dishes using ReLeSR and plated at equal densities onto standard flat-bottom 96-well microplates (Corning) coated with hESC-qualified Matrigel. Plates were subsequently fixed at daily intervals. All plates were fixed for 20 minutes with a final concentration of 3.7% formaldehyde by adding an equal volume to the 7.4% formaldehyde medium already in the wells. To stain cell nuclei, a staining solution was made by diluting Hoechst 33342 (Molecular Probes) 1:5000 in PBS and incubating for 15 minutes in the dark at room temperature. After incubation, the staining solution was removed, the cells were washed three times with PBS, and a sufficient volume of PBS (approximately 200 μL) for imaging was added to the wells.

[0213] Imaging and acquisition settings Images were acquired on a Celigo imaging cytometer (Nexcelom Bioscience). Brightfield illumination is a 1 LED-based enhanced brightfield imaging channel with uniform well illumination. 4 LED-based fluorescence channels are also present. A large-chip CCD camera with a galvanometric mirror and F-theta lens is used to acquire images at a resolution of 1 μLm / pixel. All images are at 10x magnification.

[0214] Training plates for the cell counting and density map models were imaged in both brightfield and blue channels. All other plates were imaged in brightfield. Imaging settings: Brightfield 50 ms exposure; Hoechst 250 ms exposure, excitation 377 / 50, emission 470 / 22.

[0215] Automated run until confluence Human hPSC lines were dissociated from 10 cm dishes using reLeSR and plated at equal densities onto 96-well microplates coated with hESC-competent Matrigel. After plating the 96 wells, the microplates were loaded into a prototype automated system's Cytomat™ automated incubator. Using Overlord™ automated software, the plates were configured to image all plates and upload those images to an AWS S3 bucket every 12 hours. The images were run through a model on AWS. Cell growth was tracked using the output of the density estimation technique described herein, including cell counts, heat maps, and growth curves. Plates were maintained until cells reached confluence.

[0216] Continuous execution hPSC lines were dissociated from 10 cm dishes using reLeSR and plated at four different densities into each of 96-well microplates coated with hESC-competent Matrigel. After plating the 96 wells, the microplates were loaded into a prototype automated system's Cytomat™ automated incubator. Using Overlord™ automated software, the plates were configured to image all plates and upload those images to AWS every 12 hours. The output of the density estimation technique described herein was used to determine when plates were ready for splitting and the split ratio to be used to equilibrate cell density across each microplate.

[0217] FIG. 15 illustrates an exemplary implementation of a computer system 1500 that can be used in connection with any embodiment of the technology described herein (e.g., the methods of FIGS. 2A, 2B, and 5B, etc.). The computer system 1500 includes one or more processors 1510 and one or more articles of manufacture that include a non-transitory computer-readable storage medium (e.g., memory 1520 and one or more non-volatile storage media 1530). The processor 1510 can control the reading and writing of data from the memory 1520 and the non-volatile storage 1530 in any suitable manner, and aspects of the technology described herein are not limited to any particular technology for writing or reading data. To perform any of the functions described herein, the processor 1510 can execute one or more processor-executable instructions stored in one or more non-transitory computer-readable storage media (e.g., memory 1520), which function as non-transitory computer-readable storage media that store processor-executable instructions for execution by the processor 1510.

[0218] Computer system 1500 may also include a network input / output (I / O) interface 1540 that allows the computing device to communicate with other computing devices (e.g., over a network), and may also include one or more user I / O interfaces 1550 that allow the computing device to provide output to and receive input from a user. The user I / O interfaces may include devices such as a keyboard, a mouse, a microphone, a display device (e.g., a monitor or touchscreen), speakers, a camera, and / or various other types of I / O devices.

[0219] The above-described embodiments can be implemented in a variety of ways. For example, the embodiments may be implemented using hardware, software, or a combination thereof. If implemented in software, the software code may be executed on any suitable processor (e.g., a microprocessor) or collection of processors, whether provided on a single computing device or distributed across multiple computing devices. It should be understood that any component or collection of components that perform the functions described above can be generally thought of as one or more controllers that control the functions described above. The one or more controllers can be implemented in a variety of ways, such as dedicated hardware or general-purpose hardware (e.g., one or more processors) that are programmed using microcode or software to perform the functions described above.

[0220] In this regard, it should be understood that one implementation of the embodiments described herein includes at least one computer-readable storage medium (e.g., RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disk (DVD) or other optical disk storage, magnetic cassette, magnetic tape, magnetic disk storage or other magnetic storage device, or other tangible, non-transitory computer-readable storage medium) encoded with a computer program (i.e., a plurality of executable instructions) that, when executed on one or more processors, performs the above-described functions of one or more embodiments. The computer-readable medium may be transportable such that the program stored thereon may be loaded into any computing device to implement aspects of the technology described herein. Furthermore, it should be understood that reference to a computer program that, when executed, performs any of the above-described functions is not limited to an application program running on a host computer. Rather, the terms computer program and software are used herein in a generic sense to refer to any type of computer code (e.g., application software, firmware, microcode, or any other form of computer instructions) that can be used to program one or more processors to implement aspects of the technology described herein.

[0221] The foregoing description of implementations has been provided for illustration and description, and is not intended to be exhaustive or to limit implementations to the precise forms disclosed. Modifications and variations are possible in light of the above teachings or may be acquired from practice of the implementations. In other implementations, the methods depicted in these figures may include fewer operations, different operations, operations in a different order, and / or additional operations. Moreover, non-dependent blocks may be performed in parallel.

[0222] It will be understood that the exemplary aspects as described above may be implemented in various forms of software, firmware, and hardware in the implementations shown in the figures. Furthermore, certain portions of the implementation may be implemented as "modules" that perform one or more functions. The modules may include hardware, such as a processor, an application specific integrated circuit (ASIC), or a field programmable gate array (FPGA), or a combination of hardware and software.

[0223] While several aspects and embodiments of the technology described in this disclosure have been described, it will be understood that various alterations, modifications, and improvements will readily occur to those skilled in the art. It is intended that such alterations, modifications, and improvements be made within the spirit and scope of the technology described herein. For example, those skilled in the art will readily envision numerous other means and / or structures for performing the functions and / or obtaining the results and / or one or more advantages described herein, and each such variation and / or modification is deemed to be within the scope of the embodiments described herein. Those skilled in the art will recognize or be able to ascertain using no more than routine experimentation many equivalents to the specific embodiments described herein. Accordingly, the foregoing embodiments are presented by way of example only, and it will be understood that, within the scope of the appended claims and their equivalents, embodiments of the invention may be practiced otherwise than as specifically described. Furthermore, any combination of two or more features, systems, articles, materials, kits, and / or methods described herein is within the scope of the present disclosure, unless such features, systems, articles, materials, kits, and / or methods are mutually inconsistent.

[0224] The above-described embodiments may be implemented in a variety of ways. One or more aspects and embodiments of the present disclosure involving the performance of a process or method utilize program instructions executable by a device (e.g., a computer, processor, or other device) to perform or control the performance of the process or method. In this regard, various inventive concepts may be embodied as a computer-readable storage medium (or multiple computer-readable storage media) (e.g., computer memory, one or more floppy disks, compact disks, optical disks, magnetic tapes, flash memory, circuitry in field programmable gate arrays or other semiconductor devices, or other tangible computer storage media) encoded with one or more programs that, when executed on one or more computers or other processors, perform methods that implement one or more of the various embodiments described above. The computer-readable medium may be transportable, allowing the program stored thereon to be loaded onto one or more different computers or other processors to implement the various aspects described above. In some embodiments, the computer-readable medium may be non-transitory.

[0225] As used herein, the terms "program" or "software" are used generically to refer to any type of computer code or set of computer-executable instructions that can be used to program a computer or other processor to implement various aspects as described above. It will be further understood that, according to one aspect, one or more computer programs that, when executed, perform the methods of the present disclosure need not reside on a single computer or processor, but may be distributed in a modular manner among several different computers or processors to implement various aspects of the present disclosure.

[0226] Computer-executable instructions may be in many forms, such as program modules, executed by one or more computers or other devices. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform particular tasks or implement particular abstract data types. Typically, the functionality of the program modules may be combined or distributed as desired in various embodiments.

[0227] Additionally, data structures may be stored on computer-readable media in any suitable format. For ease of explanation, data structures may be depicted as having fields that are related by their position within the data structure. Such relationships may equally be achieved by assigning storage to the fields at locations within the computer-readable media that convey the relationship between the fields. However, any suitable mechanism may be used to establish relationships between information within fields of a data structure, including the use of pointers, tags, or other mechanisms for establishing relationships between data elements.

[0228] When implemented in software, the software code may be executed on any suitable processor or collection of processors, whether provided on a single computer or distributed across multiple computers.

[0229] A computer may also have one or more input and output devices. These devices can be used, among other things, to display a user interface. Examples of output devices that can be used to provide a user interface include a printer or display screen for visually displaying output, and a speaker or other sound-generating device for audibly displaying output. Examples of input devices that can be used in a user interface include a keyboard and a pointing device such as a mouse, touchpad, or digital tablet. As another example, a computer can receive input information through speech recognition or other audio formats.

[0230] Such computers may be interconnected by one or more networks of any suitable form, such as a local area network or a wide area network such as an enterprise network, an intelligent network (IN), or the Internet, etc. Such networks may be based on any suitable technology and operate according to any suitable protocol, and may include wireless networks, wired networks, or fiber optic networks.

[0231] Also, as described, some aspects may be embodied as one or more methods. The actions performed as part of a method may be ordered in any suitable manner. Thus, embodiments may be constructed in which actions are performed in an order different from that shown, which may include performing some actions simultaneously even though shown as sequential actions in an exemplary embodiment.

[0232] All definitions defined and used herein should be understood to take precedence over dictionary definitions, definitions in documents incorporated by reference, and / or ordinary meanings of the defined terms.

[0233] The indefinite articles "a" and "an," as used in this specification and claims, unless expressly indicated otherwise, should be understood to mean "at least one."

[0234] The term "and / or," as used in the specification and claims, should be understood to mean "either or both" of the elements so conjoined, i.e., elements that are present conjunctively in some cases and disjunctively in other cases. Multiple elements listed with "and / or" should be construed in the same manner, i.e., "one or more" of the elements so conjoined. Other elements may optionally be present other than the elements specifically identified in the "and / or" clause, whether related to those specifically identified elements or not. Thus, as a non-limiting example, a reference to "A and / or B," when used in conjunction with open-ended language such as "comprising," can, in one embodiment, refer to A only (optionally including elements other than B); in another embodiment, refer to B only (optionally including elements other than A); in yet another embodiment, refer to both A and B (optionally including other elements); and so forth.

[0235] As used herein and in the claims, the phrase "at least one" when used in reference to a list of one or more elements should be understood to mean at least one element selected from any one or more elements in the list of elements, but not necessarily including at least one of each element specifically set forth in the list of elements, and not excluding any combination of elements in the list of elements. This definition also allows for the optional presence of elements other than those specifically identified in the list of elements to which the phrase "at least one" refers, whether related to the specifically identified elements or not. Thus, as a non-limiting example, "at least one of A and B" (or, equivalently, "at least one of A or B" or, equivalently, "at least one of A and / or B") can refer in one embodiment to at least one (optionally including multiple) A's in the absence of B (and optionally including elements other than B); in another embodiment to at least one (optionally including multiple) B's in the absence of A (and optionally including elements other than A); in yet another embodiment to at least one (optionally including multiple) A's and at least one (optionally including multiple) B's (and optionally including other elements); etc.

[0236] In the claims and the above specification, all transitional phrases such as "comprising," "including," "carrying," "having," "containing," "involving," "holding," "consisting of," and the like, are understood to be open-ended, i.e., meaning including but not limited to. Only the transitional phrases "consisting of" and "consisting essentially of" shall be closed or semi-closed transitional phrases, respectively.

[0237] The terms "nearly," "substantially," and "about" may be used to mean, in some embodiments, within ±20% of a target value, in some embodiments, within ±10% of a target value, in some embodiments, within ±5% of a target value, and in some embodiments, within ±2% of a target value. The terms "nearly," "substantially," and "about" may include the target value.

Claims

1. A method for adjusting the processing of a culture, wherein the culture comprises a plurality of cells, each cell in the plurality of cells has one or more cell categories selected from a plurality of cell categories, the plurality of cell categories comprises a first cell category and a second cell category, and the method is Processing images of the plurality of cells in the culture, thereby identifying one or more cell categories of cells shown in the image from among the plurality of cell categories, and processing them accordingly, Segmenting the image into multiple image segments by assigning each individual pixel in the image to a corresponding cell category among the multiple cell categories, wherein the assignment includes determining each of the multiple values ​​corresponding to each of the multiple cell categories, each of which represents the likelihood that the individual pixel corresponds to a cell in each of the multiple cell categories. The aforementioned plurality of image segments are A first image segment including pixels related to cells of the first cell category and A second image segment containing pixels associated with cells of the second cell category. Including segmentation, Processing and Based on the plurality of image segments into which the image is segmented, the amount of cells in the culture corresponding to the first cell category is identified, Adjusting the treatment of the culture based on the aforementioned amount, Methods that include...

2. Assigning the individual pixels in the image to the corresponding cell category within the plurality of cell categories includes classifying the individual pixels according to a plurality of classes, where the first class of the plurality of classes corresponds to the first cell category, and the second class of the plurality of classes corresponds to the second cell category. The method according to claim 1, wherein classifying the individual pixels includes, for each of the individual pixels, selecting a class to classify the individual pixel based on the respective determined plurality of values.

3. The aforementioned assignment is performed using a trained machine learning model, and the aforementioned assignment is The method according to claim 1 or 2, comprising processing the image using the trained machine learning model to obtain, for each of the individual pixels, the respective plurality of values ​​corresponding to the respective plurality of cell categories.

4. The method according to claim 3, wherein the trained machine learning model comprises a deep neural network model comprising one or more convolutional layers, the deep neural network model comprising a cascade of deep neural network blocks, each of the deep neural network blocks comprising its respective deep convolutional neural network (CNN), and the trained machine learning model performs computations using atlas space pyramid pooling in at least part of it.

5. The process involves processing images of the plurality of cells in the culture to estimate the culture information of the culture, further comprising estimating that the culture information of the culture indicates the density of the cells in the culture. The method according to claim 1 or 2, wherein the segmentation of the image includes segmenting the image based on the culture information.

6. The culture information is used to identify the cell coordinates of the plurality of cells. The method according to claim 5, wherein segmenting the image based on the culture information includes providing the image and the coordinates as input to a trained machine learning model to obtain an output showing the respective likelihood that each of the individual pixels corresponds to one of the plurality of cell categories.

7. The process involves processing images of the plurality of cells in the culture to estimate the culture information of the culture, further comprising estimating that the culture information of the culture indicates the density of the cells in the culture. The method according to claim 1 or 2, wherein adjusting the treatment of the culture includes adjusting the treatment of the culture based on the culture information and the amount of cells in the culture corresponding to the first cell category.

8. The culture information is used to identify the coordinates of the cells shown in the image, and the processing of the culture is adjusted based on the culture information. Using the aforementioned coordinates, the location of one or more cells in the plurality of cells is identified, Removing cells from the aforementioned identified location, The method according to claim 7, including the method described in claim 7.

9. The method according to claim 7, wherein processing images of the plurality of cells in the culture to estimate the culture information includes estimating the number of the plurality of cells, the position of at least one cell in the plurality of cells, and / or the internuclear distance between at least two cells in the plurality of cells.

10. Adjusting the processing of the culture based on the culture information and the amount of cells in the culture corresponding to the first cell category is: The method according to claim 7, comprising outputting a recommendation indicating the time for subculturing one of the plurality of cells in the culture and / or the recommended number of new cultures for dividing the culture.

11. Adjusting the treatment of the cultured material is Based on the amount of the cells in the culture corresponding to the first cell category, output a recommendation to modify the method of adding one or more substances to the culture to affect the growth of the culture, or Modify the method of adding one or more substances to the culture in such a way as to affect the growth of the culture, or Based on the amount of the cells in the culture corresponding to the first cell category, output a recommendation for subculturing the multiple cells in the culture, or Passing the cells of the aforementioned culture, or Outputting a recommendation to discard one of the multiple cells in the culture, or The method according to claim 1 or 2, comprising discarding one of the plurality of cells in the culture.

12. Comparing the amount of the cells in the culture corresponding to the first cell category with a predetermined amount, Based on the comparison, the treatment of the second culture is adjusted so that the second culture has a predetermined amount of the first cell category. The method according to claim 1 or 2, further comprising:

13. The method according to claim 1 or 2, wherein the first cell category corresponds to inducible induced pluripotent stem cells (iPSCs) and the second cell category corresponds to non-iPSCs.

14. At least one non-temporary computer-readable storage medium on which executable instructions are encoded, wherein, when executed by at least one processor, the executable instructions cause the at least one processor to execute a method for regulating the processing of a culture, the culture comprising a plurality of cells, each cell having one or more cell categories selected from a plurality of cell categories, the plurality of cell categories comprising a first cell category and a second cell category, and the method is Processing images of the plurality of cells in the culture, thereby identifying one or more cell categories of cells shown in the image from among the plurality of cell categories, and processing them accordingly, Segmenting the image into multiple image segments by assigning each individual pixel in the image to a corresponding cell category among the multiple cell categories, wherein the assignment includes determining each of the multiple values ​​corresponding to each of the multiple cell categories, each of which represents the likelihood that the individual pixel corresponds to a cell in each of the multiple cell categories. The plurality of image segments include a first image segment containing pixels related to cells of the first cell category and a second image segment containing pixels related to cells of the second cell category, and the segmentation is performed accordingly. Processing and Based on the plurality of image segments into which the image is segmented, the amount of cells in the culture corresponding to the first cell category is identified, Adjusting the treatment of the culture based on the aforementioned amount, Non-temporary computer-readable storage media, including [specific type of storage medium].

15. A cell imaging and incubation system, An imaging sensor configured to acquire images of multiple cells in a culture, An incubator configured to incubate the culture, At least one processor, An executable instruction is encoded in at least one non-temporary computer-readable storage medium, The method comprises, and the executable instruction, when executed by at least one processor, causes the at least one processor to execute a method for regulating the processing of a culture, wherein the culture comprises a plurality of cells, each cell having one or more cell categories selected from a plurality of cell categories, the plurality of cell categories comprising a first cell category and a second cell category, and the method is Processing images of the plurality of cells in the culture, thereby identifying one or more cell categories of cells shown in the image from among the plurality of cell categories, and processing them accordingly, Segmenting the image into multiple image segments by assigning each individual pixel in the image to a corresponding cell category among the multiple cell categories, wherein the assignment includes determining each of the multiple values ​​corresponding to each of the multiple cell categories, each of which represents the likelihood that the pixel corresponds to a cell in each of the multiple cell categories. The plurality of image segments include a first image segment containing pixels related to cells of the first cell category and a second image segment containing pixels related to cells of the second cell category, and the segmentation is performed accordingly. Processing and Based on the plurality of image segments into which the image is segmented, the amount of cells in the culture corresponding to the first cell category is identified, Adjusting the treatment of the culture based on the aforementioned amount, A cell imaging and incubation system, including [specific feature / feature].

16. The cell imaging and incubation system according to claim 15, further comprising a robotic system configured to transfer the cultured material in the cell imaging and incubation system between cultivation in the incubator and imaging by the imaging sensor.

17. The cell imaging and incubation system according to claim 16, wherein the robotic system is configured to transfer the culture between culture and imaging when timing conditions are met.

18. The aforementioned method, When the aforementioned timing conditions are met, the robot system is activated to move the cultured material to the imaging sensor. The imaging sensor is activated to acquire images of the plurality of cells in the culture, The cell imaging and incubation system according to claim 17, further comprising:

19. The aforementioned method, The process further includes processing the images of the plurality of cells in the culture, thereby estimating the culture information of the culture, and the culture information of the culture indicates the density of the cells in the culture. The cell imaging and incubation system according to any one of claims 15 to 18, wherein segmenting the image includes segmenting the image based on the culture information.

20. The aforementioned method, The process further includes processing the images of the plurality of cells in the culture, thereby estimating the culture information of the culture, and the culture information of the culture indicates the density of the cells in the culture. The cell imaging and incubation system according to any one of claims 15 to 18, wherein adjusting the treatment of the culture includes adjusting the treatment of the culture based on the culture information and the amount of the cells in the culture corresponding to the first cell category.

21. The cell imaging and incubation system according to claim 20, further comprising using the culture information to estimate the number of the plurality of cells, the position of at least one cell of the plurality of cells, and / or the internuclear distance between at least two cells of the plurality of cells.

22. Adjusting the treatment of the culture means outputting a recommendation to modify the method of adding one or more substances to the culture to affect the growth of the culture, based on the amount of the cells in the culture corresponding to the first cell category, or A cell imaging and incubation system according to any one of claims 15 to 18, comprising outputting a recommendation to subculture one of the plurality of cells in the culture based on the amount of the cells in the culture corresponding to the first cell category.

23. A method for adjusting the processing of a culture, wherein the culture comprises a plurality of cells, each cell in the plurality of cells has one or more cell categories selected from a plurality of cell categories, the plurality of cell categories comprises a first cell category and a second cell category, and the method is Processing images of the plurality of cells in the culture, thereby identifying one or more cell categories of cells shown in the image from among the plurality of cell categories, and processing them accordingly, Segmenting the image into multiple image segments by assigning regions of the image to corresponding cell categories among the multiple cell categories, each of which regions includes two or more individual pixels in the image, and the assignment includes determining, for each of the regions, a set of values ​​corresponding to each of the multiple cell categories, each of which values ​​represents the likelihood that the region corresponds to a cell in each of the multiple cell categories. The plurality of image segments include a first image segment containing a region related to cells of the first cell category and a second image segment containing a region related to cells of the second cell category, and the segmentation is performed accordingly. Processing and Based on the plurality of image segments into which the image is segmented, the amount of cells in the culture corresponding to the first cell category is identified, Adjusting the treatment of the culture based on the aforementioned amount, A method that includes this.

Images