Imaging-based system for monitoring the quality of cells in culture

Abstract

Described herein are techniques for modulating the treatment of cell cultures having cells corresponding to different cell categories. Some techniques may be used with a cell imaging and incubation system that includes an imaging sensor configured to obtain an image of the culture and an incubator configured to incubate the culture. Adjustment of processing of the culture may be based on processing of an image of the culture obtained by an imaging sensor of the system. The processing may include segmenting an image of the culture into a plurality of image segments by assigning individual pixels of the image to corresponding cell categories. According to some embodiments, the techniques include determining an amount of the culture corresponding to a particular cell category based on the image segments. The amount of the culture corresponding to the particular cell category may provide information for modulation of the treatment of the culture.

Description

[0001] Related Applications

[0002] This application claims priority under 35 U.S.C. §119(e) to U.S. Provisional Application No. 63 / 392,422, filed on July 26, 2022, entitled “IMAGING-BASED SYSTEM FOR MONITORING QUALITY OF CELLS IN CULTURE,” the entire contents of which are incorporated herein by reference. Background Art

[0003] Cell culture refers to the growth of cells in a controlled environment, which can be used for many purposes, such as studying the effects of drugs, modeling disease, and studying genetic variation. Cell culture management involves making decisions about how to handle the cultures to produce high-quality cells. Summary of the invention

[0004] Some embodiments provide a method of regulating treatment of a culture, the culture comprising a plurality of cells, cells in the plurality of cells having one or more corresponding cell classes selected from a plurality of cell classes, the plurality of cell classes comprising a first cell class and a second cell class, the method comprising: processing an image of the plurality of cells of the culture to identify one or more cell classes of cells depicted in the image from the plurality of cell classes, the processing comprising: segmenting the image into a plurality of image segments by assigning individual pixels in the image to corresponding cell classes in the plurality of cell classes, the assignment comprising determining, for each of the individual pixels, a corresponding plurality of values ​​corresponding to the corresponding plurality of cell classes, each of the plurality of values ​​indicating a likelihood that the pixel corresponds to a cell of the corresponding cell class in the plurality of cell classes, wherein the plurality of image segments comprises: a first image segment comprising pixels associated with cells of the first cell class; and a second image segment comprising pixels associated with cells of the second cell class; determining a quantity of cells in the culture corresponding to the first cell class based on the plurality of image segments into which the image is segmented; and regulating treatment of the culture based on the quantity.

[0005] Some embodiments provide at least one non-transitory computer-readable storage medium having executable instructions encoded thereon that, when executed by at least one processor, cause the at least one processor to perform a method of regulating processing of a culture, the culture comprising a plurality of cells, cells in the plurality of cells having one or more corresponding cell classes selected from a plurality of cell classes, the plurality of cell classes comprising a first cell class and a second cell class, the method comprising: processing an image of the plurality of cells of the culture to identify one or more cell classes of cells depicted in the image from the plurality of cell classes, the processing comprising: assigning individual pixels in the image to corresponding cell classes in the plurality of cell classes, The image is segmented into a plurality of image segments, the assignment comprising determining, for each of the individual pixels, a corresponding plurality of values ​​corresponding to a corresponding plurality of cell classes, each value in the plurality of values ​​indicating a likelihood that the pixel corresponds to a cell of a corresponding cell class in the plurality of cell classes, wherein the plurality of image segments comprises: a first image segment comprising pixels associated with cells of the first cell class; and a second image segment comprising pixels associated with cells of the second cell class; determining a quantity of cells in the culture corresponding to the first cell class based on the plurality of image segments into which the image is segmented; and adjusting treatment of the culture based on the quantity.

[0006] In some embodiments, assigning individual pixels in the image to corresponding cell categories among the multiple cell categories includes classifying the individual pixels according to multiple classes, wherein a first class among the multiple classes corresponds to the first cell class and a second class among the multiple classes corresponds to the second cell class, and classifying the individual pixels includes selecting, for each of the individual pixels, a class to which the individual pixel is to be classified based on the determined corresponding multiple values.

[0007] In some embodiments, the assignment is performed using a trained machine learning model, and the assignment includes: using the trained machine learning model to process the image to obtain the corresponding multiple values ​​corresponding to the corresponding multiple categories for each of the individual pixels.

[0008] In some embodiments, the trained machine learning model comprises a deep neural network model comprising one or more convolutional layers.

[0009] In some embodiments, the deep neural network model comprises cascaded deep neural network blocks, each of which comprises a corresponding deep convolutional neural network (CNN), and wherein the trained machine learning model performs computations at least in part using atrous spatial pyramid pooling.

[0010] In some embodiments, the deep neural network model comprises a U-net architecture.

[0011] In some embodiments, the deep neural network includes at least one million, at least five million, at least 10 million, at least 50 million, at least 100 million, at least 500 million, or at least 1 billion parameters, the values ​​of which are used as part of processing the image using the deep neural network.

[0012] Some embodiments further include processing the image of the plurality of cells of the culture to estimate culture information of the culture, the culture information of the culture indicating a cell density of the plurality of cells in the culture, wherein segmenting the image includes segmenting the image based on the culture information.

[0013] In some embodiments, the culture information is used to determine coordinates of cells in the plurality of cells, and segmenting the image based on the culture information comprises providing the image and the coordinates as inputs to a trained machine learning model to obtain an output indicating a respective likelihood that each of the individual pixels corresponds to a cell class in the plurality of cell classes.

[0014] Some embodiments further include processing the image of the plurality of cells of the culture to estimate culture information of the culture, the culture information of the culture indicating a cell density of the plurality of cells in the culture, and adjusting the processing of the culture includes adjusting the processing of the culture based on the culture information and an amount of the culture corresponding to the first cell class.

[0015] In some embodiments, the culture information is used to determine coordinates of cells depicted in the image, and wherein adjusting processing of the culture based on the culture information comprises: using the coordinates to determine a location of one or more cells in the plurality of cells; and removing cells from the determined locations.

[0016] In some embodiments, processing the images of the plurality of cells of the culture to estimate the culture information comprises estimating a number of the plurality of cells, a position of at least one cell in the plurality of cells, and / or an internuclear distance between at least two cells in the plurality of cells.

[0017] In some embodiments, adjusting processing of the culture based on the culture information and the amount of cells in the culture corresponding to the first cell type includes: outputting a recommendation indicating a time to passage cells in the plurality of cells of the culture and / or a recommended number of new cultures into which the culture is to be split.

[0018] In some embodiments, adjusting the processing of the culture includes outputting a recommendation to modify the manner in which one or more substances are added to the culture to affect growth of the culture based on the amount of cells corresponding to the first cell type in the culture.

[0019] In some embodiments, regulating the treatment of the culture comprises modifying the manner in which one or more substances are added to the culture to affect the growth of the culture.

[0020] In some embodiments, adjusting processing of the culture includes outputting a recommendation to passage cells in the plurality of cells in the culture based on an amount of cells in the culture corresponding to the first cell type.

[0021] In some embodiments, modulating the treatment of the culture comprises passaging a cell in the plurality of cells of the culture.

[0022] In some embodiments, adjusting processing of the culture includes outputting a recommendation to discard a cell from the plurality of cells of the culture.

[0023] In some embodiments, adjusting the treatment of the culture comprises discarding a cell from the plurality of cells of the culture.

[0024] Some embodiments further comprise: comparing the amount of cells in the culture corresponding to the first cell type to a predetermined amount; and based on the comparison, adjusting processing of the second culture to grow the second culture to have the predetermined amount of the first cell type.

[0025] In some embodiments, the image of the plurality of cells in culture comprises a bright field image.

[0026] Some embodiments further comprise obtaining an image of the plurality of cells of the culture by an imaging sensor of a cell imaging and incubation system.

[0027] In some embodiments, the first cell type corresponds to induced pluripotent stem cells (iPSCs), and the second cell type corresponds to non-iPSCs.

[0028] In some embodiments, the plurality of cell classes includes a third cell class corresponding to background, and the plurality of image segments further includes a third image segment including pixels associated with cells of the third cell class.

[0029] Some embodiments provide a cell imaging and incubation system, the cell imaging and incubation system comprising: an imaging sensor configured to obtain an image of a plurality of cells of a culture; an incubator configured to incubate the culture; at least one processor; and at least one non-transitory computer-readable storage medium having encoded thereon executable instructions that, when executed by the at least one processor, cause the at least one processor to perform a method of regulating treatment of a culture, the culture comprising a plurality of cells, cells in the plurality of cells having one or more corresponding cell classes selected from a plurality of cell classes, the plurality of cell classes comprising a first cell class and a second cell class, the method comprising: processing the image of the plurality of cells of the culture to identify one or more cells depicted in the image from the plurality of cell classes The processing includes: segmenting the image into a plurality of image segments by assigning individual pixels in the image to corresponding cell categories in the plurality of cell categories, the assignment including determining, for each of the individual pixels, a corresponding plurality of values ​​corresponding to the corresponding plurality of cell categories, each value in the plurality of values ​​indicating a likelihood that the pixel corresponds to a cell of a corresponding cell category in the plurality of cell categories, wherein the plurality of image segments include: a first image segment including pixels associated with cells of the first cell category; and a second image segment including pixels associated with cells of the second cell category; determining a quantity of cells in the culture corresponding to the first cell category based on the plurality of image segments into which the image is segmented; and adjusting processing of the culture based on the quantity.

[0030] Some embodiments further include a robotic system configured to transfer cultures within the cell imaging and incubation system between being cultured within the incubator and being imaged by the imaging sensor.

[0031] In some embodiments, the robotic system is configured to transfer the culture between being cultured and being imaged when timing conditions are met.

[0032] In some embodiments, the method further comprises: actuating the robotic system to move the culture to the imaging sensor when the timing condition is met; and actuating the imaging sensor to obtain an image of the plurality of cells of the culture.

[0033] Some embodiments further include an imaging device, wherein the imaging device includes: the imaging sensor; and a chamber configured to receive a well plate, wherein the well plate is configured to contain the culture.

[0034] In some embodiments, the method further comprises: processing the image of the plurality of cells of the culture to estimate culture information of the culture, the culture information of the culture indicating a cell density of the plurality of cells in the culture, wherein segmenting the image comprises segmenting the image based on the culture information.

[0035] In some embodiments, the method further comprises: processing the image of the plurality of cells of the culture to estimate culture information of the culture, the culture information of the culture indicating a cell density of the plurality of cells in the culture, wherein adjusting the processing of the culture comprises adjusting the processing of the culture based on the culture information and an amount of cells in the culture corresponding to the first cell class.

[0036] Some embodiments further comprise using the culture information to estimate a number of the plurality of cells, a location of at least one cell in the plurality of cells, and / or an internuclear distance between at least two cells in the plurality of cells.

[0037] In some embodiments, adjusting the processing of the culture includes outputting a recommendation to modify the manner in which one or more substances are added to the culture to affect growth of the culture based on the amount of cells corresponding to the first cell type in the culture.

[0038] In some embodiments, adjusting processing of the culture includes outputting a recommendation to passage cells in the plurality of cells in the culture based on an amount of cells in the culture corresponding to the first cell type.

[0039] Some embodiments provide a method of regulating treatment of a culture, the culture comprising a plurality of cells, cells in the plurality of cells having one or more corresponding cell classes selected from a plurality of cell classes, the plurality of cell classes comprising a first cell class and a second cell class, the method comprising: processing an image of the plurality of cells of the culture to identify one or more cell classes of cells depicted in the image from the plurality of cell classes, the processing comprising: segmenting the image into a plurality of image segments by assigning regions of the image to corresponding cell classes in the plurality of cell classes, each of the regions comprising two or more individual pixels in the image, the assigning comprising determining, for each of the regions, a respective plurality of values ​​corresponding to the respective plurality of cell classes, each of the plurality of values ​​indicating a likelihood that the region corresponds to a cell of the respective plurality of cell classes, wherein the plurality of image segments comprises: a first image segment comprising a region associated with cells of the first cell class; and a second image segment comprising a region associated with cells of the second cell class; determining a quantity of cells in the culture corresponding to the first cell class based on the plurality of image segments into which the image is segmented; and regulating treatment of the culture based on the quantity. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] The drawings are not intended to be drawn to scale. In the drawings, each identical or nearly identical component shown in various figures is represented by a like numeral. For clarity, not every component may be labeled in every figure. In the drawings:

[0041] Figure 1A is a diagram depicting an illustrative technique 100 for regulating treatment of a cell culture, according to some embodiments of the technology described herein.

[0042] Figure 1B is a block diagram of an example system 150 for regulating processing of a cell culture according to some embodiments of the technology described herein.

[0043] Figure 2A is a flow chart of an illustrative process 200 for regulating treatment of a cell culture in accordance with some embodiments of the technology described herein.

[0044] Figure 2B is a flow chart of an illustrative process 250 for segmenting an image into a plurality of image segments in accordance with some embodiments of the techniques described herein.

[0045] Figure 2C An example of adjusting processing of a culture based on identified image segments in accordance with some embodiments of the technology described herein is shown.

[0046] Figure 3A Shown is an example segmented image of a cell culture according to some embodiments of the technology described herein.

[0047] Figure 3B Some embodiments of the technology described herein are shown Figure 3A Example segmentation image of one cell class of the cell culture shown in .

[0048] Figure 4A Example bright field images of cell cultures are shown according to some embodiments of the technology described herein.

[0049] Figure 4B Some embodiments of the technology described herein are shown. Figure 4A Example density plots generated from the cell cultures shown in .

[0050] Figure 5A An example cell imaging and incubation system is shown according to some embodiments of the technology described herein.

[0051] Figure 5B is a flow chart of an illustrative process 550 for operating a cell imaging and incubation system according to some embodiments of the technology described herein.

[0052] Figure 6 Depicted are example processes for generating and using induced pluripotent stem cells (iPSCs) according to some embodiments of the technology described herein.

[0053] FIG. 7A to FIG. 7B Shown is the similarity between a gene expression dendrogram and a dendrogram generated from a confusion matrix when evaluating the performance of a machine learning model for clone identification according to some embodiments of the techniques described herein.

[0054] Figure 7C It is shown that segmentation techniques according to embodiments of the technology described herein can be used to distinguish pixels associated with iPSCs, non-iPSCs, and background.

[0055] Fig.7D It is shown that segmentation techniques according to embodiments of the technology described herein can be used to accurately predict the frequency of pixels associated with iPSCs.

[0056] FIG. 8A to FIG. 8C It is shown that density estimation techniques according to some embodiments of the techniques described herein can be used to generate a density map of a bright field image.

[0057] Fig. 9 An example process for training a machine learning model to generate a density map of a cell culture according to some embodiments of the technology described herein is shown.

[0058] FIG. 10A to FIG. 10B It is shown that density estimation techniques according to some embodiments of the technology described herein can be used to estimate the number of cells depicted in a bright field image.

[0059] FIG. 11A to FIG. 11C It is shown that density estimation techniques according to some embodiments of the technology described herein can be used to accurately estimate the number of cells depicted in a bright field image compared to real data.

[0060] FIG. 12A to FIG. 12B It is shown that density estimation techniques according to some embodiments of the technology described herein can be used to monitor the number of cells in the wells of a culture plate over time.

[0061] Fig.13 is a graph illustrating the number of cells in a culture corresponding to culture treatment adjustment decisions for the culture according to some embodiments of the technology described herein.

[0062] Fig.14 It is shown that culture information prediction techniques according to some embodiments of the technology described herein can be used to accurately predict the number of cells in culture.

[0063] Fig.15 is a schematic diagram of an illustrative computing device that can be used to implement various aspects described herein. DETAILED DESCRIPTION

[0064] Described herein are techniques for regulating the processing of cell cultures by imaging cell cultures and associating individual pixels in the images with corresponding cell categories using machine learning techniques. In some embodiments, the cell category corresponds to the cell type or other characteristics of a cell or cell group. In some embodiments, these techniques can be used with a cell imaging and incubation system that can include an imaging sensor configured to obtain an image of the culture and an incubator configured to incubate the culture. In some such embodiments, the regulation of the processing of the culture can be based on the processing of the image of the culture obtained by the imaging sensor of the system. The processing can include segmenting the image of the culture into multiple image fragments, which can be accomplished by assigning individual pixels of the image to corresponding cell categories based on pixel-level evaluation. According to some embodiments, these techniques include determining the amount of the culture corresponding to a specific cell category based on these image fragments. The amount of the culture corresponding to the specific cell category can provide information for the regulation of the processing of the culture. For example, such an amount can provide information for decision-making in the entire cell culture growth process.

[0065] Compared with conventional culture management techniques (including conventional manual or conventional automated culture management), the technology described herein can improve automated cell culture management by more reliably and more accurately determining the composition of cell culture. In some embodiments, the result determined using the composition is used to determine the quality of the cell culture. For example, the result can be used to determine metrics such as culture area, size (e.g., minimum and maximum size, aspect ratio), shape (e.g., roundness), texture and roughness. This metric can indicate whether the cells of the culture are healthy and / or whether the cells are differentiating, and these factors may contribute to the overall quality of the culture. Because these metrics are based on a more accurate determination of the culture composition, the final quality of the culture can be determined with a higher degree of accuracy compared to conventional methods. In some embodiments described herein, the results of the culture composition and quality determination can be used to adjust the processing of the cell culture over time. By processing based on a more accurate determination of the composition and quality of the cell culture, the processing decision may be more accurate and more timely for the current state of the culture, which makes it possible to reduce errors and alleviate the inefficiency introduced by less accurate conventional techniques.

[0066] For example, some of the techniques described herein can be used to generate recommendations for treating cell cultures to promote culture growth, prevent waste, and improve overall cell culture maintenance efficiency. Such recommendations may include recommendations for modifying the manner in which substances (e.g., growth factors) are added to or removed from the culture (e.g., the amount of substances added to and / or removed from the culture and when to do so), recommendations for indicating the time to passage cells of the culture, recommendations for indicating the number of new cultures into which the culture is to be split, recommendations for discarding cells of the culture, or other suitable culture processing steps. In some embodiments, such recommendations can be output to a user, who can implement the recommendations manually or by controlling a semi-automatic system, or to an automated system configured to implement the recommendations. In embodiments where a semi-automatic or automated system can be used to partially or fully implement the recommendations, the system can do so in response to receiving the recommendations.

[0067] Cell culture is the process of growing cells in a controlled environment for a variety of purposes, such as studying the morphology, physiology, and biochemistry of cells, monitoring the response of cells to drugs, modeling disease, evaluating genetic variation, etc. Growing cell cultures can involve isolating cells from a tissue, providing the cells with appropriate conditions (e.g., temperature, culture medium, etc.), allowing the cells to proliferate until they occupy a certain percentage of the available matrix (e.g., reaching a confluency threshold), and then passaging the cells by causing them to split and transferring them to new containers, thereby allowing the culture to continue to expand.

[0068] Maintaining and regulating the processing of cell culture may be a manual, laborious and subjective process, resulting in inefficiency and inconsistent results. Routinely, in order to produce high-quality cells of the correct cell type, cultures are frequently assessed to provide information for decision-making about culture regulation. Such decisions are usually made based on manual visual inspection. Although these manual assessments and decisions are made by highly skilled laboratory technicians, these decisions themselves are subjective and also subject to human error. Subjective assessments may be inaccurate, and may and do often result in making decisions that may adversely affect a single culture (e.g., not feeding or splitting in due time) or adversely affect the overall culture process (e.g., resources are invested in a culture that may be discarded). This is a common problem that has been widely experienced in the field of cell culture for decades, resulting in the problem of well-known time and physical resource loss.

[0069] A type of cell that is often cultured is an induced pluripotent stem cell (iPSC), which is a stem cell that can be cultured into different types of cells. In some cases, stem cell cultures can maintain an undifferentiated state, wherein each cell in the culture can be a stem cell that has not yet developed or has not yet developed into a cell with a specific function or structure. In this case, it is possible to monitor whether the cells of the culture have differentiated, and if there is no differentiation, the culture can be divided into multiple cultures (when conditions (such as time or the size of the culture) are met) to continue to cultivate a certain amount of undifferentiated stem cells. In this case, if the culture is seen to begin to differentiate, the quality of the culture may be related to how much the culture has differentiated, and the culture composed mainly of differentiated cells can be discarded.

[0070] Therefore, in some cases, cell cultures may be frequently assessed to ensure the growth of cells with the correct cell type. Although fluorescent imaging, sequencing and similar techniques can be used to assess cultures, these techniques are time-consuming and invasive and may damage cells. Therefore, it is beneficial to use faster, non-invasive techniques (such as bright field or phase microscopy) to assess cultures. However, since differentiated and undifferentiated cells have a high visual similarity, it may be challenging to accurately quantify and distinguish them using such non-invasive techniques. Therefore, it is also challenging to determine how to modify the treatment of cultures to prevent further cell differentiation. Usually, due to failure to effectively treat cultures, cultures are discarded. Any discarding of cultures means a loss of resources and time, and if differentiation cultures are not identified early, time and resources may be spent on cultivating cells that will eventually be discarded, thereby causing further inefficiency of resources and time.

[0071] Even when the cells are of the correct cell type, the culture is frequently assessed to estimate the condition or health of the cells in the culture. Decisions are made based on this estimate as to how and when to modify the way the cells are treated. As non-limiting examples, this can include determining when to change the culture medium, the rate at which to add specific growth factors, and the amount thereof. Because it is challenging to identify small changes that may occur in the culture, it is also difficult to determine when and how to modify the way the culture is treated.

[0072] Additionally or alternatively, cell cultures are frequently assessed to estimate the number of cells in the culture. This estimate is used to inform the time of passage of cells in the culture, and to determine the number of new cultures into which the culture can be divided. However, even a slight miscalculation of the number of cells may result in cells being passaged at an incorrect time (e.g., too early or too late) and the culture splitting into an incorrect number of new cultures. This may result in cell loss through apoptosis, and may affect the way in which cells differentiate, resulting in cultures that are ultimately discarded.

[0073] In many cases, the conventional approach is to have skilled laboratory personnel perform these assessments on the differentiation, cell condition or quantity of cells in the culture. As mentioned above, despite the high skill level of laboratory personnel, these assessments themselves are still subjective, and even personnel within the same organization disagree. This may result in repeated trials for culture maintenance, resulting in cell loss and / or inconsistencies between different cultures.

[0074] Various techniques have been employed to attempt to automate cell culture management. However, these techniques have limitations and do not address the problems associated with culture management identified above. For example, one conventional technique involves attempting to evaluate cell cultures using image tile-based classification. This technique relies on capturing an image of the culture and dividing the image into multiple tiles, then analyzing each tile to identify the content depicted in the tile. A tile refers to an image that depicts a portion of a larger image (e.g., an image of a cell culture). When an image is divided into multiple tiles, the multiple tiles may include images that depict overlapping portions of the larger image. This technique has inherent limitations in terms of its accuracy, as its accuracy is related to the quality of the tileization process and the size / precision of the tiles. In addition, because tile analysis involves comparing a tile with a previously viewed known tile to identify the closest match (and thereby identifying the tile as including the same content as the matched previous tile), reliable analysis of each tile depends on having a sufficient number of training tiles of each type of tile that can be viewed by the system. Compiling training data brings a heavy management burden to the implementation of the system. Increasing the size of the tiles results in a reduction in the number of tiles and may reduce the burden of data collection, but increasing the tile size results in a corresponding reduction in accuracy. These limitations in accuracy mean that conventional automated analysis techniques do not reliably outperform manual subjective interpretation by skilled laboratory technicians, which leads to the continued use of manual processes despite the limitations of manual processes discussed above. Where conventional automated analysis techniques are used, their limitations in accuracy continue to lead to errors in adjusting the processing of cell cultures.

[0075] Conventional automated techniques for cell analysis are further limited because they cannot identify cells or cell cultures that are in an intermediate state of a cycle (e.g., a cell life cycle or differentiation cycle) rather than purely in a starting state or an end state. Conventional techniques are limited to making binary decisions about what is depicted by a tile of an image. For example, conventional techniques can be used to determine whether a tile includes cells of a specific cell type, which can be undifferentiated cells or differentiated cells, or cells in a stable state of a cell life cycle. However, this technique does not capture the transition of cells or cell cultures between two different states, such as the transition of a culture from one cell type to another. Therefore, when conventional techniques are used, processing decisions are based only on whether the tile belongs primarily to one specific cell type or another specific cell type. These decisions do not take into account the presence of cell cultures that transition between cell states, or the presence of cultures that include cells at different stages of the cycle, and they may use different types or levels of processing than cultures that belong to a single non-intermediate type. These limitations mean that the composition of the culture cannot be accurately identified, which may lead to errors in adjusting the treatment of the cell culture.

[0076] About estimating the number of cells in culture, conventional automation technology relies on the fusion measurement based on culture to estimate the number of cells. Fusion refers to the amount of the culture matrix occupied by the cells of the culture. Although fusion is widely used in both automated analysis and manual analysis, the cell counting technology based on fusion is not enough to accurately estimate the number of cells in culture. The technology based on fusion will not consider the scope of cell compaction and area. As mentioned above, this miscalculation of the number of cells in culture causes errors when adjusting the processing of cell culture.

[0077] The inventors have recognized and appreciated that this challenge and inefficiency can be mitigated by improved automated systems for analyzing images of cells in culture. Some of the techniques described herein include more reliable and / or more accurate methods for quantifying the number of cells in cell culture, estimating the cell class of cells in culture based on image analysis and pixel-level segmentation, and / or adjusting the treatment of the culture based on the results of such techniques.

[0078] The inventors have developed systems and methods for regulating the processing of cell cultures. In some embodiments, these techniques include estimating culture information of cell cultures, such as information indicating density, cell count and / or cell position. For example, this can include using a machine learning model. The machine learning model can be implemented using one or more convolutional neural networks (CNNs) and can be adapted to determine the number of cells in the cell culture. In some embodiments, the model can generate information indicating the cell density in different regions of the culture, such as a density map. The information can also include the identification of the position of the individual cells in the culture, such as identified in the coordinate system of the culture or the culture image. The technology using the machine learning model improves conventional techniques, and mitigates the shortcomings of the above-mentioned shortcomings associated with cell culture management by considering cell compaction and area, thereby making the estimation of cell count more accurate. This can improve the regulation of the processing of the culture, thereby producing higher quality cells and consistency between independent cultures.

[0079] In some embodiments, the technology developed by the inventors and described in this article includes machine learning techniques for estimating the composition of cell culture. These techniques include segmenting the image of cell culture by evaluating the individual pixels of the image of the cell in the culture and identifying the corresponding cell category associated with each pixel. As discussed in more detail below, in some cases, the cell category can relate to a cell type. Based on identifying the corresponding cell category of each pixel and identifying the fragment containing the pixel associated with the common cell category, the image can be segmented into different fragments, each fragment corresponding to a cell category. In some embodiments, the culture information generated using the above-mentioned cell counting technology can be used to provide information for segmentation, thereby improving the efficiency and accuracy of the final pixel assignment. Some of the techniques described herein can make it possible to make a more accurate estimate of the cell composition of the culture by obtaining the pixel-level resolution of the composition of the cell culture. By reducing the inaccuracy of conventional techniques, these techniques improve the automated regulation of the processing of the culture, making it possible to produce consistent, high-quality cell cultures, while also making it possible to reduce waste and improve efficiency.

[0080] Cell classification can relate to one or more characteristics of a cell or cell group, such as one or more forms and / or functional characteristics of a cell. This morphological characteristic can include the shape of a cell or the shape of a cell group. In certain embodiments, this characteristic can be the observable characteristic of a cell or a cell group, such as the characteristic that can be determined by describing (for example, an image) (multiple) cells. This observable characteristic can be a marker that a cell or a cell group can show, such as a specific morphological structure (for example, a cell cluster). Based on the presence or absence of one or more markers, a cell or a cell group can be identified as being in a category or another category.

[0081] In certain embodiments, cell classification can relate to the differentiation state of (multiple) cells in certain embodiments.This differentiation state can relate to specific cells, such as specific cells are undifferentiated stem cells or are or have begun to differentiate and therefore are not undifferentiated cells, or relate to cell groups, such as the group includes all undifferentiated cells or a certain amount of undifferentiated cells.In certain embodiments, cell classification can relate to cell type, and this cell type can be the type of cells such as undifferentiated stem cells, or the specific type of cells that have begun to differentiate or are differentiating, such as having a specific anatomical structure (e.g., an organ) or being arranged to perform a specific anatomical function.In certain embodiments, the position of specific organelles in the cell can also be the factor that cell classification is based on.In the case where cell classification can relate to cell type, this cell type can include intermediate cell types, and these intermediate cell types can be the progenitor cells of other cell types during the cell development cycle.In some embodiments where cell classification relates to cell type, cell classification can be for a group of cells, and can relate to the cell of one or more types present in the group of cells.

[0082] In some embodiments, the cell category can relate to the stage of the cell cycle, which can be a process that a cell performs during its life. Such a cell cycle can be, for example, mitosis, and the stage of the cell cycle can be a stage of mitosis. The cell category can relate to the life state of the cell, such as whether the cell is dead or alive. The cell category can also relate to the proliferation level, such as the cell growth rate.

[0083] In some embodiments, the cell class can relate to the experimental population to which a cell or group of cells belongs, such as whether the cell belongs to a control population or a population of experimental subjects undergoing perturbations, genetic mutations, or other experiments.

[0084] In some embodiments, the cell class may involve a combination of the foregoing factors, or the system may operate with multiple cell classes, each involving a different one of the foregoing factors or a combination thereof. Embodiments are not limited to operating with any specific examples of cell classes described above.

[0085] In some embodiments, an image segmentation process according to the techniques described herein can analyze an image (e.g., on a pixel-by-pixel basis, as described herein) and identify a category to which each pixel is to be classified. In some such image segmentation processes, the process can estimate a value indicating the likelihood that the pixel is associated with a corresponding cell category. In doing so, in some cases, the image segmentation process can identify multiple categories for a pixel to which the pixel is to be classified, and each category in the multiple categories can be associated with a corresponding (and potentially different) value indicating likelihood. Then, viewing the value indicating likelihood can result in selecting a cell category for the pixel, such as by selecting the category with the highest likelihood metric or other evaluation (examples of which are given below).

[0086] In some embodiments, the image segmentation process can use a machine learning model to analyze the image, and the machine learning model is trained to identify the category to which each pixel is to be classified. In some embodiments, the image includes a two-dimensional (2D) data point matrix, and the pixels in the image correspond to a single data point in the 2D matrix. Therefore, the 2D matrix can be provided as an input to the machine learning model to obtain an output that identifies multiple categories to which the data point is to be classified for the data points in the 2D matrix. In some other embodiments, the image includes a 2D data point matrix, and the pixels in the image correspond to multiple data points in the 2D matrix. Therefore, the 2D matrix can be provided as an input to the machine learning model to obtain an output that identifies multiple categories to which the multiple data points are to be classified for the multiple data points in the 2D matrix.

[0087] Additionally or alternatively, in some embodiments, the image segmentation process can analyze the image and identify the category to which a region of the image is to be classified. For example, a region of an image can include two or more individual pixels in the image. In some embodiments, the two or more pixels are adjacent pixels. For example, in a 2D data point matrix, adjacent pixels include orthogonal or diagonally adjacent entries in the 2D matrix.

[0088] Various examples of ways in which these techniques and systems may be implemented are described below. However, it should be understood that the embodiments are not limited to operating according to these examples. Other embodiments are also possible.

[0089] Figure 1A is a diagram depicting an illustrative technique 100 for regulating processing of a cell culture 102 by processing an image 106 of the culture 102 using one or more machine learning models 108 to generate an output 110 including an image segment 110 - 1 and / or culture information 110 - 2 .

[0090] In some embodiments, culture 102 includes (multiple) cells of any suitable type. For example, culture 102 may include cells of the same type and / or multiple (e.g., two or more) different types of cells. As a non-limiting example, in some embodiments, culture 102 includes pluripotent stem cells. Pluripotent stem cells are undifferentiated or partially differentiated cells that have the ability to self-renew and differentiate into various types of cells. As another example, culture 102 may include induced pluripotent stem cells (iPSC), which are pluripotent stem cells derived from adult somatic cells that have been genetically reprogrammed to an embryonic stem cell-like state. Additionally or alternatively, culture 102 may include cell types differentiated into pluripotent stem cells and / or iPSC. However, it should be understood that culture 102 includes cells of any suitable type, because the various aspects of the technology described herein are not limited in this regard.

[0091] In some embodiments, culture 102 is grown in any suitable type of container. For example, the type of container can depend on the type of experiment(s) to be performed on culture 102 and / or the type of imaging sensor(s) used to capture images of culture 102. For example, culture 102 can be grown on a coverslip, in a petri dish, in a sample well, in a multi-well plate (e.g., a microplate), in a culture flask, in an OptoSelect TM chip, or using any other suitable type of container.

[0092] In some embodiments, imaging sensor(s) 104 are used to capture images 106 of culture 102. Imaging sensor(s) 104 may include any suitable type of imaging sensor, such as an imaging sensor capable of capturing bright field images, phase contrast images, and / or fluorescence images. For example, imaging sensor(s) 104 may include a microscope imaging system having one or more cameras, such as a Imaging cytometer, Optical fluidic system, Live Cell Analysis System and / or Opera High Content Screening System.

[0093] In some embodiments, imaging sensor(s) 104 automatically capture images 106. For example, imaging sensor(s) 104 may automatically capture images at specified time intervals. Additionally or alternatively, imaging sensor(s) 104 may automatically capture images 106 after detecting culture 102 in a field of view of imaging sensor(s) 104 and / or in a particular location relative to imaging sensor(s) 104. In some embodiments, imaging sensor(s) 104 captures images 106 in response to receiving a user input indicating when to capture image 106.

[0094] In some embodiments, image 106 includes an image of all or a portion of culture 102. For example, image 106 may depict one, some, or all of the wells of a multi-well plate. Additionally or alternatively, image 106 may depict one, some, or all of the cells in culture 102. In some embodiments, image 106 is a bright field image, a phase contrast image, a fluorescence image, and / or an image captured using any other suitable imaging modality. In some embodiments, any suitable image processing technique may be used to process image 106 captured using image sensor(s) 104, as aspects of the technology described herein are not limited in this regard.

[0095] In some embodiments, one or more trained machine learning models 108 are used to process the image 106 to obtain the image segment 110-1 and / or the culture information 110-2. In some embodiments, different machine learning models are used to obtain the image segment 110-1 and the culture information 110-2. For example, in Figure 1A In an embodiment, the image segment 110-1 is obtained using the machine learning model 108-1, and the culture information 110-2 is obtained using the machine learning model 108-2.

[0096] In some embodiments, image 106 includes a two-dimensional (2D) matrix of data points. In some embodiments, individual pixels in image 106 include (e.g., are composed of) individual data points in the 2D matrix. In some embodiments, a region of image 106 includes two or more individual pixels in the image. Thus, a region may include a group of two or more data points in the 2D matrix. In some embodiments, the two or more pixels are adjacent pixels, meaning that they include two or more data points that are orthogonal or diagonally adjacent to each other in the 2D matrix.

[0097] In some embodiments, image 106 (e.g., 2D matrix) is provided to machine learning model 108-1 to obtain image segment 110-1. Machine learning model 108-1 can be of any suitable type. For example, the machine learning model can be a neural network, such as a deep neural network model. The deep neural network model can have any architecture in many types of architectures and can include any suitable type of layer. For example, the deep neural network model can include a convolutional neural network (CNN), a U-Net network, a DeepLab network or any version thereof (e.g., DeepLabv1, DeepLabv2, DeepLabv3, and DeepLabv3+), or any other suitable deep learning network architecture. The neural network can have one or more convolutional layers. In some embodiments, the last layer of the deep neural network can be modified to consider a set of labels corresponding to a set of cell categories (e.g., iPSC, non-iPSC, and background).

[0098] The architecture of the deep neural network model may include one or more basic blocks implemented using ResNet, MobileNet, Xception, or any other suitable deep learning network or any variant of such a deep learning network, as the various aspects of the technology described herein are not limited in this regard. In some embodiments, the deep learning network may have any suitable depth. In some embodiments, the depth is the maximum number of sequential convolutional layers or fully connected layers on the path from the input to the output layer. For example, the deep learning network may be 18 layers deep (e.g., ResNet-18), 50 layers deep (e.g., ResNet-50), 53 layers deep (e.g., MobileNet-v2), 71 layers deep (e.g., Xception), or any other suitable depth, as the various aspects of the present technology are not limited in this regard. In some embodiments, the deep learning network has any suitable number of layers, as the various aspects of the present technology are not limited in this regard. For example, the deep learning network may have 177 total layers and 54 convolutional layers (e.g., ResNet-50). However, it should be understood that the deep learning network may have a greater or lesser number of layers. In some embodiments, the deep neural network uses any suitable resolution image, for example, an image input size of 224×224, 299×299, 1080×1080, or 1958×1958. Aspects of DeepLab are described in Chen, Liang-Chieh et al., “DeepLab: Semantic Image Segmentation with Convolutional Nets, Atrous Convolution, and Fully Connected CRFs,” IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 40, No. 4, pp. 834-848 (2017) and Chen, Liang-Chieh et al., “Encoder-decoder with atrous separable convolution for semantic image segmentation,” in: Ferrari, V., Hebert, M., Sminchisescu, C., Weiss, Y. (Eds.), ECCV 2018.LNCS, Vol. 11211, pp. 833-851 Springer, Cham (2018), each of which is incorporated herein by reference in its entirety.Aspects of ResNet are described in He, K. et al., "Deep Residual Learning for Image Recognition," CVPR (2016), which is incorporated herein by reference in its entirety.

[0099] Regardless of the specific type of machine learning model used as part of the illustrative technique 100, the machine learning model 108-1 outputs an image segment 110-1 corresponding to the image 106. The image segment may include one or more pixels associated with a cell class. Figure 1A In the embodiment shown in FIG. 1 , image segment 110 - 1 includes pixels associated with three different cell categories. For example, some image segments may include pixels associated with iPSCs, some image segments may include pixels associated with non-iPSCs, and some image segments may include pixels associated with non-cell categories. However, it should be understood that machine learning model 108 - 1 can output image segments that include pixels associated with any suitable number or type of cell categories, as aspects of the technology described herein are not limited in this regard.

[0100] In some embodiments, the image 106 is optionally provided to a machine learning model 108-2 to obtain culture information 110-2. The machine learning model 108-2 can be of any suitable type. For example, the machine learning model 108-2 can be a neural network, such as a convolutional neural network (CNN) model. In some embodiments, the neural network includes two parts, each of which is configured to process input using a convolutional layer. In some embodiments, the first part is configured to approximate cell counts and classify images into a threshold number of classes. The threshold number can be any suitable number of classes, such as 10 classes, because the various aspects of the technology described herein are not limited in this regard. In some embodiments, the first part is configured to divide the image into classes based on the approximate number of cells in the image. The information generated by the first part of the network can be used by the second part of the network to generate a density map. In some embodiments, the first part and the second part each include any suitable number of convolutional layers, because the various aspects of the technology described herein are not limited in this regard. For example, the CNN can use the architecture described in Sindagi, VA and Patel, VMJ "CNN-based Cascaded Multi-task Learning of High-level Prior and Density Estimation for Crowd Counting" arXiv:1707.09605 (2017), which is incorporated herein by reference in its entirety.

[0101] Regardless of the specific type of machine learning model used as part of the illustrative technique 100, the machine learning model 108-2 outputs culture information 110-2 corresponding to the image 106. In some embodiments, the culture information includes a cell count indicating the number of cells depicted in the image 106. In some embodiments, the culture information 110-2 indicates the physical location of the cells in the image 106. For example, the culture information 110-2 may include a three-dimensional density map, where density is distributed along the x-coordinate and y-coordinate of the image 106.

[0102] like Figure 1A As shown in the embodiment of FIG. 1 , culture information 110-2 is optionally used to obtain image segment 110-1. For example, culture information 110-2 can be provided as input to a machine learning model 108-1 for predicting image segment 110-1. Machine learning model 108-1 can process culture information 110-2 to estimate the probability of a cell or a specific cell type being present at a location associated with a pixel or pixel group.

[0103] Additionally or alternatively, culture information 110-2 is optionally compared with image fragment 110-1 to determine whether there is a correlation between them. For example, culture information 110-2 can indicate whether there are cells at the position corresponding to a specific pixel, and image fragment 110-1 can indicate whether those pixels are associated with a specific cell category. This correlation can be used to identify cells associated with a predetermined cell category and separate those cells from the position identified by it. The predetermined cell category can include any type of cell category. In some embodiments, the predetermined cell category can include one or more cell categories. For example, the predetermined cell category can include a first predetermined cell category representing a first cell type (e.g., iPSC) or a second predetermined cell category representing a second cell type (e.g., non-iPSC). In another example, the predetermined cell category can include a first predetermined cell category representing a first cell type (e.g., a neuronal cell type), a second predetermined cell category representing a second cell type (e.g., a mesenchymal cell type), or a third predetermined cell category representing a background portion (e.g., an image region that does not include cells). In another example, the predetermined cell categories may include a first predetermined cell category representing a first differentiation state of a cell (e.g., a differentiated cell), a second predetermined cell category representing a second differentiation state of a cell (e.g., an undifferentiated cell), and a third predetermined cell category representing a third differentiation state of a cell (e.g., a partially differentiated cell). In another example, the predetermined cell categories may include a first predetermined cell category representing a first phase of a cell cycle (e.g., a G1 phase), a second predetermined cell category representing a second phase of a cell cycle (e.g., an S phase), a third predetermined cell category representing a third phase of a cell cycle (e.g., a G2 phase), a fourth predetermined cell category representing a fourth phase of a cell cycle (e.g., an M phase), and a fifth predetermined cell category representing a fifth phase of a cell cycle (e.g., a G0 phase). In another example, the predetermined cell categories may include a first predetermined cell category representing a first experimental group (e.g., a control group) and a second predetermined cell category representing a second experimental group (e.g., a treatment group).

[0104] In some embodiments, image segments 110-1 and (optionally) culture information 110-2 are used to adjust treatment 112 of culture 102. For example, image segments 110-1 and / or culture information 110-2 may indicate the type, health, differentiation state, and / or number of cells in the culture. This information, in turn, may be used to inform decisions related to treatment of culture 102 and / or other cultures. For example, image segments 110-1 and / or culture information 110-2 may be used to inform decisions related to cell passaging (e.g., when to passage cells, the number of new cultures to split a culture into, and whether to discard cells in the culture). Image segments 110-1 and / or culture information 110-2 may be used to inform decisions related to how to treat the culture or future cultures (e.g., when to feed the culture, what to feed the culture with, and how much to feed). The present invention includes at least information about Figure 2C Examples of culture handling recommendations are further described.

[0105] In some embodiments, one or more users manually or semi-automatically adjust the processing 112 of the culture. For example, the illustrative technique 100 can include outputting (e.g., via a user interface) recommendations to adjust the processing 112 of the culture 102 to one or more users, and the users can perform adjustments and / or provide user input to the system that causes the system to perform the processing adjustments. In some embodiments, adjusting the processing 112 of the culture is performed automatically. For example, a processor can generate a recommendation to adjust the processing 112 of the culture and cause the system to perform the adjustments without user intervention.

[0106] Figure 1B is a block diagram of an example system 150 for regulating treatment of a cell culture according to some embodiments of the technology described herein. System 150 includes computing device(s) 180 configured to cause software 182 to execute thereon various functions related to evaluating and regulating treatment of a cell culture.

[0107] Computing device(s) 180 may be one or more computing devices of any suitable type. For example, computing device(s) 180 may be a portable computing device (e.g., a laptop computer, a smartphone) or a fixed computing device (e.g., a desktop computer, a server). When computing device(s) 180 include multiple computing devices, the devices may be physically co-located (e.g., in a single room) or distributed across multiple physical locations. In some embodiments, computing device(s) 180 may be part of a cloud computing infrastructure.

[0108] In some embodiments, computing device(s) 180 may be operated by one or more users 160, such as one or more researchers and / or other individual(s). For example, user(s) 160 may provide user input indicating that imaging sensor(s) 192 may capture images of a culture, input specifying processing or other methods to be performed on the captured images, and / or input specifying how processing of the culture may be adjusted.

[0109] like Figure 1B As shown in the embodiment of the present invention, the software 182 includes multiple modules. Each module may include processor executable instructions that, when executed by at least one computer hardware processor, cause the at least one computer hardware processor to perform the (multiple) functions of the module. Such modules are sometimes referred to as "software modules" in this article. Figure 1B The software modules shown in the figure include processor-executable instructions that, when executed by a computing device, cause the computing device to perform one or more processes, as described herein, including at least FIG. 2A to FIG. 2B and Figure 5B It should be understood that Figure 1B The modules shown in are illustrative, and in other embodiments, except Figure 1B In addition to or in lieu of the modules shown in FIG. 1 , one or more other software modules may be used to implement the software 182. In other words, the organization of the software 182 may be similar to that of Figure 1B The method shown in is different.

[0110] like Figure 1B As shown, the software 182 includes multiple software modules for evaluating and adjusting the processing of cell cultures, such as the image segmentation module 166, the culture information prediction module 168, the culture adjustment module 172, and the system automation module 162. Figure 1B In an embodiment, the software 182 further includes a machine learning model training module 164 for training one or more machine learning models and a user interface module 170 for obtaining user input.

[0111] In some embodiments, the image segmentation module 166 obtains an image from the imaging sensor(s) 192 or the image data storage device 152, obtains a trained machine learning model from the machine learning model data storage device 154, and processes the obtained image using the obtained machine learning model to segment the image. For example, the image segmentation module 166 can process the image using the machine learning model to predict, for each pixel in the image, a corresponding cell class associated with the pixel. The present invention includes at least one embodiment of the present invention. Figure 2B The process 250 describes a technique for segmenting an image.

[0112] In some embodiments, the culture information prediction module 168 obtains an image from the imaging sensor(s) 192 or the image data storage device 152, obtains a trained machine learning model from the machine learning model data storage device 154, and uses the obtained machine learning model to process the obtained image to obtain culture information corresponding to the image. For example, the culture information prediction module 168 can process the image to estimate the number of cells in the image and / or generate a density map indicating the location of cells in the image. The present invention includes at least one embodiment of the present invention. Figure 2A Act 204 of process 200 shown in describes a technique for obtaining culture information.

[0113] In some embodiments, culture adjustment module 172 obtains image segments from image segmentation module 166 and / or obtains culture information from culture information prediction module 168 and uses the obtained image segments and / or culture information to generate recommendations for adjusting the treatment of the cell culture. For example, culture adjustment module 172 can determine when to passage cells of the culture, the number of cultures to split the culture into, whether to discard the culture, whether to feed the culture, what to feed the culture with, and / or how much to feed. The present invention includes at least one embodiment of the present invention with respect to Figure 2C Exemplary recommendations for regulating treatment of cell cultures are further described.

[0114] In some embodiments, culture treatment adjustment recommendations may be output by culture adjustment module 172. For example, these recommendations may be output to user(s) 160 via user interface module 170. Additionally or alternatively, these recommendations may be output to system automation module 162.

[0115] The user interface module 170 can be a graphical user interface (GUI), a text-based user interface, and / or any other suitable type of interface through which a user can provide input and view information generated by the software 182. For example, in some embodiments, the user interface can be a web page or web application accessible through an Internet browser. In some embodiments, the user interface can be a GUI of an app executed on a user's mobile device. In some embodiments, the user interface can include a plurality of selectable elements through which a user can interact. For example, the user interface can include a drop-down list, a check box, a text field, or any other suitable element.

[0116] The system automation module 162 is configured to control one or more of the imaging sensor(s) 192, the incubator 194, and / or the robotic system(s) 196. For example, the system automation module 162 can cause the robotic system(s) 196 to manipulate cells in culture (e.g., to separate, split, or discard cells), manipulate substances about the culture (e.g., culture medium, growth factors) to modify the culture medium, and / or move the culture between the imaging sensor(s) 192 and the incubator 194. Additionally or alternatively, the system automation module 162 can cause the imaging sensor(s) 192 to capture images of the cell culture. Additionally or alternatively, the system automation module 162 can cause the incubator 194 to adjust its temperature or other settings. Example system automation software includes Overlord TM Scheduling software. This article includes at least FIG. 5A to FIG. 5B Example automated systems and techniques for using such systems are described.

[0117] In some embodiments, the system automation module 162 is configured to control the imaging sensor(s) 192, incubator 194, and / or robotic system(s) 196 in response to obtaining a culture treatment adjustment recommendation from the culture adjustment module 172. Additionally or alternatively, the system automation module 162 is configured to control the imaging sensor(s) 192, incubator 194, and / or robotic system(s) 196 independently of receiving output from the culture adjustment module 172. For example, the system automation module 162 may control the imaging sensor(s) 192, incubator 194, and / or robotic system(s) 196 periodically in response to user input from the user(s) 160 and / or in response to receiving output from another software module.

[0118] (Multiple) imaging sensors 192 include any suitable type of (multiple) imaging sensors. In some embodiments, (multiple) imaging sensors 192 are configured for bright field imaging, phase contrast imaging, and / or fluorescence imaging. For example, (multiple) imaging sensors 192 may include a microscope imaging system having one or more cameras. The present invention includes at least Figure 1A Example imaging sensor(s) are described.

[0119] Incubator 194 includes any suitable type of culture incubation system. Incubator 194 can be configured to regulate the temperature, humidity, and / or CO2 level of the environment in which the cell culture can be stored. For example, incubator 194 can include an input for CO2, a water tank for humidity, and / or a thermal regulator for temperature control. Incubator 194 can be automatic or semi-automatic, meaning that the incubator self-regulates environmental conditions (e.g., temperature, humidity, and / or CO2 level) or regulates environmental conditions in response to user input. Additionally or alternatively, incubator 194 can be manual, meaning that the user regulates the environmental conditions. Example incubators include Cytomat TM Automated incubator and LiCONic STX500 automated incubator.

[0120] The robotic system(s) 196 include any suitable robotic system configured to handle and / or manipulate cell cultures. The robotic system may include a liquid handler configured to add substances or remove substances from the culture. The robotic system may include a robotic arm configured to handle the culture (e.g., move the culture between an incubator and an imaging platform). Example robotic systems include the Hamilton Microlab STAR liquid handling system and the PF3400 SCARA robot.

[0121] like Figure 1B As shown, the example system 150 also includes an image data storage device 152 and a machine learning model data storage device 154. In some embodiments, the software 182 obtains data from the image data storage device 152, the machine learning model data storage device 154, and / or (multiple) users 160 (e.g., by uploading data). In some embodiments, the software further includes a machine learning model training module 164 for training one or more machine learning models (e.g., stored in the machine learning model data storage device 154).

[0122] In some embodiments, the image is obtained from an image data storage device 152. Image data storage device 152 may be of any suitable type (e.g., a database system, multiple files, flat files, etc.) and may store image data in any suitable manner and in any suitable format, as aspects of the technology described herein are not limited in this respect. Image data storage device 152 may be part of or external to computing device(s) 180.

[0123] In some embodiments, image data storage device 152 includes image data obtained for a cell culture, such as at least one image data storage device 152 that is provided herein. Figure 1AIn some embodiments, the stored image data may have been captured using imaging sensor(s) 192, previously uploaded by a user (e.g., user(s) 160), and / or from one or more public data storage devices. In some embodiments, a portion of the image data may be processed by image segmentation module 166 to obtain image segments. In some embodiments, a portion of the image data may be processed by culture information prediction module 168 to obtain culture information. In some embodiments, a portion of the image data may be used to train one or more machine learning models (e.g., using machine learning model training module 164).

[0124] In some embodiments, the image segmentation module 166 and / or the culture information prediction module 168 obtains (extracts or provides) the corresponding trained machine learning model from the machine learning model data storage device 154. The machine learning model can be provided via a communication network (not shown) (such as the Internet or other suitable network), as aspects of the technology described herein are not limited to any particular communication network.

[0125] In some embodiments, machine learning model data storage device 154 stores one or more machine learning models for segmenting images of cultures and / or for obtaining culture information corresponding to images of cultures. Machine learning model data storage device 154 can be of any suitable type (e.g., a database system, multiple files, flat files, etc.) and can store machine learning models in any suitable manner and in any suitable format, as aspects of the technology described herein are not limited in this regard. Machine learning model data storage device 154 can be part of or external to computing device(s) 180.

[0126] In some embodiments, the machine learning model training module 164 (referred to herein as the training module 164) can be configured to train one or more machine learning models to segment images of cell cultures and / or obtain culture information of images of cell cultures. In some embodiments, the training module 164 uses an image data training set to train the machine learning model. For example, the training module 164 can obtain training data from the image data storage device 152. In some embodiments, the training module 164 can provide (multiple) trained machine learning models to the machine learning model data storage device 154.

[0127] Figure 2Ais a flow chart of an illustrative process 200 for regulating treatment of a cell culture according to some embodiments of the technology described herein. One or more actions of process 200 can be automatically performed by any suitable computing device(s). For example, the actions(s) can be performed by a laptop computer, a desktop computer, one or more servers (in a cloud computing environment), a server computer, or a computer system as described herein. Figure 1B The computing device(s) 180 described herein Fig.15 The computer system 1500 described herein performs, and / or performs in any other suitable manner. For example, in some embodiments, action 202 can be automatically performed by any suitable computing device(s). As another example, action 204 can be automatically performed by any suitable computing device(s).

[0128] Process 200 begins at act 202, in which an image of cells of a culture is obtained. In some embodiments, the image is obtained using one or more image sensors. In some embodiments, the image is obtained from a data storage device storing images previously obtained using one or more image sensors. The one or more image sensors may include any suitable type of image sensor, as aspects of the present technology are not limited to any particular type of image sensor. For example, the one or more image sensors may include a camera configured to capture bright field images, phase contrast images, and / or fluorescent images of cells of a culture. The camera may be included in a microscope imaging system. The present invention includes at least one embodiment of the present invention. Figure 1A to Figure 1B Example imaging systems are described.

[0129] In some embodiments, the image obtained depicts a portion (e.g., some or all) of a cell culture. For example, an image may depict one, some, or all cells of a culture. Figure 4A An example brightfield image of cells in culture is shown that may be obtained at act 202. However, it should be appreciated that the image may be captured using any suitable imaging modality, such as using brightfield imaging, phase contrast imaging, and / or fluorescence imaging.

[0130] Process 200 then proceeds to (optional) action 204, in which the obtained image is processed to estimate culture information of the culture. In some embodiments, processing the image includes providing the image as input to a trained machine learning model. The machine learning model can be trained for one or more tasks. For example, the machine learning model can be trained to estimate the number of cells in the image. Additionally or alternatively, the machine learning model can be trained to estimate a density map of the distribution of density along the x-coordinate and y-coordinate of the image. In some embodiments, the predicted cell count is used to provide information for density map estimation. Figure 4B Shown for Figure 4AExample density map estimated from the brightfield image shown in .

[0131] The machine learning model can be of any suitable type. For example, the machine learning model can be a neural network, such as a convolutional neural network (CNN) model. The CNN model can have one or more convolutional layers. For example, the CNN can use the architecture described in Sindagi, VA and Patel, VMJ "CNN-based Cascaded Multi-Task Learning for Advanced Priors and Density Estimation for Crowd Counting" arXiv:1707.09605 (2017), which is incorporated herein by reference in its entirety.

[0132] In some embodiments, the culture information comprises the output of the machine learning model. For example, the culture information may include a predicted cell count. Additionally or alternatively, the culture information may indicate the physical location of the cells depicted in the image. For example, the culture information may include a density map estimated by the machine learning model. Additionally or alternatively, the culture information may indicate the coordinates of the cells in the image.

[0133] Process 200 then proceeds to act 206, in which the image is processed to identify one or more cell classes of cells depicted in the image. In some embodiments, processing the image includes segmenting the image into one or more image segments at act 206a. This can include, for example, providing the image as input to a trained machine learning model.

[0134] In some embodiments, the machine learning model is trained to predict a value corresponding to a corresponding cell category for each pixel in a plurality of (e.g., some or all) pixels in an image. For example, a value corresponding to a corresponding cell category can indicate the likelihood that a pixel corresponds to a cell of the corresponding cell category. As a non-limiting example, a machine learning model can be trained to predict a value indicating the following likelihood: the likelihood that a pixel is associated with a cell of a particular cell type, the likelihood that a pixel is associated with a cell or cell group with a particular characteristic, the likelihood that a pixel is associated with a cell with a particular differentiation state, the likelihood that a pixel is associated with a cell at a particular stage of the cell cycle, and / or the likelihood that a pixel is associated with a cell or cell group belonging to a particular experimental group. Further examples of cell categories are described above.

[0135] In some embodiments, the culture information obtained at action 204 is optionally provided as input to a machine learning model at action 206a. Coordinate information from the estimated density map can be used to indicate predicted cell locations in the image. For example, a matrix of x-coordinates and y-coordinates can be provided as input to a trained machine learning model along with the image. Additionally or alternatively, the coordinates can be used to directly label the image before providing it to the machine learning model. In some embodiments, the machine learning model is trained to predict, for each of a plurality of pixels in the image, a value indicating the likelihood of the presence of a cell at a location associated with the pixel.

[0136] The machine learning model can be of any suitable type. For example, the machine learning model can be a neural network, such as a deep neural network model. The deep neural network can have any of many types of architectures and can include any suitable type of layers. For example, the deep neural network model can include a convolutional neural network (CNN), U-Net, DeepLab and its versions (e.g., DeepLabv1, DeepLabv2, DeepLabv3, and DeepLabv3+), or any other suitable deep learning network. The neural network can have one or more convolutional layers. The architecture of the deep neural network can include one or more basic blocks implemented using ResNet, MobileNet, Xception, or any other suitable deep learning network that can be used. Aspects of DeepLab are described in Chen, Liang-Chieh et al., "DeepLab: Semantic Image Segmentation with Convolutional Nets, Dilated Convolutions, and Fully Connected CRFs," IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 40, No. 4, pp. 834-848 (2017) and Chen, Liang-Chieh et al., "Encoder-Decoder with Dilated Separable Convolutions for Semantic Image Segmentation," In: Ferrari, V., Hebert, M., Sminchisescu, C., Weiss, Y. (Eds.) ECCV 2018. LNCS, Vol. 11211, pp. 833-851 Springer, Cham (2018), each of which is incorporated herein by reference in its entirety. Aspects of ResNet are described in He, K. et al., "Deep Residual Learning for Image Recognition," CVPR (2016), which is incorporated herein by reference in its entirety.

[0137] Figure 3A shows a machine learning algorithm from training for image segmentation (e.g., Figure 1A 1 . The image is segmented into segments including pixels associated with cells of the following three cell classes: iPSCs, non-IPSCs, and background. Figure 3B The same image is shown indicating the probability of a pixel being associated with a cell of the iPSC cell class.

[0138] Process 200 then proceeds to action 208, in which the image fragments are used to determine the amount of culture corresponding to each cell category in one or more cell categories. For a specific cell category, in some embodiments, this includes determining the number of pixels associated with the cell category and / or the number of pixels associated with one or more specific cell categories compared to the total number of pixels. For example, in order to determine the amount of culture corresponding to iPSC, the number of pixels associated with iPSC can be compared with the sum of the number of pixels associated with iPSC and the number of pixels associated with non-iPSC.

[0139] Process 200 then proceeds to act 210, in which the processing of the culture is adjusted based on the amount of the culture corresponding to each of the one or more cell types. In some embodiments, this includes outputting a recommendation to adjust the processing of the culture. Non-limiting examples of the recommendation include modifying the manner in which one or more substances are added to the culture, discarding the culture, passaging the cells of the culture, and harvesting the cells for downstream applications. The present invention includes at least one embodiment of the present invention. Figure 2C Example recommendations for regulating treatment of cultures are described.

[0140] In some embodiments, adjusting the processing of the culture is performed by one or more users. For example, a recommendation to adjust the processing of the culture can be made via a user interface (such as a user interface that includes at least information about the culture). Figure 1B The user interface module 170) is output to (multiple) users.

[0141] In some embodiments, conditioning the culture is performed by an automated or semi-automated system. The system may include one or more components for performing conditioning, such as a robotic system (e.g., Figure 1B (multiple) robotic systems 196 shown in FIG. 1 ), incubators (e.g., Figure 1B ) and / or one or more imaging sensors (e.g., Figure 1B ). In some embodiments, the system performs adjustments in response to determining an amount of culture corresponding to each of one or more cell types, after a specified time has passed since the determination, and / or in response to receiving user input.

[0142] In some embodiments, adjusting the processing of the culture is further based on the culture information estimated at act 204. For example, the image segments may indicate one or more cells or groups of cells associated with a particular cell class. The manner in which the processing of cells associated with the particular cell class is adjusted may be different from the manner in which the processing of cells associated with other classes is adjusted. In some embodiments, the culture information may be used to accurately locate and process cells corresponding to pixels associated with a particular cell class. For example, when removing differentiated cells from the culture, the image segments along with the culture information may be used to locate and remove cells corresponding to pixels associated with the differentiated cell class.

[0143] Process 200 then proceeds to (optional) action 212, in which the processing of the second culture is regulated. In some embodiments, the second culture is grown in parallel with the culture undergoing analysis (referred to herein as the first culture). For example, cells from the two cultures may have been inoculated at the same time or within the same time window. In some embodiments, the cells of the second culture may be inoculated at a later time than the cells of the first culture or after the first culture is analyzed.

[0144] In some embodiments, the treatment of the second culture is adjusted at action 212 to culture the second culture into each cell category in one or more cell categories with a predetermined amount. This may include comparing the amount of the first culture corresponding to the specific cell category (determined at action 208) with the corresponding predetermined amount. If the determined amount of the first culture is equal to the predetermined amount or within a threshold percentage (e.g., 1%, 2%, 5%, 10%, etc.) of the predetermined amount, the treatment of the second culture may be adjusted in the same or similar manner as the first culture. If the determined amount of the first culture is not equal to the predetermined amount or does not fall within the threshold percentage of the predetermined amount, the treatment of the second culture may be adjusted in a different manner than the first culture. The predetermined amount may be any number of cells set by one or more users, by an automated system, and / or by a semi-automated system. In some embodiments, the predetermined amount may be at least 1 cell, 10 cells, 100 cells, 1000 cells, 10,000 cells, 100,000 cells, or more cells. In some embodiments, the predetermined amount can be up to 100,000 cells, 10,000 cells, 1000 cells, 100 cells, 10 cells, or fewer cells. In some other embodiments, the predetermined amount can be between 17,000 cells and 23,000 cells, between 17,500 cells and 22,500 cells, between 18,000 cells and 22,000 cells, between 18,500 cells and 21,500 cells, between 19,000 cells and 21,000 cells, between 19,500 cells and 20,500 cells, between 19,700 cells and 20,300 cells, between 19,800 cells and 20,200 cells, between 19,900 cells and 20,100 cells, between 19,950 cells and 20,050 cells, or in any other suitable range of cell numbers. Additionally or alternatively, in some embodiments, the predetermined amount can be any percentage of cells set by one or more users, by an automated system, and / or by a semi-automated system. In some embodiments, the predetermined amount can be at least 10% of the cells of the culture, at least 20% of the cells of the culture, at least 25% of the cells of the culture, at least 30% of the cells of the culture, at least 40% of the cells of the culture, at least 50% of the cells of the culture, at least 60% of the cells of the culture, at least 70% of the cells of the culture, at least 75% of the cells of the culture, at least 80% of the cells of the culture, at least 90% of the cells of the culture, at least 95% of the cells of the culture, between 10% and 100% of the cells of the culture, between 50% and 100% of the cells of the culture, between 75% and 95% of the cells of the culture, or in any other suitable range.Non-limiting examples of recommendations for adjusting the processing of the second culture include modifying the manner in which one or more substances are added to the culture, discarding the culture, passaging cells of the culture, and harvesting cells for downstream applications. The present invention at least includes reference to. Figure 2C Example recommendations for regulating treatment of cultures are described.

[0145] As described herein, including at least with respect to act 210, adjusting the processing of the second culture may be performed by one or more users, by an automated system, and / or by a semi-automated system.

[0146] At act 214, process 200 includes determining whether there is another image of cells of the culture to be processed. When it is determined that there is another image, acts 202-212 are repeated for the other image.

[0147] It should be understood that any combination of actions may be performed as part of process 200. Process 200 may include: Figure 2A For example, process 200 may include actions 202-214, actions 206-210, actions 202 and 206-210, actions 204-210, actions 202-210, etc.

[0148] Figure 2B 2 is a flow chart of an illustrative process 250 for segmenting an image into a plurality of image segments according to some embodiments of the techniques described herein. In some embodiments, process 250 may be used to implement act 206a of process 200. Process 250 may be performed by any suitable computing device(s) (e.g., as described herein with respect to Figure 1B The computing device(s) 180 and / or the present disclosure with respect to Fig.15 The computer system 1500 described above is used to execute.

[0149] Process 250 begins with action 252, in which pixels in the image are assigned to corresponding cell classes. As non-limiting examples, this can include assigning pixels to cell classes that represent the type of cell or cell group, assigning pixels to cell classes that represent characteristics of the cell or cell group, assigning pixels to cell classes that represent the differentiation state of the cell, assigning pixels to cell classes that represent a particular stage of the cell cycle, and / or assigning pixels to cell classes that represent the experimental group to which the cell or cell group belongs. Further examples of cell classes are described above.

[0150] In some embodiments, action 252 includes assigning pixels to a plurality of corresponding cell categories. This may include assigning pixels to any corresponding combination of cell categories. For example, this may include assigning pixels to a cell category representing a particular cell type and to a cell category representing a characteristic of the cell. As another example, this may include assigning a particular cell type to a cell category representing a differentiation state, a cell category representing a cell cycle stage, and a cell category representing a characteristic of a cell group.

[0151] Assigning individual pixels in the image to corresponding cell categories at action 252 includes determining a value corresponding to the corresponding cell category for the pixel at action 252a. In some embodiments, these value indication pixels correspond to the possibility of cells of the corresponding cell category. For example, this may include determining a first value indicating the possibility of the pixel being associated with a cell of a first cell type (e.g., iPSC) and a second value indicating the possibility of the pixel being associated with a cell of a second type (e.g., non-iPSC) for a pixel. However, it should be understood that any suitable number of values ​​(e.g., 2 values, 3 values, 4 values, 5 values, 10 values, 20 values, etc.) may be determined for a pixel.

[0152] In some embodiments, act 252a is performed using a machine learning model, such as the machine learning model described herein including at least with respect to act 206 of process 200. For example, the determined value can be obtained as an output from the machine learning model.

[0153] Process 250 then proceeds to action 252b, in which the pixel is assigned to a corresponding cell class based on the determined value. In some embodiments, this includes assigning the pixel to the cell class corresponding to the value indicating the greatest likelihood that the pixel is associated with the cell of that class. For example, for a value indicating a 40% likelihood that the pixel is associated with a cell of type A and a 60% likelihood that the pixel is associated with a cell of type B, type B may be assigned to the pixel because it corresponds to the value indicating the greatest likelihood.

[0154] In some embodiments, regardless of the cell class assigned to the pixel at action 252b, a value corresponding to each cell class can be output. For example, for a particular pixel, the output can indicate that the value indicating the likelihood that the pixel is associated with a cell of type A is 40%, and the value indicating the likelihood that the pixel is associated with a cell of type B is 60%, even though the pixel can be assigned to type B. In this way, these techniques can be used to monitor cells in a culture as they transition between cell classes. For example, these techniques can be used to determine whether a cell is transitioning between an undifferentiated state and a differentiated state. Although the pixels corresponding to those cells can be classified as "undifferentiated," the value indicating the likelihood that the pixel is associated with an undifferentiated cell can be relatively low relative to the differentiated class (e.g., 51%, 55%, 60%, etc.), which can indicate that the cell is transitioning to a differentiated state. Therefore, the treatment of the culture can be adjusted to help this transition and / or prevent this transition. This article at least includes information about Figure 2C Examples of conditioning treatments of cultures are described.

[0155] At act 254, process 250 includes determining whether another pixel exists. When it is determined that another pixel exists, acts 252-254 are repeated for the other pixel.

[0156] Figure 2C An example 280 of adjusting the processing of a culture based on the identified image segments is shown. It should be understood that although Figure 2C Examples are shown, but embodiments of the technology described herein are not limited to any particular manner of regulating processing of a culture.

[0157] Example 280-1 includes modifying the manner in which one or more materials are added to a culture to affect the growth of the culture.

[0158] In some embodiments, modifying the manner in which one or more materials are added to the culture includes modifying the type of substance added to the culture. For example, modifying the type of substance added to the culture may include adding serum, inorganic salts, buffers, carbohydrates, amino acids, vitamins, proteins, peptides, fatty acids, lipids, trace elements, antibiotics, and / or growth factors. Example substances are at least described in Chen, G. et al. "Chemically defined conditions for human iPSC derivation and culture" Nat. Methods. 2011; 8(5): 424-429, which is incorporated herein by reference in its entirety. In some embodiments, these substances vary according to the type of cells cultured and / or according to the cell fate of the undifferentiated stem cells in the culture.

[0159] In some embodiments, modifying the manner in which one or more materials are added to the culture includes modifying the rate at which the substances are added. This may include increasing or decreasing the rate at which a particular substance is added, and / or calculating the rate at which a substance may be added.

[0160] In some embodiments, modifying the manner in which one or more materials are added to the culture includes modifying the amount of the substance added. This can include, for example, increasing or decreasing the amount of one or more substances added to the culture, and / or quantifying the amount of a substance that can be added.

[0161] As a non-limiting example, maintaining iPSC in an undifferentiated state involves adding substances, such as growth factors, to the culture at specified time intervals. If iPSC begins to differentiate, the type, amount or rate of added substances can be modified. In certain embodiments, the present invention at least includes the technology described in processes 200 and 250 that can be used to determine the presence and / or relative amount of a culture in a differentiated state. The presence and / or amount of differentiation can be used to inform whether to add different substances, the amount of substances that can be added, and the rate at which substances can be added to prevent further differentiation and maintain iPSC in its undifferentiated state. For example, if the amount of a culture associated with a non-iPSC cell category (compared with the amount associated with the same iPSC cell category) exceeds 5%, exceeds 10%, exceeds 15%, exceeds 20%, exceeds 25% or exceeds 30%, then the mode of adding substances can be adjusted. For example, the mode of adjusting the addition of substances can include increasing the amount of fibroblast growth factor 2 (FGF2) in the culture.

[0162] As another example, iPSC can be differentiated into different cell types by monitoring cell culture and controlling the cell culture environment. For example, iPSC can be differentiated towards the forebrain. Forkhead Box G1 (FOXG1) is a forebrain marker, and inhibition of WNT signaling has been shown to improve FOXG1 induction. In some embodiments, the techniques described herein at least including processes 200 and 250 can be used to determine the amount of culture corresponding to the FOXG1 marker cell category. For example, these techniques can be used to process an image of the culture after the culture has been stained with FOXG1, so as to determine the likelihood that the pixel is associated with the FOXG1 marker cell category for a pixel in the image. If the amount of the culture corresponding to the FOXG1 marker cell category is less than a threshold amount, XAV939 (a WNT inhibitor) can be added to the culture to improve FOXG1 induction. Examples of neural differentiation are described by Maroof et al., "Directed differentiation and functional maturation of cortical interneurons from human embryonic stem cells," Cell Stem Cell 2013; 12(5):559-572, which is incorporated herein by reference in its entirety.

[0163] Example 280-2 includes discarding cells of culture. In some embodiments, discarding cells of culture includes discarding one, some or all cells of culture. In some embodiments, if a cell is associated with one or more predetermined cell categories, or if a certain amount of culture is associated with one or more predetermined cell categories, the cell can be discarded. For example, if the amount of culture associated with a predetermined cell category exceeds a threshold, the culture can be discarded. Additionally or alternatively, cells corresponding to pixels associated with a predetermined cell category can be discarded. As a non-limiting example, a predetermined cell category may include a non-iPSC cell category, and the culture can be discarded, and if the amount of the non-iPSC cell category (compared to the amount associated with the iPSC cell category) exceeds 5%, more than 10%, more than 15%, more than 20%, more than 25%, or more than 30%, the culture can be discarded. Additionally or alternatively, non-iPSC can be identified and discarded.

[0164] In some embodiments, discarding cells of a culture includes discarding the culture when the quality of the culture is lower than that of other cultures. For example, a well plate can have several wells (e.g., two or more), each well containing a culture. In some embodiments, a user and / or an automated system can rank the cultures (e.g., using the techniques described herein). If the ranking of a culture is not high enough, the culture can be discarded. For example, on a plate with 96 wells, 10 cultures can be selected for further growth, and the remaining cultures can be discarded.

[0165] Example 280-3 includes passaging cells of a culture. Passaging cells is the procedure of dividing (or splitting) cells of a culture into new cultures to facilitate further expansion.

[0166] In some embodiments, regulating the passage of a culture includes determining when to split the culture, determining the number of new cultures into which the culture is to be split, and / or determining whether the culture is overgrown and can be discarded. In some embodiments, these decisions can depend on one or more factors, such as the number of cells in the culture, the conditions used to maintain the cell type, the differentiation protocol, the number of downstream assays, and the type of downstream assays. The number of cells in the culture can be determined, for example, using techniques described herein including at least with respect to action 204 of process 200.

[0167] In some embodiments, the culture is ready to split when the number of cells is equal to a predetermined number or approximately equal to (e.g., within 1%, 2%, 3%, 5%, 7%, 10%, etc.) a predetermined number. The predetermined number can be any number of cells set by one or more users, by an automated system, and / or by a semi-automated system. For example, these techniques can include determining that the culture is ready to split when the number of cells in the culture is between 17,000 cells and 23,000 cells, between 17,500 cells and 22,500 cells, between 18,000 cells and 22,000 cells, between 18,500 cells and 21,500 cells, between 19,000 cells and 21,000 cells, between 19,500 cells and 20,500 cells, between 19,700 cells and 20,300 cells, between 19,800 cells and 20,200 cells, between 19,900 cells and 20,100 cells, between 19,950 cells and 20,050 cells, or in any other suitable range of cell numbers.

[0168] In some embodiments, the culture is overgrown and can be discarded when the number of cells exceeds a threshold amount. For example, the techniques can include determining that the culture is overgrown and can be discarded when the number of cells in the culture exceeds 23,000 cells, 23,500 cells, 24,000 cells, 24,500 cells, 25,000 cells, 25,500 cells, 26,000 cells, 26,500 cells, 27,000 cells, or any other suitable number of cells.

[0169] In some embodiments, these techniques can be used to determine the number of new cultures that cells are to be split into based on the growth rate of cells in culture. For example, cells growing at a faster rate can split into a greater number of new cultures. By monitoring the number of cells in culture over time (e.g., by analyzing images captured over time), the growth rate can be quantified and used to inform the split ratio.

[0170] Example 280-4 includes harvesting cells for downstream applications. In some embodiments, harvesting cells includes monitoring and / or guiding cell differentiation. For example, this can include determining when to differentiate cells and / or whether the differentiation is successful.

[0171] As a first example, the cardiomyocytes have a morphology indicative of mature, healthy cardiomyocytes. In particular, organized cardiomyocyte sarcomeres indicate mature and healthy cardiomyocytes, while disorganized cardiomyocyte sarcomeres indicate immature and unhealthy cardiomyocytes. In some embodiments, the present invention includes at least FIG. 2A to FIG. 2B The image segmentation techniques described can be used to identify pixels associated with myotomes and pixels associated with non-myotomes. The resulting image fragments can be used to determine (e.g., based on user observation and / or using one or more computing devices) whether the myotomes are organized. If it is determined that the myotomes are organized, the culture can be identified as having successfully differentiated and can be used for downstream experiments. If it is determined that the myotomes are disorganized, the culture can be identified as having not successfully differentiated and will not be used for downstream experiments.

[0172] As another example, cell clustering can be used to determine when cells can differentiate. As mentioned herein, a cell cluster can include a group of two or more cells in contact with each other. In certain embodiments, cell clustering is determined using techniques described in at least actions 204 and 206 of process 200 herein. For example, estimated density maps can be used to predict the position of cells in a culture dish, which in turn can be used to determine whether cells have formed clusters. Additionally or alternatively, image fragments can be used to identify clusters by identifying pixels associated with cells and pixels associated with non-cells.

[0173] In some embodiments, determining that the cell is ready to differentiate includes determining whether the amount of culture corresponding to the cell cluster exceeds a threshold compared to the amount of culture corresponding to the non-cluster. For example, this can include determining whether the amount of culture corresponding to the cluster exceeds at least 40%, at least 45%, at least 50%, at least 55%, at least 60%, at least 65%, at least 70%, or any other suitable threshold ratio (e.g., fraction or percentage) of the culture. If the amount exceeds the threshold, the cell can be identified as ready to differentiate. If the amount does not exceed the threshold, the cell can be identified as not ready to differentiate.

[0174] As yet another example, regulating the processing of the culture can include selecting cells and / or cultures for downstream analysis. In some embodiments, this includes selecting a cell culture with a specific growth rate, absence of one or more markers, presence of one or more markers, absence of debris, absence of necrosis, and / or a specific target expression level.

[0175] Determining whether cells have a particular growth rate can include, for example, monitoring the number of cells in the culture over time. In some embodiments, determining the number of cells in the culture includes using techniques described herein including at least with respect to act 204 of process 200. In some embodiments, if the number of cells in a culture equals or exceeds the number of cells in other cultures over a particular time interval, then the culture is selected for downstream analysis.

[0176] Determining whether the cells of the culture include a specific marker includes identifying pixels associated with the marker in the image of the culture. In some embodiments, identifying pixels associated with the marker includes using the image segmentation techniques described herein including at least action 206 of process 200 and about process 250. For example, the image of the culture can be provided to a machine learning model that is trained to predict the likelihood that the pixel is associated with the marker for each of a plurality of pixels in the image. In some embodiments, if one or more specific markers are present or absent in the sample, the culture is selected for downstream analysis. In some embodiments, if the amount of a specific marker in the culture exceeds or is less than a threshold amount, the culture is selected for downstream analysis.

[0177] For example, SSEA-3 and SSEA-4 are markers associated with pluripotency. Therefore, SSEA-3 and SSEA-4 in the culture can be monitored in order to maintain the iPSC in the culture. For example, in some embodiments, an image of a culture stained with SSEA-3 and SSEA-4 can be obtained. The image can be processed using the techniques described herein, including at least those described with respect to processes 200 and 250, to determine the likelihood that each pixel is associated with a SSEA-3 and / or SSEA-4 marker cell class for the pixels in the image. If the amount of the culture associated with the SSEA-3 and / or SSEA-4 marker cell class is below a threshold amount or below a previously determined amount, FGF can be added to the cell culture to maintain the cells in an undifferentiated state and to strive to increase SSEA-3 and SSEA-4.

[0178] Similarly, in some embodiments, determining the cell viability of the culture (e.g., whether the culture includes debris and / or necrosis) includes identifying pixels associated with debris and / or necrosis in an image of the culture. In some embodiments, identifying pixels associated with debris and / or necrosis includes using image segmentation techniques described herein including at least action 206 with respect to process 200 and with respect to process 250. For example, an image of the culture can be provided to a machine learning model that is trained to predict, for each of a plurality of pixels in the image, a likelihood that the pixel is associated with debris and / or a likelihood that the pixel is associated with necrosis. In some embodiments, if the culture does not include debris and / or necrosis, the culture is selected for downstream analysis. In some embodiments, if the amount of the culture associated with debris and / or necrosis is less than a threshold amount, the culture is selected for downstream analysis.

[0179] In some embodiments, the target expression level of the cells of the culture is measured by fluorescence. For example, this may include determining the amount of the culture associated with the fluorescence. In some embodiments, determining the amount of the culture associated with the fluorescence includes identifying pixels associated with the fluorescence in a fluorescent image of the culture using at least the techniques described herein including action 206 of process 200 and process 250. For example, a fluorescent image of the culture may be provided as input to a machine learning model that is trained to predict the likelihood that a pixel is associated with the fluorescence for each of a plurality of pixels in the image. In some embodiments, if the amount of the culture associated with the fluorescence exceeds a certain threshold, the culture may be selected for downstream analysis.

[0180] Example 280-5 includes adjusting the processing of the second culture. For example, the second culture can include a future culture or a culture grown in parallel with the first culture. In some embodiments, adjusting the processing of the second culture includes making decisions about the second culture that involve modifying the manner in which one or more substances are added to the second culture, discarding the second culture, passaging cells of the second culture, and / or harvesting cells for downstream applications.

[0181] As a first example, the techniques described herein can be used to monitor the growth rate of a cell culture. Figure 2A The process 200 described can be used to determine the amount of culture corresponding to cells and the amount of culture corresponding to background (e.g., non-cells). The determined amounts can be used to determine the fusion percentage. For example, when 80% of the culture is identified as corresponding to cells, the culture can be considered to be 80% fused.

[0182] In some embodiments, cell culture may grow rapidly and become over-fused, thereby causing damage to cells (e.g., cells may begin to die). For example, when cell culture exceeds a fusion threshold percentage (e.g., 75%, 80%, 85%, 90%, etc.), cells may be damaged. Therefore, in some embodiments, the fusion percentage can provide information for decisions related to the treatment of cell culture and / or the treatment of (multiple) future cultures. For example, if the fusion percentage exceeds the threshold percentage and the cells cannot be recovered, the cell culture can be discarded. However, if the cells are recoverable, the fusion percentage can be used to provide information for decisions related to splitting cells into one or more future cultures. For example, when the fusion percentage exceeds the threshold percentage, this can indicate that cells grow rapidly and cells can split into a relatively large number of new cultures to accommodate this growth. Additionally or alternatively, in some embodiments, the rapid growth rate of cells can indicate that future cell cultures can split at an earlier time point to prevent them from becoming over-fused.

[0183] As another example, the cell culture can have one or more predetermined cells and / or conditions. In certain embodiments, the presence of one or more predetermined cells and / or conditions is used to provide information for the decision-making related to the treatment of future cultures. The predetermined cells and / or conditions can include any cells or conditions associated with any predetermined cell culture. For example, if the predetermined cell culture includes a differentiated culture and an undifferentiated culture, the predetermined cells can include differentiated cells and undifferentiated cells. In certain embodiments, future cultures can be treated in an effort to avoid one or more predetermined cells and / or conditions (e.g., cell culture overfusion).

[0184] For example, during the growth of hematopoietic cells, cells migrate and are susceptible to separation. In some embodiments, the techniques described herein can be used to monitor the culture to determine the amount of culture corresponding to the separated cells. Figure 2A The process 200 described can be used to determine the amount of culture corresponding to the separated cell class, the amount of culture corresponding to the attached cell class, and any other suitable class (e.g., background cell class). The proportion of separated cells in the culture can be determined by comparing the amount of culture corresponding to the separated cell class with the amount of culture corresponding to the attached cell class.

[0185] To prevent this separation, an attachment matrix can be used to coat the cell culture dish. However, if the amount of the attachment matrix is ​​insufficient, the cells may still separate. In some embodiments, if the determined proportion of separated cells exceeds a threshold, the culture can be discarded. For example, if at least 45%, 50%, 55%, 60%, 70%, or any other suitable proportion of the culture corresponds to a separated cell class, the culture can be discarded. This can indicate that for future cultures of hematopoietic cells, the amount of the attachment matrix coating can be increased to prevent this separation.

[0186] As another example, during neural differentiation, the presence of mesenchymal cells and / or neurons in the culture can be monitored. In some embodiments, the techniques described herein can be used to monitor the culture to determine the amount of the culture corresponding to mesenchymal cells and / or neurons. For example, with respect to Figure 2A The process 200 described can be used to determine the amount of culture corresponding to the mesenchymal cell class, the amount of culture corresponding to the neuronal cell class, and any other suitable class (e.g., background cell class). The proportion of mesenchymal cells in the culture can be determined by comparing the amount of culture corresponding to the mesenchymal cell class with the amount of culture corresponding to the neuronal cell class.

[0187] When the determined proportion of mesenchymal cells increases, this indicates that neural differentiation is unsuccessful. In particular, in some embodiments, inappropriate concentrations of SMAD inhibitors and / or fibroblast growth factor (FGF) may lead to unsuccessful neural differentiation. Therefore, the conditions of (multiple) future cell cultures can be modified (e.g., by adjusting the concentration of SMAD inhibitors and / or FGF) to achieve successful neural differentiation.

[0188] Additionally or alternatively, in some embodiments, the relationship between the conditions of one or more cultures and the quality of those cultures can be monitored to select conditions for future culture. For example, neural differentiation can produce high percentages (e.g., 90%, 95%, 97%, 98%, 99%, 100%, etc.) of neurons and low percentages (e.g., 0%, 1%, 2%, 3%, 5%, 10%, etc.) of mesenchymal cells. In some embodiments, neural differentiation is attempted in cell cultures with different cell culture conditions (e.g., concentrations of SMAD inhibitors and / or FGF). Therefore, different cultures can grow into neurons and mesenchymal cells with different proportions. The relationship between culture conditions and corresponding proportions of neurons and mesenchymal cells can be used to select cell culture conditions for (multiple) future cell cultures that will produce high proportions of neurons and low proportions of mesenchymal cells. For example, cell culture conditions and the ratio of neurons and mesenchymal cells can be provided as input to a machine learning model that is trained to extrapolate the relationship between the two.

[0189] Figure 5A An example cell imaging and incubation system is shown in accordance with some embodiments of the technology described herein. In some embodiments, the example cell imaging and incubation system 500 includes an incubator 502 , a robotic system(s) 504 , and / or an imaging system 506 .

[0190] In some embodiments, the incubator 502 is configured to house one or more cell cultures 520. The incubator 502 can automatically, semi-automatically, or manually adjust the environment for housing one or more cultures 520. For example, the incubator 502 may include at least one of the embodiments described herein. Figure 1B The incubator 194.

[0191] In some embodiments, the robotic system(s) 504 are configured to manipulate one or more cultures 520 and / or other materials for regulating processing of one or more cultures 520. Figure 5A As shown, the robotic system(s) 504 can be configured to move one or more cultures 520 between the incubator 502 and the imaging system 506. For example, the robotic system(s) 504 can include at least one embodiment of the present invention. Figure 1B The robotic system(s) 196 .

[0192] In some embodiments, imaging system 506 may include one or more imaging sensors configured to capture images of cells of one or more cultures 520. For example, the imaging sensor(s) may be configured to capture any suitable type of image of the cells, such as a bright field image, a phase contrast image, and / or a fluorescence image. For example, the imaging sensor(s) may include at least one of the imaging sensors described herein. Figure 1B The imaging sensor(s) 192 .

[0193] In some embodiments, a computing device (not shown) of the imaging and incubation system 500 is configured to process images obtained using the imaging system 506 to obtain image segments 508 and / or culture information 510. For example, the computing device may include at least one embodiment of the present invention. FIG. 2A to FIG. 2B The techniques described are used to process the acquired images.

[0194] In some embodiments, the imaging and incubation system 500 is a fully automated or semi-automated system, meaning that it can automatically or semi-automatically (e.g., through user input) monitor the cell culture and adjust the processing of the cell culture to achieve a predetermined result. Figure 5B Techniques for operating an example imaging and incubation system 500 are described.

[0195] Figure 5B 5 is a flow chart of an illustrative process 550 for operating a cell imaging and incubation system according to some embodiments of the technology described herein. One or more actions of process 550 can be automatically performed by any suitable computing device(s). For example, the actions(s) can be performed by a laptop computer, a desktop computer, one or more servers (in a cloud computing environment), a computer system such as described herein, or a computer program product. Figure 1B The computing device(s) 180 described herein Fig.15 The computer system 1500 may be described as being implemented, and / or in any other suitable manner.

[0196] At action 552, process 550 includes determining whether a timing condition is satisfied. In some embodiments, determining whether a timing condition is satisfied includes determining whether a particular amount of time has passed since an initial time (e.g., since cells were seeded in culture). For example, this may include determining whether six hours, seven hours, eight hours, nine hours, ten hours, 11 hours, 12 hours, 13 hours, 14 hours, 16 hours, 17 hours, 18 hours, or any other suitable amount of time has passed since cells were seeded in culture, as aspects of the technology described herein are not limited in this respect.

[0197] In some embodiments, determining whether a timing condition is satisfied comprises determining whether a previous image of the culture has been acquired (e.g., using Figure 5A506 shown in ). For example, this can include determining whether six hours, seven hours, eight hours, nine hours, ten hours, 11 hours, 12 hours, 13 hours, 14 hours, 16 hours, 17 hours, 18 hours, or any other suitable amount of time has passed since the cells were seeded in culture, as aspects of the technology described herein are not limited in this respect.

[0198] If the timing condition is met at act 552, process 550 proceeds to act 554, in which the robotic system is actuated to move the culture to the imaging sensor(s). For example, the culture may be stored in an incubator, and the robotic system may be actuated to move the culture from the incubator to an imaging system having one or more imaging sensors. In some embodiments, the computing device (e.g., Figure 1B The robotic system is actuated by software executed on the computing device(s) 180 in the system. For example, Figure 1B The system automation module 162 in may be configured to actuate the robotic system(s).

[0199] Process 550 then proceeds to action 556, in which one or more imaging sensors are actuated to obtain an image of a plurality of cells of the culture. In some embodiments, the imaging sensor(s) are actuated in response to the positioning of the culture in the imaging system. For example, the imaging system may include one or more sensors (e.g., presence detection sensors) configured to detect the presence of the culture in the imaging system. Additionally or alternatively, feedback from the robotic system(s) may indicate that the robotic system(s) completed the task of moving the culture to the imaging sensor(s).

[0200] In some embodiments, the imaging sensor(s) are actuated after a timing condition is met. For example, determining whether the timing condition is met may include determining the amount of time that has elapsed since the automated action 554 was initiated. In some embodiments, the timing condition depends on the configuration of the imaging and incubation system and / or the amount of time it takes for the robotic system(s) to move the culture from the incubator to the imaging sensor(s).

[0201] In some embodiments, the imaging sensor(s) are actuated based on user input. For example, a user may specify a time to acquire an image.

[0202] In some embodiments, images obtained using (multiple imaging sensors) may include at least FIG. 2A to FIG. 2B The technology described above is used for processing.

[0203] In some embodiments, process 550 includes removing Figure 5B550 may further include, after action 556, actuating the robotic system(s) to move the culture away from the imaging sensor(s). For example, the process 550 may further include, after action 556, actuating the robotic system(s) to move the culture away from the imaging sensor(s). FIG. 2A to FIG. 2C The described techniques are used to actuate the robotic system(s) to regulate the processing of the cell culture. Additionally or alternatively, the robotic system(s) may be actuated to move the culture back to the incubator.

[0204] Example

[0205] Applications of iPSCs

[0206] Figure 6 An example process for producing and using induced pluripotent stem cells (iPSC) according to some embodiments of the technology described herein is shown. As described herein, iPSC 604 is a pluripotent stem cell derived from an adult somatic cell (e.g., a patient's cell 602). iPSC 604 has the ability to self-renew and differentiate into (i.e., produce) many different cell types (e.g., cell type 606) that constitute an adult body. Therefore, iPSC 604 has many useful applications, such as human cell and developmental modeling 608, disease modeling 610, transplantation 612, drug and genetic screening 614, cell replacement therapy 616, drug selection 618, cell-based assays, biochemical assays, target validation and de-orphanization, drug response prediction, molecular refinement and quality assurance.

[0207] Sample Automation Platform

[0208] An automated platform was assembled to monitor iPSC growth by integrating a robotic arm, an automated incubator, an imaging cytometer, and automation and control software. Specifically, the automated platform included at least the following components: a PF3400SCARA robot from Precise Automation, a Thermo Scientific TM Cytomat TM Automated incubators, Celigo image cytometers from Nexcelom Bioscience, and Overlord from Peak Analysis and Automation TM Laboratory automation software.

[0209] Example Culture Quality Assessment Techniques

[0210] Experiments were performed to assess whether brightfield data can be used to determine culture quality, and more specifically, to detect differences in cell morphology due to differentiation. Six iPSC lines (or clones) were generated using at least the same method described in the "Experimental Methods" section herein. The clones named C2 to C7 were selected because their different degrees of obvious differentiated cells were observed when continued to be passaged. PluriTest (Muller et al., 2011) is an unbiased bioinformatics method for accurately determining the pluripotency of human stem cells (Initiative, 2018), which is used to establish the quality score and ranking of individual clones. Batch mRNA sequencing was performed for all seven clones, thereby generating the PluriTest ranking from best to worst: LT, C7, C3, C4, C5, C6, C2. Various aspects of PluriTest are described in Muller et al. "A bioinformatic assay for pluripotency in human cells [Bioinformatics determination of human cell pluripotency]" Nature Methods 8, 315-317 (2011), which is incorporated herein by reference as a whole.

[0211] Different machine learning models were evaluated and selected for best accuracy when performing the following three artificial intelligence (AI) tasks to determine clone quality: distinguishing clones, classifying images as undifferentiated or differentiated, and semantic segmentation. The machine learning models evaluated included: Xception, Resnet101, Inceptionv3, Densenet201, and Mobilenetv2.

[0212] Clone identification

[0213] Four models were evaluated for their ability to distinguish clones from a single image. Of the four models tested, the method achieved the highest accuracy of 79.36%, which is lower than the expectation that AI can fully distinguish clones. However, the resulting confusion matrix can be used to generate a dendrogram based on the distance between clones ( Figure 7B ), which looks strikingly similar to the cloned gene expression dendrogram ( Fig. 7A ), which supports the view that simple bright field images contain sufficient information for evaluating human pluripotent stem cells (hPSCs). Table 1 shows the performance of each model in distinguishing clones.

[0214]

[0215] Table 1. Performance of the models in distinguishing clones.

[0216] The model was trained using MATLAB 2020b running in an AWS EC2 p3.2xlarge instance. To create a training set for clone identification, 1,000 random images were selected from each clone, for a total of 7,000 images. Of these, 60% were used for training, 20% were used for validation during training, and 20% were used to test and evaluate the trained model. The pre-trained model used for clone identification through transfer learning was Densenet201. The validation accuracy of the final model was 79.36%. Table 2 shows example hyperparameters and their corresponding values ​​for supporting transfer learning.

[0217] Hyperparameters value Solver SDGM momentum 0.9 Initial learning rate 0.001 Learning rate schedule Segmentation Learning rate reduction factor 0.1 Learning rate reduction cycle 10 L2 Regularization 1.0000e-04 Gradient Thresholding L2norm Gradient Threshold Inf Maximum cycle 10 Mini-batch size 100 Verbose 1 Verbose frequency 50 Verification frequency 20 Verify the tolerance 5 Shuffle Each cycle Execution Environment Single GPU

[0218] Table 2. Example hyperparameters.

[0219] Image Classification

[0220] For image classification (i.e., determining clone quality), training data was generated by classifying individual images of each clone as either pluripotent or differentiated until a balanced dataset of similarly sized classes was created for each clone. The model was then trained to distinguish between images showing undifferentiated hPSCs and images containing differentiated cell types, achieving 95.87% accuracy.

[0221] The model was trained using MATLAB 2020b running in an AWS EC2 p3.2xlarge instance. To create a training set for image classification (i.e., determining clone quality), images from several hPSC clones and hPSC lines from Life Technologies were used and divided into hPSC and non-hPSC classes. The original images were collected in a 6-well plate, sized 1958×1958, and tiled into four 979×979 images for training. Clonal images were selected for each class, for a total of 2400 images. Of these, 60% were used for training, 20% for validation during training, and 20% for testing and evaluating the trained model. The pre-trained model used to determine clone quality by transfer learning was Resnet101. The validation accuracy of the final model was 95.87%. Table 3 shows example hyperparameters and their corresponding values ​​for supporting transfer learning.

[0222] Hyperparameters value Solver SDGM momentum 0.9 Initial learning rate 0.001 Learning rate schedule Segmentation Learning rate reduction factor 0.1 Learning rate reduction cycle 10 L2 Regularization 1.0000e-04 Gradient Thresholding L2norm Gradient Threshold Inf Maximum cycle 10 Mini-batch size 100 Verbose 1 Verbose frequency 50 Verification frequency 10 Verify the tolerance 5 Shuffle Each cycle Execution Environment Single GPU

[0223] Table 3. Example hyperparameters.

[0224] Semantic Segmentation

[0225] Thirty-two random images were selected for each clone and pixels were drawn by the user according to three classes (undifferentiated hPSC, differentiated cells, and background). After training, the accuracy of the final model was 95.99%. Tables 4-1, 4-2, and 4-3 show the performance of each model in performing semantic segmentation on different image datasets used to train and test the final semantic segmentation model.

[0226]

[0227] Table 4-1. Performance of the models in performing semantic segmentation.

[0228]

[0229] Table 4-2. Performance of the models in performing semantic segmentation.

[0230]

[0231]

[0232] Table 4-3. Performance of the models in performing semantic segmentation.

[0233] The results can be visualized as the percentage of likelihood for each class or as a pixel plotted image for the combination. Figure 7C Pixel likelihood images of the hPSC class (middle row) and corresponding images output from a semantic segmentation model (bottom row) are shown. The frequency of hPSC pixels to total pixels containing cells was calculated to score and rank clones according to their pluripotency. Figure 7C The bar graph in indicates the ratio of iPSCs (eg, undifferentiated) to non-iPSCs. Fig.7D As shown, the frequency of undifferentiated hPSC pixels correlated closely with the percentage of triple-positive cells measured by flow cytometry, indicating that semantic segmentation can successfully estimate the cellular composition of each hPSC clone and report a quantitative score that can be used to rank the clones, as shown in Table 5.

[0234] Because spatial information is also obtained, semantic segmentation is a superior method for hPSC quality assessment compared to other models. Although there are examples of image-based classification of hPSCs using U-Net to assess the presence of differentiated cells, these examples cannot assign classifications at single-pixel resolution, which is a clear advantage of using semantic segmentation.

[0235]

[0236] Table 5. Ranking the quality of hPSC lines using different methods.

[0237] The model was trained using MATLAB 2020b running in an AWS EC2 p3.2xlarge instance. To create a training set for segmenting hPSCs, non-hPSCs, and background in images, 32 random images of each hPSC clone were selected, for a total of 224 images. Of these, 60% were used for training, 20% were used for validation during training, and 20% were used to test and evaluate the trained model. Pixel labels were created using MATLAB Image Labeler to label pixels as hPSCs, non-hPSCs, or background. The semantic segmentation network used to train the model was Deeplabv3+, and the base pretrained network was Resnet50. The final model had a validation accuracy of 95.99%, a weighted intersection-over-union (IoU) score of 0.94, and an average boundary F1 (BF) score of 0.792. The IoU and BF scores were calculated on the training dataset. IoU is the ratio of correctly classified pixels to the total number of true data and predicted pixels in that class. The BF score shows how well the predicted boundary for each class compares to the true boundary. Table 6 shows example hyperparameters and their corresponding values ​​for transfer learning.

[0238]

[0239]

[0240] Table 6. Example hyperparameters.

[0241] Example Density Estimation Technique

[0242] Experiments were performed to evaluate machine learning techniques for predicting the number of hPSCs within a colony. A convolutional neural network (CNN) was used to generate accurate counts and physical locations of cells by generating a three-dimensional density map in which the density is distributed along the x- and y-coordinates of the image. Various aspects of CNN are described in Sindagi, VA and Patel, VMJ "CNN-based Cascaded Multi-Task Learning for Advanced Priors and Density Estimation for Crowd Counting" arXiv:1707.09605 (2017), which is incorporated herein by reference in its entirety.

[0243] To test whether this CNN can be trained to detect hPSCs at single-cell resolution, aligned brightfield and Hoechst-stained images of hPSC colonies plated in a 96-well format were acquired using an automated microscope (such as the one described in the “Example Automation Platform” section). Fig. 8A and Figure 8B An example alignment of a bright field image and a Hoechst stained image is shown in . The centroids were identified using the Hoechst stained image and converted into a density map using Image-J / FIJI to be used as ground truth data for training the model. Figure 8C The corresponding density map is shown in .

[0244] like Fig. 9 As shown, the image-based CNN consists of two parallel processes filtered by a convolutional layer 904. One half (top) approximates counting and classifies the image 902 into a 10-way counting classifier, the purpose of which is to classify the image based on the approximate number of cells in the field. This information is used to provide information for the second half (bottom) to generate a density map 910, in which the local maximum density represents a single nucleus. Through repeated training rounds (cycles), the connection between the various layers of the CNN is strengthened or weakened based on the similarity of the density map generated by the model with the real data density map 908 obtained from the Hoechst staining image 906. The image is run by an intermediate model captured in an even cycle, and the result is output as a density map to capture the CNN training process. Before determining the correct density positioning, the model went through a trial and error phase from cycle 8 to cycle 48, including an inverted density map, wherein the density was assigned to the empty part of the culture dish before finding the correct approximate distribution. By cycle 60, the total colony morphology and position were correctly determined and further improved throughout the iteration. This learning process is consistent with the minimization of training loss, average error, and mean square error. The training was stopped to select the model with the smallest mean error and mean square error from cycle 680. It was found that the resulting model ignored microscopic artifacts, including particles on the bottom of the microplate, well edges, bubbles, and focal plane changes, without enhancing the original bright field images (e.g., Fig. 8A ) and used it in subsequent experiments (called the optimized model).

[0245] The optimized model is data rich; it can locate the relative position of cells in a dish, be segmented to generate counts within a specific field of view, and summarize larger areas by calculating the area under the curve. To illustrate this, the image can be sliced ​​at a given horizontal coordinate and the grayscale value plotted. FIG. 10A to FIG. 10B and FIG. 11A to FIG. 11B A comparison of the magnitude and clarity in the features of the low-contrast original image and the density map revealed by the model is shown, illustrating the informative transformation performed by the trained model. Fig. 11C As shown, the optimized model was evaluated using newly imaged data based on correlation of real data provided by fluorescence-based object detection with the model results exhibiting an R-squared value of 0.994.

[0246] In addition to the basic tasks of cell counting and finding differentiation contaminants, the generated density map also contains information that allows additional analysis. For example, it can be used to map cell positions, detect fusions, and measure internuclear distances. The method can be easily embedded in an automated process that can reach a certain scale and throughput to meet the needs of automated hPSC culture. Cell counting and quality assessment methods can be adapted to various hPSC lines and microscopes by training new models or transferring learning with as few as a single 96-well plate of hPSCs. These methods can be used as a means of establishing standards when training individuals to work with hPSC tissue culture. These methods provide rapid quality control assessments for cells cultured for use as cell replacement therapies, thereby enhancing existing validation methods such as gene expression profiling, flow cytometry, and immunocytochemistry.

[0247] This article includes at least the automation system described in the “Example Automation Platform” section for acquiring training data. Whole-well images from 27 barcoded 96-well tissue culture plates were recorded every 12 hours. The images were automatically uploaded to the cloud, and cell counts and heat maps were calculated to monitor cell growth over time, demonstrating the online performance of the automation system during hPSC growth. Fig. 12A Cell counts over time are shown. Fig. 12B A heat map of the culture plate is shown, indicating the relative number of cells in each well over time. Split decision training was performed by classifying images of hPSCs that could be fed, split, or considered over-confluent. Fig.13 The classification is shown in . The limitations of the model were determined by reducing the image area of ​​the petri dish or by merging pixels to reduce the resolution.

[0248] Density maps were generated for seven hPSC lines using the optimized model, which showed the following results compared to the Hoechst staining images: Fig.14 The accuracy and broad applicability of the model to additional hPSC lines are shown. Normal qq plots of cell counts for hPSC lines were used to determine how well the sampling of individual fields of view fit a normal distribution. As an example, if cell differentiation results in fusion across all fields of view, as shown in C2, the result is a "moderate effect" in which images with median counts are more frequently observed, which is seen as a concave downward curve. In contrast, the concave upward curve depicted by C6 represents a bimodal distribution compared to a normal distribution.

[0249] The model was trained using Amazon SageMaker. Training was performed using ml.p3.8xlarge instances. The custom model and training scripts from Sendagi et al. (Sindagi and Patel, 2017) were packaged as docker images according to the SageMaker specifications. All hyperparameters used during training were kept the same as those from Sendagi et al. (Sindagi and Patel, 2017). Training lasted approximately 4 hours.

[0250] The training data set was assembled by randomly selecting 3000 1958×1958 images from a larger data set of 4608 images collected from three 96-well plates. The size of each image was then reduced to 256×256 by randomly cutting from the image. The 3000 images were then manually sorted to remove images that were out of focus or otherwise defective to prevent the nuclear segmentation algorithm from working properly. After the 3000 images were manually sorted, 2375 images were selected for training. The fluorescent channel from each image was run through the segmentation algorithm to find the nuclear center point. Various aspects of the segmentation algorithm are described in Wang, Y. et al. "Segmentation of the Clustered Cells with Optimized Boundary Detection in Negative Phase Contrast Images" PloS One [Public Library of Science Comprehensive], 10 (2015), which is incorporated herein by reference in its entirety. These center points are then used to create a true data density map as described in (Sindagi and Patel, 2017). The training dataset is then further split into a training dataset and a validation dataset, where 80% of the data is used for training and 20% of the data is used for validation during training.

[0251] Experimental methods

[0252] Tissue culture

[0253] The hPSC line was obtained from Life Technologies (Thermo Fisher Scientific) and maintained between passages 25 and 45. This cell line is an additional reprogramming line derived from CD34+ hematopoietic cells. hPSCs were fed daily with mTeSR1, passaged using ReLeSR, and attached to 10 cm and 96-well tissue culture plates coated with hESC-qualified matrigel (Corning). The pluripotency of hPSCs was assessed by flow cytometry, karyotype abnormalities, and mycoplasma to control the quality of the culture.

[0254] Reprogrammed hPSC clones were derived from CD34+ umbilical cord blood cells (STEMCELL Technologies). Reprogramming was performed using the CytoTune-iPS2.0 Sendai reprogramming kit (Invitrogen) and following the instruction manual. Once reprogramming was complete, clones were fed daily with mTeSR1, passaged using ReLeSR, and attached to 6-well tissue culture plates coated with hESC-qualified Matrigel (Corning). Cell plating and staining for training datasets

[0255] The hPSC line was dissociated from a 10 cm culture dish using ReLeSR and plated at equal density on a standard flat-bottom 96-well microplate (Corning), which was coated with hESC-qualified matrigel. The plates were fixed at one-day intervals in the following days. All plates were fixed with formaldehyde at a final concentration of 3.7% for 20 minutes by adding an equal volume of formaldehyde to the culture medium already in the 7.4% formaldehyde wells. In order to stain the nuclei, a staining solution was prepared by diluting Hoechst33342 (Molecular Probes) in PBS to 1:5000 and incubating in the dark at room temperature for 15 minutes. After incubation, the staining solution was removed, and the cells were washed three times with PBS, and a sufficient volume (about 200 μL) of PBS was added to the wells for imaging.

[0256] Imaging and acquisition settings

[0257] Images were acquired with a Celigo imaging cytometer (Naixilon Biotech). Illumination for brightfield was an enhanced brightfield imaging channel based on 1 LED with uniform and good illumination. There was also a fluorescence channel based on 4 LEDs. Images were acquired with a large chip CCD camera with a galvanometer mirror and F-θ lens at a resolution of 1 μLm / pixel. All images were magnified 10 times.

[0258] The training plates used for cell counting and density map models were imaged in brightfield and blue channel. All other plates were imaged in brightfield. Acquisition settings: brightfield 50ms exposure; Hoechst 250ms exposure, excitation 377 / 50, emission 470 / 22.

[0259] Automatically run to fusion

[0260] Human hPSC lines were dissociated from 10 cm dishes using reLeSR and plated at equal density on 96-well microplates coated with hESC-qualified Matrigel. After plating, the 96-well microplates were loaded into the Cytomat TMIn an automated incubator. Using Overlord TM The automated software sets the plate to image all plates and uploads those images to an AWS S3 bucket every 12 hours. The images are run through the model on AWS. Cell growth is tracked using the output of the density estimation techniques described herein, including cell counts, heat maps, and growth curves. The plates are maintained until the cells grow to confluence.

[0261] Continuous operation

[0262] hPSC lines were dissociated from 10 cm dishes using reLeSR and plated at four different densities on each of 96-well microplates coated with hESC-qualified Matrigel. After plating, the 96-well microplates were loaded into the Cytomat TM In an automated incubator. Using Overlord TM The automated software sets the plate to image all plates and upload those images to AWS every 12 hours. The output of the density estimation technique described herein is used to determine when the plate is ready to split and the split ratio used to balance the cell density on each microplate.

[0263] Fig.15 The following illustrates a method that can be combined with the techniques described herein (e.g., FIG. 2A to FIG. 2B and Figure 5B An illustrative embodiment of a computer system 1500 for use with any embodiment of the method of the present invention. The computer system 1500 includes one or more processors 1510 and one or more articles of manufacture including non-transitory computer-readable storage media (e.g., memory 1520 and one or more non-volatile storage media 1530). The processor 1510 can control the writing of data to and reading of data from the memory 1520 and the non-volatile storage device 1530 in any suitable manner, as the various aspects of the technology described herein are not limited to any particular technology for writing or reading data. In order to perform any of the functions described herein, the processor 1510 can execute one or more processor-executable instructions stored in one or more non-transitory computer-readable storage media (e.g., memory 1520), which can be used as a non-transitory computer-readable storage medium to store processor-executable instructions for execution by the processor 1510.

[0264] The computer system 1500 may also include a network input / output (I / O) interface 1540 via which the computing device can communicate with other computing devices (e.g., over a network), and one or more user I / O interfaces 1550 via which the computing device can provide output to a user and receive input from a user. The user I / O interfaces may include devices such as a keyboard, a mouse, a microphone, a display device (e.g., a monitor or a touch screen), a speaker, a camera, and / or various other types of I / O devices.

[0265] The above-described embodiments may be implemented in any of a variety of ways. For example, embodiments may be implemented using hardware, software, or a combination thereof. When implemented in software, the software code may be executed on any suitable processor (e.g., microprocessor) or processor set, whether provided in a single computing device or distributed among multiple computing devices. It should be understood that any component or component set that performs the above-described functions may be generally considered to be one or more controllers that control the above-described functions. The one or more controllers may be implemented in a variety of ways, such as using dedicated hardware or using general-purpose hardware (e.g., one or more processors) that is programmed using microcode or software to perform the above-described functions.

[0266] In this regard, it should be understood that one implementation of the embodiments described herein includes at least one computer-readable storage medium (e.g., RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disk (DVD) or other optical disk storage device, magnetic cassette, magnetic tape, magnetic disk storage device or other magnetic storage device, or other tangible non-transitory computer-readable storage medium), the at least one computer-readable storage medium is encoded with a computer program (i.e., a plurality of executable instructions) that performs the above-mentioned functions of one or more embodiments when executed on one or more processors. The computer-readable medium can be transportable so that the program stored thereon can be loaded onto any computing device to implement various aspects of the technology described herein. In addition, it should be understood that the reference to a computer program that performs any of the above-mentioned functions when executed is not limited to an application program running on a host. Rather, the terms computer program and software are used herein in a general sense to refer to any type of computer code (e.g., application software, firmware, microcode, or any other form of computer instructions) that can be employed to program one or more processors to implement various aspects of the technology described herein.

[0267] The above description of the embodiments provides illustration and description, but is not intended to be exhaustive or limit the embodiments to the precise form disclosed. Modifications and variations are possible in accordance with the above teachings, or may be obtained from the practice of the embodiments. In other embodiments, the methods depicted in these figures may include fewer operations, different operations, operations in different orders, and / or additional operations. Further, non-dependent blocks may be executed in parallel.

[0268] It is apparent that in the embodiments shown in the figures, the example aspects described above can be implemented in many different forms of software, firmware, and hardware. Further, some parts of these embodiments can be implemented as "modules" that perform one or more functions. The module can include hardware (such as a processor, an application specific integrated circuit (ASIC), or a field programmable gate array (FPGA)) or a combination of hardware and software.

[0269] Having described several aspects and embodiments of the technology set forth in this disclosure, it should be understood that various changes, modifications and improvements will be readily conceived by those skilled in the art. Such changes, modifications and improvements are intended to be within the spirit and scope of the technology described herein. For example, a person of ordinary skill in the art will easily envision various other means and / or structures for performing the functions described herein and / or obtaining results and / or one or more advantages, and each of such changes and / or modifications is considered to be within the scope of the embodiments described herein. Those skilled in the art will recognize or be able to determine many equivalents of the specific embodiments described herein using only routine experiments. Therefore, it should be understood that the aforementioned embodiments are presented only by way of example, and within the scope of the appended claims and their equivalents, the inventive embodiments may be practiced in a manner different from that specifically described. In addition, any combination of two or more features, systems, products, substances, kits and / or methods described herein, if such features, systems, products, substances, kits and / or methods are not mutually inconsistent, is included within the scope of this disclosure.

[0270] The above embodiments can be implemented in any of a variety of ways. One or more aspects and embodiments of the execution of the process or method disclosed herein can utilize program instructions that can be executed by a device (e.g., a computer, a processor, or other device) to execute these processes or methods or control their execution. In this regard, different inventive concepts can be embodied as a computer-readable storage medium (or multiple computer-readable storage media) (e.g., a computer memory, one or more floppy disks, compact disks, optical disks, tapes, flash memories, circuit configurations in field programmable gate arrays or other semiconductor devices, or other tangible computer storage media), which is encoded with one or more programs, which execute one or more methods of implementing the various embodiments described above when executed on one or more computers or other processors. The one or more computer-readable media can be transportable, so that one or more programs stored thereon can be loaded onto one or more different computers or other processors to implement various aspects of the above aspects. In some embodiments, the computer-readable medium can be a non-transient medium.

[0271] The term "program" or "software" is used herein in a general sense to refer to any type of computer code or computer executable instruction set that can be employed to program a computer or other processor to implement the various aspects described above. In addition, it should be understood that, according to one aspect, one or more computer programs that perform the methods of the present disclosure when executed need not reside on a single computer or processor, but can be distributed in a modular manner among multiple different computers or processors to implement various aspects of the present disclosure.

[0272] Computer executable instructions can be in many forms, such as program modules, that are executed by one or more computers or other devices. Typically, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. Typically, in various embodiments, the functionality of the program modules can be combined or distributed as desired.

[0273] Moreover, the data structure can be stored in a computer-readable medium in any suitable form. For simplicity of presentation, the data structure can be shown as having fields that are related by location in the data structure. Similarly, such relationships can be achieved by assigning locations in a computer-readable medium that convey the relationship between the fields for storage of the fields. However, any suitable mechanism can be used to establish the relationship between the information in the fields of the data structure, including by using pointers, tags, or other mechanisms that establish relationships between data elements.

[0274] When implemented in software, the software code can be executed on any suitable processor or collection of processors, whether provided in a single computer or distributed among multiple computers.

[0275] Moreover, a computer may have one or more input and output devices. These devices may be used, among other things, to present a user interface. Examples of output devices that may be used to provide a user interface include a printer or display screen for visually presenting the output and a speaker or other sound generating device for auditorily presenting the output. Examples of input devices that may be used for a user interface include a keyboard and a pointing device, such as a mouse, a touch pad, and a digitized tablet computer. As another example, a computer may receive input information through speech recognition or in other audible formats.

[0276] Such computers can be interconnected by one or more networks of any suitable form, including local area networks or wide area networks (such as enterprise networks) and intelligent networks (IN) or the Internet. Such networks can be based on any suitable technology and can operate according to any suitable protocol, and can include wireless networks, wired networks or fiber optic networks.

[0277] Moreover, as described, some aspects may be embodied as one or more methods. The actions performed as part of a method may be ordered in any suitable manner. Thus, embodiments may be constructed in which the actions are performed in an order different from that shown, which may include performing some actions simultaneously (even though the actions are shown as being sequential in the illustrative embodiments).

[0278] All definitions, as defined and used herein, should be understood to control over dictionary definitions, definitions in documents incorporated by reference, and / or ordinary meanings of the defined terms.

[0279] The indefinite articles "a" and "an" as used herein in the specification and claims, unless expressly indicated to the contrary, should be understood to mean "at least one".

[0280] The phrase "and / or" as used herein in the specification and claims should be understood to mean "either or both" of the elements so combined, i.e., elements that are present in combination in some cases and separately in other cases. Multiple elements listed with "and / or" should be interpreted in the same manner, i.e., "one or more" of the elements so combined. Other elements other than the elements specifically identified by the "and / or" clause may optionally be present, whether related or unrelated to those elements specifically identified. Thus, as a non-limiting example, when used in conjunction with open language such as "comprising", a reference to "A and / or B" may refer to only A (optionally including elements other than B) in one embodiment; to only B (optionally including elements other than A) in another embodiment; to both A and B (optionally including other elements) in yet another embodiment; etc.

[0281] As used herein in the specification and claims, the phrase "at least one" with respect to a list of one or more elements should be understood to mean at least one element selected from any one or more elements in the list of elements, but not necessarily including at least one element of each element specifically listed in the list of elements, and not excluding any combination of elements in the list of elements. This definition also allows that there may be optionally elements other than the elements specifically identified in the list of elements to which the phrase "at least one" refers, whether related to or unrelated to those elements specifically identified. Thus, as a non-limiting example, "at least one of A and B" (or equivalently "at least one of A or B", or equivalently "at least one of A and / or B") may refer to at least one (optionally including more than one) A in one embodiment, where B is absent (and optionally including elements other than B); to at least one (optionally including more than one) B in another embodiment, where A is absent (and optionally including elements other than A); to at least one (optionally including more than one) A and at least one (optionally including more than one) B (and optionally including other elements) in yet another embodiment; etc.

[0282] In the claims and the above specification, all transitional phrases, such as "comprising", "including", "carrying", "having", "containing", "involving", "holding", "composed of", etc., should be understood as open-ended, that is, meaning including but not limited to. Only the transitional phrases "consisting of" and "consisting essentially of" should be closed or semi-closed transitional phrases, respectively.

[0283] The terms "approximately," "substantially," and "about" may be used in some embodiments to mean within ±20% of a target value, in some embodiments to mean within ±10% of a target value, in some embodiments to mean within ±5% of a target value, and in some embodiments to mean within ±2% of a target value. The terms "approximately," "substantially," and "approximately" may include the target value.

Claims

1. A method of regulating a treatment of a culture, the culture comprising a plurality of cells, the cells of the plurality of cells having one or more corresponding cell types selected from a plurality of cell types, the plurality of cell types comprising a first cell type and a second cell type, the method comprising: processing the image of the plurality of cells of the culture to identify one or more cell types of cells depicted in the image from the plurality of cell types, the processing comprising: segmenting the image into a plurality of image segments by assigning individual pixels in the image to corresponding cell classes in the plurality of cell classes, the assigning comprising determining, for each of the individual pixels, a respective plurality of values ​​corresponding to the respective plurality of cell classes, each value in the respective plurality of values ​​indicating a likelihood that the individual pixel corresponds to a cell of a respective cell class in the plurality of cell classes, The multiple image segments include: a first image segment comprising pixels associated with cells of the first cell class; and a second image segment comprising pixels associated with cells of the second cell class; determining an amount of cells corresponding to the first cell type in the culture based on the plurality of image segments into which the image is segmented; and Treatment of the culture is adjusted based on this amount.

2. The method of claim 1, wherein: Assigning the individual pixels in the image to corresponding cell classes in the plurality of cell classes comprises classifying the individual pixels according to a plurality of classes, wherein a first class in the plurality of classes corresponds to the first cell class and a second class in the plurality of classes corresponds to the second cell class, and Classifying the individual pixels includes selecting, for each of the individual pixels, a class into which the individual pixel is to be classified based on the determined corresponding plurality of values.

3. The method of claim 1 or any other preceding claim, wherein: The assignment is performed using a trained machine learning model and includes: The image is processed using the trained machine learning model to obtain, for each of the individual pixels, the corresponding plurality of values ​​corresponding to the corresponding plurality of cell classes.

4. The method of claim 3, wherein: The trained machine learning model includes a deep neural network model comprising one or more convolutional layers.

5. The method of claim 4, wherein: The deep neural network model includes cascaded deep neural network blocks, each of which includes a corresponding deep convolutional neural network (CNN), and wherein the trained machine learning model performs calculations at least in part using atrous spatial pyramid pooling.

6. The method of claim 4, wherein: This deep neural network model includes the U-net architecture.

7. The method according to any one of claims 4 to 6, wherein: The deep neural network model includes at least one million, at least five million, at least 10 million, at least 50 million, at least 100 million, at least 500 million, or at least 1 billion parameters, the values ​​of which are used as part of processing the image using the deep neural network model.

8. The method of claim 1 or any other preceding claim, further comprising: processing the image of the plurality of cells of the culture to estimate culture information of the culture, the culture information of the culture indicating a cell density of the plurality of cells in the culture, Wherein segmenting the image comprises segmenting the image based on the culture information.

9. The method according to claim 8, in, The culture information is used to determine coordinates of cells in the plurality of cells, and Wherein segmenting the image based on the culture information comprises providing the image and the coordinates as input to a trained machine learning model to obtain an output indicating a respective likelihood that each of the individual pixels corresponds to a cell class among the plurality of cell classes.

10. The method of claim 1 or any other preceding claim, further comprising: processing the image of the plurality of cells of the culture to estimate culture information of the culture, the culture information of the culture indicating a cell density of the plurality of cells in the culture, Wherein adjusting the processing of the culture includes adjusting the processing of the culture based on the culture information and the amount of cells in the culture corresponding to the first cell type.

11. The method of claim 10, wherein: The culture information is used to determine coordinates of cells depicted in the image, and wherein adjusting processing of the culture based on the culture information comprises: using the coordinates to determine the location of one or more cells in the plurality of cells; and Cells are removed from the identified locations.

12. The method of claim 10, processing the images of the plurality of cells of the culture to estimate the culture information comprises estimating the number of the plurality of cells, the position of at least one cell in the plurality of cells, and / or the internuclear distance between at least two cells in the plurality of cells.

13. The method of claim 10, wherein: Adjusting processing of the culture based on the culture information and the amount of cells in the culture corresponding to the first cell type includes: An output indicates a recommendation for a time to passage cells in the plurality of cells in the culture and / or a recommended number of new cultures into which the culture is to be split.

14. The method of claim 1 or any other preceding claim, wherein: Adjusting the processing of the culture includes outputting a recommendation to modify the manner in which one or more substances are added to the culture to affect growth of the culture based on the amount of cells corresponding to the first cell type in the culture.

15. The method of claim 1 or any other preceding claim, wherein: Modulating the treatment of the culture includes modifying the manner in which one or more substances are added to the culture to affect the growth of the culture.

16. The method of claim 1 or any other preceding claim, wherein: Adjusting processing of the culture includes outputting a recommendation to passage cells in the plurality of cells in the culture based on an amount of cells in the culture corresponding to the first cell type.

17. The method of claim 1 or any other preceding claim, wherein: Modulating the treatment of the culture includes passaging cells in the plurality of cells of the culture.

18. The method of claim 1 or any other preceding claim, wherein: Adjusting processing of the culture includes outputting a recommendation to discard a cell from the plurality of cells of the culture.

19. The method of claim 1 or any other preceding claim, wherein: Adjusting the treatment of the culture includes discarding cells from the plurality of cells of the culture.

20. The method of claim 1 or any other preceding claim, further comprising: comparing an amount of cells in the culture corresponding to the first cell type to a predetermined amount; as well as Based on the comparison, processing of the second culture is adjusted to grow the second culture to have the predetermined amount of the first cell type.

21. The method of claim 1 or any other preceding claim, wherein: The image of the plurality of cells of the culture comprises a bright field image.

22. The method of claim 1 or any other preceding claim, further comprising: An image of the plurality of cells of the culture is obtained by an imaging sensor of a cell imaging and incubation system.

23. The method of claim 1 or any other preceding claim, wherein: The first cell class corresponds to induced pluripotent stem cells (iPSCs), and the second cell class corresponds to non-iPSCs.

24. The method of claim 22 or any other preceding claim, in, The plurality of cell classes include a third cell class, the third cell class corresponding to background, and The plurality of image segments further include a third image segment including pixels associated with cells of the third cell category.

25. At least one non-transitory computer readable storage medium having encoded thereon executable instructions that, when executed by at least one processor, cause the at least one processor to perform a method of regulating a process of a culture, the culture comprising a plurality of cells, cells in the plurality of cells having one or more corresponding cell types selected from a plurality of cell types, the plurality of cell types comprising a first cell type and a second cell type, the method comprising: processing the image of the plurality of cells of the culture to identify one or more cell types of cells depicted in the image from the plurality of cell types, the processing comprising: segmenting the image into a plurality of image segments by assigning individual pixels in the image to corresponding cell classes in the plurality of cell classes, the assigning comprising determining, for each of the individual pixels, a respective plurality of values ​​corresponding to the respective plurality of cell classes, each value in the respective plurality of values ​​indicating a likelihood that the individual pixel corresponds to a cell of a respective cell class in the plurality of cell classes, The plurality of image segments include: a first image segment including pixels associated with cells of the first cell category; and a second image segment including pixels associated with cells of the second cell category; determining an amount of cells corresponding to the first cell type in the culture based on the plurality of image segments into which the image is segmented; and Treatment of the culture is adjusted based on this amount.

26. A cell imaging and incubation system comprising: an imaging sensor configured to obtain an image of a plurality of cells of the culture; an incubator configured to incubate the culture; at least one processor; as well as At least one non-transitory computer-readable storage medium having encoded thereon executable instructions that, when executed by at least one processor, cause the at least one processor to perform a method of regulating a process of a culture, the culture comprising a plurality of cells, cells in the plurality of cells having one or more corresponding cell types selected from a plurality of cell types, the plurality of cell types comprising a first cell type and a second cell type, the method comprising: processing the image of the plurality of cells of the culture to identify one or more cell types of cells depicted in the image from the plurality of cell types, the processing comprising: segmenting the image into a plurality of image segments by assigning individual pixels in the image to corresponding cell classes in the plurality of cell classes, the assigning comprising determining, for each of the individual pixels, a respective plurality of values ​​corresponding to the respective plurality of cell classes, each value in the plurality of values ​​indicating a likelihood that the pixel corresponds to a cell of a respective cell class in the plurality of cell classes, The plurality of image segments include: a first image segment including pixels associated with cells of the first cell category; and a second image segment including pixels associated with cells of the second cell category; determining an amount of cells corresponding to the first cell type in the culture based on the plurality of image segments into which the image is segmented; and Treatment of the culture is adjusted based on this amount.

27. The cell imaging and incubation system of claim 26, further comprising a robotic system configured to transfer the culture within the cell imaging and incubation system between being cultured in the incubator and being imaged by the imaging sensor.

28. The cell imaging and incubation system of claim 27, wherein: The robotic system is configured to transfer the culture between being cultured and being imaged when timing conditions are met.

29. The cell imaging and incubation system of claim 28, the method further comprising: when the timing condition is met, actuating the robotic system to move the culture to the imaging sensor; as well as The imaging sensor is actuated to obtain an image of the plurality of cells of the culture.

30. The cell imaging and incubation system of any one of claims 26 to 29, further comprising an imaging device, wherein: The imaging device comprises: the imaging sensor; and A chamber is configured to receive a well plate, wherein the well plate is configured to contain the culture.

31. A cell imaging and incubation system as claimed in any one of claims 26 to 30, wherein: The method further comprises: processing the image of the plurality of cells of the culture to estimate culture information of the culture, the culture information of the culture indicating a cell density of the plurality of cells in the culture, Wherein segmenting the image comprises segmenting the image based on the culture information.

32. A cell imaging and incubation system as claimed in any one of claims 26 to 31, wherein: The method further comprises: processing the image of the plurality of cells of the culture to estimate culture information of the culture, the culture information of the culture indicating a cell density of the plurality of cells in the culture, Wherein adjusting the processing of the culture includes adjusting the processing of the culture based on the culture information and the amount of cells in the culture corresponding to the first cell type.

33. The cell imaging and incubation system of claim 32, further comprising using the culture information to estimate a number of the plurality of cells, a position of at least one cell in the plurality of cells, and / or an internuclear distance between at least two cells in the plurality of cells.

34. A cell imaging and incubation system as claimed in any one of claims 26 to 33, wherein: Adjusting the processing of the culture includes outputting a recommendation to modify the manner in which one or more substances are added to the culture to affect growth of the culture based on the amount of cells corresponding to the first cell type in the culture.

35. A cell imaging and incubation system as claimed in any one of claims 26 to 34, wherein: Adjusting processing of the culture includes outputting a recommendation to passage cells in the plurality of cells in the culture based on an amount of cells in the culture corresponding to the first cell type.

36. A method of regulating a treatment of a culture, the culture comprising a plurality of cells, the cells of the plurality of cells having one or more corresponding cell types selected from a plurality of cell types, the plurality of cell types comprising a first cell type and a second cell type, the method comprising: processing the image of the plurality of cells of the culture to identify one or more cell types of cells depicted in the image from the plurality of cell types, the processing comprising: segmenting the image into a plurality of image segments by assigning regions of the image to corresponding cell classes of the plurality of cell classes, each of the regions comprising two or more individual pixels in the image, the assigning comprising determining, for each of the regions, a respective plurality of values ​​corresponding to the respective plurality of cell classes, each value of the respective plurality of values ​​indicating a likelihood that the region corresponds to a cell of a respective cell class of the plurality of cell classes, The plurality of image segments include: a first image segment including a region associated with cells of the first cell category; and a second image segment including a region associated with cells of the second cell category; determining an amount of cells corresponding to the first cell type in the culture based on the plurality of image segments into which the image is segmented; and Treatment of the culture is adjusted based on this amount.

Images