Cell Line Development Image Characterization with Convolutional Neural Networks

Abstract

The method includes, by a computing system, receiving a query image depicting a sampling area, processing the query image using a single cluster detection model to identify one or more regions of the query image depicting clusters within the sampling area, processing the one or more regions using a cluster validation deep learning model to determine whether each depicted cluster is a cell cluster, and determining that exactly one of the identified one or more regions depicts a cluster that is a cell cluster. The method further includes processing the region depicting the cell cluster using a morphology deep learning model to determine that only one cell is present in the cell cluster and that the morphology of the cell is acceptable.

Description

[Technical field]

[0001] The present disclosure relates to a method for cell line development (CLD) image characterization that uses machine learning algorithms, such as convolutional neural networks, to analyze and classify images. [Background technology]

[0002] The field of biological therapeutics, also known as biologics or biopharmaceuticals, is growing rapidly within the biotechnology industry. The majority of biologics are protein-based therapeutics, of which recombinant proteins are the largest group with over 400 recombinant products approved worldwide. In biopharmaceutical development, recombinant proteins are produced from monoclonal cell lines. Regulatory agencies require reasonable assurance that the producing cell line is derived from a single progenitor cell. Existing guidelines require documentation of cloning procedures with imaging techniques and / or appropriate statistical details that demonstrate that the cell lines used to generate biologics are clonally derived. There are several known methods of cloning used to generate clinical cell lines, particularly mammalian cell lines, including limiting dilution cloning, single-cell printing and fluorescence-activated cell sorting. These methods use different techniques to deposit single cells into wells of multiwell plates, each with its own advantages and disadvantages in terms of ease of use, implementation costs, and seeding and cloning efficiency. Although the intention of each cloning method is single-cell deposition, this does not always occur. Upon seeding, plate wells may contain a single cell, multiple cells, or no cells at all. Thus, the method can be followed by plate imaging to confirm that indeed only one cell is present in each plate well. Images of cell culture plates immediately after plating can serve as the only direct evidence of a cloning event and are often submitted to the U.S. Food and Drug Administration for such purposes. They are also used to determine which cells should be moved forward as having a high degree of confidence that they are clonally derived. Thus, these images play a critical role and are heavily relied upon for the development of cell lines. However, the image analysis process itself has many challenges that can affect the consistency of results as well as the overall efficiency of single cell cloning. US Patent Application Publication No. 2015 / 0087240 A1 describes a method for characterizing a cell population including a set of cell subpopulations, the method comprising: receiving image data corresponding to a set of images of the cell population taken at a series of time points; and generating an analysis based on processing the set of images according to a cell stage classification module configured to automatically identify a cell class of each of the set of cell subpopulations, and a cell graph representation module configured to characterize geometric and spatial characteristics of the set of cell subpopulations. Summary of the Invention [Means for solving the problem]

[0003] Provided herein is a method for cell line development image characterization that uses machine learning to achieve automated single-cell image review.

[0004] Clinical cell lines can be generated for the development of recombinant protein-based pharmaceuticals such as monoclonal antibodies (mAbs). Disclosed herein is an accurate and detailed automated imaging workflow for documenting single cell cloning during cell line development. Image characterization is used to describe the procedure of classifying images from cell line development testing into different categories to achieve image review. In certain embodiments, the cell line development image characterization method disclosed herein can improve defect detection in images by using machine learning algorithms. The image characterization method can use artificial neural networks to train a machine learning model that can classify images of cells. The image characterization method can use transfer learning to reduce the number of samples required to train the machine learning model. Discussed herein is an exemplary cell line development image characterization method that evaluates Chinese Hamster Ovary (CHO) cells, but in fact, implementation of the image characterization method is applicable to any mammalian cell. The method can obtain images of cells to be reviewed, classify images of cells using machine learning models, and return images with uncertain defects for manual inspection while passing images of good cells.

[0005] The CLD image characterization method described herein combines digital image processing techniques such as transformation and filtering with pattern recognition techniques such as machine learning and deep learning. The method includes multiple steps that can be performed by themselves or together, in sequential or alternative order. The process filters out bad images at each step until a selected set of good images is obtained with high confidence that each well contains a single cell. The CLD image characterization method is intended to replace the first round of manual image analysis performed by researchers. Utilizing the methods disclosed herein to perform automated cell image analysis improves the consistency of results and increases the throughput of single cell image analysis.

[0006] To achieve the goal of applying machine learning algorithms to image clinical cell line characterization, certain technical challenges exist. One technical challenge includes achieving very high accuracy, since not capturing bad images can reduce the assurance that the cell line used to generate the biologic is clonally derived. Another technical challenge is ensuring that good images are accurately identified, since classifying good images as bad images can result in a large number of clones being unnecessarily rejected, resulting in an inefficient workflow. Image analysis software associated with each imager for identifying cells in a well is commercially available. However, despite setting strict parameters, the results of the algorithms are not always reliable. The solution presented by the embodiments disclosed herein to address the above challenges includes using one or more specific architectures of machine learning algorithms, which result in high classification accuracy for both good and bad images. Although the present disclosure describes characterizing CLD images in a specific manner and via a specific system, the present disclosure contemplates characterizing any suitable image in any suitable manner and via any suitable system.

[0007] One aspect of the disclosure includes a method including receiving, by a computing system, a query image depicting a sampling region; processing the query image using a single cluster detection model to identify one or more regions of the query image depicting clusters within the sampling region; processing the one or more regions using a cluster validation deep learning model to determine whether each depicted cluster is a cell cluster; determining that exactly one of the identified one or more regions depicts a cluster that is a cell cluster; and processing the region depicting the cell cluster using a morphology deep learning model to determine that only one cell is present in the cell cluster and that the morphology of the cell is acceptable. Each depicted cluster includes a cluster outline region having a cluster outline region size within a predetermined range and at least a predetermined number of points. Identifying the region includes drawing a bounding box around the cluster outline region. Determining whether each depicted cluster is a cell cluster includes validating each depicted cluster in a corresponding location of another query image depicting the sampling region. Determining whether each depicted cluster is a cell cluster includes determining whether each depicted cluster appears as a positive cluster for having cells. Determining whether each delineated cluster is a cell cluster includes determining whether each delineated cluster does not appear as a negative cluster for having a cell. Determining that only one cell is present in the cell cluster includes evaluating the number of contours detected in the cell cluster. Determining that the morphology of the cell is acceptable includes identifying a cell center region and evaluating the cell center region for circularity. Determining that the morphology of the cell is acceptable includes identifying a bright cell center region and a dark ring-like cell membrane around the bright cell center region. Determining that the morphology of the cell is acceptable includes identifying a bright cell center region and a bright ring-like region outside the cell membrane. The query image includes a fluorescent image or a bright field image.The method further includes identifying one or more large distinct objects in the query image using a contour detection algorithm during a pre-check phase of the analysis prior to processing the query image. The method further includes determining whether the query image is below a predetermined darkness threshold during a pre-check phase of the analysis prior to processing the query image. The method further includes determining whether critical clipping of the sampling region is present in the query image prior to processing the query image. The sampling region is a well, the well having a well contour in the query image defined by a plurality of well contour points and a well contour center. Determining whether critical clipping of the sampling region is present includes applying a fitting circle having a fitting circle center and a fitting circle radius to the well contour in the query image. Determining whether critical clipping of the sampling region is present further includes calculating the fitting circle radius minus a distance of the fitting circle center to each of the plurality of well contour points to identify a difference between the well contour and the fitting circle. Determining whether critical clipping of the sampling region is present includes calculating the distance from the well contour center to an image edge minus the well contour radius to determine whether the result is less than or equal to a predetermined distance threshold. Determining whether critical clipping of the sampling region exists includes determining whether consecutive well contour points lie on a single line and whether a distance between two end points of the single line meets or exceeds a predetermined distance threshold. Determining whether critical clipping of the sampling region exists further includes determining whether the single line has a local curvature of zero.

[0008] An aspect of the disclosure includes a system including one or more processors and a non-transitory memory coupled to the processor that includes instructions executable by the processor, the system being operable, when the processor executes the instructions, to receive a query image depicting a sampling region, process the query image using a single cluster detection model to identify one or more regions of the query image depicting a cluster within the sampling region, process the one or more regions using a cluster validation deep learning model to determine whether each depicted cluster is a cell cluster, determine that exactly one of the identified one or more regions depicts a cluster that is a cell cluster, and process the region depicting the cell cluster using a morphology deep learning model to determine that only one cell is present in the cell cluster and that the morphology of the cell is acceptable. Each depicted cluster includes a cluster contour region having a cluster contour region size within a predetermined range and at least a predetermined number of points. Identifying the region includes drawing a bounding box around the cluster contour region. Determining whether each depicted cluster is a cell cluster includes validating each depicted cluster in a corresponding location of another query image depicting the sampling region. Determining whether each delineated cluster is a cell cluster includes determining whether each delineated cluster appears as a positive cluster for having cells. Determining whether each delineated cluster is a cell cluster includes determining whether each delineated cluster does not appear as a negative cluster for having cells. Determining that only one cell is present in a cell cluster includes evaluating a number of contours detected in the cell cluster. Determining that the morphology of the cell is acceptable includes identifying a cell center region and evaluating the cell center region for circularity. Determining that the morphology of the cell is acceptable includes identifying a bright cell center region and a dark ring-like cell membrane around the bright cell center region.Determining that the cell morphology is acceptable includes identifying a bright cell center region and a bright ring-like region outside the cell membrane. The query image includes a fluorescent image or a bright field image. When the processor executes the instructions, the processor is further operable to identify one or more large separate objects in the query image using a contour detection algorithm during a pre-check stage of the analysis before processing the query image. When the processor executes the instructions, the processor is further operable to determine whether the query image is below a predetermined darkness threshold during a pre-check stage of the analysis before processing the query image. When the processor executes the instructions, the processor is further operable to determine whether critical clipping of the sampling region is present in the query image before processing the query image. The sampling region is a well, the well having a well contour in the query image defined by a plurality of well contour points and a well contour center. When the processor executes the instructions, the processor is operable to determine whether critical clipping of the sampling region is present by applying a fitting circle having a fitting circle center and a fitting circle radius to the well contour in the query image. When the processor executes the instructions, the processor is operable to further determine whether critical clipping of the sampling region exists by calculating a fitting circle radius minus a distance of the fitting circle center for each of a plurality of well contour points to identify a difference between the well contour and the fitting circle. When the processor executes the instructions, the processor is operable to determine whether critical clipping of the sampling region exists by calculating a distance from the well contour center to the image edge minus the well contour radius and determining whether the result is less than or equal to a predetermined distance threshold. When the processor executes the instructions, the processor is operable to determine whether critical clipping of the sampling region exists by determining whether consecutive well contour points lie in a single line and whether the distance between two end points of the single line meets or exceeds a predetermined distance threshold.Upon executing the instructions, the processor is further operable to determine whether critical clipping of the sampling region exists by determining whether the single line has a local curvature of zero.

[0009] An aspect of the disclosure includes one or more computer-readable non-transitory storage media embodying software operable, when executed, to receive a query image depicting a sampling region, process the query image using a single cluster detection model to identify one or more regions of the query image depicting a cluster within the sampling region, process the one or more regions using a cluster validation deep learning model to determine whether each depicted cluster is a cell cluster, determine that exactly one of the identified one or more regions depicts a cluster that is a cell cluster, and process the region depicting the cell cluster using a morphology deep learning model to determine that only one cell is present in the cell cluster and that the morphology of the cell is acceptable. Each depicted cluster includes a cluster contour region having a cluster contour region size within a predetermined range and at least a predetermined number of points. Identifying the region includes drawing a bounding box around the cluster contour region. Determining whether each depicted cluster is a cell cluster includes validating each depicted cluster in a corresponding location of another query image depicting the sampling region. Determining whether each delineated cluster is a cell cluster includes determining whether each delineated cluster appears as a positive cluster for having cells. Determining whether each delineated cluster is a cell cluster includes determining whether each delineated cluster does not appear as a negative cluster for having cells. Determining that only one cell is present in a cell cluster includes evaluating a number of contours detected in the cell cluster. Determining that the morphology of the cell is acceptable includes identifying a cell center region and evaluating the cell center region for circularity. Determining that the morphology of the cell is acceptable includes identifying a bright cell center region and a dark ring-like cell membrane around the bright cell center region. Determining that the morphology of the cell is acceptable includes identifying a bright cell center region and a bright ring-like region outside the cell membrane. The query image includes a fluorescent image or a bright field image.The software, when executed, is further operable to identify one or more large distinct objects in the query image using a contour detection algorithm during a pre-check phase of the analysis prior to processing the query image. The software, when executed, is further operable to determine whether the query image is below a predetermined darkness threshold during a pre-check phase of the analysis prior to processing the query image. The software, when executed, is further operable to determine whether critical clipping of the sampling region is present in the query image prior to processing the query image. The sampling region is a well, the well having a well contour in the query image defined by a plurality of well contour points and a well contour center. The software, when executed, is operable to determine whether critical clipping of the sampling region is present by applying a fitting circle having a fitting circle center and a fitting circle radius to the well contour in the query image. The software, when executed, is further operable to determine whether critical clipping of the sampling region is present by calculating the fitting circle radius minus the distance of the fitting circle center to each of the plurality of well contour points to identify a difference between the well contour and the fitting circle. When executed, the software is operable to determine whether critical clipping of the sampling region exists by calculating the distance from the well contour center to the image edge minus the well contour radius and determining whether the result is less than or equal to a predetermined distance threshold. When executed, the software is operable to determine whether critical clipping of the sampling region exists by determining whether consecutive well contour points lie on a single line and whether the distance between two end points of the single line meets or exceeds a predetermined distance threshold. When executed, the software is further operable to determine whether critical clipping of the sampling region exists by determining whether the single line has a local curvature of zero. [Brief description of the drawings]

[0010] [Figure 1] This illustrates the challenges of single-cell image review.

[0011] [Diagram 2] 1 illustrates an exemplary process for CLD image selection.

[0012] [Diagram 3] 1 illustrates an exemplary single cell cloning workflow.

[0013] [Figure 4] 1 shows an exemplary flowchart illustrating three tasks in automated single-cell image review, including well region check, single cluster detection and validation, and morphology review.

[0014] [Diagram 5] 1 illustrates an exemplary procedure for image labeling.

[0015] [Figure 6A] An exemplary bright field (BF) image is shown, which is too dark to detect the outline of the wells and the cells cannot be clearly seen.

[0016] [Figure 6B] 1 shows an example BF image that is too dark with 52 99th percentile pixels.

[0017] [Figure 6C] 1 shows an example BF image that is too dark with 63 99th percentile pixels.

[0018] [Figure 6D] 1 shows an example BF image with 92 99th percentile pixels.

[0019] [Figure 6E]1 shows an example BF image with 246 99th percentile pixels.

[0020] [Figure 7A] 1 shows an example of a fluorescence (FL) image with debris that can be falsely detected as a cell.

[0021] [Figure 7B] 1 shows an example of a BF image with debris that can lead to erroneous detection of well contours.

[0022] [Figure 7C] 1 shows an example of an image in which debris is detected.

[0023] [Figure 8A] 1 illustrates a first well area check criterion applied to an exemplary well that calculates the fitted circle radius minus the distance of the fitted circle center to each of a number of well contour points.

[0024] [Figure 8B] 8B shows a second well area check criterion applied to the example well of FIG. 8A that calculates the maximum distance to the next point within the well contour, and a third well area check criterion applied to the example well of FIG. 8A that calculates the local curvature.

[0025] [Figure 8C] 8B illustrates a fourth well area check criterion applied to the example well of FIG. 8A that calculates the shortest distance between the well contour center and the four edges of the image.

[0026] [Figure 9] 13 shows an example image of a clipped well with clipping prediction based on maximum distance to the next point in the well contour and distance-radius calculations.

[0027] [Figure 10]1 shows an exemplary procedure for detecting clusters from FL well images using contour detection.

[0028] [Figure 11] 1 shows an example image of two clusters correctly detected using blob detection.

[0029] [Figure 12A] 1 shows an example image using a strong pixel threshold resulting in weak signals and missed cluster detections.

[0030] [Figure 12B] 1 shows an example image in which a weak pixel threshold is used, resulting in strong background signal and many false positive cluster detections.

[0031] [Figure 12C] 1 shows an example image where debris was mistaken for a cell using single cluster detection.

[0032] [Figure 13] 1 illustrates an exemplary cluster validation deep learning model for identifying cell clusters.

[0033] [Figure 14A] 1 shows an exemplary original enlarged cell cluster image in grayscale.

[0034] [Figure 14B] 14B shows a three-dimensional (3D) surface of the cell cluster of FIG. 14A.

[0035] [Figure 15] 1 illustrates an exemplary method for determining cell morphology using image processing.

[0036] [Figure 16] An example of the success of the cell contouring method is shown.

[0037] [Figure 17] Examples of imaging issues and challenges are shown, including cells at the well edge, out-of-focus images, and dark images.

[0038] [Figure 18] 1 illustrates an exemplary morphological deep learning model based on the VGG16 architecture.

[0039] [Figure 19] 1 illustrates a flowchart of an exemplary method by a computing system for performing single cluster detection, single cluster validation, and morphology review.

[0040] [Figure 20A] 1 shows an example visualization of filter patterns after the first convolutional layer.

[0041] [Figure 20B] 13 shows an example visualization of the filter patterns after the second convolutional layer.

[0042] [Figure 20C] 13 shows an example visualization of filter patterns after the fourth convolutional layer.

[0043] [Figure 21] 1 shows an example output saliency map (B) for an input image (A).

[0044] [Figure 22] 1 shows an exemplary activation-maximized image of a cell with good morphology (A) and an exemplary activation-maximized image of a cell with poor morphology (B).

[0045] [Figure 23A] 1 shows an example of a prediction error based on cells at the well edge.

[0046] [Figure 23B]1 shows an example of a prediction error based on a dark image.

[0047] [Figure 24] 1 illustrates an example of a computing system. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0048] Introduction Current implementations require instrument-generated analysis of images captured at the time of plating, also referred to as day 0 images. Imager algorithms associated with the imager are used to determine whether one or more cells are present in the well. However, variations in cell morphology and image quality can affect consistent image interpretation and therefore the accuracy of the results. Figure 1 shows an example of a single cell review challenge of a magnified black and white BF image captured on day 0. Different background lighting conditions and reflections can result in dark images, while cells outside the camera's focal plane can result in blurry images. Additionally, cells close to the well edge resulting from centrifugation can be difficult to find and lead to false negatives. Clipped and therefore incomplete images can result in cell leakage, as shown in the top left image of Figure 1 in callout 102. The presence of well artifacts can hinder visualization of cells. Varying morphologies of cells can also pose challenges for the imager analyzer, which must decipher between single whole cells and cells that have begun division, as shown in the second image at the top. All such situations can lead to errors. Therefore, there is a need for a fast and reliable method for automated CLD image characterization that can ensure that only single cells with good morphology are advanced.

[0049] Clinical cell lines can be generated for the development of recombinant protein-based pharmaceuticals such as monoclonal antibodies (mAbs). Disclosed herein is an accurate and detailed automated imaging workflow for documenting single cell cloning during cell line development. Image characterization is used to describe the procedure of classifying images from cell line development testing into different categories to achieve image review. In certain embodiments, the cell line development image characterization method disclosed herein can improve defect detection in images by using machine learning algorithms. The image characterization method can use artificial neural networks to train a machine learning model that can classify images of cells. The image characterization method can use transfer learning to reduce the number of samples required to train the machine learning model. Discussed herein is an exemplary cell line development image characterization method that evaluates Chinese Hamster Ovary (CHO) cells, but in fact, implementation of the image characterization method is applicable to any mammalian cell. The method can obtain images of cells to be reviewed, classify images of cells using machine learning models, and return images with uncertain defects for manual inspection while passing images of good cells.

[0050] The CLD image characterization method described herein combines digital image processing techniques, such as transformation and filtering, with pattern recognition techniques, such as machine learning and deep learning. The method includes multiple steps that can be performed by themselves or together, in sequential order, or in alternative orders. In one embodiment shown in FIG. 2, the process of CLD image selection includes four steps performed in a specific order. The first step is a pre-check of the plate well images acquired on day 0 to determine if the plate wells contain defects or are suitable for further analysis. During the pre-check phase of the analysis, it is determined if the image is too dark and / or if the wells contain obvious debris. In the second step, it is determined if the entire well area is captured in the image or if the image is clipped. In the third step, it is determined if the well contains only one fluorescent cell cluster. In the fourth step, it is determined if the only single fluorescent cell cluster is a single cell and if the cell has good morphology. The amount of calculations required to perform each step increases, with the third and fourth steps, which involve deep learning models, being the most computationally intensive. This process filters out bad images at each step until a selected set of good images is obtained with high assurance that each well contains a single cell. The method of CLD image characterization is intended to replace the first round of manual image analysis performed by researchers. Utilizing deep learning models to perform such automated cell image analysis will improve the consistency of results and increase the throughput of single cell image analysis.

[0051] The following sections discuss: (1) single cell cloning workflow, (2) application of CLD image characterization to the single cell cloning workflow, (2a) imager acquisition and analysis of images, (3) image pre-check, (3a) dark images, (3b) obvious debris in the well, (4) well area check, (4a) criterion 1: distance from fitting circle radius to the well contour point of the fitting circle center, (4b) criterion 2: maximum distance to the next point on the well contour, (4c) criterion 3: local curvature, (4d) criterion 4: distance between the well contour center and the four edges of the image. Various aspects of CLD image characterization are described with respect to: (4) shortest distance between clusters, (4e) well area check: exemplary image characterization results, (5) single cluster detection and validation, (5a) single cluster detection method 1: finding cluster contours, (5b) single cluster detection method 2: blob detection, (5c) single cluster validation: deep learning model, (6) morphology review, (6a) morphology review: image processing and analysis, (6b) morphology review: deep learning model, (7) cell detection and cell morphology: CNN prediction performance, and (8) systems and methods. Single Cell Cloning Workflow

[0052] Single cell cloning is a critical step during cell line development. Figure 3 shows an example of a single cell cloning workflow. In this example, Chinese Hamster Ovary (CHO) cells are stained using a fluorescent dye and seeded into a 384-well plate containing cell culture medium using limiting dilution cloning or single cell deposition. The plate is centrifuged at 1000 rpm for 3 minutes to bring the cells down to the bottom of the wells and then imaged immediately. Full resolution bright field (BF) and fluorescent (FL) images of each well are acquired using a Celigo® imager from Nexcelom, which has a resolution of 1 micron per pixel. These images are referred to herein as query images. The Celigo® imager algorithm is then used to count and determine the number of fluorescent single cells present in the wells immediately after plating or on day 0. The cells are incubated for a set number of days, for example 14 days, until the cells are confluent. At confluence, all 384 wells are imaged using the Celigo® imager. Confluence images are then analyzed using the Celigo® Imager algorithm to determine the percentage of those well areas covered by proliferating cells. Clones whose images are determined to be good images on day 0 by the Celigo® Imager algorithm and are greater than 25% confluence are seeded into 96-well plates using robotic picking to scale up for clone selection in fed-batch production cultures. The study then performs a first round of image analysis to manually review the day 0 images determined to be good by the Celigo® Imager to ensure that they are in fact good images of high titer clones. This first round of manual review performed by the researcher consists of up to 192 single-cell images per cell line and is a laborious, time-consuming, and subjective process. Additional manual reviews are performed at different stages of the cloning workflow, including a second manual review performed by a subject matter expert (SME), followed by a third manual review by a senior SME or group leader prior to cell banking.

[0053] The cell number data at plating combined with cell proliferation data after plate incubation is used to determine which clones are picked by the automated hit-picking workcell. High confidence in the results of the day 0 image analysis is desired, since only wells derived from a single cell are cultured and further scaled up for that clone.

[0054] As an optional first step in this workflow, a baseline BF image of each well containing cell culture medium can be taken prior to cell deposition to capture the presence of plate artifacts. This serves as an additional layer of information and allows subtraction of artifacts during manual image review. Identifying distinctions between the query image and the baseline image helps distinguish cells from artifacts after plating. Application of CLD image characterization to single cell cloning workflows

[0055] The method described herein is provided as a fast and reliable method for automated CLD image characterization and is intended to replace the first round of manual image analysis performed by the researcher in the exemplary single cell cloning workflow of Figure 3. However, it is understood that the method is not limited to its use and can be applied to other single cell cloning workflows, including those using different size plates, cell types, imagers, cloning methods, image types, etc. Additionally, the query image may be associated with or depict any type of sampling area and is not limited to plate wells.

[0056] The described method of CLD image characterization uses machine learning and deep learning algorithms to achieve an automated single cell image review. The image characterization applies multiple techniques to perform three specific tasks or achieve three specific goals: 1) checking well area, 2) detecting and validating single clusters, and 3) reviewing morphology. FIG. 4 shows an exemplary flow chart illustrating these three tasks or goals. For the first task of capturing as much well area as feasible, the method checks the completeness and circularity of the area of ​​each well based on the BF image. Using contour detection and Hough circle detection, wells with clipping are identified. For the second task of single cluster detection and validation, the method checks the total number of fluorescent clusters in the FL image and validates the clusters as cell clusters in the BF image. Using edge detection and deep learning, images with multiple cell clusters are identified. For the third task of morphology review, the method checks whether the cell clusters are single cells and checks the morphology of single cells in the BF image. Using edge detection and deep learning, cells are classified into different categories based on their assessed morphology.

[0057] An exemplary procedure for image labeling is shown in Figure 5. The less confident prediction is based on the probability of the predicted classification, with a lower probability meaning a lower confidence. The labeling procedure can be in order of increasing confidence, with less confident images being labeled first. As will be explained in more detail later, the labels in the first round of the procedure can be created manually or using a computer version of the method, such as a contour detection algorithm or a blob detection algorithm. The labeling procedure improves efficiency, as the researcher does not need to manually label all images. Furthermore, deep learning models can be improved by data augmentation. Data augmentation helps to improve the predictive performance of deep learning models by constructing new images from available images using image transformations, including flip, zoom, shift, rotate, etc. Although already in a usable stage, performance will improve by continuing to train the model to a more mature stage. Imager acquisition and image analysis

[0058] After the plate is seeded using another cloning method such as single cell deposition or limiting dilution cloning, and then centrifuged to bring the cells to the bottom of the well, FL and BF images of each plate well are taken. The camera is set to focus on the bottom of the well so that the cells are in the focal plane. The imager acquires a single image of the entire well without image stitching. High-resolution live-cell FL images at the time of plating reliably identify the presence and location of live CHO cells or other mammalian cells previously stained with short-lived fluorescent dyes. High-resolution BF images at the time of plating provide sufficient resolution to reliably identify the cells. In BF imaging, light may be illuminated from above the well and images may be acquired from below the well, or light may be illuminated from below the well and images may be acquired from above the well.

[0059] Querying the image acquired by the high resolution imager can capture one or more distinct objects within the well. A distinct object is a distinct object outline area identified in the image, which can be a single cell cluster, multiple cell clusters, debris or artifacts, well defects such as broken edges, or a combination thereof. After acquiring the image, optimization of the imager's single cell cloning imaging analysis algorithm is performed to identify cells within the well. For algorithms specific to the Celigo® imager, as an example, a cell size cutoff of 25 microns is set during automated image analysis to eliminate large clumps of cells. Although CHO cells have a size approaching 15-20 microns, setting the cell size filter to that range dramatically reduces the number of wells that the analysis software recognizes as having cells in the FL image analysis, thereby limiting the throughput of the system. Despite the setting, it has been observed that a few doublets and small cell clumps up to 35 microns pass through the software analysis filter. Additionally, centrifugation can cause cells to be located at or near the well edge wells rather than in the center of the well bottom. This can occur in up to 60% of the wells. In such cases, the cells are no longer in the focal plane of the camera, so the images appear blurry, making it difficult for the imager software to identify the cells. For at least these reasons, image analysis on day 0 by the Celigo® image analysis algorithm is not completely reliable.

[0060] The method of CLD image characterization described herein is provided as an alternative to the first round of manual imager analysis of day 0 images. The method utilizes machine learning and deep learning algorithms with selected parameters to ensure that cells are accurately and precisely identified within the wells. It results in more consistent and reliable cell count data at the time of plating than that obtained from the imager algorithm. Thus, when the cell count data obtained using the method disclosed herein is combined with the cell proliferation data after plate incubation to determine which clones should be advanced, there is a high degree of confidence that only wells originating from a single cell will be cultured and further scaled up for that clone.

[0061] In the following description of the methods of CLD image characterization, it is understood that the disclosed functions / algorithms and associated parameters for performing specific functions and calculations are merely exemplary, and that the use of other libraries is within the scope of this disclosure. Furthermore, because pixel ranges depend on the imager, capture resolution, algorithms used, and other factors, their values ​​may actually differ from those disclosed herein. Image pre-check

[0062] The CLD image characterization method can start with an optional pre-check of the day 0 BF images. This step is performed to determine if they have defects or are suitable for further analysis. During the pre-check phase of the analysis, it is determined if the images are too dark and / or if the wells contain obvious debris. Dark Image

[0063] The BF images are pre-reviewed for darkness. A threshold function is used to set the threshold of the binary conversion that makes the well different from the background to 25 pixels. Anything too dark can lead to errors in detecting the well contour, which can lead to errors in detecting well clipping, as will be further explained later. Images that are too dark can also lead to errors in detecting cells within the well contour. Figure 6A shows an example of a BF image that is too dark to detect the well contour 604, with the left part of the well being as dark as the background, making it impossible to identify the cell shown in box 602.

[0064] To pre-check the darkness, the 99th percentile pixel of the BF image is used as a reference. A BF image is considered too dark and is flagged if its value is less than 80 pixels. Images above this pre-defined darkness threshold are acceptable and considered good images and continue for further analysis. Images below this darkness threshold are considered bad images for containing defects and are filtered out. Figures 6B and 6C show other examples of BF images considered bad for being too dark, with 99th percentile pixels of 52 and 63, respectively, below the pre-defined darkness threshold. The exemplary BF images of Figures 6D and 6E are considered good and have 99th percentile pixels of 92 and 246, respectively, above the pre-defined darkness threshold. Visible debris in the well

[0065] FL images are pre-reviewed for the presence of obvious debris. Apparent debris can be large clear objects in the well with large clear object contour area sizes within a predefined range. The presence of such debris can lead to errors in cell detection, so images containing them are flagged. Figure 7A shows two examples of FL images with some debris identified in boxes, such as box 702, that may be erroneously detected as cells. Debris in BF images can also lead to errors in detecting well contours. In Figure 7B, an example of a BF image is shown with debris 704 that may be erroneously considered as a broken well edge.

[0066] Contour detection algorithms are used to pre-check for obvious debris. These algorithms are similar to those used for cluster detection and are described in more detail later. For FL images, the threshold for binary conversion is set to 40 pixels using a threshold function, and the contour area, or large distinguishable contour region size, is set between 5000 and 1 million pixels. For BF images, the threshold for binary conversion is set to 70 pixels, and the contour area, or large distinguishable contour region size, is set between 5000 and 1 million pixels. Images without such debris are considered good images and continue for further analysis. Images with obvious debris are considered bad due to containing defects and are excluded. Figure 7C shows additional example images in which debris 706 was detected, some showing large or opaque debris that may cover cells. Well Area Check

[0067] To ensure that a complete review of the wells to identify cells can be done, a well area check step can be performed. Clipped wells are wells that have a part of them cut off in the image. It can be the result of slight alignment issues between the camera and the well. Critical clipping is clipping of more than the width of a single cell, which can be 15-20 microns for CHO cells. Such critical clippings can hide cells in that part of the well, and therefore images with them are considered as bad images for containing defects. Non-critical clippings smaller than the width of a cell can be considered as good images. To check the well area and ensure there is no critical clipping, a tool is used to detect the contours of the wells based on the BF image, which generally has good contrast between the wells and the image background. The threshold for the binary conversion to differentiate the wells from the background is set to 25 pixels using the cv2.threshold function from the opencv package. Well contours are found using cv2.findContours function from opencv package with option cv2.CHAIN_APPROX_SIMPLE, the number of points defining the well contour is set to at least 100 pixels, and the area of ​​the well contour is set to between 1 and 5 million pixels. The well contours are fitted to a circle, fitting circle, using the Hough circle transform algorithm (cv2.HoughCircles function from opencv package) and the following parameters: method=cv2.HOUGH_GRADIENT, dp=2, minDist=1500, paraml=10, param2=30, minRadius=300, maxRadius=0. The fitting circle has a radius and a center.

[0068] An image processing tool utilizing four well area checking criteria or methods is implemented to detect well area clipping, although any number and combination of the four criteria may be used. Four methods for detecting well clipping applied to the same exemplary well are shown in Figures 8A-8C. Images determined to have critical clipping of well area are considered bad images. Those wells may be re-imaged with the same or a different imager on day 0, or the images may be excluded and not continued for further analysis. Criterion 1: Fitting circle radius - distance from fitting circle center to well contour point

[0069] This method can detect not only well clipping but also other well defects. The first well area check criterion detects the difference between the well contour (i.e., "contour") and its fitting circle (i.e., "circle fitted by contour") shown in the left image of Figure 8A. The difference is calculated as the fitting circle radius minus the distance of the fitting circle center (i.e., "circle center") to each of the multiple well contour points. The calculated difference is plotted starting from the top of the well contour and moving in a clockwise direction as shown in the right image of Figure 8A. The find_peaks function of the python scipy.signal package (parameters: prominence = 0.7, width = 100, height = 30) is used to detect the peak or maximum difference between the well contour and its fitting circle based on the calculation. The large peak in the middle corresponds to the clipping of the bottom side of the well, and the two smaller peaks after that correspond to the small clipping on the left side of the well and the small broken edge on the top left side of the well, respectively. Criterion 2: Maximum distance to the next point within the well contour

[0070] The second well area check criterion detects well clipping by detecting straight lines in the well contour shown in the left image of Figure 8B. If several consecutive well contour points lie in a single line, as happens with clipped edges, the image processing algorithm stores only the two end points of the edge. The distance between two consecutive contour points is calculated and a predefined maximum distance is used as a criterion for clipping. A predefined distance threshold is set to 150 pixels, such that a distance between consecutive well contour points of 150 pixels or more is predicted as clipping. The larger the calculated distance, the larger the clipping. In the figure, a large distance is calculated between the two end points of the edge, thereby indicating one large clipping of the bottom side of the well. One small clipping of the left side of the well is also detected. Criterion 3: Local curvature

[0071] The third well area check criterion checks the local curvature of the line of criterion 2, with the local curvature of a straight line taken as 0. The localCurvature function of the R EBImage package is used to calculate the local curvature of the well contour, with the length of the continuous curvature window set to 101 pixels. The three clipping criteria described so far are shown in the circular plots in the right image of FIG. 8B. For the bottom and left clipping regions of the well, the corresponding distances calculated for criterion 1 (solid line 802) and criterion 2 (dashed line 804) are both large, while the local curvature calculated for criterion 3 (dotted line 806) is 0. Criterion 4: Shortest distance between the center of the well outline and the four sides of the image

[0072] The fourth well area confirmation criterion shown in FIG. 8C detects whether the well contour center (i.e., "circle center") is too close to an image edge. This method for detecting well clipping is based on the fact that clipping can only occur on one of the four sides of the image and is parallel to the image side. The shortest distance between the well contour center and the four sides of the image is calculated. A well is predicted as clipped if the distance, calculated as the distance from the well contour center to the nearest image edge minus the well contour radius, is less than or equal to a predefined distance threshold. The threshold is set to 5 (i.e., distance-radius≦5 pixels). The closer the calculated distance is to the threshold, the less clipping there is. Well Area Check: Exemplary Image Characterization

[0073] In one example, the second and fourth criteria are used to detect well clipping. A well is predicted to be clipped if the maximum distance to the next point in the well contour is 150 pixels or more, or if the distance from the well contour center to the nearest image edge minus the well contour radius is 5 pixels or less. Table 1 is a confusion matrix for well clipping prediction using the second and fourth criteria, showing that the algorithm works and the criteria are good indicators of clipped wells. 390 false positives are cases where the algorithm fails to detect small clipping, which is acceptable and possible if the clipping is not significant. If the calculated distances for the two criteria are close to their respective predefined threshold distances, further review can be done to minimize the number of false positives reported. [Table 1]

[0074] FIG. 9 shows an example of a clipped well with a clipping prediction based on a calculated value of the maximum distance to the next point in the well contour (criterion 2) and the distance from the well contour center to the nearest image edge minus the well contour radius (criterion 4, i.e., distance minus radius). In the bottom left image, an accurate prediction of clipping was made. Because the size of the clipping is small, this can be considered a non-critical clipping that is not large enough to hide a cell such as the cell in box 902. In the right-most image, an incorrect prediction may have been made because the well contour is not an exact circle. Single-cluster detection and validation

[0075] The image acquired by the high resolution imager can capture one or more distinct objects in the well. The distinct objects can be single cell clusters, multi-cell clusters, debris or artifacts, well defects such as broken edges, or combinations thereof. The disclosed method of CLD image characterization can be used to distinguish clusters, which are distinct objects with specific characteristics, from other types of distinct objects. For the second task or purpose of single cluster validation, the method checks the total number of fluorescent clusters in the FL image and validates the clusters as cell clusters in the BF image. Using edge detection and deep learning, images with multiple cell clusters are identified. Images with only one cell cluster detected are considered good images and continue for further analysis. Images without cell clusters or with multiple cell clusters are considered bad images for containing defects and are excluded. Single cluster detection method 1: Finding the cluster contours

[0076] FIG. 10 shows an exemplary procedure for detecting fluorescent clusters in FL well images using a single cluster detection model. In such a procedure, software analysis filters and contour detection algorithms are used to identify points of interest within the wells. The first and second steps involve reading the FL image in grayscale (cv2.imread function with option 0) and converting it to a binary black and white image (cv2.threshold with option cv2.threshold=30 pixels). The third step finds distinct object contour regions, shown as black areas in the image (cv2.findContours function with option cv2.CHAIN_APPROX_TC89_KCOS), and adds contours around them. In the fourth step, the distinct object contour regions are then filtered based on their area and number of points. In the example of FIG. 10, the distinct contour region size is set to a predefined range of 50 to 3000 pixels, and the predefined number of points in the distinct contour region is set to at least 10 pixels. In the fifth step, a bounding box is drawn or added around the selected distinct object contour regions that meet the size and points criteria of the previous step using the cv2.rectangle function and an extension of 10 pixels on each edge of the contour. The selected distinct object contour regions are clusters. As a cluster, the distinct object contour regions are considered to be cluster contour regions that have a cluster contour region size within a predetermined range of 50 to 3000 pixels and have at least a predetermined number of points of 10 pixels.

[0077] As mentioned above, a similar contour detection algorithm is used to pre-check images for the presence of clear debris that is large distinct objects. In that application, the parameters include a predetermined large distinct object contour region size range set between 5000 and 1 million pixels. Thus, the predetermined large distinct object contour region size range for identifying clear debris is larger than the predetermined distinct object contour region size range for identifying clusters. Single Cluster Detection Method 2: Blob Detection

[0078] An alternative single cluster detection model to contour detection is blob detection. Candidate clusters are found in the FL images using a blob detection algorithm (cv2.SimpleBlobDetector_Params, cv2.SimpleBlobDetector_create, cv2.detector functions from the opencv package). The images are binary images with a pixel threshold set to 30. Filtering parameters include a predefined number of points within the individual object contour region set to at least 10 pixels, and an individual object contour region size set to a predefined range between 50 and 3000 pixels at 2052x2052 scale. A box is added around selected individual object contour regions that meet the size and points criteria of the previous step (cv2.rectangle function), extending 10 pixels on each side of the contour. These selected individual object contour regions are the clusters.

[0079] In general, the single cluster detection models of contour detection and blob detection described herein have similar performance in detecting clusters. FIG. 11 shows an example image of two clusters, one cluster in each box 1102 and 1104, correctly detected using blob detection. However, there are some challenges associated with the described single cluster detection method. One such challenge is based on threshold. Using a strong pixel threshold may result in a signal that is too weak to detect all clusters in a well, leading to false negatives. Cell leakage due to weak signals is shown in FIG. 12A. Similarly, a weak pixel threshold, such as 20 or 25 pixels, may result in too strong background signal, resulting in many false positive cluster detections. The false positives identified in some boxes 1202 in FIG. 12B may be debris or plate artifacts. Threshold-based errors are amplified in poor quality images, such as dark or out-of-focus images. Single Cluster Validation: Deep Learning Models

[0080] Cluster detection is used to count the number of clusters and identify their location within the well. However, cluster detection is limited in the information it provides about the cluster itself. Since ensuring that there is only one cell in the well on day 0 is the primary goal of single cell cloning, it would be beneficial to know if the clusters detected in the well contain at least one cell. The performance of single cluster validation after single cluster detection is a step towards achieving that goal. In single cluster detection, one or more locations and / or regions of the query image, particularly the FL image, that depict clusters within the sampling region (e.g., the well) are identified by processing the query imaging using a single cluster detection model. The location of each cluster can be returned by the single cluster detection model as coordinates within the image (e.g., x / y pixel location), a bounding box location (e.g., again, by x / y pixel boundaries), a contoured region that contains the cluster, or another similar portion of the image (also referred to throughout as the region of the query image). As shown as the final step in Figure 10, single cluster validation confirms the presence of at least one cell within the corresponding location of the BF image identified using single cluster detection, thereby ensuring that the cluster contains at least one cell. In other words, single cluster validation confirms that the cluster is a cell cluster. Images that do not have the identified image regions are considered bad images and are rejected.

[0081] In other words, single cluster validation enhances single cluster detection. It uses a cluster validation deep learning model to distinguish cell clusters from non-cell clusters, such as debris in a well. The implementation of single cluster validation can prevent clones from being discarded or moved forward due to plate artifacts and debris being misinterpreted as cells, or cells as plate artifacts and debris. One or more regions of the query image identified by the single cluster detection model as depicting clusters in the sampling area are then processed by a cluster validation deep learning model to determine whether each depicted cluster is a cell cluster. The output of the cluster validation deep learning model is a binary classification of the query image as either a good image or a bad image.

[0082] If the cluster validation deep learning model determines that exactly one of the regions depicts a cluster that is a cell cluster, the image is classified as a good image, and therefore the sampling region contains only one cell cluster. If the cluster validation deep learning model determines that none of the regions depicts a cluster that is a cell cluster, or multiple regions depict a cluster that is a cell cluster, the image is classified as a bad image, and therefore the sampling region does not contain a cell cluster or contains multiple cell clusters, respectively. In practice, generally, about 80% of the clusters identified using single cluster detection are actual cell clusters, and the rest are not cell clusters. For example, if the debris meets the same size and point criteria required for a cluster in single cluster detection, the debris can be identified as a cluster. This is shown in the example image of FIG. 12C in box 1204. The use of single cluster validation following single cluster detection verifies that there are no cells in the cluster and that the cluster is debris and not a cell cluster. Given that the single cluster does not contain cells, it is concluded that the well does not contain cells. Therefore, the image is deemed a bad image containing defects and is rejected without further analysis being performed.

[0083] An example of a cluster validation deep learning model is shown in FIG. 13. The input to the model is one or both of the regions of the FL image and the regions of the BF image, and the output from the model is the identification of cell clusters and their locations within the well. Additionally or alternatively, the model can return a binary decision of whether the entire image including the analyzed region is a good image or a bad image. In one specific embodiment, identifying cell clusters within the region of the image includes calculating a confidence score for the candidate cell clusters and determining a cell cluster designation for the candidate cell cluster based on a comparison of the confidence score to a predefined threshold score, where the cell cluster designation can be a cluster outline portion of the image that includes at least one cell. In one exemplary embodiment, the architecture of the deep learning model is a VGG16 CNN model, although other neural network model architectures may also be used.

[0084] For model training, the FL and BF images (or regions thereof) of the detected clusters using the high threshold are used in combination to train the model on what a cell cluster (i.e., a cluster positive for having cells) looks like. In the example of FIG. 13, there is one cell cluster identified in the upper strong threshold FL image in box 1302 using blob detection, which is verified in the corresponding location of the lower strong threshold BF image. Additionally, the FL and BF images of randomly selected images are used in combination to train the model on what a non-cell cluster (i.e., a cluster negative for having cells, such as debris) looks like. In this example, random selection of other regions of the well where no cell clusters are present is used to help the algorithm understand the difference between cell clusters and non-cell clusters. Such training can be more accurate as it helps in the calculation of confidence scores and confusion matrices. Deep learning models are improved by data augmentation to build new images from available images using image transformations including flip, zoom, shift, rotate, etc., which helps improve the predictive performance of the deep learning model. The FL and BF images of the clusters detected using a weak threshold to include a large number of clusters are also fed into the deep learning model, which gives a prediction for each candidate cluster. This exemplary cluster validation deep learning model has an accuracy of 99.3% on the validation data.

[0085] The deep learning model described herein for single cluster validation determines whether an image is good or bad depending on the prediction it gives to each candidate cluster. If there are two or more candidate clusters in the image that are likely to have cells, or if there are no candidate clusters in the image, the image is considered to be bad. It is filtered out and does not proceed to further analysis. If only one candidate cluster is likely to have cells, the image is considered to be good and proceeds to morphological review.

[0086] As introduced in the previous paragraph, transfer learning and image augmentation are used to improve the performance of cluster validation deep learning models. Transfer learning refers to training a base or auxiliary CNN on an auxiliary dataset and task, thereby developing certain parameters, and then utilizing a target dataset and task to further refine those parameters. Transfer learning is performed on the same model architecture using ImageNet or other custom datasets. For cluster validation in the CLD characterization method disclosed herein, a plurality of first target training samples are accessed, the first target training samples include a plurality of first target training images associated with a sampling region and a plurality of cell cluster instructions associated with the plurality of first target training images, respectively. The plurality of first auxiliary training samples are selected based on the first target training sample, the first auxiliary training samples include a plurality of first auxiliary training images and a plurality of first content categories associated with the plurality of first auxiliary training images, respectively. Then, the cluster validation deep learning model is trained by training a first auxiliary machine learning model based on the plurality of first auxiliary training samples, and training a cluster validation deep learning model based on the trained first auxiliary machine learning model and the plurality of cell cluster instructions, wherein the plurality of first content categories associated with the plurality of first auxiliary training images do not match the plurality of cell cluster instructions associated with the plurality of first target training images. Morphological Review

[0087] Once a well is determined to have no critical clipping by the well area check and contains only one cell cluster by single cluster validation, the image continues to morphological review, where image processing and analysis or deep learning models are used to determine the number of cells in the cell cluster and the acceptability of the cell morphology. Morphological review: Image processing and analysis

[0088] Clusters that are verified as cell clusters may contain, for example, a single cell, multiple overlapping cells, or cells that have already started dividing. In the latter two cases, those clones will not be advanced. With morphology review, the goal is to identify cell clusters that contain a single cell with good morphology. The method of reviewing cell morphology is based on the observation that in an image, cells have a bright center and a dark cell membrane boundary. FIG. 14A shows an example of a cell cluster with a bounding box added around it during cluster detection. The cell cluster is enlarged to provide as the original grayscale image for morphology review. FIG. 14B shows the 3D surface of the cell cluster of FIG. 14A, showing the cell center as a hill and the cell membrane as a surrounding valley. The method of reviewing morphology described herein detects and constructs cell contours, and then predicts the number of cells (e.g., the number of detected contours) and cell morphology (e.g., the circularity of the cells) based thereon.

[0089] Figure 15 shows an exemplary method of determining cell morphology using multiple image processing techniques, including nine steps. The procedure described is merely exemplary. It is understood that many other image processing techniques for reviewing cell morphology can be used. Procedures having additional or fewer steps, or steps performed in an alternative order, are within the scope of this disclosure.

[0090] Step 1: Removing well edges. In the first step, bright regions containing cells are identified in the FL image (top right image) and the corresponding location of this region in the BF image (top left image) is replaced by the median value in the BF image (bottom image). This can sufficiently remove the well edges and black background in the BF image to remove disturbances that may arise from them during further analysis.

[0091] Step 2: Enhance the image. In the second step, the image points are enhanced using B-spline interpolation (cv2.resize function with ratio 20).

[0092] Step 3: Increase contrast. In the third step, the contrast of the BF image is increased (histogram equalization using the cv2.equalizeHist function) to make the cell boundaries in the image more distinct.

[0093] Step 4: Inverse binary the image. In the fourth step, we threshold each image to inverse binary (black to white, white to black) using cv2.threshold threshold function with threshold set to 40 pixels and cv2.THRESH_BINARY_INV option.

[0094] Step 5: Adding contours to the white parts of the image. In the fifth step, contours or convex hulls are added to the white parts of the image using the cv2.findContours function with the number of contour points set to at least 10 pixels and the cell area set between 50 and 3000 pixels. The convex hull was constructed using the cv2.convexHull function.

[0095] Step 6: Focus on the white parts of the image. In the sixth step, we remove the areas with grayscale greater than 0 and focus only on the image that corresponds to the white parts of the image from the previous step.

[0096] Step 7: Make Image Binary In the seventh step, the previous image is made binary by thresholding it at 0.8 pixels (the 99th percentile of the pixels on the BF image).

[0097] Step 8: Add contours and filter by area. In the eighth step, contours are added to the previous image using the cv2.findContours function and filtered by area set between 50 and 3000 pixels.

[0098] Step 9: Morphological analysis using 3D contours In the ninth step, an analysis of the images showing the 3D surface with contours is performed.

[0099] In this example, the cell cluster is shown in the final 3D image as having two separate hills, each surrounded by a valley. Since the number of cells in a cell cluster is based on the number of contours detected, it is determined that there are two cells in the cell cluster. And since cell morphology is based on the circularity of the contours, the cell cluster is determined to have poor morphology. Since the cell cluster is determined to be not a single cell cluster, but rather a multi-cell cluster, the image is deemed to be bad for containing defects and is excluded. If the cell cluster is determined to contain only one cell and is in fact a single cell cluster with good morphology, the image is deemed to be good. Based on that good image, it can be concluded that there is only one cell in that respective well on day 0.

[0100] Figure 16 shows a successful example of the cell contour method. Using the image processing techniques and analysis described, a single contour with sufficient circularity is identified in each image, as indicated by ring 1602. Thus, the cell clusters are correctly determined to be single cell clusters whose cells have good morphology. The method works well when the images are of good quality, sufficiently bright, and in focus. Morphology Review: Deep Learning Models

[0101] As an alternative to the image processing techniques described in the previous section, deep learning models can be used to predict cell morphology. Deep learning models are advantageous because they can provide consistent results for a larger number of images compared to manual human review. They can better handle issues that may arise with images, including image quality issues such as darkness and blurriness. They can also better analyze images that have inconsistencies in cell patterns due to plate centrifugation and challenges associated with cells being close to well edges. Figure 17 shows examples of images with these issues and challenges.

[0102] Images deemed good by the cluster validation deep learning model as containing only one cell cluster are followed by morphology review. For only one region of the query image depicting a cell cluster, the CLD image characterization method further includes processing the region through a morphology deep learning model to determine 1) whether the cell cluster has exactly one or alternatively two or more cells in the cluster, and simultaneously 2) if there is only one cell in only one cell cluster, whether the morphology of the cell is acceptable (i.e., the cell has good morphology).

[0103] In a particular embodiment, the output of the morphological deep learning model is to classify a query image into one of three categories with respective confidences. The categories can be too dark for analysis, good image, and bad image. The model is trained by feeding images with one of these three labels. The morphological deep learning algorithm calculates the probabilities of these three categories for an input image and selects the category with the highest probability.

[0104] As previously mentioned, the pre-check of day 0 images involves determining whether an image is too dark for further analysis. Using a low pass filter during this initial pre-check stage of analysis, images that are very dark and below a pre-determined darkness threshold are already identified and filtered out at some point prior to analysis by the morphological deep learning model. Thus, the morphological deep learning model provides another layer of image darkness analysis. Dark images that pass the pre-check stage but are still dark and have additional issues such as blurring may not be suitable for further analysis. Both dark and out-of-focus images are identified and filtered out for containing defects, as the algorithm may not be able to correctly identify the presence of cell clusters within them or any cells within cell clusters. Too dark for analysis can also be used as a categorical label for training as one of the outputs of the deep learner to help ensure that only images that meet basic brightness criteria continue in the workflow.

[0105] An image is classified as a good image by the morphological deep learning model if there is both 1) a high probability that there is only a single cell identified in one and only one cell cluster, and 2) that single cell is likely to have good morphology. An image is classified as a bad image if not even a single cell is identified, and instead the cluster is likely not actually a cell cluster but is e.g. debris. An image is also classified as a bad image by the morphological deep learning model if multiple cells are identified in one and only one cell cluster, or if a single cell is identified in a cell cluster, but the single cell is likely to have bad morphology.

[0106] The results of the day 0 image analysis using the cluster validation and morphological deep learners are then stored and / or transmitted by or between computers for further analysis via manual human review. Manual human review can be performed using visual inspection and associated imager software, so that a manual reviewer can make the final decision on which clones should move forward in the single cell cloning workflow. Selected clones are then robotically picked and seeded into 96-well plates or other sized plates for culturing and further scaling up the clones.

[0107] In an alternative embodiment, the output of the morphological deep learning model is a classification of the query image into one of five categories with respective confidence levels. The categories can be too dark for analysis, good image, good image with concerns, bad image, and bad image with merit. In such an embodiment, the model is trained by feeding images with one of these five labels. The morphological deep learning algorithm calculates the probability of these five categories for the input image and selects the category with the highest probability. Images classified as too dark for analysis, bad images, and bad images with merit are filtered out and are not continued for further analysis. Compared to the three-category embodiment described above, the inclusion of these additional intermediate layers gives the researcher a choice during the hit-picking step. If there are not enough images classified as good images by the morphological deep learner to proceed to the next stage of the single-cell cloning workflow (i.e., combining with the analysis obtained from the confluence imaging to identify clones to culture), the researcher can manually review the images classified as good images with concerns and identify those that are in fact good images.

[0108] It is understood that the three or five categories described are merely exemplary and that the use of other labels and numbers of labels in categorizing images is within the scope of this disclosure. In the following example dataset, the labels and associated output categories are more descriptive with respect to cell detection and cell morphology. Example Data Set

[0109] In one exemplary image dataset, 309 plates with 384 wells each yielded a total of 118,656 wells available. Day 0 BF and FL images of various conditions were included. Approximately 2% of the images, or over 4000 images, were manually labeled as belonging to one of six categories: good cells, good cells but not certain, bad cells or >1 cell, bad cells but not certain, nothing (or no cells), and debris. These labels were provided to the algorithm to build a preliminary deep learning model, which was then used to label the remaining 98% of the images.

[0110] FIG. 18 illustrates an exemplary deep learning model based on the VGG16 architecture that can be used for a morphological deep learning model, according to one embodiment. In this example, there are 13 convolutional layers and 3 fully connected layers, but the model used may be constructed to have any number of layers. The input to the cov1 layer is a fixed-size 224×224 RGB image. The image passes through a stack of convolutional (conv.) layers, and filters are used with a very small receptive field of: 3×3 (this is the minimum size to capture the concepts of left / right, top / bottom, center). One configuration also utilizes a 1×1 convolution filter, which can be viewed as a linear transformation (followed by nonlinearity) of the input channels. The convolution stride is fixed at 1 pixel, and the spatial padding of the conv. layer input is such that the spatial resolution is preserved after convolution, i.e., the padding is 1 pixel for a 3×3 conv. layer. Spatial pooling is performed by five max pooling layers following some of the convex layers (not all convex layers are followed by max pooling). Max pooling is performed over a 2x2 pixel window with stride 2. Three fully connected (FC) layers follow the stack of convolutional layers, the first two with 4096 channels each, and the third performs 1000-class classification and therefore contains 1000 channels (one for each class). The final layer is a softmax layer. The configuration of the fully connected layers is the same in all networks. All hidden layers are equipped with rectified (ReLU) nonlinearities. It is understood that other known CNN architectures may be retuned and used for the same purpose and are within the scope of this disclosure.

[0111] In one exemplary embodiment of the exemplary deep learning model introduced above, 10,000 candidates were labeled with six levels: good cells, good cells but not certain, bad cells or >1 cells, bad cells but not certain, nothing (or no cells), and debris. If the morphology was deemed not certain by the model, a manual review of those images was performed. In addition to the manual labeling, the image processing analysis results described for single cluster detection (i.e., finding the cluster outline and blob detection) and cell morphology were also used to perform the initial labeling. Both BF and FL images were read in grayscale, resized to 64*64*1 using the cv2.resize function, and merged into one image with a size of 64*64*2 using the cv2.merge function, and used as the input image for the deep learning model. 10% of the data was randomly selected as the validation set, and the remaining 90% was selected as the training set.

[0112] Transfer learning and image augmentation are used to enhance model performance. As mentioned above, transfer learning refers to training a base or auxiliary CNN on an auxiliary dataset and task, thereby developing certain parameters, and then utilizing a target dataset and task to further refine those parameters. Transfer learning is performed on the same model architecture with ImageNet or other custom datasets. This procedure has been demonstrated to enable training large CNNs on a target dataset without overfitting by using significantly larger auxiliary datasets. Furthermore, previous studies have reported that transfer learning can significantly improve model generalization ability (compared to random initialization) even when the auxiliary and target datasets are very different.

[0113] For morphological review in the CLD characterization method disclosed herein, a plurality of second target training samples are accessed, the second target training samples include a plurality of second target training images associated with the sampling area, a plurality of cell instructions, and a plurality of cell morphological instructions associated with the plurality of second target training images, respectively. A plurality of second auxiliary training samples are selected based on the second target training samples, the second auxiliary training samples include a plurality of second auxiliary training images and a plurality of second content categories associated with the plurality of second auxiliary training images, respectively. Then, the morphological deep learning model is trained by training a second auxiliary machine learning model based on the plurality of second auxiliary training samples, and training the morphological deep learning model based on the trained second auxiliary machine learning model, the plurality of cell instructions, and the plurality of cell morphological instructions. The plurality of second content categories associated with the plurality of second auxiliary training images are inconsistent with the plurality of cell instructions and the plurality of cell morphological instructions associated with the plurality of second target training images.

[0114] FIG. 19 illustrates a flowchart of an exemplary method by a computing system 1900 to perform single cluster detection, single cluster validation, and morphological review to ultimately identify a sampling region containing a single cell with acceptable morphology. The method 1900 includes receiving 1902 a query image depicting the sampling region. For single cluster detection, the query image is processed 1904 using a single cluster detection model to identify 1906 one or more regions of the query image depicting a cluster within the sampling region. For single cluster validation, the method 1900 includes processing 1908 one or more regions using a cluster validation deep learning model to determine whether each depicted cluster is a cell cluster, and determining 1910 that exactly one of the identified one or more regions depicts a cluster that is a cell cluster. For morphological review, the morphological deep learning model is used to process 1912 the regions depicting the cell clusters, determining 1914 that there is only one cell within the cell cluster, and determining 1916 that the morphology of the cell is acceptable. A query image that depicts a sampling region having a single cell with acceptable morphology is considered a good image 1918.

[0115] FIG. 19 also illustrates how 1900 an image can be excluded as a bad image for containing defects. For single cluster detection, if no cluster is identified by the single cluster detection model, the image is deemed a bad image 1920. For single cluster validation, if the cluster validation deep learning model determines that none of the one or more identified regions depict a cluster that is a cell cluster, or if two or more of the one or more identified regions depict a cluster that is a cell cluster, the image is deemed a bad image 1922. For morphology review, if the morphology deep learning model determines that there are no cells or multiple cells in only one cell cluster, the image is deemed a bad image 1924. If only one cell is determined to be present in the cell cluster, but the cell morphology is determined to be unacceptable, the image is deemed a bad image 1926.

[0116] 20A through 22 show what the CNN sees when the image characterization tool is running.

[0117] Visualization of intermediate activations. Figures 20A-C show what the original input image looks like after passing through each layer of filters. Some filters are very sensitive to the patterns of cells, especially the ring-shaped valleys of the cell membrane. In the following examples, the figures were constructed using the models.Model function in the keras package. On the left side of each figure is the original raw input image, and on the right side is the image after passing through the convolutional filters, the images were manually selected. The image on the right triggers a morphological deep learning algorithm to determine whether the image is good or bad, as the output of the filter for that image. In one example, the bright central part of a cell is provided as the interest point, so when the morphological deep learning algorithm evaluates the image, it identifies the central part and evaluates it for circularity. This may be done for one layer, while other layers may look at different aspects of the image. Figure 20A shows an example visualization of the filter pattern after the first convolutional layer. Figure 20B shows an example visualization of the filter pattern after the second convolutional layer. FIG. 20C shows an example visualization of the filter patterns after the fourth convolutional layer.

[0118] Saliency visualization. A saliency map shows which parts of an image are important to the prediction of a morphological deep learning model. To construct the saliency map in the following example, the visualize_saliency function from the vis.visualization package was used with the "guided" option. Figure 21 shows an example saliency map, where the most important parts of the image are the cell center, especially the region close to the cell membrane, and the ring-shaped region outside the cell membrane. More specifically, on the left is the original raw input image (A) showing a cell with good morphology. On the right is the input image after processing by the saliency function, represented as an example of the output saliency map (B) of the input image (A). In this example, the saliency map (B) shows a cell with a bright center and a bright ring outside the membrane. Thus, the cell is likely to be predicted by the morphological deep learning model as having good morphology.

[0119] Activation Maximization. Activation Maximization constructs or generates an input image that maximizes the filter output activation. In the following example, the visual_activation function from the vis.visualization package was used to construct an activation maximized image. Figure 22 shows an example activation maximized image where the activation function maximizes the activation of the cell's output node. The activation maximized image on the left is of a cell with good morphology (A). By maximizing a particular neuron, some pattern of a good cell, such as a bright center and a dark ring-like region around it, can be detected in the center-right part of the image. The activation maximized image on the right is of a cell with poor morphology (B). Cell detection and cell morphology: CNN prediction performance

[0120] In an exemplary embodiment of the morphological deep learning model described above, about 12,000 boxed FL and BF images were manually labeled by six levels, and another about 8,000 boxed images randomly selected from the well images were labeled as "none." The overall correct classification rate of the morphological model on about 2,000 validation data is 87%, 99.8% for detecting cells (cells vs. non-cells), and 89% for four levels of cell morphology. [Table 2]

[0121] Although the correct classification rate is high for cell morphology, prediction errors may still occur based on specific challenges. Figure 23A shows an example of a prediction error based on cells at the well edge. Figure 23B shows an example of a prediction error based on a dark image. By continuing to train the morphology deep learning model with additional data to a more mature stage, it is expected that the prediction performance will improve.

[0122] As previously mentioned, proper analysis of high-resolution day 0 plate images is critical in cell line development. Implementation of the CLD image characterization method described herein utilizing deep learning models results in a more efficient workflow that allows more qualified clones to progress. It increases the consistency of single-cell image analysis, reduces the time and effort of the clone selection and manual review process, and gives extremely high assurance that cell lines derived from this workflow are clonally derived. Systems and methods

[0123] FIG. 24 illustrates an exemplary computer system 2400. In certain embodiments, one or more computer systems 2400 perform one or more steps of one or more methods described or illustrated herein. In certain embodiments, one or more computer systems 2400 provide functionality described or illustrated herein. In certain embodiments, software executing on one or more computer systems 2400 performs one or more steps of one or more methods described or illustrated herein or provides functionality described or illustrated herein. Certain embodiments include one or more portions of one or more computer systems 2400. As used herein, references to a computer system can encompass computing devices, and vice versa, where appropriate. Additionally, references to a computer system can encompass one or more computer systems, where appropriate.

[0124] The present disclosure contemplates any suitable number of computer systems 2400. The present disclosure contemplates computer system 2400 taking any suitable physical form. By way of example and not limitation, computer system 2400 may be an embedded computer system, a system on a chip (SOC), a single board computer system (SBC) (e.g., a computer on module (COM) or a system on module (SOM)), a desktop computer system, a laptop or notebook computer system, an interactive kiosk, a mainframe, a mesh of computer systems, a mobile phone, a personal digital assistant (PDA), a server, a tablet computer system, or a combination of two or more of these. Where appropriate, computer system 2400 may include one or more computer systems 2400, may be unitary or distributed, may span multiple locations, span multiple machines, span multiple data centers, reside in a cloud that may include one or more cloud components in one or more networks. Where appropriate, one or more computer systems 2400 may perform one or more steps of one or more methods described or illustrated herein without substantial spatial or temporal limitations. By way of example, and not limitation, one or more computer systems 2400 may perform one or more steps of one or more methods described or illustrated herein in real time or batch mode. One or more computer systems 2400 may, where appropriate, perform one or more steps of one or more methods described or illustrated herein at different times or in different locations.

[0125] In a particular embodiment, computer system 2400 includes a processor 2402, a memory 2404, a storage device 2406, an input / output (I / O) interface 2408, a communications interface 2410, and a bus 2412. Although this disclosure describes and illustrates a particular computer system having a particular number of particular components in a particular configuration, this disclosure contemplates any suitable computer system having any suitable number of any suitable components in any suitable configuration.

[0126] In particular embodiments, the processor 2402 includes hardware for executing instructions, such as those that make up a computer program. By way of example and not limitation, to execute an instruction, the processor 2402 may retrieve (or fetch) the instruction from an internal register, an internal cache, memory 2404, or storage device 2406, decode and execute it, and then write one or more results to an internal register, an internal cache, memory 2404, or storage device 2406. In particular embodiments, the processor 2402 may include one or more internal caches for data, instructions, or addresses. This disclosure contemplates the processor 2402 including any suitable number of any suitable internal caches, where appropriate. By way of example and not limitation, the processor 2402 may include one or more instruction caches, one or more data caches, and one or more translation lookaside buffers (TLBs). Instructions in an instruction cache may be copies of instructions in memory 2404 or storage device 2406, and the instruction cache may speed up retrieval of those instructions by the processor 2402. The data in the data cache may be a copy of data in memory 2404 or storage 2406 for instructions that operate on data, or the results of a previous instruction executed in processor 2402 for access by a subsequent instruction executed in processor 2402, or for writing to memory 2404 or storage 2406, or other suitable data. The data cache may speed up read or write operations by processor 2402. The TLB may speed up virtual address translation of processor 2402. In certain embodiments, processor 2402 may include one or more internal registers for data, instructions, or addresses. This disclosure contemplates processor 2402 including any suitable number of any suitable internal registers, where appropriate.Where appropriate, the processor 2402 may include one or more arithmetic logic units (ALUs), may be a multi-core processor, or may include one or more processors 2402. Although this disclosure describes and illustrates a particular processor, this disclosure contemplates any suitable processor.

[0127] In a particular embodiment, the memory 2404 includes a main memory for storing instructions for the processor 2402 to execute or data for the processor 2402 to operate on. By way of example and not limitation, the computer system 2400 may load instructions from another source (such as, for example, another computer system 2400) into the storage device 2406 or memory 2404. The processor 2402 may then load the instructions from the memory 2404 into an internal register or cache. To execute instructions, the processor 2402 may retrieve instructions from the internal register or cache and decode them. During or after the execution of instructions, the processor 2402 may write one or more results (which may be intermediate or final results) to an internal register or cache. The processor 2402 may then write one or more of those results to the memory 2404. In particular embodiments, the processor 2402 executes only instructions in one or more internal registers or caches or memory 2404 (as opposed to storage 2406 or elsewhere) and operates only on data in one or more internal registers or caches or memory 2404 (as opposed to storage 2406 or elsewhere). One or more memory buses (each of which may include an address bus and a data bus) may couple the processor 2402 to the memory 2404. The bus 2412 may include one or more memory buses, as described below. In particular embodiments, one or more memory management units (MMUs) reside between the processor 2402 and the memory 2404 to facilitate accesses to the memory 2404 requested by the processor 2402. In particular embodiments, the memory 2404 includes random access memory (RAM). This RAM may be volatile memory, as appropriate. Where appropriate, this RAM may be dynamic RAM (DRAM) or static RAM (SRAM). Further, where appropriate, this RAM may be single-ported or multi-ported RAM. This disclosure contemplates any suitable RAM.The memory 2404 may, where appropriate, comprise one or more memories 2404. Although this disclosure describes and illustrates a particular memory, this disclosure contemplates any suitable memory.

[0128] In particular embodiments, the storage device 2406 includes mass storage for data or instructions. By way of example and not limitation, the storage device 2406 may include a hard disk drive (HDD), a floppy disk drive, a flash memory, an optical disk, a magneto-optical disk, a magnetic tape, or a universal serial bus (USB) drive, or a combination of two or more of these. The storage device 2406 may include removable or non-removable (or fixed) media, where appropriate. The storage device 2406 may be internal or external to the computer system 2400, where appropriate. In particular embodiments, the storage device 2406 is a non-volatile solid-state memory. In particular embodiments, the storage device 2406 includes read-only memory (ROM). Where appropriate, the ROM may be a mask programmable ROM, a programmable ROM (PROM), an erasable PROM (EPROM), an electrically erasable PROM (EEPROM), an electrically alterable ROM (EAROM), or a flash memory, or a combination of two or more of these. The present disclosure contemplates the mass storage device 2406 taking any suitable physical form. The storage 2406 may include, where appropriate, one or more storage control units that facilitate communications between the processor 2402 and the storage 2406. Where appropriate, the storage 2406 may include one or more storage devices 2406. Although this disclosure describes and illustrates a particular storage device, this disclosure contemplates any suitable storage device.

[0129] In particular embodiments, I / O interface 2408 includes hardware, software, or both that provide one or more interfaces for communication between computer system 2400 and one or more I / O devices. Computer system 2400 may include one or more of these I / O devices, where appropriate. One or more of these I / O devices may enable communication between a person and computer system 2400. By way of example and not limitation, an I / O device may include a keyboard, keypad, microphone, monitor, mouse, printer, scanner, speaker, still camera, stylus, tablet, touch screen, trackball, video camera, another suitable I / O device, or a combination of two or more of these. An I / O device may include one or more sensors. This disclosure contemplates any suitable I / O devices and any suitable I / O interface 2408 therefor. Where appropriate, I / O interface 2408 may include one or more device drivers that enable processor 2402 to drive one or more of these I / O devices. I / O interface 2408 may, where appropriate, include one or more I / O interfaces 2408. Although this disclosure describes and illustrates a particular I / O interface, this disclosure contemplates any suitable I / O interface.

[0130] In certain embodiments, the communication interface 2410 includes hardware, software, or both that provide one or more interfaces for communication (e.g., packet-based communication, etc.) between the computer system 2400 and one or more other computer systems 2400 or one or more networks. By way of example and not limitation, the communication interface 2410 may include a network interface controller (NIC) or network adapter for communicating with an Ethernet or other wired-based network, or a wireless NIC (WNIC) or wireless adapter for communicating with a wireless network, such as a WI-FI network. This disclosure contemplates any suitable network and any suitable communication interface 2410 therefor. By way of example and not limitation, the computer system 2400 may communicate with one or more portions of an ad-hoc network, a personal area network (PAN), a local area network (LAN), a wide area network (WAN), a metropolitan area network (MAN), or the Internet, or a combination of multiple of these. One or more portions of one or more of these networks may be wired or wireless. By way of example, computer system 2400 may communicate with a wireless PAN (WPAN) (e.g., a BLUETOOTH WPAN, etc.), a WI-FI network, a WI-MAX network, a cellular network (e.g., a Global System for Mobile Communications (GSM) network), or any other suitable wireless network, or a combination of two or more of these. Computer system 2400 may include any suitable communications interface 2410 for any of these networks, where appropriate. Communications interface 2410 may include one or more communications interfaces 2410, where appropriate. Although this disclosure describes and illustrates a particular communications interface, this disclosure contemplates any suitable communications interface.

[0131] In particular embodiments, bus 2412 includes hardware, software, or both that couple components of computer system 2400 together. By way of example, and not limitation, bus 2412 may include an Accelerated Graphics Port (AGP) or other graphics bus, an Enhanced Industry Standard Architecture (EISA) bus, a Front Side Bus (FSB), a HyperTransport (HT) Interconnect, an Industry Standard Architecture (ISA) bus, an InfiniBand interconnect, a Low Pin Count (LPC) bus, a memory bus, a MicroChannel Architecture (MCA) bus, a Peripheral Component Interconnect (PCI) bus, a PCI Express (PCIe) bus, a Serial Advanced Technology Attachment (SATA) bus, a Video Electronics Standards Association Local (VLB) bus, or another suitable bus, or a combination of two or more of these. Bus 2412 may include one or more buses 2412, where appropriate. Although this disclosure describes and illustrates a particular bus, this disclosure contemplates any suitable bus or interconnect.

[0132] As used herein, a computer-readable non-transitory storage medium may include one or more semiconductor-based or other integrated circuits (ICs) (such as, for example, field programmable gate arrays (FPGAs) or application specific ICs (ASICs)), hard disk drives (HDDs), hybrid hard drives (HHDs), optical disks, optical disk drives (ODDs), magneto-optical disks, magneto-optical drives, floppy disks, floppy disk drives (FDDs), magnetic tapes, solid state drives (SSDs), RAM drives, secure digital cards or drives, any other suitable computer-readable non-transitory storage medium, or any suitable combination of two or more of these, where appropriate. A computer-readable non-transitory storage medium may be volatile, non-volatile, or a combination of volatile and non-volatile, where appropriate.

[0133] As used herein, "or" is inclusive and not exclusive, unless expressly indicated otherwise or indicated otherwise by context. Thus, as used herein, "A or B" means "A, B, or both," unless expressly indicated otherwise or indicated otherwise by context. Furthermore, "and" is both conjunctive and several, unless expressly indicated otherwise or indicated otherwise by context. Thus, as used herein, "A and B" means "A and B, together or separately," unless expressly indicated otherwise or indicated otherwise by context.

[0134] The scope of the present disclosure encompasses all modifications, substitutions, variations, alternatives, and alterations to the exemplary embodiments described or illustrated herein that would be understood by a person skilled in the art. The scope of the present disclosure is not limited to the exemplary embodiments described or illustrated herein. Furthermore, although the present disclosure describes and illustrates each embodiment herein as including certain components, elements, features, functions, operations, or steps, any of these embodiments may include any combination or permutation of any of the components, elements, features, functions, operations, or steps described or illustrated anywhere herein that would be understood by a person skilled in the art. Furthermore, a reference in the appended claims to an apparatus or system or a component of a system that is adapted, arranged, enabled, configured, enabled, enabled, enabled, or operates to perform a particular function encompasses that apparatus, system, or component, whether or not it or that particular function is activated, turned on, or unlocked, so long as the apparatus, system, or component is so adapted, arranged, enabled, configured, enabled, enabled, enabled, or operates. Moreover, although the present disclosure describes or illustrates particular embodiments as providing certain advantages, a particular embodiment may not provide all, some, or all of these advantages. In certain embodiments, for example, the following items are provided: (Item 1) The computing system Receiving a query image depicting a sampling region; processing the query image using a single cluster detection model to identify one or more regions of the query image that delineate clusters within the sampling region; processing the one or more regions using a cluster validation deep learning model to determine whether each delineated cluster is a cell cluster; determining that exactly one of the identified one or more regions describes a cluster that is a cell cluster; determining that only one cell is present within the cell cluster; and and processing the regions depicting the cell clusters using a morphological deep learning model to determine that the morphology of the cells is acceptable. (Item 2) 2. The method of claim 1, wherein each delineated cluster includes a cluster contour area having a cluster contour area size within a predetermined range and at least a predetermined number of points. (Item 3) 3. The method of claim 2, wherein identifying the region comprises drawing a bounding box around the cluster contour region. (Item 4) 4. The method of any one of claims 1 to 3, wherein determining whether each delineated cluster is a cell cluster comprises verifying each delineated cluster in a corresponding location of another query image depicting the sampling area. (Item 5) 5. The method of any one of items 1 to 4, wherein determining whether each delineated cluster is a cell cluster comprises determining whether each delineated cluster appears as a positive cluster for having cells. (Item 6) 6. The method of any one of items 1 to 5, wherein determining whether each delineated cluster is a cell cluster comprises determining whether each delineated cluster does not appear as a negative cluster for having cells. (Item 7) 7. The method of any one of items 1 to 6, wherein determining that only one cell is present within the cell cluster comprises assessing a number of contours detected within the cell cluster. (Item 8) 8. The method of any one of items 1 to 7, wherein determining that the morphology of the cell is acceptable comprises identifying a cell central region and evaluating the cell central region for circularity. (Item 9) 9. The method of any one of items 1 to 8, wherein determining that the morphology of the cells is acceptable comprises identifying a bright cell central region and a dark ring-like cell membrane around the bright cell central region. (Item 10) 10. The method of any one of items 1 to 9, wherein determining that the morphology of the cell is acceptable comprises identifying a bright central region of the cell and a bright ring-shaped region outside the cell membrane. (Item 11) 11. The method of any one of claims 1 to 10, wherein the query image comprises a fluorescent image or a bright field image. (Item 12) 12. The method of any one of claims 1 to 11, further comprising using a contour detection algorithm to identify one or more large distinct objects in the query image during a pre-check stage of analysis prior to processing the query image. (Item 13) 13. The method of any one of claims 1 to 12, further comprising determining whether the query image is below a predefined darkness threshold during a pre-check phase of analysis prior to processing the query image. (Item 14) Item 14. The method of any one of items 1 to 13, further comprising determining whether critical clipping of the sampling region is present in the query image before processing the query image. (Item 15) 15. The method of claim 14, wherein the sampling region is a well, the well having a well contour in the query image defined by a plurality of well contour points and a well contour center. (Item 16) determining whether critical clipping of the sampling region exists, 16. The method of claim 15, comprising applying a fitting circle having a fitting circle center and a fitting circle radius to the well contour in the query image. (Item 17) determining whether critical clipping of the sampling region exists, 17. The method of claim 16, further comprising calculating a fitting circle radius minus a distance of the fitting circle center to each of a plurality of well contour points to identify a difference between the well contour and the fitting circle. (Item 18) determining whether critical clipping of the sampling region exists, 16. The method of claim 15, comprising calculating the distance from the well contour center to an image edge minus a well contour radius and determining whether the result is less than or equal to a predetermined distance threshold. (Item 19) determining whether critical clipping of the sampling region exists, 16. The method of claim 15, comprising determining whether consecutive well contour points lie in a single line and whether a distance between two end points of said single line meets or exceeds a predetermined distance threshold. (Item 20) determining whether critical clipping of the sampling region exists, 20. The method of claim 19, further comprising determining whether the single line has a local curvature of zero. (Item 21) 1. A system comprising: one or more processors; a non-transitory memory coupled to the processor, the non-transitory memory containing instructions executable by the processor, the instructions, when executed by the processor, Receiving a query image depicting a sampling region; processing the query image using a single cluster detection model to identify one or more regions of the query image that delineate clusters within the sampling region; processing the one or more regions using a cluster validation deep learning model to determine whether each delineated cluster is a cell cluster; determining that exactly one of the identified one or more regions describes a cluster that is a cell cluster; determining that only one cell is present within the cell cluster; and and processing the regions depicting the cell clusters using a morphological deep learning model to determine that the morphology of the cells is acceptable. (Item 22) 22. The system of claim 21, wherein each delineated cluster includes a cluster contour area having a cluster contour area size within a predetermined range and at least a predetermined number of points. (Item 23) 23. The system of claim 22, wherein identifying the region includes drawing a bounding box around the cluster contour region. (Item 24) 24. The system of any one of claims 21 to 23, wherein determining whether each delineated cluster is a cell cluster comprises verifying each delineated cluster in a corresponding location of another query image depicting the sampling area. (Item 25) 25. The system of any one of items 21 to 24, wherein determining whether each delineated cluster is a cell cluster comprises determining whether each delineated cluster appears as a positive cluster for having cells. (Item 26) 26. The system of any one of items 21 to 25, wherein determining whether each delineated cluster is a cell cluster comprises determining whether each delineated cluster does not appear as a negative cluster for having cells. (Item 27) 27. The system of any one of items 21 to 26, wherein determining that only one cell is present within the cell cluster comprises assessing a number of contours detected within the cell cluster. (Item 28) 28. The system of any one of items 21 to 27, wherein determining that the morphology of the cell is acceptable comprises identifying a cell central region and evaluating the cell central region for circularity. (Item 29) 29. The system of any one of claims 21 to 28, wherein determining that the morphology of the cell is acceptable comprises identifying a bright cell central region and a dark ring-like cell membrane around the bright cell central region. (Item 30) 30. The system of any one of items 21 to 29, wherein determining that the morphology of the cell is acceptable comprises identifying a bright region at the center of the cell and a bright ring-shaped region outside the cell membrane. (Item 31) 31. The system of any one of claims 21 to 30, wherein the query image comprises a fluorescent image or a bright field image. (Item 32) When the processor executes the instructions, 32. The system of any one of claims 21 to 31, further operable to identify one or more large distinct objects in the query image using a contour detection algorithm during a pre-check stage of analysis prior to processing the query image. (Item 33) When the processor executes the instructions, 33. The system of any one of claims 21 to 32, further operable to determine whether the query image is below a predetermined darkness threshold during a pre-check phase of analysis prior to processing the query image. (Item 34) When the processor executes the instructions, Item 34. The system of any one of items 21 to 33, further operable to determine whether critical clipping of the sampling region is present in the query image before processing the query image. (Item 35) 35. The system of claim 34, wherein the sampling region is a well, the well having a well contour in the query image defined by a plurality of well contour points and a well contour center. (Item 36) When the processor executes the instructions, Item 36. The system of item 35, operable to determine whether critical clipping of the sampling region exists by applying a fitting circle having a fitting circle center and a fitting circle radius to the well contour in the query image. (Item 37) When the processor executes the instructions, Item 37. The system of item 36, further operable to determine whether critical clipping of the sampling area exists by calculating a fitting circle radius minus a distance of the fitting circle center to each of a plurality of well contour points to identify a difference between the well contour and the fitting circle. (Item 38) When the processor executes the instructions, Item 36. The system of item 35, operable to determine whether critical clipping of the sampling area exists by calculating the distance from the well contour center to an image edge minus a well contour radius and determining whether the result is less than or equal to a predetermined distance threshold. (Item 39) When the processor executes the instructions, Item 36. The system of item 35, operable to determine whether critical clipping of the sampling region exists by determining whether consecutive well contour points lie in a single line and whether a distance between two end points of the single line meets or exceeds a predetermined distance threshold. (Item 40) When the processor executes the instructions, 40. The system of claim 39, further operable to determine whether critical clipping of the sampling region exists by determining whether the single line has a local curvature of zero. (Item 41) When executed, Receiving a query image depicting a sampling region; processing the query image using a single cluster detection model to identify one or more regions of the query image that delineate clusters within the sampling region; processing the one or more regions using a cluster validation deep learning model to determine whether each delineated cluster is a cell cluster; determining that exactly one of the identified one or more regions describes a cluster that is a cell cluster; determining that only one cell is present within the cell cluster; and and processing the regions depicting the cell clusters using a morphological deep learning model to determine that the morphology of the cells is acceptable. (Item 42) Item 42. The medium of item 41, wherein each delineated cluster includes a cluster contour area having a cluster contour area size within a predetermined range and at least a predetermined number of points. (Item 43) Item 43. The media of item 42, wherein identifying the region includes drawing a bounding box around the cluster contour region. (Item 44) 44. The medium of any one of items 41 to 43, wherein determining whether each delineated cluster is a cell cluster comprises verifying each delineated cluster in a corresponding location of another query image depicting the sampling area. (Item 45) 45. The medium of any one of items 41 to 44, wherein determining whether each delineated cluster is a cell cluster comprises determining whether each delineated cluster appears as a positive cluster for having cells. (Item 46) 46. ​​The medium of any one of items 41 to 45, wherein determining whether each delineated cluster is a cell cluster comprises determining whether each delineated cluster does not appear as a negative cluster for having cells. (Item 47) 47. The medium of any one of items 41 to 46, wherein determining that only one cell is present within the cell cluster comprises assessing a number of contours detected within the cell cluster. (Item 48) 48. The medium of any one of items 41 to 47, wherein determining that the morphology of the cells is acceptable comprises identifying a cell central region and evaluating the cell central region for circularity. (Item 49) 49. The medium of any one of items 41 to 48, wherein determining that the morphology of the cells is acceptable comprises identifying a bright cell central region and a dark ring-like cell membrane around the bright cell central region. (Item 50) 50. The medium of any one of items 41 to 49, wherein determining that the morphology of the cells is acceptable comprises identifying a bright central region of the cell and a bright ring-shaped region outside the cell membrane. (Item 51) 51. The medium of any one of claims 41 to 50, wherein the query image comprises a fluorescent image or a bright field image. (Item 52) When the software is executed, 52. The medium of any one of claims 41 to 51, further operable to identify one or more large distinct objects in the query image using a contour detection algorithm during a pre-check stage of analysis prior to processing the query image. (Item 53) When the software is executed, 53. The medium of any one of claims 41 to 52, further operable to determine whether the query image is below a predetermined darkness threshold during a pre-check stage of analysis prior to processing the query image. (Item 54) When the software is executed, 54. The medium of any one of claims 41 to 53, further operable to determine whether critical clipping of the sampling region exists in the query image before processing the query image. (Item 55) 55. The medium of claim 54, wherein the sampling region is a well, the well having a well contour in the query image defined by a plurality of well contour points and a well contour center. (Item 56) When the software is executed, Item 56. The medium of item 55, operable to determine whether critical clipping of the sampling region exists by applying a fitting circle having a fitting circle center and a fitting circle radius to the well contour in the query image. (Item 57) When the software is executed, Item 57. The medium of item 56, further operable to determine whether critical clipping of the sampling area exists by calculating a fitting circle radius minus a distance of the fitting circle center to each of a plurality of well contour points to identify a difference between the well contour and the fitting circle. (Item 58) When the software is executed, Item 56. The medium of item 55, operable to determine whether critical clipping of the sampling area exists by calculating the distance from the well contour center to the image edge minus the well contour radius and determining whether the result is less than or equal to a predetermined distance threshold. (Item 59) When the software is executed, Item 56. The medium of item 55, operable to determine whether critical clipping of the sampling area exists by determining whether consecutive well contour points lie in a single line and whether a distance between two end points of the single line meets or exceeds a predetermined distance threshold. (Item 60) When the software is executed, 60. The medium of claim 59, further operable to determine whether critical clipping of the sampling region exists by determining whether the single line has a local curvature of zero.

Claims

1. The computing system receiving a query image depicting a sampling area, the sampling area being a well; determining whether a critical clipping of the sampling region exists within the query image, the critical clipping being a clipping of equal to or greater than the width of a single cell; processing the query image using a single cluster detection model to identify one or more regions of the query image that delineate clusters within the sampling region; processing the one or more regions using a cluster validation deep learning model to determine whether each delineated cluster is a cell cluster, and that one of the identified one or more regions delineates a cluster that is a cell cluster; Processing the regions depicting the cell clusters using a morphological deep learning model, the morphological deep learning model comprising: whether there is only one cell in the cell cluster; and trained to classify the regions among a plurality of categories depending on whether the morphology of the cells is acceptable.

2. The method of claim 1 , wherein each delineated cluster includes a cluster boundary area having a cluster boundary area size within a predetermined range and at least a predetermined number of points.

3. The method of claim 2 , wherein identifying the regions comprises drawing a bounding box around the cluster contour regions.

4. 4. The method of claim 1, wherein determining whether each delineated cluster is a cell cluster comprises verifying each delineated cluster in a corresponding location of another query image depicting the sampling area.

5. 5. The method of claim 1 , wherein determining whether each delineated cluster is a cell cluster comprises determining whether each delineated cluster appears as a positive cluster for having cells.

6. 6. The method of claim 1 , wherein determining whether each delineated cluster is a cell cluster comprises determining whether each delineated cluster does not appear as a negative cluster for having cells.

7. 7. The method of claim 1, wherein determining that only one cell is present within the cell cluster comprises assessing a number of contours detected within the cell cluster.

8. 8. The method of claim 1, wherein determining that the morphology of the cell is acceptable comprises identifying a cell central region and evaluating the cell central region for circularity.

9. 9. The method of claim 1, wherein determining that the morphology of the cell is acceptable comprises identifying a bright cell central region and a dark ring-like cell membrane around the bright cell central region.

10. 10. The method of claim 1, wherein determining that the cell's morphology is acceptable comprises identifying a bright central region of the cell and a bright ring-shaped region outside the cell membrane.

11. The method of claim 1 , wherein the query image comprises a fluorescent image or a bright field image.

12. 12. The method of claim 1, further comprising using a contour detection algorithm to identify one or more large distinct objects in the query image during a pre-check stage of analysis prior to processing the query image.

13. 13. The method of claim 1, further comprising determining whether the query image is below a predetermined darkness threshold during a pre-check stage of analysis prior to processing the query image.

14. The method of claim 1, wherein the well has a well contour in the query image defined by a plurality of well contour points and a well contour center.

15. determining whether critical clipping of the sampling region exists, The method of claim 14, comprising applying a fitting circle having a fitting circle center and a fitting circle radius to the well contours in the query image.

16. determining whether critical clipping of the sampling region exists, 16. The method of claim 15, further comprising calculating a fitting circle radius minus a distance of the fitting circle center to each of a plurality of well contour points to identify a difference between the well contour and the fitting circle.

17. determining whether critical clipping of the sampling region exists, 15. The method of claim 14, comprising calculating the distance from the well contour center to an image edge minus a well contour radius and determining whether the result is less than or equal to a predetermined distance threshold.

18. determining whether critical clipping of the sampling region exists, 15. The method of claim 14, comprising determining whether consecutive well contour points lie on a single line and whether a distance between two end points of said single line meets or exceeds a predetermined distance threshold.

19. determining whether critical clipping of the sampling region exists, The method of claim 18 , further comprising determining whether the single line has zero local curvature.

20. 1. A system comprising: one or more processors; a non-transitory memory coupled to the processor, the non-transitory memory containing instructions executable by the processor, the instructions, when executed by the processor, receiving a query image depicting a sampling area, the sampling area being a well; determining whether a critical clipping of the sampling region exists within the query image, the critical clipping being a clipping of equal to or greater than the width of a single cell; processing the query image using a single cluster detection model to identify one or more regions of the query image that delineate clusters within the sampling region; processing the one or more regions using a cluster validation deep learning model to determine whether each delineated cluster is a cell cluster, and that one of the identified one or more regions delineates a cluster that is a cell cluster; Processing the regions depicting the cell clusters using a morphological deep learning model, the morphological deep learning model comprising: whether there is only one cell in the cell cluster; and trained to classify the regions among a plurality of categories depending on whether the morphology of the cells is acceptable; The system is operable to:

21. 21. The system of claim 20, wherein each delineated cluster includes a cluster contour area having a cluster contour area size within a predetermined range and at least a predetermined number of points.

22. The system of claim 21 , wherein identifying the region comprises drawing a bounding box around the cluster contour region.

23. 23. The system of claim 20, wherein determining whether each delineated cluster is a cell cluster comprises verifying each delineated cluster in a corresponding location of another query image depicting the sampling area.

24. 24. The system of any one of claims 20 to 23, wherein determining whether each delineated cluster is a cell cluster comprises determining whether each delineated cluster appears as a positive cluster for having cells.

25. 25. The system of any one of claims 20 to 24, wherein determining whether each delineated cluster is a cell cluster comprises determining whether each delineated cluster does not appear as a negative cluster for having cells.

26. 26. The system of claim 20, wherein determining that only one cell is present within the cell cluster comprises assessing a number of contours detected within the cell cluster.

27. 27. The system of claim 20, wherein determining that the morphology of the cell is acceptable comprises identifying a cell central region and evaluating the cell central region for circularity.

28. 28. The system of any one of claims 20 to 27, wherein determining that the cell's morphology is acceptable comprises identifying a bright cell central region and a dark ring-like cell membrane around the bright cell central region.

29. 29. The system of any one of claims 20 to 28, wherein determining that the cell's morphology is acceptable comprises identifying a bright central region of the cell and a bright ring-shaped region outside the cell membrane.

30. 30. The system of claim 20, wherein the query image comprises a fluorescent image or a bright field image.

31. When the processor executes the instructions, 31. The system of claim 20, further operable to use a contour detection algorithm to identify one or more large distinct objects in the query image during a pre-check stage of analysis, prior to processing the query image.

32. When the processor executes the instructions, 32. A system according to any one of claims 20 to 31, further operable to determine whether the query image falls below a predefined darkness threshold during a pre-check stage of analysis, prior to processing the query image.

33. The system of claim 20, wherein the well has a well contour in the query image defined by a plurality of well contour points and a well contour center.

34. When the processor executes the instructions, 34. The system of claim 33, operable to determine whether critical clipping of the sampling region exists by applying a fitting circle having a fitting circle center and a fitting circle radius to the well contour in the query image.

35. When the processor executes the instructions, 35. The system of claim 34, further operable to determine whether critical clipping of the sampling area exists by calculating a fitting circle radius minus a distance of the fitting circle center to each of a plurality of well contour points to identify a difference between the well contour and the fitting circle.

36. When the processor executes the instructions, 34. The system of claim 33, operable to determine whether critical clipping of the sampling area exists by calculating the distance from the well contour center to an image edge minus a well contour radius and determining whether the result is less than or equal to a predetermined distance threshold.

37. When the processor executes the instructions, 34. The system of claim 33, operable to determine whether critical clipping of the sampling region exists by determining whether consecutive well contour points lie in a single line and whether a distance between two end points of the single line meets or exceeds a predetermined distance threshold.

38. When the processor executes the instructions, 38. The system of claim 37, further operable to determine whether there is critical clipping of the sampling region by determining whether the single line has zero local curvature.

39. One or more computer-readable non-transitory storage media embodying software that is operable when executed by a processor of a computing system to cause the computing system to: receiving a query image depicting a sampling area, the sampling area being a well; determining whether a critical clipping of the sampling region exists within the query image, the critical clipping being a clipping of equal to or greater than the width of a single cell; processing the query image using a single cluster detection model to identify one or more regions of the query image that delineate clusters within the sampling region; processing the one or more regions using a cluster validation deep learning model to determine whether each delineated cluster is a cell cluster, and that one of the identified one or more regions delineates a cluster that is a cell cluster; Processing the regions depicting the cell clusters using a morphological deep learning model, the morphological deep learning model comprising: whether there is only one cell in the cell cluster; and trained to classify the regions among a plurality of categories depending on whether the morphology of the cells is acceptable; A medium that allows this to be carried out.

40. 40. The medium of claim 39, wherein each delineated cluster includes a cluster boundary region having a cluster boundary region size within a predetermined range and at least a predetermined number of points.

41. 41. The medium of claim 40, wherein identifying the region comprises drawing a bounding box around the cluster outline region.

42. 42. The medium of any one of claims 39 to 41, wherein determining whether each delineated cluster is a cell cluster comprises verifying each delineated cluster in a corresponding location of another query image depicting the sampling area.

43. 43. The medium of any one of claims 39 to 42, wherein determining whether each delineated cluster is a cell cluster comprises determining whether each delineated cluster appears as a positive cluster for having cells.

44. 44. The medium of any one of claims 39 to 43, wherein determining whether each delineated cluster is a cell cluster comprises determining whether each delineated cluster does not appear as a negative cluster for having cells.

45. 45. The medium of any one of claims 39 to 44, wherein determining that only one cell is present within the cell cluster comprises assessing a number of contours detected within the cell cluster.

46. 46. ​​The medium of any one of claims 39 to 45, wherein determining that the morphology of the cell is acceptable comprises identifying a cell central region and evaluating the cell central region for circularity.

47. 47. The medium of any one of claims 39 to 46, wherein determining that the cell's morphology is acceptable comprises identifying a bright cell central region and a dark ring-like cell membrane around the bright cell central region.

48. 48. The medium of any one of claims 39 to 47, wherein determining that the cell's morphology is acceptable comprises identifying a bright region at the center of the cell and a bright ring-shaped region outside the cell membrane.

49. 49. The medium of any one of claims 39 to 48, wherein the query image comprises a fluorescent image or a bright field image.

50. When the software is executed, 50. The medium of claim 39, further operable to use a contour detection algorithm to identify one or more large distinct objects in the query image during a pre-check stage of analysis prior to processing the query image.

51. When the software is executed, 51. A medium according to any one of claims 39 to 50, further operable to determine whether the query image is below a predetermined darkness threshold during a pre-check stage of analysis prior to processing the query image.

52. The medium described in claim 39, wherein the well has a well contour in the query image defined by a plurality of well contour points and a well contour center.

53. When the software is executed, 53. The medium of claim 52, operable to determine whether critical clipping of the sampling region exists by applying a fitting circle having a fitting circle center and a fitting circle radius to the well contour in the query image.

54. When the software is executed, 54. The medium of claim 53, further operable to determine whether critical clipping of the sampling area exists by calculating a fitting circle radius minus a distance of the fitting circle center to each of a plurality of well contour points to identify a difference between the well contour and the fitting circle.

55. When the software is executed, 53. The medium of claim 52, operable to determine whether critical clipping of the sampling area exists by calculating the distance from the well contour center to an image edge minus a well contour radius and determining whether the result is less than or equal to a predetermined distance threshold.

56. When the software is executed, 53. The medium of claim 52, operable to determine whether critical clipping of the sampling region exists by determining whether consecutive well contour points lie in a single line and whether a distance between two end points of the single line meets or exceeds a predetermined distance threshold.

57. When the software is executed, 57. The medium of claim 56, further operable to determine whether there is critical clipping of the sampling region by determining whether the single line has zero local curvature.

Images