Systems and methods for facilitating clonal selection

Abstract

A method for facilitating clonal selection includes generating a time series of images of a well containing culture medium, including a first image and a subsequent second image. The method also includes analyzing the first image with one or more processors to detect one or more candidate objects shown in the first image, and for each of the candidate objects, determining whether the object is a single cell by analyzing the image of the object using a convolutional neural network. The method further includes analyzing the second image with the processor to detect cell colonies shown in the second image, and determining whether the colonies are formed from only one cell based on whether the processor determines that at least each candidate object is a single cell. The method further includes generating output data by the processor indicating whether the colonies are formed from only one cell.

Description

[Technical Field]

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS Priority is claimed to U.S. Provisional Patent Application No. 62 / 774,154, filed November 30, 2018, the entire contents of which are incorporated herein by reference.

[0002] This application relates generally to clonal selection techniques, and more specifically to techniques for determining whether a cell colony is formed from a single clone / cell. [Background technology]

[0003] In various cell line development processes, clonal selection requires knowledge of whether a cell colony formed from a single clonal cell. For example, individual cells can be seeded into separate wells (e.g., of a 96-well plate) using flow cytometry techniques such as fluorescence-activated cell sorting (FACS), followed by culturing the cells for an appropriate period (e.g., 14 days). Throughout the culture period, digital images of the wells are captured at appropriate time intervals, such as daily or every 7 days. The analyst reviews the well image captured at the end of the culture period (e.g., day 14) to identify cell colonies. If a particular well contains a colony, the analyst can also review one or more previous images of the same well (e.g., images captured on the first day of culture) to determine whether the colony formed solely from a single clone. If the analyst can confirm that the colony formed from a single clone, that particular sample can be transferred to one or more additional stages of the cell line development process. If not, the sample can be discarded.

[0004] This process is very time-consuming and labor-intensive. For example, it is not uncommon for hundreds of man-hours to be required to analyze microscopic images for a single project. Furthermore, when attempting to identify progenitor cells of a colony, it can be difficult for analysts to accurately identify single cells within a well. For example, if the imaged object is out of focus, a false positive can occur (i.e., incorrectly labeling the object as a single cell). As another example, doublets or "stacked" cells can be difficult to distinguish from single cells. False positives (and / or false negatives) can also result from other factors, such as debris, shadows created by the well walls, and / or anomalies on the inner perimeter of the well walls. Summary of the Invention [Means for solving the problem]

[0005] Embodiments described herein relate to systems and methods that improve upon conventional visual inspection techniques used for clonal selection. In particular, in some embodiments, an automated visual inspection system captures a series of digital images for each well in a well plate at intervals over a culture period (e.g., several days or weeks). Depending on the embodiment, images may be captured only on the first and last days of the culture period, daily, or on another suitable time schedule. The system can identify candidate object colonies derived from a single cell. Based on the identified candidate objects, the system can reposition the well for further image capture. Cells from wells identified as containing colonies derived from a single cell can be transferred to a new culture environment and cultured. As used herein, a "well" refers to any laboratory-scale cell culture environment that allows for optical inspection of its contents. While wells on a multi-well plate are discussed as examples herein, it should be understood that whenever "well" and "well plate" are referenced, unless otherwise specified, these terms are intended to encompass any suitable laboratory-scale cell culture environment that allows for optical inspection of its contents.

[0006] For a given well, computer image processing techniques can be applied to subsequent images (e.g., images captured on the final day of the culture period) to determine whether the well contains a cell colony. For example, a convolutional neural network (CNN) can be used to classify objects in the well medium as cell colonies. The CNN can include any number of convolutional layers suitable for two-dimensional convolution (e.g., to detect features such as edges in the image), pooling layers (e.g., downsampling layers to reduce computation while preserving the relative position of features), and fully connected layers. Alternatively, one or more other image processing techniques, such as traditional image filtering, edge detection, and / or pattern detection techniques, can be used to detect colonies. If a cell colony is detected in a well, one or more previous images of the well (e.g., images from the first day of culture) can be analyzed to determine whether the colony formed from a single clone / cell. This determination can involve multiple steps. First, an image of the entire well is analyzed to identify any objects in the well that are candidates for being single cells (e.g., objects that are neither too large nor too small to be single cells). The image of each candidate object is then input to a CNN (e.g., a second CNN if the first CNN was used to detect cell colonies) that classifies the candidate object according to object type. In some embodiments, the image of each candidate object is extracted from the image of the whole well and includes, or simply consists of, a set of pixels that represent that candidate object. In some embodiments, the image of each candidate object further includes a zoomed-in image of the candidate object, e.g., to overcome image resolution limitations associated with currently available imaging technology. For example, a first, lower magnification imaging unit can be used to capture an image of the whole well for the purpose of identifying the candidate objects, while a second, higher magnification imaging unit can be used to capture zoomed-in images of each of the identified candidate objects.Use of the second imaging unit may further include positioning the wells or portions of the wells in the optical path of the second imaging unit, for example, by shifting the well plate a small distance in the x and / or y directions.

[0007] For a given candidate object, the CNN may output a binary classification (e.g., "single cell" vs. "not a single cell") or may classify the object according to one of three or more types (e.g., "single cell," "doublet," "empty cell," "debris," etc.). If the candidate object is classified as a single cell (or another class corresponding to a single living cell), the cell colony may be determined to be derived from that cell. This determination may also depend on one or more other factors, such as the location of the single cell within the well compared to the location of the detected cell colony within the well. Furthermore, in some embodiments, one or more additional images of the well corresponding to one or more other times during the culture period may be analyzed to confirm that the cell colony formed from a single cell. For example, the additional images may be processed to detect one or more intermediate growth stages of the cell colony.

[0008] If it cannot be determined that the cell colony formed from a single cell, the sample in that well may be discarded. However, if the origin of a single cell can be determined (e.g., with at least some threshold level of accuracy, such as 90% or 99%), the sample may be transferred to one or more additional stages of the cell line development process. For example, the cells of the sample (e.g., as single cells and / or as part of a cell-containing sample that may also include other cells) may be transferred to a new culture environment and cultured. Cell lines can be used for any of a variety of purposes, depending on the embodiment. For example, cell lines can be used to provide cells for producing biopharmaceutical antibodies or hybrid molecules (e.g., drugs, including bispecific T-cell engager (BiTE®) antibodies such as BLINCYTO® (blinatumomab), or monoclonal antibodies), or to provide cells for research and / or development purposes. "Accuracy" has its ordinary and accustomed meaning as would be understood by one of skill in the art in light of this disclosure. It refers to the probability that a result is neither a false negative nor a false positive. For example, accuracy can be calculated as 100% - (Type I error) - (Type II error).

[0009] Those skilled in the art will understand that the figures described herein are included for illustrative purposes and are not intended to limit the present disclosure. The figures are not necessarily to scale, emphasis instead being placed upon illustrating the principles of the present disclosure. It will be understood that in some instances, various aspects of the described embodiments may be shown exaggerated or enlarged to facilitate an understanding of the described embodiments. In the figures, like reference numerals generally refer to functionally similar and / or structurally similar components throughout the various views. [Brief explanation of the drawings]

[0010] [Figure 1] FIG. 1 is a simplified block diagram of an example system in which the techniques described herein may be implemented. [Figure 2A]2 illustrates an exemplary visual inspection system that may be used in the system of FIG. 1. [Figure 2B] 2 illustrates an exemplary visual inspection system that may be used in the system of FIG. 1. [Figure 2C] 2 illustrates an exemplary visual inspection system that may be used in the system of FIG. 1. [Figure 3] FIG. 1 is a flow diagram of an exemplary process for determining whether to use or discard a well sample. [Figure 4] 1 illustrates an exemplary embodiment and scenario in which well images are generated by a visual inspection system at two different times. [Figure 5] 1 illustrates an exemplary embodiment and scenario in which well images are generated by a visual inspection system at three different times. [Figure 6] 1 illustrates an embodiment and scenario where multiple candidate objects are identified in a well image. [Figure 7] 1 shows an exemplary zoomed-in image of a candidate object. [Figure 8] FIG. 1 is a flow diagram of an exemplary method for facilitating clonal selection. DETAILED DESCRIPTION OF THE INVENTION

[0011] The various concepts introduced above and described in more detail below can be implemented in any of many ways, and the described concepts are not limited to any particular implementation manner. Example embodiments are provided for illustrative purposes.

[0012] FIG. 1 is a simplified block diagram of an exemplary system 100 in which the techniques described herein may be implemented. System 100 includes a visual inspection system 102 communicatively coupled to a computer system 104. Visual inspection system 102 includes hardware (e.g., a well plate stage, one or more lenses and / or mirrors, an imager, etc.), as well as firmware and / or software, configured to capture digital images of wells within a well plate. An exemplary embodiment of visual inspection system 102 is shown in FIGS. 2A-2C. Referring first to FIG. 2A, visual inspection system 102 may include a stage 202 configured to accept a well plate 204 containing multiple wells (not shown in FIG. 2A). Well plate 204 may be of any suitable size, any suitable shape, and have any suitable number of wells (e.g., 6, 24, 96, 384, 1536, etc.) arranged thereon. Furthermore, the wells may be arranged in any suitable pattern on well plate 204, such as, for example, a 2:3 rectangular matrix.

[0013] The visual inspection system 102 further includes an illumination system 208, a first imager 210 configured to acquire wide-field images, and in some embodiments (for purposes described further below), a second imager 212 configured to acquire high-magnification images. In other embodiments, the visual inspection system may omit the imager 212. The illumination system 208 may include any suitable number and / or type of light source configured to generate source light, which illuminates each well in the well plate 204 when the well is positioned in the optical path of the imager 210 or imager 212. The imager 210 includes a telecentric lens 220 and a wide-field camera 222. The telecentric lens 220 may be a high-fidelity telecentric lens with 1x magnification, and the wide-field camera 222 may be, for example, a charge-coupled device (CCD) camera. Imager 210 is configured and positioned relative to stage 202 such that it can capture images each showing a single entire well (with a resolution suitable for resolving candidate objects within the well from the rest of the well, e.g., with the single well occupying substantially all of the image or at least a majority of the image) when the wells are appropriately positioned on stage 202 and illuminated by illumination system 208. Imager 212 includes objective lens 230, mirror 232, and high-resolution camera 234. Objective lens 230 may be a 20x magnification, long-working distance objective lens (or lens system), and high-resolution camera 234 may be, for example, another CCD. Mirror 232 may enable imager 212 to have an appropriately low profile.

[0014] In some embodiments, each well of the well plate 204 has one or more transparent and / or opaque portions. For example, each well may be completely transparent or may have a transparent bottom with opaque sidewalls. Each well may be generally cylindrical or have another suitable shape (e.g., cubic, etc.). The visual inspection system 102 images the wells of the well plate 204, for example, sequentially imaging each well in the well plate 204. To this end, the visual inspection system 102 is configured to move the stage 202 along one or more axes (e.g., x and y) to sequentially align each well with the optical path of the illumination system 208 and the imager 210 for individual well analysis. For example, the stage 202 may be coupled to one or more motorized actuators. Once each well is aligned with the optical path of the illumination system 208 and the imager 210, the imager acquires one or more images of the illuminated well. Any cells in a given well may generally lie in a flat plane above the base of the well, in which case the well may be imaged from a top-down or bottom-up perspective. In such embodiments, the visual inspection system 102 may also be configured to move the stage 202 in the vertical (z) direction to maintain focus on a flat, thin layer in which cells may reside. Vertical actuation may also enable three-dimensional scanning of the well sample (e.g., to detect cells adhering to the sidewall of the well above the well base). The visual inspection system 102 may also apply any suitable techniques to mitigate vibration and / or mechanically support high-fidelity imaging, both of which may be important when high-magnification imaging is used.

[0015] 2B and 2C show the configuration of visual inspection system 102 when imager 210 and imager 212 are each used to capture an image of a particular well (in this scenario, well 240 located in one corner of well plate 204). For clarity, some components of visual inspection system 102 are omitted from FIGS. 2B and 2C. As seen in FIGS. 2B and 2C, well 240 can be selectively positioned in the optical path of imager 210 or imager 212 by moving stage 202 relative to imagers 210 and 212. (For clarity, stage 202 is not explicitly shown in FIGS. 2B and 2C, but it will be understood that stage 202 is appropriately positioned; see, e.g., FIG. 2A.) In various embodiments, this can be achieved by moving stage 202 or by moving imagers 210 and 212.

[0016] 2A-2C depict only one embodiment of visual inspection system 102, and it will be understood that other embodiments are possible. For example, visual inspection system 102 may include additional imagers similar to imagers 210 and / or 212 (e.g., for three-dimensional imaging). As a further example, visual inspection system 102 may instead illuminate the sample using an oblique illumination source rather than a backlight, or may instead be configured to utilize fluorescence imaging or phase-contrast-based imaging. Additionally, although not shown in FIGS. 2A-2C, visual inspection system 102 may include one or more communication interfaces and processors to enable communication with computer system 104 and to provide local control of the operation of stage 202, illumination system 208, and / or imagers 210 and 212 (e.g., in response to commands received from computer system 104).

[0017] 1 , computer system 104 is generally configured to control / automate the operation of visual inspection system 102 and to receive and process images captured / generated by visual inspection system 102, as described further below. Computer system 104 is also coupled to training server 106 via network 108. Network 108 may be a single communication network or may include multiple communication networks of one or more types (e.g., one or more wired and / or wireless local area networks (LANs) and / or one or more wired and / or wireless wide area networks (WANs), such as the Internet). As discussed further below, training server 106 is generally configured to train one or more machine learning (ML) models 109, which training server 106 transmits to computer system 104 via network 108 to enable computer system 104 to detect and / or classify objects in images generated by visual inspection system 102. In various embodiments, the training server 106 may provide the ML models 109 as a "cloud" service (e.g., Amazon Web Services), or the training server 106 may be a local server. In alternative embodiments, the ML models 109 are transferred to the computer system 104 by techniques other than remote download (e.g., by physically transferring a portable storage device to the computer system 104), in which case the system 100 may not include the network 108. In other embodiments, one, some, or all of the ML models 109 may be trained on the computer system 104 and then uploaded to the server 106. In other embodiments, the computer system 104 itself performs the model training, in which case the system 100 may omit both the network 108 and the training server 106.In yet other embodiments, some or all of the components of computer system 104 shown in FIG. 1 (e.g., one, some, or all of modules 120-126) are instead included in visual inspection system 102, in which case visual inspection system 102 can communicate directly with training server 106 over network 108.

[0018] 1, computer system 104 includes a processing unit 110, a network interface 112, and a memory unit 114. However, in some embodiments, computer system 104 includes two or more computers that are co-located with each other or remote from each other. In these distributed embodiments, the operations described herein related to processing unit 110, network interface 112, and / or memory unit 114 may be divided among multiple processing units, network interfaces, and / or memory units, respectively.

[0019] The processing unit 110 includes one or more processors, each of which may be a programmable microprocessor that executes software instructions stored in the memory unit 114 to perform some or all of the functionality of the computer system 104 as described herein. The processing unit 110 may include, for example, one or more graphics processing units (GPUs) and / or one or more central processing units (CPUs). Alternatively, or in addition, some of the processors in the processing unit 110 may be other types of processors (e.g., application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), etc.), and some of the functionality of the computer system 104 described herein may instead be implemented in hardware. The network interface 112 may include any suitable hardware (e.g., front-end transmitter and receiver hardware), firmware, and / or software configured to communicate with the training server 106 over the network 108 using one or more communication protocols. For example, the network interface 112 may be or include an Ethernet interface, allowing the computer system 104 to communicate with the training server 106 over the Internet, an intranet, or the like. Memory unit 114 may include one or more volatile and / or non-volatile memories, including one or more of any suitable memory types, such as read-only memory (ROM), random-access memory (RAM), flash memory, solid-state drive (SSD), hard disk drive (HDD), etc. Collectively, memory unit 114 may store one or more software applications, data received / used by those applications, and data output / generated by those applications.

[0020] Memory unit 114 stores software instructions for clone selection application 118 that, when executed by processing unit 110, identify wells / samples in which cell colonies developed from a single clone. While various modules of application 118 are discussed below, it will be understood that the modules may be distributed among different software applications and / or the functionality of any one of such modules may be divided among different software applications.

[0021] A visual inspection system (VIS) control module 120 of application 118 controls / automates the operation of visual inspection system 102 via commands or other messages so that images of samples in the wells of well plate 204 can be generated with little or no human intervention. Visual inspection system 102 may transmit captured images to computer system 104 for storage in memory unit 114 or another suitable memory not shown in FIG. 1 .

[0022] The cell colony detection module 122 of the application 118 attempts to identify / detect any cell colonies shown in the well images received from the visual inspection system 102. The cell colony detection module 122 may detect cell colonies using, for example, a machine learning model such as a convolutional neural network (CNN) or using a non-machine learning algorithm (visual analytics software) that detects colonies without the need for training. The operation of the cell colony detection module 122 is discussed in further detail herein.

[0023] The object detection module 124 of the application 118 attempts to identify / detect any individual objects (1) that appear in the well images received from the visual inspection system 102 and (2) that potentially may be single cells. However, due to limitations in the models or algorithms employed by the object detection module 124 and / or limitations in the resolution of the images processed by the object detection module 124, the object detection module 124 may not be able to confirm with sufficient accuracy or confidence whether each detected object is actually a single cell. Therefore, each object detected by the object detection module 124 is initially viewed only as a “candidate” for being a single cell. The object detection module 124 may detect objects using a relatively simple machine learning model or using a non-machine learning algorithm (visual analytics software) that detects objects without requiring training. In some embodiments, the object detection module 124 utilizes OpenCV to process images and detect objects therein. In some embodiments, object detection module 124 only outputs objects that are above a minimum threshold size (e.g., a threshold pixel width or number of pixels) and / or below a maximum threshold size to avoid identifying objects that may not be single cells. For example, a minimum threshold size may filter out dead pixels and very small contaminants in an image, while a maximum threshold size may filter out very large contaminants or bubbles, etc. The operation of object detection module 124 is discussed in further detail herein.

[0024] The object classification module 126 of the application 118 attempts to classify individual objects detected by the object detection module 124. In some embodiments, the object classification module 126 processes subsets of the same well image processed by the object detection module 124. For example, the object detection module 124 can accept subsets of image pixels as input, with each subset corresponding to a single candidate object (possibly including pixels corresponding to a small region surrounding the candidate object). In such embodiments, the visual inspection system 102 may not include the imager 212. Alternatively, the VIS control module 120 may cause the visual inspection system 102 to capture additional, higher magnification images of each candidate object using the imager 212 and then input those zoomed-in images to the object classification module 126. In either case, the object classification module 126 may use a CNN to classify the objects. In some embodiments, the CNN requires significantly more processing power and / or processing time than the model or algorithm applied by the object detection module 124. The operation of the object classification module 126 and embodiments in which higher magnification images are captured and processed are described in further detail herein.

[0025] Operation of system 100, according to some embodiments, will now be described with reference to FIGS. 1 and 2A-2C. Initially, training server 106 trains ML model 109 using data stored in training database 130. ML model 109 may include, for example, a first CNN implemented by cell colony detection module 122 and a second CNN implemented by object classification module 126. Training database 130 may include a single database stored in a single memory (e.g., HDD, SSD, etc.), a single database stored across multiple memories, or multiple databases stored in one or more memories. For each different model in ML model 109, training database 130 may store a corresponding set of training data (e.g., input / feature data and corresponding labels). To train the CNN implemented by object classification module 126, for example, training database 130 may include multiple input datasets, each corresponding to an image of an object, along with a label indicating the correct object classification for that image. To improve classification accuracy, the training data may include images containing a wide variety of object types (e.g., single cells, doublets, different types of debris, different types of wall anomalies, etc.) and / or images captured under a wide variety of conditions (e.g., lighting that produces shadows on various types of well walls, etc.). The labels may represent classifications of the training images provided by human analysts of the corresponding images. In some embodiments, the training server 106 uses additional labeled datasets in the training database 130 to validate the generated ML models 109 (e.g., to confirm that a given one of the ML models 109 provides at least some minimum acceptable accuracy). The training server 106 then provides the ML models 109 to the computer system 104 (e.g., via remote download over the network 108). In some embodiments, the training server 106 also continuously updates / refines one or more ML models 109.For example, after the ML model 109 is initially trained to provide a sufficient level of accuracy, the visual inspection system 102 or computer system 104 may provide additional images to the training server 106 over time, and the training server 106 may use supervised or unsupervised learning techniques to further improve the accuracy of the model.

[0026] Wells (e.g., wells 240 in FIGS. 2B and 2C ) in the well plate 204 of the visual inspection system 102 are at least partially filled, either automatically or manually, with media containing nutrients (e.g., amino acids, vitamins, etc.), growth factors, and / or other components appropriate for the cells. For example, each well may be at least partially filled. In some embodiments and / or scenarios, an attempt is made to inoculate each well with one and only one clone / cell. For example, flow cytometry techniques, such as fluorescence-activated cell sorting (FACS) techniques, may be used to inoculate each well with a single clone.

[0027] The well plate 204 is then loaded onto the stage 202, and the VIS control module 120 causes the visual inspection system 102 to move the stage 202 in small increments (e.g., in the x and / or y directions) and activate the imager 210 (and possibly the illumination system 208) in a synchronized manner, causing the imager 210 to capture at least one image for each well. This initial image of each well may be captured immediately after inoculation during the first day of culture and may show the entire area of ​​each well (e.g., from a bottom-up view). The visual inspection system 102 may store each well image locally or may immediately transfer each image to the computer system 104 (e.g., for storage in the memory unit 114).

[0028] The process of imaging the wells can be repeated at regular or irregular intervals, depending on the embodiment. For example, VIS control module 120 can cause visual inspection system 102 to image each one of the wells once a day or once a week over a given culture period (e.g., 10 days, 14 days, etc.). Alternatively, the wells can be imaged only at the beginning and end of the culture period (e.g., days 1 and 14 of a 14-day culture period). As well as generating well images, or in batches after a subset (or all) of the images have been generated, visual inspection system 102 transmits the images to computer system 104 for automated analysis. Similar to the process of capturing well images, the process of transferring images to computer system 104 can be automated (e.g., triggered by a command from VIS control module 120).

[0029] Generally, application 118 attempts to detect cell colonies in images of wells and then, for each detected colony, identifies whether the colony was formed solely from a single clone / cell. To this end, cell colony detection module 122 (e.g., using a CNN, which is one copy of ML model 109) may analyze each of the well images from the end of the culture period. For each colony that colony detection module 122 detects in a given well, object detection module 124 may analyze images from the first day of that same well to detect any objects therein. However, either generally or under certain conditions (e.g., shadows from the well walls), object detection module 124 may not be able to accurately distinguish a single cell from debris, well wall anomalies, and / or other objects. Therefore, for each “candidate” object identified by object detection module 124, object classification module 126 instead attempts to classify the object. As noted above, object classification module 126 may use any suitable classification scheme, as long as the classification reveals whether the depicted object is a single cell (or a single living cell). In some embodiments, object classification module 126 accomplishes its classification task by using a CNN (a copy of ML model 109) to process a particular portion of the well image that has already been analyzed by object detection module 124. For example, object classification module 126 may apply just those pixels that correspond to a candidate object (and possibly a small surrounding region) as input to the CNN, and the CNN output is the predicted class / type for that candidate object.

[0030] However, in other embodiments, the resolution of the images analyzed by object detection module 124 is simply not sufficient to provide high-accuracy classification of candidate objects. Thus, in these embodiments, VIS control module 120 may cause visual inspection system 102 to capture one or more additional images of at least some of the wells immediately after inoculation on the first day of culture. For example, for a given well, visual inspection system 102 may send the initial “Day 1” image to computer system 104 when that image is captured, and object detection module 124 may analyze the image for candidate objects immediately thereafter. For each candidate object detected in the image / well, VIS control module 120 may cause visual inspection system 102 to shift stage 202 by a small amount so that the candidate object is in the optical path of higher magnification imager 212, e.g., approximately centered in this optical path. To appropriately adjust the position of stage 202, the location of each candidate object within the well may be determined based on the image of the entire well (i.e., the “Day 1” image processed by object detection module 124). The visual inspection system 102 transmits each higher magnification image to the computer system 104, where the object classification module 126 processes / analyzes the object images using a CNN to classify the indicated candidate objects.

[0031] The CNN used by object classification module 126 (and possibly by cell colony detection module 122) may include any suitable number of convolutional layers for two-dimensional convolution (e.g., to detect features such as edges in an image), any suitable number of pooling layers (e.g., downsampling layers to reduce computation while preserving the relative positions of features), and any suitable number of fully connected layers (e.g., to provide high-level inference based on features). Alternatively (e.g., if visual inspection system 102 implements three-dimensional imaging techniques), the CNN of object classification module 126 may utilize three-dimensional convolution to detect features in three dimensions. Regardless of whether a two-dimensional or three-dimensional CNN is used, the CNN may also provide a probability for each prediction / classification indicating the likelihood that the prediction / classification is accurate. In some embodiments, object classification module 126 defaults to a particular classification (e.g., “unknown” or “not a single cell”) if an object in a given image cannot be classified with at least a threshold probability / confidence level.

[0032] Based on whether the cell colony in the sample / well formed from a single clone, application 118 can determine whether a given well sample should be discarded or proceed to the next stage of cell line development. However, depending on the embodiment, determining whether a colony formed from a single clone may involve more than detecting a single cell in the sample / well. For example, application 118 can conclude that a colony formed from a single clone only if both (1) object classification module 126 classifies a candidate object in a well as a single cell, and (2) the location of that single cell in the well overlaps (or is within a threshold distance, etc.) with the location of the colony in the well. Various other ways in which a determination may be made are discussed herein. It will be understood that a conclusion that a cell colony did not form / develop from a single cell may mean that there is positive visual evidence that the colony formed from two or more cells, or that there simply is not enough visual evidence to confidently determine how the colony formed.

[0033] The application 118 may display an indication in a user interface whether a given well sample should be discarded or proceed to the next development stage, and / or may communicate with another application and / or computer system to trigger, for example, automatic discard or a cell line development stage. Cell line development may be for any suitable purpose, depending on the embodiment and / or scenario. For example, the cell line may be used to develop biopharmaceutical antibodies or hybrid molecules (e.g., bispecific T-cell engager (BiTE®) antibodies such as BLINCYTO® (blinatumomab), or monoclonal antibodies, etc.), or for research and / or development purposes.

[0034] 3 is a flow diagram of an exemplary process 300 for determining whether to use or discard a sample in a well. Process 300 may be performed, for example, by system 100 of FIG. 1 and (in some embodiments) may be fully automated. In exemplary process 300, in a preliminary stage 302, individual wells (e.g., wells in well plate 204) are seeded with clones (cells) using FACS subcloning. During an initial stage 304 of the culture period (referred to as "Day 1" for simplicity, although it will be understood that initial stage 304 may occur during other periods immediately following seeding), at least one image of each well is generated (e.g., by imager 210 of visual inspection system 102 as controlled by VIS control module 120). The "Day 1" image may be captured at the beginning of the first culture day (e.g., immediately after seeding the wells).

[0035] In a subsequent stage 306, at least one additional image of each well is generated (e.g., by imager 210) at approximately the midpoint of the culture period (referred to for simplicity as "day 7," but it will be understood that the approximate midpoint can occur at other time periods, e.g., within 5-7 days, 5-8 days, 5-9 days, 6-7 days, 6-8 days, 6-9 days, 7-8 days, or 7-9 days). In some embodiments, stage 306 can occur earlier or later, stage 306 may be omitted, or process 300 can include additional stages in which well images are generated (e.g., once a day, or every other day, etc.). Referring to stage 308, at the end of the culture period, yet another image (or another set of images) of each well is generated (e.g., by imager 210) (referred to for simplicity as "day 14," but it will be understood that the end of the culture period can be at a different time period). In some embodiments, the culture period is longer or shorter than 14 days.

[0036] In FIG. 3 , stages 310, 312, and 314 represent operations that occur for each sample / well. In stage 310, images from at least stages 304 and 308 (e.g., “Day 1” and “Day 14”) are analyzed to determine whether a cell colony is present in the well, and if so, whether the colony is formed from a single clone. Stage 310 may include, for example, the operations of cell colony detection module 122, object detection module 124, and object classification module 126, as described above. Stage 310 may occur entirely after stage 308 or may occur at other times during process 300. As described above in connection with FIG. 1 , for example, it may be necessary to analyze at least some well images in early stage 304 (e.g., immediately after inoculation, such as on the first day of culture). Thus, stage 310 may include object detection occurring in parallel with stage 304, which may further include acquiring additional zoomed-in images (e.g., using imager 212) of each detected candidate object. In these embodiments, analysis of each candidate object (e.g., by object classification module 126) may occur in parallel with (or immediately after) stage 304, or may occur later, such as at stage 308. In some embodiments, stage 310 also includes analysis of the "day 7" image generated in stage 306. For example, stage 310 can determine whether a particular cell colony formed from a single clone not only by identifying a single clone within a well, but also by analyzing the "day 7" image of the well to detect colonies at intermediate growth stages.

[0037] If it is not determined in stage 310 that a particular cell colony formed from a single clone (e.g., it cannot be determined with sufficient confidence that the colony formed from a single clone), flow proceeds to stage 312, where the corresponding sample is discarded. Conversely, if it is determined in stage 314 that the colony formed from a single clone, flow proceeds to stage 314, where the corresponding sample is forwarded to the next stage of the cell line development process. For example, cells of the sample (e.g., as part of a cell-containing sample) can be transported and cultured in a new culture environment. Information regarding cell culture can be found, for example, in Green and Sambrook, "Molecular Cloning: A Laboratory Manual" (4th edition) Cold Spring Harbor Laboratory Press 2012, which is incorporated herein by reference in its entirety.

[0038] 4 illustrates an exemplary scenario in which images 400, 402 of a well 410 (e.g., one of the wells in well plate 204) are generated by a visual inspection system (e.g., visual inspection system 102) at two different times. Specifically, a first image 400 of well 410 is captured immediately after inoculation of well 410 (e.g., at stage 304 of process 300), while a second image 402 of well 410 is captured at the end of the incubation period (e.g., at stage 308 of process 300). Images 400 and 402 may be captured, for example, by imager 210. Each of images 400, 402 represents a bottom-up perspective of well 410 and shows the entire contents of well 410. It will be understood that in some embodiments, well images 400, 402 also include some area outside the perimeter of well 410 (e.g., if the images are rectangular).

[0039] 1 , the cell colony detection module 122 of the application 118 analyzes the image 402 (e.g., using a CNN) at or after the end of the culture period (e.g., at stage 308 of the process 300) to detect cell colonies 412 within the well 410. In some embodiments, in response to detecting the cell colonies 412, the application 118 may analyze the image 400 using the object detection module 124 (e.g., using OpenCV) to detect any candidate objects within the well 410 at the start of the culture. As can be seen in the exemplary scenario of FIG. 4 , the image 400 shows only a single object 414 within the well 410. Therefore, the object detection module 124 outputs an indication of only the single candidate object. The application 118 may then extract the image pixels corresponding to the object 414 and provide that image portion as input to the object classification module 126 (e.g., another CNN), which analyzes the image portion and classifies the object 414 (e.g., as a single cell, or doublet, debris, etc.).

[0040] In some embodiments, as discussed herein, accurate classification may include a higher magnification of the candidate object. In such embodiments, application 118 may analyze image 400 immediately after it is generated (e.g., immediately after inoculation). In particular, object detection module 124 may analyze image 400 to detect object 414, and in response, VIS control module 120 may move stage 202 so that object 414 is approximately centered in the optical path of a higher magnification imager (e.g., imager 212) and cause that imager to capture a zoomed-in image of object 414 (not shown in FIG. 4 ) immediately after image 400 is captured. Object classification module 126 may then analyze the zoomed-in image to classify object 414. Thereafter, at or after the end of the culture period (e.g., at stage 308 of process 300, such as after day 14), cell colony detection module 122 analyzes image 402 to detect cell colonies 412.

[0041] Regardless of when the cell colony 412 is detected and the object 414 is classified, the classification (and possibly the location of the colony 412 and object 414 within the well 410) can be used to determine whether the colony 412 was formed from a single clone. In an alternative scenario in which the object detection module 124 detects multiple candidate objects in the image 400, the object classification module 126 can also analyze the images of the remaining candidate objects to determine whether any one or more of those objects are single cells.

[0042] If the object in image 400 is not classified as a single cell, application 118 may conclude that cell group 412 did not form from a single clone. Conversely, if object 414 (or another object in image 400) is classified as a single cell, application 118 may determine that cell group 412 formed from a single clone. However, in some embodiments, this latter determination may also be based on one or more other factors. For example, application 118 may determine that cell group 412 formed from a single clone only if the location of object 414 in well 410 overlaps with the location of cell group 412 in well 410. As another example, application 118 may determine that cell group 412 formed from a single clone only if (1) the location of object 414 in well 410 overlaps with the location of cell group 412 in well 410, and (2) no other candidate objects in image 400 are classified as cells (e.g., as single cells or doublets).

[0043] In some embodiments, one or more intermediate well images are used to determine or confirm that a cell colony has developed from a single clone. One such embodiment, discussed herein with reference to FIG. 5, illustrates an exemplary embodiment and scenario in which images 500, 502, and 504 of a well 510 are generated by a visual inspection system (e.g., visual inspection system 102) at three different times. Specifically, a first image 500 of a well 510 is captured immediately after inoculation of the well 510 (e.g., at stage 304 of process 300), a second image 502 of the well 510 is captured near the midpoint of the culture period (e.g., at stage 306 of process 300), and a third image 504 of the well 510 is captured at the end of the culture period (e.g., at stage 308 of process 300). Images 500, 502, and 504 may be captured, for example, by imager 210. Each of the images 500, 502, 504 represents a bottom-up perspective of the well 510 and shows the entire contents of the well 510. It will be appreciated that, similar to images 400, 402 of Figure 4, in some embodiments, the images 500, 502, 504 may also include an area outside the perimeter of the well 510.

[0044] 1 , cell colony detection module 122 of application 118 analyzes image 504 (e.g., using a CNN) at or after the end of the culture period (e.g., at stage 308 of process 300) to detect cell colonies 520 within well 510. In some embodiments, in response to detecting cell colonies 520, application 118 may analyze image 500 using object detection module 124 (e.g., using OpenCV) to detect any candidate objects within well 510 at the start of culture. As can be seen in the exemplary scenario of FIG. 5 , image 500 shows only a single object 522 within well 510. Thus, object detection module 124 outputs an indication of only the single candidate object. The application 118 may then extract the image pixels corresponding to the object 522 and provide that image portion as input to the object classification module 126 (e.g., another CNN), which analyzes the image portion and classifies the object 522 (e.g., as a single cell, or doublet, debris, etc.).

[0045] In some embodiments, as discussed herein, accurate classification may include a higher magnification of the candidate object. In such embodiments, application 118 may analyze image 500 immediately after it is generated (e.g., immediately after inoculation, such as at stage 304 of process 300). In particular, object detection module 124 may analyze image 500 to detect object 522, and in response, VIS control module 120 may move stage 202 so that object 522 is at least approximately centered in the optical path of a higher magnification imager (e.g., imager 212) and cause that imager to capture a zoomed-in image of object 522 (not shown in FIG. 4 ). Object classification module 126 may then analyze the zoomed-in image to classify object 522. Subsequently, at or after the end of the culture period (e.g., at stage 308 of process 300), cell colony detection module 122 may analyze image 504 to detect cell colonies 520.

[0046] Regardless of when cell colony 520 is detected and object 522 is classified, the classification (and possibly the location of colony 520 and object 522 within well 510) can be used to determine whether colony 520 formed from a single clone. As described above, the presence of other candidate objects in image 500 and their classification by object classification module 126 may also influence that determination. Additionally, in this embodiment, application 118 analyzes image 502. For example, cell colony detection module 122 may analyze image 502 (e.g., using the same CNN used to analyze image 504) to detect colony 524. Application 118 may then compare the location of colony 524 to the location of object 522 and / or the location of cell colony 520, and / or compare the size of colony 524 (e.g., pixel width, total number of pixels, etc. corresponding to colony 524 in image 502) to the size of colony 520 (e.g., pixel width, total number of pixels, etc. corresponding to colony 520 in image 504). In one exemplary embodiment, application 118 determines that cell colony 520 was formed from a single clone only if (1) the location of colony 524 in well 510 overlaps with the location of colony 520 in well 510, and (2) the size of colony 524 is smaller than the size of colony 520.

[0047] 4 and 5 are provided for illustrative purposes, and it will be understood that other numbers of images (e.g., 4, 5, 10, etc.) can be analyzed, and / or images from different days within the culture period can be analyzed to determine whether a cell colony formed from a single clone. Furthermore, algorithms other than those described above can alternatively or additionally be used to determine whether a cell colony formed from a single clone. For example, a series of other image processing functions (e.g., intensity thresholding, blob analysis, and subsequent binary morphology operators) can alternatively or additionally be used to determine whether a cell colony formed from a single clone.

[0048] 6 and 7 correspond to example embodiments and scenarios in which multiple candidate objects are detected and then zoomed-in images of the objects are obtained for more accurate classification. Referring first to FIG. 6, image 600 (which may be the entire image or only a portion of the image) shows well 610 (e.g., one of the wells in well plate 204). Image 600 may be captured, for example, by imager 210. In image 600, object detection module 124 has detected a first candidate object 612 and a second candidate object 614. The rectangles surrounding objects 612, 614 merely indicate that the objects have been detected and may or may not actually be generated or displayed on any user interface.

[0049] Because application 118, in this particular example, is unable to accurately classify objects 612, 614 based on image 600 (or at least is unable to accurately classify similarly sized objects on a sufficiently consistent basis), VIS control module 120 may be configured to cause visual inspection system 102 (e.g., using imager 212) to capture zoomed-in images of objects 612, 614. FIG. 7 shows various exemplary zoomed-in images 700, 702, 704, 706, any one of which may correspond to, for example, one of candidate objects 612, 614. Specifically, image 700 is a high-magnification image of a single cell 710, image 702 is a high-magnification image of an anomaly 720 in a well wall 722, image 704 is a high-magnification image of a doublet 730, and image 706 is a high-magnification image of debris 740. The object classification module 126 may use a CNN trained to classify objects according to all of these classes, some of these classes (e.g., "single cell" vs. "other"), and / or one or more other classes (e.g., "empty cell," "bubble," etc.).

[0050] 8 is a flow diagram of an exemplary method 800 for facilitating clonal selection (e.g., for a cell line development process). Method 800 may be performed by one or more portions of system 100 (e.g., visual inspection system 102 and computer system 104), or another suitable system. As a more specific example, block 802 of method 800 may be performed by visual inspection system 102 of FIGS. 1 and 2, while blocks 804-812 may be performed by computer system 104 (e.g., by processing unit 110 in executing instructions stored in memory unit 114).

[0051] At block 802 of method 800, a plurality of time-series images of a well containing culture medium are generated by a first imaging unit (e.g., by imager 210). The culture medium may include cell nutrients, growth factors, etc., and has previously been inoculated (e.g., using a flow cytometry technique such as a FACS technique). The generated images include at least a first time-series image generated at a first time (e.g., immediately after inoculation, such as during the first day of the culture period) and a second time-series image generated at a second subsequent time (e.g., the last day of the culture period). The plurality of images may also include one or more additional time-series images generated at a time between the first and second time-series images.

[0052] At block 804, the first time-series images are analyzed (e.g., using OpenCV and / or other suitable object detection and / or segmentation software) to detect one or more candidate objects shown in the first time-series images. Block 804 may include detecting only objects that are within the shown well and that are below a certain maximum size (e.g., number of pixels). Depending on the embodiment, block 804 may or may not include segmenting the first time-series images to identify which image pixels correspond to the objects.

[0053] In block 806, for each candidate object detected in block 804, an image of the candidate object is analyzed using a CNN to determine whether the candidate object is a single cell. If candidate object segmentation was performed in block 806, for example, block 806 may include analyzing a set of pixels corresponding to each candidate object. Alternatively, method 800 may further include, for each candidate object detected in block 804, generating a zoomed-in image of a portion of a well containing the candidate object, in which case block 806 may include analyzing each zoomed-in image using a CNN. The zoomed-in image may be generated using a second, higher magnification imaging unit, such as imager 212. Generating the zoomed-in image may include shifting the well (e.g., by moving a stage on which a well plate containing the well is positioned) so that the well is aligned, for example, with the optical path of the second imaging unit.

[0054] Regardless of the type of image being analyzed (e.g., a segmented image portion or a new zoomed-in image), the CNN used in block 804 may be configured / trained to classify objects according to different possible object types, including at least one type corresponding to a single cell (or single living cell, etc.). For example, the CNN may classify a given object as one of "single cell," "doublet," and "debris," or as one of "single cell" and "other."

[0055] In some embodiments, blocks 804 and 806 occur after block 802 is completed. In other embodiments, blocks 804 and 806 occur before the second time-series image is generated in block 802. For example, blocks 804 and 806 may occur almost immediately after the first time-series image is generated in block 802.

[0056] In block 808, the second time series images are analyzed to detect cell colonies shown therein. The second time series images may be analyzed, for example, using a CNN (e.g., a different CNN than the one used in block 806) or using one or more other image processing functions (e.g., intensity thresholding, blob analysis, binary morphological operators, etc.). In various embodiments, block 808 may occur several days after blocks 804 and 806 (e.g., if blocks 804 and 806 occur immediately after the first time series images are generated) or immediately after blocks 804 and 806 (e.g., if all image analysis occurs after block 802 is completed). However, in other embodiments, blocks 804 and 806 occur after (e.g., in response to) the determination made in block 808. That is, a system implementing method 800 may attempt to detect a single cell in a well if, and only if, a cell colony has already been detected in the well.

[0057] At block 810, based at least in part on the determination made at block 806 (i.e., based on whether each of the at least one or more candidate objects is a single cell), it is determined whether the cell colony detected at block 808 was formed from only one cell. For example, at block 810, if the candidate object includes two or more cells, or if there are no living objects, such as single cells, among the candidate objects, it may be determined that the colony was not formed from a single cell. It will be understood that a conclusion that the colony was not formed from a single cell may mean that there is positive visual evidence that the colony was formed from two or more cells, or that there is simply not enough visual evidence to determine how the colony was formed.

[0058] In some embodiments, the determination in block 810 is also based on one or more other factors. If the first candidate object in the first time-series image is determined to be a single cell in block 806, for example, block 810 may include comparing the location of the candidate object in the well to the location of a cell colony in the well. Additionally or alternatively, if a third time-series image is generated at a time between the first and second time-series images, and if the first candidate object in the first time-series image is determined to be a single cell in block 806, block 810 may include analyzing the third time-series image to determine whether the candidate object has developed into a cell colony. For example, an intermediate colony may be detected in the third time-series image, and the cell number, size, and / or location of the intermediate colony may be analyzed and compared to the cell number, size, and / or location of the cell colony detected in the second time-series image.

[0059] At block 812, output data is generated. The output data indicates whether the cell colony formed from only one cell according to the determination made at block 810. The output data may be displayed to a user on a user interface of the computing device and / or transmitted to one or more other software modules and / or computer systems, for example. In some embodiments, the output data notifies a user or one or more software modules or systems that the well containing the cell colony should be selected for one or more additional stages of cell line development (if it was determined at block 810 that the colony formed from only one cell) or that the contents of the well should be discarded (if it was determined at block 810 that the colony did not form from only one cell). In some embodiments, method 800 further includes using the contents of the well to develop a cell line (e.g., for biopharmaceutical manufacturing, for research and development, etc.), e.g., by transferring the cells from the well to a new culture environment and culturing the cells in the new culture environment, or further includes discarding the contents of the well based on the output data.

[0060] While the systems, methods, devices, and components thereof have been described with reference to exemplary embodiments, they are not limited thereto. The detailed description is to be construed as an example only and does not describe every possible embodiment of the invention, as describing every possible embodiment would be impractical, if not impossible. Many alternative embodiments can be implemented using either current technology or technology developed after the filing date of this patent, and such embodiments will still fall within the scope of the claims that define the invention.

[0061] Those skilled in the art will understand that numerous modifications, changes, and combinations can be made to the above-described embodiments without departing from the scope of the present invention, and that such modifications, changes, and combinations are to be construed as being within the scope of the inventive concept.

Claims

1. 1. A method for facilitating clonal selection, comprising: generating, by a first imaging unit, a plurality of time series images of a well containing a medium, the plurality of time series images including a first time series image generated at a first time and a second time series image generated at a second time after the first time; detecting, by one or more processors, one or more candidate objects shown in the first time-series images by analyzing the first time-series images; for each of the one or more candidate objects, determining whether the candidate object is a single cell by analyzing an image of the candidate object using a convolutional neural network by the one or more processors; detecting cell colonies shown in the second time series images by analyzing the second time series images with the one or more processors; determining whether the cell colony is formed from only one cell based at least in part on whether each of the one or more candidate objects is determined to be a single cell by the one or more processors; generating, by the one or more processors, output data indicative of whether the cell colony was formed from only one cell; A method comprising:

2. the method including using the convolutional neural network to determine that a first candidate object of the one or more candidate objects is a single cell; determining whether the cell colony is formed from only one cell further comprises comparing a position of the first candidate object in the well with a position of the cell colony in the well. The method of claim 1.

3. the plurality of time-series images further includes a third time-series image generated at a third time between the first time and the second time, the method including using the convolutional neural network to determine that a first candidate object of the one or more candidate objects is a single cell; determining whether the cell colony is formed from only one cell further comprises determining whether the first candidate object has developed into the cell colony by analyzing at least the third time-series images.

3. The method according to claim 1 or 2.

4. 4. The method of claim 3, wherein determining whether the first candidate object has developed into the cell colony comprises determining whether the third time series images show an intermediate cell colony having one or both of: (i) a smaller number of cells than the cell colony; or (ii) a smaller size than the cell colony.

5. 4. The method of claim 3, wherein determining whether the first candidate object has developed into the cell colony comprises: (i) determining that the third time series images show the intermediate cell colony; and (ii) comparing the position of the intermediate cell colony within the well with the position of the cell colony within the well and / or the position of the first candidate object within the well.

6. the first imaging unit generates the plurality of time-series images at a first magnification level; the method further comprising generating, for each of the one or more candidate objects shown in the first time series images, a zoomed-in image of a portion of the well containing the candidate object by a second imaging unit providing a second magnification level greater than the first magnification level; determining whether the candidate object is a single cell comprises determining whether the candidate object is a single cell by analyzing the zoomed-in image using the convolutional neural network. The method according to any one of claims 1 to 5.

7. The method of claim 6 , wherein generating the zoomed-in image includes shifting the well so that the well is aligned with an optical path of the second imaging unit.

8. 8. The method of claim 1, wherein determining whether the candidate object is a single cell comprises using the convolutional neural network to classify the candidate object as one of a plurality of possible object types, the plurality of object types including a type corresponding to a single cell.

9. 9. The method of claim 1, wherein detecting the cell colonies shown in the second time series of images comprises processing the second time series of images using another convolutional neural network.

10. the method comprising determining that the cell colony was formed from only one cell; the method further comprising transporting the cells from the well to a new culture environment and culturing the cells in the new culture environment. The method according to any one of claims 1 to 9.

11. the method comprising determining that the cell colony was formed from only one cell; the method further comprising using the contents of the wells to develop a cell line for producing a biopharmaceutical. The method according to any one of claims 1 to 9.

12. the method comprising determining that the cell colony was not formed from only one cell; The method further comprises discarding the contents of the well. The method according to any one of claims 1 to 11.

13. One or more non-transitory computer-readable media storing instructions that, when executed by one or more processors, cause the one or more processors to: receiving a plurality of time series images of a well containing a medium, the plurality of time series images being generated by a first imaging unit and including a first time series image generated at a first time and a second time series image generated at a second time later than the first time; detecting one or more candidate objects shown in the first time-series images by analyzing the first time-series images; for each of the one or more candidate objects, determining whether the candidate object is a single cell by analyzing an image of the candidate object using a convolutional neural network; analyzing the second time series of images to detect cell colonies shown in the second time series of images; determining whether the cell colony is formed from only one cell based at least in part on whether each of the one or more candidate objects is determined to be a single cell; generating output data indicative of whether the cell colony was formed from only one cell; One or more non-transitory computer-readable media.

14. The instructions cause the one or more processors to: upon determining that the first candidate object is a single cell, determining whether the first candidate object has developed into a cell colony by comparing the location of the first candidate object in the well with the location of the cell colony in the well; 14. One or more non-transitory computer-readable media as recited in claim 13.

15. the plurality of time-series images further includes a third time-series image generated at a third time between the first time and the second time, and the instructions cause the one or more processors to: determining whether the first candidate object has developed into the cell colony by analyzing at least the third time-series images when determining whether the cell colony is formed from only one cell; 15. One or more non-transitory computer-readable media according to claim 13 or 14.

16. the plurality of time series images generated by the first imaging unit are generated at a first magnification level; the instructions further cause the one or more processors to receive, for at least one of the one or more candidate objects, a zoomed-in image of a portion of the well containing the candidate object, the zoomed-in image being generated at a second magnification level greater than the first magnification level; the instructions cause the one or more processors to determine whether the candidate object is a single cell by analyzing the zoomed-in image using the convolutional neural network. One or more non-transitory computer-readable media according to any one of claims 13 to 15.

17. 17. The one or more non-transitory computer-readable media of claim 16, wherein the zoomed-in image is generated by a second imaging unit, and the instructions further instruct the one or more processors to instruct a stage containing the well disposed thereon to align the well with an optical path of the second imaging unit.

18. 18. The one or more non-transitory computer-readable media of any one of claims 13-17, wherein the output data either (i) indicates that the contents of the well can be used to develop a cell line for producing a biopharmaceutical, or (ii) indicates that the contents of the well should be discarded.

19. 1. A system comprising:

1. A visual inspection system comprising: a stage configured to receive a well plate; a first imaging unit configured to generate images of wells in the well plate on the stage, each image corresponding to a single well; A visual inspection system; 1. A computer system comprising: one or more processors; one or more memories storing instructions that, when executed by the one or more processors, provide the computer system with: instructing the first imaging unit to generate a plurality of time series images of a well containing a medium, the plurality of time series images including a first time series image generated at a first time and a second time series image generated at a second time later than the first time; detecting one or more candidate objects shown in the first time-series images by analyzing the first time-series images; for each of the one or more candidate objects, determining whether the candidate object is a single cell by analyzing an image of the candidate object using a convolutional neural network; analyzing the second time series of images to detect cell colonies shown in the second time series of images; determining whether the cell colony is formed from only one cell based at least in part on whether each of the one or more candidate objects is determined to be a single cell; one or more memories storing instructions for generating output data indicative of whether the cell colony was formed from only one cell; and a computer system including: A system comprising:

20. The instructions may include: upon determining that the first candidate object is a single cell, determining whether the first candidate object has developed into a cell colony by comparing the location of the first candidate object in the well with the location of the cell colony in the well; 20. The system of claim 19.

21. the plurality of time-series images further includes a third time-series image generated at a third time between the first time and the second time, and the instructions to the computer system include: determining whether the first candidate object has developed into the cell colony by analyzing at least the third time-series images when determining whether the cell colony is formed from only one cell; 21. A system according to claim 19 or 20.

22. the first imaging unit is configured to generate the plurality of time-series images at a first magnification level; the visual inspection system further comprising a second imaging unit configured to generate a zoomed-in image of a portion of the well in the well plate; the second imaging unit provides a second magnification level greater than the first magnification level; the instructions further cause the computer system to, for each of at least one of the one or more candidate objects, instruct the second imaging unit to generate a zoomed-in image of a portion of the well containing the candidate object; the instructions cause the computer system to determine whether the candidate object is a single cell by analyzing the zoomed-in image using the convolutional neural network. A system according to any one of claims 19 to 21.

23. 23. The system of claim 22, wherein the zoomed-in image is generated by a second imaging unit, and the instructions further cause the one or more processors to instruct the stage, including the well disposed thereon, to align the well with an optical path of the second imaging unit.

Images