Systems and methods for facilitating clone selection

Abstract

To provide systems and methods for improving conventional visual inspection technique used for clone selection.SOLUTION: A method comprises generating time-series images of a well including a first image and a following second image. The method also includes detecting one or more candidate objects depicted in the first image by analyzing the first image, and for each of the candidate objects, determining whether the object is a single cell by analyzing an image of the object using a convolutional neural network. The method further includes detecting a cell colony depicted in the second image by analyzing the second image, and determining whether the colony was formed from a single cell at least based on whether each candidate object was determined to be a single cell or not. The method further includes generating output data to indicate whether the colony was formed from a single cell.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS Priority is claimed to U.S. Provisional Patent Application No. 62 / 774,154, filed November 30, 2018, the entire contents of which are incorporated herein by reference.

[0002] This application relates generally to clonal selection techniques, and more specifically to techniques for determining whether a cell population is formed from a single clone / cell. [Background technology]

[0003] In various cell line development processes, clonal selection requires knowledge of whether a cell colony formed from a single clonal cell. For example, individual cells may be inoculated into separate wells (e.g., of a 96-well plate) by flow cytometry techniques such as fluorescence-activated cell sorting (FACS) techniques, and then the cells may be cultured for an appropriate period of time (e.g., 14 days). Throughout the culture period, digital images of the wells are captured at appropriate time intervals, such as daily or every 7 days. The analyst reviews the well image captured at the end of the culture period (e.g., day 14) to identify cell colonies, and if a particular well contains a colony, the analyst may also review one or more previous images of the same well (e.g., an image captured on the first day of culture) to determine whether the colony formed from only a single clone. If the analyst can confirm that the colony formed from a single clone, that particular sample may be forwarded to one or more additional stages of the cell line development process. If not, the sample may be discarded.

[0004] This process is very time-consuming and labor-intensive. For example, it is not uncommon for hundreds of man-hours to be required to analyze microscopic images for a single project. Furthermore, it can be difficult for analysts to accurately identify single cells within a well when attempting to identify progenitor cells of a colony. For example, false positives can occur when an imaged object is out of focus (i.e., incorrectly labeling the object as a single cell). As another example, doublets or "stacked" cells can be difficult to distinguish from single cells. False positives (and / or false negatives) can also result from other factors, such as debris, shadows created by the well walls, and / or anomalies on the inner circumference of the well walls. Summary of the Invention [Means for solving the problem]

[0005] The embodiments described herein relate to systems and methods that improve upon conventional visual inspection techniques used for clonal selection. In particular, in some embodiments, an automated visual inspection system captures a series of digital images for each well in a well plate at intervals over the culture period (e.g., days or weeks). Depending on the embodiment, images may be captured only on the first and last days of the culture period, daily, or on other suitable time schedules. The system may identify candidate object colonies derived from a single cell. Based on the identified candidate objects, the system may move the location of the well for further image capture. Cells from wells identified as containing colonies derived from a single cell may be transferred and cultured in a new culture environment. As used herein, a "well" refers to any laboratory-scale cell culture environment that allows for optical inspection of its contents. Although wells on a multi-well plate are discussed as examples herein, it should be understood that whenever "well" and "well plate" are referred to, unless otherwise specified, these terms are intended to encompass any suitable laboratory-scale cell culture environment that allows for optical inspection of its contents.

[0006] For a given well, computational image processing techniques can be applied to subsequent images (e.g., images captured on the final day of the culture period) to determine whether the well contains a cell colony. For example, a convolutional neural network (CNN) can be used to classify objects in the well medium as cell colonies. The CNN can include any number of convolutional layers suitable for two-dimensional convolution (e.g., to detect features such as edges in the image), pooling layers (e.g., downsampling layers to reduce computation while preserving the relative positions of features), and fully connected layers. Alternatively, one or more other image processing techniques can be used to detect the colonies, such as traditional image filtering, edge detection, and / or pattern detection techniques. If a cell colony is detected in the well, one or more previous images of the well (e.g., images from the first day of culture) can be analyzed to determine whether the colony formed from a single clone / cell. This determination can include multiple steps. First, an image of the entire well is analyzed to identify any objects in the well that are candidates to be single cells (e.g., objects that are neither too large nor too small to be single cells). The image of each candidate object is then input to a CNN (e.g., a second CNN if the first CNN was used to detect cell clusters) that classifies the candidate object according to object type. In some embodiments, the image of each candidate object is extracted from the image of the whole well and includes, or is simply composed of, a set of pixels that indicate that candidate object. In some embodiments, the image of each candidate object further includes a zoomed-in image of the candidate object, e.g., to overcome image resolution limitations associated with currently available imaging techniques. For example, a first, lower magnification imaging unit can be used to capture an image of the whole well for purposes of identifying the candidate object, while a second, higher magnification imaging unit can be used to capture a zoomed-in image of each of the identified candidate objects.Use of the second imaging unit may further include placing the well or a portion of the well in the optical path of the second imaging unit, for example by shifting the well plate a small distance in the x and / or y directions.

[0007] For a given candidate object, the CNN may output a binary classification (e.g., "single cell" vs. "not a single cell"), or may classify the object according to one of three or more types (e.g., "single cell," "doublet," "empty cell," "debris," etc.). If the candidate object is classified as a single cell (or another class corresponding to a single live cell), the cell colony may be determined to be derived from that cell. This determination may also depend on one or more other factors, such as the location of the single cell in the well compared to the location of the detected cell colony in the well. Furthermore, in some embodiments, one or more additional images of the well, corresponding to one or more other times during the culture period, may be analyzed to confirm that the cell colony was formed from a single cell. For example, the additional images may be processed to detect one or more intermediate growth stages of the cell colony.

[0008] If it cannot be determined that the cell colony was formed from a single cell, the sample in that well may be discarded. However, if the origin of a single cell can be determined (e.g., with at least some threshold level of accuracy, such as 90% or 99%), the sample may be transferred to one or more additional stages of the cell line development process. For example, the cells of the sample (e.g., as single cells and / or as part of a cell-containing sample that may also include other cells) may be transferred to a new culture environment and cultured. The cell line may be used for any of a wide variety of purposes, depending on the embodiment. For example, the cell line may be used to provide cells for producing biopharmaceutical antibodies or hybrid molecules (e.g., drugs, including bispecific T-cell engager (BiTE®) antibodies such as BLINCYTO® (blinatumomab), or monoclonal antibodies, etc.), or to provide cells for research and / or development purposes. "Accuracy" has its ordinary and customary meaning as would be understood by one of skill in the art in light of this disclosure. It refers to the probability that a result is neither false negative nor false positive. For example, accuracy can be calculated as 100%-(Type I error)-(Type II error).

[0009] Those skilled in the art will appreciate that the figures described herein are included for illustrative purposes and are not intended to limit the disclosure. The figures are not necessarily to scale, with emphasis instead being placed on illustrating the principles of the disclosure. It will be understood that in some instances, various aspects of the described embodiments may be shown exaggerated or enlarged to facilitate understanding of the described embodiments. In the drawings, identical reference numbers generally refer to functionally similar and / or structurally similar components throughout the various views. [Brief description of the drawings]

[0010] [Figure 1] FIG. 1 is a simplified block diagram of an example system in which the techniques described herein may be implemented. [Figure 2A]2 illustrates an exemplary visual inspection system that may be used in the system of FIG. 1. [Figure 2B] 2 illustrates an exemplary visual inspection system that may be used in the system of FIG. 1. [Figure 2C] 2 illustrates an exemplary visual inspection system that may be used in the system of FIG. 1. [Diagram 3] FIG. 1 is a flow diagram of an exemplary process for determining whether to use or discard a well sample. [Figure 4] 1 illustrates an example embodiment and scenario in which well images are generated by a visual inspection system at two different times. [Diagram 5] 1 illustrates an example embodiment and scenario in which well images are generated by a visual inspection system at three different times. [Figure 6] 1 illustrates an embodiment and scenario in which multiple candidate objects are identified in a well image. [Figure 7] 1 shows an exemplary zoomed-in image of a candidate object. [Figure 8] FIG. 1 is a flow diagram of an exemplary method for facilitating clonal selection. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0011] The various concepts introduced above and described in more detail below can be implemented in any of numerous ways, and the concepts described are not limited to any particular implementation manner. Example embodiments are provided for illustrative purposes.

[0012] FIG. 1 is a simplified block diagram of an exemplary system 100 in which the techniques described herein may be implemented. System 100 includes a visual inspection system 102 communicatively coupled to a computer system 104. Visual inspection system 102 includes hardware (e.g., a well plate stage, one or more lenses and / or mirrors, an imager, etc.), as well as firmware and / or software, configured to capture digital images of wells within a well plate. An exemplary embodiment of visual inspection system 102 is illustrated in FIGS. 2A-2C. Referring first to FIG. 2A, visual inspection system 102 may include a stage 202 configured to receive a well plate 204 including a plurality of wells (not shown in FIG. 2A). Well plate 204 may be of any suitable size, any suitable shape, and have any suitable number of wells (e.g., 6, 24, 96, 384, 1536, etc.) disposed thereon. Furthermore, the wells may be arranged in any suitable pattern on well plate 204, such as, for example, a 2:3 rectangular matrix.

[0013] The visual inspection system 102 further includes an illumination system 208, a first imager 210 configured to acquire a wide field of view image, and in some embodiments (for purposes further described below), a second imager 212 configured to acquire a high magnification image. In other embodiments, the visual inspection system may omit the imager 212. The illumination system 208 may include any suitable number and / or type of light source configured to generate source light, which illuminates each well in the well plate 204 when the well is placed in the optical path of the imager 210 or imager 212. The imager 210 includes a telecentric lens 220 and a wide field of view camera 222. The telecentric lens 220 may be a high fidelity telecentric lens with 1x magnification, and the wide field of view camera 222 may be, for example, a charge-coupled device (CCD) camera. The imager 210 is configured and positioned relative to the stage 202 such that it can capture images each showing a single entire well (with a resolution suitable for resolving candidate objects in the well from the rest of the well, e.g., with the single well occupying substantially all of the image or at least a majority of the image) when the wells are appropriately positioned on the stage 202 and illuminated by the illumination system 208. The imager 212 comprises an objective lens 230, a mirror 232, and a high-resolution camera 234. The objective lens 230 may be a 20x magnification, long working distance objective lens (or lens system), and the high-resolution camera 234 may be, for example, another CCD. The mirror 232 may enable the imager 212 to have a suitably low profile.

[0014] In some embodiments, each well of the well plate 204 has one or more transparent and / or opaque portions. For example, each well may be completely transparent or may have a transparent bottom with opaque sidewalls. Each well may be generally cylindrical or have other suitable shapes (e.g., cubic, etc.). The visual inspection system 102 images the wells of the well plate 204, e.g., sequentially images each well in the well plate 204. To this end, the visual inspection system 102 is configured to move the stage 202 along one or more (e.g., x and y) axes to sequentially align each well with the optical path of the illumination system 208 and the imager 210 for individual well analysis. For example, the stage 202 may be coupled to one or more motorized actuators. Once each well is aligned with the optical path of the illumination system 208 and the imager 210, the imager acquires one or more images of the illuminated well. Any cells in a given well may generally lie in a flat plane on the base of the well, in which case the well may be imaged from a top-down or bottom-up perspective. In such an embodiment, the visual inspection system 102 may also be configured to move the stage 202 in the vertical (z) direction to keep the focus on a flat, thin layer in which the cells may reside. Vertical actuation may also enable three-dimensional scanning of the well sample (e.g., to detect cells adhering to the sidewall of the well above the well base). The visual inspection system 102 may also apply any suitable techniques to mitigate vibration and / or mechanically support high fidelity imaging, both of which may be important when high magnification imaging is used.

[0015] 2B and 2C show the configuration of the visual inspection system 102 when the imager 210 and the imager 212 are each used to capture an image of a particular well (in this scenario, a well 240 located in one corner of the well plate 204). For clarity, some components of the visual inspection system 102 are omitted in FIGS. 2B and 2C. As seen in FIGS. 2B and 2C, the well 240 can be selectively positioned in the optical path of the imager 210 or the imager 212 by moving the stage 202 relative to the imagers 210 and 212 (for visibility, the stage 202 is not explicitly shown in FIGS. 2B and 2C, but it will be understood that the stage 202 is appropriately positioned; see, e.g., FIG. 2A). In various embodiments, this can be accomplished by moving the stage 202 or by moving the imagers 210 and 212.

[0016] It will be appreciated that Figures 2A-2C depict only one embodiment of visual inspection system 102, and other embodiments are possible. For example, visual inspection system 102 may include additional imagers similar to imagers 210 and / or 212 (e.g., for three-dimensional imaging). As a further example, visual inspection system 102 may instead illuminate the sample using an oblique illumination source rather than a backlight, or may instead be configured to utilize fluorescent imaging or phase-contrast based imaging. Additionally, although not shown in Figures 2A-2C, visual inspection system 102 may include one or more communication interfaces and processors to enable communication with computer system 104 and provide local control of the operation of stage 202, illumination system 208, and / or imagers 210 and 212 (e.g., in response to commands received from computer system 104).

[0017] 1, computer system 104 is generally configured to control / automate the operation of visual inspection system 102 and to receive and process images captured / generated by visual inspection system 102, as described further below. Computer system 104 is also coupled to training server 106 via network 108. Network 108 may be a single communication network or may include one or more types of multiple communication networks (e.g., one or more wired and / or wireless local area networks (LANs) and / or one or more wired and / or wireless wide area networks (WANs), such as the Internet). As discussed further below, training server 106 is generally configured to train one or more machine learning (ML) models 109, which training server 106 transmits to computer system 104 via network 108 to enable computer system 104 to detect and / or classify objects in images generated by visual inspection system 102. In various embodiments, the training server 106 may provide the ML models 109 as a "cloud" service (e.g., Amazon Web Services), or the training server 106 may be a local server. In alternative embodiments, the ML models 109 are transferred to the computer system 104 by techniques other than remote download (e.g., by physically transferring a portable storage device to the computer system 104), in which case the system 100 may not include the network 108. In other embodiments, one, some, or all of the ML models 109 may be trained on the computer system 104 and then uploaded to the server 106. In other embodiments, the computer system 104 itself performs the model training, in which case the system 100 may omit both the network 108 and the training server 106.In yet other embodiments, some or all of the components of computer system 104 shown in FIG. 1 (e.g., one, some, or all of modules 120-126) are instead included in visual inspection system 102, in which case visual inspection system 102 can communicate directly with training server 106 over network 108.

[0018] The computer system 104 may be a general-purpose computer specially programmed to perform the operations discussed herein, or may be a dedicated computing device. As can be seen in FIG. 1, the computer system 104 comprises a processing unit 110, a network interface 112, and a memory unit 114. However, in some embodiments, the computer system 104 includes two or more computers that are co-located with each other or remote from each other. In these distributed embodiments, the operations described herein related to the processing unit 110, the network interface 112, and / or the memory unit 114 may be divided among multiple processing units, network interfaces, and / or memory units, respectively.

[0019] The processing unit 110 includes one or more processors, each of which may be a programmable microprocessor that executes software instructions stored in the memory unit 114 to perform some or all of the functionality of the computer system 104 as described herein. The processing unit 110 may include, for example, one or more graphics processing units (GPUs) and / or one or more central processing units (CPUs). Alternatively, or in addition, some of the processors in the processing unit 110 may be other types of processors (e.g., application specific integrated circuits (ASICs), field programmable gate arrays (FPGAs, etc.)), and some of the functionality of the computer system 104 described herein may instead be implemented in hardware. The network interface 112 may include any suitable hardware (e.g., front-end transmitter and receiver hardware), firmware, and / or software configured to communicate with the training server 106 over the network 108 using one or more communication protocols. For example, the network interface 112 may be or may include an Ethernet interface, enabling the computer system 104 to communicate with the training server 106 over the Internet, an intranet, or the like. The memory unit 114 may include one or more volatile and / or non-volatile memories, including one or more of any suitable memory type, such as read-only memory (ROM), random access memory (RAM), flash memory, solid-state drive (SSD), hard disk drive (HDD), etc. Collectively, the memory unit 114 may store one or more software applications, data received / used by those applications, and data output / generated by those applications.

[0020] Memory unit 114 stores software instructions for a clone selection application 118 that, when executed by processing unit 110, identifies wells / samples in which cell populations developed from a single clone. Various modules of application 118 are discussed below, although it will be understood that the modules may be distributed among different software applications and / or the functionality of any one of such modules may be divided among different software applications.

[0021] A visual inspection system (VIS) control module 120 of application 118 controls / automates the operation of visual inspection system 102 via commands or other messages so that images of samples in the wells of well plate 204 can be generated with little or no human intervention. Visual inspection system 102 may transmit captured images to computer system 104 for storage in memory unit 114, or another suitable memory not shown in FIG.

[0022] The cell colony detection module 122 of the application 118 attempts to identify / detect any cell colonies shown in the well images received from the visual inspection system 102. The cell colony detection module 122 may detect cell colonies using, for example, a machine learning model such as a convolutional neural network (CNN) or using a non-machine learning algorithm (visual analytics software) that detects colonies without the need for training. The operation of the cell colony detection module 122 is discussed in more detail herein.

[0023] The object detection module 124 of the application 118 attempts to identify / detect any individual objects that (1) appear in the well images received from the visual inspection system 102 and (2) potentially may be single cells. However, due to limitations in the models or algorithms employed by the object detection module 124 and / or due to limitations in the resolution of the images processed by the object detection module 124, the object detection module 124 may not be able to ascertain with sufficient accuracy or confidence whether each detected object is in fact a single cell. Thus, each object detected by the object detection module 124 is initially viewed only as a "candidate" for being a single cell. The object detection module 124 may detect objects using relatively simple machine learning models or using non-machine learning algorithms (visual analytics software) that detect objects without the need for training. In some embodiments, the object detection module 124 utilizes OpenCV to process images and detect objects therein. In some embodiments, the object detection module 124 outputs only objects above a minimum threshold size (e.g., a threshold pixel width or number of pixels) and / or below a maximum threshold size to avoid identifying objects that may not be single cells. For example, a minimum threshold size may filter out dead pixels and very small contaminants in an image, a maximum threshold size may filter out very large contaminants or bubbles, etc. The operation of the object detection module 124 is discussed in more detail herein.

[0024] The object classification module 126 of the application 118 attempts to classify individual objects detected by the object detection module 124. In some embodiments, the object classification module 126 processes subsets of the same well images processed by the object detection module 124. For example, the object detection module 124 can accept subsets of image pixels as input, with each subset corresponding to a single candidate object (possibly including pixels corresponding to a small region surrounding the candidate object). In such an embodiment, the visual inspection system 102 may not include the imager 212. Alternatively, the VIS control module 120 may cause the visual inspection system 102 to capture additional, higher magnification images of each candidate object using the imager 212 and then input those zoomed-in images to the object classification module 126. In either case, the object classification module 126 may use a CNN to classify the objects. In some embodiments, the CNN requires significantly more processing power and / or processing time than the models or algorithms applied by the object detection module 124. The operation of the object classification module 126, and embodiments in which higher magnification images are captured and processed, are described in further detail herein.

[0025] The operation of the system 100, according to some embodiments, will now be described with reference to FIG. 1 and FIG. 2A-2C. Initially, the training server 106 trains the ML model 109 using data stored in the training database 130. The ML model 109 may include, for example, a first CNN implemented by the cell colony detection module 122 and a second CNN implemented by the object classification module 126. The training database 130 may include a single database stored in a single memory (e.g., HDD, SSD, etc.), a single database stored across multiple memories, or multiple databases stored in one or more memories. For each different model in the ML model 109, the training database 130 may store a corresponding set of training data (e.g., input / feature data and corresponding labels). To train the CNN implemented by the object classification module 126, for example, the training database 130 may include multiple input datasets, each corresponding to an image of an object, along with a label indicating the correct object classification of that image. To improve classification accuracy, the training data may include images containing a wide variety of object types (e.g., single cells, doublets, different types of debris, different types of wall anomalies, etc.) and / or images captured under a wide variety of conditions (e.g., lighting that produces shadows on the walls of various types of wells, etc.). The labels may represent classifications of the training images provided by human analysts of the corresponding images. In some embodiments, the training server 106 uses additional labeled datasets in the training database 130 to validate the generated ML models 109 (e.g., to confirm that a given one of the ML models 109 provides at least some minimum acceptable accuracy). The training server 106 then provides the ML models 109 to the computer system 104 (e.g., via remote download over the network 108). In some embodiments, the training server 106 also continuously updates / improves one or more of the ML models 109.For example, after the ML model 109 is initially trained to provide a sufficient level of accuracy, the visual inspection system 102 or computer system 104 may provide additional images to the training server 106 over time, and the training server 106 may use supervised or unsupervised learning techniques to further improve the accuracy of the model.

[0026] Wells (e.g., wells 240 in FIGS. 2B and 2C) in the well plate 204 of the visual inspection system 102 are at least partially filled, either automatically or manually, with media containing nutrients (e.g., amino acids, vitamins, etc.), growth factors, and / or other components appropriate for the cells. For example, each well may be at least partially filled. In some embodiments and / or scenarios, an attempt is made to inoculate each well with one and only one clone / cell. For example, a flow cytometry technique, such as fluorescence activated cell sorting (FACS) technique, may be used to inoculate each well with a single clone.

[0027] The well plate 204 is then loaded onto the stage 202, and the VIS control module 120 causes the visual inspection system 102 to move the stage 202 in small increments (e.g., in the x and / or y directions) and activate the imager 210 (and possibly the illumination system 208) in a synchronized manner, causing the imager 210 to capture at least one image for each well. This initial image of each well may be captured immediately after inoculation during the first day of culture and may show the entire area of ​​each well (e.g., from a bottom-up view). The visual inspection system 102 may store each well image locally, or may immediately transfer each image to the computer system 104 (e.g., for storage in the memory unit 114).

[0028] The process of imaging the wells may be repeated at regular or irregular intervals, depending on the embodiment. For example, the VIS control module 120 may cause the visual inspection system 102 to image each one of the wells once a day, once a week, etc., over some given culture period (e.g., 10 days, 14 days, etc.). Alternatively, the wells may only be imaged at the beginning and end of the culture period (e.g., days 1 and 14 of a 14-day culture period). Once the well images are generated, or in a batch after a subset (or all) of the images have been generated, the visual inspection system 102 transmits the images to the computer system 104 for automated analysis. As with the process of capturing well images, the process of transferring images to the computer system 104 may be automated (e.g., triggered by a command from the VIS control module 120).

[0029] In general, application 118 attempts to detect cell colonies in images of the wells, and then, for each detected colony, identifies whether the colony was formed from only a single clone / cell. To this end, cell colony detection module 122 (e.g., using a CNN that is one copy of ML model 109) may analyze each of the well images from the end of the culture period. For each colony that colony detection module 122 detects in a given well, object detection module 124 may analyze a first day image of that same well to detect any objects therein. However, object detection module 124 may not be able to accurately distinguish a single cell from debris, well wall anomalies, and / or other objects, either generally or in the presence of certain conditions (e.g., shadows from the well walls). Thus, for each "candidate" object identified by object detection module 124, object classification module 126 instead attempts to classify the object. As noted above, object classification module 126 may use any suitable classification scheme, so long as the classification reveals whether the depicted object is a single cell (or a single live cell). In some embodiments, the object classification module 126 accomplishes its classification task by using a CNN (a copy of the ML model 109) to process certain portions of the well image that have already been analyzed by the object detection module 124. For example, the object classification module 126 can apply just those pixels that correspond to a candidate object (and possibly a small surrounding region) as input to the CNN, and the CNN output is the predicted class / type for that candidate object.

[0030] However, in other embodiments, the resolution of the images analyzed by object detection module 124 is simply not sufficient to provide high accuracy classification of candidate objects. Thus, in these embodiments, VIS control module 120 may cause visual inspection system 102 to capture one or more additional images of at least some of the wells immediately after inoculation on the first day of culture. For example, for a given one of the wells, visual inspection system 102 may send the first "day 1" image to computer system 104 when that image is captured, and object detection module 124 may analyze the image for candidate objects immediately thereafter. For each candidate object detected in the image / well, VIS control module 120 may cause visual inspection system 102 to shift stage 202 by a small amount so that the candidate object is in the optical path of higher magnification imager 212, for example, approximately centered in this optical path. To appropriately adjust the position of stage 202, the position of each candidate object in the well may be determined based on the image of the entire well (i.e., the "day 1" image processed by object detection module 124). The visual inspection system 102 transmits each higher magnification image to the computer system 104 where the object classification module 126 processes / analyzes the object images using CNN to classify the indicated candidate objects.

[0031] The CNN used by the object classification module 126 (and possibly the CNN used by the cell colony detection module 122) may include any suitable number of convolutional layers for 2D convolution (e.g., to detect features such as edges in an image), any suitable number of pooling layers (e.g., downsampling layers to reduce computation while preserving the relative location of features), and any suitable number of fully connected layers (e.g., to provide high-level inference based on features). Alternatively (e.g., if the visual inspection system 102 implements 3D imaging techniques), the CNN of the object classification module 126 may utilize 3D convolution to detect features in 3 dimensions. Regardless of whether a 2D or 3D CNN is used, the CNN may also provide a probability for each prediction / classification indicating the likelihood that the prediction / classification is accurate. In some embodiments, the object classification module 126 defaults to a particular classification (e.g., “unknown” or “not a single cell”) if an object in a given image cannot be classified with at least a threshold probability / confidence level.

[0032] The application 118 can determine whether a given well sample should be discarded or proceed to the next stage of cell line development based on whether the cell colony in the sample / well formed from a single clone. However, depending on the embodiment, determining whether a colony formed from a single clone may include more than detecting a single cell in the sample / well. For example, the application 118 can conclude that a colony formed from a single clone only if both (1) the object classification module 126 classifies a candidate object in the well as a single cell, and (2) the location of that single cell in the well overlaps (or is within a threshold distance, etc.) with the location of the colony in the well. Various other ways in which a determination may be made are discussed herein. It will be appreciated that a conclusion that a cell colony did not form / develop from a single cell may mean that there is positive visual evidence that the colony formed from two or more cells, or it may mean that there is simply not enough visual evidence to confidently determine how the colony formed.

[0033] The application 118 may cause a user interface to display an indication of whether a given well sample should be discarded or proceed to the next development stage, and / or may communicate with another application and / or computer system to trigger, for example, automatic discard or a cell line development stage. Cell line development may be for any suitable purpose, depending on the embodiment and / or scenario. For example, the cell line may be used to develop biopharmaceutical antibodies or hybrid molecules (e.g., bispecific T-cell engager (BiTE®) antibodies such as BLINCYTO® (blinatumomab), or monoclonal antibodies, etc.), or for research and / or development purposes.

[0034] FIG. 3 is a flow diagram of an exemplary process 300 for determining whether to use or discard a sample in a well. Process 300 may be performed, for example, by system 100 of FIG. 1 and may (in some embodiments) be fully automated. In exemplary process 300, in a preliminary stage 302, individual wells (e.g., wells in well plate 204) are inoculated with clones (cells) using FACS subcloning. During an initial stage 304 of the culture period (referred to as "Day 1" for brevity, but it will be understood that initial stage 304 may occur during other periods immediately following inoculation), at least one image of each well is generated (e.g., by imager 210 of visual inspection system 102 as controlled by VIS control module 120). The "Day 1" image may be captured at the start of the first culture day (e.g., immediately after inoculating the wells).

[0035] In a subsequent stage 306, at least one additional image of each well is generated (e.g., by imager 210) at approximately the midpoint of the culture period (referred to as "day 7" for simplicity, but it will be understood that the approximate midpoint may occur within other time periods, e.g., 5-7 days, 5-8 days, 5-9 days, 6-7 days, 6-8 days, 6-9 days, 7-8 days, or 7-9 days). In some embodiments, stage 306 may occur earlier or later, stage 306 may be omitted, or process 300 may include additional stages in which well images are generated (e.g., once a day, or every other day, etc.). Referring to stage 308, at the end of the culture period, yet another image (or another set of images) of each well is generated (e.g., by imager 210) (referred to as "day 14" for simplicity, but it will be understood that the end of the culture period may be a different period). In some embodiments, the culture period is longer or shorter than 14 days.

[0036] In FIG. 3, stages 310, 312, and 314 represent operations that occur for each sample / well. In stage 310, images from at least stages 304 and 308 (e.g., “Day 1” and “Day 14”) are analyzed to determine whether a cell colony is present in the well, and if so, whether the colony is formed from a single clone. Stage 310 may include operations of cell colony detection module 122, object detection module 124, and object classification module 126, for example, as described above. Stage 310 may occur entirely after stage 308, or may occur at other times during process 300. As described above in connection with FIG. 1, for example, at an early stage 304 (e.g., immediately after inoculation, such as on the first day of culture), it may be necessary to analyze at least some well images. Thus, stage 310 may include object detection that occurs in parallel with stage 304, which may further include acquiring additional zoomed-in images (e.g., using imager 212) of each detected candidate object. In these embodiments, analysis of each candidate object (e.g., by object classification module 126) may occur in parallel with (or immediately after) stage 304, or may occur later, such as at stage 308. In some embodiments, stage 310 also includes analysis of the "day 7" image generated in stage 306. For example, stage 310 may determine whether a particular cell colony was formed from a single clone not only by identifying a single clone within a well, but also by analyzing the "day 7" image of the well to detect colonies at intermediate growth stages.

[0037] If it is not determined at stage 310 that a particular cell colony formed from a single clone (e.g., it cannot be determined with sufficient confidence that the colony formed from a single clone), flow proceeds to stage 312, where the corresponding sample is discarded. Conversely, if it is determined at stage 314 that the colony formed from a single clone, flow proceeds to stage 314, where the corresponding sample is forwarded to the next stage of the cell line development process. For example, the cells of the sample (e.g., as part of a sample containing cells) may be transported and cultured in a new culture environment. Information regarding cell culture can be found, for example, in Green and Sambrook, “Molecular Cloning: A Laboratory Manual” (4th edition) Cold Spring Harbor Laboratory Press 2012, which is incorporated herein by reference in its entirety.

[0038] 4 illustrates an exemplary scenario in which images 400, 402 of a well 410 (e.g., one of the wells in a well plate 204) are generated by a visual inspection system (e.g., visual inspection system 102) at two different times. Specifically, a first image 400 of the well 410 is captured immediately after inoculation of the well 410 (e.g., at stage 304 of process 300), while a second image 402 of the well 410 is captured at the end of the incubation period (e.g., at stage 308 of process 300). Images 400 and 402 may be captured, for example, by imager 210. Each of images 400, 402 represents a bottom-up perspective of well 410 and shows the entire contents of well 410. It will be appreciated that in some embodiments, well images 400, 402 also include some area outside the perimeter of well 410 (e.g., if the images are rectangular).

[0039] 1, the cell colony detection module 122 of the application 118 analyzes the image 402 (e.g., using CNN) at or after the end of the culture period (e.g., at stage 308 of the process 300) to detect cell colonies 412 in the well 410. In some embodiments, in response to detecting the cell colonies 412, the application 118 may analyze the image 400 using the object detection module 124 (e.g., using OpenCV) to detect any candidate objects in the well 410 at the start of the culture. As seen in the example scenario of FIG. 4, the image 400 shows only a single object 414 in the well 410. Thus, the object detection module 124 outputs an indication of only the single candidate object. The application 118 may then extract image pixels corresponding to the object 414 and provide that image portion as input to an object classification module 126 (e.g., another CNN), which analyzes the image portion and classifies the object 414 (e.g., as a single cell, or doublet, debris, etc.).

[0040] In some embodiments, as discussed herein, accurate classification may include a higher magnification of the candidate object. In such embodiments, application 118 may analyze image 400 immediately after image 400 is generated (e.g., immediately after inoculation). In particular, object detection module 124 may analyze image 400 to detect object 414, and in response, VIS control module 120 may move stage 202 so that object 414 is approximately centered in the optical path of a higher magnification imager (e.g., imager 212) and cause that imager to capture a zoomed-in image of object 414 (not shown in FIG. 4) immediately after image 400 is captured. Object classification module 126 may then analyze the zoomed-in image to classify object 414. Thereafter, at or after the end of the culture period (e.g., at stage 308 of process 300, such as after day 14), cell colony detection module 122 may analyze image 402 to detect cell colonies 412.

[0041] Regardless of when the cell colony 412 is detected and the object 414 is classified, the classification (and possibly the location of the colony 412 and object 414 within the well 410) can be used to determine whether the colony 412 was formed from a single clone. In an alternative scenario in which the object detection module 124 detects multiple candidate objects in the image 400, the object classification module 126 can also analyze the images of the remaining candidate objects to determine whether any one or more of those objects are single cells.

[0042] If the object in the image 400 is not classified as a single cell, the application 118 may conclude that the cell group 412 was not formed from a single clone. Conversely, if the object 414 (or another object in the image 400) is classified as a single cell, the application 118 may determine that the cell group 412 was formed from a single clone. However, in some embodiments, this latter determination may also be based on one or more other factors. For example, the application 118 may determine that the cell group 412 was formed from a single clone only if the location of the object 414 in the well 410 overlaps with the location of the cell group 412 in the well 410. As another example, the application 118 may determine that the cell group 412 was formed from a single clone only if (1) the location of the object 414 in the well 410 overlaps with the location of the cell group 412 in the well 410, and (2) no other candidate objects in the image 400 are classified as cells (e.g., as single cells or doublets).

[0043] In some embodiments, one or more intermediate well images are used to determine or confirm that a cell colony has developed from a single clone. One such embodiment discussed herein with reference to FIG. 5 illustrates an exemplary embodiment and scenario in which images 500, 502, 504 of a well 510 are generated by a visual inspection system (e.g., visual inspection system 102) at three different times. Specifically, a first image 500 of a well 510 is captured immediately after inoculation of the well 510 (e.g., at stage 304 of process 300), a second image 502 of the well 510 is captured near the midpoint of the culture period (e.g., at stage 306 of process 300), and a third image 504 of the well 510 is captured at the end of the culture period (e.g., at stage 308 of process 300). Images 500, 502, and 504 may be captured, for example, by imager 210. Each of the images 500, 502, 504 represents a bottom-up view of the well 510 and shows the entire contents of the well 510. It will be appreciated that, similar to images 400, 402 of FIG. 4, in some embodiments, images 500, 502, 504 may also include some area outside the perimeter of the well 510.

[0044] 1, the cell colony detection module 122 of the application 118 analyzes the image 504 (e.g., using CNN) at or after the end of the culture period (e.g., at stage 308 of the process 300) to detect cell colonies 520 within the well 510. In some embodiments, in response to detecting the cell colonies 520, the application 118 may analyze the image 500 using the object detection module 124 (e.g., using OpenCV) to detect any candidate objects within the well 510 at the start of the culture. As seen in the exemplary scenario of FIG. 5, the image 500 shows only a single object 522 within the well 510. Thus, the object detection module 124 outputs an indication of only the single candidate object. The application 118 may then extract the image pixels corresponding to the object 522 and provide that image portion as input to an object classification module 126 (e.g., another CNN), which analyzes the image portion and classifies the object 522 (e.g., as a single cell, or doublet, debris, etc.).

[0045] In some embodiments, as discussed herein, accurate classification may include a higher magnification of the candidate object. In such embodiments, application 118 may analyze image 500 immediately after image 500 is generated (e.g., immediately after inoculation, such as stage 304 of process 300). In particular, object detection module 124 may analyze image 500 to detect object 522, and in response, VIS control module 120 may move stage 202 so that object 522 is at least approximately centered in the optical path of a higher magnification imager (e.g., imager 212) and cause that imager to capture a zoomed-in image of object 522 (not shown in FIG. 4). Object classification module 126 may then analyze the zoomed-in image to classify object 522. Thereafter, at or after the end of the culture period (e.g., stage 308 of process 300), cell colony detection module 122 may analyze image 504 to detect cell colonies 520.

[0046] Regardless of when the cell colony 520 is detected and the object 522 is classified, the classification (and possibly the location of the colony 520 and the object 522 within the well 510) can be used to determine whether the colony 520 was formed from a single clone. As described above, the presence of other candidate objects in the image 500 and their classification by the object classification module 126 may also affect that determination. In addition, in this embodiment, the application 118 analyzes the image 502. For example, the cell colony detection module 122 may analyze the image 502 (e.g., using the same CNN used to analyze the image 504) to detect the colony 524. The application 118 may then compare the location of the colony 524 to the location of the object 522 and / or the location of the cell colony 520, and / or compare the size of the colony 524 (e.g., the pixel width or total number of pixels corresponding to the colony 524 in the image 502, etc.) to the size of the colony 520 (e.g., the pixel width or total number of pixels corresponding to the colony 520 in the image 504, etc.). In one exemplary embodiment, application 118 determines that cell colony 520 was formed from a single clone only if (1) the location of colony 524 in well 510 overlaps with the location of colony 520 in well 510 and (2) the size of colony 524 is smaller than the size of colony 520.

[0047] 4 and 5 are provided for illustrative purposes, and it will be appreciated that other numbers of images (e.g., 4, 5, 10, etc.) can be analyzed and / or images from different days within the culture period can be analyzed to determine whether a cell colony formed from a single clone. Additionally, algorithms other than those described above may alternatively or additionally be used to determine whether a cell colony formed from a single clone. For example, a series of other image processing functions (e.g., intensity thresholding, blob analysis, and subsequent binary morphology operators) may alternatively or additionally be used to determine whether a cell colony formed from a single clone.

[0048] 6 and 7 correspond to an example embodiment and scenario in which multiple candidate objects are detected and then zoomed-in images of the objects are obtained for more accurate classification. Referring first to FIG. 6, an image 600 (which may be the entire image or only a portion of the image) shows a well 610 (e.g., one of the wells in the well plate 204). The image 600 may be captured, for example, by the imager 210. In the image 600, the object detection module 124 has detected a first candidate object 612 and a second candidate object 614. The rectangles surrounding the objects 612, 614 simply indicate that the objects have been detected and may or may not actually be generated or displayed on any user interface.

[0049] Because application 118, in this particular example, is unable to accurately classify objects 612, 614 based on image 600 (or at least is unable to accurately classify similarly sized objects on a sufficiently consistent basis), VIS control module 120 may be configured to cause visual inspection system 102 (e.g., using imager 212) to capture zoomed-in images of objects 612, 614. FIG. 7 illustrates various exemplary zoomed-in images 700, 702, 704, 706, any one of which may correspond to, for example, one of candidate objects 612, 614. Specifically, image 700 is a high-magnification image of a single cell 710, image 702 is a high-magnification image of an anomaly 720 in a well wall 722, image 704 is a high-magnification image of a doublet 730, and image 706 is a high-magnification image of debris 740. The object classification module 126 may use a CNN trained to classify objects according to all of these classes, some of these classes (e.g., "single cell" vs. "other"), and / or one or more other classes (e.g., "empty cell", "bubble", etc.).

[0050] 8 is a flow diagram of an exemplary method 800 for facilitating clonal selection (e.g., for a cell line development process). Method 800 may be performed by one or more portions of system 100 (e.g., visual inspection system 102 and computer system 104), or another suitable system. As a more specific example, block 802 of method 800 may be performed by visual inspection system 102 of FIGS. 1 and 2, while blocks 804-812 may be performed by computer system 104 (e.g., by processing unit 110 in executing instructions stored in memory unit 114).

[0051] At block 802 of method 800, a plurality of time series images of a well containing culture medium are generated by a first imaging unit (e.g., by imager 210). The culture medium may include cell nutrients, growth factors, etc., and has been previously inoculated (e.g., using a flow cytometry technique such as a FACS technique). The generated images include at least a first time series image generated at a first time (e.g., immediately after inoculation, such as during the first day of the culture period), and a second time series image generated at a second subsequent time (e.g., the last day of the culture period). The plurality of images may also include one or more additional time series images generated at a time between the first time series image and the second time series image.

[0052] At block 804, the first time series images are analyzed (e.g., using OpenCV and / or other suitable object detection and / or segmentation software) to detect one or more candidate objects shown in the first time series images. Block 804 may include detecting only objects that are within the shown well and that are below a certain maximum size (e.g., number of pixels). Block 804 may or may not include segmenting the first time series images to identify which image pixels correspond to the objects, depending on the embodiment.

[0053] In block 806, for each candidate object detected in block 804, an image of the candidate object is analyzed using a CNN to determine whether the candidate object is a single cell. If segmentation of the candidate object was performed in block 806, for example, block 806 may include analyzing a set of pixels corresponding to each candidate object. Alternatively, method 800 may further include, for each candidate object detected in block 804, generating a zoomed-in image of a portion of a well containing the candidate object, in which case block 806 may include analyzing each zoomed-in image using a CNN. The zoomed-in image may be generated using a second, higher magnification imaging unit, such as imager 212. Generating the zoomed-in image may include shifting the well (e.g., by moving a stage on which a well plate containing the well is placed) so that the well is aligned, for example, with an optical path of the second imaging unit.

[0054] Regardless of the type of image being analyzed (e.g., a segmented image portion or a new zoomed-in image), the CNN used in block 804 may be configured / trained to classify objects according to different possible object types, including at least one type corresponding to a single cell (or single living cell, etc.). For example, the CNN may classify a given object as one of "single cell," "doublet," and "debris," or as one of "single cell" and "other."

[0055] In some embodiments, blocks 804 and 806 occur after block 802 is completed. In other embodiments, blocks 804 and 806 occur before the second time series image is generated in block 802. For example, blocks 804 and 806 may occur almost immediately after the first time series image is generated in block 802.

[0056] In block 808, the second time series images are analyzed to detect cell clusters shown therein. The second time series images may be analyzed, for example, using a CNN (e.g., a different CNN than that used in block 806) or using one or more other image processing functions (e.g., intensity thresholding, blob analysis, binary morphology operators, etc.). In various embodiments, block 808 may occur several days after blocks 804 and 806 (e.g., if blocks 804 and 806 occurred immediately after the first time series images were generated) or immediately after blocks 804 and 806 (e.g., if all image analysis occurs after block 802 is completed). However, in other embodiments, blocks 804 and 806 occur after (e.g., in response to) the determination made in block 808. That is, the system implementing method 800 may attempt to detect a single cell in a well if, and only if, a cell cluster has already been detected in the well.

[0057] At block 810, based at least in part on the determination made at block 806 (i.e., based on whether each of the at least one or more candidate objects is a single cell), it is determined whether the cell colony detected at block 808 was formed from only one cell. For example, at block 810, it may be determined that the colony was not formed from a single cell if the candidate object includes two or more cells, or if there are no objects such as single living cells among the candidate objects. It will be appreciated that a conclusion that the colony was not formed from a single cell may mean that there is positive visual evidence that the colony was formed from two or more cells, or that there is simply not enough visual evidence to determine how the colony was formed.

[0058] In some embodiments, the determination in block 810 is also based on one or more other factors. If the first candidate object in the first time series image is determined to be a single cell in block 806, for example, block 810 may include comparing the location of the candidate object in the well to the location of a cell colony in the well. Additionally or alternatively, if a third time series image is generated at a time between the first time series image and the second time series image, and if the first candidate object in the first time series image is determined to be a single cell in block 806, block 810 may include analyzing the third time series image to determine whether the candidate object has developed into a cell colony. For example, an intermediate colony may be detected in the third time series image, and the cell number, size, and / or location of the intermediate colony may be analyzed and compared to the cell number, size, and / or location of the cell colony detected in the second time series image.

[0059] At block 812, output data is generated. The output data indicates whether the cell colony formed from only one cell according to the determination made at block 810. The output data may be displayed to a user on a user interface of the computing device and / or transmitted to one or more other software modules and / or computer systems, for example. In some embodiments, the output data informs a user, or one or more software modules or systems, that the well containing the cell colony should be selected for one or more additional stages of cell line development (if the colony was determined to have formed from only one cell at block 810) or that the contents of the well should be discarded (if the colony was not determined to have formed from only one cell at block 810). In some embodiments, method 800 further includes using the contents of the well to develop the cell line (e.g., for biopharmaceutical manufacturing, for research and development, etc.), for example, by transferring the cells from the well to a new culture environment and culturing the cells in the new culture environment, or further includes discarding the contents of the well based on the output data.

[0060] Although the systems, methods, devices, and components thereof have been described with respect to exemplary embodiments, they are not limited thereto. The detailed description is to be construed as an example only, and does not describe all possible embodiments of the invention, since describing all possible embodiments would be impractical, if not impossible. Many alternative embodiments can be implemented using either current technology or technology developed after the filing date of this patent, and such embodiments still fall within the scope of the claims that define the invention.

[0061] Those skilled in the art will understand that numerous modifications, alterations, and combinations can be made to the above-described embodiments without departing from the scope of the present invention, and such modifications, alterations, and combinations are to be construed as falling within the scope of the inventive concept.

Claims

1. 1. A method for facilitating clonal selection, comprising: generating, by a first imaging unit, a plurality of time series images of a well containing a medium, the plurality of time series images including a first time series image generated at a first time and a second time series image generated at a second time after the first time; detecting, by one or more processors, one or more candidate objects shown in the first time-series images by analyzing the first time-series images; for each of the one or more candidate objects, determining, by the one or more processors, whether the candidate object is a single cell by analyzing an image of the candidate object using a convolutional neural network; detecting cell populations shown in the second time series images by analyzing the second time series images with the one or more processors; determining whether the cell population is formed from only one cell based at least in part on whether each of the one or more candidate objects is determined to be a single cell by the one or more processors; generating, by the one or more processors, output data indicative of whether the cell colony is formed from only one cell; A method comprising:

2. the method including determining, using the convolutional neural network, that a first candidate object of the one or more candidate objects is a single cell; determining whether the cell colony is formed from only one cell further comprises comparing a position of the first candidate object in the well with a position of the cell colony in the well. The method of claim 1.

3. the plurality of time-series images further includes a third time-series image generated at a third time between the first time and the second time, the method including determining, using the convolutional neural network, that a first candidate object of the one or more candidate objects is a single cell; determining whether the cell colony is formed from only one cell further comprises determining whether the first candidate object has developed into the cell colony by analyzing at least the third time series images. The method according to claim 1 or 2.

4. 4. The method of claim 3, wherein determining whether the first candidate object has developed into the cell colony comprises determining whether the third time series images show an intermediate cell colony having one or both of: (i) a smaller number of cells than the cell colony; or (ii) a smaller size than the cell colony.

5. 4. The method of claim 3, wherein determining whether the first candidate object has developed into the cell community comprises: (i) determining that the third time series images show the intermediate cell community; and (ii) comparing a position of the intermediate cell community within the well with a position of the cell community within the well and / or a position of the first candidate object within the well.

6. the first imaging unit generating the plurality of time series images at a first magnification level; the method further comprising generating, for each of the one or more candidate objects shown in the first time series images, a zoomed-in image of a portion of the well including the candidate object by a second imaging unit providing a second magnification level greater than the first magnification level; determining whether the candidate object is a single cell includes determining whether the candidate object is a single cell by analyzing the zoomed-in image using the convolutional neural network. The method according to any one of claims 1 to 5.

7. The method of claim 6 , wherein generating the zoomed-in image includes shifting the well so that the well is aligned with an optical path of the second imaging unit.

8. 8. The method of claim 1, wherein determining whether the candidate object is a single cell comprises using the convolutional neural network to classify the candidate object as one of a plurality of possible object types, the plurality of object types including a type corresponding to a single cell.

9. 9. The method of claim 1, wherein detecting the cell populations shown in the second time series images comprises processing the second time series images using another convolutional neural network.

10. the method comprising determining that the cell colony is formed from only one cell; The method further comprises transporting the cells from the well to a new culture environment and culturing the cells in the new culture environment. The method according to any one of claims 1 to 9.

11. the method comprising determining that the cell colony is formed from only one cell; The method further comprises using the contents of the wells to develop a cell line for producing a biopharmaceutical. The method according to any one of claims 1 to 9.

12. the method comprising determining that the cell colony was not formed from only one cell; The method further comprises discarding the contents of the well. The method according to any one of claims 1 to 11.

13. One or more non-transitory computer-readable media storing instructions that, when executed by one or more processors, cause the one or more processors to: receiving a plurality of time series images of a well containing a medium, the plurality of time series images being generated by a first imaging unit and including a first time series image generated at a first time and a second time series image generated at a second time after the first time; detecting one or more candidate objects shown in the first time-series images by analyzing the first time-series images; for each of the one or more candidate objects, determining whether the candidate object is a single cell by analyzing an image of the candidate object using a convolutional neural network; detecting a cell colony shown in the second time series of images by analyzing the second time series of images; determining whether the cell population is formed from only one cell based at least in part on whether each of the one or more candidate objects is determined to be a single cell; generating output data indicative of whether the cell colony was formed from only one cell; One or more non-transitory computer-readable media.

14. The instructions cause the one or more processors to: upon determining that a first candidate object is a single cell, determining whether the first candidate object has developed into a cell colony by comparing a position of the first candidate object in the well with a position of the cell colony in the well.

14. One or more non-transitory computer-readable media as recited in claim 13.

15. the plurality of time series images further includes a third time series image generated at a third time between the first time and the second time, and the instructions cause the one or more processors to: determining whether the first candidate object has developed into the cell colony by analyzing at least the third time series images when determining whether the cell colony is formed from only one cell; 15. One or more non-transitory computer-readable media according to claim 13 or 14.

16. the plurality of time series images generated by the first imaging unit are generated at a first magnification level; The instructions further cause the one or more processors to receive, for at least one of the one or more candidate objects, a zoomed-in image of a portion of the well containing the candidate object, the zoomed-in image being generated at a second magnification level greater than the first magnification level; the instructions cause the one or more processors to determine whether the candidate object is a single cell by analyzing the zoomed-in image using the convolutional neural network. One or more non-transitory computer readable media according to any one of claims 13 to 15.

17. 17. The one or more non-transitory computer-readable media of claim 16, wherein the zoomed-in image is generated by a second imaging unit, and the instructions further instruct the one or more processors to instruct a stage containing the well disposed thereon to align the well with an optical path of the second imaging unit.

18. 18. The one or more non-transitory computer readable media of any one of claims 13-17, wherein the output data either (i) indicates that the contents of the well can be used to develop a cell line for producing a biopharmaceutical, or (ii) indicates that the contents of the well should be discarded.

19. 1. A system comprising:

1. A visual inspection system comprising: a stage configured to receive a well plate; a first imaging unit configured to generate images of wells in the well plate on the stage, each image corresponding to a single well; A visual inspection system; 1. A computer system comprising: one or more processors; One or more memories storing instructions that, when executed by the one or more processors, provide the computer system with: directing the first imaging unit to generate a plurality of time series images of a well containing a medium, the plurality of time series images including a first time series image generated at a first time and a second time series image generated at a second time after the first time; detecting one or more candidate objects shown in the first time-series images by analyzing the first time-series images; for each of the one or more candidate objects, determining whether the candidate object is a single cell by analyzing an image of the candidate object using a convolutional neural network; detecting a cell colony shown in the second time series of images by analyzing the second time series of images; determining whether the cell population is formed from only one cell based at least in part on whether each of the one or more candidate objects is determined to be a single cell; one or more memories storing instructions for generating output data indicative of whether the cell colony was formed from only one cell; a computer system including: A system comprising:

20. The instructions include: upon determining that a first candidate object is a single cell, determining whether the first candidate object has developed into a cell colony by comparing a position of the first candidate object in the well with a position of the cell colony in the well.

20. The system of claim 19.

21. the plurality of time-series images further includes a third time-series image generated at a third time between the first time and the second time, and the instructions further comprising: determining whether the first candidate object has developed into the cell colony by analyzing at least the third time series images when determining whether the cell colony is formed from only one cell; 21. A system according to claim 19 or 20.

22. the first imaging unit is configured to generate the plurality of time series images at a first magnification level; the visual inspection system further comprising a second imaging unit configured to generate a zoomed-in image of a portion of the well in the well plate; the second imaging unit provides a second magnification level greater than the first magnification level; the instructions further cause the computer system to, for each of at least one of the one or more candidate objects, instruct the second imaging unit to generate a zoomed-in image of a portion of the well containing the candidate object; the instructions cause the computer system to determine whether the candidate object is a single cell by analyzing the zoomed-in image using the convolutional neural network. A system according to any one of claims 19 to 21.

23. 23. The system of claim 22, wherein the zoomed-in image is generated by a second imaging unit, and the instructions further cause the one or more processors to instruct the stage, including the well disposed thereon, to align the well with an optical path of the second imaging unit.

Images