System and method for assessing cell growth rate

Abstract

A method for facilitating growth assessment of a cell line includes generating an image of a well containing a medium inoculated with at least one cell of the cell line. The method also includes generating a downsampled segmentation map including pixels indicating the estimated presence or absence of cell colonies in a corresponding portion of the well image. Generating the downsampled segmentation map includes inputting the well image to a fully convolutional neural network having multiple convolutional layers. The method also includes (i) inputting colony size information (including the downsampled segmentation map and / or pixel counts derived therefrom) into a cell growth assessment algorithm to identify a growth classification or score for the cell line and display the growth classification or score, and / or (ii) displaying the colony size information, thereby facilitating manual cell growth assessment.

Description

Technical Field

[0001] This application generally relates to clone selection techniques for cell line development, and more specifically to techniques for assessing growth / proliferation associated with specific clones.

Background Art

[0002] As the demand for higher performance cells increases, the throughput rate of the cell line development process is becoming increasingly important. However, the process is complex, and each clone / cell line has characteristics that make it unique, and it may be necessary to sort through hundreds or thousands of potential candidates to find the "best" clone for a particular application. Clone selection is typically performed by assessing various characteristics that tend to indicate how suitable each clone is for a commercial formulation. For example, it is desirable for a clone to produce high-quality protein and be robust against environmental strains. Another major characteristic is the growth profile of the clone, i.e., the growth / proliferation rate of the clone over a given time period. To assess the growth profile, individual cells of the cell line are typically inoculated into separate wells (e.g., of a 96-well plate) using a flow cytometry technique such as fluorescence-activated cell sorting (FACS), and then the cells are cultured over an appropriate period (e.g., 14 days). Throughout the culture period, digital images of the wells are captured at suitable time intervals such as daily or every few days. A professional analyst examines the well images over time to assess the growth rate of the clone.

Summary of the Invention

Problems to be Solved by the Invention

[0003] However, this manual examination / assessment is very time-consuming and cumbersome, and generally requires a lot of man-hours for the analysis of microscopic images.

Means for Solving the Problems

[0004] The embodiments described herein automate at least a portion of the visual inspection process (e.g., to standardize and increase throughput), and in particular, reduce various difficulties inherent in the automated visual inspection process (e.g., difficulties associated with counting cells in high-density cell colonies and / or difficulties associated with generating a training library, etc.), thereby improving conventional visual inspection techniques for assessing the growth / proliferation rate of clones used to inoculate one or more wells (e.g., of a 96-well plate). The automated techniques described herein can be used, for example, during the clone selection phase of cell line development. As used herein, the term "well" refers to any laboratory-scale cell culture environment that enables optical inspection of its contents. Although wells on a multi-well plate are discussed as examples herein, it should be understood that whenever the terms "well" and "well plate" are mentioned, these terms are considered to encompass any suitable laboratory-scale cell culture environment that enables optical inspection of its contents, unless otherwise specified. The terms "clone" and "cell line" are used interchangeably herein.

[0005] More specifically, the well images can be processed by a computer system to automate one or more steps that facilitate a manual growth assessment process by a user, or can automate the entire growth assessment process. In some embodiments, the well images are captured at regular or irregular intervals (e.g., daily or every two days) after inoculation of the wells, and at least some of the well images are processed by a convolutional neural network (“CNN”). For example, the CNN can be a novel modified version of the fully convolutional network (“FCN”) recently described by J. Long, E. Shelhamer, and T. Darrell in Fully Convolutional Networks for Semantic Segmentation, Computer Vision and Pattern Recognition, 2015 (“Long et al.”), the entirety of which is incorporated herein by reference. In particular, in some of these embodiments, the FCN of Long et al. is modified by omitting any transposed (or “deconvolution”) layers such that the FCN does not upsample and thus outputs a “heatmap” that is smaller than the original well image. Thus, while the FCN of Long et al. outputs a full segmentation image of the same size as the original / input image, the FCN of the present disclosure outputs a smaller downsampled segmentation map. The pixels of the downsampled segmentation map can indicate the inferred presence or absence of cell colonies in the corresponding portion of the larger well image. Thus, the FCN effectively acts as a filter that scans the well images to detect cell colonies, and the number of pixels representing cell colonies in the downsampled segmentation map (e.g., via pixel counting as described herein) functions as a rough indicator of colony size.

[0006] Compared with the full-size segmented images output by the FCN of Long et al., the downsampled segmentation maps generated by the FCN of the present disclosure generally have lower accuracy on the segment boundaries (i.e., on the cell colony boundaries in the present application), and may produce "vagrant" cells (i.e., cells displaced from the main population of cell colonies) that are not considered. These may initially appear to be a major drawback of the technique, but relatively imprecise colony boundaries / areas such as those provided by the downsampled segmentation maps can be sufficient for performing reliable growth assessments, and it has been found that the number of vagrant cells is typically small enough to be safely ignored for the purpose of cell growth assessment. Furthermore, when cell colonies grow relatively large, it has been found that the approximate relative cell colony size in well images (captured over multiple days) can be a reliable indicator for assessing growth without the need for precise cell counting. Still further, the training of the FCN of the present disclosure can take far less time and / or cost than the training of conventional FCNs such as the FCN described by Long et al. In particular, the architecture of the FCN of the present disclosure can be trained using smaller cropped image "patches" instead of full well images, thus eliminating the need to label every pixel of the full well image when constructing the training library. This can significantly reduce the amount of time required to manually label images for the training library.

[0007] In some embodiments, different techniques are used to assess cell growth / proliferation at the early stages of seeding. For example, in the first few days after seeding, cell counting techniques can be used on well images captured before the colonies grow to a high enough density to obscure a large number of cells (e.g., due to "stacking" of cells in the colonies). For example, the systems and methods disclosed herein can identify precise cell counts in one or more early-stage well images using fully convolutional regression networks (i.e., "FCRN") as described by W. Xie, J. A. Noble, and A. Zisserman in Microscopy Cell Counting and Detection with Fully Convolutional Regression Networks, Computer Methods in Biomechanics and Biomedical Engineering: Imaging & Visualization, 2018, Vol. 6, No. 3, pp. 283-292 ("Xie et al."), the entirety of which is incorporated herein by reference.

[0008] Additionally or alternatively, in some embodiments, the systems and methods disclosed herein can determine whether a given cell colony (e.g., represented in a downsampled segmentation map) is derived from one cell, and if not, display information indicating that the colony and / or well should be ignored for growth assessment purposes. Various techniques that can be used to determine whether a colony is derived from one cell are described in PCT Patent Application No. PCT / US19 / 63177, entitled "Systems and Methods for Facilitating Clone Selection," filed November 26, 2019 (PCT Patent Application International Publication No. WO 2020 / 0112723 Pamphlet), the entire disclosure of which is incorporated herein by reference.

[0009] In some embodiments, one or more outputs of the steps or algorithms described above are analyzed by a user (e.g., a scientist or process engineer) to perform a final assessment of the growth profile of the clone. Alternatively, in some embodiments, one or more outputs of the steps or algorithms described above are automatically analyzed using a higher-level algorithm to generate an overall cell growth classification or score for the clone. For example, a cell growth assessment algorithm can operate on a precise count of one or more early-stage well images and can also operate on other metrics of colony size at one or more late stages (e.g., the number of pixels classified as belonging to a colony in a downsampled segmentation map). The cell growth assessment algorithm can then output a growth score for the clone or can output a binary indicator (e.g., "good" or "bad") or other suitable growth classification for the clone. The cell growth assessment algorithm can, for example, compare various pixel sizes / numbers and / or cell counts (or the change over time of those quantities) to each threshold. As another example, the cell growth assessment algorithm can include a neural network that accepts various pixel sizes / numbers and / or cell counts as inputs and outputs a growth classification or score. Alternatively, growth information (e.g., day-specific cell counts and / or colony pixel sizes, etc.) can be displayed to the user to assist the user in assessing the growth profile of the clone.

[0010] If the clone is manually or automatically determined to have an appropriate growth profile and any other relevant criteria are met (e.g., regarding one or more product quality metrics), the clone can proceed to one or more additional stages of the cell line development process. For example, the cells of the cell line can be introduced into a new culture environment and cultured. The cell line can be used for any of a variety of purposes depending on the embodiment. For example, the cell line can be used to provide cells that produce antibodies or hybrid molecules for biopharmaceuticals (e.g., drugs including bispecific T cell engager (BiTE®) antibodies such as BLINCYTO® (blinatumomab), or monoclonal antibodies, etc.), or to provide cells for research and / or development purposes.

[0011] Those skilled in the art will understand that the figures described herein are included for illustrative purposes and are not intended to limit the present disclosure. The drawings are not necessarily to scale and instead focus on showing the principles of the present disclosure. In some cases, various aspects of the described embodiments may be shown exaggerated or enlarged to make the described embodiments easier to understand. In the drawings, like reference numerals generally refer to functionally and / or structurally similar components throughout the various figures.

Brief Description of the Drawings

[0012]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5A

Figure 5B

Figure 6

Figure 7

Mode for Carrying Out the Invention

[0013] The various concepts described above as introductions and discussed in more detail below can be implemented in any of a number of ways, and the concepts described are not limited to any particular implementation. Examples of embodiments are provided for purposes of illustration.

[0014] FIG. 1 is a simplified block diagram of an exemplary system 100 in which the techniques described herein may be implemented. System 100 includes a visual inspection system 102 communicatively coupled to a computer system 104. The visual inspection system 102 includes hardware (e.g., a well plate stage, a light source, one or more lenses and / or mirrors, an imager, etc.), as well as firmware and / or software, and is configured to capture digital images of wells within a well plate. An example embodiment of the visual inspection system 102 is shown in FIG. 2, which omits some components for clarity. In the embodiment of FIG. 2, the visual inspection system 102 may include a stage (not shown in FIG. 2) configured to receive a well plate 204 that houses several wells 206. The well plate 204 may be of any suitable size, any suitable shape, and have any suitable number (e.g., 6, 24, 96, 384, 1536, etc.) of wells disposed thereon. Further, the wells 206 may be disposed in any suitable pattern on the well plate 204, such as, for example, a 2:3 rectangular matrix.

[0015] The visual inspection system 102 further includes an imager 210 configured to acquire a wide-field image. In some embodiments, the visual inspection system 102 can also include one or more additional imagers (e.g., the high-magnification imager described in PCT Patent Application No. PCT / US19 / 63177 (PCT Patent Application International Publication No. WO 2020 / 0112723 Pamphlet)). The visual inspection system 102 also includes an illumination system (not shown in FIG. 2), which can include any suitable number and / or type of light sources configured to generate source light and illuminate the wells 206 in the well plate 204 when the wells 206 are positioned in the optical path of the imager 210. The imager 210 includes a telecentric lens 220 and a wide-field camera 222. The telecentric lens 220 can be a high-fidelity telecentric lens with a magnification of 1×, and the wide-field camera 222 can be, for example, a charge-coupled device (CCD) camera. In some embodiments, the imager 210 is configured to capture, at an appropriate resolution level, an image showing the entire single well 206 when the well 206 is appropriately positioned on the stage and illuminated by the illumination system, and is positioned relative to the stage. In some embodiments where the visual inspection system 102 includes a second imager for high magnification / resolution, the second imager includes a magnifying objective lens (e.g., an objective lens with a magnification of 20× and a long working distance) and a high-resolution camera (e.g., another CCD camera).

[0016] In some embodiments, each well 206 of the well plate 204 has one or more transparent and / or opaque portions. For example, each of the wells 206 may be entirely transparent or may have a transparent bottom with opaque sidewalls. Each well 206 may generally be cylindrical or may have other suitable shapes (e.g., cube, etc.). The visual inspection system 102 may image (e.g., sequentially) all of the wells 206 of the well plate 204. For this purpose, the visual inspection system 102 may be configured to move the stage along one or more (e.g., x and y) axes to continuously align each well 206 with the optical paths of the illumination system and imager 210 for individual well analysis. For example, the stage may be coupled to one or more electric actuators. When each well 206 is aligned with the optical paths of the illumination system and imager 210, the imager 210 acquires one or more images of the illuminated well 206. Any cells within a given well 206 may generally be in a flat plane on the base of the well 206, in which case the well 206 may be imaged from a top-down or bottom-up perspective. In such embodiments, the visual inspection system 102 may also be configured to move the stage in the vertical (z) direction to continue focusing on the flat and thin layer where cells may be present. The visual inspection system 102 can also apply any suitable techniques to reduce vibration and / or mechanically support high-fidelity imaging, both of which can be particularly important when high-magnification imaging is used.

[0017] FIG. 2 merely shows an example embodiment of the visual inspection system 102, and it is understood that other embodiments are possible. Further, the visual inspection system 102 of the example of FIG. 2 may include other components in addition to those described above. For example, the visual inspection system 102 may include one or more communication interfaces to enable communication with the computer system 104 and one or more processors to provide local control of the stage, illumination system, and / or imager 210 (e.g., in response to commands received from the computer system 104).

[0018] Referring again to FIG. 1, computer system 104 generally controls / automates the operation of visual inspection system 102 and may be configured to receive and process images captured / generated by visual inspection system 102 as further described hereinbelow. Computer system 104 is also coupled to training server 106 via network 108. Network 108 may be a single communication network or may include multiple communication networks of one or more types (e.g., one or more wired and / or wireless local area networks (LANs) and / or one or more wired and / or wireless wide area networks (WANs) such as the Internet). Training server 106 is generally configured to train one or more machine learning models, and training server 106 enables computer system 104 to access the machine learning models via network 108, thereby enabling one or more image processing operations to be performed on images generated by visual inspection system 102. In various embodiments, training server 106 can provide its machine learning models as a "cloud" service (e.g., Amazon Web Services), or training server 106 may be a local server. In alternative embodiments, the machine learning models are transferred to computer system 104 by network download or other techniques (e.g., by physically transferring a portable storage device to computer system 104). In other embodiments, computer system 104 itself performs model training, in which case system 100 may omit both network 108 and training server 106. In yet other embodiments, some or all of the components of computer system 104 shown in FIG. 1 (e.g., one, several, or all of the modules described hereinafter) are instead included in visual inspection system 102, in which case visual inspection system 102 can communicate directly with training server 106 via network 108.

[0019] The computer system 104 can be a general-purpose computer specially programmed to perform the operations discussed herein, or it can be a dedicated computing device. As can be seen from FIG. 1, the computer system 104 includes a processing unit 110, a network interface 112, a display unit 114, and a memory unit 116. However, in some embodiments, the computer system 104 includes two or more computers that are located in the same place as each other or are separated from each other. In these distributed embodiments, the operations described herein related to the processing unit 110, the network interface 112, the display unit 114, and / or the memory unit 116 can each be divided among a plurality of processing units, network interfaces, memory units, and / or display units.

[0020] The processing unit 110 includes one or more processors, and each of the one or more processors can be a programmable microprocessor that executes software instructions stored in the memory unit 116 to perform some or all of the functions of the computer system 104 described herein. The processing unit 110 can include, for example, one or more graphics processing units (GPUs) and / or one or more central processing units (CPUs). Alternatively or in addition, some of the processors within the processing unit 110 can be other types of processors (e.g., application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), etc.), and some of the functions of the computer system 104 described herein can be implemented in hardware instead.

[0021] The network interface 112 can include any suitable hardware (e.g., front-end transmitter and receiver hardware), firmware, and / or software configured to communicate with the training server 106 via the network 108 using one or more communication protocols. For example, the network interface 112 can be or include an Ethernet interface that enables the computer system 104 to communicate with the training server 106 via the Internet or an intranet, etc.

[0022] The display unit 114 can include one or more output devices such as a computer monitor or a touch screen, can be integrated into the devices of the computer system 104, or can operate as a peripheral device. The display unit 114 can utilize any suitable one or more display technologies (e.g., LED, OLED, LCD, etc.). Although not shown in FIG. 1, the computer system 104 can also include one or more input devices (e.g., a keyboard, a microphone, a touch screen, etc.) configured to receive user input. The display unit 114 (and any input devices) can include not only appropriate hardware but also related firmware and / or software (e.g., display driver software).

[0023] The memory unit 116 can include one or more volatile memories and / or non-volatile memories. Any suitable one or more memory types can be included, such as read-only memory (ROM), random access memory (RAM), flash memory, solid-state drive (SSD), hard disk drive (HDD), etc. Collectively, the memory unit 116 can store instructions of one or more software applications, data received / used by those applications, and data output / generated by those applications (when executed by the processing unit 110).

[0024] In one example system 100, one such application is the cell growth assay application 118. Generally, when executed by the processing unit 110, the cell growth assay application 118 executes one or more algorithms that, depending on the embodiment, provide an output to assist a user (e.g., when presented on the display unit 114) in performing a “final” or overall growth assay of a particular cell line, or use such output to automatically generate an overall growth assay of the cell line. Various modules of application 118 are discussed below, but it is understood that those modules can be distributed among different software applications and / or that the functionality of any one of such modules can be divided among different software applications.

[0025] Application 118 can include a visual inspection system (VIS) control module 120, a colony size module 122, and a cell count module 124. In some embodiments, application 118 includes more, fewer, and / or different modules. For example, application 119 can exclude the VIS control module and / or the cell count module 124.

[0026] The VIS control module 120 controls / automates the operation of the visual inspection system 102 via commands or other messages to enable it to generate images of samples within the wells of the well plate 204 with little or no human intervention. The visual inspection system 102 may transmit the captured images to the computer system 104 for storage in the memory unit 116 or another suitable memory not shown in FIG. 1.

[0027] The colony size module 122 generally processes the well images received from the visual inspection system 102 to determine which portions of each image represent cell colonies (or portions of cell colonies). The colony size module 122 may use a fully convolutional neural network (“FCN”) 130 to process each well image to generate a “heat map” having a pixel size smaller (i.e., fewer pixels) than the well image. More precisely, the output of the FCN 130 may be a downsampling segmentation map that provides a relatively low-resolution indication of which portions of the well image represent cell colonies or colony portions. The FCN 130 and the downsampling segmentation map generated by the FCN 130 are discussed in more detail below with reference to FIG. 6. FIG. 1 shows the FCN 130 that exists in (i.e., is stored in the memory of) the training server 106, reflecting one embodiment in which the colony size module 122 executes the FCN 130 by utilizing a web service or accessing a local server. However, as described herein, other embodiments (e.g., storing and accessing the FCN 130 locally on the computer system 104) are possible.

[0028] The cell counting module 124 generally processes the well images received from the visual inspection system 102 to identify the exact number of cells in each well image. In some embodiments, the cell counting module 124 processes each well image to identify each individual cell in the image (e.g., as opposed to doublets or debris), and then sums the number of individual cells thus identified to determine the total cell count. For example, the cell counting module 124 may execute a convolutional neural network to identify / classify individual cells (e.g., as described in PCT Patent Application No. PCT / US19 / 63177 (PCT Patent Application International Publication No. WO 2020 / 0112723 Pamphlet)). In other embodiments, the cell counting module 124 uses a technique that does not require object detection (i.e., without first identifying individual cells). In the embodiment shown in FIG. 1, for example, the cell counting module 124 inputs the well image to a fully convolutional regression network (FCRN) 132 that determines the cell count of a given well image based on density estimation, such as described by Xie et al., rather than object detection or segmentation. The RCRN 132 may be, for example, the "FCRN-A" or "FCRN-B" discussed by Xie et al., or another suitable type of FCRN. FIG. 1 shows the FCRN 132 existing in the training server 106 (i.e., stored in the memory of the training server 106), reflecting an embodiment in which the cell counting module 124 executes the FCRN 132 by utilizing a web service or accessing a local server. However, as noted above, other embodiments (e.g., storing and accessing the FCRN 132 locally on the computer system 104) are also possible.

[0029] As discussed in more detail below with reference to FIG. 3, application 118 may utilize cell count module 124 on one or more relatively early stage well images (e.g., from the first few days after inoculation), and may utilize colony size module 122 on one or more late stage well images (e.g., about one week after inoculation where precise cell counting becomes unrealistic due to cell stacking). Further, as also discussed below in relation to FIG. 3, application 118 may execute a higher level algorithm to assess the growth of specific clones based on the outputs generated by cell count module 124 and colony size module 122.

[0030] The operation of system 100 according to some embodiments will now be described with reference to FIGS. 1 and 2. First, training server 106 (i.e., one or more processors of server 106) uses the data stored in training database 130 to train FCN 130 and FCRN 132 (or in other embodiments, different types of machine learning models utilized by modules 122, 124). Training database 130 may include a single database stored in a single memory (e.g., HDD, SSD, etc.), a single database stored across multiple memories, or multiple databases stored in one or more memories. For each of neural networks 130, 132, training database 130 may store a corresponding set of training data (e.g., input / image data and corresponding labels).

[0031] In some embodiments, the training images of FCN130 (stored in training database 140) are image "patches" that are smaller cropped versions of the full well images. For example, the training images can be obtained by automatically capturing well images using the eye inspection system 102 as described above (and / or using one or more other similar systems), and then manually or automatically cropping the well images so that each cropped image (image patch) shows only a portion of the well and its contents. For example, if the full well image has a size of 3333×3333 pixels (or 3495×3495 pixels, etc.), each image patch has a size of only 64×64 pixels (or 32×32 pixels or 128×128 pixels, etc.). The absence of transposed layers / upsampling in FCN130 enables the use of this cropping technique for training, thereby significantly reducing the burden of generating the training image library. For example, instead of the user labeling each of many pixels of each well image, the user only needs to provide a single label (e.g., a single label that applies to all 64×64 pixels of the image patch) for each image patch. In other embodiments, finer labeling of the image patches is used (e.g., labeling of pixel subsets within each image patch). In some other embodiments, the training images of FCRN132 (which are also stored in training database 140) are also image patches derived from well images, but are labeled pixel by pixel.

[0032] After the training images are manually labeled (e.g., using labeling software that presents the training images in a GUI and accepts labels as user input), the training server 106 uses the appropriate images and labels to train FCN130 and FCRN132. For example, the training server 106 can train FCN130 using image patches with a single label and can train FCRN132 using image patches labeled pixel by pixel.

[0033] After server 106 trains FCN 130 and FCRN 132 and validates the trained models 130, 132, computing system 104 may use the trained models 130, 132 to evaluate (or facilitate the evaluation of) the growth rate of a particular clone used to inoculate well 206 in well plate 204. First, each well 206 within well plate 204 of vision inspection system 102 may be automatically or manually filled with a medium containing nutrients (such as amino acids, vitamins, etc.), growth factors, and / or other components appropriate for the cells. In some embodiments and / or scenarios, an attempt is made to inoculate each well with a unique clone of cells. For example, flow cytometry techniques such as fluorescence-activated cell sorting (FACS) techniques can be used to inoculate each well 206 with a single clone of cells. Alternatively, a single well 206 may be seeded with multiple cells of a clone.

[0034] Next, well plate 204 is loaded onto the stage, and VIS control module 120 moves vision inspection system 102 in small increments (e.g., in the x and / or y directions) on the stage and synchronously activates imager 210 (and optionally the illumination system), whereby imager 210 captures at least one image of each well 206. This initial image of each well 206 may be captured immediately after inoculation during the first day of culture and may show the entire area of each well 206 (e.g., from a bottom-up view). Vision inspection system 102 may locally store each well image or may immediately transfer each image to computer system 104 (e.g., for storage in memory unit 116).

[0035] The process of imaging well 206 may be repeated at regular or irregular intervals depending on the embodiment and / or scenario. For example, the VIS control module 120 may cause the visual inspection system 102 to image each well 206 once a day, once every two days, or once every three days, etc., over some predefined culture period (e.g., 10 days, 14 days, etc.). Alternatively (e.g., in some embodiments where application 118 omits the cell counting module 124), well 206 may be imaged only at the beginning and end of the culture period (e.g., day 1 and day 14 of a 14-day culture period) or only at the beginning, midpoint, and end of the culture period. As well images are generated or in a batch after a subset (or all) of the images are generated, the visual inspection system 102 sends the images to the computer system 104 for automated analysis. Similar to the process of capturing well images, the process of transferring the images to the computer system 104 may be automated (e.g., triggered by a command from the VIS control module 120).

[0036] Either when the well image is received or at some time thereafter (e.g., after the entire culture period has ended), application 118 generally processes the well image to assess cell growth or facilitate manual cell growth assessment as discussed above. FIG. 3 shows various algorithms that application 118 may implement for this purpose as part of an example process (at least partially) executed by system 100. However, it is understood that application 118 may use other suitable algorithms as additional or alternative. All well images discussed in relation to FIG. 3 may be captured, for example, by imager 210 of the visual inspection system 102 when controlled by the VIS control module 120.

[0037] In one example of process 300, in preliminary stage 302, FACS subcloning is used to inoculate clonal cells into individual wells (e.g., well 206 within well plate 204). In other embodiments, flow cytometry techniques other than FACS subcloning may be used in stage 302. During the initial stage of the culture period (e.g., immediately after inoculating well 206 on the first day), in a first stage 304-1, at least one image of each well 206 is generated, and a first set 306-1 of well images (e.g., 96 images for a 96-well plate) is generated. In a second stage 304-2, at least one additional image of each well 206 is generated, and another set 306-2 of well images is generated. Stage 304-2 may be performed, for example, one day after inoculation or after some other appropriate time period (e.g., two or three days after stage 304-1, etc.). In a third stage 304-3, at least one additional image of each well 206 is generated, and another set 306-3 of well images is generated. Stage 304-3 may be performed, for example, two days after inoculation or after some other appropriate time period (e.g., one or two days after stage 304-2, etc.). This may be done for N stages, where N is any appropriate integer greater than 1, and the Nth stage 304-N may be performed at the end or near the end of the culture period. However, in some embodiments where application 118 estimates only colony size, well images may be captured in only one stage (e.g., only in stage 304-N, which may be the end of the culture period). In some embodiments, stage 304-1 is performed on the first day of culture, and subsequent stages 304-2 through 304-N are performed at one- or two-day intervals. The entire culture period may be, for example, 14 days, and stages 304-1 through 304-14 are performed regularly at one-day intervals to generate 14 sets 306-1 through 306-14 of well images.

[0038] FIG. 4 shows examples of images 400, 402, and 404 that may correspond to a single well 206 at each of three different times (e.g., stages 304-1, 304-2, and 304-3). Each of the images 400, 402, 404 represents a bottom-up or top-bottom view of the well 206 and shows the entire contents of the well 206. In some embodiments, it is understood that the well images 400, 402, 404 can also include some areas outside the perimeter of the well 206 (e.g., if the image is rectangular). As seen in FIG. 4, in this example, the well image 400 includes a single clone cell 410, the well image 402 includes a very small colony 420 of five cells, and the well image 404 includes a colony 430 that is moderately large such that many cells are stacked (i.e., partially or fully obscuring other cells of the colony 430).

[0039] In the example embodiment of FIG. 3, the well image sets 306-1 and 306-2 are processed via a cell counting algorithm 310 implemented by the cell counting module 124, while the remaining well image sets 306-3 through 306-N are processed via a colony size algorithm 320 implemented by the colony size module 122. In particular, embodiments of the cell counting algorithm 310 may include processing each image of a given well image set 306 using FCRN132 (or another suitable model utilized by module 124), and embodiments of the colony size algorithm 320 may include processing each image of a given well image set 306 using FCN130 (or another suitable model utilized by module 122). For example, the output of algorithm 310 can be the cell count of each well image, and / or the output of algorithm 320 can be a downsampled segmentation map (or the number of pixels in the map representing the colony).

[0040] Figure 3 shows well image sets 306-1 and 306-2 being processed via algorithm 310 and well image sets 306-3 through 306-N being processed via algorithm 320. However, in other embodiments and / or scenarios, different well image sets (and / or a different number of well image sets) may be processed by one or both of algorithms 310, 320. For example, if the cell counting algorithm 310 can safely assume that each well 206 initially contains only one clone cell, it may not be necessary to process any image (e.g., image 400) of well image set 306-1. As another example, module 124 may apply the cell counting algorithm 310 to more than the first two stages 306-1, 306-2, and / or module 122 may apply the colony size algorithm 320 to fewer stages than the last (N - 2) stages 306-3 through 306-N. In some embodiments and / or scenarios, one or more of the well image sets 306 are not processed by either module 122 or module 124.

[0041] In some embodiments, application 118 dynamically determines, in each of one or more of stage 304, whether application 118 should apply algorithm 310 and / or algorithm 320 to each well image set 306. For example, for a given stage 304, module 124 may first attempt to apply cell counting algorithm 310 to one, some, or all of the images in set 306. If algorithm 310 is unable to identify (or at least not identify with a threshold confidence level or the like) the cell count for some or all of the well images in set 306 (e.g., images exceeding a threshold number), module 122 may process image set 306 to generate a downsampled segmentation map and / or a corresponding pixel count indicative of colony size. In the example of FIG. 4, for example, algorithm 310 may successfully count the cells in well images 400 and 402, but may fail to count the cells in well image 404. In response to this failure, algorithm 320 may process well image 430 to generate a downsampled segmentation map indicative of the approximate shape / area / boundary of colony 430 (e.g., excluding stray cells that do not contact other cells in colony 430).

[0042] In one example of process 300, application 118 also operates on the outputs of algorithms 310, 320 to implement a cell growth assessment algorithm 330 that performs an “overall” growth assessment of the clone under consideration. Algorithm 330 can take any suitable form. For a given well 206, for example, algorithm 330 can compare the cell count generated by cell count algorithm 310 to a day / phase-specific threshold (and / or compare the absolute, percentage, or ratio increase between phases 304 to a threshold, etc.), and / or compare the pixel count generated by colony size algorithm 320 to a day / phase-specific threshold (and / or compare the absolute, percentage, or ratio increase between phases 304 to a threshold, etc.). Application 118 can then output a growth indication (e.g., a score or a classification such as “good,” “medium,” “bad,” or “neither”) based on the comparison. In other embodiments, implementation of algorithm 330 includes running another machine learning model that operates on the outputs of algorithms 310 and / or 320. For example, training server 106 can accept inputs from algorithms 310, 320 (e.g., day / phase-specific cell counts and day / phase-specific colony pixel counts / sizes for each well image, for example) and further train and / or score a neural network configured to output a growth classification (e.g., “good,” “medium,” “bad,” or “neither”) or score.

[0043] In some embodiments, cell growth assessment algorithm 330 also performs one or more other operations as a check of process 300. For example, algorithm 330 can use one or more neural networks (e.g., trained and / or stored by server 106) to confirm that the cell colonies detected by algorithm 320 originated from only one clone cell. For example, algorithm 330 can accomplish this via any suitable technique described in PCT patent application No. PCT / US19 / 63177 (published as PCT patent application international publication No. WO 2020 / 0112723).

[0044] In some embodiments, application 118 does not include / support cell growth assessment algorithm 330. Instead, the display unit 114 is caused to display the output of algorithm 310 (e.g., cell count per well per day / phase) and the output of algorithm 320 (e.g., downsampling segmentation map per well per day / phase and / or pixel count indicating how many pixels in the map correspond to areas where cell colonies are presumed to exist). The user of computer system 104 can then view the information on display unit 114 and more subjectively assess the growth profile of the clone. Additionally or alternatively, in some embodiments, application 118 does not include / support cell count algorithm 310.

[0045] The assessment by a user observing the output of cell growth assessment algorithm 330 or the output of algorithms 310 and / or 320 can indicate whether a clone should be rejected or whether to proceed to the next cell line development stage. Cell line development can be for any suitable purpose, depending on the embodiment and / or scenario. For example, the cell line can be for developing antibodies or hybrid molecules for biopharmaceuticals (e.g., bispecific T cell engagers (BiTE®) antibodies such as BLINCYTO® (blinatumomab), or monoclonal antibodies, etc.), or for use in research and / or development purposes. As an example, the next stage of cell line development that the clone proceeds to (or does not proceed to) can include introducing the cells of the cell line into a new culture environment (e.g., a bioreactor). Information regarding cell culture can be found, for example, in Green and Sambrook, “Molecular Cloning: A Laboratory Manual” (4th edition) Cold Spring Harbor Laboratory Press 2012, which is hereby incorporated by reference in its entirety.

[0046] As described herein, the FCN 130 implemented by the colony size module 122 (e.g., to execute algorithm 320) can be a modified version of the FCN knighted in Long et al., which is built on top of an old CNN such as the CNN 500 shown in FIG. 5A. As can be seen in FIG. 5A, the CNN 500 operates on the input image 502 and outputs a classification 504. The CNN 500 generally includes one or more convolutional layers 510 that detect features within the image 502. The earlier convolutional layers generally detect lower-level features such as edges or corners, while the later convolutional layers generally detect more abstract features such as overall shapes. Although not shown in FIG. 5A, the convolutional layer 510 may be interspersed with any suitable number of pooling layers (i.e., downsampling layers that reduce computation while maintaining the relative locations of features) and / or rectified linear unit (ReLU) layers that apply activation functions. The CNN 500 also generally includes any suitable number (greater than 0) of fully connected layers 512 that provide high-level inferences based on the features detected by the earlier layers. Specifically, the fully connected layer 512 identifies the classification 504 based on the features detected by the earlier layers.

[0047] Figure 5B shows an FCN 520, an example that operates on the input image 522 to output a segmented image 524 that is the same size as the input image 522, and shows the classification of each pixel of the input image 522 (e.g., "cat" vs. "non - cat" or "cat" vs. "ground" vs. "sky", etc.) rather than a single overall image classification. Similar to the CNN 500, the FCN 520 includes one or more convolutional layers 530 in which pooling and / or ReLU layers (not shown in Figure 5B) may be interspersed. However, unlike the CNN 500, the FCN 520 does not include any fully - connected layers that specify an overall image classification, and instead includes additional convolutional layers 532. An additional convolutional layer 534 generates a heatmap (i.e., a down - sampled segmentation map) of the image 522. The heatmap is provided to an appropriate number (greater than 0) of transposed layers 534, also called "de - convolutional layers". The transposed layers 534 upsample the heatmap to provide a classification for each pixel of the entire image 522, and the classification can then be represented as the segmented image 524. The operation of the FCN 520 is described in more detail in Long et al.

[0048] An example of a modified FCN 600 is shown in FIG. 6. The modified FCN 600 can be used, for example, as the FCN 130 of FIG. 1. Although referred to herein as a "modified" FCN, it is understood that the FCN 600 does not necessarily need to be actively modified from any particular starting point. Instead, the term "modified" is used to indicate that the architecture of the FCN 600 is different from the architecture of the FCN described in Long et al.

[0049] The modified FCN600 operates on an image 602 (e.g., in this case, an image from one of the well image sets 306) to output a downsampled segmentation map 604, which is smaller than the input image 602 (i.e., consists of fewer pixels). Each of some or all of the pixels of map 604 may indicate a classification (e.g., "colony" or "non - colony") identified by the modified FCN600. Similar to FCN520, the modified FCN600 includes one or more convolutional layers 610 in which pooling and / or ReLU layers (not shown in FIG. 6) may be interspersed. Also similar to FCN520, the modified FCN600 includes one or more additional convolutional layers 612 instead of fully - connected layers (i.e., the modified FCN600 does not include fully - connected layers). However, unlike FCN520, the modified FCN600 does not include any transposed layers to upscale map 604 to generate a pixel - by - pixel classified full - size image. Instead, map 604 may be the final output of the modified FCN600. The operation of convolutional layers 610, additional convolutional layers 612, and any pooling and / or other (e.g., ReLU) layers may be described in Long et al.

[0050] Based on that pixel - by - pixel classification, map 604 indicates that the modified FCN600 has inferred that a cell colony is present or not present in the well image (e.g., image 602). Due to the smaller pixel size (i.e., fewer pixels), map 604 necessarily has a lower resolution than the full - size segmented image, and thus, the precision in representing the boundaries / shapes of cell colonies is lower. In some embodiments, application 118 upscales map 604 using simple interpolation (e.g., before display to the user) to match the pixel size of the input image 602 to assist in the identification of cell colony regions.

[0051] The modified FCN600 provides a lower resolution / precision than the full-size segmented image of the FCN520. However, as described above, the modified FCN600 has the advantage that training can be achieved without pixel-by-pixel labeling of the full-size well image. Furthermore, it has been found that the relative imprecision of the modified FCN600 is not very important for growth profile assessment, and thus it is substantially more important that the generation of the training image library is relatively easy. Additionally, the absence of a transposed layer in the modified FCN600 results in fewer processing resources and / or a faster inference / classification time in the modified FCN600 compared to the FCN620.

[0052] FIG. 7 is a flow diagram of an example method 700 for facilitating the growth assessment of a cell line (e.g., for clone selection during the cell line development process). The method 700 can be performed by one or more parts of the system 100 (e.g., the visual inspection system 102 and the computer system 104), or by another suitable system. As a more specific example, block 702 can be performed by the visual inspection system 102, while blocks 704 and 706 (or 704, 708, and 710) can be performed by the computer system 104 (e.g., by the processing unit 110 when executing instructions of the colony size module 122 stored in the memory unit 116).

[0053] In block 702 of method 700, a first well image is generated. The first well image is an image of a well containing a medium inoculated with at least one cell of the cell line (e.g., an image similar to well image 404). In block 704, a downsampling segmentation map is generated using an FCN. The FCN can be, for example, the modified FCN600 of FIG. 6. The pixels of the map indicate the presumed presence or absence of cell colonies in the corresponding part of the first well image. Block 704 can include inputting the first well image into the FCN. The FCN need not include a transposed layer (e.g., like the FCN600), in which case the map output by the FCN is smaller than the first well image (i.e., has fewer pixels).

[0054] Depending on the embodiment and / or scenario, method 700 may include block 706 and / or may include a combination of blocks 708 and 710. In block 706, a display of colony size information (e.g., triggered via a command or other message) facilitates a manual / user cell growth assessment of the cell line. For example, application 118 may generate a graphical user interface (GUI) or information filling the GUI, and cause the display unit 114 to present the colony size information within the GUI. The colony size information may include a downsampled segmentation map (generated in block 704) and / or pixel counts and possibly other related information derived from the downsampled segmentation map (e.g., by module 124 or another part of application 118 that processes the map). In some embodiments of the method and system, the "pixel count" is the count of all pixels in the map classified as being within a cell colony (e.g., the "pixel area" of the cell colony) by the FCN. Alternatively, the "pixel count" may be a count of how many pixels are in the maximum dimension (e.g., width or length) of the cell colony shown in the map or another suitable type of pixel count indicating colony size.

[0055] In block 708, by inputting the colony size information (i.e., the downsampled segmentation map and / or pixel counts and possibly other related information) into a cell growth assessment algorithm (e.g., algorithm 330), a growth classification or score of the cell line is determined. Next, in block 710, a display of the growth classification or score (e.g., triggered via a command or other message) facilitates a determination of whether to advance the clone to the next stage of cell line development. For example, application 118 may generate a GUI or information filling the GUI, and cause the display unit 114 to present the classification or score within the GUI.

[0056] In some embodiments, method 700 includes one or more additional blocks not shown in FIG. 7. For example, block 702 may include generating a first well image at a first time and further generating a second well image (of the same well) at a later second time, and method 700 may include an additional block in which the count of at least a subset of the cells present in the well at the second time is determined by processing the second well image. Method 700 may then include displaying both the colony size information and the count. For example, the second time may be the end of a culture period during which a single cell of the cell line is capable of forming a colony. By way of example, the second time may be about 7 days after the first time, such as 4 - 10 days after the first time, 5 - 9 days after the first time, 6 - 8 days after the first time, 6 - 7 days after the first time, or 7 - 8 days after the first time. By way of example, the second time may be about 14 days after the first time, such as 11 - 17 days after the first time, 12 - 16 days after the first time, 13 - 15 days after the first time, 13 - 14 days after the first time, or 14 - 15 days after the first time.

[0057] As another example, in some embodiments where method 700 includes block 708, method 700 includes an additional block in which the cell line is selectively used or not used in subsequent stages of cell line development, at least in part based on the growth classification or score of the cell line. The subsequent stage may be, for example, a new culture environment in which the cell line is selectively advanced or not advanced.

[0058] As yet another example, method 700 may include one or more blocks related to training the FCN. For example, method 700 may include a first additional block in which a plurality of well training images are received (e.g., from vision inspection system 102 or computer system 104 by training server 106), a second additional block in which a plurality of image patches are generated by cropping at least each well training image to a smaller size (e.g., by training server 106), a third additional block in which one or more user-provided labels for each image patch (e.g., a single user-provided label for each image patch) are received (e.g., from computer system 104, another computing system, or a peripheral input device of server 106 by training server 106), and / or a fourth additional block in which the FCN is trained using the image patches and the corresponding user-provided labels (e.g., by training server 106).

[0059] By way of example and not limitation, the present disclosure contemplates at least the following examples.

Example

[0060] Example 1. A method for promoting the growth assay of cell lines, comprising: generating, by an imaging unit, a first well image of a well containing a medium inoculated with at least one cell of the cell line; generating, by one or more processors, a downsampling segmentation map including pixels indicating the estimated presence or absence of cell colonies in corresponding portions of the first well image, wherein generating the downsampling segmentation map includes inputting the first well image into a fully convolutional neural network having a plurality of convolutional layers, and the downsampling segmentation map has fewer pixels than the first well image; and the following: (i) one or more processors input colony size information into a cell growth assay algorithm to specify a growth classification or score of the cell line, wherein the colony size information includes the downsampling segmentation map and / or a pixel count derived from the downsampling segmentation map, specifying, and causing, by one or more processors, the growth classification or score to be displayed; and (ii) causing, by one or more processors, the colony size information to be displayed, thereby promoting manual cell growth assay. A method including one or both of the above.

[0061] Example 2. The method according to Example 1, wherein generating the downsampling segmentation map includes inputting the first well image into a fully convolutional neural network including a plurality of convolutional layers and not including a transposed layer.

[0062] Example 3. Generating the first well image includes generating the first well image at a first time, and further includes generating a second well image at a second time earlier than the first time. The method further includes one or more processors processing the second well image to specify a count of at least a subset of the cells present in the well at the second time, according to the method of Example 1 or 2.

[0063] Example 4. The method includes displaying colony size information, and the method further includes, by one or more processors, displaying a count, the method according to Example 3.

[0064] Example 5. The method includes identifying a growth classification or score of a cell line and displaying the growth classification or score by inputting at least colony size information and a count into a cell growth assessment algorithm, the method according to Example 3 or 4.

[0065] Example 6. The method includes identifying a growth classification or score of a cell line by inputting at least colony size information into a cell growth assessment algorithm, the colony size information including a pixel count derived from a downsampling segmentation map, the pixel count being a count of how many pixels in the downsampling segmentation map were classified as belonging to a cell colony by a fully convolutional neural network, and the cell growth assessment algorithm identifying the growth classification or score by at least partially comparing the pixel count to a threshold pixel count, the method according to any one of Examples 1 to 5.

[0066] Example 7. The method according to any one of Examples 1 to 6 further includes selectively using or not using a cell line in a subsequent stage of a cell line development process, at least partially based on the growth classification or score of the cell line.

[0067] Example 8. Selectively using a cell line in a subsequent stage of a cell line development process includes selectively advancing the cell line to a new culture environment, and not selectively using a cell line in a subsequent stage of a cell line development process includes not advancing the cell line to a new culture environment, the method according to Example 7.

[0068] Example 9. Prior to generating the downsampling segmentation map, receiving a plurality of well-training images, generating a plurality of image patches by cropping each of at least the plurality of well-training images to a smaller size, receiving a user-provided label for each of the plurality of image patches, and training a fully convolutional neural network using the plurality of image patches and the user-provided labels for the plurality of image patches. The method according to any one of Examples 1 to 8, further comprising .

[0069] Example 10. The method according to Example 9, wherein receiving a user-provided label for each of the plurality of image patches includes receiving only one user-provided label for each of the plurality of image patches.

[0070] Example 11. The method according to any one of Examples 1 to 10, further comprising using the display of colony size information for manual cell growth assessment.

[0071] Example 12. One or more non-transitory computer-readable media storing instructions that, when executed by one or more processors, cause the one or more processors to receive a first well image of a well containing a medium inoculated with at least one cell of a cell line, generate a downsampling segmentation map including pixels indicative of a presumed presence or absence of cell colonies in corresponding portions of the first well image, wherein generating the downsampling segmentation map includes inputting the first well image into a fully convolutional neural network having a plurality of convolutional layers, and the downsampling segmentation map has fewer pixels than the first well image, and perform one or both of the following: (i) input colony size information into a cell growth assessment algorithm to identify a growth classification or score of the cell line, wherein the colony size information includes the downsampling segmentation map and / or a pixel count derived from the downsampling segmentation map, and display the growth classification or score; and (ii) display the colony size information, thereby facilitating manual cell growth assessment.

[0072] Example 13. The one or more non-transitory computer-readable media according to Example 12, wherein the fully convolutional neural network includes a plurality of convolutional layers and does not include a transposed layer.

[0073] Example 14. The instructions further cause the one or more processors to generate a first well image at a first time, generate a second well image at a second time earlier than the first time, and process the second well image to identify a count of at least a subset of the cells present in the well at the second time.

[0074] Example 15. One or more non-transitory computer-readable media according to Example 14, wherein the instructions cause one or more processors to display colony size information and further cause the one or more processors to display a count.

[0075] Example 16. One or more non-transitory computer-readable media according to Example 14 or 15, wherein the instructions cause one or more processors to input at least colony size information and a count into a cell growth assessment algorithm to identify a growth classification or score of a cell line and to display the growth classification or score.

[0076] Example 17. One or more non-transitory computer-readable media according to any one of Examples 12 to 16, wherein the instructions cause one or more processors to input at least colony size information into a cell growth assessment algorithm to identify a growth classification or score of a cell line, the colony size information includes a pixel count derived from a downsampling segmentation map, the pixel count is a count of how many pixels in the downsampling segmentation map are classified as belonging to a cell colony by a fully convolutional neural network, and the cell growth assessment algorithm identifies the growth classification or score by at least partially comparing the pixel count with a threshold pixel count.

[0077] Example 18. A system, a visual inspection system, comprising a stage configured to receive a well plate and an imaging unit configured to generate an image of a well in the well plate on the stage, and a computer system, the computer system including one or more processors and one or more memories for storing instructions, the instructions, when executed by the one or more processors, cause the computer system to instruct the imaging unit to generate a first well image of a well containing a medium inoculated with at least one cell of a cell line, and generate a downsampling segmentation map including pixels indicating the presumed presence or absence of cell colonies in the corresponding portion of the first well image, generating the downsampling segmentation map including inputting the first well image into a fully convolutional neural network having a plurality of convolutional layers, the downsampling segmentation map having fewer pixels than the first well image, generating, and the following: (i) the one or more processors input colony size information into a cell growth assessment algorithm to identify a growth classification or score of the cell line, the colony size information including the downsampling segmentation map and / or a pixel count derived from the downsampling segmentation map, identifying and displaying the growth classification or score and (ii) the one or more processors display the colony size information, thereby facilitating manual cell growth assessment, a system that performs one or both of displaying.

[0078] Example 19. The system according to Example 18, wherein the fully convolutional neural network includes a plurality of convolutional layers and does not include a transposed layer.

[0079] Example 20. The instructions cause the computer system to instruct the imaging unit to generate a first well image at a first time, and the instructions cause the computer system to generate a second well image at a second time earlier than the first time and to further instruct the imaging unit to identify a count of at least a subset of the cells that were present in the well at the second time by processing the second well image, the system according to Example 18 or 19.

[0080] Example 21. The instructions cause the computer system to display colony size information, and the instructions cause the computer system to further display a count, the system according to Example 20.

[0081] Example 22. The instructions cause the computer system to identify a growth classification or score of a cell line and to display the growth classification or score by inputting at least colony size information and a count into a cell growth assessment algorithm, the system according to Example 20 or 21.

[0082] The systems, methods, apparatuses, and their components have been described from the perspective of exemplary embodiments, but they are not limited to these exemplary embodiments. The detailed description is to be construed as illustrative only and it is not possible, and indeed unrealistic, to describe all possible embodiments. Thus, not all possible embodiments of the invention are described. Using either current technology or technology developed after the filing date of this patent, many alternative embodiments can be implemented and they remain within the scope of the claims that define the invention.

[0083] Those skilled in the art will appreciate that various modifications, variations, and combinations can be made to the above embodiments without departing from the scope of the invention, and such modifications, variations, and combinations are to be construed as being within the scope of the concept of the invention.

Claims

1. A method for promoting the growth assay of a cell line, comprising: generating, by an imaging unit, a first well image of a well containing a medium inoculated with at least one cell of the cell line; generating, by one or more processors, a downsampling segmentation map including pixels indicative of the presumed presence or absence of cell colonies in corresponding portions of the first well image, wherein generating the downsampling segmentation map includes inputting the first well image into a fully convolutional neural network having a plurality of convolutional layers, the downsampling segmentation map having fewer pixels than the first well image; and the following: (i) the one or more processors input colony size information into a cell growth assay algorithm to identify a growth classification or score of the cell line, the colony size information including the downsampling segmentation map and / or a pixel count derived from the downsampling segmentation map, and the one or more processors display the growth classification or score; and (ii) the one or more processors display the colony size information, thereby facilitating manual cell growth assay, either or both of which; and a method comprising the above.

2. The method according to claim 1, wherein generating the downsampling segmentation map includes inputting the first well image into a fully convolutional neural network including a plurality of convolutional layers and no transposed layer.

3. Generating the first well image includes generating the first well image at a first time and further generating a second well image at a second time earlier than the first time, the method further comprising the one or more processors processing the second well image to identify a count of at least a subset of the cells present in the well at the second time, according to claim 1 or 2.

4. The method includes displaying the colony size information. The method according to claim 3, further comprising causing the one or more processors to display the count.

5. The method comprises: identifying the growth classification or score of the cell line by inputting at least the colony size information and the count into the cell growth assessment algorithm; displaying the growth classification or score; The method according to claim 3 or 4.

6. The method comprises identifying the growth classification or score of the cell line by inputting at least the colony size information into the cell growth assessment algorithm, wherein the colony size information includes the pixel count derived from the downsampling segmentation map, and the pixel count is a count of how many pixels in the downsampling segmentation map are classified as belonging to cell colonies by the fully convolutional neural network; The method according to any one of claims 1 to 5, wherein the cell growth assessment algorithm identifies the growth classification or score by at least partially comparing the pixel count with a threshold pixel count.

7. The method according to any one of claims 1 to 6, further comprising selectively using or not using the cell line in subsequent stages of the cell line development process, at least partially based on the growth classification or score of the cell line.

8. Selectively using the cell line in the subsequent stage of the cell line development process includes selectively advancing the cell line to a new culture environment, and / or Not selectively using the cell line in the subsequent stage of the cell line development process includes not advancing the cell line to a new culture environment. The method according to claim 7.

9. Prior to generating the downsampling segmentation map, receiving a plurality of well training images; generating a plurality of image patches by cropping each of the plurality of well training images to a smaller size; receiving user-provided labels for each of the plurality of image patches; training the fully convolutional neural network using the plurality of image patches and the user-provided labels for the plurality of image patches. The method according to any one of claims 1 to 8, further comprising

10. The method according to claim 9, wherein receiving the user-provided label for each of the plurality of image patches includes receiving only one user-provided label for each of the plurality of image patches.

11. The method according to any one of claims 1 to 10, further comprising using the display of the colony size information for manual cell growth assessment.

12. One or more non-transitory computer-readable media storing instructions that, when executed by one or more processors, cause the one or more processors to receive a first well image of a well containing a medium inoculated with at least one cell of a cell line; generate a downsampled segmentation map including pixels indicating a presumed presence or absence of cell colonies in a corresponding portion of the first well image, wherein generating the downsampled segmentation map includes inputting the first well image into a fully convolutional neural network having a plurality of convolutional layers, wherein the downsampled segmentation map has fewer pixels than the first well image; and (i) inputting the colony size information into a cell growth assessment algorithm to identify a growth classification or score for the cell line, the colony size information including the downsampled segmentation map and / or a pixel count derived from the downsampled segmentation map, identifying and displaying the growth classification or score; and (ii) displaying the colony size information, thereby facilitating manual cell growth assessment. either or both of One or more non-transitory computer-readable media for causing the above to be performed.

13. The one or more non-transitory computer-readable media according to claim 12, wherein the fully convolutional neural network includes a plurality of convolutional layers and does not include a transposed layer.

14. The instructions cause the one or more processors to generate the first well image at a first time, The instructions cause the one or more processors to generate a second well image at a second time earlier than the first time. By processing the second well image, identifying the count of at least a subset of the cells that were present in the well at the second time, The one or more non-transitory computer-readable media according to claim 12 or 13, further causing the above to be performed.

15. The instructions cause the one or more processors to display the colony size information, The one or more non-transitory computer-readable media according to claim 14, wherein the instructions further cause the one or more processors to display the count.

16. The instructions cause the one or more processors to By inputting at least the colony size information and the count into the cell growth assessment algorithm, identifying the growth classification or score of the cell line, Displaying the growth classification or score, The one or more non-transitory computer-readable media according to claim 14 or 15, causing the above to be performed.

17. The instructions cause the one or more processors to identify the growth classification or score of the cell line by inputting at least the colony size information into the cell growth assessment algorithm. The colony size information includes the pixel count derived from the downsampling segmentation map. The pixel count is the count of how many pixels in the downsampling segmentation map were classified as belonging to cell colonies by the fully convolutional neural network. The one or more non-transitory computer-readable media according to any one of claims 12 to 16, wherein the cell growth assessment algorithm identifies the growth classification or score by at least partially comparing the pixel count with a threshold pixel count.

18. A system, A visual inspection system, A stage configured to receive a well plate and An imaging unit configured to generate an image of a well in the well plate on the stage A visual inspection system comprising: A computer system, Comprising, the computer system One or more processors, One or more memories storing instructions, Including, when the instructions are executed by the one or more processors, causing the computer system to Instructing the imaging unit to generate a first well image of a well containing a medium inoculated with at least one cell of the cell line; Generating a downsampling segmentation map including pixels indicating the presumed presence or absence of cell colonies in corresponding portions of the first well image, wherein generating the downsampling segmentation map includes inputting the first well image into a fully convolutional neural network having a plurality of convolutional layers, and the downsampling segmentation map has fewer pixels than the first well image; as follows: (i) By inputting colony size information into a cell growth assessment algorithm to identify a growth classification or score of the cell line, wherein the colony size information includes the downsampling segmentation map and / or a pixel count derived from the downsampling segmentation map, identifying and displaying the growth classification or score; and (ii) Displaying the colony size information, thereby facilitating manual cell growth assessment; either one or both of; A system for causing to perform.

19. The system according to claim 18, wherein the fully convolutional neural network includes a plurality of convolutional layers and does not include a transposed layer.

20. The instruction causes the computer system to instruct the imaging unit to generate the first well image at a first time, The instruction causes the computer system to generate a second well image at a second time earlier than the first time, and by processing the second well image, further instruct the imaging unit to identify a count of at least a subset of the cells present in the well at the second time. The system according to claim 18 or 19.

21. The instruction causes the computer system to display the colony size information, The system according to claim 20, wherein the instruction further causes the computer system to display the count.

22. The instruction causes the computer system to By inputting at least the colony size information and the count into the cell growth assessment algorithm, identifying the growth classification or score of the cell line; displaying the growth classification or score; The system according to claim 20 or 21, which causes the above to be performed.

Images