Data-driven predictive modeling for cell line selection in biopharmaceutical production

Abstract

To provide methods for predicting a relative rank of cell lines advanced from a clone generation and analysis process according to a certain product quality attribute.SOLUTION: A method for facilitating selection of master cell lines from among candidate cell lines that produce recombinant proteins comprises: receiving first attribute values for the candidate cell lines measured using an opto-electronic cell line generation and analysis system; acquiring second attribute values that include one or more attribute values measured at a cell pool screening stage of the candidate cell lines; determining ranking of the candidate cell lines according to a product quality attribute associated with hypothetical small-scale screening cultures, where the determination of the ranking comprises predicting, for each of the candidate cell lines, a value of the product quality attribute by analyzing the first and second multiple attribute values using a machine learning-based regression estimator, and comparing the predicted values; and presenting an indication of the ranking to a user via a user interface.SELECTED DRAWING: Figure 7

Description

[Technical field]

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS Priority is claimed to U.S. Provisional Patent Application No. 62 / 841,186, filed April 30, 2019, and U.S. Provisional Patent Application No. 63 / 014,398, filed April 23, 2020, the disclosures of which are incorporated by reference in their entireties herein.

[0002] This application relates generally to cell line (clone) selection techniques, and more specifically to techniques for predicting the relative rank of cell lines that have progressed from a clone generation and analysis process according to specific product quality attributes. [Background technology]

[0003] Large complex molecules (e.g., proteins), known in the biopharmaceutical industry as biologics, are derived from living systems. The general workflow for the development of a biologic begins with research and development. In this early stage, a disease or indication that represents a significant unmet medical need is targeted. Researchers determine promising drug candidates based on a suitable target product profile governing aspects such as safety, efficacy, and route of administration. Ultimately, a combination of in vitro studies and computational models selects a particular molecule as the top drug candidate for a particular disease and target population. After the top candidate is selected, the molecular blueprint is formalized into a gene and the gene of interest is inserted into an expression vector. The expression vector is then inserted into a host cell in a process known as gene transfer. If gene transfer is successful, the cell is able to incorporate the gene of interest into its own production mechanisms, ultimately acquiring the ability to produce the desired pharmaceutical product.

[0004] Since each cell has unique characteristics, the product produced by each cell will vary slightly, for example, in terms of productivity (e.g., titer) and product quality. In general, it is more desirable to produce drugs with consistently high titers and consistently high quality for economic and safety reasons. High concentration or titer of product helps to reduce the manufacturing footprint required to achieve the desired production volume, thus saving both capital and operating costs. High product quality ensures that a greater percentage of the drug is safe, effective, and usable, which also saves costs. In the context of cell line development, product quality attributes are evaluated through assays performed on the product of interest. These assays often include chromatographic analysis, which is used to determine attributes such as the degree of glycosylation and other factors such as the percentage of unusable protein due to cleavage (clipping) or aggregation (aggregates).

[0005] Based on criteria related to productivity and product quality, the "best" cell lines or clones are selected in a process known as "cell line selection," "clone selection," or "clone screening." The selected cell lines / clones are used for a master cell bank, which serves as a uniform starting point for all future production (e.g., clinical and commercial).

[0006] Ensuring consistent product batches helps promote more uniform and predictable pharmacokinetic and pharmacodynamic responses in patients. However, when a "pool" of heterogeneous cells obtained after gene transfer is used to generate a product of interest, there may be many variants of the generated product. This is because during gene transfer, the gene of interest is integrated into the candidate host cells in various ways. For example, differences in copy number (i.e., the number of integrated copies of the gene of interest) and other differentiation factors may occur between the unique footprints of different cells. The production of the product of interest may also vary due to slight differences in the internal machinery of individual cells, including the nature of post-translational modifications. These variations are undesirable, especially considering the need to ultimately control and guarantee a measured and safe response of the patient. Thus, typically, the cell lines of the master cell bank are required to be "clonally derived", i.e., the master cell bank contains only cells derived from a common single cellular ancestor. This theoretically helps to ensure a large degree of uniformity in the produced drug, although there will be small but inevitable differences due to natural genetic variations due to random mutations as cells divide. Therefore, the clone screening process is critical in distributing not only productive and high-quality starting material but also the only cell line that fits the "clone-derived" requirements.

[0007] Figure 1 shows a typical clone screening process 10. The first stage 11 shows the traditional microtiter plate-based method of clone generation and expansion, which can take 2-3 weeks. Hundreds of pooled heterogeneous cells are sorted into single cell cultures by processes such as fluorescence-activated cell sorting (FACS) or limiting dilution. After recovery into healthy and stable populations, these clone-derived cells are analyzed and the selected population is transferred to stage 12. In stage 12, the clonal cells in small vessels such as spin tubes, 24-well plates or 96-deep well plates are cultured in "small-scale cell cultures" (e.g., 10-day fed-batch). In this small-scale process, boluses of nutrients are added periodically to obtain different measurements of cell growth and viability. Typically, hundreds or thousands of these small-scale cultures are run in parallel. At the end of the culture (e.g., day 10), the cells are harvested for assays and analysis.

[0008] By analyzing the growth and productivity characteristics of the clones in the small-scale culture at stage 12, the "top" or "best" clones (e.g., top 4) are selected for scale-up culture, which is performed in the third stage 14. The scale-up (or "large-scale") process is useful compared to the small-scale culture at stage 12 because it better represents the process that will ultimately be used in clinical and commercial manufacturing. The scale-up process may be performed, for example, in 3-5 liter perfusion bioreactors with a 15-day culture. These perfusion bioreactors accommodate a more efficient transfer of waste products and nutrients, thereby increasing the overall productivity of the culture. Perfusion bioreactors are also typically associated with a larger number of measured variables, such as routine and continuous process conditions and metabolite concentrations, allowing for tighter control and monitoring.

[0009] After the scale-up process in stage 14, media and product are collected and analyzed. Finally, in the fourth stage 16, the scale-up product that produces the highest titer and exhibits the best product quality attributes (PQAs) is typically selected as the "best" or "winning" clone. Finally, in the fifth stage 18, the winning clone is used as a master cell bank for use in future clinical and commercial manufacturing. Summary of the Invention [Problem to be solved by the invention]

[0010] The above-mentioned kind of conventional clone screening process is very resource intensive, typically taking several months and requiring hundreds or thousands of assays and cell cultures. However, as the pace of biotechnology quickens and as more emphasis is placed on processing more molecules in the early stage pipeline, the need for faster clone screening increases. Furthermore, conventional clone screening process lacks standardized criteria for selecting which clones to proceed to the next stage / bioprocess and ultimately selecting the winning clone, in part because the unique combination of modality, composition and sequence characteristics for each different drug candidate means that different factors may be more or less important. [Means for solving the problem]

[0011] Embodiments described herein relate to systems and methods for creating, evaluating and / or applying predictive models of cell line and bioprocess performance in clonal selection. In particular, robust machine learning models are created and used to reduce development timelines and resource usage while improving performance.

[0012] In one aspect, one or more machine learning algorithms can be used to predict the performance of each and every clone in a hypothetical scale-up (bioreactor) culture based on measurements and other data on real-world small-scale cultures of these same clones. Large-scale culture performance can be predicted for a hypothetical / hypothetical number of days of culture (e.g., 15 days of culture), with each prediction being made almost instantly. Depending on the embodiment, this process can result in the selection of better clones / cell lines (i.e., clones that are more likely to perform better in large-scale culture) for scale-up experiments, or can even result in the selection of a "winning" clone without performing any scale-up experiments (e.g., by selecting the clone with the best predicted bioreactor performance), which can shave a month or more off the critical path of a biologics program.

[0013] Using the predictive models described herein, higher producing and / or higher quality clones can be identified compared to the traditional "funnel" approach (i.e., going from stage 12 to stage 14 and then to stage 16 in FIG. 1). This improvement occurs because small scale results, despite having some similarities, are not fully representative of scale-up results. In other words, simply selecting clones with the best productivity and / or best product quality according to some pre-determined criteria at stage 12 will not necessarily result in the best productivity and / or best product quality (according to the same criteria) at stage 14.

[0014] Furthermore, interpretable machine learning algorithms can be used to identify the most important input features (e.g., measurements of small-scale cultures) to achieve accurate prediction. This can be useful, given that in any given clone screening program, a very large number of attributes (e.g., more than 600) can be tracked. Thus, for example, it is possible to make sufficiently accurate predictions using a relatively small number of input features (e.g., about 10 features), eliminating the need to measure a large number of other attributes. Knowledge of the correlation between measurements and desired prediction targets can provide scientific insights and also generate hypotheses for further research that can lead to future bioprocess improvements.

[0015] In another embodiment, in addition to or instead of the above process, one or more machine learning algorithms may be used to select which clones should proceed from the subcloning stage to small-scale screening culture (e.g., stage 11 to stage 12 in FIG. 1). Typically, clones with both high cell productivity scores and high cell numbers at the end of the subcloning stage have been considered to be the best candidates to achieve high performance in small-scale screening culture (fed-batch experiments). This approach typically results in approximately 30-100 clones progressing to the fed-batch stage. However, the machine learning algorithms described herein can improve this process by analyzing various attributes of candidate clones at both the subcloning stage and the preceding cell pool stage and predicting certain product quality attributes (e.g., titer, cell growth or specific productivity) resulting from a virtual small-scale (e.g., fed-batch) culture experiment. The microtiter plate-based method of clone generation and growth (i.e., subcloning stage 11 in FIG. 1) may be replaced with the use of more efficient, high-throughput, and high-content screening tools, such as, for example, the Berkeley Lights Beacon™ optical-electronic cell line generation and analysis system. After predicting product quality attribute values ​​for candidate cell lines, the candidates are ranked according to the predicted values, thereby facilitating the selection of a smaller subset of candidate clones for the next stage of cell line development. Advantageously, the rankings made according to these values ​​can be highly accurate with certain machine learning models, even if the underlying prediction values ​​show relatively low accuracy and therefore appear to be inadequate on the surface. Depending on the embodiment, this process may require less resource usage (e.g., in terms of time, cost, effort, equipment, etc.) and / or provide better standardization when selecting candidate clones / cell lines for small-scale screening culture (i.e., clones that are more likely to be those that show the best performance in small-scale culture). For example, reducing the number of cells that are advanced to the fed-batch stage may free up the ability to test other cell lines for other drug products.In some embodiments, the small-scale screening stage may be skipped entirely based on the ranking of the various cell lines (e.g., by proceeding directly from stage 11 to stage 14 of process 10).

[0016] Those skilled in the art will appreciate that the figures described herein are included for illustrative purposes and are not intended to limit the present disclosure. The figures are not necessarily to scale, with emphasis instead being placed on illustrating the principles of the present disclosure. It should be understood that in some instances, various aspects of the described embodiments may be shown exaggerated or enlarged to facilitate understanding of the described embodiments. In the drawings, like reference numbers throughout the various views generally refer to functionally similar and / or structurally similar components. [Brief description of the drawings]

[0017] [Figure 1] 1 illustrates various stages of a typical clone screening process. [Diagram 2] FIG. 1 is a simplified block diagram of an exemplary system capable of implementing the techniques of the first aspect of the invention described herein. [Diagram 3] FIG. 1 is a flow diagram of an example process for generating a machine learning model specialized for a particular use case. [Figure 4A] 1 illustrates exemplary performance of various models in a variety of different use cases. [Figure 4B] 1 illustrates exemplary performance of various models in a variety of different use cases. [Figure 5A] 1 shows exemplary feature importance metrics for a variety of different use cases and models. [Figure 5B] 1 shows exemplary feature importance metrics for a variety of different use cases and models. [Figure 5C] 1 shows exemplary feature importance metrics for a variety of different use cases and models. [Figure 5D] 1 shows exemplary feature importance metrics for a variety of different use cases and models. [Figure 6A] 13A-13C show screenshots provided by an exemplary user interface for setting use case parameters and analyzing predicted outputs, respectively. [Figure 6B] 13A-13C show screenshots provided by an exemplary user interface for setting use case parameters and analyzing predicted outputs, respectively. [Figure 7] FIG. 1 is a flow diagram of an exemplary method for facilitating the selection of a master cell line from among candidate cell lines producing a recombinant protein. [Figure 8] FIG. 2 is a simplified block diagram of an exemplary system capable of implementing the techniques of the second aspect of the invention described herein. [Figure 9] 1 is an exemplary graphical output showing the relationship between cell number and cell productivity score for a selection of cell lines. [Figure 10] 1 illustrates an exemplary process for generating and evaluating a machine learning model. [Figure 11A] 1 illustrates an example output from a regression estimator that may be used for feature reduction. [Figure 11B] 1 illustrates an example output from a regression estimator that may be used for feature reduction. [Figure 12A] Shows observed model performance and / or feature importance for various models and target product quality attributes. [Figure 12B] Shows observed model performance and / or feature importance for various models and target product quality attributes. [Figure 12C] Shows observed model performance and / or feature importance for various models and target product quality attributes. [Figure 12D] Shows observed model performance and / or feature importance for various models and target product quality attributes. [Figure 12E] Shows observed model performance and / or feature importance for various models and target product quality attributes. [Figure 12F]Shows observed model performance and / or feature importance for various models and target product quality attributes. [Figure 12G] Shows observed model performance and / or feature importance for various models and target product quality attributes. [Figure 13A] A comparison of model predicted rankings with rankings based on real-world fed-batch cultures is shown. [Figure 13B] A comparison of model predicted rankings with rankings based on real-world fed-batch cultures is shown. [Figure 13C] A comparison of model predicted rankings with rankings based on real-world fed-batch cultures is shown. [Figure 14] FIG. 1 is a flow diagram of an exemplary method for facilitating the selection of a cell line to proceed to the next stage of cell line screening among multiple candidate cell lines producing a recombinant protein. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0018] The various concepts introduced above and discussed in more detail below can be implemented in any of numerous ways, and the concepts described are not limited to any particular manner of implementation. Example embodiments are provided for illustrative purposes.

[0019] 2 is a simplified block diagram of an exemplary system 100 that may implement the techniques of the first aspect described herein. The system 100 includes a computing system 102 communicatively connected to a training server 104 via a network 106. In general, the computing system 102 is configured to use one or more machine learning (ML) models 108 trained by the training server 104 to predict large-scale (bioreactor) cell culture performance (e.g., productivity and / or product quality attributes) of specific cell lines based on small-scale culture measurements of those cell lines and optionally other parameters (e.g., modalities).

[0020] The network 106 may be a single communication network or may include multiple communication networks of one or more types (e.g., one or more wired and / or wireless local area networks (LANs) and / or one or more wired and / or wireless wide area networks (WANs), such as the Internet). In various embodiments, the training server 104 trains and / or uses the ML models 108 as a "cloud" service (e.g., Amazon Web Services), or the training server 104 may be a local server. However, in the illustrated embodiment, the ML models 108 are trained by the server 104 and transferred to the computing system 102 over the network 106 as needed. In other embodiments, one, some, or all of the ML models 108 may be trained on the computing system 102 and then uploaded to the server 104. In still other embodiments, the computer system 102 trains and maintains / stores the models 108, in which case the system 100 may omit both the network 106 and the training server 104.

[0021] 2 illustrates a scenario in which the computing system 102 makes predictions based on measurements of a particular small-scale cell culture 110. The culture 110 may be, for example, a culture of a particular cell line (e.g., derived from Chinese Hamster Ovary (CHO) cells) in a single container such as a well or vial. The cell line of the culture 110 may be any suitable cell line that produces recombinant proteins and may be of any particular modality. The cell line may be, for example, a monoclonal antibody (mAb) producing cell line or a cell line that produces bispecific or other multispecific antibodies. It will be appreciated that the computing system 102 may also make predictions based on measurements of cells cultured in a microfluidic environment, such as an opto-electronic device as described herein.

[0022] The one or more analytical instruments 112 are collectively configured to obtain physical measurements used by the computing system 102 to make predictions, as discussed further below. The analytical instruments 112 may obtain measurements directly and / or obtain or facilitate obtaining indirect or "soft" sensor measurements. As used herein, the term "measurement" may refer to a value that is directly measured / sensed by an analytical instrument (e.g., one of the instruments 112), a value that the analytical instrument calculates based on one or more direct measurements, or a value that another device (e.g., the computing system 102) calculates based on one or more direct measurements. The analytical instruments 112 may include fully automated instruments and / or instruments that require human assistance. By way of example only, the analytical instrument 112 may include one or more chromatographic instruments (e.g., instruments configured to perform size exclusion chromatography (SEC), cation exchange chromatography (CEX) and / or hydrophilic interaction chromatography (HILIC)), one or more instruments configured to obtain measurements to determine the titer of a target product, one or more devices configured to directly or indirectly measure metabolite concentrations in culture media (e.g., glucose, glutamine, etc.), and the like.

[0023] The computer system 102 may be a general-purpose computer specially programmed to perform the operations discussed herein, or may be a dedicated computing device. As can be seen in FIG. 2, the computing system 102 includes a processing unit 120, a network interface 122, a display 124, a user input device 126, and a memory unit 128. However, in some embodiments, the computing system 102 includes two or more computers that are co-located with each other or remote from each other. In these distributed embodiments, the operations described herein related to the processing unit 120, the network interface 122, and / or the memory unit 128 may be divided among multiple processing units, network interfaces, and / or memory units, respectively.

[0024] The processing unit 120 includes one or more processors, each of which may be a programmable microprocessor that executes software instructions stored in the memory unit 128 to perform some or all of the functionality of the computing system 102 as described herein. The processing unit 120 may include, for example, one or more central processing units (CPUs) and / or one or more graphics processing units (GPUs). Alternatively or in addition, some of the processors in the processing unit 120 may be other types of processors (e.g., application specific integrated circuits (ASICs), field programmable gate arrays (FPGAs), etc.), and some of the functionality of the computing system 102 described herein may instead be implemented in hardware. The network interface 122 may include any suitable hardware (e.g., front-end transmitter and receiver hardware), firmware, and / or software configured to communicate with the training server 104 over the network 106 using one or more communication protocols. For example, the network interface 122 may be or include an Ethernet interface that enables the computing system 102 to communicate with the training server 104 over the Internet, an intranet, or the like.

[0025] The display 124 may use any suitable display technology (e.g., LED, OLED, LCD, etc.) to present information to a user, and the user input device 126 may be a keyboard or other suitable input device. In some embodiments, the display 124 and the user input device 126 are integrated into a single device (e.g., a touch screen display). In general, the display 124 and the user input device 126 may be combined to allow a user to interact with a graphical user interface (GUI) provided by the computing system 102 (e.g., an interface such as that described below with reference to FIGS. 6A and 6B ). However, in some embodiments, the computing system 102 does not include the display 124 and / or the user input device 126, or one or both of the display 124 and the user input device 126 are included in another computer or system (e.g., a customer device) that is communicatively coupled to the computing system 102.

[0026] The memory unit 128 may include one or more volatile and / or non-volatile memories. It may include one or more of any suitable memory types, such as read only memory (ROM), random access memory (RAM), flash memory, solid state drive (SSD), hard disk drive (HDD), etc. Collectively, the memory unit 128 may store one or more software applications, data received / used by those applications, and data output / generated by those applications. These applications include a large-scale predictive application 130 that, when executed by the processing unit 120, predicts the performance (e.g., productivity and / or product quality attributes) of a particular cell line in a hypothetical / hypothetical large-scale culture based on small-scale measurements obtained by the analytical instrument 112 (and possibly also based on other information such as modality). Various modules of the application 130 are described below, although it will be understood that those modules may be distributed among different software applications and / or the functionality of any one of such modules may be divided among two or more software applications.

[0027] The data collection unit 132 of the application 130 collects values ​​of various attributes associated with a small scale cell culture, such as the culture 110. For example, the data collection unit 132 can receive measurements directly from the analytical instrument 112. Additionally or alternatively, the data collection unit 132 can receive information stored in a measurements database (not shown in FIG. 2 ) and / or information entered by a user (e.g., via the user input device 126). For example, the data collection unit 132 can receive information such as modality, target drug product, drug protein scaffold type, and / or any other suitable information entered by a user and / or stored in a database. Additionally or alternatively, the data collection unit can receive measurements from an opto-electronic device, as described herein.

[0028] For a given small-scale cell culture corresponding to a particular cell line, the prediction unit 134 of the application 130 operates based on the attribute values ​​collected by the data collection unit 132 and uses a local machine learning model 136 to output one or more predicted attribute values ​​corresponding to a hypothetical / hypothetical large-scale culture. That is, the attribute values ​​collected by the data collection unit 132 are used as inputs / features for the machine learning model 136. The attributes whose values ​​are predicted may include one or more productivity metrics (e.g., titer) and / or one or more product quality metrics (e.g., SEC main peak, low molecular weight peak and / or high molecular weight peak percentage). In the illustrated embodiment, the machine learning model 136 is a local copy of one of the models 108 trained by the training server 104, which may be stored, for example, in the RAM of the memory unit 128. However, as mentioned above, the server 104 may utilize all of the models 108 in other embodiments, in which case a local copy need not be present in the memory unit 128.

[0029] The visualization unit 138 of the application 130 generates a user interface that allows the user to input information indicative of the use case (e.g., which large-scale attribute values ​​to predict, modality, etc.) via the user input device 126, and allows the user to observe a visual representation of the predictions made by the prediction unit 134 (and / or other information derived therefrom) via the display 124. Exemplary screenshots of user interfaces that may be generated by the visualization unit 138 are described below with reference to Figures 6A and 6B.

[0030] The operation of system 100 according to one embodiment will now be described in further detail with respect to a particular scenario in which application 130 is used to predict large scale performance for a number of different cell lines (clones) in small scale culture, including a particular cell line in small scale cell culture 110. By doing so, a better selection of cell lines may be identified for scale-up (e.g., for stage 14 in process 10 of FIG. 1 ), or the scale-up stage may be skipped entirely (e.g., by passing directly from stage 12 to stage 16 of process 10 based on predictions for the various cell lines).

[0031] Initially, the training server 104 trains the machine learning models 108 using data stored in the training database 140. The machine learning models 108 can include many different types of machine learning based regression estimators (e.g., decision tree regressor models, random forest regressor models, linear support vector regression models, extreme gradient boosting (xgboost) regressor models, etc.) and possibly also one or more models not based on regression (e.g., neural networks). Furthermore, in some embodiments, the models 108 can include more than one model of any given type (e.g., two or more models of the same type trained on different historical data sets and / or using different feature sets). Furthermore, different models of the models 108 can be trained to predict different large scale culture attribute values ​​(e.g., titer or chromatographic SEC values, etc.). As will be further discussed below with reference to FIGS. 4A and 4B, each of the machine learning models 108 can be optimized (trained and tuned) for a particular specification case or for a particular class of specification cases. Additionally, as discussed further below with reference to Figures 5A-5D, each of the machine learning models 108 can be used to identify which features (e.g., small scale culture attribute values) are most predictive of a particular large scale culture attribute value, and / or can be trained or retrained using a feature set that includes only those features that are most predictive of a particular large scale culture attribute value.

[0032] The training database 140 may include a single database stored in a single memory (e.g., HDD, SSD, etc.) or multiple databases stored in one or more memories. For each different model in the machine learning models 108, the training database 140 may store a corresponding set of training data (e.g., input / feature data and corresponding labels), possibly with overlap between training datasets. For example, to train a model that predicts titer, the training database 140 may include multiple feature sets (each of which includes historical small-scale culture measurements made by one or more analytical instruments (e.g., analytical instruments 112 and / or similar instruments) and possibly other information (e.g., modality)) along with a label for each feature set. In this example, the label for each feature set indicates a large-scale culture titer value (e.g., end-point titer at day 15) measured when the cell line from the small-scale culture was scaled up in a bioreactor. In some embodiments, all features and labels are numeric and non-numeric classifications or categories are mapped to numeric values ​​(e.g., allowed values ​​for modality feature / input [Bispecific Format 1, Bispecific Format 2, Bispecific Format 1 or 2] are mapped to values ​​[10, 01, 00]).

[0033] In some embodiments, the training server 104 uses additional labeled datasets in the training database 140 to validate the trained machine learning models 108 (e.g., to verify that a given one of the machine learning models 108 provides at least some minimum acceptable accuracy). Validation of models 108 is further discussed below with reference to FIG. 3. In some embodiments, the training server 104 also continually updates / refines one or more machine learning models 108. For example, after a machine learning model 108 has been initially trained to provide a sufficient level of accuracy, additional measurements, both small scale (features) and large scale (labels), may be used to improve predictive accuracy.

[0034] The application 130 may retrieve a particular one of the machine learning models 108 corresponding to a desired use case from the training server 104 via the network 106 and the network interface 122. The use case may be, for example, one indicated by a user via a user interface (e.g., as described below with reference to FIG. 6A). Upon retrieving the model, the computing system 102 saves a local copy as a local machine learning model 136. In other embodiments, as described above, the model is not retrieved and instead the input / feature data is sent to the training server 104 (or another server) as needed to use the appropriate one of the models 108.

[0035] The data collection unit 132 collects the necessary data according to the feature set used by the model 136. For example, the data collection unit 132 may communicate with the analytical instruments 112 to collect measurements of titers, chromatographic values, metabolite concentrations, and / or other specific attributes of the small scale cell culture 110. In one such embodiment, the data collection unit 132 sends commands to one or more analytical instruments 112, causing the one or more instruments to automatically collect the desired measurements. In another embodiment, the data collection unit 132 collects measurements of the small scale cell culture 110 by communicating with a different computing system (not shown in FIG. 2 ) that is coupled to (and optionally controls) the analytical instruments 112. As mentioned above, the data collection unit 132 may also receive information (e.g., modality, target drug product, etc.) entered by a user. In some embodiments, some user-inputted information collected by the data collection unit 132 is used to select an appropriate one of the models 108, while other user-inputted information collected by the data collection unit 132 is used as (or used to obtain) one or more features / inputs into the selected model.

[0036] After the data collection unit 132 collects attribute values ​​associated with the small scale cell cultures 110 (and possibly other data such as target drug products) and used as inputs / features by the local machine learning model 136, the prediction unit 134 causes the model 136 to operate based on those inputs / features and output predictions of one or more large scale cell culture attribute values ​​for the same cell line. It should be appreciated that in some embodiments and / or scenarios, the local machine learning model 136 may include two or more models, each predicting / outputting a different large scale culture attribute value.

[0037] The large scale culture attribute values ​​output by the model 136 may include, for example, one or more productivity attributes, such as titer or viable cell density (VCD), and / or one or more product quality attribute values, such as SEC main peak (MP) percentage, SEC low molecular weight (LMW) peak percentage, and / or SEC high molecular weight (HMW) peak percentage. The visualization unit 138 may cause a user interface depicted on the display 124 to present the predicted attribute values ​​and / or other information derived from the predicted attribute values. For example, the visualization unit 138 may cause the user interface to present an indication of whether the predicted attribute values ​​meet one or more cell line selection criteria (e.g., after the application 130 compares the attribute values ​​to one or more respective thresholds).

[0038] The above process can be repeated for many different cell lines, each of which is used for a small scale cell culture similar to small scale cell culture 110. For example, computing system 102 (or another computing system not shown in FIG. 2) may cause analytical instrument 112 to continuously acquire measurements from hundreds or thousands of small scale cell cultures (each of which comprises a different clone / cell line), and prediction unit 134 may cause model 136 to operate on each set of measurements (and possibly other data) to output a respective large scale prediction or set of predictions.

[0039] The prediction unit 134 may cause the predictions made by the model 136 for each cell line and / or information obtained from each set of predictions to be stored in the memory unit 128 or another suitable memory / location. After the predictions have been made and stored for all cell lines under consideration, a "winning" cell line may be selected (e.g., similar to stage 16 of FIG. 1). The selection of the winning cell line may be fully automated according to some criteria specific to the use case (e.g., by assigning specific weights to productivity and product quality attributes and then comparing the scores) or may involve human interaction (e.g., by simply displaying the predicted large-scale attribute values ​​to the user via the display 124). Alternatively, after the predictions have been made and stored for all cell lines under consideration, a subset of cell lines may be selected for scale-up (e.g., similar to stage 14 of FIG. 1). Again, this selection may be fully automated according to some criteria specific to the use case or may involve human interaction.

[0040] As described above, the training server 104 can train a number of different predictive models 108 that are particularly well suited to a particular specification case or a particular class of specification cases. Additionally, interpretable machine learning models can be used to avoid the time and cost of having to perform and collect a large number of small-scale analytical measurements (and possibly other information). For example, the training server 104 can train one of the models 108 on hundreds of features (e.g., about 600 features), after which the training server 104 (or a human reviewer) can analyze the trained model (e.g., the weights assigned to each feature) to determine the most predictive features (e.g., about 10 features). That particular model or a new version of that model trained using only the most predictive features can then be used with a much smaller feature set. Identifying highly predictive features can also be useful for other purposes, such as providing new scientific insights that can generate new hypotheses (which can in turn lead to improvements in the bioprocess).

[0041] Various techniques for determining which model is best suited for a particular use case and for identifying the most predictive features for a given model or use case are described below with reference to FIGS.

[0042] Generally, a model that performs well for a particular use case can be identified by training many different models and comparing their results using historical training data generated from previous clone screening runs. Historical data can include small-scale cell line development data (e.g., small-scale fed-batch measurement data) and scaled-up bioreactor data (e.g., perfusion bioreactor measurements). Additionally, historical data can include both categorical data, such as media type and modality, and numerical data, such as metabolite concentrations and titer values. For small-scale cell line development data (also referred to herein simply as "cell line development data" or "CLD data"), growth factors such as viability, VCD, and glucose concentration can be collected periodically over time (e.g., on different days of a 10-day culture). For scaled-up bioreactor data (also referred to herein as "bioprocess development data" or "BD data"), these attributes and possibly additional attributes, such as pH level and dissolved oxygen concentration, can be collected and recorded in association with each feature set. The bioreactor data may also include data that serve as labels for various feature sets, such as product titers and other analytical results from the assays (e.g., results of SEC and / or CEX analyses).Various measures can be taken to ensure a robust training data set (e.g., providing standardized heterogeneous data, removing outliers, imputing missing values, etc.).

[0043] In some embodiments, special feature engineering techniques are used to extract or derive useful features. For example, convolutional neural networks (or APIs that automatically extract summary statistics from time data, such as tsfresh) can be used to detect time dependencies between various attributes (e.g., high correlation between VCD on day 0 of small-scale culture and VCD on day 6 of small-scale culture). These time dependencies can be used to extract / derive useful features for model training. Other feature engineering techniques can also be used, such as variance thresholding, principal component analysis (PCA), mutual information regression, analysis of variance (ANOVA), and removal of features with high covariance.

[0044] In any supervised machine learning regression model, generated using historical training data, the task is to convert the input / feature data x into predicted values.

number

number

number

[0045] FIG. 3 shows a modular and flexible process 200 that can be used as a framework for identifying models that perform well for each of a number of different use cases. Initially, in stage 202, relevant data corresponding to a given specification case is selected from among the available historical data. A "specification case" can be defined in a variety of ways, depending on how one determines which data is relevant for that use case. For example, a use case can be defined as a specific target variable (y), a specific modality or set of modalities, and possibly one or more specific constraints on the feature data set. As a more specific example, a use case can correspond to (1) end point titers for large-scale cultures (bioreactors) as target variables, (2) all modalities (e.g., monoclonal antibodies and any bispecific or multispecific formats that may be considered), and (3) using only cell line development historical data as (and / or to derive) features for the training data. Conversely, another use case may correspond to using (1) chromatographic analysis results for large scale cultures as target variables (e.g., SEC main peak), (2) a single modality (e.g., a particular monoclonal antibody or bispecific or multispecific antibody format), and (3) both cell line development history data and bioreactor history data as (and / or to derive) training data features.

[0046] A model library for the use case is registered in stage 204. Stage 204 involves the selection of a number of candidate machine learning models / estimators that may or may not turn out to be particularly suitable for predicting target attribute values ​​for the use case. In order to obtain accurate and interpretable results, some or all of the machine learning models selected in stage 204 should satisfy two criteria. First, machine learning models that can assign weights to input features are preferred because they can account for the relative importance of each input feature with respect to predicting the target output. Second, sparsity-induced machine learning models are preferred (e.g., models that initially accept many attribute values ​​as features, but only require a small subset of these attribute values ​​as features to make accurate predictions). This property reduces overfitting by eliminating features that do not significantly affect the target outcome, while also improving interpretability. Sparsity-induced models can also save time and costs because there is no need to measure the excluded attribute values. Regression models / estimators based on decision trees (e.g., decision / ID tree models, random forest models, xgboost models, gradient boosting models, etc.) or other machine learning algorithms (e.g., support vector machines (SVMs) with linear and / or radial basis function kernels, elastic nets, etc.) are particularly suitable to meet both criteria above. Although not traditionally considered to be interpretable, in some embodiments one or more neural networks may also be selected in stage 204.

[0047] In stage 206, the machine learning pipeline is designed to train each model considered for the use case (i.e., each model selected for the library in stage 204). For example, stage 206 may include performing k-fold validation for each model (e.g., k=10 if the model is trained and evaluated 10 times across different 90 / 10 partitions of the dataset selected in stage 202). Within the machine learning pipeline, the dataset selected in stage 202 may first be transformed by standard scaling, such as by normalizing the mean of each feature with zero (μ=0) and standard deviation with one (σ=1). This allows the importance of each feature to be considered on an equal basis, without being biased by unequal magnitudes of raw values ​​corresponding to different features.

[0048] After normalization, the hyperparameters of the model are tuned. For example, a Bayesian search technique can be used to tune the hyperparameters. This technique performs a Bayesian guided search that is computationally more efficient than grid search or random search, but obtains a similar level of performance as random search. Relatively simple algorithms, such as non-boosting and non-neural network algorithms, may use a relatively small number of iterations of Bayesian search (e.g., 10), while more complex algorithms, such as gradient boosting, xgboost, and neural network algorithms, may use a relatively large number of iterations of Bayesian search (e.g., 30) due to the higher dimensional search space. The hyperparameters may be selected through k-fold validation (e.g., k=5). Each trained model with tuned hyperparameters is then evaluated using a test dataset. The coefficient of determination (R 2 ) and root mean square error (RMSE). The RMSE can be calculated as follows:

number

number

[0049] In stage 208, the best model for the use case is selected according to some criteria. For example, the "best" model may be the model that has the lowest average RMSE over 10 cross-validation folds after 90 / 10k-fold validation (per Equation 3 above) among all the models used to enroll the model library in stage 204 and trained in stage 206. RMSE is chosen based on the R 2 It can be a better metric than R 2 The metric can sometimes take extreme negative values ​​in some cross-validation sets, which can distort the dynamics of model comparisons when averaged. RMSE may be utilized more than Mean Absolute Error (MAE) to penalize larger errors between predictions and actuals.

[0050] Then, in stage 210, a final production model for the use case is output. The final production model may be of the same type as the model selected in stage 208, but may be retrained on the entire dataset selected in stage 202 to obtain better (e.g., optimal) hyperparameters. By training on the entire dataset, the final production model may generalize better and may exhibit a similar or higher level of average accuracy compared to that obtained during cross-split validation. The final production model is then saved as a trained model and is ready to make predictions for new experiments.

[0051] In one embodiment, the process 200 is performed by the training server 104 of FIG. 2 (possibly with human input at various stages, such as defining the use case and / or registering candidate models in a model library). The process 200 may be repeated for each use case and for any suitable number of use cases (e.g., 5, 10, 100, etc.). As final production models for different use cases are output at each iteration of stage 210, the training server 104 may add those final production models to the machine learning models 108. Thereafter, and before making predictions for different clones / cell lines of the small-scale cell culture (e.g., culture 110) in the manner discussed above with reference to FIG. 2, the computing system 102 or the training server 104 may select an appropriate final production model from the models 108. This selection may be based on a user input indicating the use case (e.g., as described below with reference to FIG. 6A) and based on an algorithm or mapping (e.g., performed by application 130) that matches the user-specified use case to the final production model. Alternatively, if an exact match does not exist, such an algorithm may match the user-specified use case to a final production model of model 108 adapted to the use case that is most similar to the user-specified use case (e.g., as determined by calculating vector distances between the numerical parameters that define the use case, where categorical parameters such as modality are mapped to numerical values).

[0052] As mentioned above, it can be advantageous to reduce the number of features required for a particular model. Thus, when the "best" model from stage 208 is retrained in stage 210, only the features that are most predictive of the desired output (e.g., titer, etc.) can be utilized. To identify a smaller feature set, process 200 can perform recursive feature elimination (RFE), which allows for a recursive reduction of explanatory features used in the final production model, discarding the least important features. The RFE algorithm trains on the data by utilizing a subset of features to obtain optimal model performance with respect to a constraint on the number of features. Pairing RFE with a sparsity-guided model / estimator, such as a decision tree or elastic net, can further reduce the number of explanatory features with a trade-off that increases interpretability at the expense of model accuracy. Through RFE, elbow plots can be used to determine the "sweet spot" or inflection point between interpretability and accuracy.

[0053] In addition to determining the accuracy of each model in the model library, it may be important to know the prediction interval (also known as the "confidence" interval). For example, a model with slightly less accuracy may be preferred over a model with more accuracy, if the less accurate model has a much tighter prediction interval. However, complex machine learning algorithms may only generate point predictions without intervals. Therefore, in some embodiments, a conformal prediction framework is utilized. Conformal prediction intervals allow for the assignment of error bounds to each new observation and can be used as a wrapper for any machine learning estimator. This framework is applicable when the training data and the test data are assumed to come from the same distribution. If this exchangeability condition is met, a subset of the training data can be utilized to construct a non-fitting function against which the underlying sample distribution is measured.

[0054] In one embodiment, the "misfit" API is used with an induced conformal prediction framework, which allows training the model only once, just before prediction intervals are generated in parallel for all new observations. The induced conformal prediction framework requires a disjoint calibration set of the training set. This helps to build robust prediction intervals, but removing samples from the training set to build the misfit function reduces the statistical power of the model. A normalization process (e.g., by a KNN-based approach) can be used to generate specific decision boundaries for each prediction.

[0055] The prediction intervals generated by the conformal forecasting framework include a proportion of future observations equal to 1-α, where α is the significance level, but the width of the intervals generated depends heavily on the underlying function. Naturally, narrower intervals yield greater confidence in the point forecast.

[0056] 4A and 4B show exemplary model performance (here RMSE over 10 folds of cross-validation) for many different use cases. In all use cases shown, the target variables (attribute values) are either large-scale (bioreactor) endpoint titers or large-scale SEC analysis metrics. Bioreactor endpoint titers can represent product concentration yields from cell culture fluid (HCCF) harvested on the last day (e.g., day 15) of perfusion bioreactor culture. This is a weighted average composite titer from culture supernatant and perfusion permeate. Endpoint titers are used to evaluate productivity. SEC analysis evaluates the chromatographic peak profile of the product based on the size of the protein. Three elution peaks are usually separated into three classifications: low molecular weight (LMW), main peak (MP) and high molecular weight (HMW). High quality clones ideally have high SEC MP, low SEC LMW and low SEC HMW. MP represents usable product, LMW represents cleavage clipping and HMW represents assembled aggregates. SEC is one of several core analyses typically used to assess product quality.

[0057] In Figures 4A and 4B, "CLD" refers to cell line development indicating that for that use case, small scale culture data is used to train the model, and "BD" refers to bioprocess development indicating that for that use case, large scale culture data is also used to train the model. Thus, for example, the use case "Titer-All Modalities-CLD" is one in which the target attribute value is bioreactor endpoint titer, all modalities (e.g., mAbs and bispecific or multispecific antibodies) are included, and only small scale culture data is used to train the model. For each model within each plot, the thin horizontal line (with short vertical lines at each end) represents the full RMSE range across the 10-fold cross-validation, the thick horizontal lines represent the + / - standard deviation range for the RMSE, and the vertical lines within the thick horizontal lines represent the average RMSE across all 10 folds.

[0058] For example, as can be seen in FIG. 4A, the Random Forest regressor model provides the lowest average RMSE for the use cases “Potency-All Modalities-CLD” and “Potency-Bispecific-CLD”, the xgboost model provides the lowest average RMSE for the use cases “Potency-mAb-CLD” and “Potency-All Modalities-CLD+BD”, the decision tree model provides the lowest average RMSE for the use case “Potency-Bispecific-CLD+BD”, and the SVM (Linear Kernel) model provides the lowest average RMSE for the use case “Titer-mAb-CLD+BD”. As can be seen in FIG. 4B, the xgboost model provides the lowest average RMSE for the use cases “SEC MP-all modalities-CLD”, “SEC MP-bispecific-CLD”, “SEC MP-mAb-CLD”, “SEC MP-all modalities-CLD_BD” and “SEC MP-mAb-CLD+BD”, while the SVM (linear kernel) model provides the lowest average RMSE for the use case “SEC MP-bispecific-CLD+BD”.

[0059] Although not shown in FIG. 4B, similar results can be obtained for SEC HMW and SEC LMW. For the target attribute value of SEC HMW, the decision tree model provides the lowest average RMSE for the use cases "SEC HMW-All Modalities-CLD", "SEC LMW-All Modalities-CLD", "SEC LMW-Bispecific-CLD" and "SEC LMW-All Modalities-CLD+BD", the xgboost model provides the lowest RMSE for the use cases "SEC HMW-Bispecific-CLD", "SEC HMW-mAb-CLD", "SEC HMW-Bispecific-CLD+BD", "SEC HMW-mAb-CLD+BD" and "SEC LMW-Bispecific-CLD+BD", the random forest model provides the lowest RMSE for the use case "SEC HMW-All Modalities-CLD+BD", the elastic net provides the lowest RMSE for the use case "SEC LMW-mAb-CLD", and the SVM (Linear Kernel) model provides the lowest RMSE for the use case "SEC LMW-mAb-CLD+BD".

[0060] In some embodiments, the application 130 of the computing system 102 of FIG. 2 determines a use case (target attribute values, modality, and type of dataset) for a given collection of candidate clones / cell lines based on user input (e.g., entered via the display 124) and requests a corresponding one of the models 108 from the training server 104. For example, the models 108 may include all of the "lowest average RMSE" models listed above, and the server 104 or computing system 102 may store a database that associates each of these models with the use case (or use cases) for which the model provided the lowest average RMSE. The server 104 or computing system 102 may then access the database to select the best model appropriate for the determined use case. In an alternative embodiment, the computing system 102 sends data indicative of the use case to the training server 104, and in response, the training server 104 selects a corresponding one of the models 108 and sends the model to the computing system 102 for storage as a local machine learning model 136. In yet other embodiments, as discussed above, the selected model may be utilized remotely from the computing system 102 (eg, at the server 104).

[0061] In some cases, a user may wish to test more than one use case to select a winning clone or to select a set of clones to be scaled up in a bioreactor for further screening. In these cases, application 130 (or a remote server such as server 104) may select and run multiple models that are all used to make large-scale predictions for each clone / cell line. For example, a user may wish to consider both titer and SEC main peak at large scale when selecting a winning clone. Thus, application 130 may select and / or run a first machine learning model (e.g., a random forest model) for the use case corresponding to endpoint titer and a second machine learning model (e.g., an xgboost model) for the use case corresponding to SEC main peak. As another example, a user may wish to consider titer, SEC main peak, SEC low molecular weight, and SEC high molecular weight at large scale when selecting a winning clone, and application 130 may select and / or run a random forest model for titer, an xgboost model for SEC main peak, and a decision tree model for both SEC low molecular weight and SEC high molecular weight.

[0062] As mentioned above, an interpretable model may be preferred to identify which inputs / features are most predictive of a particular target attribute value. For example, a tree-based learning method may output metrics that indicate how important each feature is for the purpose of reducing the mean squared error of the model when the feature is used as a node in a decision tree. Additionally, a coefficient plot may represent the normalized directional coefficients that weight each input / feature when predicting the target attribute value.

[0063] Figures 5A-5D show exemplary feature importance metrics for a variety of different use cases and a variety of different models. Figure 5A shows feature importance plots and coefficient plots for models predicting large-scale (bioreactor) endpoint titers, and Figure 5B shows feature importance plots for titer predictions filtered by modality. From these two plots, we can see that "CLD-titer x SEC main peak - day 10" is a consistently high importance feature for models derived using only CLD (cell line development) data. We can also see that VCD is a particularly important attribute in predicting titers than specific productivity (denoted as "qp" and having units of pg per cell per day). This indicates that for the purpose of generating high titers, it is more important to have better cell growth in the culture than to have high specific productivity. The term "iVCD" in Figure 5A refers to the integrated VCD, which accounts for the total volume (cells x days) in the reactor.

[0064] Figure 5C shows feature importance plots and coefficient plots for models predicting large-scale (bioreactor) end-point SEC main peaks, and Figure 5D shows feature importance plots for SEC main peak predictions filtered by modality. These plots show that modality and modifications to the protein scaffold are important determinants of SEC main peaks. For example, day 0 CLD modality (converted to a numerical value) has a strong negative correlation with SEC main peaks, indicating that molecules corresponding to bispecific formats generally have lower predicted SEC main peaks. The term "project" in Figure 5D refers to an index of a specific project and therefore a specific product.

[0065] In some embodiments, the training server 104 of FIG. 2 trains any given one of the machine learning models 108 using the N most important features for the particular use case and model (N being a predefined positive integer such as 10 or a number that exceeds a threshold importance metric for all features, etc.), and only these N features are collected by the data collection unit 132 for processing by the local model 136. In some embodiments, N is determined using recursive feature elimination (RFE), as described above. Through RFE, the training server 104 may perform multiple iterations of training to reduce the final number of inputs / features used to make predictions. As described above, the ideal number of features (i.e., the number of features used to train the various models 108 used in production) may be selected by examining elbow plots that graph the number of features against model performance, e.g., with an inflection point in each such graph that represents the "sweet spot" between accuracy and interpretability.

[0066] For the features discussed above, any suitable attributes may be used (e.g., to initially train the various models and possibly train the final production model if the features are of sufficient importance). A non-exhaustive list of possible attributes / features for both the Cell Line Development (CLD) and Bioprocess Development (BD) datasets is shown in Table 1 below.

[0067] [Table 1]

[0068] [Table 2]

[0069] [Table 3]

[0070]

Table 4

[0071] As mentioned above, the one or more machine learning models (e.g., of model 108) selected (e.g., by application 130 or server 104) to make predictions for large-scale cultures may depend on a use case or set of use cases input by a user via a graphical user interface. FIG. 6A shows an example screenshot 400 of such a user interface, which application 130 may have presented, for example, on display 124. As seen in the example embodiment of FIG. 6A, the user interface may enable a user to (1) input two target attributes (i.e., large-scale bioreactor attributes to be predicted by the corresponding machine learning models), (2) indicate whether the input / features should include only cell line development data or both cell line development and bioprocess development (bioreactor) data, (3) indicate one or more modalities being considered, and (4) indicate a desired prediction / confidence interval. Based on the user input, the application 130 or server 104 can select an appropriate model from the models 108 to make the prediction, i.e., the final production model resulting from stage 210 of the process 200 for each of the use cases indicated by the user. In the exemplary screenshot 400, it can be seen that a single set of user inputs can correspond to two use cases (i.e., one for each of the two target attributes, each of which includes the same user-selected data set and modality). The selected models can be downloaded as local models (e.g., each similar to model 136) or can remain on the server 104 for use in the cloud service. The user's activation of the "predict" control is detected by the application 130 (or server 104), and in response, the application 130 (or server 104) applies the model to each feature set to predict each large-scale attribute value. It will be understood that in other embodiments, the user interface can be provided with different user controls than those shown in FIG. 6A.

[0072] The predictions made by the selected / applied model may be presented to the user in any suitable manner. An example of such a presentation is shown in screenshot 410 of FIG. 6B, which corresponds to an embodiment in which predictions for all clones / cell lines can be shown simultaneously. In FIG. 6B, each clone / cell line is plotted as a dark circle on a two-dimensional graph. In the results shown in the exemplary scenario of FIG. 6B, a user wanting a clone with a high SEC main peak and high titer would select (or alternatively have application 130 automatically select) one or both of the two clones in the upper right corner of the graph as the top clone. In some embodiments, application 130 also allows the user to toggle the display of prediction intervals for each prediction. Additionally, in some embodiments, application 130 allows the user to view feature importance and / or coefficient plots (e.g., plots similar to those shown in FIGS. 5A-5D) associated with the various models / predictions.

[0073] 7 is a flow diagram of an exemplary method 500 for facilitating selection of a master cell line from among candidate cell lines producing a recombinant protein. The method 500 may be performed by the processing unit 120 of the computing system 102 when executing software instructions of an application 130 stored in the memory unit 128, or may be performed by one or more processors of, for example, the server 104 (e.g., in the implementation of a cloud service).

[0074] At block 502, attribute values ​​associated with a small scale cell culture are received for a particular cell line. At least some of the received attribute values ​​are measurements of the small scale cell culture (e.g., one or more media properties such as endpoint titer, SEC MP, SEC LMW, SEC HMW, VCD, viability, glucose or other metabolite concentration, and / or any other CLD measurements shown in Table 1 above). In some embodiments, the attribute values ​​may be received from opto-electronic equipment as described herein. In some embodiments and / or scenarios, other data is also received at block 502, such as user input data (e.g., an identifier for the particular cell line, a modality of a drug produced using the particular cell line, an indication of a drug product produced using the particular cell line, and / or a protein scaffold type associated with a drug produced using the particular cell line). Additionally, in some embodiments, one or more attribute values ​​associated with a large scale cell culture may be received (e.g., in embodiments where a small scale culture is scaled up to perform a large scale measurement at day 0, to better predict large scale performance at day 15 without necessarily performing a full scale culture).

[0075] In some embodiments, the small scale culture attribute values ​​received at block 502 include measurements taken on different days of the small scale culture. For example, a first attribute value may be the titer value on day 10 of the small scale culture (e.g., the endpoint titer of a 10 day culture) and a second attribute value may be the VCD value on day 0 of the small scale culture. As a further example, a third attribute value may be the VCD value on day 6 of the small scale culture, and so on. In other exemplary embodiments, the combination of small scale measurements may be the same as or similar to that shown with the label "CLD" in any of the plots of Figures 5A-5D.

[0076] In block 504, for a particular cell line, one or more attribute values ​​associated with the virtual large-scale cell culture are predicted by analyzing the attribute values ​​received in block 502 (and optionally user input data) using at least a machine learning based regression estimator (e.g., a decision tree regression estimator, a random forest regression estimator, an xgboost regression estimator, a linear SVM regression estimator, etc.). The predicted attribute values ​​may include, for example, potency (e.g., endpoint potency) and / or one or more product quality attribute values ​​(e.g., chromatographic measurements such as SEC main peak, SEC LMW and / or SEC HMW).

[0077] In block 506, the predicted attribute values ​​and / or an indication of whether the predicted attribute values ​​meet one or more cell line selection criteria (e.g., above or below a certain threshold) are presented to the user via a user interface (e.g., a user interface corresponding to screenshot 410 of FIG. 6B) to facilitate selection of a desired cell line for use in drug product manufacturing. For example, a user may proceed directly from such a display to select a "winning" cell line, or may use the displayed information to identify which cell lines should be scaled up in real-world bioreactors for validation and / or further clonal screening (with selection of winning clones occurring at a subsequent stage).

[0078] In some embodiments, method 500 includes one or more additional blocks not shown in Figure 7. For example, method 500 may include two additional blocks, both of which occur before block 502: a first additional block in which data indicative of a use case is received from a user via a user interface (e.g., a user interface corresponding to screenshot 400 of Figure 6A), and a second additional block in which a machine learning based regression estimator (e.g., from among models 108) is selected from among a plurality of estimators (each of these estimators is designed / optimized for a different use case) based on the data indicative of the use case. For example, the user input data may indicate at least one of one or more attribute values ​​associated with the virtual large-scale cell culture, indicate a modality of the drug to be generated, and possibly other parameters (e.g., parameters indicative of a range of a dataset, such as the CLD and BD datasets discussed above).

[0079] In more specific embodiments and scenarios, the user input data indicative of the use case may include data indicative of at least one titer associated with the virtual large-scale cell culture, and block 504 may include analyzing the plurality of attribute values ​​using a decision tree regression estimator, a random forest regression estimator, an xgboost regression estimator, or a linear SVM regression estimator (e.g., according to the results discussed above in connection with FIG. 4A). As another specific embodiment and scenario, the user input data indicative of the use case may include data indicative of at least one chromatographic measurement (e.g., SEC main peak) associated with the virtual large-scale cell culture, and block 504 may include analyzing the plurality of attribute values ​​using an xgboost regression estimator (e.g., according to the results discussed above in connection with FIG. 4B).

[0080] In embodiments in which a machine learning based regression estimator is selected from among a plurality of estimators, method 500 may include an additional block in which, for each of the estimators, a feature set is determined that is most predictive of the output of the estimator, In such embodiments, block 502 may include receiving only those attribute values ​​that are included in the most predictive feature set.

[0081] 8 is a simplified block diagram of an exemplary system 800 that may implement the techniques of the second aspect described herein. The system 800 includes a computing system 802 communicatively coupled to a training server 804 via a network 806. In general, the computing system 802 is configured to use one or more machine learning (ML) models 808 trained by the training server 804 to determine / predict a ranking of candidate cell lines according to each of one or more product quality attributes (e.g., specific productivity, titer, and / or cell growth) in a virtual small-scale screening culture (e.g., fed-batch culture) based on measurements by a clone (or cell line) generation and analysis system 850 and measurements on one or more cell pools 810.

[0082] Network 806 may be similar to network 106 of FIG. 2 and / or training server 804 may be similar to training server 104. In the illustrated embodiment, machine learning models 808 are trained by training server 804 and then transferred to computing system 802 over network 806 as needed. However, in other embodiments, one, some, or all of ML models 808 may be trained on computing system 802 and then uploaded to server 804. In other embodiments, computing system 802 trains and maintains / stores ML models 808, in which case system 800 may omit both network 806 and training server 804. In yet another embodiment, training server 804 provides access to models 808 as a web service (e.g., computing system 802 provides input data that server 804 uses to make predictions with one or more models 808, and server 804 returns the results to computing system 802).

[0083] Each of the cell pools 810 can be, for example, a pool of transfected cells (e.g., Chinese Hamster Ovary (CHO) cells) in a single container such as a well or vial. The cell pools 810 can be any suitable pool of cells that produce a recombinant protein and that have been scaled up through successive cell passages in a selective growth medium, and can be of any modality. The cells can be, for example, cells that produce a recombinant protein such as a monoclonal antibody (mAb) or cells that produce a recombinant protein such as a bispecific or other multispecific antibody. However, in general, the cells of each of the pools 810 are not all derived from a clone.

[0084] The one or more analytical instruments 812 are collectively configured to obtain physical measurements of the cell pool 810 that may be used by the computing system 802 to make predictions, as discussed further herein. The analytical instruments 812 may obtain measurements directly and / or obtain or facilitate obtaining indirect or "soft" sensor measurements. As discussed above, the term "measurement" as used herein may refer to a value that is directly measured / sensed (e.g., by one of the instruments 812), a value that is calculated based on one or more direct measurements, or a value that a device other than the measurement device (e.g., the computing system 802) calculates based on one or more direct measurements. The analytical instruments 812 may be similar to the analytical instruments 112 of FIG. 2, such as chromatographic or optical sensors described herein. The analytical instruments 812 may include, for example, one or more devices specifically configured to measure cell pool viable cell density (VCD), cell pool viability (VIA), time integrated viable cell density (IVCD), and cell pool specific productivity.

[0085] The clone generation and analysis system 850 can be any suitable (preferably high-throughput) subcloning system. In some embodiments, the clone generation and analysis system 850 is a Berkeley Lights Beacon system. As can be seen in FIG. 8, the system 850 includes an analysis unit 852 and a cell line generation and propagation unit 854. The cell line generation and propagation unit 854 can be a culture chip that includes multiple physically isolated pens that are perfused by a microfluidic channel. The unit 854 can be, for example, an OptoSelect™ Berkeley Lights chip. Each pen can receive transfected cells from a cell pool using a light projection pattern that activates a photoconductor that gently flicks the cells to manipulate them (e.g., as provided by Berkeley Lights OptoElectro™ positioning technology) and contains the cells (and other generated cells of the cell line) throughout the cell line generation and analysis process.

[0086] The analysis unit 852 of the cell line generation and analysis system 850 is configured to measure physical properties of cells in the clone generation and expansion unit 854. The analysis unit 852 can include one or more sensors or instruments for directly obtaining measurements and / or can obtain or facilitate the obtaining of indirect or "soft" sensor measurements. The instruments of the analysis unit 852 can include fully automated instruments and / or instruments requiring human assistance. By way of example only, the instruments of the analysis unit 852 (e.g., sensors or other instruments integrated within or interfaced with the unit 854) can include one or more imaging devices (e.g., cameras and / or microscopes) and associated software configured to directly or indirectly measure cell number or cell proliferation, as well as one or more devices configured to directly or indirectly measure cell productivity by performing a secretion assay (e.g., a diffusion-based fluorescent assay that binds to antibodies produced by cells on a chip, such as a secretion assay using the Spotlight HuIg2 assay (or Spotlight assay)).

[0087] Computing system 802 may be, for example, a general-purpose computer similar to computing system 102. As can be seen in Figure 8, computing system 802 includes a processing unit 820, a network interface 822, a display 824, a user input device 826, and a memory unit 828. Processing unit 820, network interface 822, display 824, and user input device 826 may be, for example, similar to processing unit 120, network interface 122, display 124, and user input device 126, respectively, of Figure 2.

[0088] The memory unit 828 may be similar to the memory unit 128 of FIG. 2. Collectively, the memory unit 828 may store one or more software applications, data received / used by those applications, and data output / generated by those applications. These applications include a small-scale predictive application 830 that, when executed by the processing unit 820, ranks candidate cell lines according to each of one or more product quality attributes (e.g., specific productivity, titer, and / or cell growth) in a virtual small-scale screening culture (e.g., stage 12 of FIG. 1 ) based on measurements obtained by the analytical instruments 812 and the analytical unit 852, and possibly also based on other information (e.g., modality, cell pool identifier, etc.). Various units of the application 830 are discussed below, although it will be understood that those units may be distributed among different software applications and / or the functionality of any one of such units may be divided among two or more software applications.

[0089] In some embodiments, the computing system 802, training server 804 and network 806 are computing system 102, training server 104 and network 106, respectively, and the memory units (128 and 828) store both a small scale prediction application 830 and a large scale prediction application 130. That is, the systems (10 and 800) may be capable of predicting both small scale and large scale performance, and FIG. 8 represents a different use case than that shown in FIG. 2.

[0090] The data collection unit 832 of the application 830 generally collects values ​​of various attributes associated with the cell pool 810 and the cell line generation and expansion unit 854. For example, the data collection unit 832 can receive measurements directly from the analytical instrument 812 and / or the analytical unit 852. Additionally or alternatively, the data collection unit 832 can receive information stored in a measurements database (not shown in FIG. 8 ) and / or information entered by a user (e.g., via the user input device 826). For example, the data collection unit 832 can receive modality, target drug product, drug protein scaffold type, and / or any other suitable information entered by a user and / or stored in a database.

[0091] The prediction unit 834 of the application 830 generally operates based on the attribute values ​​collected by the data collection unit 832 and uses a local machine learning model 836 to predict product quality attribute values ​​of virtual small-scale screening cultures of different candidate cell lines and to rank the cell lines using the predicted values. In the illustrated embodiment, the machine learning model 836 is a local copy of one of the models 808 trained by the training server 804 and may be stored, for example, in the RAM of the memory unit 828. However, as mentioned above, the server 804 may utilize / execute the model 808 in other embodiments, in which case a local copy need not be present in the memory unit 828.

[0092] A visualization unit 838 of the application 830 generates a user interface that presents the rankings (determined by the prediction unit 834) to the user. The visualization unit 838 may also enable the user, via the user input device 826 and the display 824, to interact with the presented data from the prediction unit 834 and / or to input parameters for a particular prediction or ranking (e.g., selecting product quality attributes according to which predicted performances should be ranked, etc.).

[0093] The operation of system 800 according to one embodiment will now be described in further detail for a particular scenario in which application 830 is used to determine one or more cell line rankings according to one or more small scale culture product quality attributes. Ranking cell lines in this manner may standardize the methodology for selecting top cell lines, may identify a better selection of cell lines for small scale screening, or may skip the small scale screening stage altogether (e.g., by skipping directly from stage 11 to stage 14 of process 10 based on the rankings of the various cell lines).

[0094] Initially, the training server 804 trains the machine learning models 808 using the data stored in the training database 840. The machine learning models 808 may include a number of different types of machine learning based regression estimators (e.g., random forest regression models, extreme gradient boosting (xgboost) regression models, linear regression models, ridge regression models, lasso regression models, principal component analysis (PCA) with linear regression models, partial least squares (PLS) regression, etc.) and possibly one or more models not based on regression (e.g., neural networks). Furthermore, in some embodiments, the models 808 may include two or more models of any given type (e.g., two or more models of the same type trained on different historical datasets and / or using different feature sets). Furthermore, different models of the models 808 may be trained to predict values ​​of different product quality attributes (e.g., titer, growth or specific productivity, etc.), facilitating ranking of cell lines according to their different product quality attributes (by the prediction unit 834). Additionally, machine learning model 808 may be used to identify, for each of one or more small scale culture product quality attributes, which features (e.g., which attribute values ​​from the cell pool stage and / or the clone generation and analysis stage) are most predictive of the relative performance of the candidate cell lines. Model 808 may also be trained or retrained using a feature set that includes only the most predictive features.

[0095] Training database 840 may include a single database stored in a single memory (e.g., HDD, SSD, etc.), multiple databases stored in a single memory, a single database stored in multiple memories, or multiple databases stored in multiple memories. For each different model in machine learning models 808, training database 840 may store a corresponding set of training data (e.g., input / feature data and corresponding labels), possibly with overlap between training datasets. To train a model that predicts the titer of a virtual small-scale culture, for example, training database 840 may include multiple training datasets with their labels, each of which includes historical measurements of cell pool titers, cell productivity scores, and / or other measurements made by one or more instruments (e.g., analytical instruments 812, instruments of analytical unit 852, and / or other instruments / sensors). In this example, the label of each training dataset indicates the actual measured titer for that cell line at the small-scale culture stage.

[0096] In some embodiments, the training server 804 uses additional labeled datasets in the training database 840 to validate the trained machine learning models 808 (e.g., to confirm that a given one of the machine learning models 808 provides at least some minimum acceptable accuracy). In some embodiments, the training server 804 also continually updates / refines one or more of the machine learning models 808. For example, after a machine learning model 808 has been initially trained to provide a sufficient level of accuracy, additional measurements at the cell pool and subcloning stages (features) and the small-scale culture stage (labels) may be used to improve the predictive accuracy.

[0097] After the models 808 are fully trained, the application 830 can retrieve, from the training server 804 via the network 806 and the network interface 822, a particular one of the machine learning models 808 (which corresponds to a particular product quality attribute for which a ranking of candidate cell lines is desired). By way of example, the product quality attribute can include cell proliferation and the machine learning model can include PLS; or the product quality attribute can include specific productivity and the machine learning model can include PCA; or the product quality attribute can include titer and the machine learning model can include a ridge regression model. The product quality attribute can be one indicated by a user via a user interface (e.g., via a user interface generated by the user input device 826 and the display 824 and the visualization unit 838) or based on any other suitable input. Upon retrieving the model, the computing system 802 saves a local copy as a local machine learning model 836. In other embodiments, as described above, the model is not retrieved, and instead, the input / feature data is sent to the training server 804 (or another server) as needed to use the appropriate one of the models 808.

[0098] The data collection unit 832 collects the necessary data according to the feature set used in the model 836. For example, the data collection unit 832 may communicate with the analytical instruments 812 and the analytical unit 852 to collect measurements of titer, pool VCD, pool VIA, cell count, cell productivity score, and / or other specific attributes of the cell pools 810 and / or the cell line generation and expansion unit 854. In one such embodiment, the data collection unit 832 sends commands to one or more of the analytical instruments 812 and the analytical unit 852 to cause the one or more to automatically collect the desired measurements. In another embodiment, the data collection unit 832 collects measurements of the cell pools 810 and the cell line generation and expansion unit 854 by communicating with a different computing system (not shown in FIG. 8 ) that is coupled to (and optionally controls) the analytical instruments 812 and / or the analytical unit 852. As mentioned above, the data collection unit 832 may also receive information (e.g., modality) entered by a user. In some embodiments, the application 830 uses some user-input information collected by the data collection unit 832 to select an appropriate one of the models 808 and uses other user-input information collected by the data collection unit 832 as (or to calculate) one or more features / inputs into the selected model.

[0099] After the data collection unit 832 collects attribute values ​​associated with the cell pool 810 and the cell line generation and expansion unit 854, and attribute values ​​used as inputs / features by the local machine learning model 836, the prediction unit 834 runs the model 836 on these inputs / features to predict the value of the product quality attribute of interest (e.g., titer, growth or specific productivity) for each of the candidate cell lines. The prediction unit 834 then compares the predicted values ​​with each other to order / rank the cell lines from best to worst or worst to best. Importantly, while machine learning models may generally have low accuracy for predicting important product quality attributes in small scale cultures, certain models (e.g., as discussed herein) have nevertheless been found to be good at predicting relative values, such that the ranking of candidate cell lines is generally accurate even if the predicted values ​​used for their ranking have low accuracy.

[0100] The visualization unit 838 may cause a user interface presented on the display 824 to display the determined rankings of the cell lines. The above process may be repeated by retrieving different models of models 808 specifically trained for one or more other product quality attributes of interest, collecting (by data collection unit 832) the inputs / features used by those models, using the models (e.g., by prediction unit 834) to predict other product quality attributes for each of the candidate cell lines, and ranking (e.g., by prediction unit 834) the candidate cell lines according to those other product quality attributes. The visualization unit 838 may then cause a user interface to present all of the rankings of the cell lines (e.g., one for titer, one for cell growth, and one for specific productivity) to enable the user to make a more informed choice as to which cell line or lines to advance to (or possibly bypass) the small-scale culture stage.

[0101] The prediction unit 834 may store in the memory unit 828 or another suitable memory / location the predictions made by the model 836 for each set of candidate cell lines and / or the corresponding rankings. After the predictions and / or rankings have been made and stored for all candidate cell lines under consideration and for all product quality attributes of interest, a "winning" portion of the candidate cell lines may be selected for advancement to the small scale culture stage (e.g., stage 12 of FIG. 1). The selection of the winning cell line may be fully automated according to some criteria specific to the product quality attribute (e.g., by assigning specific weights to the titer, cell growth and specific productivity rankings and then comparing the resulting scores) or may involve human interaction (e.g., by displaying the predicted rankings to the user via the display 824). The winning cell line may then be advanced to the small scale cell culture stage (e.g., stage 12 of FIG. 1) or in some embodiments may be advanced to an earlier stage (e.g., stage 14 of FIG. 1) bypassing the small scale cell culture stage.

[0102] In some embodiments, the computing system 802 is configured to identify which cell lines should be subjected to the procedures discussed above, i.e., which cell lines should be used as "candidate" cell lines. For example, the computing system 802 (e.g., the application 830 or another application) may analyze the results of cell count and diffusion assays (obtained by the data collection unit 832 from the analysis unit 852 of the cell line generation and analysis system 850) to determine which cell lines have the highest potential and should be advanced for further cell line development and screening. Cell lines with both high cell productivity scores and high cell counts are considered the best candidates to achieve high performance in small-scale screening cultures. The identification of candidate cell lines may be performed automatically by the processing unit 820 or the prediction unit 834, or in combination with a user manually weighing these factors via the user input device 826. The identification may also be strictly manual, in which case the user, via the user input device 826, evaluates the scores shown on the display 824 and selects which cell lines should be candidates. 9 shows an exemplary graphical output 860 of the display 824 showing a plot of cell number versus cell productivity score (Spotlight assay score) for a selection of cell lines. The cell lines that the user would like to select as candidate cell lines are, for example, surrounded by a dashed line. Various techniques for determining which model is best suited to predict a given product quality attribute ranking for a virtual small-scale screening culture and to identify the most predictive features / inputs for a given model and / or product quality attribute are now described with reference to FIGS. 10-12G.

[0103] FIG. 10 shows an example of a modular and flexible process 900 that provides a data preparation and model selection framework. In particular, the process 900 can be used as a framework to identify well-performing models for predicting values ​​of different product quality attributes and facilitating ranking of cell lines according to those attributes (e.g., by the prediction unit 834). At a high level, the process 900 includes a stage or step 902 for aggregating data, a stage 910 for data pre-processing, and a stage 920 for defining a model. In general, a well-performing model for a particular attribute value can be identified by training many different models using historical training data generated from previous cell line screening runs and comparing their results. For example, the attribute can include cell proliferation and the machine learning model can include PLS; or the attribute can include specific productivity and the machine learning model can include PCA; or the attribute can include titer and the machine learning model can include a ridge regression model. Various measures can be taken to ensure a robust training dataset (e.g., providing standardized heterogeneous data, removing outliers, imputing missing values, etc.). In some embodiments, special feature engineering techniques are used to extract or derive the best representation of the predictor variables to improve the effectiveness of the model. To avoid overfitting, in some embodiments, feature reduction can be performed. Models may be evaluated using metrics such as, for example, root mean square error (RMSE) to measure the accuracy of the predictions, and Spearman rho to measure the correctness of the ranking order.

[0104] In step 902, the training server 804 receives data from the training database 840 or any other suitable database. This step may include entering user input via the user input device 826, where the user defines possible predictor variables and product quality attribute values ​​to be predicted by the machine learning regression estimator (model). The predictor variables may include cell pool data as well as data collected in the cell line generation and analysis system. Although other embodiments may use other subcloning systems, the following discussion refers to an example in which Berkeley Lights' Beacon (abbreviated herein as "BLI") is used for the cell line generation and analysis system. The predicted variables may be defined, for example, as data collected during a clonal fed-batch experiment. First, in step 902, appropriate data is selected from among the available historical data. Furthermore, the historical data may include both categorical data, such as modality, and numerical data, such as cell number and titer. The cell pool data may include, for example, data regarding modality, VCD, pool viability, pool titer, pool specific productivity, and pool time integral VCD. Growth factors such as VCD and viability can be collected periodically over time (e.g., on different days of a 10-day culture). Cell line generation and growth data (BLI data) can include data on, for example, cell productivity score, BLI specific productivity, cell number, time-integrated VCD, doubling time, etc. Growth factors measured by BLI, such as cell number, can also be collected periodically over time (e.g., on different days after feeding into a clonal generation and growth unit, such as unit 854). When these cell lines are advanced to the next stage of cell line development (e.g., stage 12 in FIG. 1), small-scale culture (e.g., fed-batch culture) data reflecting outcomes such as titer, specific productivity, and / or cell growth measurements can serve as labels for the various feature sets. A non-limiting list of possible attributes / features is provided below in Table 2 for both the cell pool dataset (pool data) and the cell line generation and analysis dataset (BLI data) as well as the fed-batch predictor variables.

[0105] In the exemplary process 900, the data pre-processing stage 910 includes steps 912-918. In step 912, the training data is evaluated and cleaned, including treatment of missing data and treatment of outliers. For example, missing records (e.g., pooled VCD data for empty pens), zero values ​​(e.g., values ​​that were not recorded), incomplete data sets (e.g., for scenarios where data collection was not completed from a cell pool for a cell line until the end of a fed-batch experiment), outliers, and data from inconclusive experiments may be removed. In some embodiments, when using a combined data set, some data values ​​may need to be adjusted to compensate for instrument variability.

[0106] In step 914, useful features are extracted or derived from the dataset using special feature engineering techniques to find the best representation of the predictor variables to enhance the effectiveness of the model. The data can be visualized for underlying relationships to determine which feature engineering steps should be evaluated for performance improvement. For example, the best representation of the predictor variables can be (i) a transformation of the predictor, (ii) an interaction of two or more predictors such as a product or ratio, (iii) a functional relationship between the predictors, or (iv) an equivalent re-representation of the predictor. The assay or proliferation values ​​can be scaled relative to cells of the same cohort to give an unbiased view of proliferation and assay scores. From these observations, features can be calculated and added to the predictor dataset (e.g., squared cell counts, squared pool titers, etc.).

[0107] Step 914 may include converting categorical variables to numerical values. For example, for a categorical variable of modality, monoclonal (mAb) modalities may be converted to "10," certain bispecific modalities may be converted to "00," and so on. In a data pre-processing step 916, the training data may be filtered to include only the features selected in steps 912 and 914 above, and to defined targets / predictors (e.g., fed-batch titer, growth, and specific productivity).

[0108] When training and comparing machine learning models, k-fold cross-validation can be used to measure model performance and select optimal hyperparameters. Thus, in step 918, the training data can be split into training and testing datasets for k-fold cross-validation to avoid training and testing on the same samples. For example, the number of splits can be defined by the number of subcloning projects used in the training dataset (e.g., for k=6, the model is trained and evaluated 6 times across different 5 / 1 partitions of the dataset).

[0109] Stage 920 defines the machine learning model and includes steps 922-928. At a high level, stage 920 may include setting a regressor and a scaling method (step 922), training a predictive model by running the preprocessed data of stage 910 through each model in a model library over a range of hyperparameters (step 924), defining and calculating model performance metrics (step 926), and outputting the final production model (step 928).

[0110] An exemplary step 922 registers the model library and sets the scaling method for each selected regression model. Preferably, some or all of the machine learning models selected for testing in step 922 meet two criteria: (i) provide a quantitative output, and / or (ii) be interpretable (e.g., by providing coefficient weights or feature importance weights). Machine learning models that can assign weights to input features are generally preferred because they can account for the relative importance of each input feature with respect to predicting the target output. Sparsity-induced machine learning models are also generally preferred (e.g., models that initially accept many attribute values ​​as features, but only require a small subset of these attribute values ​​as features to make an accurate prediction). This property reduces overfitting by eliminating features that do not significantly affect the target outcome, while also improving interpretability. Regression models / estimators based on decision trees (e.g., random forest regression models, extreme gradient boosting (xgboost) regression models) or other machine learning algorithms (e.g., linear regression models, ridge regression models, lasso regression models, principal component analysis (PCA) with linear regression models or partial least squares (PLS) regression models, etc.) may be particularly suitable to meet both criteria above. Although not traditionally considered interpretable, in some embodiments, one or more neural networks may be selected in step 922. Step 922 may also include setting hyperparameter ranges for the selected regression models.

[0111] An exemplary step 924 involves training predictive models. For example, step 924 may train models selected for inclusion in the library for each target product quality attribute of interest against the full set of feature data preprocessed in steps 912 and 914, and cross-validate over the ranges of hyper-parameters defined in step 922. Step 924 may include performing k-fold validation on the dataset defined in step 918 for each model.

[0112] An exemplary step 926 is to calculate performance metrics using the trained models. For each of the k folds, an algorithm performance metric, such as RMSE (for accuracy in predicting the target product quality attribute) and / or Spearman's rho (for ranking accuracy), may be calculated for each of the predictive models trained in step 924. Then, each trained model with tuned hyper-parameters is evaluated using one of the folds as a test data set, and the model with the best metric (e.g., highest Spearman's rho or lowest RMSE) for each predicted product quality attribute is selected. The performance metrics of the iterative runs may be saved and an average of the k folds (e.g., 6 folds) may be calculated to compare model performance. The calculation of the RMSE metric is shown in Equation 2 above. Spearman's rho may be calculated as follows:

number

[0113] Although counterintuitive, as discussed above, the ability of certain machine learning models to correctly rank cell lines (according to the relative values ​​of the product quality attributes predicted by the models) can far exceed their ability to accurately predict product quality attributes. For example, certain machine learning models have been found to have relatively low accuracy in predicting the value of a particular product quality attribute at the fed-batch stage, but perform well in predicting values ​​in a relative sense (e.g., in terms of whether the predicted value is greater or less than the values ​​the model predicts for other cell lines). In the context of selecting cell lines to proceed to the next development stage, this ability to accurately rank cell lines may be sufficient, since knowing which cell lines to proceed to the next stage is generally more important than predicting accurate and precise product quality attributes. Thus, Spearman's rho may be a preferred metric to calculate in step 926 (rather than, e.g., RMSE).

[0114] In step 928, the "best" model is output / identified as the final production model based on the calculated metrics (e.g., the model with the highest Spearman's rho or lowest RMSE). If the best model is interpretable, step 928 may include determining the importance of each feature in making the prediction. For example, step 928 may include determining feature importance based on coefficient weights (e.g., generated by a Lasso regression model) or feature importance weights (e.g., generated by a tree-based model such as XGBoost). Output from these interpretable models (e.g., a display of reduced parameters by a Lasso sparsity induced model or a feature importance plot showing how frequently each variable was split in training the tree for an XGBoost model) may be analyzed by the training server 804 or a human reviewer (via visualization unit 838) to determine the most predictive features (e.g., 2-10 features) for each relative ranking of candidate cell lines according to predicted product quality attribute values. For example, FIG. 11A is an example output 930 from a Lasso regression model when predicting fed-batch titer, showing that pool titer is more predictive of fed-batch titer than cell productivity score (here the "Spotlight" assay score), and that cell productivity score is more predictive of fed-batch titer than cell count (which has no or very little predictive power for fed-batch titer). Similarly, FIG. 11B shows an example feature importance plot 932 of an xgboost regression model predicting fed-batch titer, showing strong feature importance of pool titer and cell productivity score (Adj_Au) relative to other features used. The results show that the model is more predictive of fed-batch titer than features based on, for example, cell number (e.g., cell number squared or "CC 2"). A new version of that model trained using only the winning / best model or the most predictive features can then be used with a much smaller feature set. The model can then be saved (e.g., by training server 804, in model 808) as a trained model and used to make predictions in new experiments (e.g., by prediction unit 834). Identifying highly predictive features can also be useful for other purposes, such as providing new scientific insights that can generate new hypotheses (which can in turn lead to improvements in bioprocesses).

[0115] Any suitable attributes may be used for the features discussed above (e.g., to initially train the various models and possibly train the final production model if the features are of sufficient importance). A non-limiting list of possible attributes / features is provided below in Table 2 for both the cell pool dataset (Pool Data) and the Cell Line Generation and Analysis dataset (BLI Data).

[0116] [Table 5]

[0117] [Table 6]

[0118] 12A is a bar graph 934 showing the performance of the best model (output of step 928 of process 900) against baseline performance using Spearman's rho metrics (here by cross-validation over 6 fold) for the product quality attributes of cell growth, specific productivity and titer. Each of the attributes was measured at the end point of the small scale cell culture process (here day 10 of a fed-batch experiment). In this example, the specific productivity performance "baseline" is a linear regression on cell productivity score, where higher cell productivity scores correspond to higher predicted specific productivity. Similarly, the growth performance baseline is a linear regression on cell number, where higher cell numbers correspond to higher predicted growth, and the titer performance baseline is a linear regression on cell productivity score and cell number, where higher scores on both correspond to higher predicted titers.

[0119] As seen in FIG. 12A, the predictive capabilities of the machine learning models identified / output in step 928 of process 900 (further discussed with reference to FIGS. 12B-12G) exceed the baseline performance for ranking candidate cell lines in all three target product quality attributes. The largest gains were seen in the model predicting proliferation ranking, where the model showed a rank correlation of ρ=0.283 compared to the baseline ρ=0 (no predictive capability). The model from step 928 showed only a slight improvement in predicting specific productivity, where the rank correlation increased from ρ=0.468 to the baseline ρ=0.492, which may mean that cell productivity scores alone can explain most of the ordinal differences in specific productivity rank. The model from step 928 showed a moderate increase in performance predicting titer, where the rank correlation increased from ρ=0.245 to ρ=0.342.

[0120] Different regression estimators in the model library 922 have been found to be better suited to predicting different target product quality attribute values. For example, using the model identification / definition procedure outlined in stage 920, the computing system 802 may test multiple regression estimators using the dataset defined in stage 910 and cross-validate each regression model over a range of hyper-parameters. Figures 12B-12G show examples of the relative performance of different regression estimators in predicting specific performance attribute values ​​and the respective selected features used to build each selected model using the feature reduction method described herein with reference to step 928. The regression estimator exhibiting the "best" performance was selected as the model with the highest average Spearman rho across all cell lines after optimizing the associated hyper-parameters (if any). The average RMSE is also shown in Figures 12B, 12D and 12F, however, for reasons described elsewhere herein (i.e., the importance of relative / ranking accuracy versus absolute accuracy), this metric was not used in model selection.

[0121] As can be seen in table 936 shown in Figure 12B, the best regression estimator for predicting titer was found to be Ridge Regression with hyperparameter lambda equal to 1.3. This performance was closely followed by four other models: Linear Regression, Lasso Regression with lambda equal to 0.001, PCA with two principal components, and PLS with two principal components. Table 938 in Figure 12C shows the two attributes (pool titer and cell productivity score (Spotlight assay score)) that were analyzed by the model selected in feature reduction.

[0122] Table 940 in Figure 12D shows that the best predictor of specific productivity was PCA with two principal components. Table 942 in Figure 12E shows the eight attributes analyzed by the model selected in feature reduction. For the first PCA component, the pool titer, cell productivity score (Spotlight assay score) and specific productivity values ​​in the cell line production and analysis system are more important, while for the second PCA component, the scaled values ​​of these metrics (normalization of the different characteristics of each cell line) are more important.

[0123] Table 944 in FIG. 12F shows that the best regression estimator predicting proliferation was found to be PLS with one principal component. Table 946 in FIG. 12G shows the nine attributes analyzed by the model selected in feature reduction. The model generally placed more weight on the pooled data than on the data collected on the Berkeley Lights system. In particular, pooled titer, pooled IVCD and pooled viable cell density on days 6 and 8 were most important, while cell number was weighted less.

[0124] In addition to using Spearman's rho, other measures or visualizations may be used to determine the ranking accuracy of various models. Such an evaluation may be expressed, for example, as a comparison of the ranking determined by the model to the actual rank of the same cell lines in real-world fed-batch experiments. The evaluation may also be evaluated for the model's ability to capture the top cell lines (e.g., the top 4 cell lines) for each target product attribute in real-world fed-batch experiments, for example, by showing whether these top cell lines appear anywhere near the top (e.g., within the top 50%) of the cell lines ranked by the model results. Figures 13A-13C show examples of such evaluation results. Each of Figures 13A-13C shows six bar graphs, each representing the evaluation results for one of the six evaluated data sets. The top 50% of the ranked cell lines are shown as white bars and the bottom 50% of the ranked cell lines are shown as shaded bars. For a model that perfectly predicts the rankings, a given bar graph would have all white bars located to the left (along the x-axis) of all the shaded bars. The height of each bar represents the relative value of the product quality attribute represented in real-world small-scale cell cultures for each cell line.

[0125] Referring first to FIG. 13A, exemplary results 950 correspond to the predicted ranking of cell lines according to the product quality attribute titer (in this example titer measured on day 10 of a fed-batch, small-scale culture). As can be seen in FIG. 13A, a 50% reduction in export using this model (i.e., of cell lines that have progressed to the fed-batch stage) is too aggressive, eliminating some of the top real-world cell lines. In this example, at least 38 clones must be exported from Dataset 4 to ensure that all of the top 4 clones are selected.

[0126] FIG. 13B shows an exemplary result 952 corresponding to the predicted ranking of cell lines according to the product quality attribute specific productivity (specific productivity (qP) at day 10 of fed-batch, small scale culture in this example). The model prediction of specific productivity was encouraging. For example, halving the number of exports would result in only missing one of the top four clones across all cell lines. The maximum number of clones (from the predicted ranking) required to capture a top four clone was 31, with datasets 5 and 6 each identifying all of their top four clones within the top eight clones predicted by the model.

[0127] FIG. 13C shows an exemplary result 954 corresponding to the predicted ranking of cell lines according to the product quality attribute cell growth (in this example, IVCD at day 10 of a fed-batch, small-scale culture). The model prediction of growth indicates that the best indicator is not the growth in the cell line production and growth unit, but the pool from which the clones originate. However, as shown by datasets 3 and 5, the model did not predict that some of the top growing clones were in the top 50%. However, this information is still valuable when compared to a baseline with no predictive ability for cell number (as measured by cell line production and growth unit). Based on the results from dataset 4, a minimum of 37 clones must be exported to ensure that the top 4 clones are exported / advanced.

[0128] 14 is a flow diagram of an exemplary method 960 for facilitating the selection of cell lines to proceed to a next cell line screening stage (e.g., stage 12 of FIG. 1) from among candidate cell lines producing recombinant protein. Part or all of method 960 may be performed by one or more processors of processing unit 820 of computing system 802 or server 804 (e.g., in the execution of a cloud service), for example, upon execution of software instructions of application 830 stored in memory unit 828.

[0129] At block 962, a first plurality of attribute values ​​are measured for a plurality of candidate cell lines using an opto-electronic cell line generation and analysis system (e.g., system 850 of FIG. 8). The opto-electronic cell line generation and analysis system may, for example, perform optical and assay measurements on the candidate cell lines at block 962. In some embodiments, such measurements are performed at least in part by measuring at least cell counts and cell productivity scores in a plurality of physically isolated pens within the opto-electronic cell line generation and analysis system. In some of these embodiments, block 962 further includes generating cells of the candidate cell line using the opto-electronic cell line generation and analysis system by at least moving individual cells to different ones of the physically isolated pens having one or more photoconductors activated by light patterns and containing the individual cells in their respective pens throughout the cell line generation and analysis process. Additionally, block 962 may include measuring different values ​​of the first plurality of attribute values ​​on different days of the cell line generation and analysis process. More generally, the first plurality of attribute values ​​may include any values ​​of attributes that can be measured by the analysis unit 852, as discussed elsewhere herein, and / or may include any suitable attribute values ​​that can be measured using the optical-electronic cell line generation and analysis system.

[0130] In block 964, a second plurality of attribute values ​​is obtained for the candidate cell line. The second plurality of attribute values ​​includes one or more attribute values ​​measured at a cell pool screening stage of the candidate cell line. The attribute values ​​measured in block 964 may include, for example, pool titer, VCD, and / or pool viability. In some embodiments and / or scenarios, other attribute values ​​are instead or additionally obtained in block 964, such as values ​​calculated based on one or more direct measurements (e.g., time-integrated VCD, pool specific productivity, etc.) or values ​​calculated by a device other than the measurement device (e.g., computing system 802) based on one or more direct measurements and / or user-input values ​​(e.g., modality). In some embodiments, some of the attribute values ​​obtained in block 964 are measurements obtained periodically over time (e.g., on different days). For example, a first attribute value may be a VCD value on day 0 of a cell pool, a second attribute value may be a VCD value on day 3 of the same cell pool, and so on. More generally, the second plurality of attribute values ​​may include values ​​of any of the attributes that may be measured by the analytical instrument 812 or associated with the cell pool 810 as discussed elsewhere herein, and / or may include values ​​of other suitable attributes associated with the cell pool.

[0131] In block 966, a ranking of the candidate cell lines is determined according to a product quality attribute associated with the virtual small-scale screening culture for the candidate cell lines. Block 966 includes predicting a value of the product quality attribute for each of the candidate cell lines by analyzing the first plurality of attribute values ​​measured in block 962 and the second plurality of attribute values ​​obtained in block 964 using a machine learning based regression estimator. Block 968 also includes comparing the predicted values ​​(i.e., to rank the candidate cell lines (e.g., in order from best to worst in terms of predicted value)). In some embodiments, the predicted value is a predicted value of a cell growth metric. In other embodiments, the predicted value is a titer, a non-productivity metric, or any other suitable indicator of performance in the virtual small-scale culture screening stage. The machine learning based regression estimator can be any suitable type of regression estimator (e.g., ridge, lasso, PCA, PCS, xgboost, etc.). In other embodiments, other types of machine learning models may be used (eg, by the prediction unit 834) to make the prediction in block 966 (eg, neural networks, etc.).

[0132] In some embodiments, block 966 includes at least (i) predicting titer for each of the plurality of candidate cell lines by analyzing the first and second plurality of attribute values ​​using a machine learning based regression estimator, and (ii) determining a ranking according to titer by comparing the predicted titers. In some of these embodiments, the first plurality of attribute values ​​include values ​​based on a cell productivity score (e.g., the score itself or a value derived therefrom) and / or the second plurality of attribute values ​​include values ​​based on a cell pool titer (e.g., the cell pool titer itself or a value derived therefrom). The machine learning based regression estimator that analyzes these attributes can be, for example, a ridge regression estimator.

[0133] In other embodiments, block 966 includes at least (i) predicting, for each of the plurality of candidate cell lines, a non-productivity metric by analyzing a first plurality of attribute values ​​and a second plurality of attribute values ​​using a machine learning based regression estimator, and (ii) determining a ranking according to non-productivity by comparing the predicted non-productivity metrics. In some of these embodiments, the first plurality of attribute values ​​includes a value based on cell productivity score and a value based on cell count, and / or the second plurality of attribute values ​​includes a value based on cell pool titer. The machine learning based regression estimator that analyzes these attributes can be, for example, a PCA regression estimator with two principal components.

[0134] In still other embodiments, block 966 includes at least (i) predicting cell proliferation metrics for each of the plurality of candidate cell lines by analyzing a first plurality of attribute values ​​and a second plurality of attribute values ​​using a machine learning based regression estimator, and (ii) comparing the predicted cell proliferation metrics to determine a ranking according to cell proliferation. In some of these embodiments, the first plurality of attribute values ​​includes values ​​based on cell count, and the second plurality of attribute values ​​includes values ​​based on cell pool time integrated viable cell density (iVCD), values ​​based on cell pool viable cell density (VCD) on different days, and values ​​based on cell pool viability on different days. The machine learning based regression estimator that analyzes these attributes can be, for example, a PLS regression estimator with one principal component.

[0135] At block 968, a representation of the rankings (e.g., an ordered list, a bar graph, etc.) is presented to the user via a user interface. For example, block 968 may include generating or displaying a GUI (e.g., by visualization unit 838) and causing the GUI to be presented on a display (e.g., display 824). In one embodiment, the presentation of the representation is triggered by sending data indicative of the rankings to another computing device or system, which uses the data to display and present the GUI.

[0136] In some embodiments, method 960 includes one or more additional blocks not shown in Figure 14. For example, method 960 may include an additional block (e.g., before block 962) in which the performance of the machine learning based regression estimator is evaluated by calculating an average Spearman ranking correlation coefficient for at least the machine learning based regression estimator (e.g., as calculated according to Equation 4). As another example, method 960 may include a further block in which one or more of the candidate cell lines are advanced to a next stage of cell line screening (e.g., a fed-batch cell culture stage) based on the ranking determined in block 966.

[0137] Aspects of the invention include the following.

[0138] Aspect 1. A method of facilitating cell line selection from among a plurality of candidate cell lines that produce a recombinant protein, the method comprising: measuring a first plurality of attribute values ​​for the plurality of candidate cell lines using an optical-electronic cell line generation and analysis system; obtaining, by one or more processors, a second plurality of attribute values ​​for the plurality of candidate cell lines, the second plurality of attribute values ​​comprising one or more attribute values ​​measured at a cell pool screening stage of the plurality of candidate cell lines; determining, by the one or more processors, a ranking of the plurality of candidate cell lines according to a product quality attribute associated with a virtual small-scale screening culture for the plurality of candidate cell lines, the determining comprising: (i) predicting a value of the product quality attribute for each of the plurality of candidate cell lines by analyzing the first plurality of attribute values ​​and the second plurality of attribute values ​​using a machine learning based regression estimator, and (ii) comparing the predicted values; and presenting an indication of the ranking to a user via a user interface.

[0139] Embodiment 2. The method of embodiment 1, wherein measuring the first plurality of attribute values ​​using the optical-electronic cell line generation and analysis system comprises performing a plurality of optical and assay measurements on a plurality of candidate cell lines.

[0140] Embodiment 3. The method of embodiment 2, wherein performing a plurality of optical and assay measurements on a plurality of candidate cell lines comprises measuring at least cell count and cell productivity score in a plurality of physically isolated pens in an opto-electronic cell line generation and analysis system, the method further comprising generating cells of the plurality of candidate cell lines by at least transferring individual cells to different pens of the plurality of physically isolated pens having one or more photoconductors activated by a light pattern using the opto-electronic cell line generation and analysis system, and containing the individual cells within their respective pens throughout the cell line generation and analysis process.

[0141] Example 4. The method of example 3, wherein measuring a first plurality of attribute values ​​comprises measuring a first attribute value corresponding to the first measurement of the attribute; and a second attribute value corresponding to the second measurement of the attribute, wherein the first measurement and the second measurement are taken on different days of the cell line generation and analysis process.

[0142] Embodiment 5. The method of any one of embodiments 1-4, wherein obtaining the second plurality of attribute values ​​includes receiving one or more of: a measured cell pool titer; a measured cell pool viable cell density (VCD); or a measured cell pool viability.

[0143] Embodiment 6. The method of any one of embodiments 1-5, wherein obtaining a second plurality of attribute values ​​comprises receiving attribute values ​​measured on different days of the cell pool screening stage.

[0144] Embodiment 7. The method of any one of embodiments 1-6, wherein the one or more product quality attributes include cell proliferation metrics.

[0145] Embodiment 8. The method of any one of embodiments 1-6, wherein the one or more product quality attributes include one or more of: (i) potency; or (ii) specific productivity metrics.

[0146] Embodiment 9. The method of any one of embodiments 1-8, wherein determining the ranking comprises at least: (i) for each of a plurality of candidate cell lines, predicting a titer by analyzing a first plurality of attribute values ​​and a second plurality of attribute values ​​using a machine learning based regression estimator; and (ii) determining the ranking according to titer by comparing the predicted titers; the first plurality of attribute values ​​comprises values ​​based on a cell productivity score; and the second plurality of attribute values ​​comprises values ​​based on a cell pool titer.

[0147] Embodiment 10. The method of embodiment 9, wherein predicting the titer comprises analyzing the first plurality of attribute values ​​using a ridge regression estimator.

[0148] Embodiment 11. The method of any one of embodiments 1-8, wherein determining the ranking comprises at least: (i) for each of the plurality of candidate cell lines, predicting a non-productivity metric by analyzing a first plurality of attribute values ​​and a second plurality of attribute values ​​using a machine learning based regression estimator; and (ii) determining the ranking according to non-productivity by comparing the predicted non-productivity metrics; the first plurality of attribute values ​​comprises values ​​based on cell productivity score and values ​​based on cell count; and the second plurality of attribute values ​​comprises values ​​based on cell pool titer.

[0149] Aspect 12. The method of aspect 11, wherein predicting the non-productivity metric includes using a principal component analysis (PCA) regression estimator having two principal components.

[0150] Embodiment 13. The method of any one of embodiments 1-8, wherein determining the ranking comprises at least: (i) predicting, for each of a plurality of candidate cell lines, cell growth metrics by analyzing a first plurality of attribute values ​​and a second plurality of attribute values ​​using a machine learning based regression estimator; and (ii) comparing the predicted cell growth metrics to determine the ranking according to cell growth; the first plurality of attribute values ​​comprises values ​​based on cell count; and the second plurality of attribute values ​​comprises values ​​based on cell pool titer, values ​​based on cell pool time integrated viable cell density (iVCD), values ​​based on cell pool viable cell density (VCD) at different days, and values ​​based on cell pool viability at different days.

[0151] Embodiment 14. The method of embodiment 13, wherein predicting the cell proliferation metric comprises using a partial least squares (PLS) regression estimator with one principal component.

[0152] Aspect 15. The method of any one of aspects 1-14, further comprising evaluating the performance of the machine learning based regression estimator by calculating Spearman's rho or average Spearman's rho for at least the machine learning based regression estimator.

[0153] Embodiment 16 The method of any one of embodiments 1 to 15, further comprising advancing one or more cell lines of the plurality of candidate cell lines to a next cell line screening stage based on the ranking.

[0154] Embodiment 17 The method of embodiment 16, wherein the next cell line screening stage is a fed-batch cell culture stage.

[0155] Aspect 18. One or more non-transitory computer-readable media storing instructions that, when executed by one or more processors of a computing system, cause the computing system to perform the method of any one of aspects 1-15.

[0156] Aspect 19. A computing system comprising: one or more processors; and one or more non-transitory computer-readable media storing instructions that, when executed by the one or more processors, cause the computing system to perform any one of the methods of aspects 1-15.

[0157] Aspect 20. A method for facilitating selection of a master cell line from among candidate cell lines producing a recombinant protein, the method comprising: receiving, by one or more processors of a computing system, a plurality of attribute values ​​associated with a small scale cell culture for the particular cell line, where at least some of the plurality of attribute values ​​are measurements of the small scale cell culture; predicting, by the one or more processors, one or more attribute values ​​associated with a virtual large scale cell culture for the particular cell line by analyzing the plurality of attribute values ​​associated with the small scale cell culture using at least a machine learning based regression estimator, where the predicted one or more attribute values ​​comprise potency and / or one or more product quality attribute values; and causing, by the one or more processors, to present to a user via a user interface one or both of (i) the predicted one or more attribute values ​​and (ii) an indication of whether the predicted one or more attribute values ​​satisfy one or more cell line selection criteria, to facilitate selection of a master cell line for use in drug product manufacturing.

[0158] Aspect 21. The method of aspect 20, wherein analyzing the plurality of attribute values ​​using a machine learning based regression estimator includes analyzing the plurality of attribute values ​​using a decision tree regression estimator.

[0159] Aspect 22. The method of aspect 21, wherein analyzing the plurality of attribute values ​​using a machine learning based regression estimator includes analyzing the plurality of attribute values ​​using a random forest regression estimator.

[0160] Aspect 23. The method of aspect 21, wherein analyzing the plurality of attribute values ​​using a machine learning based regression estimator includes analyzing the plurality of attribute values ​​using an xgboost regression estimator.

[0161] Aspect 24. The method of aspect 20, wherein analyzing the plurality of attribute values ​​using a machine learning based regression estimator includes analyzing the plurality of attribute values ​​using a linear support vector machine (SVM) regression estimator.

[0162] Aspect 25. The method of aspect 20, wherein analyzing the plurality of attribute values ​​using a machine learning based regression estimator includes analyzing the plurality of attribute values ​​using an elastic net estimator.

[0163] Embodiment 26. The method of any one of embodiments 20-25, wherein the predicted one or more attribute values ​​include one or more product quality attributes.

[0164] Embodiment 27 The method of embodiment 26, wherein the predicted one or more product quality attribute values ​​include one or more predicted chromatographic measurements.

[0165] Embodiment 28. The method of any one of embodiments 20-27, further comprising receiving user input data from a user via a user interface, the user input data including one or more of an identifier for the particular cell line, a modality of a drug produced using the particular cell line, an indication of a drug product produced using the particular cell line, or a protein scaffold type associated with a drug produced using the particular cell line, and wherein analyzing the plurality of attribute values ​​associated with the small scale cell culture using a machine learning based regression estimator further comprises analyzing the user input data using the machine learning based regression estimator.

[0166] Embodiment 29. The method of any one of embodiments 20 to 28, wherein receiving a plurality of attribute values ​​associated with the small scale cell culture comprises receiving one or more of: a measured titer of the small scale cell culture; a measured viable cell density of the small scale cell culture; or a measured viability of the small scale cell culture.

[0167] Embodiment 30. The method of any one of embodiments 20 to 29, wherein receiving a plurality of attribute values ​​associated with the small scale cell culture comprises receiving one or more characteristics of a medium of the small scale cell culture.

[0168] Embodiment 31 The method of embodiment 30, wherein receiving one or more characteristics of the medium comprises receiving a measured glucose concentration of the medium.

[0169] Embodiment 32. The method of any one of embodiments 20-31, wherein receiving a plurality of attribute values ​​associated with the small scale cell culture comprises receiving a first attribute value corresponding to a first measurement of an attribute associated with the small scale cell culture; and a second attribute value corresponding to a second measurement of an attribute associated with the small scale cell culture, wherein the first measurement and the second measurement are taken on different days of the small scale cell culture.

[0170] Embodiment 33. The method of any one of embodiments 20-32, further comprising receiving, by the one or more processors and from a user via a user interface, data indicative of a use case prior to receiving a plurality of attribute values ​​associated with the small scale cell culture, and selecting, by the one or more processors and based on the data indicative of the use case, a machine learning based regression estimator from among the plurality of estimators, each of the plurality of estimators designed for a different use case.

[0171] Aspect 34. The method of aspect 33, wherein receiving data indicative of the use case includes receiving at least data indicative of (i) at least one of one or more attribute values ​​associated with the virtual large-scale cell culture, and (ii) a modality of the drug to be produced.

[0172] Embodiment 35. The method of embodiment 34, wherein receiving data indicative of the use case includes receiving data indicative of a titer associated with at least the virtual large-scale cell culture; and analyzing the plurality of attribute values ​​using a machine learning based regression estimator includes analyzing the plurality of attribute values ​​using (i) a decision tree regression estimator, (ii) a random forest regression estimator, (iii) an xgboost regression estimator, or (iv) a linear support vector machine (SVM) regression estimator.

[0173] Aspect 36. The method of aspect 34, wherein receiving data indicative of the use case includes receiving data indicative of chromatographic measurements associated with at least the virtual large-scale cell culture; and analyzing the plurality of attribute values ​​using a machine learning based regression estimator includes analyzing the plurality of attribute values ​​using an xgboost regression estimator.

[0174] Embodiment 37. The method of embodiment 33, further comprising: for each estimator of the plurality of estimators, determining, by the one or more processors, a set of features that are most predictive of the output of the estimator; and receiving a plurality of attribute values ​​associated with the small scale cell culture comprises receiving only attribute values ​​that are included within the set of features determined for the machine learning based regression estimator.

[0175] Embodiment 38. The method of any one of embodiments 20-37, further comprising measuring, by one or more analytical devices, at least some of a plurality of attribute values ​​associated with the small scale cell culture.

[0176] Embodiment 39. The method of any one of embodiments 20 to 38, wherein receiving the plurality of attribute values ​​includes receiving measurements from an optical-electronic cell line generation and analysis system.

[0177] Aspect 40. One or more non-transitory computer-readable media storing instructions that, when executed by one or more processors of a computing system, cause the computing system to perform the method of any one of aspects 20-39.

[0178] Aspect 41. A computing system comprising: one or more processors; and one or more non-transitory computer-readable media storing instructions that, when executed by the one or more processors, cause the computing system to perform any one of the methods of aspects 20-39.

[0179] Although the systems, methods, devices, and components thereof have been described in terms of exemplary embodiments, the systems, methods, devices, and components thereof are not limited thereto. The detailed description is to be construed as an example only, and does not describe all possible embodiments of the invention, since describing all possible embodiments would be impractical, if not impossible. Many alternative embodiments can be implemented using either current technology or technology developed after the filing date of this patent, and such embodiments will still fall within the scope of the claims that define the invention.

[0180] Those skilled in the art will understand that various modifications, alterations and combinations can be made to the above-described embodiments without departing from the scope of the present invention, and such modifications, alterations and combinations are to be construed as being within the scope of the concept of the present invention.

Claims

1. 1. A method for facilitating the selection of a cell line from among a plurality of candidate cell lines that produce a recombinant protein, comprising: measuring a first plurality of attribute values ​​for said plurality of candidate cell lines using an optical-electronic cell line generation and analysis system; obtaining, by one or more processors, a second plurality of attribute values ​​for the plurality of candidate cell lines, the second plurality of attribute values ​​comprising one or more attribute values ​​measured at a cell pool screening stage of the plurality of candidate cell lines; determining, by one or more processors, a ranking of the plurality of candidate cell lines according to a product quality attribute associated with a virtual small-scale screening culture for the plurality of candidate cell lines, comprising: (i) predicting a value of the product quality attribute for each of the plurality of candidate cell lines by analyzing the first plurality of attribute values ​​and the second plurality of attribute values ​​using a machine learning based regression estimator; and (ii) comparing the predicted values; and Presenting the display of the ranking to a user via a user interface. The method includes:

2. 2. The method of claim 1, wherein measuring the first plurality of attribute values ​​using the optical-electronic cell line generation and analysis system comprises performing a plurality of optical and assay measurements on the plurality of candidate cell lines.

3. performing the plurality of optical and assay measurements on the plurality of candidate cell lines includes measuring at least a cell number and a cell productivity score in a plurality of physically isolated pens in the optical-electronic cell line generation and analysis system, the method comprising: generating cells of said plurality of candidate cell lines by at least transferring individual cells to different pens of said plurality of physically separated pens having one or more photoconductors activated by a light pattern using said optical-electronic cell line generation and analysis system, and containing said individual cells within their respective pens throughout a cell line generation and analysis process. The method of claim 2 , further comprising:

4. Measuring the first plurality of attribute values ​​comprises: a first attribute value corresponding to a first measurement of the attribute; and a second attribute value corresponding to a second measurement of the attribute; wherein the first measurement and the second measurement are taken on different days of the cell line generation and analysis process.

5. Obtaining the second plurality of attribute values ​​includes: Measured cell pool titer; Measured cell pool viable cell density (VCD); or Measured cell pool viability The method of claim 1 , further comprising receiving one or more of:

6. 2. The method of claim 1, wherein obtaining the second plurality of attribute values ​​comprises receiving attribute values ​​measured on different days of the cell pool screening stage.

7. The method of claim 1 , wherein the one or more product quality attributes comprises cell proliferation metrics.

8. 2. The method of claim 1, wherein the one or more product quality attributes include one or more of: (i) potency; or (ii) non-productivity metrics.

9. determining the ranking includes at least: (i) predicting a potency for each of the plurality of candidate cell lines by analyzing the first plurality of attribute values ​​and the second plurality of attribute values ​​using the machine learning based regression estimator; and (ii) determining the ranking according to potency by comparing the predicted potencies; the first plurality of attribute values ​​includes values ​​based on a cell productivity score; and The method of claim 1 , wherein the second plurality of attribute values ​​comprises values ​​based on cell pool titers.

10. 10. The method of claim 9, wherein predicting the titer comprises analyzing the first plurality of attribute values ​​using a ridge regression estimator.

11. determining the ranking includes at least: (i) predicting, for each of the plurality of candidate cell lines, a non-productivity metric by analyzing the first plurality of attribute values ​​and the second plurality of attribute values ​​using the machine learning based regression estimator; and (ii) determining the ranking according to non-productivity by comparing the predicted non-productivity metrics; The first plurality of attribute values ​​includes a value based on a cell productivity score and a value based on a cell number; and The method of claim 1 , wherein the second plurality of attribute values ​​comprises values ​​based on cell pool titers.

12. The method of claim 11 , wherein predicting the non-productivity metric includes using a principal component analysis (PCA) regression estimator having two principal components.

13. determining the ranking includes at least: (i) predicting, for each of the plurality of candidate cell lines, a cell proliferation metric by analyzing the first plurality of attribute values ​​and the second plurality of attribute values ​​using the machine learning based regression estimator; and (ii) comparing the predicted cell proliferation metrics to determine the ranking according to cell proliferation; the first plurality of attribute values ​​includes values ​​based on cell count; and 2. The method of claim 1, wherein the second plurality of attribute values ​​comprises a value based on cell pool titer, a value based on cell pool time integrated viable cell density (iVCD), a value based on cell pool viable cell density (VCD) on different days, and a value based on cell pool viability on different days.

14. 14. The method of claim 13, wherein predicting the cell proliferation metric comprises using a partial least squares (PLS) regression estimator with one principal component.

15. 2. The method of claim 1, further comprising evaluating a performance of the machine learning based regression estimator by calculating Spearman's rho or an average Spearman's rho for at least the machine learning based regression estimator.

16. 2. The method of claim 1, further comprising advancing one or more cell lines of said plurality of candidate cell lines to a next cell line screening stage based on said ranking.

17. 17. The method of claim 16, wherein the subsequent cell line screening stage is a fed-batch cell culture stage.

18. One or more non-transitory computer-readable media storing instructions that, when executed by one or more processors of a computing system, cause the computing system to perform the method of any one of claims 1 to 15.

19. 1. A computing system comprising: one or more processors; and One or more non-transitory computer readable media storing instructions that, when executed by said one or more processors, cause said computing system to perform the method of any one of claims 1 to 15. A computing system comprising:

20. 1. A method for facilitating the selection of a master cell line among candidate cell lines producing a recombinant protein, comprising: receiving, by one or more processors of a computing system, a plurality of attribute values ​​associated with a small scale cell culture for a particular cell line, at least some of the plurality of attribute values ​​being measurements of the small scale cell culture; predicting, by the one or more processors, for the particular cell line, one or more attribute values ​​associated with the virtual large-scale cell culture by analyzing the plurality of attribute values ​​associated with the small-scale cell culture using at least a machine learning based regression estimator, wherein the predicted one or more attribute values ​​include titer and / or one or more product quality attribute values; and causing the one or more processors to present to a user via a user interface one or both of (i) the predicted one or more attribute values, and (ii) an indication of whether the predicted one or more attribute values ​​meet one or more cell line selection criteria, to facilitate selection of the master cell line for use in drug product manufacturing. The method includes:

21. 21. The method of claim 20, wherein analyzing the plurality of attribute values ​​using a machine learning based regression estimator comprises analyzing the plurality of attribute values ​​using a decision tree regression estimator.

22. 22. The method of claim 21, wherein analyzing the plurality of attribute values ​​using a machine learning based regression estimator comprises analyzing the plurality of attribute values ​​using a random forest regression estimator.

23. 22. The method of claim 21, wherein analyzing the plurality of attribute values ​​using a machine learning based regression estimator comprises analyzing the plurality of attribute values ​​using an xgboost regression estimator.

24. 21. The method of claim 20, wherein analyzing the plurality of attribute values ​​using a machine learning based regression estimator comprises analyzing the plurality of attribute values ​​using a linear support vector machine (SVM) regression estimator.

25. 21. The method of claim 20, wherein analyzing the plurality of attribute values ​​using a machine learning based regression estimator comprises analyzing the plurality of attribute values ​​using an elasticity net estimator.

26. The method of claim 20 , wherein the predicted one or more attribute values ​​include the one or more product quality attributes.

27. 27. The method of claim 26, wherein the predicted one or more product quality attribute values ​​comprise one or more predicted chromatographic measurements.

28. From the user via the user interface: an identifier for said particular cell line; Modalities of drugs produced using said particular cell lines; an indication of the drug product produced using the particular cell line; or The drug-related protein scaffold produced using the particular cell line and receiving user input data including one or more of:

21. The method of claim 20, wherein analyzing the plurality of attribute values ​​associated with the small scale cell cultures using the machine learning based regression estimator further comprises analyzing the user input data using the machine learning based regression estimator.

29. Receiving the plurality of attribute values ​​associated with the small scale cell culture comprises: the measured titer of said small scale cell culture; the measured viable cell density of the small scale cell culture; or Measured viability of the small-scale cell culture 21. The method of claim 20, comprising receiving one or more of:

30. 21. The method of claim 20, wherein receiving the plurality of attribute values ​​associated with the small scale cell culture comprises receiving one or more characteristics of a medium for the small scale cell culture.

31. 31. The method of claim 30, wherein receiving the one or more characteristics of the medium comprises receiving a measured glucose concentration of the medium.

32. Receiving the plurality of attribute values ​​associated with the small scale cell culture comprises: a first attribute value corresponding to a first measurement of an attribute associated with the small scale cell culture; and a second attribute value corresponding to a second measurement of the attribute associated with the small scale cell culture; 21. The method of claim 20, comprising receiving a first measurement and a second measurement, wherein the first measurement and the second measurement are taken on different days of the small scale cell culture.

33. prior to receiving the plurality of attribute values ​​associated with the small scale cell culture, receiving, by the one or more processors and from a user via a user interface, data indicative of a use case; and selecting, by the one or more processors and based on the data indicative of the use case, the machine learning based regression estimator from among a plurality of estimators.

21. The method of claim 20, further comprising: wherein each of the plurality of estimators is designed for a different use case.

34. 34. The method of claim 33, wherein receiving data indicative of the use case comprises receiving at least data indicative of (i) at least one of the one or more attribute values ​​associated with the virtual large scale cell culture, and (ii) a modality of a drug to be produced.

35. Receiving data indicative of the use case includes receiving data indicative of a titer associated with at least the virtual large scale cell culture; and 35. The method of claim 34, wherein analyzing the plurality of attribute values ​​using a machine learning based regression estimator comprises analyzing the plurality of attribute values ​​using (i) a decision tree regression estimator, (ii) a random forest regression estimator, (iii) an xgboost regression estimator, or (iv) a linear support vector machine (SVM) regression estimator.

36. Receiving data indicative of the use case includes receiving data indicative of chromatographic measurements associated with at least the virtual large-scale cell culture; and 35. The method of claim 34, wherein analyzing the plurality of attribute values ​​using a machine learning based regression estimator comprises analyzing the plurality of attribute values ​​using an xgboost regression estimator.

37. determining, for each estimator of the plurality of estimators, by the one or more processors, a set of features that are most predictive of an output of the estimator; and 34. The method of claim 33, wherein receiving the plurality of attribute values ​​associated with the small scale cell culture comprises receiving only attribute values ​​that are included within the set of features determined for the machine learning based regression estimator.

38. 21. The method of claim 20, further comprising measuring at least some of the plurality of attribute values ​​associated with the small scale cell culture with one or more analytical instruments.

39. 21. The method of claim 20, wherein receiving the plurality of attribute values ​​comprises receiving measurements from an optical-electronic cell line generation and analysis system.

40. One or more non-transitory computer readable media storing instructions that, when executed by one or more processors of a computing system, cause the computing system to perform the method of any one of claims 20 to 39.

41. 1. A computing system comprising: one or more processors; and One or more non-transitory computer readable media storing instructions which, when executed by said one or more processors, cause said computing system to perform the method of any one of claims 20 to 39. A computing system comprising:

Images