Alternative titer targeting systems and methods

Abstract

Embodiments relate to alternative titer target production. The subject matter of the embodiments of the present invention provides a computer-implemented method, a computer system, and a computer-readable storage medium for predicting the outcome of an alternative titer target production method.

Description

TECHNICAL FIELD

[0001] The systems, methods, and computer programs disclosed herein relate to biopharmaceutical production.

[0002] CROSS-REFERENCE TO RELATED APPLICATIONS

[0003] This application claims the benefit of U.S. Provisional Patent Application No. 63 / 537,583, filed September 11, 2023, the disclosure of which is incorporated by reference in its entirety. BACKGROUND

[0004] Production of biologics requires a series of complex processes, which are broadly classified into drug substance (DS) production and drug product (DP) production. DS production is characterized by key unit operations of cell culture and purification, while DP production encompasses key steps of sterile fill-finish. Sterile fill-finish unit operations include mixing of DS with excipients, including bulking, filling, lyophilization, capping, inspection, and packaging by validated processes. More specifically, bags of frozen drug substance are thawed, pooled, and partially diluted with formulation buffer. Next, the partially diluted bulk is tested for potency, sterile filtered, and further diluted with formulation buffer to a predetermined target potency concentration (also referred to as bulk potency target). Subsequently, the sterile bulk is filled into glass vials, lyophilized, and capped to produce drug product (DP) vials.

[0005] In the inspection step, quality control (QC) tests the DP vials according to standard operating procedures (SOPs) to ensure that all critical quality attributes (CQAs) are within the specification limits, which is a requirement for commercialization release. The specification limits for various DP CQAs are regulated by health authorities to ensure the safety and effectiveness of the produced DP. Typically, individual vials are tested to be within the specification limits, however, in some cases, an average in-process action limit can be implemented. DP batches with CQAs outside the validated range are considered non-compliant for commercialization release. Therefore, a continuous process validation (CPV) plan is implemented to provide continuous assurance that the production process remains in control (validated state) during commercial production, demonstrates understanding of process variability at commercial scale, and drives process changes.

[0006] The CQA of primary interest is the potency of the DP vials, which is monitored by the CPV program and is known to be controlled by the batch potency target concentration set by the DP production's scale-up phase. Intuitively, increasing or decreasing the batch potency target will cause the DP vial potency distribution mean to shift proportionally and affect the fraction of DP batches that are out-of-specification (OOS) at release. Since process optimization is desirable, the refinement of the batch potency target offers the opportunity to improve process capability (robustness) by reducing the probability (risk) of OOS at release.

[0007] A given (current) batch potency target is set for the DP production process for each fill size (dosage strength) of the DP. The current batch potency target for the DP is typically changed based on production experience (subject matter expertise) to ensure business and quality requirements are met. Historically, a statistical evaluation of the theoretical (alternative) potency targets is performed prior to making batch potency target adjustments, including estimating the OOS risk at release for each alternative potency target of interest. While commercial software exists to perform these analyses, the analyses are typically time-consuming to perform in an ad hoc manner due to the manual steps involved in the workflow. The complete workflow required to evaluate alternative potency targets involves a series of manual steps involving data collection, data pre-processing, and statistical calculations. Automated statistical analysis workflows are desirable to facilitate timely responses, particularly when OOS events require decisions related to good manufacturing practice (GMP).

[0008] In addition to limiting the efficiency of the overall workflow, commercial software is also limited in terms of customizability and accessibility. The functionality and user interface (UI) of commercial software is typically fixed unless a major update is made to the software, which does not accommodate changing user needs. Furthermore, traditional commercial software requires installation on a user's local operating system and requires local computing resources, which can limit future accessibility and scalability for the entire organization. This is in contrast to cloud-based software, which relies on a network of remote servers to provide computing resources (i.e., storage, processing power, etc.).

[0009] There is a need for automated evaluation of alternative potency targets and methods. Development of a system would be used to facilitate efficient statistical evaluation of alternative potency targets and potentially prevent future deviations, product loss, and related research efforts. Deployment of any system would facilitate end-user accessibility and scalability by eliminating overhead associated with managing their local computing environment (i.e., installation of programming languages and software packages).

[0010] In summary, there is a need for systems and methods for implementing a potency target system that can be used to help production process scientists efficiently evaluate alternative potency targets for DPs in biologic product manufacturing. Furthermore, there is a need for software systems that are quickly, efficiently, and effectively deployed to achieve such results. These and other issues are addressed by embodiments of the present invention. SUMMARY

[0011] In a first aspect, embodiments provide a computer-implemented method comprising: providing a front-end dashboard comprising user inputs for inputting parameters into a data acquisition module for determining a replacement potency target, and outputs for displaying processed results; providing a back-end for: pre-processing data from the data acquisition module, including normalization; applying statistical modeling calculations using out-of-specification (OOS) risk selection criteria to the pre-processed data; and outputting the results; wherein the method determines the replacement potency target, and the outputs display corresponding OOS risks.

[0012] Embodiments provide a computer-implemented method and system, wherein the user input parameters comprise a drug product, a fill size, a batch selection, and a production date range.

[0013] Embodiments provide a computer-implemented method and system, wherein the data acquisition module further comprises a source data system.

[0014] Embodiments provide a computer-implemented method and system, wherein the data acquisition module further comprises a data acquisition system.

[0015] Embodiments provide a computer-implemented method and system, wherein the data acquisition module further comprises a cloud-based data storage.

[0016] Embodiments provide a computer-implemented method and system, wherein the data acquisition module further comprises cloud-based computing.

[0017] Embodiments provide a computer-implemented method and system, wherein the pre-processing step further comprises normalizing the data by statistical analysis.

[0018] Embodiments provide a computer-implemented method, wherein the data is normalized by calculating Pnormalized according to the following equation:

[0019]

[0020] wherein:

[0021] P = potency value,

[0022] P CT = current batch potency target, and

[0023] P TM = batch potency target at production.

[0024] Embodiments provide a computer-implemented method, wherein the OOS risk selection criteria for candidate replacement potency targets can be determined from a model, wherein:

[0025] 1) : OOS risk that at least one individual replicate result does not meet the upper release limit, given that the calculated CV (coefficient of variation) is within user-defined limits;

[0026] 2) : OOS risk that at least one individual replicate result does not meet the lower release limit, given that the calculated CV is within user-defined limits;

[0027] 3) : OOS risk that the average result does not meet the lower average release limit, given that the calculated CV is within user-defined limits; and

[0028] 4) : OOS risk of failing in at least one of the above cases 1-3.

[0029] Embodiments provide a computer-implemented method, wherein the data collection module further comprises a cloud-based relational database.

[0030] Embodiments provide a computer-implemented method, wherein data querying is enabled between the user input and the cloud-based relational database.

[0031] In another aspect, embodiments provide a computer-implemented system, comprising: a front-end dashboard comprising user input for inputting user input parameters into a data collection module for determining a surrogate potency target, and output for displaying processed results; a back-end for: pre-processing data from the data collection module, including normalization; applying statistical modeling calculations using out-of-specification (OOS) risk selection criteria to the pre-processed data; and outputting the results; wherein the system determines the surrogate potency target, and the output displays the corresponding OOS risk.

[0032] Embodiments provide a computer-implemented system, wherein the user input parameters comprise drug product, fill size, batch selection, and production date range.

[0033] Embodiments provide a non-transitory computer readable storage medium having software instructions stored thereon that, when executed by a processor of a computer system, cause the computer system to perform a method of allowing a user to input user input parameters into a data acquisition module to determine a surrogate potency target, and an output for displaying processed results; pre-processing data from the data acquisition module, including normalization; and applying statistical modeling calculations using Out of Specification (OOS) risk selection criteria to the pre-processed data; and outputting the results; wherein the method determines the surrogate potency target, and the output displays the corresponding OOS risk.

[0034] These and other features of the present teachings are set forth herein. BRIEF DESCRIPTION OF DRAWINGS

[0035] Those skilled in the art will understand that the drawings described below are for illustrative purposes only. The drawings are not intended to limit the scope of the present teachings or the claims in any way.

[0036] Figure 1 Embodiments of a computer system pipeline architecture for a potency target system in a cloud-based environment are shown. The main components of the data pipeline include source data systems, data integration systems, cloud data storage, and cloud computing (Domino Data Lab) platforms. The source data systems are illustrative and serve as examples of some of the data science systems that can be used.

[0037] Figure 2 Embodiments of a potency target system architecture are shown.

[0038] Figure 3 A flowchart for a statistical computing workflow is shown, including confidence interval estimation. User inputs including formulation, fill size, and production date range define the relevant potency data for analysis. Historical batch potency target adjustment information is used to normalize raw potency result data to the current batch production potency target. Potency data is then simulated at the surrogate potency target(s) of interest and used for OOS risk estimation at release.

[0039] Figure 4 A data range portion of the potency target system UI is shown, with Drug A selected to be obtained from an online data source, Discoverant, https: / / www.3ds.com / products-services / biovia / products / manufacturing-analytics / biovia-discoverant / (see also Discoverant User Guide. Aegis Analytical Corporation, 2001).

[0040] The dashboard allows users to use a combination of release valence results from Discoverant and offline data sources (such as .csv or .xlsx files). In this example, the online data source "Discoverant" is used when no offline data source is uploaded.

[0041] Figure 5 The data filter section of the valence target system is shown, allowing selection of the fill size #1 between the production start date 01 / 01 / 2022 and the production end date 04 / 01 / 2023. As shown in the batch selection drop-down list, a total of 17 DP batches are available for further analysis.

[0042] Figure 6 The data normalization section of the potency target system is shown, which is used to normalize the potency result of drug A at release for fill size #1. The release-time potency results of all available data are normalized to the current batch potency target of "118" IU / mL.

[0043] Figure 7 This section illustrates the OOS risk estimation (simulation) parameter settings for the potency target system. The selected parameter settings and specification limits are for illustrative purposes. Specification limits A, B, and C are used for drug A fill size #1, representing low vial size, high vial size, and low average size, respectively. The selection of simulation parameters is based on user guide recommendations.

[0044] Figure 8 Bar graphs show the valence target system for different types of release OOS risk estimates for each different alternative valence target. Error bars represent three standard deviations (SD) limits, and the height of the bars is the median of the OOS risk estimate. OOS risk estimates are provided for low vial, high vial, low average, and overall.

[0045] Figure 9 The OOS risk estimation (simulation) parameter settings section of the potency target system is shown, with specified assay uncertainty. The specification limits for fill size #1 of drug A and the simulation parameters remain unchanged (with...). Figure 7 compared to).

[0046] Figure 10 Bar graphs show different types of release OOS risk estimates (based on specified measurement uncertainties) for each of the different alternative valence targets. Error bars represent three SD limits, and the height of the bars is the median of the OOS risk estimate. OOS risk estimates are provided for low vial, high vial, low average, and overall.

[0047] Figure 11is a block diagram of an exemplary computer system of the present disclosure, which is suitable for determining one or more surrogate potency results of a surrogate potency method based on at least two surrogate potency process parameters. DETAILED DESCRIPTION

[0048] Embodiments will be stated more particularly below, without distinguishing between aspects of the embodiments (methods, computer systems, computer-readable storage media). Rather, the following discussion will be understood to apply equally to all aspects of the embodiments, regardless of context (method, computer system, computer-readable storage media) in which they occur.

[0049] If in this specification or claims, steps are recited in a sequential order, this does not necessarily mean that the embodiments are limited to that order. Rather, it is contemplated that the steps can be performed in different orders or in parallel to each other, unless one step builds on another step, which absolutely requires the latter to be performed subsequently (however, this is clear in individual cases). The recited order is therefore an embodiment of the present disclosure.

[0050] As used herein, the articles “a” and “an” are intended to include one or more items, and can be used interchangeably with the phrase “one or more.” As used in the specification and claims, the singular forms “a,” “an” and “the” include plural referents unless the context clearly dictates otherwise. Where only one item is intended, the term “one” or similar language is used. Also, as used herein, the terms “has,” “have,” “having,” or the like are intended to be open-ended terms. Further, the phrase “based on” is intended to mean “based, at least in part, on” unless explicitly stated otherwise. Moreover, the phrase “based on” can mean “in response to” and indicates that a specified operation is triggered in response to a condition, as appropriately mentioned herein, by an electronic device (e.g., controller, processor, computing device, etc.) automatically.

[0051] The operations in accordance with the teachings herein can be performed by a computer system at least one specifically programmed for the desired purposes or a general-purpose computer system specifically configured for the desired purposes by means of at least one computer program stored in a typical non-transitory computer readable storage medium.

[0052] A “computer system” is a system for electronic data processing, which processes data by means of programmable computing rules. Such a system typically comprises a “computer”, which unit comprises a processor for performing logical operations and peripheral devices.

[0053] In computer technology, "peripheral device" refers to all devices that are connected to a computer and serve to control the computer and / or as input and output devices. Examples thereof are monitors (screens), printers, scanners, mice, keyboards, drives, cameras, microphones, loudspeakers, etc. Internal ports and expansion cards are also considered peripheral devices in computer technology.

[0054] Today's computer systems are often divided into desktop PCs, portable PCs, laptops, notebooks, netbooks and tablet PCs, and so-called handheld devices (e.g. smartphones); all of which can be used to execute embodiments.

[0055] The term "non-transitory" as used herein is intended to exclude transitory propagating signals or waves, but otherwise include any volatile or non-volatile computer storage technology suitable for a system.

[0056] The term "computer" is intended to be broadly interpreted to encompass any type of electronic device with data processing capabilities, including, by way of non-limiting examples, personal computers, servers, embedded cores, computing systems, communication devices, processors (e.g., digital signal processors (DSPs)), microcontrollers, field-programmable gate arrays (FPGAs), system-on-a-chip (SoC) systems, and other electronic computing devices.

[0057] The term "process" as used above is intended to encompass any type of computation or manipulation or transformation of data representing physical (e.g., electronic) phenomena as physical (e.g., electronic) phenomena, which can occur or reside in, for example, the registers and / or memories of at least one computer or processor. The term processor includes a single processing unit or multiple distributed or remote processing units.

[0058] "Computer-readable medium" refers to any component capable of embodying, storing, communicating, propagating or transporting one or more of an executable code or data.

[0059] "Computer-readable storage medium" refers to any computer readable medium capable of storing data (e.g., executable code).

[0060] "Executable code" refers to a set of instructions in any language, which can be used by or in conjunction with a physical computing device such as a computer.

[0061] "Display device" refers to any component capable of displaying results digitally, electronically or visually.

[0062] "Memory" refers to any computer readable medium capable of storing data and / or executable code.

[0063] "Network" means two or more physical computing devices connected by any type of communication line or communication protocol.

[0064] "Output" means one or more of transmitting, conveying, or displaying information, data, executable code, or any result from executing executable code.

[0065] "Processor" means a component capable of executing a method encoded by executable code.

[0066] "Receive" means one or more of accepting, retrieving, or obtaining information, data, executable code, or any result from executing executable code.

[0067] "Server" means a device or machine that serves a network.

[0068] "Data" means any information in any language capable of being used or combined with a physical computing device (e.g., a computer).

[0069] "Digital" means that the weed map can be processed by a machine such as a computer system.

[0070] Some embodiments of the disclosure will be described below with reference to the accompanying drawings, in which some but not all embodiments of the disclosure are shown. Indeed, various embodiments of the disclosure can be embodied in many different forms and should not be construed as limited to the embodiments set forth herein; rather, these example embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the disclosure to those skilled in the art.

[0071] Biopharmaceutical production requires a series of highly regulated steps. Throughout the various stages of drug product (DP) production, various critical quality attributes (CQAs) are tested according to regulatory specifications guidelines. DP potency concentration measures the dose strength of a particular DP, which is a very interesting CQA, as each DP fill size is different according to its batch potency target. Therefore, when implementing a DP batch potency target change, the probability (risk) of the theoretical (alternative) potency target being out of specification (OOS) needs to be assessed.

[0072] Data

[0073] According to the provisions of many production standard operating procedures (SOPs), the produced DP is measured multiple times at many QC analysis laboratories before commercial release to test various CQAs. As shown in Figure 1 the potency results of individual vials (along with other CQAs) are assessed before commercial release and stored in an internal database.

[0074] Data sources and pipelines

[0075] As Figure 1 shown, some internal data systems include several source data systems, a data integration system, and cloud data storage.

[0076] Individual vial potency results and related metadata (e.g., batch production date, fill size, product, etc.) for a release DP batch are initially obtained through various source data systems. Using the BIOVIA Discoverant data integration system, data from the various source data systems can be obtained.

[0077] According to the provisions of the product production standard operating procedures (SOP), a produced DP batch is measured multiple times at a QC analysis laboratory prior to commercial release to detect various CQAs. The measurement results are stored in a laboratory information management system (LIMS) database, and the batch information is stored in an enterprise resource planning (ERP) system database.

[0078] The source data from the LIMS and ERP databases are ingested into cloud-based storage for use in software applications. Amazon Web Services (AWS) is used to provide the cloud storage. Specifically, a PostgreSQL relational database (AWS-RDS) is used to store the data for easy access by the applications.

[0079] Within the scope of the embodiments, other data integration systems can be used with the present embodiments. For example, other data integration systems include ETQ Reliance QMS, Arena PLM and QMS, Greenlight Guru, Teamcenter, Qualio, Ideagen Quality Management, MasterControl Quality Management System, G2 Deals, Virgin Pulse, Propel, and Orcanos. Other data integration systems known or used in the art can be used with the present embodiments.

[0080] Analysis Groups (AGs) are defined in Discoverant to periodically query various source data systems for the most recent and available data and place it in context by batch number. Amazon Web Services (AWS) provides a variety of cloud computing services that can be used to support cloud deployment of the system. External data access is a feature of Discoverant that enables third-party systems to retrieve and use Discoverant AG data. An Extract-Transform-Load (ETL) pipeline (e.g., AWS Glue) was developed to periodically extract historical process data from Discoverant, transform it into an appropriate data structure (e.g., a spreadsheet), and load it into a cloud storage system. For potency target systems, a PostgreSQL relational database (AWS-RDS) was used to store the data to facilitate system access.

[0081] Within the scope of embodiments, other open source relational databases that connect with a wide range of applications and services can be employed in addition to PostgreSQL. For example, other embodiments can include, but are not limited to, Microsoft SQL Server, MSQL, Oracle Database, MongoDB, Apache Cassandra, MySQL, Redis, DynamoDB, IBM Cloudant, Orient DB, Jaguar DB, CouchDB, Firebase, ArangoDB, CockroachDB, RethinkDB, ScaleGrid, RavenDB, Intersystems IRIS, Knack, MariaDB, and Airtable. Other relational databases known or used in the art can be used with the present embodiments.

[0082] System Architecture

[0083] The potency target system includes multiple modules that work in concert, including data acquisition, pre-processing, statistical calculations, front-end visualization, and reporting. An overview of these modules is shown in Figure 2 .

[0084] The front end is an interactive dashboard that allows users to specify drug product potency data to query from AWS-RDS and to evaluate alternative potency targets. The data is then pre-processed by the backend Python library before performing statistical calculations based on user-specified simulation parameters. The output is provided in graphical and tabular formats, displayed on the dashboard’s user interface (UI) and captured in an automatically generated PDF report. Throughout the development cycle, software engineering best practices of object-oriented programming, version control (GitLab), and inline code documentation were employed throughout the python codebase (backend and frontend) to facilitate efficient collaboration among developers working on the potency target library.

[0085] Within the scope of embodiments, other alternatives to GitLab can be used with the present embodiments. Version control hosting software is a widely used technology that is suitable for users searching for simple software solutions with customization. Other solutions can include software that includes project management and further software development. Other options for GitLab can include, but are not limited to, GitHub, CloudBees, G2 Deals, CircleCl, Jenkins, Red Hat Ansible Automation Platform, Azure Pipelines, Bitbucket, Azure DevOps Server, Copado Cl / CD, and Jira.

[0086] Within the scope of embodiments, other code libraries or libraries can be used with the present embodiments. For example, Python code library tools provide the functionality to create, open, read, and index data and tables in formats for use with many popular software products. Other code library tools include Java, NodeJS, PHP, Ruby, Golang, and Scala. Other embodiments known in the art can be used with the present embodiments.

[0087] Further, other libraries can be used with the present embodiments, including Tkinter / pyQt, Pandas / Numpy, Tensorflow / Pytorch, Matplotlib / Bokeh, and Numpy / (Are you kidding me).

[0088] Within the scope of embodiments, other languages or code can be used with the present embodiments. For example, C++, C#, or R can be used with the present embodiments instead of Python. Further, other languages or code known in the art can be employed.

[0089] Below, the system backend development flow is discussed. Subsequently, the front end UI (dashboard) development and overall validation process are detailed in other sections, respectively.

[0090] System Backend (Python Library) Development

[0091] The backend is a statistical software library developed in python programming language to provide the necessary functionalities for the frontend dashboard, while cleverly modularizing the python code for future reusability. As shown in Figure 2 Figure 1, the steps required for potency target evaluation require data collection, data pre-processing, and statistical computation. Each of these steps is developed as part of a separate code module, which leverages other open-source python libraries to facilitate development. The methods taken for data pre-processing and statistical computation are detailed in other sections respectively.

[0092] Data Pre-processing

[0093] For each dosage strength (file size), the DP batch average potency value is calculated as the overall historical average of normalized potency measurements. Potency data is normalized before being used for statistical analysis. For the fill size of interest, if the dataset contains final container potency results with different batch potency targets, the final container potency results from batches produced with previous potency targets will be normalized to the current potency target. For example, the final container potency result (P) is divided by the batch potency target at the time of production, then multiplied by the current batch potency target, to calculate the normalized final container potency result (P Normalized ):

[0094]

[0095] where:

[0096] P = potency value,

[0097] P CT = current batch potency target, and

[0098] P TM = batch potency target at the time of production.

[0099] According to Equation 1, if there is no batch potency target change for the DP of a given fill size, the normalized potency result (P Normalized ) will be equal to the original potency result (P).

[0100] Statistical Computation and Modeling Methods

[0101] The probability of a DP potency result exceeding the acceptable limit is influenced by multiple factors. The potency target of the batch influences the position of the distribution mean relative to the release limit. At a fixed batch potency target, the average potency result of a DP lot can be higher or lower than the distribution mean due to process factors that cause lot-to-lot variability and potential laboratory-to-laboratory test differences. In addition, process fill weight variations and test method precision also cause variability between individual vial potency results during lot release testing. It is challenging to analytically determine the joint probability distribution of this situation. Therefore, Monte Carlo simulation is used to combine the within-lot and lot-to-lot probability distributions and numerically estimate the overall risk of a DP potency result exceeding the release limit for an alternative batch potency target option.

[0102] Within the scope of the embodiments, other simulation methods and software can be used with the embodiments of the present invention. For example, other simulation alternatives to Monte Carlo simulation known in the art can be employed.

[0103] The ultimate goal of this statistical calculation module is to calculate the OOS risk for a candidate batch potency target. The following types of OOS risk can be calculated numerically:

[0104] 1) : The OOS risk that at least one individual replicate result does not meet the upper release limit, given that the calculated CV is within the user-defined limit.

[0105] 2) : The OOS risk that at least one individual replicate result does not meet the lower release limit, given that the calculated CV is within the user-defined limit.

[0106] 3) : The OOS risk that the average result does not meet the lower average release limit, given that the calculated CV is within the user-defined limit; and

[0107] 4) : The OOS risk of failing at least one of the above.

[0108] For example, the OOS risk of the above types produces the best result for a candidate potency target. For example, , , and The batch potency target can fall within a given limit for each possible case. The accuracy and precision of this prediction depends on the data set available and used in the calculation. Obviously, a larger data set is preferred. However, in some cases, this can not be possible. For example, the data set contains only a few batches outside of the specification, which makes a reliable estimation of the OOS risk challenging in providing sufficient accuracy and precision. Therefore, it is counterintuitive to use the above module to calculate or predict the batch potency target. The capability of the present method and system lies in being able to provide accurate and precise results using the above module with almost no raw or actual data or input provided about the batch potency target. Both software and hardware can be used to simulate an accurate and precise data set. This can result in faster and cheaper results for the user. Moreover, by way of example, the above module can also be employed in the presence of a large amount of available actual historical data set.

[0109] For a given type of OOS risk, the following section details the statistical analysis of the DP potency release data performed as part of the Monte Carlo simulation.

[0110] Monte Carlo simulation

[0111] The size of the data set (number of batches) is not large enough to perform a numerical calculation of the OOS risk. Therefore, a Monte Carlo simulation method is used to generate a number of synthetic potency data points. As a first step, sample statistics are estimated from the historical potency results normalized at release to characterize the DP potency distribution mean and the factors that contribute to the variability of the potency results. The DP potency distribution mean (μ) is estimated as the overall mean of all potency results at release for all DP batches. While the variance component analysis (VCA) is a standard statistical method to decompose the total variability into the variability explained by different factors. More specifically, the VCA is applied to the potency data set to determine the between-batch variability (i.e., process variability), the between-lab variability, and the within-batch or assay variability (σ2) as a percentage of the total variability.

[0112] For a specified potency target of interest, the jth batch mean potency result (Yj) and the kth single replicate result (Yjk) are randomly generated using normal distributions described by Equations (2) and (3), respectively.

[0113]

[0114]

[0115] is the estimate of the DP potency distribution mean,

[0116] and​​​​​ are the replacement and current batch potency target, respectively, and , and are the estimates of between-batch variability, between-lab variability, and within-batch (assay variability), respectively.

[0117] A total of simulated batches are generated using equations (2) and (3), each containing potency values for the number of replicates. The number of replicates is determined according to the production specification and fill size of the drug product. Discussion of the appropriate number of simulated batches is provided below.

[0118] For each simulated batch, the coefficient of variation (CV) is calculated using the individual potency values for the i-th simulated batch and compared to the test method sample suitability criterion CV ). If the results of the batch meet the suitability criterion , the simulated batch is considered valid, otherwise it is classified as invalid.

[0119] Finally, for each valid simulated batch, a determination of OOS or non-OOS (1 or 0) is made according to the OOS risk definition. The results for all simulated batches are averaged to calculate the respective OOS risk. For example:

[0120]

[0121] where:

[0122]

[0123] is the minimum potency result value for the j-th valid simulated batch, containing replicates.

[0124] The number of simulated DP batches at the replacement potency target of interest is determined according to the convergence of the estimated OOS risk at release. Convergence is determined by the convergence criterion , where the difference in the estimated OOS risk at release between adjustments in does not exceed .

[0125] Convergence is assessed iteratively and the specific value of is determined through discussion with the process expert, discussing what a meaningful difference in risk is and how it will impact the decision. Convergence can be assessed iteratively and can be determined through discussion with the process expert The specific value of

[0126] Quantifying sampling uncertainty

[0127] To estimate the sampling uncertainty of the point estimates of OOS risk, a bootstrap method was used to construct confidence intervals. The same procedure previously described in the modeling method was repeated times using data subsets generated by resampling the historical normalized potency results. Specifically, resampling describes a process of randomly selecting potency results from the original set of potency results and replacing them until a data set of the same size is constructed. Under the bootstrap method, each data point from the original data set is allowed to be picked as many times as the size of the original data set, or not at all. For each risk type, the OOS risk was estimated and the respective confidence interval (C.I.) was calculated using the appropriate percentiles of the OOS risk distribution generated from the OOS risk estimates. For example, the and percentiles of the bootstrap distribution were used to construct the C.I. Figure 3 The workflow of OOS risk calculation and confidence interval calculation at release is shown.

[0128] Simulated bootstrap data sets The number of bootstrap data sets was determined based on the convergence of the OOS risk estimates at release. Convergence was determined by the convergence criterion where the difference in the OOS risk estimates (upper and lower limits of the C.I.) between adjustments in did not exceed Alternatively, convergence can be tested iteratively and the specific value of was determined through discussion with process experts, discussing what a meaningful difference in risk is and how it would affect decision making.

[0129] System front-end (dashboard) development

[0130] The system front-end included an interactive dashboard that allowed users to evaluate alternative potency targets using OOS risk at release as an evaluation metric. Users could perform intermediate analyses including distribution visualization, variance component analysis, and Shewhart control charts to further explore historical data. A modular python library (back-end) was developed to include querying data from the RDS database, data processing, and statistical calculations. The dashboard leveraged the back-end library to perform the required calculations under the hood and is described in this document (see System back-end Python library development).

[0131] The dashboard itself is implemented using Python's Dash. Dash is an open-source framework for building data visualization interfaces; it provides a python interface for creating flexible interactive code and customizable systems that directly connect to analysis code. Dash unifies the graphics library Plot and other dashboard components with the general web system framework Flask. The use of an open-source data visualization tool addresses many of the limitations imposed by commercial software, including limitations in transparency and customizability. The system is currently hosted through a cloud-based data science platform called Domino Data Lab (available over the internet from their website), which facilitates the development and deployment of various data science systems.

[0132] Alternative cloud and non-cloud based data science platforms exist that can be employed with embodiments of the present invention. For example, other platforms can include MATLAB, Posit, Alteryx, RapidMiner, SAP HANA Cloud, Qubole, Databricks Lakehouse Platform, and IBM Watson Studio, instead of using Domino Data Lab. Other platforms known in the art can be used with embodiments of the present invention.

[0133] Aspects of the present teachings will be further appreciated upon considering the following examples, which are not to be construed as limiting the scope of the present teachings in any manner.

[0134] The next section introduces several examples of potency target system, emphasizing how it can be used to support biologic product manufacturing. First, a demonstration of how end users interface with the UI to obtain relevant data for analysis is described, followed by examples of how the output of the potency target system can be used in practice. For simplicity, the following sections summarize the actual inputs and values.

[0135] Figure 4 An illustration of how the dashboard inputs can be used to obtain potency results at release for a theoretical DP "Drug Product A" is depicted.

[0136] After obtaining all available historical potency results at release from Discoverant, the user can choose to further specify a particular fill size of interest and limit the analysis to a specific batch production date range (by selecting a start and end date).

[0137] Figure 5The UI is shown as it is used to limit the historical release potency results for Drug A to a specific fill size, "Fill Size #1", produced between January 1, 2022 and April 1, 2023. The "Select Batches" section of this step dynamically updates the available batches in the drop-down list to provide the user with visual feedback on the number of available batches given the data filtering criteria used, as well as the option to deselect specific DP batches from the analysis.

[0138] Once the obtained release potency results are filtered based on the desired fill size and batch production date range, the user can use historical process knowledge of batch potency target changes prior to the analysis to normalize all potency results to the current batch potency target. For Fill Size #1 of Drug A, the batch potency target was 123 IU / mL starting at Date #1. At Date #2, the batch potency target was lowered by 3 IU / mL. Finally, at Date #3, the batch potency target was lowered again by 2 IU / mL. Figure 6 It is described how the user would use the potency target system to normalize the release time potency results to the current batch potency target of 118 IU / mL.

[0139] The following subsections present examples of the potency target system assuming the input data has been collected, filtered, and normalized as outlined herein.

[0140] Example 1: Alternate Potency Target Evaluation

[0141] Recall that the potency target system was developed to evaluate DP alternate potency targets using release OOS risk as the evaluation criteria. For Fill Size #1 of Drug A, alternate potency target values of 106 IU / mL, 110 IU / mL, 115 IU / mL, 118 IU / mL, and 123 IU / mL are of interest. Prior to the release OOS risk estimates for each alternate potency target can be computed, the user must set the required simulation parameters in the OOS Risk Estimate (Simulation) Parameters section of the potency target system (as described herein).

[0142] Figure 7 It is depicted how the UI is used to input the required simulation parameters. For each alternate potency target, several types of release OOS risk estimates are estimated (vial low, vial high, average low, and overall). The analysis results are summarized in a table on the dashboard UI; however, the results are most easily interpreted when visually presented.

[0143] Figure 8 It is shown how the bar chart presented using the dashboard is used to easily evaluate the alternate potency targets based on the release OOS risk.

[0144] Based on the assessment of the surrogate potency targets, it is identified that the surrogate potency target of 115 IU / mL has the lowest estimated risk of an OOS at release (for all types). Thus, for Drug A fill size #1, the optimal surrogate potency target can be relatively quickly decided when considering the risk of an OOS at release. There can be other cases where the selection of the optimal surrogate potency target cannot be straightforward. For example, the potency target for a batch that estimates an acceptable risk of an OOS can not be within the practical range, or can have a lower risk of an OOS for a given type of OOS, but a higher risk of an OOS for another type of OOS (i.e., lower risk of vial low OOS, but higher risk of vial high OOS). In such cases, the production process scientist must utilize their subject matter expertise to consider which factors are useful when considering potential surrogate potency targets.

[0145] Example 2: Assessment of a new assay based on the risk of a release OOS

[0146] As previously mentioned, individual vial potency results are affected by assay variability (this is explicitly shown by Equation 3). Thus, it is also possible that assay variability can affect the estimation of the risk of an OOS at release. Thus, an additional example has been identified in which the potency target system can be used to assess a new potency assay (when assuming that the theoretical assay variability is known). The potency target system dashboard provides an optional user input panel labeled “Specify Assay Uncertainty (CV%)” to assess the risk of an OOS at release for an assay with a theoretical assay uncertainty in CV%. By providing an input for the assay uncertainty, the user can assess the risk of an OOS at release for surrogate potency targets using the new assay uncertainty, rather than using the assay uncertainty estimated from historical potency results. As previously mentioned, the results estimation of the risk of an OOS at release for each surrogate potency target can be most easily visually assessed using a bar chart.

[0147] When compared to the results seen in Figure 7 Figure 10 The corresponding surrogate potency targets show a lower risk of an OOS at release. Thus, in this illustrative example, the theoretical assay (with an assay uncertainty of 2% CV) improves process robustness by reducing the risk of an OOS at release. With the overall reduction in the risk of an OOS at release, the surrogate potency targets of 115 IU / mL and 118 IU / mL now both appear to be viable options when considering the implementation of a new potency assay. As mentioned in the previous use case (Example 1), other factors besides the risk of an OOS at release will be considered when determining the appropriate surrogate potency target. In addition to the risk of an OOS at release, the production process expert will also make a comprehensive assessment of these considerations to guide the decision regarding the appropriateness of a surrogate potency target.

[0148] ​Example 3: Assessing the action limit and potency target

[0149] So far, the assessment of surrogate potency targets with fixed specifications and action limits has been discussed. However, in practice, there can be situations where adjusting the action limits for various CQAs is considered. For example, as a mitigation measure, the batch average (lower limit) action release limit for DP potency can be re-evaluated. In this case, it can be useful to assess the release OOS risk estimates for surrogate potency targets while considering multiple candidate average action limits. Table 1 illustrates how the overall release risk of OOS for surrogate potency targets of 115 IU / mL, 118 IU / mL, and 123 IU / mL can be estimated using average action limits of 98 IU / mL, 103 IU / mL, and 108 IU / mL

[0150] Table 1: Selected results for overall OOS risk at release for formulation A fill size #1 given various combinations of average potency limit and batch potency target

[0151]

[0152] In practice, the results shown in Table 1 can be achieved by utilizing the potency target system as described earlier, i.e., running simulations with different settings of average action limit and tabulating the results of each run as a row of a table. The results shown in Table 1 can be used to support establishing the appropriate surrogate potency targets and average potency limits needed to achieve maximum process capability (lowest risk of DP potency failure).

[0153] One aspect of the embodiments includes the process of developing and deploying a software system for simplifying the assessment of surrogate potency targets for DPs in biologic product manufacturing. The system is deployed using various cloud-based technologies, and an interactive dashboard is developed to enable production process scientists to effectively use the system. The utility of the system is illustrated through several examples involving the assessment of surrogate potency targets, the assessment of new potency assays, and the assessment of both specification limits and surrogate potency targets.

[0154] In general, potency target systems are developed in biologic product manufacturing to support GMP decisions and improve the efficiency of the overall decision-making process. Before the results generated by the system can be used to support GMP decisions, the system needs to be fully validated to provide certainty in its results and confirm the accuracy of its reported results.

[0155] Other embodiments can involve establishing a continuous integration and continuous deployment (CI / CD) pipeline to improve the frequency of system delivery to end users by automating code testing and builds and automating system deployment to cloud servers. Implementation of a CI / CD pipeline will require establishing production and development environments, where the development environment will serve as a safe space for code developers to implement new functionality for the system, and the production environment will remain for deploying tested updates to the final system. The benefit of implementing a CI / CD pipeline is to minimize the risk of the final system delivered to end users being affected by accident. A natural extension is to integrate the set of validation steps described for GMP systems into the CI / CD pipeline. Only validated changes that are made in the development environment (i.e., pass unit testing, integration testing, statistical result validation, etc.) are pushed to the production environment. Ideally, this implementation will ensure that updates to the software system have been validated for GMP use before reaching end users.

[0156] Embodiments of the invention can be performed using a computer system. Figure 11 An example computer system 200 is shown. In connection therewith, the computer system 200 can be configured by way of executable instructions to implement various algorithms and other operations described herein.

[0157] The example computer system 200 can include, for example, one or more servers, workstations, personal computers, notebook computers, tablet computers, smartphones, other suitable computing devices, combinations thereof, and the like. Moreover, the computer system 200 can include a single computing device, or it can include multiple computing devices that are located adjacent to one another or distributed across a geographic area and coupled by one or more networks. Such networks can include, without limitation, the Internet, an intranet, a private or public local area network (LAN), a wide area network (WAN), a mobile network, a telecommunications network, combinations or other suitable networks thereof, and the like.

[0158] In summary, the illustrated computer system 200 includes a processing unit 202 and a memory 204 coupled to (and in communication with) the processing unit 202. The processing unit 202 can include, without limitation, one or more processors (e.g., in a multi-core configuration, etc.), including central processing units (CPUs), microcontrollers, reduced instruction set computer (RISC) processors, and systems on a chip (SoCs), programmable logic devices (PLDs), gate arrays, and / or any other circuit or processor designed to carry out the functions described herein. The above list is exemplary only, and thus is not intended to limit the definition and / or meaning of the term processing unit in any way.

[0159] As described herein, the memory 204 is one or more devices that enable information, such as executable instructions and / or other data, to be stored and retrieved. The memory 204 can include one or more computer-readable storage media, such as, but not limited to, dynamic random access memory (DRAM), static random access memory (SRAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), solid state memory devices, flash memory drives, CD-ROM, thumb drives, magnetic tapes, hard drives, and / or any other type of physical or tangible computer-readable media. The memory 204 can be configured to store, without limitation, process parameters, outputs, models, and / or other types of data (and / or data structures) suitable for use as described herein, etc. In various embodiments, computer-executable instructions can be stored in the memory 204 to be executed by the processing unit 202 to cause the processing unit 202 to perform one or more of the functions described herein, such that the memory 204 is a physical, tangible, and non-transitory computer-readable storage medium. It should be understood that the memory 204 can include a variety of different memories, each implementing one or more of the functions or processes described herein.

[0160] In example embodiments, the computer system 200 includes an output unit 206 coupled to (and in communication with) the processing unit 202. The output unit 206 outputs or presents information to a user of the computer system 200 by, for example, displaying and / or otherwise outputting information such as, but not limited to, outputs, process parameters, and / or any other type of data. It should also be understood that, in some embodiments, the output unit 206 can include a display device, such that various interfaces (e.g., systems (web-based or otherwise), etc.) can be displayed at the computer system 200 and at the display device to display such information and data, etc. In some examples, the computer system 200 can cause interfaces to be displayed at a display device of another computing device, including, for example, a server hosting a website having a plurality of web pages, or a server interacting with a web system employed at another computing device, etc. The output unit 206 can include, but is not limited to, a liquid crystal display (LCD), a light emitting diode (LED) display, an organic LED (OLED) display, an “e-ink” display, combinations thereof, etc. In some embodiments, the output unit 206 can include a plurality of units.

[0161] The computer system 200 also includes an input device 208 that receives input from a user. The input device 208 is coupled to the processing unit 202 (and in communication with the processing unit 202) and can include, for example, a keyboard, a pointing device, a mouse, a stylus, a touch-sensitive panel (e.g., a touchpad or a touch screen, etc.), another computing device, and / or an audio input device. Further, in some example embodiments, a touch screen, such as a touch screen included in a tablet or similar device, can serve as both an output unit 206 and an input device 208. In at least one example embodiment, the output unit 206 and the input device 208 can be omitted. Additionally, the computer system 200 shown includes a network interface 210 coupled to the processing unit 202 (and in some embodiments, to the memory 204) (and in communication with the processing unit 202). The network interface 210 can include, but is not limited to, a wired network adapter, a wireless network adapter, a telecommunications adapter, or other device designed to communicate over one or more different networks. The network interface 210 and / or the input device 208 are referred to herein as receiving units.

[0162] In this manner, the computer system 200 can include the processing unit 202 and the computer-readable storage medium or memory 204 coupled to the processing circuitry, where the processing circuitry is configured to execute computer-readable program code instructions stored in the memory 204. It will also be appreciated that one or more functions, as well as combinations of functions, can be implemented by special-purpose computer systems and / or processing circuitry executing program code instructions, or combinations of special-purpose hardware and program code instructions, that specify the functions.

[0163] The execution of instructions by the processing unit or the storage of instructions in the computer-readable storage medium support combinations of operations for performing the specified functions. For example, the instructions input to the processing unit for storage in the computer-readable storage medium can include input parameters for a data acquisition module to determine a replacement potency target. Further, a non-transitory computer-readable storage medium having software instructions stored thereon, which, when executed by a processor of a computer system, cause the computer system to perform a method of: allowing a user to input user input parameters into a data acquisition module to determine a replacement potency target, and an output for displaying a processed result; pre-processing data from the data acquisition module, including normalization; and applying a statistical modeling calculation using an out-of-specification (OOS) risk selection criterion to the pre-processed data; and outputting the result; wherein the method determines the replacement potency target, and the output displays a corresponding OOS risk.

[0164] While embodiments of the application have been described with reference to particular examples, various modifications and changes can be implemented and equivalents can be substituted for elements without departing from the true spirit and scope of the appended claims. The specification and examples are, therefore, to be regarded in an illustrative rather than a restrictive sense.

Claims

1. A computer-implemented method comprising: (a) providing a front end dashboard comprising user inputs for inputting user input parameters into a data acquisition module to determine a replacement potency target, and outputs for displaying processed results; (b) providing a back end for: pre-processing data from the data acquisition module, including normalization; applying statistical modeling calculations using out-of- specification (OOS) risk selection criteria to the pre-processed data; and outputting the results; wherein the method determines the replacement potency target, and the outputs display respective OOS risks.

2. The computer-implemented method of claim 1, wherein the user input parameters comprise a drug product, a fill size, a batch selection, and a production date range.

3. The computer-implemented method of claim 1, wherein, The data acquisition module further comprises a source data system.

4. The computer-implemented method of claim 3, wherein, The data acquisition module further comprises a data acquisition system.

5. The computer-implemented method of claim 4, wherein, The data acquisition module further comprises a cloud-based data storage.

6. The computer-implemented method of claim 5, wherein, The data acquisition module further comprises cloud-based computing.

7. The computer-implemented method of claim 1, wherein, The pre-processing step further comprises normalizing the data by statistical analysis.

8. The computer-implemented method of claim 7, wherein, The data is normalized by calculating Pnormalized according to the following equation: ， where: P = potency value, P CT = current batch potency target, and P TM = Batch potency target at production.

9. The computer-implemented method of claim 1, wherein, OOS risk selection criteria for candidate replacement potency targets can be determined from the model, wherein: 1) : if the calculated CV is within the user-defined limits, at least one individual replicate result does not meet the OOS risk for the upper release limit; 2) : false if the calculated CV is within the user-defined limits and at least one individual replicate result does not meet the lower release limit OOS risk; 3) : if the calculated CV is within the user-defined limits, then the average result is not at risk of being OOS below the lower average release limit; and 4) OOS risk in the above at least one case.

10. The computer-implemented method of claim 1, wherein, The data acquisition module further comprises a cloud-based relational database.

11. The computer-implemented method of claim 10, wherein, Data queries can be made between the user inputs and the cloud-based relational database.

12. A computer-implemented system comprising: (a) a front end dashboard comprising user inputs for inputting user input parameters into a data acquisition module to determine a replacement potency target, and outputs for displaying processed results; (b) a back end for: pre-processing data from the data acquisition module, including normalization; applying statistical modeling calculations using out-of- specification (OOS) risk selection criteria to the pre-processed data; and outputting the results; wherein the system determines the replacement potency target, and the outputs display respective OOS risks.

13. The computer-implemented system of claim 12, wherein, The user input parameters comprise a drug product, a fill size, a batch selection, and a production date range.

14. The computer-implemented system of claim 12, wherein, The data acquisition module further comprises a source data system.

15. The computer-implemented system of claim 14, wherein, The data acquisition module further comprises a data acquisition system.

16. The computer-implemented system of claim 15, wherein, The data acquisition module further comprises a cloud-based data storage.

17. The computer-implemented system of claim 16, wherein, The data acquisition module further comprises cloud-based computing.

18. A non-transitory computer-readable storage medium having software instructions stored thereon that, when executed by a processor of a computer system, cause the computer system to perform the following method: (a) allowing a user to input user input parameters into a data acquisition module to determine a replacement potency target, and outputs for displaying processed results; (b) pre-processing data from the data acquisition module, including normalization; and (c) applying statistical modeling calculations using out-of-specification (OOS) risk selection criteria to the pre-processed data; and outputting the results; wherein the method determines the replacement potency target, and the outputs display respective OOS risks.

Images