Component matching decision support tool

Abstract

A method for reducing variability of a combined device includes identifying potential combinations of at least a first and a second component, each combination capable of forming one or more units of the combined device. The method also includes, for each potential combination, predicting a quality or outcome of the unit of the combined device when formed from at least the first and second components of the combination by applying values ​​of one or more properties of at least the first component of the combination and values ​​of one or more properties of the second component of the combination as inputs to a predictive model. The method also includes selecting a subset of combinations from among the potential combinations based on the predicted quality or outcome of the potential combinations and providing an indication of the selected subset of combinations.

Description

[Technical Field]

[0001] This application relates generally to the assembly or manufacture of devices, and more particularly to techniques for selecting particular components or component batches / lots to be used when assembling or manufacturing devices. [Background technology]

[0002] In many manufacturing / production situations, different types of components are combined to form a combination device. In the pharmaceutical industry, for example, a fluid pharmaceutical is often combined with (i.e., used for filling) a syringe to produce a fluid-filled syringe. As another example, a fluid-filled syringe can be combined with (i.e., filled) an autoinjector subassembly to produce a complete autoinjector. As yet another example, a lyophilized pharmaceutical can be combined with (i.e., used for filling) a vial or other container. In each of these and other situations, multiple batches of each component may be available (e.g., in inventory) for combination with one another. Due to component variability (e.g., batch-level variability), the manner in which any two components are paired (or, more generally, two or more components are matched) for a given unit of a combination device can affect the way individual component tolerances "stack" in the final product, leading to a wider distribution of possible values ​​for the characteristics of the combined devices (e.g., the values ​​of the device's functional output, such as the injection time of an autoinjector). Even when the tolerances of the individual components are tightly controlled, the properties of the final product may exhibit a relatively wide range of values, potentially leading to increased rejection rates, more user (e.g., customer or patient) complaints, and / or other problems. Furthermore, making the tolerances of the individual components tight enough to meet the tolerance targets in the final product may be prohibitively expensive. Summary of the Invention [Means for solving the problem]

[0003] To address some of the above-mentioned shortcomings of current / conventional practices, embodiments described herein include systems and methods that use data associated with different components (e.g., batch-specific or unit-specific data for the different components) to better pair or match those components (e.g., specific components or specific batches of components) when manufacturing or assembling a combined device. In this way, variability (e.g., around nominal specifications) of the combined device can be reduced, thereby providing more consistent product performance (e.g., fewer defects and / or fewer user complaints) compared to traditional process control measures. The term “component” is used broadly herein to refer to any physical part of a combined device (e.g., a structural subcomponent, a raw material, or a fluid, lyophilized drug, or other substance used to fill another component acting as a container, etc.), unless the context of use clearly dictates a more specific meaning. Furthermore, the term “combined device” is used broadly herein to refer to any device manufactured or assembled using two or more components and may be a “final” product (e.g., for sale or distribution) or an intermediate stage of a product, unless the context of use clearly dictates a more specific meaning.

[0004] In the techniques disclosed herein, a predictive machine learning model predicts at least one characteristic or outcome of a unit of a combined device for each of several different combinations of components that may be used to form the combined device (e.g., batch 1 of component A with batch 1 of component B and batch 1 of component A with batch 2 of component B, etc.). The predicted characteristic may be, for example, a measure of device quality or performance (e.g., the standard deviation of a characteristic of the combined device across multiple units manufactured). As another example, the predicted outcome may be an indication of whether or how frequently a user complaint (either general or a specific type of user complaint) will occur. To make each prediction, the model operates on (i.e., accepts as input) values ​​of one or more characteristics of each component considered for the combination. These model inputs may include actual measurements, identifiers, other predicted (or estimated) values, and / or other types of upstream manufacturing data associated with the components (e.g., associated with a particular component batch). After the model predicts the attribute or outcome (or attributes and / or outcomes) of each of the various component combinations, an optimizer (e.g., a linear optimizer) operates on the predictions to determine which components (or component batches, etc.) should be matched to "best" (e.g., optimally) meet a desired measure of device quality or performance (e.g., a desired value of the attribute / attributes and / or outcome / outcomes predicted by the model).

[0005] In particular, the use of the techniques disclosed herein for pairing prefilled syringe batches with autoinjector subassembly batches has been shown to reduce lot-to-lot variability in average injection times by 30% to 45% compared to traditional processes that randomly pair batches. Generally, as the number of available component batches / lots increases and / or the number of components required to manufacture / assemble a particular combined device increases, the benefits of matching those component batches / lots using the techniques disclosed herein can correspondingly increase. Furthermore, software tools implementing the techniques described herein can facilitate a human user's understanding of why a particular combination of components or component batches is advantageous. For example, using such tools, a user can develop a set of heuristics or "rules of thumb" that enable intuitive pairing of components to offset different sources of variability. In general, the techniques described herein can help reduce lot-to-lot variability in manufactured combined devices and / or improve user (e.g., customer and / or patient) experience.

[0006] Those skilled in the art will appreciate that the figures described herein are included for purposes of illustration and not limitation of the present disclosure. The figures are not necessarily to scale, emphasis instead being placed upon illustrating the principles of the present disclosure. It should be understood that in some instances, various aspects of the described implementations may be shown exaggerated or enlarged to facilitate understanding of the described implementations. In the drawings, like reference numerals generally refer to functionally similar and / or structurally similar components throughout the various views. [Brief explanation of the drawings]

[0007] [Figure 1] FIG. 1 is a simplified block diagram of an example system that implements a component matching tool to match components to reduce variability across manufactured combined device units. [Figure 2A]FIG. 1 illustrates components of an exemplary combined device to which the component matching tool may match a particular component or a particular batch of components. [Figure 2B] FIG. 1 illustrates components of an exemplary combined device to which the component matching tool may match a particular component or a particular batch of components. [Figure 2C] An example of equipment / product lineage is shown below. [Figure 3A] 2 illustrates an example process that may be implemented at least in part by the computing system of FIG. 1. [Figure 3B] 1 illustrates a pairing determination stage in an example manufacturing process. [Figure 4] 2 illustrates an example user interface that may be generated and / or populated by the entity matching tool of FIG. 1. [Figure 5] 10A-10C show plots illustrating example predictions of average injection times and average actuation forces for potential syringe stoppers, syringe barrels, and drug combinations. [Figure 6] 1 shows a plot comparing the predicted mean injection time of an autoinjector when the autoinjector components are matched randomly, when the autoinjector components are matched using heuristic rules, and when the autoinjector components are matched using a linear optimizer. [Figure 7] 10 shows an exemplary SHAP and feature importance plot for an embodiment in which the combination device is an autoinjector. [Figure 8] 1 is a flow diagram of an example method for reducing variability in a combined device. DETAILED DESCRIPTION OF THE INVENTION

[0008] The various concepts described introductory above and discussed in more detail below can be implemented in any of numerous ways, and the described concepts are not limited to any particular implementation style. Example implementations are provided for illustrative purposes.

[0009] FIG. 1 is a simplified block diagram of an example system 100 in which the techniques disclosed herein may be implemented. The example system 100 includes M units of component A in batch 102A and N units of component B in batch 102B (M and N are the same or different integers, and one or both numbers are greater than 1). Component A and component B may be any physical components that can be combined / assembled with each other to form a combined device. The combined device may be any device manufactured or assembled using two or more such components, and may be a “final” device / product (e.g., for sale or distribution) or an intermediate stage of the device / product (e.g., a subassembly of equipment). FIGS. 2A and 2B provide specific, non-limiting examples of component A and / or component B. For example, component A can be syringe 200 of Figure 2A, including barrel 202, plunger / stopper 204, flange 206, needle 208, and possibly needle shield 210, and component B can be a fluid pharmaceutical that fills syringe 200, such that the resulting combined device is a fluid-filled syringe. As another example, component A can be the combination of barrel 202 (integral with flange 206) and (pre-staked) needle 208 of Figure 2A, and component B can be plunger / stopper 204 of Figure 2A.

[0010] 2B shows yet another example corresponding to subsequent manufacturing / assembly stages where component A is fluid-filled syringe 222, component B includes both front and rear autoinjector subassemblies 224 and 226, and the combined device is a complete autoinjector. Alternatively, system 100 may treat subassemblies 224 and 226 as separate components and match a batch of fluid-filled syringes 222 (component A), a batch of front autoinjector subassemblies 224 (component B), and a batch of rear autoinjector subassemblies 226 (component C, not shown in FIG. 1). It will be appreciated that the techniques described herein can be applied to any number of components and can be applied multiple times in successive manufacturing stages (e.g., to reduce variability in the fluid-filled syringes and also in subsequent manufacturing stages to reduce variability in the final autoinjector).

[0011] In some embodiments, system 100 matches one set of multiple batches with one or more batches in another set. For example, batch 102A may correspond to a set of 10 batches of component A, while batch 102B may correspond to one batch (or 10 batches, 20 batches, etc.) of component B. While the description herein primarily relates to matching different lots or batches of components (e.g., each “batch” consists of units manufactured in the same process run and / or the same time window, etc., and on the same equipment), in some embodiments, system 100 instead matches components on a unit-by-unit basis (e.g., where all units of each component are tested to provide measurement data that can be analyzed, rather than simply sampling a subset of batch units for measurement). Accordingly, it is understood that embodiments described herein with respect to matching “batches” may instead match single units, collections of units of any size (e.g., collections that do not correspond to “batches” defined by a manufacturing process), or sets of multiple batches.

[0012] Furthermore, while most embodiments described herein involve matching components of different types, in some embodiments, the combined device is formed from at least two units of the exact same component, and batches 102A and 102B are subsets of the entire batch of components (i.e., components A and B are the same component). Furthermore, while most embodiments described herein involve pharmaceutical devices (e.g., autoinjectors), in other embodiments, the combined device is a non-pharmaceutical device, such as a home appliance or appliance subassembly, an automobile or automobile subassembly, a medical device or medical device subassembly, etc. For example, in an automobile embodiment, components A and B may be the cylinder head and piston of the engine, respectively.

[0013] In general, the matching techniques described herein can be used to match components at any level of component genealogy within a manufacturing process (e.g., one or more hierarchical levels each including one or more subassemblies or subcomponents and a final assembly / product). Certain circumstances and / or qualities may make a given manufacturing process more suitable for improvement through the techniques disclosed herein, including: (1) data (e.g., manufacturing data) from one or more components of the process is available before the corresponding manufacturing state begins; (2) batch (or other grouping) genealogy data linking which components will be used in a given final product or final product batch is available; (3) the available process component data can predict characteristics of interest in the final product; and (4) most predicted characteristics include characteristics of two or more components that will be combined (rather than just one component without the opportunity to make any pairing decisions).

[0014] 2C shows an example of one such device / product genealogy 250, where different batches of different components are combined to create subassemblies and ultimately different lots of final products. When the term "component" is used herein, it is understood that all boxes under the "final product" layer (e.g., "Component C," "Subassembly AB," etc.) can be viewed as combinable components. Thus, genealogy 250 represents a manufacturing process in which the first level includes three separate components and the second level includes four separate components (two batches designated "Component C" and two batches designated "Component D").

[0015] In some embodiments, different batches of one component (e.g., component A or component B) are of the same general component type but are not identical. Referring again to FIG. 1 , for example, all batches 102A of component A may be the same syringe batch (e.g., the same part number or part number assembly), while different batches 102B of component B may be different fluid pharmaceuticals. That is, the techniques described herein may be used to combine different batches of a single syringe type with different pharmaceuticals. This can be advantageous because it is generally known that the impact of component variability is not consistent across different pharmaceuticals due to different pharmaceutical characteristics (e.g., viscosity, protein concentration, etc.). Thus, optimal matching of drug types with different container batches and / or optimal matching of prefilled syringes containing different drug types with different autoinjector subassembly batches can reduce variability in the final autoinjector.

[0016] System 100 includes a characterization system 104 generally configured to measure and / or collect data representative of one or more characteristics of each batch 102A and each batch 102B. Characterization system 104 may include separate devices for measuring the characteristics of the two batches 102A, 102B (e.g., when the types of component A and component B are sufficiently different to require different measurement devices or when components A and B are manufactured in different locations). If component A is a syringe and component B is a fluid pharmaceutical, for example, characterization system 104 may include a first sensor (e.g., a camera) and associated device (e.g., an optical comparator and a computing device that stores measurements) that automatically and / or manually measures the inner diameter of the barrel of a unit of component A, and a second sensor (e.g., a viscometer) and associated device (e.g., a computing device that stores measurements) that automatically and / or manually measures the viscosity of the pharmaceutical. The characterization system 104 may include and / or collect data from multiple data sources (eg, different organizations, systems, etc.), and the data may correspond to one or more time periods.

[0017] In some embodiments, the characterization system 104 samples only a subset of the units in each batch and identifies representative characteristic values ​​for that batch (e.g., average bore diameter, average viscosity, etc.) Generally, the characteristic values ​​may be summary statistics from one or more tests and / or processes associated with each of the different components, such as mean, median, minimum, maximum, standard deviation, and specific quartile values.

[0018] In other embodiments (e.g., when components are matched individually rather than in batches), characterization system 104 measures every unit of component A and every unit of component B. In some embodiments, characterization system 104 includes devices and associated equipment (e.g., sensor controllers, computing devices, etc.) that indirectly measure (i.e., "soft sense") the values ​​of particular properties. For example, characterization system 104 may include a Raman spectrometer that analyzes Raman scans of a fluid pharmaceutical to identify values ​​indicative of chemical composition and molecular structure.

[0019] System 100 also includes a computing system 110 coupled to characterization system 104. Computing system 110 may include a single computing device or multiple computing devices (e.g., one or more servers and one or more client devices) that are co-located or remote from one another. In the example embodiment shown in FIG. 1 , computing system 110 includes one or more processors 120, a network interface 122, a display 124, a user input device 126, and a memory 128. In some embodiments, computing system 110 includes a portion of characterization system 104.

[0020] Each of the processors 120 may be a programmable microprocessor that executes software instructions stored in memory 128 to perform some or all of the functions of the computing system 110, as described herein. Alternatively, one or more of the processors 120 may be other types of processors (e.g., application specific integrated circuits (ASICs), field programmable gate arrays (FPGAs), etc.).

[0021] Network interface 122 may include any suitable hardware (e.g., front-end transmitter and receiver hardware), firmware, and / or software configured to communicate with external devices and / or systems (e.g., characterization system 104, another computing system that receives the optimal combination indication from computing system 110, a server that provides an interface between computing system 110 and characterization system 104, etc.) using one or more communication protocols. For example, network interface 122 may be or include an Ethernet interface. Although not shown in FIG. 1 , computing system 110 may communicate with characterization system 104 and / or any device that provides an interface between computing system 110 and characterization system 104 over a single communication network or over multiple communication networks of one or more types (e.g., one or more wired and / or wireless local area networks (LANs) and / or one or more wired and / or wireless wide area networks (WANs), such as the Internet or an intranet, etc.).

[0022] Display 124 may use any suitable display technology (e.g., LED, OLED, LCD, etc.) to present information to a user, and user input device 126 may be a keyboard or other suitable input device. In some embodiments, display 124 and user input device 126 are integrated into a single device (e.g., a touchscreen display). In general, display 124 and user input device 126 may be combined to enable a user to view and / or interact with a visual presentation (e.g., a graphical user interface or displayed information) output by computing system 110, for purposes such as, for example, informing a user of recommended component batch combinations.

[0023] Memory 128 may include one or more physical memory devices or units, including volatile and / or nonvolatile memory, and may include memory located on various computing devices in computing system 110. Any suitable type or types of memory may be used, such as read-only memory (ROM), solid-state drive (SSD), hard disk drive (HDD), etc. Memory 128 stores instructions for one or more software applications, including component matching tool 130. Component matching tool 130, when executed by some or all of processors 120, is generally configured to (1) collect component batch data generated by characterization system 104, (2) determine which combination of batches 102A and 102B should provide better characteristics and / or results for the combined device unit (e.g., a narrower distribution of a particular characteristic of the combined device and / or fewer user complaints), and (3) provide an indication of those combinations to a user and / or another computing system or application. To this end, component matching tool 130 includes training module 140, combination identification module 142, prediction module 144, and optimization module 146. Modules 140-146 may be separate software components or modules of component matching tool 130, or may simply represent functionality of component matching tool 130 that is not necessarily divided among different components / modules. For example, in some embodiments, prediction module 144 and training module 140 are included in a single software module. Furthermore, in some embodiments, different modules 140-146 may be distributed among multiple copies of component matching tool 130 (e.g., running on different devices in computing system 110) or among different types of applications stored and executed on one or more devices in computing system 110. The operation of each of modules 140-146 is described in further detail below with reference to the operation of system 100.

[0024] As also described in further detail below, computing system 110 is configured to access history database 150 for training purposes. History database 150 may store parameter values ​​associated with past characterization data collected by characterization system 104 and / or collected by other similar devices or systems. For example, history database 150 may store individual (or average, arithmetic mean, etc.) viscosity and / or protein concentration of a fluid pharmaceutical, syringe barrel diameter and / or glide force, etc., and possibly other related parameter values ​​(e.g., time). In some embodiments, history database 150 additionally or alternatively stores categorical values ​​for particular components or component batches, such as data indicating whether a pharmaceutical fluid batch was observed to contain particles above a threshold size (e.g., above 20 μm). History database 150 may also store “label” information representing the actual attributes and / or results of a unit of a combined device formed from a particular combination of batches of different components. In some embodiments, each label is an actual measured value associated with a resulting lot of a combined device. For example, each label may be the average injection time or average actuation force for a batch of autoinjectors formed using a particular combination of component batches, or a value or set of values ​​defining a probability distribution of injection times or actuation forces for a batch of autoinjectors. Alternatively or additionally, each label may be an indication of whether a particular type of user complaint (e.g., patient complaint, customer complaint, etc.) or user complaints in general exceeded some threshold number or frequency for the resulting combined device, or the exact number, frequency, and / or likelihood of such complaints, etc. Database 150 may be stored in permanent memory in memory 128, in a different permanent memory in computing system 110, or in another device or system. In some embodiments, computing system 110 accesses database 150 over the Internet using network interface 122.

[0025] Training module 140 of component matching tool 130 uses characteristic (e.g., measurement) data and associated labels stored in historical database 150 to train predictive model 132, which is then used by prediction module 144 to make predictions of combined device units resulting from particular combinations of component batches. Predictive model 132 may be a decision tree ensemble model (e.g., a gradient boosted tree model), a neural network, a support vector machine (SVM) model, a decision tree model, or any other suitable type of model. As a more specific example, predictive model 132 may be an XG-Boost model, which has been shown to deliver superior performance for the disclosed techniques compared to certain other predictive model types (e.g., linear regression, LASSO, and random forest models). The predictive model 132 may include, for example, a classifier that predicts a particular classification of a combined device resulting from a particular combination of component batches (e.g., "within specification tolerance" or "not within specification tolerance" or "good user experience," "moderate user experience," and "poor user experience," etc.), a model that predicts a particular numerical value or values ​​(e.g., mean infusion time, standard deviation of infusion time, interquartile range of infusion time, etc.), or a model that predicts a complete probability distribution (e.g., where the predictive model 132 includes a modular neural network (MNN)). The predictive model 132 may be created using, for example, open source Python libraries.

[0026] In some embodiments, the predictive model 132 is retrained / improved, or a new predictive model is trained for a particular use case. For example, the predictive model 132 or a new model may be trained using historical data specific to the manufacturing location to be used, the revision / version or type of pharmaceutical being manufactured, a particular time period, etc. Additionally, in embodiments in which one or more labels represent user experience, the labels may be limited to, among other things, relevant geographies (e.g., global, US, or non-US), age groups, end-use environments (e.g., sample devices, devices for sale, or medical use devices), etc.

[0027] Generating a predictive model 132 (e.g., partially or entirely by training module 140, perhaps with some user input) may include, for example: (1) preprocessing data (preparing historical data for use by predictive model 132 by normalizing and scaling features, converting data types, and inputting missing data); (2) separating the preprocessed data into training and test / validation sets (to avoid overfitting the data and to estimate the performance of predictive model 132 on unfamiliar data); and (3) selecting a model type (determining which supervised machine learning algorithm to use for a given prediction). ), (4) selecting and cross-validating hyperparameters (determining discretionary parameters to tune a given model for a particular dataset), (5) performing model fitting (using the training data and tuning parameters to create a predictive model capable of selecting an optimal loss function to minimize), (6) evaluating the predictive model 132 (using the trained predictive model 132 to evaluate model performance by making predictions on a test set), and (7) determining feature importance (providing a visual representation of how the trained predictive model 132 arrives at a predicted output value based on input data). Tuning parameters for the XG Boost model may include, for example, maximum depth, number of estimators, minimum child weights, learning rate, column sample by tree, subsample, alpha, gamma, and lambda. Model performance may be measured, for example, by the coefficient of determination (R 2 ) metric.

[0028] As discussed in more detail below, combination identification module 142 is generally configured to identify specific combinations of component batches (e.g., different combinations each pairing one of batches 102A with one of batches 102B), and prediction module 144 will then predict one or more attributes and / or results of the combined device resulting from that combination. As also discussed in more detail below, optimization module 146 is generally configured to process the combination-specific predictions output by prediction module 144 to generate a set of combinations to be used in manufacturing / assembly or a set of recommended combinations for manufacturing / assembly.

[0029] As described above, computing system 110 may include one device or multiple devices, and if it includes multiple devices, they may be co-located or remotely distributed (e.g., with Ethernet and / or Internet communication between the different devices). In one embodiment, for example, a first server of computing system 110 (including module 140) trains predictive model 132, a second server of computing system 110 collects measurements and / or other data from characterization system 104, and a third server of computing system 110 (including modules 142, 144, 146) receives measurements and / or other data from the second server and uses a copy of the trained predictive model 132 to generate predictions based on the received measurements and / or other data. As another example, the third server in the above example does not store a copy of the trained predictive model 132 but instead utilizes predictive model 132 by providing measurements to the second server (e.g., if predictive model 132 is available via a web service configuration). As used herein, unless the context of the term clearly dictates otherwise, terms such as "running," "using," or "implementing" a model (e.g., predictive model 132) are used broadly to encompass the alternative of directly running a locally stored model or requesting another device (e.g., a remote server) to run the model. It is understood that still other configurations and distributions of functionality beyond those shown in FIG. 1 and / or described herein are possible and within the scope of the invention disclosed herein.

[0030] The operation of system 100 will now be described in more detail with reference to both the components of Figure 1 and process 300 shown in Figure 3A. During an initial training phase 302, component matcher 130 retrieves historical data 304 (e.g., past component measurements and associated labels) from history database 150, and training module 140 uses the retrieved historical data 304 to train predictive model 132. For example, if component A is a syringe and component B is a pharmaceutical product, historical data 304 may include: (1) for each batch of syringes represented in history database 150, data indicative of the glide force, shield removal force, barrel inner diameter, break test results, maximum insert trim outer diameter, maximum insert outer diameter, and / or insert silicone weight, e.g., average values ​​for the batch; and (2) for each batch and / or type of drug represented in history database 150, drug identifier and / or other information, e.g., drug viscosity, protein concentration, manufacturing location, and / or density, e.g., using average values ​​where appropriate. As another example, if component A is a prefilled syringe and component B includes a rear autoinjector subassembly, historical data 304 may include: (1) for each batch of prefilled syringes represented in historical database 150, data indicative of the extrusion force, breakloose force, protein concentration, characteristics (e.g., whether particles above a certain size were observed in the pharmaceutical product), and / or any of the above syringe-specific characteristics (e.g., glide force, barrel inner diameter, etc.), e.g., average values ​​for the batch; and (2) for each batch of rear autoinjector subassemblies represented in historical database 150, specification number, saline release test results (injection time and / or actuation force), and / or spring data (e.g., spring force), e.g., using average values ​​where appropriate.

[0031] The training module 140 trains the predictive model 132 to predict a particular attribute of the combined device, such as injection time or actuation force (e.g., if the combined device is a prefilled syringe or a fully automatic injection device). The value may be, for example, an expected average value of the property for all combined devices formed from a particular batch associated with a particular combination (e.g., Batch 1 of component A combined with Batch 2 of component B), or an expected metric indicating the variability of the property (e.g., an expected standard deviation). In other embodiments, the training module 140 trains the predictive model 132 to predict a probability distribution of combined devices when formed from a particular combination of components.

[0032] In some embodiments, predictive model 132 includes one or more preliminary model stages. For example, a first model stage of predictive model 132 may convert inputs from historical data 150 into values ​​reflecting a reduced set of dimensions. As a more specific example, predictive model 132 may apply a principal component analysis (PCA) technique to a set or subset of inputs to reduce the set or subset to n dimensions, and then apply the n values ​​(one per dimension) as inputs to a subsequent model stage (e.g., a decision tree ensemble model or a neural network) that outputs a predicted value. In such an embodiment, training module 140 may perform the dimensionality reduction in stage 302 and then apply the inputs to the remainder of predictive model 132 for training purposes.

[0033] Stage 302 may also include validating and / or qualifying the trained predictive model 132 (e.g., using a portion of the historical data 304 that was not used for training). Once satisfactorily trained and possibly validated / qualified, the prediction module 144 of the component matching tool 130 may use the predictive model 132 in stage 310 to predict values ​​of one or more combined device properties for each combination of components or component batches under consideration. However, prior to stage 310, the combination identification module 142 of the component matching tool 130 determines which combinations of component batches (and thus which subsets of the new data 306) should be analyzed. The new data 306 includes data indicative of properties of component batches to be assembled to form new units of combined devices, and may include the same types of measurements and / or other component data (e.g., syringe glide force, pharmaceutical identifiers, etc.) as described above with respect to the historical data 304.

[0034] Combination identification module 142 may determine combinations to consider by, for example, receiving a list of batch identifiers of different components (e.g., component A and component B) manually entered by a user (e.g., via user input device 126) or from another computing system, storage device, or application, and then automatically generating each possible permutation based on the batch identifier list. In other embodiments, combination identification module 142 in step 308 also filters out (omits) combinations that are not feasible or practical (e.g., it is not possible to combine two batches due to the timing / sequencing of the manufacturing process) so as not to waste processing resources in step 310. In other embodiments, combination identification module 142 identifies / determines appropriate component / batch combinations to consider simply by receiving an indication of appropriate component / batch combinations to consider (e.g., manually entered by a user or from another computing system, storage device, or application). In some embodiments and / or situations, the output of (and / or input to) combination identification module 142 is limited based on time, e.g., so that a component batch can only be reserved in a "holding" area for a limited time to avoid major disruptions to the manufacturing process.

[0035] In step 310, the prediction module 144 applies a portion / subset of the new data 306 corresponding to the components / batches of combinations identified in block 308 as input to the predictive model 132. The prediction module 144 may operate on the data 306 sequentially for each identified combination, i.e., by predicting the attribute or outcome of interest for one combination and then processing the next combination. Alternatively (e.g., if the processor 120 is capable of implementing multiple instances of the prediction module 144), the component matcher 130 may perform predictions for multiple combinations in parallel. As mentioned above, the predictive model 132 may, in some embodiments, include a dimensionality reduction step, in which case step 310 includes reducing the data dimensionality before inputting the dimensionality-reduced data into the predictive model 132.

[0036] The output in step 310 includes predictions of one or more attributes and / or outcomes of each combination identified in step 308 (i.e., attributes and / or outcomes of the same type as those represented by the labels used in training step 302). In step 312, optimization module 146 applies the predicted attributes and / or outcomes of all identified combinations as inputs to an optimizer to determine which set of combinations provides the "best" (e.g., optimal) performance. The optimizer may be a linear optimizer. For example, optimization module 146 may use PuLP (a Python linear programming application programming interface (API)) to define the objective function and invoke an external solver.

[0037] In some embodiments where the prediction module 144 predicts only one characteristic (e.g., the standard deviation of infusion times across all combined device units resulting from a particular component combination), the optimization module 146 solves the following objective function:

number

[0038] [Table 1]

[0039] To minimize the objective function in this example, the optimization module 146 determines which of the columns X1-X6 leads to the smallest sum and selects that as the desired set of combinations.

[0040] In some embodiments, optimization module 146 minimizes an objective function that includes multiple terms, each similar to those shown in the function above, e.g., with one term for each attribute or outcome predicted by prediction module 144. For example, the objective function may include a first term corresponding to the average infusion time and a second term corresponding to the average number of complaints (e.g., 0.1 if 1 complaint is predicted per 10 units). Some or all of the individual terms in the objective function may be weighted according to the perceived importance of each term to the overall performance of the combined device.

[0041] The optimization module 146 provides an output indicating the set / permutation of combinations that minimizes the objective function, and in step 314, a lot of combined equipment is manufactured (e.g., using commercial-scale production equipment) using the indicated combinations. In some embodiments, the combination matching tool 130 causes a display (e.g., display 124) to present a user interface that shows the resulting combinations to a user, who then (perhaps after manual review / verification) takes one or more actions to ensure that production uses those combinations in step 314. In other embodiments, the combination matching tool 130 sends data indicative of the resulting combinations to another computing system or application, and the computing system or application causes the manufacturing process to automatically use those combinations (e.g., by controlling appropriate conveying / routing equipment).

[0042] In some embodiments, measurements of the combined device manufactured in step 314 are obtained manually or automatically and used as labels to compare against the predictions made in step 310. For each of one or more batch combinations, the corresponding input data (from new data 306) applied to predictive model 132, for example, along with the corresponding label, can be fed back to training module 140, which can use the data and label to improve / update predictive model 132 through further training.

[0043] FIG. 3B illustrates the pairing decision stage in an example manufacturing process 320. In the first stage, different drugs (and possibly different batches of each drug) are paired with other components (and possibly different batches of those other components). For example, different fluid pharmaceuticals may be paired with different types of assembled syringes and / or different batches of those syringes. The second stage shown in FIG. 3B may represent, for example, the pairing of a particular filled syringe with a particular autoinjector subassembly. The component matching tool 130 may match components and / or batches in an attempt to optimize one or more characteristic values, such as minimizing the probability of glass breakage, minimizing the standard deviation from the desired injection time, etc. (as shown by the dashed lines in FIG. 3B). Typically, as the stage number increases (i.e., closer to the final product), data availability and predictive power increase, but the opportunity for improvement through matching decisions decreases.

[0044] FIG. 4 illustrates an example user interface 400 that may be generated and / or populated by the component matching tool 130 of FIG. 1 or a similar application in some embodiments. FIG. 4 depicts an embodiment in which component matching is performed in two sequential stages: a pre-fill stage in which batches / lots of empty syringes are matched with different pharmaceutical preparations (“Drug 1,” “Drug 2,” and “Drug 3”), and a post-fill stage in which those filled (“prefilled”) syringes are matched with batches / lots of autoinjector subassemblies. The quantities shown in FIG. 4 are purely for illustrative purposes and do not necessarily represent realistic values. It is further understood that any suitable units may be associated with the values ​​(e.g., Newtons for average actuation force and syringe glide force, seconds or milliseconds for average injection time, etc.).

[0045] In the row of user interface 400 labeled "Desired Specifications and Quantity," the user can select (e.g., via user input device 126) the particular medication to be considered (in the illustrated example, "Drug 1" and "Drug 2"). In the row labeled "Pre-Fill Data," the user can enter (e.g., via user input device 126) and / or view (e.g., via display 124) the values ​​of properties associated with a particular lot / batch of the selected medication and a particular lot / batch of syringes that can be used to hold those medications. In the illustrated example, the values ​​of the properties of the plunger lot / batch are also displayed, and component matching tool 130 matches batches of medications, syringes, and plungers. For example, with reference to FIG. 1 , the medication is component A, the syringes are component B, while the plungers may be component C, not shown in FIG. 1 . In the row labeled "Post-Fill Data," the user can enter (e.g., via user input device 126) and / or view (e.g., via display 124) the values ​​of characteristics associated with a particular lot / batch of prefilled syringes (for each drug) and the values ​​of characteristics associated with a particular lot / batch of autoinjector subassemblies.

[0046] In the example user interface 400, the results for each selected drug are shown in the first two rows. In particular, the first row shows the optimized batch pairing of drugs with syringes and plungers, as well as plots of predicted injection times and actuation forces (for each selected drug), and the second row shows predicted average injection times and average actuation forces for each selected drug.

[0047] FIG. 5 shows plot 500 illustrating example predictions of average injection times and average actuation forces for potential syringe stopper, syringe barrel, and drug combinations. In some embodiments, combination matching tool 130 generates and / or populates visualizations that include plots similar to plot 500, for example, to help a user understand how different drugs affect component batch variability. In FIG. 5, each circle represents a different combination of syringe stopper / barrel batch and drug type, with enlarged circles representing combinations using one particular batch / lot of syringe barrels. As can be seen from FIG. 5, filling syringe barrels from this particular batch with the drug represented by Drug 3 (at location 502) leads to the shortest injection time.

[0048] 6 shows a plot 600 comparing the predicted standard deviation of the mean injection time of an autoinjector when the autoinjector components are matched randomly (i.e., similar to conventional approaches), when the autoinjector components are matched using heuristic rules (i.e., heuristic rules that attempt to offset the injection time contributions of prefilled syringes and autoinjector subassemblies), and when the autoinjector components are matched using a linear optimizer (i.e., a tool similar to component matching tool 130). In particular, as shown in FIG. 6, simulations indicate that when prefilled syringes are optimally paired with power injector subassemblies, lot-to-lot variability in arithmetic mean injection times is reduced by 30-45%.

[0049] The above discussion assumes that it is already known which characteristic values ​​should be used in determining component matching (i.e., which characteristics values ​​should be measured / collected for use as inputs to the predictive model 132). However, it can be difficult to understand which characteristic values ​​(and which components at various genealogy levels) are most strongly correlated with the characteristics of the final device / product. Furthermore, such correlations may change over time, manufacturing location, product revisions, etc. Therefore, a generalization tool is needed that quickly identifies correlations across representative aspects of the available data.

[0050] To this end, component matching tool 130 (or another software tool) may generate and / or populate a visualization such as that shown as plot 700 in FIG. 7 . Plot 700 represents an example of Shapley additive explanation (SHAP) and feature importance values ​​for an embodiment in which the combined device is an autoinjector. Plot 700 can help a user understand the importance of particular component characteristics and how they may be used, for example, to counteract each other. For example, an over-concentration of positive SHAP values ​​for subassembly mean injection time and an over-concentration of negative SHAP values ​​for pre-filled syringe (PFS) extrusion force suggests that the two characteristics may be particularly useful for offsetting each other to achieve reduced variability in the final autoinjector. A user may be able to apply such information, for example, to create or refine heuristic rules for matching batches of autoinjector components or to identify areas to focus on during autoinjector component manufacturing (e.g., areas where specification tolerances should be tightened). The component matching tool 130 (or another software tool) may additionally or alternatively generate and / or populate a visualization with one or more other types of visualizations, such as a correlation heat map or pair plot. For example, a correlation heat map may visually show (e.g., using a color scheme) assembly-day correlation, an arithmetic mean customer complaint metric, one or more process parameters, one or more statistics (e.g., mean and standard deviation) of one or more final lot parameters, one or more statistics (e.g., mean and standard deviation) of one or more component parameters, etc.

[0051] In some embodiments, the following process is performed (by the component matching tool 130 or other software) to create the predictive model 132: (1) the predictive model 132 is first trained and evaluated using the original input set of data features (characteristics), (2) the features are ordered by importance as calculated using SHAP, (3) the least important input data features are removed from the included features, (4) the predictive model 132 is retrained using a new, smaller set of data features / characteristics, (5) steps 2-4 are repeated until the input data feature list reaches the desired length, and (6) the entire process is (optionally) repeated to identify differences between model performance and the selected features / characteristics caused by random seeding of training and test sets between runs.

[0052] 8 is a flow diagram of an example method 800 for reducing variability in a combined device. Method 800 may be performed, in whole or in part, by a computing system (e.g., one or more computing devices) such as, for example, computing system 110 of FIG. 1 (e.g., by processor 120 executing instructions of component matching tool 130).

[0053] In block 802, potential combinations of a first component and a second component (e.g., components A and B in FIG. 1 ) are identified. Each potential combination can form one or more units of a combination device (e.g., an autoinjector). For example, each potential combination can be a combination of a particular batch of a first component with a particular batch of a second component. Block 802 may include receiving a first set of identifiers for different batches of the first component and a second set of identifiers for different batches of the second component to form different pairs / permutations of the first set of identifiers and the second set of identifiers, respectively, or may simply include receiving an indication of each potential combination. In some embodiments, block 802 includes selectively eliminating infeasible combinations of components.

[0054] At block 804, for each potential combination identified at block 802, a characteristic or outcome of a unit of the combined device (if formed from the first and second components of that combination) is predicted. Block 804 includes applying values ​​of one or more characteristics of the first component of the combination and values ​​of one or more characteristics of the second component of the combination as inputs to a predictive model (e.g., predictive model 132). In embodiments where the potential combinations are combinations of batches (i.e., two or more sets) of first and second components, the characteristic may be a characteristic that statistically represents each batch (e.g., average barrel bore diameter, etc.). Block 804 may include, for example, a numerical value (e.g., mean or standard deviation), a category (e.g., "likely high frequency of complaints" or "unlikely high frequency of complaints"), or a probability distribution for the particular characteristic.

[0055] At block 806, a subset of combinations is selected from among the potential combinations identified at block 802 based on predicted characteristics or outcomes of the potential combinations (i.e., based on the characteristics or outcomes predicted at block 804). Block 806 may include solving an objective function using the predicted characteristics or outcomes as inputs to the objective function. In some embodiments, block 806 includes selecting the subset based on one or more characteristics (e.g., infusion time standard deviation) and one or more outcomes (e.g., presence, amount, frequency, or likelihood of user complaints).

[0056] An indication of the selected subset of combinations is provided at block 808. For example, block 808 may include causing a display (e.g., display 124) to indicate the selected subset of combinations to a user and / or may include transmitting data indicative of the selected subset of combinations to a computing system or application.

[0057] In some embodiments, method 800 also includes one or more additional blocks not shown in Figure 8. For example, method 800 may further include, for each combination of potential combinations, predicting one or more additional attributes or outcomes of the unit of the combined device (when formed from the first and second components of the combination) using one or more additional predictive models. In these embodiments, the objective function may include multiple function terms each corresponding to a different attribute or outcome of the combined device.

[0058] As another example, method 800 may include an additional block after block 808 in which manufacturing equipment (e.g., commercial production equipment) assembles multiple units of the combined device according to a subset of the combinations selected in block 806 and indicated in block 808.

[0059] Embodiments of the present disclosure relate to non-transitory computer-readable storage media having computer code thereon for performing various computer-implemented operations. The term "computer-readable storage medium" is used herein to include any medium capable of storing or encoding a sequence of instructions or computer code for performing the operations, methods, and techniques described herein. The medium and computer code may be those specially designed and constructed for the purposes of the embodiments of the present disclosure, or may be of the kind well known and available to those skilled in the computer software arts. Examples of computer-readable storage media include, but are not limited to, magnetic media such as hard disks, floppy disks, and magnetic tape; optical media such as CD-ROMs and holographic devices; magneto-optical media such as optical disks; and hardware apparatuses specially configured for storing and executing program code, such as ASICs, programmable logic devices ("PLDs"), and ROM and RAM devices.

[0060] Examples of computer code include machine code, such as that produced by a compiler, and files containing relatively high-level code that is executed by a computer using an interpreter or compiler. For example, an embodiment of the present disclosure may be implemented using Java, C++, or other object-oriented programming language and development tools. Further examples of computer code include encryption and compression code. Furthermore, an embodiment of the present disclosure may be downloaded as a computer program product, which may be transferred from a remote computer (e.g., a server computer) to a requesting computer (e.g., a client computer or a different server computer) over a transmission channel. Another embodiment of the present disclosure may be implemented in hardwired circuitry in place of or in combination with machine-executable software instructions.

[0061] As used herein, the singular terms "a," "an," and "the" can include plural references unless the context clearly indicates otherwise.

[0062] As used herein, the terms "connect," "connected," and "connection" refer to an operative coupling or linking (and are represented by connections drawn in the drawings). Connected components may be directly or indirectly coupled to each other, for example, through a set of separate components.

[0063] As used herein, the terms "nearly," "substantially," "substantial," and "about" are used to describe and account for small variations. When used in conjunction with events or circumstances, these terms may refer not only to the exact occurrence of the event or circumstance, but also to events or circumstances occurring very close to the exact occurrence of the event or circumstance. For example, when used in conjunction with a numerical value, these terms may refer to a variation of ±10% or less of the numerical value, such as ±5% or less, ±4% or less, ±3% or less, ±2% or less, ±1% or less, ±0.5% or less, ±0.1% or less, or ±0.05% or less. For example, two numerical values ​​can be considered "substantially" identical if the difference between the two numerical values ​​is ±10% or less of the mean of the values, such as ±5% or less, ±4% or less, ±3% or less, ±2% or less, ±1% or less, ±0.5% or less, ±0.1% or less, or ±0.05% or less.

[0064] Furthermore, amounts, ratios, and other numerical values ​​may be presented herein in a range format. It should be understood that such range format is used for convenience and brevity and includes numerical values ​​expressly stated as the limits of the range, but should be understood flexibly to include all individual numerical values ​​or subranges subsumed within the range, as if each numerical value and subrange were expressly stated.

[0065] While the present disclosure has been described and illustrated with reference to specific embodiments thereof, these descriptions and illustrations are not intended to limit the disclosure. Those skilled in the art will recognize that various modifications may be made and equivalents may be substituted without departing from the true spirit and scope of the present disclosure, as defined by the appended claims. Illustrations may not necessarily be to scale. Differences between the technical representations in the present disclosure and the actual devices may exist due to manufacturing processes, tolerances, and / or other reasons. There may be other embodiments of the present disclosure that are not specifically illustrated. The specification and drawings (other than as claimed) are to be considered illustrative rather than restrictive. Changes may be made to adapt a particular situation, material, composition of matter, technique, or process to the objective, spirit, and scope of the present disclosure. All such modifications are intended to be within the scope of the claims appended hereto. While the techniques disclosed herein have been described with reference to particular operations being performed in a particular order, it will be understood that these operations may be combined, subdivided, or reordered to form equivalent techniques without departing from the teachings of this disclosure. Accordingly, unless specifically indicated herein, the order and grouping of operations is not a limitation of this disclosure.

Claims

1. 1. A method for reducing variability in a combined device, comprising: identifying, by one or more processors, a plurality of potential combinations of at least a first component and a second component, each of the potential combinations capable of forming one or more units of a combined device; For each of the potential combinations, predicting, by the one or more processors, a quality or outcome of the unit of the combined apparatus when formed from at least the first and second components of the combination by applying as inputs to a predictive model at least (i) values ​​of one or more properties of the first component of the combination and (ii) values ​​of one or more properties of the second component of the combination; selecting, by the one or more processors, a subset of combinations from among the potential combinations based on the predicted properties or outcomes of the potential combinations; providing, by the one or more processors, an indication of the selected subset of combinations; A method comprising:

2. each of the potential combinations is a different pair consisting of (i) the batch of the first component and (ii) the batch of the second component; For each of the potential combinations, each value of the one or more properties of the first component is statistically representative of a respective batch of the first component; The method of claim 1 , wherein each value of the one or more properties of the second component is statistically representative of a respective batch of the second component.

3. identifying the plurality of potential combinations receiving a first set of identifiers for different batches of the first component; receiving a second set of identifications of different batches of the second component; identifying the plurality of potential combinations by forming different pairs, each pair consisting of an identifier from the first set and an identifier from the second set; The method of claim 2 , comprising:

4. 4. The method of claim 1, wherein identifying the plurality of potential combinations includes at least one of eliminating infeasible combinations or receiving an indication of each of the plurality of potential combinations.

5. 5. The method of claim 1, wherein selecting the subset of combinations comprises solving an objective function using the predicted qualities or outcomes of the potential combinations as inputs to the objective function.

6. The method further includes, for each combination of the potential combinations, using one or more additional predictive models to predict one or more additional attributes or outcomes of the unit of the combined device when formed from the first and second components of the combination; The method of claim 5 , wherein the objective function includes multiple functional terms, each corresponding to a different attribute or outcome of the combined device.

7. the combination device is a filled syringe; the first component is a syringe; The method of any one of claims 1 to 6, wherein the second component is a fluid pharmaceutical.

8. 8. The method of claim 7, wherein the one or more characteristics of the first component include one or more of glide force, shield removal force, barrel diameter, break test result, pin diameter, or pin weight, and the one or more characteristics of the second component include one or more of viscosity, density, or protein concentration.

9. the combination device is an automatic injector; the first component is a pre-filled syringe; The method of any one of claims 1 to 6, wherein the second component is a power injector subassembly.

10. 10. The method of claim 9, wherein the one or more characteristics of the first component include one or more of extrusion force, sliding yield stress, protein concentration, or particle properties, and the one or more characteristics of the second component include one or more of specification number, saline release test results, injection time, actuation force, or spring force.

11. 11. The method of any one of claims 7 to 10, wherein predicting the quality or outcome of the units of the combination device comprises at least one of: (a) predicting a value indicative of variability in injection time or actuation force across the units of the combination device; (b) predicting a probability distribution of the characteristics of the combination device across the units of the combination device; or (c) predicting the existence, amount, frequency, or likelihood of a user complaint associated with the units of the combination device.

12. One or more non-transitory computer-readable media, When executed by one or more processors, the one or more processors: identifying a plurality of potential combinations of at least a first component and a second component, each of the potential combinations capable of forming one or more units of a combined device; For each of the potential combinations, predicting the characteristics or outcome of the unit of the combined device when formed from at least the first and second components of the combination by applying as inputs to a predictive model at least (i) the value of one or more properties of the first component of the combination and (ii) the value of one or more properties of the second component of the combination; selecting a subset of combinations from among the potential combinations based on the predicted properties or outcomes of the potential combinations; providing an indication of the selected subset of combinations; One or more non-transitory computer-readable media having stored thereon instructions to cause the

13. each of the potential combinations is a different pair consisting of (i) the batch of the first component and (ii) the batch of the second component; For each of the potential combinations, each value of the one or more properties of the first component is statistically representative of a respective batch of the first component; 13. The one or more non-transitory computer-readable media of claim 12, wherein each value of the one or more characteristics of the second components is statistically representative of a respective batch of the second components.

14. 14. The one or more non-transitory computer-readable media of claim 13, wherein identifying the plurality of potential combinations comprises eliminating infeasible combinations.

15. 15. The one or more non-transitory computer-readable media of any one of claims 12-14, wherein selecting the subset of combinations comprises solving an objective function using the predicted qualities or outcomes of the potential combinations as inputs to the objective function.

16. the combination device is a filled syringe; the first component is a syringe; 16. The one or more non-transitory computer-readable media of any one of claims 12 to 15, wherein the second component is a fluid pharmaceutical.

17. 16. The one or more non-transitory computer-readable media of claim 15, wherein the one or more characteristics of the first component include one or more of a glide force, a shield removal force, a barrel diameter, a break test result, a pin diameter, or a pin weight, and the one or more characteristics of the second component include one or more of a viscosity, a density, or a protein concentration.

18. the combination device is an automatic injector; the first component is a pre-filled syringe; 16. The one or more non-transitory computer-readable media of any one of claims 12 to 15, wherein the second component is an automatic injector subassembly.

19. 20. The one or more non-transitory computer-readable media of claim 18, wherein the one or more characteristics of the first component include one or more of an extrusion force, a sliding yield stress, a protein concentration, or a particle property, and the one or more characteristics of the second component include one or more of a specification number, a saline release test result, an injection time, an actuation force, or a spring force.

20. 20. The one or more non-transitory computer-readable media of any one of claims 16-19, wherein predicting the characteristics or outcomes of the units of the combined device comprises at least one of: (a) predicting a value indicative of a variability in injection time or actuation force across the units of the combined device; (b) predicting a probability distribution of the characteristics of the combined device across the units of the combined device; or (c) predicting the existence, amount, frequency, or likelihood of a user complaint associated with the units of the combined device.

Images