Component matching decision support tool

Abstract

A method for reducing variability of a combined device includes identifying potential combinations of at least a first and a second component, each combination capable of forming one or more units of the combined device. The method also includes, for each potential combination, predicting a quality or outcome of the unit of the combined device when formed from at least the first and second components of the combination by applying values ​​of one or more properties of at least the first component of the combination and values ​​of one or more properties of the second component of the combination as inputs to a predictive model. The method also includes selecting a subset of combinations from among the potential combinations based on the predicted quality or outcome of the potential combinations and providing an indication of the selected subset of combinations.

Description

[Technical field]

[0001] This application relates generally to the assembly or manufacture of devices, and more particularly to techniques for the selection of particular components or component batches / lots to be used when assembling or manufacturing a device. [Background technology]

[0002] In many manufacturing / production situations, different types of components are combined to form a combination device. In the pharmaceutical industry, for example, a fluid pharmaceutical is often combined (i.e., used for filling) with a syringe to produce a fluid-filled syringe. As another example, a fluid-filled syringe can be combined (i.e., filled) with an autoinjector subassembly to produce a complete autoinjector. As yet another example, a lyophilized pharmaceutical can be combined (i.e., used for filling) with a vial or other container. In each of these and other situations, multiple batches of each component may be available (e.g., in inventory) for combination with one another. Due to component variability (e.g., batch-level variability), the manner in which any two components are paired (or, more generally, two or more components are matched) for a given unit of a combination device may affect the way that individual component tolerances "stack" in the final product, leading to a wider distribution of possible values ​​for a characteristic of the paired device (e.g., the value of a functional output of the device, such as injection time for an autoinjector). Even when the tolerances of the individual components are tightly controlled, the properties of the final product may exhibit a relatively wide range of values, potentially leading to increased rejection rates, more user (e.g., customer or patient) complaints, and / or other problems. Furthermore, making the tolerances of the individual components tight enough to meet the tolerance targets in the final product may be prohibitively expensive. Summary of the Invention [Means for solving the problem]

[0003] To address some of the above shortcomings of current / conventional implementations, embodiments described herein include systems and methods that use data associated with different components (e.g., batch-specific or unit-specific data for different components) to better pair or match those components (e.g., specific components or specific batches of components) when manufacturing or assembling a combined device. In this manner, variability (e.g., around nominal specifications) of the combined device can be reduced, thereby providing more consistent product performance (e.g., fewer defects and / or fewer user complaints) compared to traditional process control measures. The term "component" is used broadly herein to refer to any physical part of a combined device (e.g., a structural subcomponent, a raw material, or a fluid, lyophilized drug, or other substance used to fill another component that acts as a container, etc.), unless the context of its use clearly indicates a more specific meaning. Additionally, the term "combined device" is used broadly herein to refer to any device that is manufactured or assembled using two or more components and may be a "final" product (e.g., for sale or distribution) or an intermediate stage of a product, unless the context of its use clearly indicates a more specific meaning.

[0004] In the techniques disclosed herein, a predictive machine learning model predicts at least one attribute or outcome of a unit of a combined device for each of several different combinations of components that may be used to form the combined device (e.g., batch 1 of component A with batch 1 of component B and batch 1 of component A with batch 2 of component B, etc.). The predicted attribute may be, for example, a measure of device quality or performance (e.g., standard deviation of a characteristic of the combined device across multiple units manufactured). As another example, the predicted outcome may be an indication of whether or not a user complaint (user complaints in general or a specific type of user complaint) will occur or how frequently such a complaint will occur. To make each prediction, the model operates on (i.e., accepts as input) values ​​of one or more characteristics of each component considered for the combination. These model inputs may include actual values, identifiers, other predicted (or estimated) values, and / or other types of upstream manufacturing data associated with the components (e.g., associated with a particular component batch). After the model predicts the attribute or outcome (or attributes and / or outcomes) of each of the various component combinations, an optimizer (e.g., a linear optimizer) operates on the predictions to determine which components (or component batches, etc.) should be matched to "best" (e.g., optimally) meet a desired measure of device quality or performance (e.g., a desired value of the attribute / attributes and / or outcome / outcomes predicted by the model).

[0005] In particular, it has been shown that the use of the techniques disclosed herein for pairing prefilled syringe batches with autoinjector subassembly batches can reduce lot-to-lot variability in average injection times by 30%-45% compared to conventional processes of randomly pairing batches. In general, as the number of component batches / lots available increases and / or the number of components required to manufacture / assemble a particular combined device increases, the benefit of using the techniques disclosed herein to match batches / lots of those components can correspondingly increase. Furthermore, software tools implementing the techniques described herein can facilitate understanding by a human user of why a particular combination of components or component batches is advantageous. For example, using such tools, a user can develop a set of heuristics or "rules of thumb" that allow intuitive pairing of components that offset different sources of variability. In general, the techniques described herein can help reduce lot-to-lot variability in manufactured combined devices and / or improve user (e.g., customer and / or patient) experience.

[0006] Those skilled in the art will appreciate that the figures described herein are included for illustrative purposes and are not intended to limit the disclosure. The figures are not necessarily to scale, with emphasis instead being placed on illustrating the principles of the disclosure. It should be understood that in some instances, various aspects of the described implementations may be shown exaggerated or enlarged to facilitate understanding of the described implementations. In the drawings, like reference numbers generally refer to functionally similar and / or structurally similar components throughout the various views. [Brief description of the drawings]

[0007] [Figure 1] FIG. 1 is a simplified block diagram of an example system implementing a component matching tool to match components to reduce variability across manufactured combined device units. [Figure 2A]1 shows components of a combination device as examples to which the component matching tool of FIG. 1 may match a particular component or a particular batch of components. [Figure 2B] 1 shows components of a combination device as examples to which the component matching tool of FIG. 1 may match a particular component or a particular batch of components. [Figure 2C] An example of equipment / product lineage is shown below. [Figure 3A] 2 illustrates an example process that may be implemented, at least in part, by the computing system of FIG. 1. [Figure 3B] 1 illustrates a pairing determination stage in an example manufacturing process. [Figure 4] 2 illustrates an example user interface that may be generated and / or populated by the entity matching tool of FIG. 1; [Diagram 5] 13A-13C depict plots showing example predictions of average injection times and average actuation forces for potential syringe stoppers, syringe barrels, and drug combinations. [Figure 6] 1 shows a plot comparing the predicted mean injection time of an autoinjector when matching autoinjector components randomly, when matching autoinjector components using heuristic rules, and when matching autoinjector components using a linear optimizer. [Figure 7] 13 shows an exemplary SHAP and feature importance plot for an embodiment in which the combination device is an autoinjector. [Figure 8] 1 is a flow diagram of an example method for reducing variability in a combined device. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0008] The various concepts described introductory above and discussed in more detail below can be implemented in any of numerous ways, and the described concepts are not limited to any particular implementation manner, examples of which are provided for illustrative purposes.

[0009] FIG. 1 is a simplified block diagram of an example system 100 in which the techniques disclosed herein may be implemented. The example system 100 includes M units of component A in batches 102A and N units of component B in batches 102B (M and N are the same or different integers, one or both of the numbers being greater than 1). Component A and component B may be any physical components that can be combined / assembled with each other to form a combined device. The combined device may be any device manufactured or assembled using two or more such components, and may be a "final" device / product (e.g., for sale or distribution) or an intermediate stage of the device / product (e.g., a subassembly of an instrument). FIG. 2A and FIG. 2B provide specific non-limiting examples of component A and / or component B. For example, component A can be the syringe 200 of Figure 2A, including the barrel 202, the plunger / stopper 204, the flange 206, the needle 208, and possibly the needle shield 210, and component B can be a fluid formulation that fills the syringe 200, such that the resulting combined device is a fluid-filled syringe. As another example, component A can be the combination of the barrel 202 (integral with the flange 206) and the (pre-staked) needle 208 of Figure 2A, and component B can be the plunger / stopper 204 of Figure 2A.

[0010] 2B illustrates yet another example corresponding to subsequent manufacturing / assembly stages where component A is a fluid-filled syringe 222 and component B includes both a front autoinjector subassembly 224 and a rear autoinjector subassembly 226, with the combined device being a complete autoinjector. Alternatively, system 100 may treat subassemblies 224 and 226 as separate components and match a batch of fluid-filled syringes 222 (component A), a batch of front autoinjector subassembly 224 (component B), and a batch of rear autoinjector subassembly 226 (component C, not shown in FIG. 1). It will be appreciated that the techniques described herein may be applied to any number of components and may be applied multiple times in successive manufacturing stages (e.g., to reduce variability in the fluid-filled syringes and also in subsequent manufacturing stages to reduce variability in the final autoinjector).

[0011] In some embodiments, system 100 matches a set of multiple batches with one or more batches in another set. For example, batch 102A may correspond to a set of 10 batches of component A, while batch 102B may correspond to one batch (or 10 batches, 20 batches, etc.) of component B. Although the description herein primarily relates to matching different lots or batches of components (e.g., each "batch" consists of units manufactured in the same process run and / or in the same time window, etc., and on the same equipment), in some embodiments system 100 instead matches components on a unit-by-unit basis (e.g., where all units of each component are tested to provide measurement data that can be analyzed, rather than simply sampling a subset of the batch units for measurement). Thus, it is understood that the embodiments described herein with respect to matching "batches" may instead match single units, match collections of units of any size (e.g., collections that do not correspond to "batches" defined by a manufacturing process), or match sets of multiple batches.

[0012] Further, while most of the embodiments described herein involve matching components of different types, in some embodiments the combined device is formed from at least two units of the exact same components, and batches 102A and 102B are subsets of the entire batch of components (i.e., components A and B are the same components). Further, while most of the embodiments described herein involve medical devices (e.g., autoinjectors), in other embodiments the combined device is a non-medical device, such as a home appliance or a home appliance subassembly, an automobile or an automobile subassembly, a medical device or a medical device subassembly, etc. For example, in an automobile embodiment, components A and B may be the cylinder head and piston of an engine, respectively.

[0013] In general, the matching techniques described herein may be used to match components at any level of component genealogy within a manufacturing process (e.g., one or more hierarchical levels each including one or more subassemblies or subcomponents and a final assembly / product). Certain circumstances and / or qualities may make a given manufacturing process more suitable for improvement through the techniques disclosed herein, including: (1) data (e.g., manufacturing data) from one or more components of the process is available before the corresponding manufacturing state begins, (2) batch (or other grouping) genealogy data is available that links which components will be used in a given final product or final product batch, (3) the available process component data can predict the characteristics of interest of the final product, and (4) most predicted characteristics include characteristics of two or more components that will be combined (rather than only one component without the opportunity to make any pairing decisions).

[0014] 2C shows an example of one such machine / product genealogy 250, where different batches of different components are combined to create subassemblies and ultimately different lots of final products. When the term "component" is used herein, it is understood that all boxes under the "final product" layer (e.g., "component C", "subassembly AB", etc.) can be viewed as combinable components. Thus, genealogy 250 represents a manufacturing process where the first level includes three separate components and the second level includes four separate components (two batches are shown for "component C" and two batches are shown for "component D").

[0015] In some embodiments, different batches of one component (e.g., component A or component B) are of the same general component type, but are not exactly the same. Referring again to FIG. 1, for example, all batches 102A of component A may be the same batch of syringes (e.g., the same part number or assembly of part numbers), while different batches 102B of component B may be different fluid pharmaceuticals. That is, the techniques described herein may be used to combine different batches of a single syringe type with different pharmaceuticals. This may be advantageous, since it is generally known that the effect of component variability is not constant across different drugs due to different drug characteristics (e.g., viscosity, protein concentration, etc.). Thus, optimal matching of drug types with different container batches and / or optimal matching of prefilled syringes containing different drug types with different autoinjector subassembly batches may reduce the variability of the final autoinjector.

[0016] The system 100 includes a characterization system 104 generally configured to measure and / or collect data representative of one or more characteristics of each batch 102A and each batch 102B. The characterization system 104 may include separate devices to measure the characteristics of the two batches 102A, 102B (e.g., if the types of component A and component B are sufficiently different to require different measurement devices or if component A and component B are manufactured at different locations, etc.). If component A is a syringe and component B is a fluid pharmaceutical, for example, the characterization system 104 may include a first sensor (e.g., a camera) and associated device (e.g., an optical comparator and a computing device that stores the measurements) that automatically and / or manually measures the barrel bore diameter of a unit of component A, and a second sensor (e.g., a viscometer) and associated device (e.g., a computing device that stores the measurements) that automatically and / or manually measures the pharmaceutical viscosity. The characterization system 104 may include and / or collect data from multiple data sources (eg, different organizations, systems, etc.), and the data may correspond to one or more time periods.

[0017] In some embodiments, the characterization system 104 samples only a subset of the units in each batch and identifies a representative characteristic value for that batch (e.g., average bore diameter, average viscosity, etc.) In general, the characteristic values ​​may be summary statistics from one or more tests and / or processes associated with each of the distinct components, such as mean, median, minimum, maximum, standard deviation, and specific quartile values.

[0018] In other embodiments (e.g., when components are matched individually rather than in batches), the characterization system 104 measures every unit of component A and every unit of component B. In some embodiments, the characterization system 104 includes devices and associated equipment (e.g., sensor controllers, computing devices, etc.) that indirectly measure (i.e., "soft sense") values ​​of particular properties. For example, the characterization system 104 may include a Raman spectrometer that analyzes a Raman scan of the fluid pharmaceutical to determine values ​​indicative of chemical composition and molecular structure.

[0019] System 100 also includes a computing system 110 coupled to characterization system 104. Computing system 110 may include a single computing device or multiple computing devices (e.g., one or more servers and one or more client devices) that are co-located or remote from one another. In the example embodiment shown in FIG. 1, computing system 110 includes one or more processors 120, a network interface 122, a display 124, a user input device 126, and a memory 128. In some embodiments, computing system 110 includes a portion of characterization system 104.

[0020] Each of the processors 120 may be a programmable microprocessor that executes software instructions stored in memory 128 to perform some or all of the functions of the computing system 110, as described herein. Alternatively, one or more of the processors 120 may be other types of processors (e.g., application specific integrated circuits (ASICs), field programmable gate arrays (FPGAs), etc.).

[0021] Network interface 122 may include any suitable hardware (e.g., front-end transmitter and receiver hardware), firmware, and / or software configured to communicate with external devices and / or systems (e.g., characterization system 104, another computing system that receives the optimal combination indication from computing system 110, or a server that provides an interface between computing system 110 and characterization system 104, etc.) using one or more communications protocols. For example, network interface 122 may be or include an Ethernet interface. Although not shown in FIG. 1 , computing system 110 may communicate with characterization system 104 and / or any device that provides an interface between computing system 110 and characterization system 104 over a single communications network or over one or more types of multiple communications networks (e.g., one or more wired and / or wireless local area networks (LANs), and / or one or more wired and / or wireless wide area networks (WANs), such as the Internet or an intranet, etc.).

[0022] Display 124 may use any suitable display technology (e.g., LED, OLED, LCD, etc.) to present information to a user, and user input device 126 may be a keyboard or other suitable input device. In some embodiments, display 124 and user input device 126 are integrated into a single device (e.g., a touch screen display). In general, display 124 and user input device 126 may be combined to enable a user to view and / or interact with a visual presentation (e.g., a graphical user interface or displayed information) output by computing system 110, for purposes such as, for example, informing a user of recommended component batch combinations.

[0023] Memory 128 may include one or more physical memory devices or units, including volatile and / or non-volatile memory, and may include memory located in various computing devices of computing system 110. Any suitable type or types of memory may be used, such as read only memory (ROM), solid state drive (SSD), hard disk drive (HDD), etc. Memory 128 stores instructions for one or more software applications, including component matching tool 130. Component matching tool 130, when executed by some or all of processors 120, is generally configured to (1) collect component batch data generated by characterization system 104, (2) determine which combination of batches 102A and 102B should provide better characteristics and / or results for a unit of combined equipment (e.g., a narrower distribution of a particular characteristic of the combined equipment and / or fewer user complaints, etc.), and (3) provide an indication of those combinations to a user and / or another computing system or application. To this end, the component matching tool 130 includes a training module 140, a combination identification module 142, a prediction module 144, and an optimization module 146. The modules 140-146 may be separate software components or modules of the component matching tool 130, or may simply represent functionality of the component matching tool 130 that is not necessarily divided among different components / modules. For example, in some embodiments, the prediction module 144 and the training module 140 are included in a single software module. Moreover, in some embodiments, the different modules 140-146 may be distributed among multiple copies of the component matching tool 130 (e.g., running on different devices in the computing system 110) or among different types of applications stored and executed on one or more devices of the computing system 110. The operation of each of the modules 140-146 is described in further detail below with reference to the operation of the system 100.

[0024] As also described in further detail below, the computing system 110 is configured to access the historical database 150 for training purposes. The historical database 150 may store parameter values ​​associated with past characterization data collected by the characterization system 104 and / or collected by other similar devices or systems. For example, the historical database 150 may store the individual (or average, arithmetic mean, etc.) viscosity and / or protein concentration of the fluid pharmaceutical, the syringe barrel diameter and / or glide force, etc., and possibly other related parameter values ​​(e.g., time). In some embodiments, the historical database 150 additionally or alternatively stores categorical values ​​of particular components or component batches, such as data indicating whether a pharmaceutical fluid batch was observed to contain particles above a threshold size (e.g., above 20 μm). The historical database 150 may also store "label" information that represents the actual attributes and / or results of a unit of the combined device formed from a particular combination of batches of different components. In some embodiments, each label is an actual measured value associated with a resulting lot of the combined device. For example, each label may be the average injection time or average actuation force of a batch of autoinjectors formed using a particular combination of component batches, or a value or set of values ​​defining a probability distribution of injection times or actuation forces of a batch of autoinjectors, etc. Alternatively or additionally, each label may be an indication of whether a particular type of user complaint (e.g., patient complaints, customer complaints, etc.) or user complaints in general exceeded some threshold number or frequency for the resulting combined device, or the exact number, frequency, and / or likelihood of such complaints, etc. Database 150 may be stored in a permanent memory in memory 128, in a different permanent memory of computing system 110, or in another device or system. In some embodiments, computing system 110 accesses database 150 over the Internet using network interface 122.

[0025] The training module 140 of the component matching tool 130 uses the characteristic (e.g., measurement) data and associated labels stored in the historical database 150 to train a predictive model 132, which is then used by the prediction module 144 to make predictions of the units of the combined device that will result from a particular combination of component batches. The predictive model 132 can be a decision tree ensemble model (e.g., a gradient boosted tree model), a neural network, a support vector machine (SVM) model, a decision tree model, or any other suitable type of model. As a more specific example, the predictive model 132 can be an XG boost model, which has been found to deliver superior performance for the disclosed techniques compared to certain other predictive model types (e.g., linear regression, LASSO, and random forest models). The predictive model 132 may include, for example, a classifier that predicts a particular classification of a combined device that will be generated as a result of a particular combination of component batches (e.g., "within specification tolerance" or "not within specification tolerance" or "good user experience", "medium user experience", and "poor user experience", etc.), a model that predicts a particular numerical value or values ​​(e.g., mean infusion time, standard deviation of infusion time, interquartile range of infusion time, etc.), or a model that predicts a complete probability distribution (e.g., where the predictive model 132 includes a modular neural network (MNN)). The predictive model 132 may be created using, for example, an open source Python library.

[0026] In some embodiments, the predictive model 132 is retrained / improved or a new predictive model is trained for a particular use case. For example, the predictive model 132 or a new model may be trained using historical data specific to the manufacturing location to be used, the revision / version or type of pharmaceutical being manufactured, a particular time period, etc. Additionally, in embodiments where one or more labels represent user experience, the labels may be limited to the relevant geography (e.g., global, US, or non-US), age group, end use environment (e.g., sample device, device for sale, or medical use device), among others.

[0027] Generating a predictive model 132 (e.g., partially or entirely by training module 140, perhaps with some user input) may include, for example, the following: (1) preprocessing data (preparing historical data for use by predictive model 132 by normalizing and scaling features, converting data types, and inputting missing data); (2) separating the preprocessed data into training and test / validation sets (to avoid overfitting the data and to estimate the performance of predictive model 132 on unseen data); and (3) selecting a model type (determining which supervised machine learning algorithm to use for a given prediction). ), (4) select and cross-validate hyperparameters (determine discretionary parameters to tune a given model for a particular dataset), (5) perform model fitting (use the training data and tuning parameters to create a predictive model capable of selecting the optimal loss function to minimize), (6) evaluate the predictive model 132 (assess model performance by using the trained predictive model 132 to make predictions on a test set), and (7) determine feature importance (provide a visual representation of how the trained predictive model 132 arrives at a predicted output value based on the input data). Tuning parameters for the XG Boost model may include, for example, maximum depth, number of estimators, minimum child weights, learning rate, column sample by tree, subsample, alpha, gamma, and lambda. Model performance may be measured, for example, using the coefficient of determination (R 2 ) metric.

[0028] As discussed in more detail below, the combination identification module 142 is generally configured to identify specific combinations of component batches (e.g., different combinations each pairing one of batches 102A with one of batches 102B), and the prediction module 144 will then predict one or more attributes and / or results of the combined device resulting from the combination. As also discussed in more detail below, the optimization module 146 is generally configured to process the combination-specific predictions output by the prediction module 144 to generate a set of combinations to be used in manufacturing / assembly or a set of combinations recommended for manufacturing / assembly.

[0029] As discussed above, the computing system 110 may include one device or multiple devices, and if multiple devices, may be co-located or remotely distributed (e.g., with Ethernet and / or Internet communication between the different devices). In one embodiment, for example, a first server (including module 140) of the computing system 110 trains the predictive model 132, a second server of the computing system 110 collects measurements and / or other data from the characterization system 104, and a third server (including modules 142, 144, 146) of the computing system 110 receives measurements and / or other data from the second server and uses a copy of the trained predictive model 132 to generate predictions based on the received measurements and / or other data. As another example, the third server in the above example does not store a copy of the trained predictive model 132, but instead utilizes the predictive model 132 by providing measurements to the second server (e.g., if the predictive model 132 is available via a web service configuration). As used herein, unless the context of the term clearly indicates otherwise, terms such as "running," "using," or "implementing" a model (e.g., predictive model 132) are used broadly to encompass the alternative of directly running a locally stored model or requesting another device (e.g., a remote server) to run the model. It is understood that still other configurations and distributions of functionality beyond those shown in FIG. 1 and / or described herein are possible and are within the scope of the invention disclosed herein.

[0030] The operation of the system 100 will now be described in more detail with reference to both the components of Figure 1 and the process 300 shown in Figure 3A. In an initial training phase 302, the component matching tool 130 retrieves historical data 304 (e.g., past component measurements and associated labels) from the historical database 150, and the training module 140 uses the retrieved historical data 304 to train the predictive model 132. For example, if component A is a syringe and component B is a pharmaceutical product, the historical data 304 may include: (1) for each batch of syringes represented in the historical database 150, data indicative of the glide force, shield removal force, barrel bore, break test results, pusher trim maximum outer diameter, pusher maximum outer diameter, and / or pusher silicone weight, e.g., average values ​​for the batch, and (2) for each batch and / or type of pharmaceutical product represented in the historical database 150, a pharmaceutical product identifier and / or other information, e.g., pharmaceutical product viscosity, using average values ​​where appropriate, protein concentration, manufacturing location, and / or density. As another example, if component A is a prefilled syringe and component B includes a rear autoinjector subassembly, historical data 304 may include: (1) for each batch of prefilled syringes represented in historical database 150, data indicative of the extrusion force, breakloose force, protein concentration, characteristics (e.g., whether particles above a certain size were observed in the pharmaceutical product), and / or any of the above syringe-specific characteristics (e.g., glide force, barrel bore, etc.), e.g., average values ​​for the batch, and (2) for each batch of rear autoinjector subassemblies represented in historical database 150, specification number, saline release test results (injection time and / or actuation force), and / or spring data (e.g., spring force), e.g., using average values ​​where appropriate.

[0031] The training module 140 trains the predictive model 132 to predict a particular attribute of the combined device, such as injection time or actuation force (e.g., if the combined device is a prefilled syringe or a fully automatic injection device). The value may be, for example, an expected average value of the property for all combined devices formed from a particular batch associated with a particular combination (e.g., combination of batch 1 of component A with batch 2 of component B) or an expected metric indicating the variability of the property (e.g., expected standard deviation). In other embodiments, the training module 140 trains the predictive model 132 to predict a probability distribution of the combined device when formed from a particular combination of components.

[0032] In some embodiments, the predictive model 132 includes one or more preliminary model stages. For example, a first model stage of the predictive model 132 may convert inputs from the historical data 150 into values ​​reflecting a reduced set of dimensions. As a more specific example, the predictive model 132 may apply a principal component analysis (PCA) technique to a set or subset of inputs to reduce the set or subset to n dimensions, and then apply the n values ​​(one per dimension) as inputs to a subsequent model stage (e.g., a decision tree ensemble model or a neural network) that outputs a predicted value. In such an embodiment, the training module 140 may perform the dimensionality reduction in stage 302 and then apply the inputs to the remainder of the predictive model 132 for training purposes.

[0033] Stage 302 may also include validating and / or qualifying the trained predictive model 132 (e.g., using a portion of the historical data 304 that was not used for training). Once satisfactorily trained and possibly validated / qualified, the prediction module 144 of the component matching tool 130 may use the predictive model 132 in stage 310 to predict values ​​of one or more combined device properties for each combination of components or component batches under consideration. However, prior to stage 310, the combination identification module 142 of the component matching tool 130 determines which combinations of component batches (and thus which subsets of the new data 306) should be analyzed. The new data 306 includes data indicative of properties of component batches to be assembled to form a new unit of combined device, and may include the same types of measurements and / or other component data (e.g., syringe glide force, pharmaceutical identifiers, etc.) as described above with respect to the historical data 304.

[0034] The combination identification module 142 may determine the combinations to consider, for example, by receiving a list of batch identifiers of different components (e.g., component A and component B) manually entered by a user (e.g., via the user input device 126) or from another computing system, storage device, or application, and then automatically generating each possible permutation based on the batch identifier list. In other embodiments, the combination identification module 142 in step 308 also filters out (omits) combinations that are not feasible or practical (e.g., it is not possible to combine two batches due to the timing / sequencing of the manufacturing process) so as not to waste processing resources in step 310. In other embodiments, the combination identification module 142 identifies / determines appropriate component / batch combinations to consider simply by receiving an indication of appropriate component / batch combinations to consider (e.g., manually entered by a user or from another computing system, storage device, or application). In some embodiments and / or situations, the output of the combination identification module 142 (and / or the input to the combination identification module 142) is limited based on time, e.g., so that a component batch can only be reserved in a “holding” area for a limited amount of time to avoid significant disruptions to the manufacturing process.

[0035] In step 310, the prediction module 144 applies a portion / subset of the new data 306 corresponding to the components / batches of combinations identified in block 308 as input to the predictive model 132. The prediction module 144 may operate on the data 306 sequentially for each identified combination, i.e., by predicting the attribute or outcome of interest for one combination and then processing the next combination. Alternatively (e.g., if the processor 120 is capable of implementing multiple instances of the prediction module 144), the component matcher 130 may perform predictions for multiple combinations in parallel. As mentioned above, the predictive model 132 may in some embodiments include a dimensionality reduction step, in which case step 310 includes reducing the data dimensionality before inputting the dimensionality reduced data into the predictive model 132.

[0036] The output in step 310 includes predictions of one or more attributes and / or outcomes of each combination identified in step 308 (i.e., attributes and / or outcomes of the same type as those represented by the labels used in training step 302). In step 312, the optimization module 146 applies the predicted attributes and / or outcomes of all identified combinations as inputs to an optimizer to determine which set of combinations provides the "best" (e.g., optimal) performance. The optimizer may be a linear optimizer. For example, the optimization module 146 may use PuLP (Python linear programming application programming interface (API)) to define the objective function and call an external solver.

[0037] In some embodiments where the prediction module 144 predicts only one attribute (e.g., the standard deviation of infusion times across all combined device units resulting from a particular component combination), the optimization module 146 solves the following objective function:

number

[0038] [Table 1]

[0039] To minimize the objective function in this example, the optimization module 146 determines which of the columns X1-X6 leads to the minimum sum and selects that as the desired set of combinations.

[0040] In some embodiments, the optimization module 146 minimizes an objective function that includes multiple terms, each similar to those shown in the function above, e.g., with one term for each attribute or outcome predicted by the prediction module 144. For example, the objective function may include a first term corresponding to the average infusion time and a second term corresponding to the average number of complaints (e.g., 0.1 if 1 complaint is predicted per 10 units). Some or all of the individual terms of the objective function may be weighted depending on the perceived importance of each term to the overall performance of the combined device.

[0041] The optimization module 146 provides an output indicative of the set / permutation of combinations that minimizes the objective function, and in step 314, a lot of the combined equipment is manufactured (e.g., using commercial-scale production equipment) using the indicated combinations. In some embodiments, the combination matching tool 130 causes a display (e.g., display 124) to present a user interface that shows the resulting combinations to a user, who then (possibly after manual review / verification) takes one or more actions to ensure that production in step 314 uses those combinations. In other embodiments, the combination matching tool 130 transmits data indicative of the resulting combinations to another computing system or application, and the computing system or application causes the manufacturing process to automatically use those combinations (e.g., by controlling appropriate handling / routing equipment).

[0042] In some embodiments, measurements of the combined devices manufactured in step 314 are obtained manually or automatically and used as labels to compare against the predictions made in step 310. For each of one or more batch combinations, the corresponding input data (from new data 306), e.g., applied to the predictive model 132 along with the corresponding label, can be fed back to the training module 140, which can use the data and labels to improve / update the predictive model 132 through further training.

[0043] FIG. 3B illustrates the pairing decision stages in an example manufacturing process 320. In a first stage, different drugs (and possibly different batches of each drug) are paired with other components (and possibly different batches of those other components). For example, different fluid pharmaceuticals may be paired with different types of assembled syringes and / or different batches of those syringes. The second stage shown in FIG. 3B may represent, for example, the pairing of a particular filled syringe with a particular autoinjector subassembly. The component matching tool 130 may match components and / or batches in an attempt to optimize one or more characteristic values, such as minimizing glass breakage probability, minimizing standard deviation from a desired injection time, etc. (as shown by the dashed lines in FIG. 3B). Typically, as the stage number increases (i.e., closer to the final product), there is greater data availability and predictive power, but fewer opportunities for improvement through matching decisions.

[0044] FIG. 4 illustrates an example user interface 400 that may be generated and / or populated by the component matching tool 130 of FIG. 1 or a similar application in some embodiments. FIG. 4 illustrates an embodiment in which component matching is performed in two successive stages: a pre-fill stage in which batches / lots of empty syringes are matched with different pharmaceutical products ("Drug 1", "Drug 2", and "Drug 3"), and a post-fill stage in which those filled ("prefilled") syringes are matched with batches / lots of autoinjector subassemblies. The quantities shown in FIG. 4 are purely for illustrative purposes and are not necessarily real values. It is further understood that any suitable units may be associated with the values ​​(e.g., Newtons for average actuation force and syringe glide force, seconds or milliseconds for average injection time, etc.).

[0045] In the row of user interface 400 labeled "Desired Specifications and Quantities," the user may select (e.g., via user input device 126) a particular drug to be considered (in the illustrated example, "Drug 1" and "Drug 2"). In the row labeled "Pre-Fill Data," the user may enter (e.g., via user input device 126) and / or view (e.g., via display 124) values ​​of properties associated with a particular lot / batch of the selected drug and a particular lot / batch of syringes that may be used to hold those drugs. In the illustrated example, values ​​of properties of the pusher lot / batch are also displayed, and component matching tool 130 matches batches of drugs, syringes, and pushers. For example, with reference to FIG. 1, the drug is component A and the syringes are component B, while the pushers may be component C, not shown in FIG. 1. In the row labeled "Post-Fill Data," the user can enter (e.g., via user input device 126) and / or view (e.g., via display 124) values ​​of characteristics associated with a particular lot / batch of prefilled syringes (for each medication) and values ​​of characteristics associated with a particular lot / batch of autoinjector subassemblies.

[0046] In the example user interface 400, the results for each selected drug are shown in the first two rows. In particular, the first row shows the optimized batch pairing of the drug with the syringe and plunger, as well as plots of the predicted injection time and actuation force (for each selected drug), and the second row shows the predicted average injection time and average actuation force values ​​for each selected drug.

[0047] FIG. 5 illustrates a plot 500 showing an example of predicted average injection times and average actuation forces for potential syringe stopper, syringe barrel, and drug combinations. In some embodiments, the combination matching tool 130 generates and / or populates visualizations including plots similar to plot 500, for example, to help a user understand how different drugs affect component batch variability. In FIG. 5, each circle represents a different combination of syringe stopper / barrel batch and drug type, with the enlarged circle representing a combination using one particular batch / lot of syringe barrels. As can be seen from FIG. 5, filling a syringe barrel from this particular batch with a drug represented as Drug 3 (at location 502) leads to the shortest injection time.

[0048] 6 shows a plot 600 comparing the predicted standard deviation of the mean autoinjector injection time when the autoinjector components are matched randomly (i.e., similar to conventional approaches), when the autoinjector components are matched using heuristic rules (i.e., heuristic rules that attempt to offset the injection time contributions of prefilled syringes and autoinjector subassemblies), and when the autoinjector components are matched using a linear optimizer (i.e., a tool similar to component matching tool 130). In particular, as shown in FIG. 6, simulations indicate that when prefilled syringes are optimally paired with autoinjector subassemblies, lot-to-lot variability in arithmetic mean injection times is reduced by 30-45%.

[0049] The above discussion assumes that it is already known which characteristic values ​​should be used in determining component matching (i.e., characteristics whose values ​​should be measured / collected for use as input to the predictive model 132). However, it can be difficult to understand which characteristic values ​​(and which components at various genealogy levels) are most strongly correlated with the characteristics of the final device / product. Furthermore, such correlations may vary across time, manufacturing locations, product revisions, etc. Thus, a generalization tool is needed that quickly identifies correlations across representative aspects of the available data.

[0050] To this end, the component matching tool 130 (or another software tool) may generate and / or populate a visualization such as that shown as plot 700 in FIG. 7. Plot 700 represents an example of Shapley additive explanation (SHAP) and feature importance values ​​in an embodiment where the combined device is an autoinjector. Plot 700 may help a user understand the importance of certain component characteristics and how they may be used, for example, to counteract one another. For example, an over-concentration of positive SHAP values ​​for subassembly mean injection time and an over-concentration of negative SHAP values ​​for pre-filled syringe (PFS) push force suggests that the two characteristics may be particularly useful for offsetting one another to achieve reduced variability in the final autoinjector. A user may be able to apply such information, for example, to create or refine heuristic rules for matching batches of autoinjector components or to identify areas to focus on (e.g., areas to tighten specification tolerances) during the manufacture of autoinjector components. The component matching tool 130 (or another software tool) may additionally or alternatively generate and / or populate the visualization with one or more other types of visualizations, such as a correlation heat map or pair plots. For example, a correlation heat map may visually show (e.g., using a color scheme) assembly-day correlation, an arithmetic mean customer complaint metric, one or more process parameters, one or more statistics (e.g., mean and standard deviation) of one or more final lot parameters, one or more statistics (e.g., mean and standard deviation) of one or more component parameters, etc.

[0051] In some embodiments, the following process is performed to create the predictive model 132 (by the component matching tool 130 or other software): (1) the predictive model 132 is first trained and evaluated using the original input set of data features (characteristics), (2) the features are ordered by importance as calculated using SHAP, (3) the least important input data features are removed from the included features, (4) the predictive model 132 is retrained using a new, smaller set of data features / characteristics, (5) steps 2-4 are repeated until the input data feature list reaches the desired length, and (6) the entire process is (optionally) repeated to identify differences between model performance and the selected features / characteristics resulting from random seeding of training and test sets between runs.

[0052] 8 is a flow diagram of an example method 800 for reducing variability in combined devices. Method 800 may be performed, in whole or in part, by a computing system (e.g., one or more computing devices) such as, for example, computing system 110 of FIG. 1 (e.g., by processor 120 executing instructions of component matching tool 130).

[0053] At block 802, potential combinations of a first component and a second component (e.g., components A and B of FIG. 1) are identified. Each potential combination can form one or more units of a combination device (e.g., an autoinjector). For example, each potential combination can be a combination of a particular batch of a first component with a particular batch of a second component. Block 802 may include receiving a first set of identifiers of different batches of the first component and a second set of identifiers of different batches of the second component, forming different pairs / permutations of the first set of identifiers and the second set of identifiers, respectively, or may simply include receiving an indication of each potential combination. In some embodiments, block 802 includes selectively omitting infeasible combinations of components.

[0054] At block 804, for each potential combination identified at block 802, a characteristic or outcome of a unit of the combined device (if formed from the first and second components of the combination) is predicted. Block 804 includes applying values ​​of one or more characteristics of the first component of the combination and values ​​of one or more characteristics of the second component of the combination as inputs to a predictive model (e.g., predictive model 132). In embodiments where the potential combinations are combinations of batches (i.e., two or more sets) of first and second components, the characteristics may be characteristics that are statistically representative of each batch (e.g., average barrel bore diameter, etc.). Block 804 may include, for example, a numerical value (e.g., mean or standard deviation), a category (e.g., "likely to have high frequency complaints" or "likely to have high frequency complaints"), or a probability distribution for a particular characteristic.

[0055] At block 806, a subset of combinations is selected from among the potential combinations identified at block 802 based on predicted attributes or outcomes of the potential combinations (i.e., based on the attributes or outcomes predicted at block 804). Block 806 may include solving an objective function using the predicted attributes or outcomes as inputs to the objective function. In some embodiments, block 806 includes selecting the subset based on one or more attributes (e.g., infusion time standard deviation) and one or more outcomes (e.g., the presence, amount, frequency, or likelihood of user complaints).

[0056] An indication of the selected subset of combinations is provided at block 808. For example, block 808 may include causing a display (e.g., display 124) to indicate the selected subset of combinations to a user and / or may include transmitting data indicative of the selected subset of combinations to a computing system or application.

[0057] In some embodiments, method 800 also includes one or more additional blocks not shown in Figure 8. For example, method 800 may further include predicting, for each combination of potential combinations, one or more additional attributes or outcomes of the unit of the combined device (when formed from the first and second components of the combination) using one or more additional predictive models. In these embodiments, the objective function may include multiple function terms each corresponding to a different attribute or outcome of the combined device.

[0058] As another example, method 800 may include an additional block after block 808 in which a manufacturing machine (e.g., a commercial production machine) assembles a plurality of units of the combined apparatus according to a subset of the combinations selected in block 806 and indicated in block 808.

[0059] An embodiment of the present disclosure relates to a non-transitory computer-readable storage medium having computer code for performing various computer-implemented operations. The term "computer-readable storage medium" is used herein to include any medium capable of storing or encoding a set of instructions or computer code for performing the operations, methods and techniques described herein. The medium and computer code may be of a type that is specially designed and constructed for the purposes of the embodiments of the present disclosure, or may be of a type that is well known and available to those skilled in the computer software art. Examples of computer-readable storage media include, but are not limited to, magnetic media such as hard disks, floppy disks and magnetic tapes, optical media such as CD-ROMs and holographic devices, magneto-optical media such as optical disks, and hardware devices specially configured to store and execute program code, such as ASICs, programmable logic devices ("PLDs"), and ROM and RAM devices.

[0060] Examples of computer code include machine code, such as that produced by a compiler, and files containing relatively high-level code executed by a computer using an interpreter or compiler. For example, an embodiment of the present disclosure may be implemented using Java, C++, or other object-oriented programming languages ​​and development tools. Further examples of computer code include encryption and compression code. Furthermore, an embodiment of the present disclosure may be downloaded as a computer program product, which may be transferred from a remote computer (e.g., a server computer) to a requesting computer (e.g., a client computer or a different server computer) over a transmission channel. Another embodiment of the present disclosure may be implemented in hardwired circuitry in place of, or in combination with, machine-executable software instructions.

[0061] As used herein, the singular terms "a," "an," and "the" can include plural references unless the context clearly indicates otherwise.

[0062] As used herein, the terms "connect," "connected," and "connection" refer to an operative coupling or linking (and connections depicted in the drawings represent). Connected components may be directly or indirectly coupled to one another, for example, through a set of separate components.

[0063] As used herein, the terms "nearly", "substantially", "substantial" and "about" are used to describe and explain small variations. When used in conjunction with an event or circumstance, these terms may refer to not only the exact occurrence of the event or circumstance, but also to the occurrence of an event or circumstance that is very close. For example, when used in conjunction with a numerical value, these terms may refer to a variation of ±10% or less of the numerical value, such as ±5% or less, ±4% or less, ±3% or less, ±2% or less, ±1% or less, ±0.5% or less, ±0.1% or less, or ±0.05% or less. For example, two numerical values ​​can be considered "substantially" identical if the difference between the two numerical values ​​is ±10% or less of the average of the numerical values, such as ±5% or less, ±4% or less, ±3% or less, ±2% or less, ±1% or less, ±0.5% or less, ±0.1% or less, or ±0.05% or less.

[0064] Additionally, amounts, ratios, and other numerical values ​​may be presented herein in a range format, with the understanding that such range format is used for convenience and brevity and includes numerical values ​​expressly specified as the limits of the range, but should be understood in a flexible manner to include all individual numerical values ​​or subranges subsumed within the range, as if each numerical value and subrange were expressly specified.

[0065] Although the present disclosure has been described and illustrated with reference to specific embodiments thereof, these descriptions and illustrations are not intended to limit the present disclosure. Those skilled in the art will understand that various modifications may be made and equivalents may be substituted without departing from the true spirit and scope of the present disclosure as defined by the appended claims. The illustrations may not necessarily be to scale. Due to manufacturing processes, tolerances, and / or other reasons, there may be differences between the technical expressions in the present disclosure and the actual devices. There may be other embodiments of the present disclosure that are not specifically illustrated. The present specification and drawings (other than those in the claims) should be regarded as illustrative rather than restrictive. Changes may be made to adapt a particular situation, material, composition of matter, technique, or process to the objective, spirit, and scope of the present disclosure. All such modifications are intended to be within the scope of the claims appended hereto. Although the techniques disclosed herein have been described with reference to particular operations being performed in a particular order, it will be understood that these operations may be combined, sub-divided, or re-ordered to form equivalent techniques without departing from the teachings of this disclosure. Thus, unless specifically indicated herein, the order and grouping of operations is not intended to be limiting of the disclosure.

Claims

1. 1. A method for reducing variability in a combined device, comprising: identifying, by one or more processors, a plurality of potential combinations of at least a first component and a second component, each of the potential combinations capable of forming one or more units of a combined device; for each of the potential combinations, predicting, by the one or more processors, a quality or outcome of the unit of the combined apparatus when formed from at least the first and second components of the combination by applying as inputs to a predictive model at least (i) values ​​of one or more properties of the first component of the combination and (ii) values ​​of one or more properties of the second component of the combination; selecting, by the one or more processors, a subset of combinations from among the potential combinations based on the predicted properties or outcomes of the potential combinations; providing, by the one or more processors, an indication of the selected subset of combinations; The method includes:

2. each of the potential combinations is a different pair of (i) the batch of the first component and (ii) the batch of the second component; For each of the potential combinations, each value of the one or more properties of the first component is statistically representative of a respective batch of the first component; The method of claim 1 , wherein each value of the one or more characteristics of the second components is statistically representative of a respective batch of the second components.

3. Identifying the plurality of potential combinations comprises: receiving a first set of identifiers for different batches of the first component; receiving a second set of identifications of different batches of the second component; and identifying the plurality of potential combinations by forming different pairs, each pair consisting of an identifier from the first set and an identifier from the second set; The method of claim 2 , comprising:

4. 4. The method of claim 1, wherein identifying the plurality of potential combinations comprises at least one of: eliminating infeasible combinations; or receiving an indication of each of the plurality of potential combinations.

5. 5. The method of claim 1, wherein selecting the subset of combinations comprises solving an objective function using the predicted qualities or outcomes of the potential combinations as inputs to the objective function.

6. The method further includes, for each of the potential combinations, using one or more additional predictive models to predict one or more additional attributes or outcomes of the unit of the combined device when formed from the first and second components of the combination; The method of claim 5 , wherein the objective function includes a number of functional terms, each functional term corresponding to a different attribute or outcome of the combined device.

7. the combination device is a filled syringe; the first component is a syringe; The method according to any one of claims 1 to 6, wherein the second component is a fluid pharmaceutical.

8. 8. The method of claim 7, wherein the one or more characteristics of the first component include one or more of glide force, shield removal force, barrel diameter, break test result, pin diameter, or pin weight, and the one or more characteristics of the second component include one or more of viscosity, density, or protein concentration.

9. the combination device is an automatic injector; the first component is a pre-filled syringe; The method of any one of claims 1 to 6, wherein the second component is an automatic injector subassembly.

10. 10. The method of claim 9, wherein the one or more characteristics of the first component include one or more of an extrusion force, a sliding yield stress, a protein concentration, or a particle property, and the one or more characteristics of the second component include one or more of a specification number, a saline release test result, an injection time, an actuation force, or a spring force.

11. 11. The method of any one of claims 7 to 10, wherein predicting the quality or outcome of the units of the combination device comprises at least one of: (a) predicting a value indicative of a variability in injection time or actuation force across the units of the combination device; (b) predicting a probability distribution of the characteristics of the combination device across the units of the combination device; or (c) predicting the presence, amount, frequency, or likelihood of a user complaint associated with the units of the combination device.

12. One or more non-transitory computer-readable media, When executed by one or more processors, the one or more processors identifying a plurality of potential combinations of at least a first component and a second component, each of the potential combinations being capable of forming one or more units of a combined device; for each of said potential combinations, predicting a quality or outcome of said unit of said combined apparatus when formed from at least said first and second components of said combination by applying as inputs to a predictive model at least (i) values ​​of one or more properties of said first component of said combination and (ii) values ​​of one or more properties of said second component of said combination; selecting a subset of combinations from among the potential combinations based on the predicted properties or outcomes of the potential combinations; providing an indication of the selected subset of combinations; One or more non-transitory computer-readable media having instructions stored thereon to cause the

13. each of the potential combinations is a different pair of (i) the batch of the first component and (ii) the batch of the second component; For each of the potential combinations, each value of the one or more properties of the first component is statistically representative of a respective batch of the first component; 13. The one or more non-transitory computer-readable media of claim 12, wherein respective values ​​of the one or more characteristics of the second components are statistically representative of respective batches of the second components.

14. 14. The one or more non-transitory computer-readable media of claim 13, wherein identifying the plurality of potential combinations comprises eliminating infeasible combinations.

15. 15. The one or more non-transitory computer-readable media of any one of claims 12-14, wherein selecting the subset of combinations comprises solving an objective function using the predicted qualities or outcomes of the potential combinations as inputs to the objective function.

16. the combination device is a filled syringe; the first component is a syringe; The one or more non-transitory computer readable media of any one of claims 12 to 15, wherein the second component is a fluid pharmaceutical.

17. 16. The one or more non-transitory computer-readable media of claim 15, wherein the one or more characteristics of the first component include one or more of a glide force, a shield removal force, a barrel diameter, a break test result, a pin diameter, or a pin weight, and the one or more characteristics of the second component include one or more of a viscosity, a density, or a protein concentration.

18. the combination device is an automatic injector; the first component is a pre-filled syringe; The one or more non-transitory computer readable media of any one of claims 12 to 15, wherein the second component is an automatic injector subassembly.

19. 20. The one or more non-transitory computer-readable media of claim 18, wherein the one or more characteristics of the first component include one or more of an extrusion force, a sliding yield stress, a protein concentration, or a particle property, and the one or more characteristics of the second component include one or more of a specification number, a saline release test result, an injection time, an actuation force, or a spring force.

20. 20. The one or more non-transitory computer-readable media of any one of claims 16-19, wherein predicting the quality or outcome of the units of the combination device comprises at least one of: (a) predicting a value indicative of a variability in injection time or actuation force across the units of the combination device; (b) predicting a probability distribution of the characteristics of the combination device across the units of the combination device; or (c) predicting the existence, amount, frequency, or likelihood of a user complaint associated with the units of the combination device.

Images