Joint application of design of experiments (DOE) and machine learning (ML) in a design space

Abstract

Disclosed herein are methods and systems for the optimization of target-guided machine learning (ML) and Design of Experiments (DOE). More particularly, some embodiments focus on ML / DOE for an adhesive material design space, although the disclosure is not intended to be limited to this particular field of use. In some embodiments, an active learning ML / DOE process is incorporated with target-guided consideration. For example, a space-filling DOE approach assisted with ML, e.g., SVM-based ML, may be used to identify a design space for adhesive materials. The disclosed techniques have shown a 2x increase in efficiency over using DOE alone. The target-guided process may involve: 1) augmenting space filling design (SFD) ranges; 2) narrowing ML model prediction ranges; and 3) selecting validation experimental runs. Refining the design space in a target-guided fashion may also help to eliminate the inclusion of "outlier" runs in the modeling process and improve predictive capabilities for small datasets.

Description

Attorney Docket No.: 2024P00238WO_shlJOINT APPLICATION OF DESIGN OF EXPERIMENTS (DOE) AND MACHINE LEARNING (ML) IN A DESIGN SPACETechnical Field

[0001] This disclosure is related generally to methods and systems for the optimization of target-guided machine learning (ML) and Design of Experiments (DOE). More particularly, some embodiments focus on ML / DOE for an adhesive material design space, although the disclosure is not intended to be limited to this particular field of use.Background

[0002] Design of Experiments (DOE) is a systematic statistical approach for planning, conducting, and analyzing experiments to understand how multiple input variables (i.e., “factors” or “parameters,” as will be used herein) affect an output variable (i.e., a “response”). By testing various parameter levels simultaneously, DOE aims to provide efficient identification of significant factors and their interactions, leading to the optimization of processes, products, and systems with minimal effort and cost.

[0003] DOE can help to uncover how various factors interact with each other, i.e., determining the effect or dependence that one factor may have on the level of another factor. Typically, a DOE plan is created, defining the factors, their levels, and the experimental runs needed to evaluate their effects. Experiments are then performed according to the DOE plan, with multiple factors often being changed simultaneously to efficiently explore the system’s behavior over the experimentation design space.

[0004] The experimental data that is produced by the various experimental runs may then be statistically analyzed to determine which factors have a significant impact on the response variable and to understand the relationships and interactions between them. The results may then be interpreted to identify the optimal settings for the input factors to achieve a desired outcome, meet a set of desired experimental design criteria, or to solve a product design problem.

[0005] DOE is typically more efficient than a “one-factor-at-a-time” (OF AT) approach because it investigates multiple factors and their interactions simultaneously, saving time and resources. DOE can also provide a deeper understanding of a process or system by revealing the cause-and-effect relationships and the interactions between different factors.

[0006] However, if the experimental run points within a design space for a given DOE are chosen randomly — or if they are chosen according to an inefficient (or unintelligent) spaceAttorney Docket No.: 2024P00238WO_shl filling design (SFD) approach — there may still be unnecessary inefficiencies in the DOE process, thereby requiring additional experimental runs within the design space to identify a required number of sets of parameter / factor values that satisfy all of the experimental design criteria for a given DOE.

[0007] Accordingly, there is a need for a more efficient, intelligent, and “target-guided” approaches to refining the experimentation design space for a given DOE, thereby addressing the shortcomings of the existing DOE systems and solutions — and especially in the space of adhesive material design.Summary

[0008] According to some embodiments, a method of performing a targeted Design of Experiments (DOE) is disclosed herein, the method comprising the following operations: (a) determining a space filling design (SFD) approach for a first parameter experimentation design space, wherein the first parameter experimentation design space comprises a dimension for each of a first plurality of parameters; (b) obtaining data produced by a first number of experiments over the first parameter experimentation design space according to the determined SFD approach; (c) performing a machine learning (ML) operation on data produced by the first number of experiments to generate a model for the first parameter experimentation design space; (d) refining the first parameter experimentation design space based, at least in part, on the portion of the first parameter experimentation design space where the model predicts that each of the first plurality of parameters meets a plurality of experimental design criteria; repeating steps (a)-(d) at least one time for the refined first parameter experimentation design space; and identifying, from the refined first parameter experimentation design space, at least one set of values, wherein each value in a set of values comprises a value for one of the first plurality of parameters, and wherein each set of values meets the plurality of experimental design criteria.

[0009] According to some such embodiments, the determined SFD approach comprises a Latin hypercube sampling (LHS). Other exemplary SFD approaches may include: uniform, minimum potential, maximum entropy, Gaussian Process Optimal, sphere packing, and / or Fast Flexible.

[0010] According to other such embodiments, performing the ML operation comprises performing a regression operation. Exemplary non-linear regression operations may include: exponential, logistic, power, and / or other forms of polynomial regressions. Other examples ofAttorney Docket No.: 2024P00238WO_shl regression operations may include: linear regression operations, or generalized regression operations (e.g., which add a penalty term to a linear regression operation).

[0011] According to other such embodiments, performing the ML operation comprises using a support vector machine (SVM). Other exemplary ML operations may include: Artificial Neural Networks (ANNs), Random Forests, k-Nearest Neighbors (KNN), Logistic Regression, generalized regression, partial least squares, ordinary least squares, discriminant analysis, decision tree (i.e., partition) analysis, and / or other ensemble methods, like Gradient Boosting.

[0012] According to other such embodiments, refining the first parameter experimentation design space further comprises: generating, based on the generated model, a contour plot over the first parameter experimentation design space for each of the first plurality of parameters. For example, the common overlpping regions in the first parameter experimentation design space where each of the first plurality of parameters indicates that the plurality of experimental design criteria (also referred to herein as “responses”) are met may be used to generate the bounds of the refined first parameter experimentation design space referred to in (d), above.

[0013] According to other such embodiments, repeating steps (a)-(d) at least one time for the refined first parameter experimentation design space comprises: repeating steps (a)-(d) at least three times. For example, performing additional iterations of refinement on the first paramter experimentaiton design space may increase the chances (and numbers) of identifying a set of values (wherein each value in a set corresponds to one of the first plurality of parameters) that would meet each of the plurality of experimental design criteria, i.e., as compared to a DOE approach that does not use the targeted guidance and refinement techniques of the present Application.

[0014] According to other such embodiments, the operations further comprise: comparing results of modeling from two or more different ML algorithms; and selecting, based on the comparing, an ML algorithm to use to perform predictive modeling over the first parameter experimentation design space (e.g., selecting the ML algorithm having the highest R-squared, lowest Root Average Square Error (RASE), or other associated model prediction accuracy metric).

[0015] It is to be understood that the various exemplary embodiments described above in this Section could be implemented as a method, embodied in the form of non-transitory computer readable instructions, and / or performed by an apparatus or system, depending on the needs of a given implementation.Attorney Docket No.: 2024P00238WO_shl Description of the Drawings

[0016] The present application is further understood when read in conjunction with the appended drawings. For the purpose of illustrating the subject matter, there are shown in the drawings exemplary embodiments of the subject matter; however, the presently disclosed subject matter is not limited to the specific methods, devices, and systems disclosed. In the drawings:

[0017] Fig. 1 illustrates an exemplary system for a target-guided, ML-based approach to DOE, according to the present disclosure.

[0018] Fig. 2 illustrates exemplary contour profiles for parameters in an experimentation design space, according to the present disclosure.

[0019] Fig. 3 illustrates a plot of an exemplary adhesive design space and an exemplary adhesion versus flow output graph, according to the present disclosure.

[0020] Fig. 4 illustrates an exemplary flow chart of a method of performing optimized, target- guided machine learning (ML) and Design of Experiments (DOE), according to the present disclosure.

[0021] Fig. 5 illustrates, in block diagram form, an exemplary simplified multifunctional electronic device, according to the present disclosure.

[0022] Aspects of the disclosure will now be described in detail with reference to the drawings, wherein like reference numbers refer to like elements throughout, unless specified otherwise.Detailed Description of Illustrative Embodiments

[0023] Disclosed herein are methods and systems for the optimization of target-guided machine learning (ML) and Design of Experiments (DOE). More particularly, some embodiments focus on ML / DOE for an adhesive material design space, although the disclosure is not intended to be limited to this particular field of use. In some embodiments, an active learning ML / DOE process is incorporated with target-guided consideration. For example, a space-filling DOE approach assisted with ML, e.g., SVM-based ML, may be used to identify a design space for adhesive materials. The disclosed techniques have shown a 2x increase in efficiency over using DOE alone. The target-guided process may involve: 1) augmenting space filling design (SFD) ranges (which can help resolve ambiguities that result from a single design); 2) narrowing ML model prediction ranges; and 3) selecting validation experimental runs. Refining the design space in a target-guided fashion may also help eliminate the inclusion of “outlier” runs in the modeling process and improve predictive capabilities for small datasets.Attorney Docket No.: 2024P00238WO_shl

[0024] A Target-Guided, ML-based Approach to Design of Experiments (DOE)

[0025] Fig. 1 illustrates an exemplary system 100 for a target-guided, ML-based approach to DOE, according to the present disclosure. As shown in Fig. 1, according to some embodiments disclosed herein, an active learning process may be utilized to identify (and refine) a parameter experimentation design space over time. First, at 102, the system may obtain (or make) an initial guess or estimate as to the overall bounds of the parameter experimentation design space. These bounds may be based on, e.g., prior experimentation, scientific knowledge / research, the understanding of the entities controlling the experimental design, etc. Preferably, the initial design space guess should be large enough to cover all likely ranges of all parameter values that are a part of the experimental design. If an initial guess is too constrained, it’s possible that optimal design parameter values may never be identified, i.e., due to never being considered as part of the design space.

[0026] Next, at 104, a space filling design (SFD) module, e.g., in the form of computer code or routines executing on an electronic device comprising one or more processors, may select both an initial number of experiments (e.g., 2 runs, 3 runs, 5 runs, 8, runs, etc.) 105 and where to place those experiments within the parameter experimentation design space. For example, one approach is the so-called Latin hypercube sampling, which attempts to generate a random or near-random sampling of parameter values from a multidimensional distribution. Other SFD approaches may include: uniform, minimum potential, maximum entropy, Gaussian Process Optimal, sphere packing, and / or Fast Flexible. (Although the examples described herein are largely two dimensional, i.e., focusing on a hypothetical “Additive #1” and a hypothetical “Additive #2,” it is to be understood that the techniques disclosed herein generalize to any multidimensional distribution, e.g., a design space with 3, 4, or 5 or more parameters, etc.

[0027] Next, at 106, the initial number of experiments (e.g., 2 runs, 3 runs, 5 runs, 8, runs, etc.) determined at 105 may be performed, e.g., using parameters as determined by the SFD approach implemented by module 104.

[0028] Next, at 108, based, at least in part, on the results of experiments 106, a ML / SVM modeling module may be utilized to generate a model for the first parameter experimentation design space. As described above, in some implementations, a suitable non-linear regression operation may be employed to model the data (whereas traditional DOE typically uses linear regressions).

[0029] Next, at block 110, using the predictive power from the ML model generated by ML / SVM modeling module 108, an overlapping contour plot may be generated over theAttorney Docket No.: 2024P00238WO_shl experimental design space for each experimental design criteria. Contour plots may provide an intuitive way to visualize or compute ranges of parameter values where multiple design criteria are predicted to be satisfied simultaneously. Exemplary contour profile plots are shown and described in greater detail below with reference to Fig. 2.

[0030] Next, at 112, the design space may be refined, e.g., based on the aforementioned regions of the contour plots where the parameter values are predicted to satisfy multiple design criteria simultaneously. Exemplary refinement of a parameter experimentation design space is shown and described in greater detail below with reference to graph 230 of Fig. 2.

[0031] Finally, the active learning process 100 may return to block 104 to complete a desired number of iterative refinement processes. According to some implementations, at least 3 iterations of design space refinement may be performed in order to identify one or more sets of parameter values that satisfy all experimental design criteria. Of course, the number of iterative design space refinement operations performed may vary based on the needs of a given DOE.

[0032] Target-guided predictive modeling block 150 shows a detailed view of an exemplary final predictive modeling process that can be performed once a sufficient amount of experimental data has been obtained. According to some implementations, a target-guided predictive modeling process 150 may begin at 152 by defining the modeling design space. Next, a set of validation rules 154 may be defined for the modeling process, e.g., to avoid overfitting of any model. Next, at block 156, a comparison of a number of ML models may be conducted, e.g., to see which ML model(s) have the best predictive accuracy and / or an acceptable blend of predictive accuracy and processing intensiveness, etc. for a given response. According to some such embodiments, multiple ML algorithms may be used to create models on the same training set. The prediction of each ML model on designated validation runs may then be compared with the respective validation run’s experimental value. The difference between the prediction and the experimental observation is called the prediction error (or, alternatively, the residual, or the bias, etc.). The ML model that exhibits the least amount of prediction error may then be selected as a predictive model for a subsequent parameter optimization process. According to some implementations, each parameter for which a response is predicted may have a distinct predictive ML model. Thus, in the case of two or more responses (e.g., flow index and adhesion, as in the examples described herein), multiple models may be optimized (i.e., so-called “multiple response optimization,” or “MRO”). MRO may be based, at least in part, on an importance or priority level assigned to each response in the experiment and / or any other optimization criteria that may be specified. Model predictionAttorney Docket No.: 2024P00238WO_shl accuracy metrics used in such a comparison process may include R-squared values, Root Average Square Error (RASE) values, and / or other associated model prediction accuracy metrics as desired for a given implementation. Finally, at block 158 a predictive design space may be constructed, based on the selected / winning ML model from block 156.

[0033] Exemplary Contour Profiles

[0034] Fig. 2 illustrates exemplary contour profiles 200 for parameters in an experimentation design space, according to the present disclosure. As shown in contour graphs 210 / 220 / 230, the horizontal axes of the graphs correspond to a value for Additive #1, while the vertical axes of the graphs correspond to a value for Additive #2. In this example, the two experimental design criteria relate to a flow index value (see graph 210) and an adhesion value (see graph 220), although it is to be understood that these two experimental design criteria are merely exemplary, and more (or fewer) design criteria could be used in a given DOE process.

[0035] Turning first to contour graph 210, a flow index contour profile graph for the exemplary DOE is shown. The lines in contour graph 210 represent combinations of Additive #1 and Additive #2 values that lead to a particular flow index value. For example, all the points along the flow index contour labeled 2.5 reflect combinations of Additive #1 and Additive #2 values that lead to a flow index value of 2.5 in the produced adhesive material, while all the points along the flow index contour labeled 1.5 reflect combinations of Additive #1 and Additive #2 values that lead to a flow index value of 1.5 in the produced adhesive material. In this example, 2.0 represents a hypothetical flow index criterion value, values above which (i.e., shaded areas 215B of contour graph 210) do not meet the flow index criterion, and values below which (i.e., non-shaded areas 215 A of contour graph 210) do meet the flow index criterion.

[0036] Turning next to contour graph 220, an adhesion contour profile graph for the exemplary DOE is shown. The lines in contour graph 220 represent combinations of Additive #1 and Additive #2 values that lead to a particular adhesion value (e.g., in terms of pounds per square inch (PSI) or megapascals (MPa)). For example, all the points along the adhesion contour labeled 2.5 reflect combinations of Additive #1 and Additive #2 values that lead to an adhesion value of 2.5 in the produced adhesive material, while all the points along the adhesion value contour labeled 17.5 reflect combinations of Additive #1 and Additive #2 values that lead to an adhesion value of 17.5 in the produced adhesive material. In this example, 10.8 represents a hypothetical adhesion value criterion value, values below which (i.e., shaded areas 225B of contour graph 220) do not meet the adhesion criterion, and values above which (i.e., nonshaded areas 225 A of contour graph 220) do meet the adhesion criterion.Attorney Docket No.: 2024P00238WO_shl

[0037] Turning next to overlaid contour graph 230, an overlay of the flow index contour graph 210 and the adhesion contour graph 220 is shown. As may now be appreciated, when aligned and overlaid, only the areas of the design space that would remain unshaded in both graphs 210 and 220 (i.e., representing the intersection of non-shaded areas 215A and 225A) will remain unshaded in overlaid contour graph 230. This region, i.e., the portion of the parameter experimentation design space where the flow-related criterion and the adhesion-related criterion are both predicted to be met, is illustrated as thickly-outlined intersectional region 240 in overlaid contour graph 230.

[0038] According to some implementations, a region bounding the intersectional region 240 may be defined for each refinement iteration of the targeted DOE process. In the example of Fig. 2, dashed line bounding box 235 serves as the exemplary bounding region for the intersectional region 240. As may now be appreciated, bounding box 235 may be defined to span from the lowest Additive #1 value satisfying the flow-related criterion and the adhesion- related criterion (i.e., about 0.15) to the highest Additive #1 value satisfying the flow-related criterion and the adhesion-related criterion (i.e., about 0.31) in the horizontal axial direction. Similarly, bounding box 235 may be defined to span from the lowest Additive #2 value satisfying the flow-related criterion and the adhesion-related criterion (i.e., about 0.5) to the highest Additive #2 value satisfying the flow-related criterion and the adhesion-related criterion (i.e., about 1.45) in the vertical axial direction. It is to be understood that other shapes for the bounding box 235 may also be possible (e.g., more complex polygonal or multidimensional shapes), and that rectangular bounding box 235 is shown for ease of illustration.

[0039] As described herein, bounding box 235 represents a “refined” first parameter experimentation design space that could be used in the next iteration of a target-guided DOE process, i.e., to narrow down the experimental region in which the next determined number of experimental runs are drawn from. With an intelligently refined parameter experimentation design space and ML model design, it is expected that a greater percentage of subsequent experimental runs will meet all the experimental design criteria, leading to both better efficiency in identifying the experimentation design space — and better precision in the predictive modeling.

[0040] Exemplary Adhesive Design Space Graph and Exemplary Adhesion-versus-Flow Output Graph

[0041] Fig. 3 illustrates a plot of an exemplary adhesive design space graph 350 and an exemplary adhesion versus flow output graph 300, according to the present disclosure.Attorney Docket No.: 2024P00238WO_shl

[0042] Turning first to exemplary adhesive design space graph 350, the horizontal axis of the graph again corresponds to a value for Additive #1, while the vertical axis of the graph corresponds to a value for Additive #2. The five plotted points labeled: 1.1, 1.2, 1.3, 1.4, and 1.5 represent a first “phase” or iteration of the space filling design process for determining the parameters of the experimental runs. As shown, the five plotted points: 1.1, 1.2, 1.3, 1.4, and 1.5 form the largest “ring” within the experimentation design space, as they are based on the first / initial guess of the design space, which is preferably set initially to the largest reasonably- likely design space where successful values are likely to be found, so as to avoid missing potentially-successful areas of the universe of design space values. In the example of Fig. 3, the filled-in plotted points represent combinations of Additive #1 and Additive #2 values where the adhesive material met all the experimental design criteria (in this exemplary experiment, related to flow index and adhesion), and the non-filled plotted points represent combinations of Additive #1 and Additive #2 values where the adhesive material did not meet all the experimental design criteria. Notably, only experiments run 1.4 (i.e., element numeral 355) from the first phase met all the experimental design criteria.

[0043] Based on the ML modeling performed during the first phase of experimentation, a second set of five plotted points labeled: 2.1, 2.2, 2.3, 2.4, and 2.5 represent a second phase or iteration of the space filling design process for determining the parameters of the experimental runs. As shown, the five plotted points: 2.1, 2.2, 2.3, 2.4, and 2.5 form a slightly tighter “ring” within the experimentation design space than the five plotted points 1 x from the first phase of experimentation, as they are based on the ML modeling from the first phase. As expected, the plotted points from the second phase are somewhat evenly distributed around the successful1.4 experimental run from the first phase. In the second phase, both experimental runs 2.3 and2.4 (i.e., element numeral 360) from the second phase met all the experimental design criteria, meaning that 3 cumulative sets of successful criteria-meeting parameter values for Additive #1 and Additive #2 have already been identified across the first two phases of experimental runs.

[0044] Finally, and again based on the ML modeling performed during the second phase of experimentation, a third set of five plotted points labeled: 3.1, 3.2, 3.3, 3.4, and 3.5 represent a third phase or iteration of the space filling design process for determining the parameters of the experimental runs. As shown, the five plotted points: 3.1, 3.2, 3.3, 3.4, and 3.5 form an even tighter “ring” within the experimentation design space than the five plotted points 2.x from the second phase of experimentation, as they are based on the ML modeling from the second phase. As expected, the plotted points from the third phase are somewhat evenly distributed aroundAttorney Docket No.: 2024P00238WO_shl the successful 1.4, 2.3, and 2.4 experimental runs from the first and second phases. In the third phase, both experimental runs 3.1 and 3.3 (i.e., element numeral 365) from the third phase met all the experimental design criteria, meaning that 5 cumulative sets of successful criteria- meeting parameter values for Additive #1 and Additive #2 have been identified across the first three phases, totaling just 15 experimental runs.

[0045] Turning next to exemplary adhesion versus flow output graph 300, it may be seen that the same fifteen experimental run plotted points from exemplary adhesive design space graph 350 are shown in adhesion versus flow output space (i.e., a so-called “performance space”). In graph 300, the horizontal axis of the graph corresponds to a value for flow index, while the vertical axis of the graph corresponds to a value for adhesion. As mentioned above, the experimental design criteria in this example are met for flow index values less than 2.0 and adhesion values greater than 10.8. In other words, plotted points appearing both above adhesion criterion line 305 and to the left of flow criterion line 310 represent combinations of Additive #1 and Additive #2 values that resulted in an adhesive material meeting all of the experimental design criteria. In this case, those plotted points (representing experimental runs: 1.4, 2.3, 2.4, 3.1, and 3.3) are enclosed by dashed line ring 315. The other plotted points on graph 300 e.g., those enclosed by dashed line ring 320 represent combinations of Additive #1 and Additive #2 values that resulted in an adhesive material that did not meet all of the experimental design criteria. In fact, points in the lower-right quadrant of graph 300 in this example (i.e., experimental runs 1.2 and 1.3) represent combinations of Additive #1 and Additive #2 values that did not meet any of the experimental design criteria. This is somewhat to be expected as the 1 x experimental runs from the first phase represent a time before the initial experimental design space had been refined, e.g., according to the various techniques described herein. According to some embodiments, eliminating runs that do not meet any of the design criteria (e.g., experimental runs 1.2 and 1.3, in this example) at the predictive modeling stage (as described above with reference to block 150), i.e., as opposed to at the data preprocessing stage, may be advantageous, e.g., so that the model space will not need to get too wide (or cover regions of the design space that cannot meet any of the design criteria).

[0046] Exemplary Methods

[0047] Fig. 4 illustrates a method of performing optimized, target-guided machine learning (ML) and Design of Experiments (DOE), according to the present disclosure. At step 402 of method 400, the joint ML / DOE system can begin by determining a space filling design (SFD) approach to use for a first parameter experimentation design space, wherein the first parameterAttorney Docket No.: 2024P00238WO_shl experimentation design space comprises a dimension for each of a first plurality of parameters. According to some such embodiments, the determined SFD approach comprises a Latin hypercube sampling (LHS). Other exemplary SFD approaches may include: uniform, minimum potential, maximum entropy, Gaussian Process Optimal, sphere packing, and / or Fast Flexible.

[0048] At step 404 of method 400, the joint ML / DOE system can obtain data produced by a first number of experiments over the first parameter experimentation design space according to the determined SFD approach. According to some implementations, the first number of experiments could be performed by the same entity that is using the joint ML / DOE system. Alternatively, the data produced by the first number of experiments could be obtained from another entity, e.g., via a storage repository or online resource. According to some implementations, the number of experiments used with the determined SFD approach could be 2, 3, 5, 8, or more experiments.

[0049] At step 406 of method 400, the joint ML / DOE system can perform a machine learning (ML) operation on the data that is produced by the first number of experiments to generate a model for the first parameter experimentation design space. According to some implementations, performing the ML operation comprises using a support vector machine (SVM). Other exemplary ML operations may include: Artificial Neural Networks (ANNs), Random Forests, k-Nearest Neighbors (KNN), Logistic Regression, and / or other ensemble methods, like Gradient Boosting.

[0050] At step 408 of method 400, the joint ML / DOE system can optionally refine the first parameter experimentation design space based, at least in part, on the portion of the first parameter experimentation design space where the model predicts that each of the first plurality of parameters meets a plurality of experimental design criteria (e.g., as shown and described above with reference to Fig. 2, region 235). According to some such implementations, each refinement of the first parameter experimentation design space may reduce the design space in one or more dimensions to include only those ranges of values for the one or more corresponding parameters wherein the ML model predicts that all experimental design criteria would be met by the combination of parameters.

[0051] At step 410 of method 400, the joint ML / DOE system can repeat steps 402 / 404 / 406 at least one time for the refined first parameter experimentation design space. In other words, the refined (e.g., reduced in size) first parameter experimentation design space may then be used in a subsequent iteration to obtain data produced by another number of experiments based onAttorney Docket No.: 2024P00238WO_shl points in the refined first parameter experimentation design space that were selected according to the determined SFD approach. In theory, subsequent iterations of steps 402 / 404 / 406 / 408 should produce a higher and higher percentage of sets of values that meet all the experimental design criteria, since, with each iteration, the experiments target in on portions of the experimentation design space that have been predicted (via previous ML-based modeling) to be likely to produce successful parameter values. In some implementations, preferably at least 3 iterations of steps 402 / 404 / 406 / 408 may be performed before identifying one or more sets of values that meet all the experimental design criteria.

[0052] At step 412, if further parameter experimentation design space refinement is still desired (i.e., “YES” at step 412), the method 400 may continue iterating by returning to step 402 and using a determined SFD approach do select new experimentation points in the most recently-refined experimentation design space. If, instead, at step 412, no further parameter experimentation design space refinement is desired (i.e., “NO” at step 412), the method 400 may proceed to step 414. According to some embodiments, once data from a sufficient number of experimental runs has been obtained, a prediction modelling processing may be performed (e.g., as described above with reference to block 150). During this process, various ML models may be utilized and their respective prediction errors on designated experimental validation runs may be computed. According to some such embodiments, at the conclusion of the predictive modelling process, the ML model having the least prediction error for each response (e.g., one “best” predictive ML model for adhesion and one “best” predictive ML model for flow, in the examples described herein) may be selected to generate the prediction model(s) for subsequent optimization purposes.

[0053] Finally, at step 414, the joint ML / DOE system can identify, e.g., from the (most- recently) refined first parameter experimentation design space (and using the generated prediction model(s)), at least one set of values (wherein each value in a set of values comprises a value for one of the first plurality of parameters), wherein each set of values meets the plurality of experimental design criteria. According to some embodiments, a multiple response optimization (MRO) process may be run to determine one or more sets of values that meet the plurality of experimental design criteria, as well as applying any other optimization criteria that may be a part of the DOE. For example, in one implementation, it may be desirable to have one response (e.g., flow) meet at least a minimum threshold value, while maximizing another response (e.g., adhesion). After step 414 (and / or once a desired number of sets of valuesAttorney Docket No.: 2024P00238WO_shl meeting the experimental design criteria and any optimization criteria have been identified), the method 400 can end.

[0054] Exemplary Electronic Device

[0055] Referring now to Fig. 5, a simplified functional block diagram of an illustrative multifunctional electronic device 500, e.g., for use in a closed-loop adhesive control system according to various aspects of the disclosure, is shown. Multifunction electronic device 500 may include processor 510, memory 520, storage device 530, user interface 540, display 550, communications circuitry 560, and communications bus 570. Multifunction electronic device 500 may be, for example, a personal electronic device such as a personal digital assistant (PDA), mobile telephone, or a tablet computer.

[0056] Processor 510 may execute instructions necessary to carry out or control the operation of many functions performed by device 500. Processor 510 may, for instance, drive display 550 and receive user input from user interface 540. User interface 540 may allow a user to interact with device 500. For example, user interface 540 can take a variety of forms, such as a button, keypad, dial, a click wheel, keyboard, display screen and / or a touch screen. For example, electronic device 500 may be utilized by a user to access a web portal and customize, view, and / or analyze various parameters related to a closed-loop adhesive control system, as described in the various embodiments above.

[0057] Processor 510 may also, for example, be a system-on-chip such as those found in mobile devices and include a dedicated graphics processing unit (GPU). Processor 510 may be based on reduced instruction-set computer (RISC) or complex instruction-set computer (CISC) architectures or any other suitable architecture and may include one or more processing cores. Memory 520 may include one or more different types of media used by processor 510 to perform device functions. For example, memory 520 may include memory cache, read-only memory (ROM), and / or random access memory (RAM). Storage 530 may store media (e.g., audio, image and video files), computer program instructions or software, preference information, device profile information, and any other suitable data. Storage 530 may include one more non-transitory storage mediums including, for example, magnetic disks (fixed, floppy, and removable) and tape, optical media such as CD-ROMs and digital video disks (DVDs), and semiconductor memory devices such as Electrically Programmable Read-Only Memory (EPROM), and Electrically Erasable Programmable Read-Only Memory (EEPROM). Memory 520 and storage 530 may be used to tangibly retain computer program instructions or code organized into one or more modules and written in any desired computer programmingAttorney Docket No.: 2024P00238WO_shl language. When executed by, for example, processor 510 such computer program code may implement one or more of the methods described herein.

[0058] While systems and methods have been described in connection with the various embodiments of the various figures, it will be appreciated by those skilled in the art that changes could be made to the embodiments without departing from the broad inventive concept thereof. It is understood, therefore, that this disclosure is not limited to the particular embodiments disclosed, and it is intended to cover modifications within the spirit and scope of the present disclosure as defined by the claims.

Claims

Attorney Docket No.: 2024P00238WO_shlWhat is claimed is:

1. A method of performing a targeted Design of Experiments (DOE), comprising the following operations:(a) determining a space filling design (SFD) approach for a first parameter experimentation design space, wherein the first parameter experimentation design space comprises a dimension for each of a first plurality of parameters;(b) obtaining data produced by a first number of experiments over the first parameter experimentation design space according to the determined SFD approach;(c) performing a machine learning (ML) operation on data produced by the first number of experiments to generate a model for the first parameter experimentation design space;(d) refining the first parameter experimentation design space based, at least in part, on the portion of the first parameter experimentation design space where the model predicts that each of the first plurality of parameters meets a plurality of experimental design criteria; repeating steps (a)-(d) at least one time for the refined first parameter experimentation design space; and identifying, from the refined first parameter experimentation design space, at least one set of values, wherein each value in a set of values comprises a value for one of the first plurality of parameters, and wherein each set of values meets the plurality of experimental design criteria.

2. The method of claim 1, wherein the determined SFD approach comprises a Latin hypercube sampling (LHS).

3. The method of claim 1, wherein performing the ML operation comprises performing a non-linear regression operation.

4. The method of claim 1, wherein performing the ML operation comprises using a support vector machine (SVM).Attorney Docket No.: 2024P00238WO_shl5. The method of claim 1, wherein refining the first parameter experimentation design space comprises: generating, based on the generated model, a contour plot over the first parameter experimentation design space for each of the first plurality of parameters.

6. The method of claim 1, wherein repeating steps (a)-(d) at least one time for the refined first parameter experimentation design space comprises: repeating steps (a)-(d) at least three times.

7. The method of claim 1, further comprising: comparing results of modeling from two or more different ML algorithms; and selecting, based on the comparing, an ML algorithm to use to perform predictive modeling over the first parameter experimentation design space.

8. One or more non -transitory computer-readable media storing computer-executable instructions, which, when executed by one or more processors of an electronic device, cause the electronic device to perform a targeted Design of Experiments (DOE) method, comprising the following operations:(a) determining a space filling design (SFD) approach for a first parameter experimentation design space, wherein the first parameter experimentation design space comprises a dimension for each of a first plurality of parameters;(b) obtaining data produced by a first number of experiments over the first parameter experimentation design space according to the determined SFD approach;(c) performing a machine learning (ML) operation on data produced by the first number of experiments to generate a model for the first parameter experimentation design space;(d) refining the first parameter experimentation design space based, at least in part, on the portion of the first parameter experimentation design space where the model predicts that each of the first plurality of parameters meets a plurality of experimental design criteria; repeating steps (a)-(d) at least one time for the refined first parameter experimentation design space; andAttorney Docket No.: 2024P00238WO_shl identifying, from the refined first parameter experimentation design space, at least one set of values, wherein each value in a set of values comprises a value for one of the first plurality of parameters, and wherein each set of values meets the plurality of experimental design criteria.

9. The one or more non-transitory computer-readable media of claim 8, wherein the determined SFD approach comprises a Latin hypercube sampling (LHS).

10. The one or more non-transitory computer-readable media of claim 8, wherein performing the ML operation comprises performing a non-linear regression operation.

11. The one or more non-transitory computer-readable media of claim 8, wherein performing the ML operation comprises using a support vector machine (SVM).

12. The one or more non-transitory computer-readable media of claim 8, wherein refining the first parameter experimentation design space comprises: generating, based on the generated model, a contour plot over the first parameter experimentation design space for each of the first plurality of parameters.

13. The one or more non-transitory computer-readable media of claim 8, wherein repeating steps (a)-(d) at least one time for the refined first parameter experimentation design space comprises: repeating steps (a)-(d) at least three times.

14. The one or more non-transitory computer-readable media of claim 8, wherein the operations further comprise: comparing results of modeling from two or more different ML algorithms; and selecting, based on the comparing, an ML algorithm to use to perform predictive modeling over the first parameter experimentation design space.

15. An electronic device, comprising: memory; one or more processors; andAttorney Docket No.: 2024P00238WO_shl one or more instructions stored in the memory, which, when executed by the one or more processors cause the electronic device to perform a targeted Design of Experiments (DOE) method, comprising the following operations:(a) determining a space filling design (SFD) approach for a first parameter experimentation design space, wherein the first parameter experimentation design space comprises a dimension for each of a first plurality of parameters;(b) obtaining data produced by a first number of experiments over the first parameter experimentation design space according to the determined SFD approach;(c) performing a machine learning (ML) operation on data produced by the first number of experiments to generate a model for the first parameter experimentation design space;(d) refining the first parameter experimentation design space based, at least in part, on the portion of the first parameter experimentation design space where the model predicts that each of the first plurality of parameters meets a plurality of experimental design criteria; repeating steps (a)-(d) at least one time for the refined first parameter experimentation design space; and identifying, from the refined first parameter experimentation design space, at least one set of values, wherein each value in a set of values comprises a value for one of the first plurality of parameters, and wherein each set of values meets the plurality of experimental design criteria.

16. The electronic device of claim 15, wherein the determined SFD approach comprises a Latin hypercube sampling (LHS).

17. The electronic device of claim 15, wherein performing the ML operation comprises performing a non-linear regression operation.

18. The electronic device of claim 15, wherein performing the ML operation comprises using a support vector machine (SVM).Attorney Docket No.: 2024P00238WO_shl19. The electronic device of claim 15, wherein refining the first parameter experimentation design space comprises: generating, based on the generated model, a contour plot over the first parameter experimentation design space for each of the first plurality of parameters.

20. The electronic device of claim 15, wherein the operations further comprise: comparing results of modeling from two or more different ML algorithms; and selecting, based on the comparing, an ML algorithm to use to perform predictive modeling over the first parameter experimentation design space.

Images