Intelligent selection of inputs for fast reasoning and quantization uncertainty for active learning

Abstract

A system for inference of active learning in a model and intelligent selection of inputs to quantify uncertainty includes calculating an information gain by combining one or more indicators of interest and uncertainty in the input using information compression based on a first set of data of one or more pairs of values as the input and output. One or more inputs having a higher potential information gain than the other inputs are selected to query the model and record values for one or more outputs. The model calculates measurements of the performance output based on the first set of data. The model is updated based on the estimated metric of performance of interest and the quantization of the associated uncertainty, and the model is output to alter the process based on the estimated metric of performance of interest and the quantization of the associated uncertainty.

Description

Technical Field

[0001] The present disclosure relates to selecting inputs for active learning. In particular, the present disclosure relates to a system and method for fast inference of arbitrary statistics of expensive functions using active learning and intelligent selection of inputs with quantified uncertainty. Background Art

[0002] Many applications utilize performance estimation systems to predict part life based on usage. This allows for an estimate of part failure. Typical measurement systems require extensive information and expensive experiments to generate these estimates. BRIEF DESCRIPTION OF THE DRAWINGS

[0003] The illustrative embodiments may take the form of various components and arrangements of components. Illustrative embodiments are shown in the accompanying drawings, and like reference numerals may indicate corresponding or similar parts throughout the various figures. The drawings are intended only to illustrate the embodiments and are not to be construed as limiting the present disclosure. The novel aspects of the present disclosure should be apparent to one of ordinary skill in the relevant art in view of the following enabling description of the drawings.

[0004] Figure 1 A system according to an exemplary embodiment is shown.

[0005] Figure 2 According to one embodiment Figure 1 A block diagram of the equipment of the system;

[0006] Figure 3 is a flowchart of a machine learning network method according to one embodiment; and

[0007] Figure 4 is a flowchart of a method for performing fast inference and quantization according to one embodiment. DETAILED DESCRIPTION

[0008] Before discussing the drawings of the present disclosure in detail, several terms and concepts are covered to facilitate a detailed description of the exemplary embodiments depicted in the drawings. However, it should be noted that these terms and concepts do not limit the present disclosure, and a person of ordinary skill in the relevant art will readily recognize modifications and variations applicable to the following auxiliary systems and / or concepts, which do not limit the scope of the present teachings.

[0009] In order to predict component failures, such as turbine failures, wing failures, etc., it may be necessary to analyze various statistical data. In some example embodiments, the statistical data that need to be learned and analyzed may include statistics of physical quantities (such as aerodynamic efficiency, maximum principal stress, spalling area, etc.), which can be modeled as complex and expensive computer models or physical experiments with real-valued parameters or inputs while using a limited number of data points.

[0010] Traditional methods for inferring the statistics of interest when performing predictions involve expensive experiments, potentially thousands of experimental runs, and computer code that includes statistical uncertainties (e.g., manufacturing tolerances in the inputs or noise in the output measurements). These uncertainties can affect the inferred value of the statistics. Therefore, by accurately quantifying the input uncertainties of machine learning models, such as manufacturing tolerances and / or design uncertainties, it is possible to obtain less expensive and more productive data for inferring metrics of interest, such as statistical expectations, tail probabilities, extreme values of the underlying physical response, etc.

[0011] Figure 1 1 shows a system 100 according to an exemplary embodiment. The system 100 includes a device 200, which will be described below. Figure 2 As described in Figure 1 As shown, the intelligent, probabilistic, and dynamic selection of experiments includes maximizing information gain, introducing input uncertainty and estimating a metric of interest (MOI), and quantifying the uncertainty in these estimates. In various embodiments, data points (e.g., old points and new points) can be utilized and input into the apparatus 200. In various embodiments, the information gain can include a measure of how much entropy is reduced when a data set is segmented, for example, by comparing the entropy of a data set before and after augmentation of the data set.

[0012] In various embodiments, the system 100 described herein can be applied to any expensive experiment, such as combustion tests that measure outputs such as temperature and pressure, and input parameters such as fuel combustion and flow rate, as well as situations where physical processes are modeled as complex, computationally expensive computational models. In various embodiments, for example, the design of turbine blades using computational models can be used. For example, the system 100 can be applied to the design of high-pressure turbine blades (airfoil design) using computational fluid dynamics models.

[0013] The high-pressure turbine (HPT) of a jet engine consists of several stages of blades (also called airfoils). In order to achieve key values of key indicators such as the maximum value of pressure distribution and the tail value of rated efficiency, the design of the shape of these blades may be an important indicator to help the blade design.

[0014] In various embodiments, the shape of a blade can be mapped to a finite number of parameters, such as the height of the airfoil, the width of the airfoil, and so on. The underlying complex physics associated with blade design can be represented by complex mathematical equations. For example, engineers and scientists might program these complex methods using computer codes that accurately simulate the underlying physical processes, albeit at a very high computational cost. For example, running a simulation using methods such as large eddy simulation can require several days of computation.

[0015] To study the effects of shape parameters on pressure distribution and efficiency, and to obtain key metrics such as the maximum or tail values of these outputs, it may be necessary to query the code across hundreds or thousands of parameter combinations. Therefore, even parallel computing may not allow for a full exploration of the input space to determine the parameters that best suit the designer's requirements.

[0016] Furthermore, there may be uncertainty in the values of input parameters. In various embodiments, this uncertainty can be characterized by upper and lower bounds on the nominal values of the parameters. In a physical sense, the uncertainty around the nominal values represents unknown effects of environmental factors, defects in the manufacturing process, operational uncertainties, and the like.

[0017] Thus, in various example embodiments, the methods described herein may provide for optimal and efficient allocation of computing resources while aiming to learn optimal values for output metrics while incorporating and quantifying the impact of uncertainty in input parameters on the metrics.

[0018] Furthermore, the methods and systems described herein can be used in any computer-aided analysis (CFD, FEA, etc.) in which the optimal design is obtained along candidates for equally optimal solutions for extreme tail probability values of maximum stress and / or other outputs of the FEA, including uncertainties within a reasonable amount of CPU time (and therefore is tolerable).

[0019] Thus, in various embodiments, the methods and systems described herein may be applied to any robust design application where a metric such as the maximum or minimum value of a code output, such as stress, force, pressure, displacement, temperature, displacement, vibration, flow rate, heat transfer rate, life, etc., is of interest and where the inputs vary (e.g., CAE simulations).

[0020] In various embodiments, the device 200 can run experiments (e.g., training iterations) using a machine learning model to produce a probabilistic compilation of results that produces more accurate predictions in each iteration. Figure 1 As shown in the figure, a plot of MOI versus number of analyses shows that each iteration produces a more accurate estimate with less uncertainty. The machine learning model uses the learned data to improve its accuracy with each iteration.

[0021] In various embodiments, the input data may include design parameters and / or process parameters and previously available information thereon. The prior information may include manufacturing or operational uncertainties, and may also include material constants and / or stress distributions. In various embodiments, the MOI may include statistical data regarding performance and / or manufacturing (e.g., efficiency, weight variation, fuel consumption variation, etc.).

[0022] In various embodiments, a method for estimating key statistics of MOI (e.g., related to tail probability, probability of failure, quantity, reliability, risk, and uncertainty quantification) is described herein. In various embodiments, information gain is estimated in user-defined MOI multivariate statistics to intelligently and adaptively select design points to run experiments.

[0023] In various embodiments, data such as input uncertainty (e.g., manufacturing tolerances, noise measurements, etc.) are introduced and the uncertainty of the statistical data as the MOI is estimated to define an interval around the statistical inference value. The different MOIs inferred directly affect the selection of the operating point.

[0024] In various embodiments, input uncertainties may include airfoil height, manufacturing tolerances, and / or airfoil width, where turbine blade design improvements are desired. In various embodiments, MOIs associated with calculating blade design examples may include statistical data from code / experimental outputs. MOIs may include tail probability, maximum stress value, mean stress value, and / or variance of blade efficiency. In various embodiments, selecting an input query sequence facilitates determining accurate values for these MOIs. In various embodiments, the MOIs may be user-defined.

[0025] Figure 2 According to one embodiment Figure 1 Now refer to the block diagram of the equipment of the system. Figure 2 Exemplary components in the device 200 according to various aspects of the present disclosure include, for example, a database 210, one or more processors 220, at least one memory 230, and a network interface 240. In certain aspects, the device 200 may include a graphics processing unit (GPU) 250, which can be used to process machine learning network models.

[0026] The database 210 may be located in a storage device. The term "storage device" may refer to any device or material from which information can be accessed, reproduced, and / or stored in electromagnetic or optical form for access by a computer processor. For example, a storage device may be a volatile memory such as RAM, a non-volatile memory such as flash memory that permanently stores digital data until it is intentionally erased, a magnetic device such as a hard drive, and a CD, DVD, Blu-ray Disc TM etc. optical media, and so on.

[0027] In various aspects, data can be stored on device 200, including, for example, user preferences, historical data, and / or other data. The data can be stored in database 210 and sent to processor 220 via a system bus.

[0028] As will be described in greater detail later herein, the processor 220 performs various processes based on instructions that may be stored in the server memory 230 and utilizing data from the database 210. Figure 1 , a request from a user device (such as a mobile device or a client computer) can be transmitted to the server through the server's network interface 240. Figure 2 The diagram is exemplary and those skilled in the art will understand that other components may exist in the controller 200. For the sake of clarity, Figure 2 These other components are not shown.

[0029] Figure 3 3 is a flow chart of a machine learning network method 300 according to one embodiment. In various embodiments, the method 300 may be executed / run multiple times to achieve a desired confidence level result. At block 310, a small initial design of experiments (DOE) is performed (e.g., a training iteration of machine learning).

[0030] In various embodiments, experimental runs may be performed at different values of, for example, airfoil height and / or airfoil width at block 310. Code output of efficiency and / or pressure values for the blade may be recorded.

[0031] In various embodiments, a run can represent data points (e.g., values of input parameters and corresponding values of outputs). The inputs for the initial run can be selected using a DOE (such as an Optimal Latin Hypercube (OLH)). In various embodiments, the initial number of runs can be user input. For example, the number of run points, airfoil height, and width values can be modified at each step / iteration.

[0032] In various embodiments, the input uncertainty can be user-defined. For example, it can represent the manufacturing tolerance around the nominal values of the airfoil height and width. This can be converted into a probability distribution, such as a normal or uniform probability distribution.

[0033] Once the initial run is complete, the initial data structure can be used to create a proxy model linking inputs to outputs.In various embodiments, an input or inputs for running the computational model can be identified that are most informative about the MOI.

[0034] The information gain is then calculated (block 320). In various embodiments, the information gain may be calculated according to the following equation:

[0035] [I(Q[f(x)])], equation (1),

[0036] Where I represents the value of the calculated information gain, Q is a statistic, which can be an operator such as expectation, tail probability, variance, etc., f(x) represents the basis function or physical process modeled by expensive simulation or experiment, and x represents the operation and design input of the system. In various embodiments, for example, f(x) can model a physical process with input variables x that generate output values f(x). When the input variables x vary, Q[] can provide statistics on the output values of the basis function f(x). In various embodiments, I() can represent the information gain based on the statistics provided by Q[].

[0037] As mentioned above, the information gain can be calculated in the MOI according to the above equation 1 as a function of the input parameter. The input parameter is denoted by x. To achieve this, a probabilistic surrogate model and its ability to quickly evaluate within the input space can be utilized to allow the calculation of a current estimate of the MOI.

[0038] The average value of information gain can be calculated using a probabilistic quantification of uncertainty in the input (e.g., described below at block 413), which can be approximated by calculating the difference between the prior and posterior probability distributions of the metric of interest. The information gain is calculated as a difference according to, for example, the equation shown below at block 414, further details of which are provided below. In various embodiments, the equation describes a probabilistic representation of all metrics of interest Q1 to Qm conditioned on the observed data set, and the observed data set, given by The value of the potential operating point represented by The probability representation of all indicators of interest Q1 to Qm conditioned on possible unobserved values of the performance output at the operating point is represented by In various embodiments, the operating point can be selected by optimizing or maximizing the value of the information gain formula.

[0039] In various embodiments, once the information gain is formulated / calculated, it can be optimized using any optimizer, such as gradient descent, BFGS, etc., to return the set of input candidates with the maximum information gain. One example method for maximizing information gain is to use a space-filling grid of points using an OLH-type DOE method. The information gain at each such candidate is evaluated, and the gain corresponding to the maximum information gain is selected.

[0040] When multiple points are selected for parallel execution, they can be selected by looking at the information gain values within them or separated by distance. The code can be run for five selected values of airfoil height and width, and the values of two outputs (e.g., efficiency and pressure) recorded. These units (e.g., outputs) can be added to the initial data, and the process repeated until the budget is exhausted or the value and uncertainty of the metric are acceptable, either by visual inspection or by meeting some threshold based on domain knowledge.

[0041] If the iteration provides the desired level of results for the stop criteria input (block 330), the method accepts the output (block 350). In various embodiments, for example, the operating point (input) query code obtains values for blade efficiency and / or pressure distribution.

[0042] Once the metric values (e.g., efficiency maximum, pressure tail, and uncertainty) are determined, they can be incorporated into downstream analyses / applications (e.g., airline customer pricing quotes around tail probability and / or blade efficiency). The uncertainty around the estimate is an indication of the confidence level around the model.

[0043] If the iterations do not provide the desired level of results for the stopping criteria input (block 330), then intelligently selected operating points (e.g., points expected to have high information gain) are added to the observed data, and the method returns to block 320. In block 340, EIG[I] represents the expected information gain and is an average approximation of the information gain I.

[0044] Figure 4 is a flow chart of a method 400 for performing fast inference and quantization according to one embodiment. In various embodiments, the method 400 may perform the Figure 3 In various embodiments, X n Can be related to the above Figure 3 The x described in n =f(x n )). In various embodiments, p(x) can be related to the distribution of values of x. In various embodiments, information gain can be related to Figure 3 The above-mentioned I in is related to, for example, the entropy difference calculated on the prior and posterior data. Therefore, in various examples, information gain calculates the reduction in entropy, providing information about which information gain is most correlated (e.g., the largest reduction in entropy).

[0045] In block 401, using initial data (e.g., D n ={X n ,y n}). In block 402, a distribution p(x) is created and manipulated using inputs, and in block 403, a quantity of interest (e.g., Q|f|:L2(X)→R) is utilized. In various embodiments, Q|f| is a statistical operator such as a statistical expectation, variance, tail probability, etc., and f is a real-valued function that accepts a vector-valued input x.

[0046] In various embodiments, D n represents the input value (X n ) and the corresponding performance output value (y n) data set. The letter n represents the number of such experiments / runs / iterations. Information from blocks 401, 402, and 403 is input into blocks 411, 412, and 413, respectively. In various embodiments, block 411 performs a probability function approximation. In various embodiments, the probability function approximation can be performed according to the following formula: p(Θ,φ|Dn) or p(f(x)|D n ), where p represents a probabilistic quantization of the parameters, such as a probability distribution. This probabilistic approximation allows analytical sampling of the underlying function.

[0047] At block 412 , the probabilistic quantification of the uncertainty in the input is used to compute an average of information gain at block 413 , which can be approximated by computing the difference between the prior and posterior probability distributions of the metric of interest.

[0048] At block 414, information gain is calculated as the difference according to, for example, the equation shown in block 414, which in various embodiments describes all metrics of interest Q1 to Q2 conditioned on the observed data set. m The probability representation of The value of the potential operating point represented by All the indicators of interest Q1 to Q2 are conditioned on the possible unobserved values of the performance output at this operating point. m In various embodiments, the operating point can be selected by optimizing or maximizing the value of the above information gain formula.

[0049] At block 426 , if sufficient calculations have been performed to meet the criteria for acceptable performance, the method may stop and continue to block 428 where a quantification of the uncertainty is determined and a final estimate of the MOI value is quantified.

[0050] If not enough calculations have been performed at block 426, the method continues to block 427 where the point with the maximum value of the information gain is selected and the method continues to block 429. In various embodiments, arg max represents the actual value of the operating point corresponding to the highest value of the "information gain" calculation I(x).

[0051] At block 429 , experiments / simulations are performed at the selected points from block 427 and data is collected to be fed back into block 411 for further analysis.

[0052] The aspects disclosed herein are examples of the present disclosure and can be embodied in various forms. For example, although some aspects herein are described as separate aspects, each aspect herein can be combined with one or more other aspects herein. The specific structural and functional details disclosed herein should not be interpreted as limitations, but rather as a basis for the claims and as a representative basis for teaching those skilled in the art to use the present disclosure in various ways in almost any appropriately detailed structure.

[0053] For example, in order to achieve key values of key indicators, such as the maximum value of pressure distribution and the tail value of rated efficiency, the design of the shape of the turbine blade may be an important indicator to help design the blade. For example, the above describes a system and method for determining these values for designing turbine blades.

[0054] According to the present disclosure, the phrases "in one aspect," "in aspects," "in various aspects," "in some aspects," or "in other aspects" may each refer to one or more of the same or different aspects.

[0055] Further aspects of the disclosure are provided by the subject matter of the following paragraphs.

[0056] A system for intelligent selection of inputs for inference and quantification of uncertainty for active learning in a model comprises: a processor; and a memory comprising instructions that, when executed by the processor, cause the system to at least: calculate information gain based on a first set of data comprising one or more pairs of values as input and output by combining one or more indicators of interest and uncertainties in the inputs using information compression; select one or more of the inputs having a potential information gain higher than other inputs to query the model and record the values of one or more of the outputs; calculate a measurement of the performance output based on the first set of data by the model; estimate an indicator of performance of interest and quantify the uncertainty associated with the estimate of the indicator of performance of interest; update the model based on the estimated indicator of performance of interest and the quantification of the associated uncertainty; and output the estimated indicator of performance of interest and the quantification of the associated uncertainty to change a process based on the estimated indicator of performance of interest and the quantification of the associated uncertainty.

[0057] A system as in any preceding clause, wherein the first set of data comprises manufacturing tolerances.

[0058] A system as in any preceding clause, wherein the manufacturing tolerance is an uncertainty associated with one or more input values.

[0059] A system as in any preceding clause, wherein the one or more outputs are results of a simulation.

[0060] A system as in any preceding clause, wherein the indicator of performance of interest comprises engine performance output.

[0061] A system as in any preceding clause, wherein the engine performance output comprises mechanical power of the engine.

[0062] A system as in any preceding clause, wherein the engine performance output comprises a maximum efficiency of the engine.

[0063] A system as in any preceding clause, wherein the first set of data comprises tabular data.

[0064] The system of any preceding clause, wherein the first set of data comprises design of experiments (DOE) points.

[0065] A system as claimed in any preceding clause, wherein the first set of data comprises measurements of engine performance output.

[0066] The system of any preceding clause, wherein the one or more indicators of interest comprise one or more of the outputs.

[0067] A processor-implemented method for fast inference of arbitrary statistics of expensive functions using active learning in a model and intelligent selection of inputs with quantified uncertainty, comprising: calculating information gain based on a first set of data having one or more pairs of values as inputs and outputs by combining one or more indicators of interest and uncertainties in the inputs using information compression; selecting one or more of the inputs having a potential information gain higher than other inputs to query the model and recording the values of one or more of the outputs; calculating a measurement of the performance output based on the first set of data by the model; estimating an indicator of performance of interest and quantifying the uncertainty associated with the estimate of the indicator of performance of interest; updating the model based on the estimated indicator of performance of interest and the quantification of the associated uncertainty; and outputting the estimated indicator of performance of interest and the quantification of the associated uncertainty to change a process based on the estimated indicator of performance of interest and the quantification of the associated uncertainty.

[0068] The processor-implemented method of any preceding clause, wherein the first set of data comprises manufacturing tolerances.

[0069] The processor-implemented method of any preceding clause, wherein the manufacturing tolerance is an uncertainty associated with one or more input values.

[0070] The processor-implemented method of any preceding clause, wherein the one or more outputs are results of a simulation.

[0071] A processor-implemented method as described in any preceding clause, wherein the indicator of performance of interest comprises engine performance output.

[0072] A processor-implemented method as described in any preceding clause, wherein the engine performance output comprises mechanical power of the engine.

[0073] A processor implemented method as described in any preceding clause, wherein the engine performance output comprises a maximum efficiency of the engine.

[0074] A processor-implemented method as described in any preceding clause, wherein the first set of data comprises tabular data.

[0075] The processor-implemented method of any preceding clause, wherein the first set of data comprises design of experiments (DOE) points.

[0076] Those skilled in the relevant art will appreciate that various adaptations and modifications of the above embodiments may be constructed without departing from the scope and spirit of the present disclosure. Therefore, it should be understood that within the scope of the appended claims, the teachings described herein may be practiced beyond the specific description herein.

Claims

1. A system for intelligent selection of inputs for active learning inference and quantified uncertainty in models, characterized in that The system comprises: processor; a memory comprising instructions that, when executed by the processor, cause the system to at least: Based on a first set of data comprising one or more pairs of values as input and output, calculating information gain by combining one or more metrics of interest and uncertainties in the input using information compression; selecting one or more of the inputs having a higher potential information gain than other inputs to query the model and recording the value of one or more of the outputs; calculating, by the model, a measure of a performance output based on the first set of data; estimating a measure of performance of interest and quantifying uncertainty associated with the estimate of the measure of performance of interest; updating the model based on the estimated indicator of the performance of interest and a quantification of the associated uncertainty; and The estimated indicator of the performance of interest and the quantification of the associated uncertainty are output to modify a process based on the estimated indicator of the performance of interest and the quantification of the associated uncertainty.

2. The system according to claim 1, wherein: in, The first set of data includes manufacturing tolerances.

3. The system according to claim 2, characterized in that in, The manufacturing tolerance is the uncertainty associated with one or more input values.

4. The system according to claim 1, wherein: in, The one or more outputs are results of the simulation.

5. The system according to claim 1, wherein: in, The performance indicator of interest includes engine performance output.

6. The system according to claim 5, characterized in that in, The engine performance output includes mechanical power of the engine.

7. The system according to claim 5, characterized in that in, The engine performance output includes a maximum efficiency of the engine.

8. The system according to claim 1, wherein: in, The first set of data includes tabular data.

9. The system according to claim 8, characterized in that in, The first set of data includes design of experiments (DOE) points.

10. The system according to claim 8, wherein: in, The first set of data includes measurements of engine performance output.

Images