Calibration System and Method for Calibrating an Industrial System Model Using Simulation Failure

Abstract

A calibration system and method for calibrating a model of the dynamics of an industrial system is provided. The calibration method includes estimating the success or failure of a simulation by simulating the model multiple times with different combinations of parameters within a range of acceptable parameter values. The calibration system is iteratively trained to define the likelihood of failure of the simulation of the model and for a probabilistic parameter-cost mapping between various combinations of different values ​​of the parameters of the model and their corresponding calibration errors until a termination condition is met. Furthermore, once the termination condition is met, the model is calibrated using an optimal combination of parameters that is most likely to minimize the calibration error in the probabilistic parameter-cost mapping.

Description

Technical Field

[0001] The present disclosure generally relates to calibration systems and methods for calibrating industrial system models, and more particularly to calibration systems and methods for calibrating industrial system models based on simulation failures.

Background Art

[0002] Advances in modeling and computing have led to the emergence of high-fidelity digital twin models that can simulate the dynamics of a wide range of industrial systems. A digital twin is a virtual representation of a system, with data updated in real time to support decision-making using simulation, machine learning, and inference. Digital twins use predictive models of dynamic systems designed for simulation. Predictive models are often formulated as black-box modeling software that prohibits users from accessing the underlying model structure.

[0003] The approach of formulating predictive models as black-box models can be used for various reasons. These reasons include the complexity that occurs in software developed and improved over years or decades, limited application programming interfaces to simplify interactions with large-scale software architectures, or the desire to avoid the widespread dissemination of trade secrets or proprietary information encoded in the model.

[0004] Even though users do not have access to this model structure, they often have to estimate the parameters of this model structure in order to calibrate the model to match the observed data. Existing parameter estimation algorithms applicable to black-box systems are designed under the assumption that the model can be simulated forward in time over any period of interest for any set of parameters within an acceptable search region.

[0005] However, in reality, this assumption is invalidated by many existing simulation-oriented models due to multiscale mechanics, large non-linearity, and numerically stiff behavior. Even with recent solvers, forward simulations of such models often take a significant amount of time due to small step sizes and event-driven behavior in continuous / discrete hybrid systems. Also, the shape of the acceptable parameter set that leads to effective simulations is often complex and unintuitive. Thus, when parameters are selected from a user-defined search space, simulations often end in failure. In the worst case, the candidate parameter set can make the simulation extremely slow and ultimately fail, wasting a large amount of computation. Such behavior can be a major obstacle when performing calibration methods on black-box models.

[0006] Therefore, there is a need for an efficient system and / or method that can estimate near-optimal parameters without performing large-scale simulations, thereby avoiding excessive time and resource consumption without compromising simulation performance. SUMMARY OF THE INVENTION PROBLEMS TO BE SOLVED BY THE INVENTION

[0007] Accordingly, an object of some embodiments is to provide a calibration system and a calibration method that learn from simulation failures and incorporate this information to increase the probability of selecting a set of parameters that lead to simulation success while avoiding excessive time and resource consumption.

[0008] In some embodiments, in a high-fidelity digital model (also referred to as a "model") that can simulate the dynamics of a wide range of industrial systems, it is often necessary to calibrate or estimate a set of optimal parameters in terms of fitness in order to reflect the observed behavior of the industrial system. Searching for one or more optimal values of the parameters to explore the parameter space (i.e., the allowable parameter range) is an inevitable part of the calibration process, but the model is rarely designed to be valid for arbitrarily large parameter spaces.

[0009] Some embodiments are based on the recognition that applying existing calibration methods often results in repeated model evaluations for the parameters, which can cause the simulation to become significantly slower or even fail completely. In general, the shape of the sub-region within the parameter space that can cause simulation failure is unknown.

[0010] Such problems are particularly common in building energy models. For example, an energy model may attempt to describe the temporal behavior of an occupied building using a closed-loop space conditioning system associated with the building. The time constants of the closed-loop space conditioning system typically vary by several orders of magnitude from milliseconds to several weeks and do not exhibit a clear separation of these time scales. Also, these closed-loop space conditioning systems often exhibit distinct hybrid behavior due to equipment cycle behavior, discontinuities in the derivatives of vapor compression cycles, and discontinuities in solar heat gain at sunrise and sunset. Nonlinear interactions between parameters can also cause the simulation behavior to deteriorate. For example, as a result of changing the parameters of an equipment model, the cooling capacity is lost, and correspondingly, the room temperature rises significantly for a particular set of building model parameters. Therefore, the allowable parameter search space is very complex and difficult to capture without significant time investment by a specialized user.

Means for Solving the Problem

[0011] To mitigate the above difficulties, some embodiments of the present disclosure learn a failure region within a parameter space from an online simulation and inform a Bayesian optimization (BO) algorithm to avoid the failure region while searching for optimal parameters in an acceptable parameter search space, providing a calibration system that uses a failure-robust Bayesian optimization (FR-BO) algorithm. As a result, the convergence of the optimizer is accelerated, and it is prevented that time is wasted by attempting simulations of parameters with a high probability of failure. In particular, FR-BO identifies regions within the parameter space where the system is likely to fail by using the simulated data. This region corresponds to combinations of values of a plurality of parameters within an acceptable parameter range (also referred to as the "parameter space"). Further, some embodiments of the present disclosure provide an acquisition function based on a constrained BO acquisition function that enables the search for optimizer candidates that not only fit well with the operational data but also have a low likelihood of causing simulation failures. The operational data represents measured values of the operation of an industrial system.

[0012] In particular, in some embodiments, the calibration system calibrates a model of the dynamics of an industrial system. The model describes changes in the state of the industrial system in response to the control of the industrial system according to a sequence of control inputs for fitting to the operational data, and the operational data represents measured values of the operation of the industrial system, including the values of the control inputs and the corresponding values of the state of the industrial system. The calibration system simulates at least one operation of the industrial system multiple times, and each simulation cycle includes estimating the success or failure of the simulation by executing a model of the industrial system having different combinations of parameter values selected from within a range of acceptable parameter values.

[0013] In some embodiments, the calibration system includes a failure classifier. The failure classifier is trained to define the likelihood of failure of the operation of the industrial system for an acceptable range of values of the model's parameters using training data that includes combinations of values of selected parameters for which the success or failure of the estimated simulation has been labeled. The calibration system further includes a probabilistic parameter-cost regressor. The parameter-cost regressor is trained to map between different combinations of values of the parameters of the industrial system model and their corresponding calibration errors.

[0014] In some embodiments, the parameter-cost regressor is repeatedly trained until an end condition is met. The probabilistic parameter-cost regressor is based on a Bayesian optimization technique. To perform the iteration, a calibration acquisition function of the first two order moments of the calibration error and a combination of parameters that is most likely to minimize the calibration error are identified according to the probabilistic parameter-cost regressor. The probabilistic parameter-cost regressor defines the first two order moments of the calibration error. In some embodiments, the at least first two order moments may include the mean of the calibration error and the variance of the calibration error (also referred to as the "confidence range"). For example, for a particular combination of different values of different parameters of a thermodynamics model, the probabilistic parameter-cost regressor provides not only the calibration error but also a confidence range centered on the calibration error. Accordingly, the convergence rate of Bayesian optimization is increased and the computational load of the calibration system is reduced.

[0015] For example, some embodiments are intended to select different combinations of parameters (also referred to as "data points") that must then be queried. In such cases, querying different combinations of parameters means simulating the state of the industrial system (such as the thermal state within an environment based on a thermodynamics model) on a model of the industrial system using different combinations of parameters and the values of the received control inputs corresponding to the values of the state of the received industrial system. Some embodiments select different combinations of parameters to be queried next by using an acquisition function for the first two moment functions of the calibration error.

[0016] The acquisition function selects different combinations of parameters to be queried next by using a probabilistic mapping provided by a probabilistic parameter-cost regressor. In some embodiments, the acquisition function is maximized by a calibration system so as to select different combinations of parameters that are most likely to be the global minimum in the probabilistic parameter-cost regressor. Thus, the acquisition function is used as guidance for selecting different combinations of parameters to be queried next. Therefore, the calibration system selects different combinations of parameters that are most likely to be the global minimum in the probabilistic parameter-cost mapping according to the acquisition function for the first two moment functions of the calibration error.

[0017] Furthermore, the calibration system determines the likelihood of failure of the operation of the industrial system for a range of acceptable parameter values from the selected different combinations of parameters of the model. Therefore, the calibration system uses training data that includes the values of the selected combinations of parameters for which the estimated simulation success or failure of the model is labeled. Based on the results of this likelihood, the probabilistic parameter-cost mapping is updated.

[0018] In the next iteration, the calibration system selects a new combination of different parameters that is most likely to be the global minimum in the updated probabilistic parameter-cost mapping, and estimates the likelihood of failure of the operation of the industrial system for the selected values of this new combination of different parameters. Subsequently, the calibration system updates the updated probabilistic parameter-cost mapping again using Bayesian optimization based on the estimated calibration error for the new combination of different parameters. Similarly, the probabilistic parameter-cost mapping is repeatedly calculated / updated until the termination condition is satisfied. In one embodiment, the termination condition includes the number of iterations defined by the user.

[0019] The calibration system monitors whether the termination condition is satisfied. If the termination condition is satisfied, the calibration system outputs the optimal combination of different parameters of the model of the industrial system that is most likely to be the global minimum in the probabilistic parameter-cost mapping.

[0020] The optimal combination of different parameters of the industrial system model may be determined offline (i.e., in advance). In addition to or instead of this, in some embodiments, the optimal combination of different parameters of the industrial system model may be determined online, i.e., in real time. For example, a calibration system may be integrated with a heating, ventilation, and air conditioning (HVAC) system installed to harmonize the environment, and the calibration system may output the optimal combination of different parameters during the operation of the HVAC system. Further, in some alternative embodiments, the optimal combination of different parameters of the thermodynamics model may be determined offline, and the thermodynamics model may be updated online. For example, the optimal combination of different parameters of the thermodynamics model is determined offline and presented to the controller of the HVAC system. The controller may determine the control input to the actuator of the HVAC system based on the optimal combination of different parameters. However, when the physical structure of the building and / or the actuator of the HVAC system changes, the calibration system may update the thermodynamics model online by estimating in real time a new optimal combination of different parameters in response to the change in the physical structure of the building and / or the actuator of the HVAC system. Further, the new optimal combination of different parameters may be presented to the controller of the HVAC system. As a result, the controller may determine the control input to the actuator of the HVAC system based on the new optimal combination of different parameters.

[0021] In some embodiments, Bayesian optimization includes a probabilistic parameter-cost regressor that uses a Gaussian process (GP) to provide a probabilistic mapping, and an acquisition function that exploits the probabilistic mapping provided by the probabilistic parameter-cost regressor to direct queries of combinations of different parameters of the resulting industrial system model.

[0022] Some embodiments are based on the recognition that the inversion and determinant operations typically used in GPs result in cubic complexity with respect to the number of data points, suggesting that it is not practical to use GPs in medium or high-dimensional parameter spaces, as typically many samplings and evaluations of the calibration cost function are required to find the optimal solution in such spaces.

[0023] Accordingly, one embodiment of the present disclosure provides a calibration system for calibrating a model of the dynamics of an industrial system, where the model describes changes in the state of the industrial system in response to control of the industrial system according to a sequence of control inputs for fitting to operating data, and the operating data represents measured values of the operation of the industrial system, including values of the control inputs and corresponding values of the state of the industrial system. The calibration system includes at least one processor and a memory storing instructions, which, when executed by the at least one processor, cause the calibration system to perform: simulating the operation of the industrial system multiple times, where each simulation cycle includes executing the model of the industrial system with different combinations of values of the parameters selected from within a range of acceptable parameter values to estimate success or failure of the simulation; further causing the calibration system to train a failure classifier that defines a likelihood of failure of the operation of the industrial system for the range of acceptable parameter values of the model using training data including combinations of values of the selected parameters labeled with the estimated success or failure of the simulation; and training a parameter-cost regressor for probabilistic parameter-cost mapping between different combinations of values of the parameters of the model of the industrial system and their corresponding calibration errors.The above parameter-cost regressor is repeatedly trained until an end condition is satisfied. To perform the repetition, the at least one processor is configured to identify a combination of parameters that is most likely to minimize the calibration error according to the probabilistic parameter-cost mapping, execute an acquisition function for the first two order moments of the calibration error, adjust the identified parameters according to the failure classifier, update the identified combination of parameters using a failure-robust acquisition function, reduce the likelihood of failure of the operation of the industrial system using the model with the updated parameters, simulate the operation of the industrial system controlled by the control input in the operation data using the model with the updated parameters to generate a simulated state of the industrial system, and update the probabilistic parameter-cost mapping based on the updated parameters, the simulated state of the industrial system, and the calibration error between the corresponding state of the industrial system in the operation data. The calibration system is further configured to calibrate the model of the industrial system using the optimal combination of parameters that is most likely to minimize the calibration error in the probabilistic parameter-cost mapping according to the acquisition function when the end condition is satisfied.

[0024] Accordingly, another embodiment of the present disclosure provides a calibration method for calibrating a model of the dynamics of an industrial system, where the model describes changes in the state of the industrial system in response to control of the industrial system according to a sequence of control inputs for fitting to operational data, and the operational data represents measured values of the operation of the industrial system, including the values of the control inputs and the corresponding values of the state of the industrial system. The calibration method includes simulating the operation of the industrial system a plurality of times, each simulation cycle including executing the model of the industrial system with different combinations of values of the parameters selected from within a range of acceptable parameter values to estimate success or failure of the simulation, and the calibration method further includes training a failure classifier that defines the probability of failure of the operation of the industrial system for the range of acceptable parameter values of the model using training data including the combinations of values of the selected parameters labeled with the estimated success or failure of the simulation, and training a parameter-cost regressor for probabilistic parameter-cost mapping between different combinations of values of the parameters of the model of the industrial system and their corresponding calibration errors. The parameter-cost regressor is repeatedly trained until an end condition is met.Further, to perform the iteration, the calibration method identifies a combination of parameters that is most likely to minimize the calibration error according to the probabilistic parameter-cost mapping, executes an acquisition function for the first two order moments of the calibration error, and updates the identified combination of parameters using an acquisition function that is robust to failure to adjust the identified parameters according to the failure classifier, thereby reducing the likelihood of failure of the operation of the industrial system using the model with the updated parameters, simulating the operation of the industrial system controlled by the control input in the operation data using the model with the updated parameters to generate a simulated state of the industrial system, and updating the probabilistic parameter-cost mapping based on the updated parameters, the simulated state of the industrial system, and the calibration error between the corresponding state of the industrial system in the operation data. The calibration method further includes calibrating the model of the industrial system using the optimal combination of parameters that is most likely to minimize the calibration error in the probabilistic parameter-cost mapping according to the acquisition function when the end condition is satisfied.

Brief Description of the Drawings

[0025]

Figure 1A

Figure 1B

Figure 1C

Figure 1D

Figure 2

Figure 3

Figure 4A

Figure 4B

Figure 5A

Figure 5B

Figure 5C

Figure 5D

Figure 6

Figure 7

Figure 8

Best Mode for Carrying Out the Invention

[0026] In the following description, for the sake of explanation, numerous specific details are set forth in order to provide a thorough understanding of the present disclosure. It will be apparent to those skilled in the art, however, that the present disclosure may be practiced without these specific details. In other instances, devices and methods are shown in block diagram form only to avoid obscuring the present disclosure.

[0027] As used in this specification and the claims, the terms “for example,” “as an example,” and “such as,” as well as each of the verbs “comprising,” “having,” “including,” and other verb forms thereof, when used in conjunction with a listing of one or more components or other items, are to be construed as open-ended, meaning that the listing is not to be considered as excluding further components or items. The term “based on” means at least in part based on. Further, it should be understood that the style and terminology used in this specification are for the purpose of description and should not be regarded as limiting. Any headings used in this specification are for convenience only and have no legal or limiting effect.

[0028] Generally, simulation is a very useful and valuable work tool. Simulation is used in industrial fields and can enable learning and testing of the behavior of systems. Simulation provides a low-cost, safe, and fast analysis tool. It also provides advantages achievable with many different system configurations. With the progress of modeling and computing, high-fidelity digital models that can simulate the dynamics of a wide range of industrial systems have been developed. In these models, in many cases, calibration or estimation of an optimal set of parameters is required. Practical calibration methods are often designed to estimate parameters that are close to optimal without performing large-scale simulations, which would consume a great deal of time and resources without correspondingly enhancing the simulation performance.

[0029] Therefore, various methods are used to learn the parameters to avoid wasting excessive time and resources in simulating these models. However, it is difficult to learn the parameters from limited data. For example, the Bayesian optimization (BO) method is an effective way to learn the parameters in a few shots based on limited data, that is, the number of evaluations of the cost function (equivalently, the simulation of the model) is significantly less than that of the population-based method. Furthermore, Bayesian optimization can be made into a powerful and easy-to-use learning tool for model calibration by essentially balancing exploration and exploitation and incorporating non-convex constraints through a modified acquisition function.

[0030] Some embodiments of the present disclosure are based on the recognition that for any set of parameters within an acceptable search region, the model may not be simulated forward in time over any period of interest. Thus, the model simulation may not be completed within the period of interest.

[0031] Furthermore, some embodiments of the present disclosure are based on the recognition that by using the simulation data associated with the failure of the model simulation, the optimal values of the parameters over the entire acceptable parameter space with a low probability of failure can be learned. Thus, the model can be trained to select the parameter values that reduce the chance of simulation failure and, as a result, reduce the excessive consumption of resources for simulation failures.

[0032] Accordingly, the present disclosure provides a calibration system based on a failure-robust Bayesian optimization (FR-BO) method. The FR-BO method includes various modules that identify regions within the parameter space where the industrial system is likely to fail by using simulated data. Subsequently, a specific acquisition function is designed based on a constrained BO acquisition function that enables the search for optimizer candidates that not only fit well with the operational data but also have a low likelihood of causing simulation failures. With reference to FIGS. 1A and 1B, the calibration system using FR-BO will be further described.

[0033] FIG. 1A shows a working environment 100 of a calibration system 101 for calibrating a model of an industrial system 103 according to some embodiments of the present disclosure. The model of the industrial system 103 mimics the dynamics of the industrial system 103. In particular, the model of the industrial system 103 corresponds to a software model (also referred to as a mathematical model). In the working environment 100 of the calibration system, the calibration system 101 and the industrial system 103 are coupled to each other via a network 105. The components described within the working environment 100 may be further divided into two or more components and / or combined in any suitable configuration. Further, one or more components may be rearranged, changed, added, and / or deleted.

[0034] In some embodiments, the calibration system 100 includes a transceiver 101a configured to exchange information through the network 105. For example, the transceiver 101a receives, via the network 105, a model of the industrial system 103 and data indicating measured values of the operation of the industrial system 103. The industrial system 103 may correspond to any system that is controlled, such as a heating, ventilation, and air conditioning (HVAC) system.

[0035] Industrial systems 103, such as HVAC systems, include actuators such as an indoor fan, an outdoor fan, and an expansion valve actuator. These actuators can be controlled according to corresponding control inputs, for example, according to the speed of the indoor fan, the speed of the outdoor fan, the position of the expansion valve, the speed of the compressor, and the like. In addition to or instead of this, in some embodiments, the control input may include a temperature value and / or a humidity value. In response to the control of the actuators of the HVAC system according to the corresponding control inputs, the thermal state in the environment changes.

[0036] According to another embodiment, the control input is determined based on different parameters of a model of the industrial system 103. The model of the industrial system 103 is a digital twin model of the dynamics of the operation of the industrial system 103, and the model of the industrial system 103 describes the changes in the state of the industrial system 103. Thus, the control input is variable and the parameter values are fixed. For example, in the case of a digital twin model corresponding to an air conditioning (AC) system, the control input may include the required temperature indoors (e.g., 23 degrees), the speed of the fan, etc., which are variable. However, the parameters may include the volume of the room to be air-conditioned, and the volume of the room is fixed.

[0037] For example, in an HVAC system, different parameters of a thermodynamics model define the physical structure of one or a combination of a building, an actuator of the HVAC system, and the layout of the HVAC system to harmonize with the environment. For example, different parameters of the thermodynamics model include parameters of a building where the HVAC system is used, such as the thickness of the building floor, the infrared emissivity of the building roof, the solar radiation emissivity of the building roof, the air infiltration rate, the indoor air heat transfer coefficient (HTC), and the outdoor air HTC. In addition, different parameters of the thermodynamics model may include HVAC parameters, such as an outdoor HEX heat transfer coefficient (HTC) adjustment factor, an indoor HEX HTC adjustment factor, an indoor HEX Lewis number, an outdoor HEX vapor HTC, an indoor HEX vapor HTC, an outdoor HEX liquid HTC, an indoor HEX liquid, an outdoor HEX two-phase HTC, and an indoor HEX two-phase HTC.

[0038] In some embodiments, network 105 may be a wireless communication network, such as cellular, Wi-Fi®, Internet, local area network, etc. In some alternative embodiments, network 105 may be a wired communication network. The calibration system 101 may obtain simulation data by executing a model of the industrial system 103 multiple times. The calibration system 101 learns, from the simulation data, values of different combinations of parameters that increase the likelihood of failure of the model simulation by using the FR-BO method. In some embodiments, the calibration system 101 may be pre-trained with the simulation data of the model to determine an optimal combination of values of different parameters that may lead to a successful simulation.

[0039] Furthermore, the transceiver 101a transmits an optimal combination of different parameters to the industrial system 103 via the network 105. Additionally, in some embodiments, the industrial system 103 may include a controller. The controller determines control inputs for the actuators of the industrial system 103 based on the optimal combination of different parameters received from the calibration system 101. Further, the state of the actuators of the industrial system 103 may be controlled based on the control inputs in order to obtain an output or a specific state of the industrial system 103 desired by the user.

[0040] Furthermore, the mathematical formulation used to model the industrial system 103 and the challenges faced in determining the optimal values of the parameters for different combinations of the model's parameters will be described in detail as follows.

[0041]

Number

[0042]

Number

[0043]

Number

[0044]

Number

[0045]

Number

[0046]

Number

[0047] Therefore, some embodiments use Bayesian Optimization (BO) methods, which, like in the case of black box model calibration, are effective in finding the global optimum of functions where gradients are not available and evaluation is costly.

[0048]

Number

[0049]

Number

[0050] In some cases, the model tends to indicate simulation failures in the failure region Θ F ⊂ Θ, and the failure does not always occur instantaneously. For example, in case (iv), the failure is flagged after a potentially long specified end time. As a result, data-driven algorithms designed without relying on simulation failures may continue to compute optimizer candidates that exist in the failure region Θ F This can lead to a decrease in the performance of the algorithm and a waste of a large amount of computational resources and CPU time.

[0051]

Number

[0052]

Number

[0053] Figure 1B shows the modules of the calibration system 101 according to some embodiments of the present disclosure. The calibration system 101 is used to calibrate the mechanical model of the industrial system 103. The model is calibrated so that it can explain the changes in the state of the industrial system 103 in response to the control of the industrial system 103 according to a sequence of control inputs for fitting to the operation data. The operation data represents the measured values of the operation of the industrial system 103, including the values of the control inputs and the corresponding values of the state of the industrial system 103.

[0054]

Number

[0055]

Number

[0056]

Number

[0057]

Number

[0058]

Number

[0059]

Number

[0060] Failure region Θ F Based on the estimation of, the simulation model 111 is improved by refining the acceptable parameter range (also referred to as the "acceptable parameter space") Θ.

[0061] In some embodiments, the calibration system 101 obtains a predefined condition for the failure region Θ corresponding to the simulation model 111. F The calibration system 101 is further configured to determine whether the failure region Θ of the improved simulation model 111 satisfies a predefined condition for the failure region. The failure region Θ F The predefined condition for the failure region may define the size of the failure region Θ within the acceptable parameter space Θ corresponding to different combinations of parameters. If it is determined that the failure region of the improved model does not satisfy the predefined condition, for example, if the failure region is not less than the predefined size of the failure region Θ F The calibration system 101 iteratively improves the model 111 and re - estimates the failure region until the estimated failure region Θ F satisfies the predefined condition for the failure region Θ F For example, when the failure region is not less than the predefined size of the failure region Θ F is not less than the predefined size of the failure region Θ F The calibration system 101 further includes an acquisition function module 107d. The acquisition function module 107d selects parameters so as to avoid the failure region Θ

[0062] by using an acquisition function. To that end, the acquisition function module 107d obtains the first two - order moments of the calibration error from the parameter - cost regressor module 107c and executes an acquisition function for the first two - order moments. By executing the acquisition function, the combination of parameters with the highest probability of minimizing the calibration error is identified according to the probabilistic parameter - cost mapping. In some embodiments, different combinations of parameters are selected to query the next combination of different parameters by using the acquisition function of the first two - order moments of the calibration error. F

[0063] ​In particular, the acquisition function module 107d selects a combination of different parameters to query next by using the probabilistic mapping provided by the parameter-cost regressor module 107c. This will be further described with reference to FIG. 1D.

[0064] In one embodiment, the acquisition function is maximized by the calibration system 101 such that, at each iteration "j" of FR-BO, it selects a combination of different parameters that is most likely to be the global minimum in the parameter-cost mapping provided by the parameter-cost regressor module 107c. Thus, the acquisition function is used as guidance for selecting a combination of different parameters to query next. Therefore, the calibration system 101 selects a combination of different parameters that is most likely to be the global minimum in the probabilistic parameter-cost mapping according to the acquisition function of the first two order moments of the calibration error. The acquisition function module 107d uses at least one of the acquisition functions, namely, expected improvement (EI), lower confidence bound, or entropy search. In a preferred embodiment, the acquisition function module 107d uses the EI acquisition function.

[0065]

Number

[0066] In this way, the FR-BO module 107 is repeatedly trained with simulation data until the termination condition is satisfied. Failure-Robust Bayesian Optimization (FR-BO) Algorithm:

[0067] The FR-BO module 107 implements the FR-BO algorithm using modules 107a to 107d (as shown in Figure 1B). The FR-BO module 107 learns the failure regions from the data obtained during the simulation through a probabilistic classifier such as a scalable variational Gaussian process classifier (VGPC). By using the probability of simulation failure or simulation success through an active learning method, the Bayesian optimization step is accelerated, and furthermore, it leads to the best place to simulate the dynamic model M T (θ), and a better estimate of the failure region boundary is obtained. Furthermore, by incorporating the information from the VGPC through a constrained weighted acquisition function, it is guaranteed that the parameters are selected while avoiding the failure regions.

[0068]

Number

[0069]

Number

[0070] Each failure label is denoted as "+1" if the simulation fails and "-1" if the simulation does not fail. If the simulation fails, the corresponding cost function value is set to an undefined value (or null value), such as NaN. If the simulation succeeds, the cost function generates a real-valued scalar.

[0071]

Number

[0072]

Number

[0073]

Number

[0074]

Number

[0075]

Number

[0076]

Number

[0077]

Number

[0078]

Number

[0079]

Number

[0080] Therefore, the active learning method guides the calibration system 101 to the best place to simulate the dynamic model M T (θ) in order to accelerate the Bayesian optimization step and obtain a better estimate of the failure region boundary.

[0081]

Number

[0082]

Number

[0083]

Number

[0084] In some embodiments, by using induced variables such as in VGPC, the scalability of GP can be significantly improved.

[0085]

Number

[0086]

Number

[0087]

Number

[0088]

Number

[0089]

Number

[0090] Some embodiments are based on the recognition that it is costly to retrain VGPC every time a new data sample is collected. However, as long as VGPC has been initially trained on some data, the frequency of retraining VGPC can be reduced. In such cases, the frequency of retraining VGPC depends on the problem. In some embodiments, VGPC is retrained when the FR-BO operation remains at a local optimum for a predetermined number of iterations.

[0091] Figure 1C shows a graphical representation of the mean and confidence range output from the probabilistic parameter-cost regressor module 107c of a calibration system, according to some embodiments of the present disclosure. In particular, Figure 1C shows a graph of parameter (x) on the x-axis and a dependent function (f(x)) on the y-axis. This graph shows the variation of the dependent function f(x) for different values of parameter X within the range of acceptable parameters X (in this case, [-1 to 3]). For simplicity of explanation, the graph in Figure 1C includes only one parameter and one dependent function. However, in a real model of the industrial system 103, the graph will include multiple parameters and multiple dependent functions.

[0092] Referring to Figure 1C, the points (such as point 121) within the graph represent samples / observations, the curve 123 represents the mean, and the shaded region 125 represents the confidence range. Thus, this graph provides information regarding the estimated value of the dependent function f(x) with respect to the estimated value of parameter X. The calibration system 101 uses this graph to determine one or more failure regions within the range of acceptable parameters X that may lead to simulation failures. From the graph in Figure 1C, it can be determined that if a value of X is selected in the range after 1.25, the simulation is likely to fail. This region within the acceptable space (or range) of X is called the failure region. This is because if a value of X is selected within the failure region (i.e., 1.25 to 2), the simulation of the model is likely to fail.

[0093] In some embodiments, based on a graph corresponding to a model having multiple parameters (such as the graph in FIG. 1C), the calibration system 101 may determine the failure region as a combination of values of multiple parameters that increase the likelihood of failure of the operation of the industrial system 103. The calibration system 101 uses the FR-BO module 107 to learn the failure regions of different parameters used in the model of the industrial system 103, and thus accurately estimate the values of different parameters that lead to successful simulation. In this way, the calibration system 101 saves excessive time and other resources involved in the simulation.

[0094] FIG. 1D shows a graphical representation of an acquisition function executed by the calibration system 101 according to some embodiments of the present disclosure. In particular, FIG. 1D shows a graph including a curve 127 of an acquisition function for selecting a combination of different parameters to be queried next. The acquisition function selects a combination of different parameters to be queried next by using the probabilistic mapping provided by the probabilistic parameter-cost module 107c. In some embodiments, the acquisition function is maximized by the calibration system 101 such that it selects a combination of different parameters that is most likely to be the global minimum 129 in the probabilistic mapping for querying. Thus, the acquisition function is used as guidance for selecting a combination of different parameters to be queried next.

[0095] Figure 2 shows a block diagram of a calibration system 200 for calibrating an industrial system 103 according to some embodiments of the present disclosure. The calibration system 200 can have several interfaces for connecting the calibration system 200 to other systems and devices. For example, a network interface controller (NIC) 201 is adapted to connect the calibration system 200 to a network 205 through a bus 203. Through the network 205, the calibration system 200 may receive data 207 wirelessly or wired, and the data 207 indicates measured values of the operation of the industrial system, including values of control inputs to actuators of the industrial system and values of the states of the industrial system caused by the operation of the industrial system according to the values of the control inputs.

[0096] For example, the industrial system 103 corresponds to an HVAC system. The calibration system 200 may receive, wirelessly through the network 205, values of control inputs to actuators of the HVAC system and values of the thermal state at the location of the environment caused by the operation of the HVAC system according to the values of the control inputs. In some cases, data 207 indicating measured values of the operation of the HVAC system and values of the thermal state at the location of the environment may be received through an input interface 209.

[0097] The calibration system 200 includes a processor 211 configured to execute stored instructions and a memory 213 that stores instructions executable by the processor 211. The processor 211 may be a single-core processor, a multi-core processor, a computing cluster, or any number of other configurations. The memory 213 can include random access memory (RAM), read-only memory (ROM), flash memory, or any other suitable memory system. The processor 211 is connected through a bus 203 to one or more input / output devices. Additionally, the calibration system 200 includes a storage device 215 adapted to store different modules storing executable instructions for the processor 211. The storage device 215 can be implemented using a hard drive, an optical drive, a thumb drive, an array of drives, or any combination thereof.

[0098] The storage device 215 is configured to store the FR-BO module 107. The FR-BO module 107 is configured to determine optimal values of different parameter combinations to avoid simulation failures of a simulation model (not shown in FIG. 2) corresponding to the industrial system 103. Therefore, the FR-BO module 107 is trained based on training data including various combinations of different parameters that result in simulation failures or simulation successes.

[0099] The FR-BO module 107 includes a parameter-cost regressor module 107c for obtaining a probabilistic parameter-cost mapping. The parameter-cost regressor module 107c is repeatedly trained to map between different combinations of values of the model's parameters and the corresponding calibration errors until an end condition is met. The parameter-cost regressor module 107c is further configured to determine the first two moment orders (i.e., the mean μ and the standard deviation σ) of the calibration error.

[0100] The FR-BO module 107 is further configured to estimate the likelihood of success and failure of simulations for different ranges (or spaces) of acceptable parameters by learning the set failure regions. To this end, the FR-BO module 107 uses a failure classifier module 107b. The failure classifier module 107b is trained using training data to define the likelihood of failure of the operation of the industrial system 103 for the range of values of the acceptable parameters of the model. The training data includes combinations of values of selected parameters that are labeled with the estimated success or failure of the simulation.

[0101] The FR-BO module 107 further includes an acquisition function module 107d. The acquisition function module 107d selects parameters so as to avoid the failure regions corresponding to the selected parameters by using an acquisition function. To this end, the acquisition function module 107d obtains the first two order moments of the calibration error from the parameter-cost regressor module 107c and executes an acquisition function on the first two order moments to identify the combination of parameters that is most likely to minimize the calibration error according to a probabilistic parameter-cost mapping. In some embodiments, different combinations of parameters are selected to query the next combination of different parameters by using the acquisition function of the first two order moments of the calibration error.

[0102] Furthermore, the FR-BO module 107 includes an acquisition function module 107a that is robust to failure. The acquisition function module 107a that is robust to failure is configured to adjust the parameter values in the identified parameter combinations based on the output of the acquisition function module 107d and the output of the failure classifier module 107b. Thus, the acquisition function module 107a that is robust to failure generates a failure-robust EI (FREI) acquisition function by incorporating the probabilistic output (obtained from the failure classifier module 107b) into the acquisition function. Further, the values of the updated parameter combinations are used to train the FR-BO module 107 in the next iteration. The values of the updated parameter combinations are used to simulate the simulation model in the next iteration.

[0103] In addition, the calibration system 200 may include an output interface 217. In some embodiments, the calibration system 200 is further configured to present an optimal combination of different parameters of the model of the industrial system 103 to the controller 219 of the industrial system 103 via the output interface 217. The controller 219 is configured to generate a control input for the actuator of the industrial system 103 based on the optimal combination of different parameters of the model.

[0104]

Number

[0105]

Number

[0106] Furthermore, the allowable parameter space of the parameters associated with the Robertson system is defined in the logarithmic space and is given by log 10 Θ={[-3,-1]×[2,12]×[5,15]}z. Thus, θ1 is in the interval [10 -3 ,10 -1can exist in, and θ2 can exist in the interval [10 -2 , 10 12 . Similarly, θ3 can exist in the interval [10 5 , 10 15 . For each θ sampled on Θ, the Robertson system is simulated over T. In an exemplary embodiment, the Robertson system may be simulated using a Radau IIA solver that is well-suited for the numerical integration of stiff systems.

[0107]

Number

[0108] Therefore, the scaling matrix W is set to W = diag[1, 5×10 4 , 1]. Further, 300 data points are collected for VGPC using active learning, then 700 BO iterations are performed, and VGPC is retrained every 100 iterations during BO.

[0109] Figure 3 shows a first table (Table I) including hyperparameters of a Robertson system for performing a failure-robust Bayesian optimization (FR-BO) method according to some embodiments of the present disclosure. In particular, the first table (Table I) includes hyperparameters used for the VGPC and GPR learners required for FR-BO.

[0110]

Number

[0111] Figure 4A shows a graphical representation of a comparison between the true output and the estimated output of a Robertson system according to some embodiments of the present disclosure. This comparison is calculated in an inspection scenario over the period of [10 -5 , 10 5 . From Figure 4A, it is clear that the estimated values of the parameters are very close to the actual values of the parameters.

[0112] Furthermore, referring to FIG. 4B, two slices of the failure regions learned by the VGPC are shown.

[0113] FIG. 4B shows a slice of the parameter space, along with the failure probability estimated by the failure classifier module 107b of the calibration system 101, according to some embodiments of the present disclosure. The left slice of FIG. 4B is obtained by fixing θ2 to its estimated value and varying θ1 and θ3 over a subset of Θ. Similarly, the right slice of FIG. 4B is obtained by fixing θ1 and varying the remaining two parameters θ2 and θ3. Both slices clearly show that the VGPC has identified regions where the model simulation is likely to indicate failure. Furthermore, it is evident that such regions can have large curvatures. Thus, even for low-dimensional dynamic systems, it is advantageous to use an advanced non-linear classifier such as the VGPC rather than a linear classifier.

[0114] In another exemplary embodiment, the parameters of a dynamic model in a digital twin of a building are estimated by using the calibration system 101. A simulation model of building energy behavior for a digital twin application is designed to predict multi-scale non-linear dynamics resulting from temporal interactions between many different building subsystems.

[0115] FIG. 5A shows a scenario including the thermodynamics of a building 500 and a heating, ventilation, and air conditioning (HVAC) 501 system calibrated by a calibration system 101 according to an exemplary embodiment of the present disclosure. The HVAC system 501 is installed within a building (also referred to as a “building system”) 500 to condition the environment of this building. A model of the vapor compression cycle of the HVAC system 501 including a compressor, a condenser heat exchanger, an electronic expansion valve, and an evaporator heat exchanger has been constructed. The behavior of the vapor compression cycle is governed by heat exchangers over the targeted time scale. Therefore, the model of the vapor compression cycle uses a dynamic model of the heat exchanger and algebraic (static) models of the compressor and the electronic expansion valve. To simplify the explanation, the lumped parameter method is used to characterize the dynamics of the refrigerant flow in the heat exchanger. In certain embodiments, a Helmholtz equation of state-based model may be used to describe refrigerant properties, while an ideal mixture gas may be used for the moist air model.

[0116] A simple isenthalpic model of the electronic expansion valve and the mass flow rate is adjusted in the vicinity of zero flow rate to prevent the tendency of the derivative of the mass flow rate to go to infinity. The flow coefficient is obtained through calibration against experimental data. The operation of the compressor may be described by relating the volumetric efficiency and the isentropic efficiency to the suction pressure, the discharge pressure, and the compressor frequency.

[0117] In an exemplary embodiment, the building model of the building 500 is constructed from the open-source Modelica Buildings library. The room model is based on the physics-based behavior of basic materials and commonly used components, and the zone air model is a mixed air single-node model having one bulk air temperature that interacts with all of the indoor radiation surfaces and heat loads.

[0118] According to an embodiment, the building model may be parameterized by the thickness of the roof 503, the thickness of the outer wall 505, and the thickness of the floor 507. The thickness of the roof is determined according to the thickness of the concrete layer 509, the thickness of the insulation layer 511, and the size of the plenum 513. The thickness of the floor 507 is determined according to the thickness of the concrete slab 515 and the thickness of the carpet tile 517.

[0119] For example, the building model consists of a single-story house based on the 2009 IECC standard. This single-story house has, for example, a floor area of 112.24 m and a height of 2.6 m, is oriented along the azimuth, and has a maximum of 3 occupants. Each outer wall also has a window 519 sized (e.g., 1.52 m × 2.72 m) to capture solar heat gain within the space of the building 500. Also included are a 10 cm thick concrete slab and 2 m of soil below the building 500 to characterize the interaction with the temperature boundary condition below the building 500 set at a constant temperature of 21°C. In addition, a peaked attic with a maximum height of 1.5 m is included so that the building model includes two thermal zones.

[0120] Furthermore, this building model is connected to a vapor compression cycle model, and a proportional-integral (PI) controller 521 is implemented on the heat pump. The heat pump adjusts the superheat temperature of the evaporator by using the compressor frequency to adjust the room temperature and the position of the expansion valve. The PI controller 521 also defines the lower and upper limits of the actuator while maintaining stability by implementing anti-windup. The resulting building envelope / HVAC combined model (i.e., a thermodynamics model) is simulated using the Atlanta-Hartsfield TMY3 file, with a convective and radiative heat load of 2 W / m 2 and a 2The latent heat load was incorporated during the hours between 8:00 am and 6:00 pm, and there were weather disturbances outside of these hours. Furthermore, such a model was exported from Modelica using the Functional Mockup Interface, and the resulting functional mockup unit (FMU) was imported into Python using the FMPy package, enabling seamless integration of the advanced machine learning module.

[0121] The inputs and outputs of the thermodynamics model may be selected to be similar to the inputs and outputs that would be observed in an actual experimental setup. The inputs to the heat pump include the room temperature setpoint, the evaporator superheat setpoint, and the indoor and outdoor fan speeds. The inputs to the building model include the convective load, the radiative load, and the latent heat load, along with the weather variables given in the TMY3 standard. Such heat loads can be estimated quite accurately by means of occupant detection, load surveys, or other similar methods.

[0122]

Number

[0123]

Number

[0124] Furthermore, the parameters selected to represent the HVAC system 501 and the building 500 are provided in Table II shown in Figure 5B.

[0125] Figure 5B shows a second table (Table II) that includes a comparison between the best estimate value 523 of parameter 525 after a predetermined number of iterations during calibration and the true value 527 of parameter 525 of the building and HVAC system. It can be observed that the value of parameter 525 estimated using the FR-BO approach is very close to the true value 527 of parameter 525.

[0126] Further, comparisons between the measured simulation outputs and the estimated simulation outputs of building 500 and HVAC system 501 are shown in FIGS. 5C-5D.

[0127] Figure 5C shows the estimated output and the true output associated with building 500 according to an exemplary embodiment of the present disclosure. The estimated output is provided by a model of building system 500 calibrated using the FR-BO approach (using calibration system 101). In the plot of FIG. 5C, the circles are the true data points and the continuous line is the estimated output from the model simulation. It is clear from FIG. 5C that the output error of building system 500 is small, which is even more evident from the second Table II (FIG. 5B) where most of the parameters are estimated with high accuracy.

[0128] Figure 5D shows the estimated output and the true output associated with HVAC system 501 according to an exemplary embodiment of the present disclosure. The estimated output is provided by a model of HVAC system 501 calibrated using the FR-BO approach (used by the proposed calibration system 101). In the plot of FIG. 5D, the circles are the true data points and the continuous line is the estimated output from the model simulation. It can be observed from FIG. 5D that the output error of HVAC system 501 is small, which is even more evident from the second Table II (FIG. 5B) where most of the parameters are estimated with high accuracy.

[0129] Furthermore, the proposed calibration system 101 also evaluates the effect of reducing the time wasted in simulations that failed during the optimization of the integrated building system (i.e., building system 500 and HVAC system 501).

[0130] FIG. 6 shows the regret and number of failures of simulations of an integrated building system including an HVAC system 501 and a building 500 according to an exemplary embodiment of the present disclosure. For evaluation, the VGPC is allowed to collect data by first iterating active learning 450 times before starting the optimization process. In the left sub-plot 601 of FIG. 6, it can be observed that the number of simulation failures has decreased by incorporating the FREI function. This improves the convergence of the Bayesian optimization step, so that FR-BO converges to its optimal solution in about 700 iterations, resulting in cost reduction.

[0131] In the right sub-plot 603 of FIG. 6, it can be observed that the total time used for Bayesian optimization is higher for FR-BO (15 hours vs. 9 hours), and the total time wasted on simulation failures is significantly less (15 hours vs. 23 hours). This corresponds to saving 8 hours of wall time.

[0132] FIG. 7 shows the steps of a calibration method 700 for calibrating a model of an industrial system 103 according to an exemplary embodiment.

[0133] In step 701, the operation of the industrial system 103 is simulated multiple times. In each simulation cycle, the execution of the model of the industrial system 103 includes estimating the success or failure of the simulation by variously combining the values of parameters selected from within a range of acceptable parameter values. The industrial system 103 can correspond to any system that can be modeled, such as a building system, an HVAC system, etc.

[0134] In step 703, the failure classifier 107b is trained to define the likelihood of failure of the operation of the industrial system 103 with respect to the range of values of the acceptable parameters of the model. The training data used to train the failure classifier 107b includes combinations of values of the selected parameters, labeled with the estimated success or failure of the simulation.

[0135] In step 705, the parameter-cost regressor 107c is repeatedly trained for probabilistic parameter-cost mapping between different combinations of values of the parameters of the model of the industrial system 103 and their corresponding calibration errors.

[0136] To repeatedly train the parameter-cost regressor 107c, in step 705a, an acquisition function of the first two order moments of the calibration error is executed to identify the combination of parameters that is most likely to minimize the calibration error according to the probabilistic parameter-cost mapping.

[0137] In step 705b, the likelihood of failure of the operation of the industrial system 103 is reduced using a model with updated parameters by adjusting the identified parameters according to the failure classifier module 107b and updating the identified combination of parameters using the acquisition function module 107a that is robust to failure.

[0138] In step 705c, the simulated state of the industrial system 103 is generated by simulating the operation of the industrial system 103 controlled by the control input in the operation data using a model with updated parameters. Further, the probabilistic parameter-cost mapping is updated based on the updated parameters, the simulated state of the industrial system 103, and the calibration error between the corresponding state of the industrial system 103 in the operation data.

[0139] In step 707, according to the acquisition function, the model of the industrial system 103 is calibrated using the optimal combination of parameters that is most likely to minimize the calibration error in the probabilistic parameter-cost mapping.

[0140] Some embodiments are based on the recognition that the simulation model 111 can be improved based on the identification of failure regions. Therefore, the calibration system 101 is configured to learn the failure regions. A failure region is a sub-region of the acceptable parameter space where the simulation model 111 fails in the simulation for one or more combinations of parameters. Thus, the failure region is deterministic. To identify the failure region, the calibration system 101 uses a failure classifier module 107b that is trained to estimate the failure regions within the acceptable parameter space for different combinations of parameters. Based on this training, the failure classifier module 107b estimates the failure regions within the acceptable parameter space with a certain probability. Thus, the output of the failure classifier module 107b is probabilistic.

[0141] Based on the failure regions estimated by the failure classifier module 107b, the calibration system 101 is further configured to iteratively improve the acceptable parameter space until the estimated failure regions meet predetermined conditions. The predetermined conditions may include the size of the failure region. Further details regarding the improvement of the simulation model (or the improvement of the acceptable parameter space) are provided below with reference to FIG. 8.

[0142] FIG. 8 shows the steps of a method 800 for improving the acceptable parameter space of the simulation model 111 based on failure regions according to an exemplary embodiment. FIG. 8 is described below in relation to FIG. 7. In an exemplary embodiment, the method 800 may be executed during the training of the failure classifier module 107b in step 703 of the calibration method 700. In another embodiment, the method 800 may be executed using a pre-trained failure classifier module 107b.

[0143] In step 801, a simulation model 111 (also referred to as a "model") corresponding to the industrial system 103 is obtained by the calibration system 101. In an exemplary embodiment, the simulation model 111 may be predefined in the calibration system 101.

[0144] In step 803, a failure of the simulation model 111 is defined. For example, a failure may be defined as an unsuccessful forward-in-time simulation of the model 111 until a desired final time. In another embodiment, a failure may be defined as any simulation of the model 111 that takes longer than a predefined time interval for simulating the model 111. For example, if the model 111 is configured to complete a simulation within a predefined time interval (e.g., 5 seconds), a failure is defined as any simulation that takes longer than the predefined time interval, i.e., 5 seconds.

[0145] In step 805, the failure region is estimated using the FR-BO module 107. The failure region is determined based on the definition of failure defined in step 803. Therefore, the FR-BO module 107 uses the failure classifier module 107b. The failure classifier module 107b is trained to estimate the failure region based on values of different combinations of parameters.

[0146] In step 807, based on the failure region estimated in step 805, the model 111 is improved, i.e., the allowable parameter space of the parameters in different combinations of parameters is improved.

[0147] In step 809, it is determined whether the failure region of the improved model satisfies a predetermined condition regarding the failure region. The predetermined condition regarding the failure region may define the size of the failure region within an acceptable parameter space corresponding to different combinations of parameters. If it is determined that the failure region of the improved model does not satisfy the predetermined condition, for example, if the failure region is not less than a predefined size of the failure region, method 800 iteratively returns to step 807, improves the improved model again, and estimates the failure region again.

[0148] On the other hand, if it is determined that the failure region of the improved model satisfies the predetermined condition, method 800 proceeds to step 811 and calibrates the parameters of model 111 using the determined improved model. Embodiments:

[0149] The above description provides only embodiments as specific examples and is not intended to limit the scope of the disclosure, applicability, or configuration. Rather, the above description of the embodiments as specific examples will provide those skilled in the art with an explanation that enables the realization of one or more embodiments as specific examples. Various modifications are intended to be made to the functions and configurations of the elements without departing from the spirit and scope of the disclosed subject matter recited in the appended claims.

[0150] Specific details have been provided in the above description to obtain a thorough understanding of the embodiments. However, those skilled in the art can understand that the embodiments can be implemented without these specific details. For example, the systems, processes, and other elements in the disclosed subject matter may be shown as components in the form of block diagrams in order not to obscure the embodiments with unnecessary details. In other examples, well-known processes, structures, and technologies may be shown without unnecessary details in order not to obscure the embodiments. Further, like reference numerals and names in the various drawings indicate like elements.

[0151] Also, individual embodiments may be described as processes shown as flowcharts, flow diagrams, data flow diagrams, structure diagrams, or block diagrams. A flowchart may describe the operations as a sequential process, but many of the operations can be executed in parallel or simultaneously. Additionally, the order of the operations may be rearranged. A process may end when its operations are completed, but may have additional steps that are not discussed or included in the figure. Further, all of the operations in any specifically described process may not occur in all embodiments. A process may correspond to a method, function, procedure, subroutine, subprogram, etc. When a process corresponds to a function, the end of the function may correspond to returning the function to the calling function or the main function.

[0152] Furthermore, embodiments of the disclosed subject matter may be implemented, at least in part, either manually or automatically. The manual or automatic implementation may be executed or at least assisted through a machine, hardware, software, firmware, middleware, microcode, hardware description language, or any combination thereof. When implemented in software, firmware, middleware or microcode, the program code or code segments for performing the required tasks may be stored on a machine-readable medium. A processor (or processors) may perform the required tasks.

[0153] The various methods or processes outlined in this specification may be encoded as software executable on one or more processors employing any one of a variety of operating systems or platforms. Additionally, such software may be described using any of a plurality of suitable programming languages and / or programming or scripting tools, and may be compiled as executable machine language code or intermediate code to be executed on a framework or virtual machine. Typically, the functionality of program modules may be combined or distributed as desired in various embodiments.

[0154] Embodiments of the present disclosure may be implemented as a method, and an example thereof is provided. The order of operations performed as part of this method may be determined in any suitable manner. Accordingly, an embodiment may be configured such that operations are performed in an order different from the order illustrated, which may include performing some operations simultaneously that are shown as a series of operations in the exemplary embodiment. Although the present disclosure has been described with reference to several preferred embodiments, it should be understood that various other adaptations and modifications can be made within the spirit and scope of the present disclosure. Accordingly, it is the aspect of the appended claims to cover all such variations and modifications that fall within the true spirit and scope of the present disclosure.

Claims

1. A calibration system for calibrating a model of the dynamics of an industrial system, wherein the model describes changes in the state of the industrial system in response to control of the industrial system according to a sequence of control inputs for fitting to operating data, the operating data indicating measured values of the operation of the industrial system, including values of the control inputs and corresponding values of the state of the industrial system, the calibration system comprising at least one processor and a memory storing instructions which, when executed by the at least one processor, cause the calibration system to perform multiple simulations of the operation of the industrial system, each simulation cycle including executing the model of the industrial system with different combinations of values of the parameters selected from within a range of acceptable parameter values, and estimating success or failure of the simulation based on at least one condition related to the simulation being satisfied, and further causing the calibration system to train a failure classifier that defines the likelihood of failure of the operation of the industrial system for the range of acceptable parameter values of the model using training data including combinations of values of the selected parameters labeled with the estimated success or failure of the simulation, and train a parameter-cost regressor for probabilistic parameter-cost mapping between different combinations of values of the parameters of the model of the industrial system and their corresponding calibration errors, the calibration errors being based on the simulated state of the industrial system for different combinations of parameter values and the corresponding state of the industrial system in the operating data, the parameter-cost regressor being repeatedly trained until an end condition is satisfied, and for performing the repetitions, the at least one processor Execute an acquisition function for the first two order moments of the calibration error, configured to identify a combination of parameters that is most likely to minimize the calibration error according to the probabilistic parameter-cost mapping, Update the identified combination of parameters using a failure-robust acquisition function that adjusts the identified parameters according to the failure classifier, thereby reducing the likelihood of failure of the operation of the industrial system using the model with the updated parameters, Generate a simulated state of the industrial system by simulating the operation of the industrial system controlled by the control input in the operation data using the model with the updated parameters, Configured to update the probabilistic parameter-cost mapping based on the updated parameters, the simulated state of the industrial system, and the calibration error between the corresponding state of the industrial system in the operation data, Further to the calibration system, A calibration system that, when the end condition is satisfied, causes the model of the industrial system to be calibrated using the optimal combination of parameters that is most likely to minimize the calibration error in the probabilistic parameter-cost mapping according to the acquisition function.

2. The at least one processor is further configured to Estimate a failure region within the range of acceptable parameter values in the selected combination of parameters, the failure region including at least one combination of parameter values that cause a failure of the operation of the industrial system, and further The calibration system according to claim 1, configured to provide a probability of simulation failure over the entire range of acceptable parameter values in the selected combination of parameters based on the estimated failure region.

3. The training data further includes a value of a cost function, the cost function corresponding to the calibration error, and the value of the cost function corresponding to at least one of a real-valued scalar when the simulation of the model is successful or a null value when the simulation of the model is unsuccessful. The calibration system according to claim 1.

4. To obtain the failure region, the at least one processor is further configured to obtain the training data for learning the failure region by sampling from within the range of the acceptable parameters, the failure region being learned using a scalable variational Gaussian process classifier (VGPC). The calibration system according to claim 2.

5. The parameter-cost regressor is iteratively trained using Bayesian optimization, the Bayesian optimization including a cost / reward function model that uses a Gaussian process (GP) to calculate a probabilistic GP surrogate model for providing a probabilistic parameter-cost mapping. The calibration system according to claim 4.

6. The at least one processor is further configured to obtain predetermined conditions regarding the failure region, and configured to iteratively estimate the failure region until the estimated failure region satisfies the predetermined conditions regarding the failure region. The calibration system according to claim 4.

7. The prior function of the VGPC is initialized based on the training data and a user-specified kernel function. The calibration system according to claim 5.

8. The Bayesian optimization includes an acquisition function that utilizes the probabilistic mapping provided by the probabilistic GP surrogate model to direct queries of different combinations of parameters resulting therefrom. The calibration system according to claim 5.

9. The Bayesian optimization constructs the acquisition function using the mean and variance of the surrogate GP model, and selects the combination of the parameters that is most likely to minimize the calibration error according to the probabilistic parameter-cost mapping, for the calibration system according to claim 5.

10. The acquisition function includes an expected improvement (EI) acquisition function, or the user-specified kernel function includes at least one of a squared exponential kernel or a Matérn kernel, for the calibration system according to claim 7.

11. The at least one processor is further configured to obtain the probability of simulation failure across the range of acceptable parameter values in the selected combination of the parameters using the scalable VGPC, The obtained probability of simulation failure is used by the acquisition function to form a failure-robust EI (FREI) acquisition function, for the calibration system according to claim 8.

12. A calibration method for calibrating a model of the dynamics of an industrial system, wherein the model describes the change in the state of the industrial system in response to the control of the industrial system according to a sequence of control inputs for fitting to operating data, and the operating data represents measured values of the operation of the industrial system including the values of the control inputs and the corresponding values of the state of the industrial system, and the calibration method includes simulating the operation of the industrial system a plurality of times, each simulation cycle including estimating the success or failure of the simulation by executing the model of the industrial system having different combinations of the parameter values selected from within the range of acceptable parameter values, and the calibration method further includes Training a failure classifier that defines the likelihood of failure of the operation of the industrial system for the range of acceptable parameter values of the model using training data that includes combinations of values of the selected parameters labeled with the presumed success or failure of the simulation. Training a parameter-cost regressor for probabilistic parameter-cost mapping between different combinations of values of the parameters of the model of the industrial system and their corresponding calibration errors, where the calibration error is based on the simulated state of the industrial system for different combinations of parameter values and the corresponding state of the industrial system in the operation data, and the parameter-cost regressor is repeatedly trained until an end condition is met. To perform the iteration, the calibration method Executing an acquisition function of the first two order moments of the calibration error to identify the combination of parameters that is most likely to minimize the calibration error according to the probabilistic parameter-cost mapping. Updating the combination of the identified parameters by adjusting the identified parameters according to the failure classifier using a failure-robust acquisition function to reduce the likelihood of failure of the operation of the industrial system using the model with the updated parameters. Generating a simulated state of the industrial system by simulating the operation of the industrial system controlled by the control input in the operation data using the model with the updated parameters. Updating the probabilistic parameter-cost mapping based on the updated parameters, the calibration error between the simulated state of the industrial system and the corresponding state of the industrial system in the operation data. The calibration method further When the end condition is satisfied, calibrating the model of the industrial system by using the optimal combination of the parameters that is most likely to minimize the calibration error in the probabilistic parameter-cost mapping according to the acquisition function. **Claim 13** Further comprising estimating a failure region within the range of acceptable parameter values in the selected combination of parameters, the failure region including at least one combination of parameter values that causes a failure in the operation of the industrial system, and further, The calibration method according to claim 12, further comprising providing a probability of simulation failure across the entire range of acceptable parameter values in the selected combination of parameters based on the estimated failure region. **Claim 14** To obtain the failure region, the calibration method further Comprises obtaining training data for learning the failure region by sampling from within the range of acceptable parameters, the failure region being learned using a scalable variational Gaussian process classifier (VGPC). The calibration method according to claim 13. **Claim 15** The parameter-cost regressor is iteratively trained using Bayesian optimization, the Bayesian optimization including a cost / reward function model that uses a Gaussian process (GP) to calculate a probabilistic GP surrogate model for providing probabilistic parameter-cost mapping. The calibration method according to claim 13. **Claim 16** The method further Obtaining a predetermined condition regarding the failure region, and Iteratively estimating the failure region until the estimated failure region satisfies the predetermined condition regarding the failure region. The calibration method according to claim 13.

Images