Methods for secure training of dynamic models

Abstract

The present invention relates to a computer-implemented method for safely and actively training a computer-aided model for modeling the time series of a physical system using a Gaussian process, the method comprising the following steps: specifying a safety threshold α; initializing by executing a safe initial curve as an input value on the system; creating an initial regression model and an initial safety model; repeatedly implementing the steps of updating the regression model, updating the safety model, determining a new curve segment, executing the determined new curve segment on the physical system and measuring output values, and incorporating the new output values ​​into the regression model and the safety model; until N cycles have been implemented; and outputting the regression model and the safety model.

Description

Technical Field

[0001] The present invention relates to a method for active learning using safety conditions for modeling dynamic systems on the basis of Gaussian processes with the aid of time series; a system trained in this way; a computer program comprising instructions, which are designed to perform the method when the computer program is executed on a computer; a machine-readable storage medium on which the computer program is stored; and a computer designed to perform the method. Background Art

[0002] Safe exploration in active learning is known from J. Schreiter, D. Nguyen-Tuong, M. Eberts, B. Bischoff, H. Markert, and M. Toussaint, “Safe Exploration for Active Learning with Gaussian Processes” (ECML / PKDD, Vol. 9286, 2015). Specifically, point-by-point data is detected in a stationary state.

[0003] Active learning involves sequential data identification to learn an unknown function. Data points are selected sequentially for identification to maximize the availability of information needed to approximate the unknown function. The general goal is to create an accurate model without providing more information than necessary. This makes the model more effective because potentially expensive measurements can be avoided.

[0004] Active learning is widely used for data classification, for example for image labeling. For active learning in time series models representing physical systems, the data must be generated in such a way that the relevant dynamic processes can be detected.

[0005] This means that the physical system must be stimulated by the dynamic motion within the input range via the input curve in such a way that the collected data, i.e., the input and output curves, contain as much information as possible about the dynamics. Examples of usable input curves include, in particular, sine, ramp, and step functions, as well as white noise. However, when stimulating the physical system, additional safety requirements must be taken into account. When dynamically exploring the input range, the stimulation must not damage the physical system.

[0006] It is therefore important to identify regions within which dynamic excitation can be performed safely. Summary of the Invention

[0007] Compared to the prior art, the method proposed in this application is a computer-implemented method for safely and actively training a time-series-based model of a physical system using a Gaussian process. The method comprises the following steps: specifying a safety threshold α; initializing by executing a safe initial curve on the system as input, creating an initial regression model and an initial safety model; repeatedly performing the following steps: updating the regression model; updating the safety model; determining a new curve segment; executing the determined new curve segment on the physical system and measuring the output value; incorporating the new output value into the regression model and the safety model; until N cycles have been performed; and finally updating and outputting the regression model and the safety model. This method has the advantage of combining dynamic exploration, active exploration, and safe exploration.

[0008] In this context, "dynamic exploration" refers to the acquisition of information under changing conditions in the system to be measured. The goal of active exploration is to acquire information as quickly as possible, performing this acquisition as sequentially as possible, thereby enabling the acquisition of a large amount of information in a short period of time. In other words, the information yield from a single measurement is maximized. Finally, safe exploration ensures minimal damage to the system to be measured.

[0009] With the method according to the invention, these three types of exploration can be combined.

[0010] By means of the measures listed below, advantageous expansions and improvements of the method proposed in the present application mentioned above can be achieved.

[0011] The present invention discloses an active learning environment with dynamic exploration (active learning) for time series models based on Gaussian processes, which takes safety into account by deriving suitable criteria for the dynamic exploration of input regions.

[0012] Active learning is useful in many applications, such as simulation and predictive computing. The goal of a learning method is typically to create a model that describes reality. To this end, a real process, system, or object (hereinafter referred to as the target) is measured to detect information about the target. The created model of reality can then be used in place of the target in the simulation or predictive computing. This approach offers the advantage of saving resources, meaning that the process does not need to be repeated, as this typically consumes resources, or the object or system is not subjected to the process to be simulated and could be damaged, destroyed, or altered.

[0013] Advantageously, the model describes reality as accurately as possible. In the present invention, it is particularly advantageous to be able to use active learning while taking into account safety conditions. These safety conditions should ensure that the target to be detected is affected as little negatively / severely as possible, for example, in the sense of damage to the object or system.

[0014] The present invention uses a Gaussian process with a time series structure, for example, a nonlinear exogenous structure or a nonlinear autoregressive exogenous structure. Dynamic exploration of the input region generates input and output curves or output measurements suitable for the time series model. The output measurements, in other words, data identifiers, serve as information for the time series model. The input curve is parameterized as successive curve regions, for example, successive segments of a ramp function or step function, which are determined step by step using an exploratory approach based on given safety requirements and previous observations.

[0015] Taking into account the previous observations, the respective subsequent segment is determined in such a way that the information gain with respect to the criteria of the model is maximized.

[0016] A Gaussian process with a nonlinear exogenous structure and a suitable heuristic criterion is used as the time series model. A separate Gaussian process model is used to predict the safe input range for given safety requirements. The input curve segments are determined by solving an optimization problem with additional conditions to account for the safety prediction.

[0017] An exemplary application of the present invention is, for example, a test bench for internal combustion engines, where the processes within the machine must be simulated. Parameters to be measured include, for example, pressure values, exhaust gas volumes, consumption values, and power values. Another application is learning a dynamic model for a robot controller, where a dynamic model is learned that maps joint positions to the robot's joint torques, which can be used to control the robot. This model can be learned actively through exploration of the joint area, but this must be done in a safe manner to avoid exceeding the joint's motion limits, which could damage the robot. Another application is learning a dynamic model that replaces physical sensors. Data used to learn this model can be generated and measured on a physical system through active exploration. Safe exploration is essential, as measurements in unsafe areas could damage the physical system. Another application is learning the properties of chemical reactions, where safety requirements may involve parameters such as temperature, pressure, and acidity. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Embodiments of the present invention are shown in the drawings and explained in detail in the following description.

[0019] Figure 1 A flow chart 100 of a method for securely training a computer-assisted model is shown;

[0020] Figure 2 A flow chart 200 of a method for securely training a computer-assisted model is shown. DETAILED DESCRIPTION

[0021] Should implement unknown functions In the case of time series models, such as the well-known nonlinear exogenous (NX) models, the input range consists of discrete values, so-called conditioning variables.

[0022] Use x k For time k, the following applies: in, represents a discrete regulation curve. is the dimension of the input region Π of the system, is the dimension of the NX structure and is the dimension of X.

[0023] Element u k It is measured by a physical system and does not have to be equidistant. For reasons of simpler notation, equidistant is taken as an example. Usually, the control curve is a continuous signal and can be clearly controlled.

[0024] In the learning environment of the model, n successive curve segments are The data is observed in the form of the input curve τ i is a matrix consisting of m input points of dimension d, that is, Output curve ρ i contains m corresponding output measurements, that is:

[0025] The next curve segment τ to be input as stimulus to the physical system should now be determined as follows n+1 , thereby increasing the information gain in the modeling of f while taking into account safety conditions

[0026]

[0027] In order to approximate the function f, a Gaussian process (hereinafter abbreviated as GP) is used, which is represented by its mean function μ(x) and its covariance function k(xi , x j ), that is to say, to stipulate

[0028] Assuming noisy observations of the input and output curves, the common distribution according to the Gaussian process is given by in is the vector connecting the output curves and is the matrix containing the input curve. The covariance matrix is ​​obtained by To illustrate, we use a Gaussian kernel as the covariance function, that is, Its passage In addition, the zero vector Assuming it is the average value, the nm-dimensional identity matrix I and σ 2 Assume that is the output noise variance.

[0029] Given a common distribution, the predicted τ can be used for a new curve segment τ * Distribution Expressed as

[0030] [Equation A:]

[0031] in,

[0032] [Equation B:]μ(τ * )=k(τ * ,T n ) T (K n +σ 2 I) -1 P n

[0033] [Equation C:]∑(τ * )=k ** (τ * ,τ * )-k(τ * ,T n ) T (K n +σ 2 I) -1 k(τ * ,T n ),

[0034] in, is a matrix, where In addition, the matrix Contains the core evaluation of τw with respect to the previous n input curves. Since the covariance matrix is ​​fully filled, the input point x is not only fully associated with the curve segment but also transcends different curves when using the correlation for planning the next curve. Since the matrix K n +σ 2 I may have high dimensionality n·m, so its inversion may be time-consuming, and therefore GP-Approximations techniques can be used.

[0035] The safety state of the system is described by an unknown function g, where The unknown function assigns a safety value z to each input point x, which serves as a safety indicator. The value z is determined using information from the system and is set such that for all values ​​of z greater than or equal to zero, the corresponding input point x is considered safe.

[0036] Such a safety value z depends on the corresponding system and can, as explained above, embody system-dependent values, which are used for safe or unsafe pressure values, exhaust gas volumes, consumption values, power values, joint position values, movement limits, sensor values, temperature values, acidity values, etc.

[0037] The value of z is usually continuous and indicates the distance between a given point x and the unknown safety limit within the input region. Therefore, a given function g or its estimate can be used to determine the safety level for the curve τ. A curve is classified as safe if the probability that the safety value z of the curve is greater than zero is sufficiently high, that is, Where α∈[0,1] represents the threshold at which τ is unsafe. in GP can be used to approximate the function g. For a given curve segment τ * The predicted distribution of is calculated as:

[0038] [Equation D:] where μ g (τ * ) and ∑ g (τ * ) are the corresponding mean and covariance values. The variable μ g and ∑ g Calculated as shown in Equation 2 or 3, but where As all i The target vector is connected. By using GP to approximate g, the safety condition ξ(τ) for the curve τ can be calculated as follows:

[0039] [Equation E:]

[0040] Usually, the calculation of ξ(τ) is difficult to solve by analytical methods and therefore some kind of approximation, such as Monte Carlo simulation or expectation propagation (English: “Expectation Propagation”), can be used.

[0041] In order to effectively select the optimal τ, the curve must be parameterized in a suitable manner. One possibility is to already perform the parameterization in the input region. The parameterization of the curve can be implemented, for example, as a ramp or step function.

[0042] Using the predicted distribution according to equation A and the safety condition according to equation E for curve parameterization, the next curve segment τ can be obtained by solving the following optimization problem with the additional condition n+1 (η * ):

[0043] [Equation F:]

[0044] [Equation G:] Thus ξ(η)>1-α,

[0045] where η∈Π denotes the curve parameterization and Indicates the best standard

[0046] According to Equation F, the prediction variance μ from Equation A is used for exploration. The prediction variance is the covariance matrix, which is obtained by the best standard is mapped to real numbers as shown in Equation F. For For example, different optimal criteria can be used depending on the system. For example, these can be: the determinant, that is, equivalent to maximizing the volume of the prediction confidence ellipsoid of the multinormal distribution; the trace, that is, equivalent to maximizing the average prediction variance; or the largest eigenvalue, that is, equivalent to maximizing the largest axis of the prediction confidence ellipsoid. However, other optimization criteria can also be considered.

[0047] refer to Figure 1 In step 120, initialization is performed by executing n0 safe initial curves. Here, a regression process and a safe process, a Gaussian process, are also created. The initial curve is located in a small safe region, within which the exploration begins. This small safe region is selected in advance based on prior knowledge of the system. The initial curve is obtained by To determine, where n = n0.

[0048] Then, in step 160, a new curve segment τ is determined by optimizing > according to equations F and G. n+1 .

[0049] Next, in step 170, the determined curve segment τ n+1 is used as input and ρ is measured on the physical system in this region n+1 and ζ n+1 .

[0050] Thereafter, in step 150, the regression process and the safety process are updated. Update the regression model f under the condition of , and use Equation D to Update the security model g.

[0051] Here, steps 150 to 170 are looped N times. In addition to the previously specified number of loops, automatic termination after reaching a termination criterion is also conceivable. The termination criterion may be based, for example, on the training error (a measure of the error in the model predictions and the system response) or on the additional possible information gain (if the optimality criterion becomes too small).

[0052] Following this, the regression model and the safety model are output in step 190 .

[0053] refer to Figure 2 The implementation 200 of the method is explained. In step 210, a safety threshold is defined. Here, a value between 0 and 1 is selected for α. Then, in step 220, initialization is performed by executing n0 safe initial curves, Here, n=n0. A regression process and a safety process, a Gaussian process, are also created. The initial curve lies within a small safety region, within which the exploration begins. This small safety region is selected in advance based on prior knowledge of the system.

[0054] Next, the part of the method comprising steps 240 to 280 is executed N times, where k is a running variable, that is, the current cycle is displayed. Figure 1 As in the example, in addition to the previously specified number of cycles, automatic termination after reaching a termination criterion can also be considered. The termination criterion can be based on the training error (error measure of model prediction and system response) or on additional possible information gain (if the optimality criterion becomes too small).

[0055] First, in step 240, according to equation A, using In step 250, the regression model f is updated according to equation D using In the case of , the security model g is updated. In the first cycle of steps 240 to 280, steps 240 and 250 can be omitted.

[0056] Then, in step 260, a new curve segment τ is determined by optimizing η according to equations F and G. n+1 .

[0057] Thereafter, in step 270, the determined curve segment τ n+1 is used as input and ρ is measured on the physical system in this region n+1 and ζ n+1 .

[0058] Then, in step 280, the input and output curves processed in the previous step are added to or superior.

[0059] After the repetition of steps 240 to 280 is completed, step 290 is performed in which the regression model and the safety model are updated and output.

[0060] For example, by updating the ranking of the matrix (rank-one update), the incremental update of the GP model for new data, that is, steps 150 or 240 and 250, can be effectively implemented. Although the NX structure is described here in conjunction with the GP model for time series modeling, it is also possible to use the general exogenous case of nonlinear autoregression, that is, the GP with the NARX input structure, where x k =(y k ,y k-1 ,...,y k-q ,u k ,u k-1 ,....,u k-d ). In order to optimize and plan the next curve segment, for example, the predicted mean value can be used Used for y k However, in the case of NARX, the input excitation of the system is controlled by the regulation quantity u k To implement.

Claims

1. A computer-implemented method (200) for safely and actively training a time series-based model of a physical system using a Gaussian process, the method comprising the following steps: Specify (210) a safety threshold α; Initializing (220) by executing a secure initial curve as an input value on the system, creating an initial regression model and an initial security model; Repeat the following steps: updating (240) the regression model; updating (250) the security model; determining (260) a new curve segment; executing (270) the determined new curve segment on the physical system and measuring an output value; Including (280) new output values ​​into the regression model and the security model; Until N cycles have been implemented; and The regression model and the security model are updated and output (290). 2 . The method according to claim 1 , wherein the new curve segment is determined ( 260 ) in such a way that the information gain is maximized while satisfying safety criteria of the safety model. 3 . The method according to claim 1 , wherein the new curve segment is determined ( 260 ) using a covariance matrix.

4. The method according to claim 1 or 2, wherein the new curve segment is determined (260) by optimization under additional conditions.

5. The method according to claim 1 or 2, wherein the system is a test bench for an internal combustion engine, a robotic control mechanism, a physical sensor or a chemical reaction.

6. A method according to claim 1 or 2, wherein the safety model includes the values ​​of the system and more precisely includes at least one value among pressure value, exhaust gas volume, consumption value, power value, joint position value, movement limit, sensor value, temperature value and acidity value.

7. A computer-aided model, which is trained using the method according to any one of claims 1 to 6. 8 . Computer program product, comprising a computer program, the computer program comprising instructions which are designed to perform the method according to claim 1 when the computer program is executed on a computer.

9. A machine-readable storage medium on which the computer program according to claim 8 is stored. 10 . A device configured to carry out the method according to claim 1 .

Images