Method for processing variance of Gaussian process prediction of embedded system

Abstract

The invention relates to a method for processing variance of Gaussian process prediction of an embedded system. The invention further relates to a computer program, a device and a storage medium for this purpose.

Description

Technical Field

[0001] The invention relates to a method for processing the variance of a Gaussian process prediction of an embedded system and further relates to a computer program, a device and a storage medium for this purpose. Background Art

[0002] Embedded systems are used in many technical environments. For example, embedded systems are also provided in vehicles in order to perform various monitoring, control or regulation functions. Furthermore, embedded systems can also be used to perform calculations or predictions using models such as Gaussian processes, which can be based on sensor data from the vehicle.

[0003] Gaussian processes (GP) can be used in particular as a probabilistic method for nonparametric regression. For example, in embedded systems, special hardware accelerators exist in the prior art that allow efficient calculation of the expected value of a Gaussian process prediction.

[0004] The computational and storage requirements of the predictions are scaled in particular with O(M), where M is the number of control points. The number of control points can be a hyperparameter and can be freely selected in an application-specific manner, wherein the number of control points can influence the accuracy of the approximation. In particular, the control points are located in the same space as the data input used. For example, there are also solutions in which the control points are located in different spaces.

[0005] To accurately compute the variance of the prediction, the memory and runtime requirements may be in the range of O(M 2 ), and its computation is therefore not supported by current hardware accelerators. In particular, the standard algorithm for computing the variance of the GP prediction uses O(M 2 ) and is therefore not suitable for embedded systems. For example, there are applications that require predictions of uncertainty. An example of this is safety-critical applications. For this purpose, a second model, a so-called watchdog, is usually trained, which decides whether the KI model is credible. However, this may lead to problems if the GP and the watchdog estimate the uncertainty differently. It may therefore be better to use the GP variance directly.

[0006] The article by ELENKOV, Martin [et al.]: Estimation Methods for Viscosity, Flow Rate and Pressure from Pump-motor Assembly Parameters. In: Sensors, Vol. 20, 2020, S. 1451 describes an estimation method for blood flow rate, pressure difference and viscosity using a Gaussian process regression model. Summary of the invention

[0007] The subject matter of the invention is a method having the features of claim 1, a computer program having the features of claim 8, a device having the features of claim 9 and a computer-readable storage medium having the features of claim 10. Further features and details of the invention are derived from the respective dependent claims, the description and the drawings. Features and details described in conjunction with the method according to the invention naturally also apply in conjunction with the computer program according to the invention, the device according to the invention and the computer-readable storage medium according to the invention, and vice versa, so that with respect to the disclosure for the individual inventive aspects, reference is always made to one another or can be made to one another.

[0008] The subject of the invention is in particular a method for processing the variance of a Gaussian process prediction of an embedded system, said method comprising the following steps:

[0009] - Provide sensor data from embedded systems,

[0010] - performing a Gaussian process prediction, in which the expected value of the target variable is determined based on the sensor data,

[0011] - determining at least one sample, preferably at least one Monte Carlo sample, based on the sensor data in order to determine a variance of the determined expected value using the at least one sample,

[0012] - classifying the Gaussian process prediction based on the analysis of the determined variance, wherein at least one class of the classification classifies the accuracy and / or inaccuracy of the Gaussian process prediction, and preferably classifies the Gaussian process prediction as inaccurate,

[0013] - Initiate measures based on the classification predicted by the Gaussian process.

[0014] The embedded system can be, in particular, a computer or microcomputer, such as in a vehicle, which is incorporated into a technical context. Here, the computer or microcomputer can, for example, take over a monitoring, control or regulation function or can be responsible for a form of data or signal processing. Gaussian process prediction preferably involves estimating or predicting the value of a function or process at one or more unobserved points based on existing control points. In particular, a Gaussian process is a probability model used for estimating a function. A Gaussian process models a function as a distribution over possible functions, and can advantageously provide not only an estimate of the function variation process but also information about the uncertainty of the estimate. In particular, a Monte Carlo sample is a type of sample used in a Monte Carlo simulation. Monte Carlo simulation is preferably a method of using random numbers to generate numerical results and solve complex problems that may be difficult or impossible to solve. In the context of the present invention, a Monte Carlo sample can be an example of a function extracted from a Gaussian process. Each sample can represent a possible function, which can describe given data. By extracting and analyzing many of these samples, it can be helpful to gain an understanding of which functions are possible or more targeted and which are not. This method can advantageously assess whether a Gaussian process prediction is plausible, which is preferably expressed by a classification as inaccurate. In addition, the use of samples can advantageously reduce the computational effort required for this purpose.

[0015] It is also conceivable that the sensor data are obtained from measurements of at least one virtual sensor, wherein the target variable is a measured variable predicted based on the sensor data by a Gaussian process prediction. A virtual or indirect sensor is preferably a software function, with which a variable that cannot be measured or is difficult to measure can be derived or estimated from other measurable or available sensor data and information. Thus, a virtual sensor is in particular not a real sensor, but a simulation of the correlation of a representative measured variable with respect to a target variable. Alternatively, the sensor data can also be obtained from measurements of a real sensor.

[0016] Gaussian process prediction can be performed using a large number of initial functions. The initial functions can be so-called prior functions that reflect the original knowledge or original assumptions about the function before evidence-based updates through observed data.

[0017] The method may further comprise the following steps:

[0018] - provide control points in the sensor data,

[0019] - Extract the function value of the control point,

[0020] - select a function from a large number of initial functions,

[0021] - determining a deviation between the function value at the control point and the result of the application of the selected function in order to provide a correction term for the selected function based on the deviation,

[0022] - Perform Gaussian process prediction using the correction term and the selected function.

[0023] The control points may be data points in the sensor data that are used to predict the Gaussian process. The control points are in particular located in the same space as the sensor data, but do not have to be selected from a set of data points in the sensor data. For example, there are also schemes in which the control points are in different spaces. The number of control points may vary depending on the application. More control points may allow for a more accurate estimate, but also require more computing power. Fewer control points may result in a smoother prediction, but may also collect less detail.

[0024] Here, the above steps may be performed for each sample in the at least one sample. In addition, the step of performing Gaussian process prediction may be performed by the embedded system, and the remaining steps may be performed by an external data processing device. By dividing the steps, the required computing power of the embedded system may be advantageously reduced.

[0025] According to an advantageous development of the invention, provision can be made that within the scope of the analysis, an evaluation of the exceeding of a defined threshold value of the variance is evaluated, wherein the exceeding of the defined threshold value is specific to an interpolation or an extrapolation, wherein in the presence of an extrapolation the Gaussian process prediction is classified as inaccurate. By identifying whether an interpolation or an extrapolation is present, it can be determined, for example in a simplified manner, whether the sensor data are close to the training data, with which the Gaussian process prediction has been trained. If the sensor data are far from the training data, it is therefore particularly possible that the Gaussian process prediction is unreliable or inaccurate.

[0026] Preferably, it can be provided that the measure comprises at least one of the following steps:

[0027] - reject Gaussian process predictions,

[0028] - Trigger an alert so that the user is informed that the Gaussian prediction was classified as inaccurate.

[0029] In the event that a Gaussian process prediction is rejected, a remeasurement of the sensor data can be triggered, for example, or an alternative model can be selected. For example, an alert can be determined for the driver of the vehicle so that the driver can take measures or can visit a repair or service point.

[0030] It is conceivable that performing the Gaussian process prediction further comprises the following steps:

[0031] The approximation is performed by means of a linear model in order to reduce the computational effort for determining the variance of the expected value by means of the approximation.

[0032] The Gaussian process function predicted by the Gaussian process can be approximated or approximated by a linear model

[0033]

[0034] This interpretation allows, in particular, efficient determination of approximate solutions for the corresponding expected value and variance predictions.

[0035] Furthermore, it is conceivable to perform the Gaussian process prediction using a machine learning model, wherein the machine learning model is based on a training comprising the following steps:

[0036] - providing training data, wherein the training data represent sensor data obtained by measurements of at least one sensor,

[0037] - provide control points in the training data so that they provide reference points for training the machine learning model,

[0038] -Training a machine learning model, wherein the machine learning model is trained so that the machine learning model determines a function so as to determine an expected value of a target parameter based on training data.

[0039] Preferably, within the scope of the invention, provision can be made for the method to be applied in the case of a vehicle or a robot or a technical system, wherein the embedded system is arranged in the vehicle or the robot or the technical system. The vehicle can be designed, for example, as a motor vehicle and / or a passenger vehicle and / or an autonomous vehicle. The vehicle can have a vehicle device, for example, for providing an autonomous driving function and / or a driver assistance system. The vehicle device can be implemented for at least partially automatically controlling and / or accelerating and / or braking and / or steering the vehicle.

[0040] The subject of the present invention is also a computer program, in particular a computer program product, which comprises instructions which, when executed by a computer, cause the computer to carry out the method according to the invention. The computer program according to the invention thus brings the same advantages as have already been described in detail with respect to the method according to the invention.

[0041] The subject matter of the present invention is also a device for data processing which is set up to carry out the method according to the invention. For example, a computer can be provided as the device, which executes the computer program according to the invention. The computer can have at least one processor for executing the computer program. A non-volatile data memory can also be provided, in which the computer program is stored and from which the computer program can be read out by the processor for execution.

[0042] The subject of the present invention may also be a computer-readable storage medium having a computer program according to the present invention and / or comprising instructions which, when executed by a computer, cause the computer to perform the method according to the present invention. The storage medium is, for example, constructed as a data memory, such as a hard disk and / or a non-volatile memory and / or a memory card. The storage medium may, for example, be integrated into a computer.

[0043] Furthermore, the method according to the present invention can also be implemented as a computer-implemented method. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] Further advantages, features and details of the invention are apparent from the following description, in which exemplary embodiments of the invention are described in detail with reference to the accompanying drawings. The features mentioned in the claims and in the description may be essential to the invention individually or in any combination. Among them:

[0045] Figure 1 Schematic visualization showing a method, a device, a storage medium and a computer program according to embodiments of the present invention,

[0046] Figure 2 A schematic diagram showing a vehicle, an embedded system and sensors according to an embodiment of the present invention is shown. DETAILED DESCRIPTION

[0047] exist Figure 1 Schematically shown in FIG. 1 are a method 100 , a device 10 , a storage medium 15 and a computer program 20 according to an embodiment of the present invention.

[0048] In a first step 101 of a method 100 according to an embodiment, sensor data of an embedded system 1 is provided. In a second step 102 of the embodiment, a Gaussian process prediction is performed, wherein an expected value of a target variable is determined based on the sensor data. In a third step 103, at least one sample is determined based on the sensor data in order to determine the variance of the expected value using the at least one sample. The sample is preferably a Monte Carlo sample. In a fourth step 104, the Gaussian process prediction is classified based on an analysis of the determined variance, wherein at least one category of the classification classifies the Gaussian process prediction as inaccurate. In a fifth step 105, measures are initiated according to the classification of the Gaussian process prediction.

[0049] exist Figure 2 Schematically shows a vehicle 3, an embedded system 1 and a sensor 2. In one embodiment, the embedded system 1 is arranged in the vehicle 3 and is preferably connected to the sensor 2. The sensor 2 may be a virtual sensor 2.

[0050] One aspect of the invention relates in particular to an algorithm for embedded systems, which algorithm computes the variance of a Gaussian process (GP) in O(M). This algorithm is known, for example, from the work in [1], wherein the references are listed in square brackets at the end of this description. One aspect of the invention is in particular the implementation of this algorithm on an embedded system.

[0051] As mentioned above, one aspect of the present invention is to compute the variance of the GP prediction with O(M) computational and storage requirements. This preferably allows for simple application of the GP model to safety-critical applications in embedded systems. Thus if the variance prediction is too high, the system can advantageously react appropriately.

[0052] In particular, the exact computation and storage requirements for variance prediction are O(M 2 ). An approximate method for calculating the variance of Monte Carlo samples is described below. The extraction of Monte Carlo samples can advantageously scale with O(M).

[0053] There may be two target systems for using the possible method of the invention: a first system that can efficiently compute the mean (English: Mean) or expected value of the GP predictions as a black box function, and a second system that can perform a set of suitable elementary operations in a freely selectable order.

[0054] The calculation of the variance may advantageously allow a statement to be made as to whether the GP is acting in interpolation (near the training data) or in extrapolation (outside the training data).In the second case, a reaction may be triggered because the prediction is classified as no longer credible.

[0055] For example, in the prior art a watchdog is often used to determine whether the GP is in interpolation or extrapolation. However, this can be problematic because the mean prediction and the variance prediction are decoupled. This can lead to problems: for example, the watchdog is already in extrapolation while the GP is still in interpolation, whereby a reaction may be triggered too early and a false alarm may be triggered. In another case, the watchdog is still in interpolation while the GP is already in extrapolation, whereby an untrustworthy prediction may be used and thus again a safety risk may arise.

[0056] The invention shows in particular how an algorithm for determining the variance of a GP can be calculated in an embedded system [1]. Thus, if the uncertainty of the prediction is too great, an appropriate reaction can be triggered.

[0057] The invention is particularly relevant in the context of a "virtual sensor" when the output of the GP is used for regulation, control or monitoring applications (eg on-board diagnostics).

[0058] In one embodiment of the present invention, the following setting can be assumed: a training data set should be given (“train” means “training”), where is the input of the nth sample and is the output of the nth sample. The goal is preferably to learn the function The function takes as input x n Assigned to output y n Furthermore, a probabilistic setting can be assumed, in which the function f~GP(0, k) has a GP prior with zero mean. The kernel function is preferably given by The control points are given by And its probability output Y ind ~N(μ ind ,∑ ind ) characterization, where The number of control points M can be a hyperparameter and can be freely chosen by the user. The goal of the algorithm can be to * , efficiently determine the mean value E[y * ] and variance Var[y * ].

[0059] Standard prediction: The standard formula for sparse GP prediction is based on [2]:

[0060] E[y * ]=K(x * , X ind )α,

[0061] Var[y * ]=K(x * , x * )-K(x * , X ind )Ω(X ind , x * ),

[0062] Where α = K(X ind , X ind ) -1 μ ind and Ω=K(X ind ,X ind ) -1 (K(X ind , X ind )+∑ ind )K(X ind , X ind ) -1 The terms α and Ω can be precomputed since they are not related to the test point x. * After caching, calculate E[y *] or Var[y * ] has a storage and runtime complexity of O(M) or O(M 2 ). This is probably why current hardware accelerators only offer mean prediction as a function. Variance prediction might cost too many resources.

[0063] Interpretation as linear regression: Using the kernel trick according to [3], the kernel function k(·,·) can be expressed as the inner product in the reproducing kernel Hilbert space (RKHS) H

[0064] k(x,x′)=<ψ(x),ψ(x′)> H ,

[0065] in can be a mapping to a (possibly infinite) feature space. For many kernel functions, the inner product can be further approximated as

[0066] <ψ(x),ψ(x′)> H ≈φ(x) T φ(x′)

[0067] in Preferably it is a finite mapping. For example, this is possible for all fixed kernels, especially for the RBF kernel.

[0068] The GP function can then be obtained through the linear model

[0069]

[0070] In particular, this interpretation of GP allows efficient determination of approximate solutions for the mean and variance predictions. Here, it is preferred to first calculate the posterior distribution p(w|D train ), in order to subsequently determine the predicted probability p(y * |x * )=∫p(y * |w,x * )p(w|D train )dw.

[0071] The quality of the approximation can be compared with the training data set D as follows train The larger the data set, the worse the approximation works.

[0072] In particular, it is shown below how the mean and variance of GP predictions can be efficiently computed using Matheron's rule [1].

[0073] In particular, Matheron's rule shows that, instead of drawing samples directly from the posterior, one can also draw samples from the prior and then correct them appropriately:

[0074] y * |y ind =f prior (x * )+k(x * , X ind )K(X ind , X ind ) -1 (y ind -f prior (X ind )),

[0075] where y ind ~N(μ ind ,∑ ind )und f prior ~GP(0,k(·,·)).

[0076] Thus, Monte Carlo samples can be generated from the GP posterior as follows:

[0077] 1. First, you can extract the control point: y ind ~N(μ ind ,∑ ind ),

[0078] 2. Then we can extract the function from the prior: f prior ~GP(0, k(·, ·))(“prior” means “prior”),

[0079] 3. Then we can determine β = K(X ind , X ind ) -1 (y ind -f prior (X ind )),

[0080] 4. For the test point x * , then we can calculate the Monte Carlo prediction: * =f prior (x * )+k(x * , X ind )β

[0081] By drawing S Monte Carlo samples, the variance of the GP prediction can be determined.

[0082] Such a calculation may be advantageous in several respects: Steps 1-3 must in particular be calculated only once (offline). Preferably, the single step that must be calculated in the embedded system is step 4.

[0083] Step 2 in particular requires extracting a function from the GP prior. This may only be possible conditionally if the standard GP formula is used. However, if the Gaussian process is approximated as a linear regression, it is preferably simple, as already explained above: after extracting the weights w and the feature map φ, the function can be evaluated in O(L) at any input. Step 4 preferably consists of two parts: the prior and the correction term. The calculation criterion for the correction term is in particular the same as for the standard mean prediction and scales in O(M). Special hardware accelerators can be used for this calculation.

[0084] How to use the algorithm on an embedded system is described below according to an embodiment.

[0085] In offline mode, for example the following calculations cannot be performed on the target hardware.

[0086] The input can be a kernel function: k(·,·), an inductive input: X ind , distribution on the inducing output: y ind ~N(μ ind ,∑ ind ) and the number of Monte Carlo samples: S.

[0087] The following should be an example of the pseudo code for the calculation:

[0088] FOR s=1,...SDO

[0089] Sample inducing outputs:

[0090] Sample a fumction from the prior:w (s) ~N(0,1),φ (s) ~p(φ)

[0091]

[0092]

[0093] DONE

[0094] In online mode, the following calculations can be performed on the target hardware.

[0095] The input can be a kernel function: k(·,·), an inductive input: X ind , test input (English: Testinput) x *, the number of Monte Carlo samples: S, distribution (English Mapping) and weight

[0096] According to one embodiment, case 1 is an embedded system that can calculate the mean value (English Mean) or expected value of the GP prediction as a black box function. In other words, in this case, there is especially an embedded system that is only supplemented with a variance calculation.

[0097] The calculation shall be described below by means of an exemplary pseudocode:

[0098] FOR s=1,...SDO

[0099]

[0100] Computek * = k(x * , X ind )(on accelerated hardware)

[0101] (on accelerated hardware)

[0102]

[0103] DONE

[0104] Memory: O(SL+MD+SM)

[0105] Running time: O(SL+SMD)

[0106] According to one embodiment, case 2 is an embedded system that can execute a set of suitable basic operations in a freely selectable order. In other words, in this case there is especially an embedded system that itself calculates or can calculate all the calculations of the present invention.

[0107] The following should describe the calculation via pseudocode:

[0108] Computek * = k(x * , X ind )

[0109] FOR s=1,...,SDO

[0110]

[0111]

[0112]

[0113] DONE

[0114] Memory: O(SL+MD+SM)

[0115] Running time: O(SL+MD+SM)

[0116] In particular, within the scope of the present invention, it is assumed that a GP with a mean value or expected value of zero (zero mean in English) is used: f ≈ GP (0, k).

[0117] In possible other applications, a Gaussian process with a non-zero mean or expected value (English: Non-Zero Mean), f~GP(m, k), can be used, where the mean function For example from domain knowledge.

[0118] The mean or expected value forecast can then preferably be calculated by the following formula

[0119] y * =m(x * )+k(x * , X ind )K(X ind , X ind ) -1 (y ind -m(x * )).

[0120] In hybrid modeling (combining the KI model with physical prior knowledge), this type of model is especially also called "Delta model": the GP can correct the predictions of the physical model. If the test input x * Stay away from X ind , then k(x * , X ind ) preferably tends to zero and takes over the physical model. At the same time, the variance of the prediction may increase.

[0121] It can be determined that this formula is very similar to the Monte Carlo prediction formula given above, where m(x * ) shall be responsible for, prior (x * This may allow to interpret the algorithm as a mixture model with a stochastic physical model, where the linear approximation of the GP prior plays the role of the physical model.

[0122] The above description of the embodiments describes the present invention only in the scope of examples. Of course, if technically reasonable, the individual features of the embodiments can be freely combined with each other without departing from the scope of the present invention.

[0123] [1] Wilson, James et al. "Efficiently sampling functions from Gaussian process posteriors." International Conference on Machine Learning. 2020.

[0124] [2] Titsias, Michaelis. “Variational learning of inducing variables in sparse Gaussian processes” Artificial Intelligence and Statistics, 2009.

[0125] [3] Bernhard et al. “Learning with Kernels:Support Vector Machines, Regularization, Optimization and Beyond.” MIT press, 2001.

Claims

1. A method (100) for processing variance of a Gaussian process prediction of an embedded system (1), the method comprising the following steps: - providing (101) sensor data of the embedded system (1), - performing (102) said Gaussian process prediction, wherein an expected value of a target variable is determined based on said sensor data, - determining (103) at least one sample, preferably at least one Monte Carlo sample, based on the sensor data in order to determine a variance of the determined expected value using the at least one sample, - classifying the Gaussian process prediction based on an analysis of the determined variance (104), wherein at least one category of the classification classifies an inaccuracy of the Gaussian process prediction, - initiating (105) measures based on the classification predicted by said Gaussian process, The Gaussian process prediction is performed using a plurality of initial functions, wherein the method further comprises the following steps: - providing control points in said sensor data, - Extract the function value of the control point, - select a function from a large number of initial functions, - determining a deviation between the function value at said control point and the result of the application of the selected function in order to provide a correction term for the selected function based on the deviation, - performing said Gaussian process prediction using the correction term and the selected function, The above steps are performed for each of at least one sample. The step of performing the Gaussian process prediction is performed by the embedded system, and the remaining steps are performed by an external data processing device.

2. The method (100) according to claim 1, characterized in that: The sensor data are obtained from measurements of at least one virtual sensor (2), wherein a target variable is a measured variable predicted by a Gaussian process prediction based on the sensor data.

3. The method (100) according to any one of the preceding claims, characterized in that As part of the analysis, an analysis of the exceeding of a defined threshold value of the variance is evaluated, wherein the exceeding of the defined threshold value is specific to an interpolation or an extrapolation, wherein in the case of an extrapolation the Gaussian process prediction is classified as inaccurate.

4. The method (100) according to any one of the preceding claims, characterized in that The measures include at least one of the following steps: - reject the Gaussian process prediction, - Triggering an alert to prompt the user that the Gaussian prediction is classified as inaccurate.

5. The method (100) according to any one of the preceding claims, characterized in that Executing (102) the Gaussian process prediction further comprises the following steps: The approximation is performed by means of a linear model in order to reduce the computational effort for determining the variance of the expected value by means of the approximation.

6. The method (100) according to any one of the preceding claims, characterized in that The Gaussian process prediction is performed using a machine learning model, wherein the machine learning model is based on training, the training comprising the following steps: - providing training data, wherein the training data represent sensor data obtained by measurements of at least one sensor (2), - providing control points in the training data so as to provide reference points for training the machine learning model through the control points, -Training the machine learning model, wherein the machine learning model is trained so that the machine learning model determines a function in order to determine an expected value of the target parameter based on the training data.

7. The method (100) according to any one of the preceding claims, characterized in that The method (100) is used in a vehicle (3), wherein the embedded system (1) is arranged in the vehicle (3).

8. A computer program (20) comprising instructions which, when executed by a computer (10), cause the computer to perform the method (100) according to any one of the preceding claims.

9. A device (10) for data processing, which is configured to carry out the method (100) according to any one of claims 1 to 7.

10. A computer-readable storage medium (15) comprising instructions which, when executed by a computer (10), cause the computer to perform the method (100) according to any one of the preceding claims 1 to 7.

Images