Method for processing the variance of a Gaussian process prediction of an embedded system

Abstract

The invention relates to a method for processing a variance of a Gaussian process prediction of an embedded system. The invention further relates to a computer program, a device, and a storage medium for this purpose.

Description

[0001] The invention relates to a method for processing a variance of a Gaussian process prediction of an embedded system. Furthermore, the invention relates to a computer program, a device, and a storage medium for this purpose. State of the art

[0002] Embedded systems are used in a wide variety of technical environments. For example, embedded systems are also used in vehicles to perform various monitoring, control, or regulation functions. Furthermore, embedded systems also perform calculations or predictions using models such as the Gaussian process, which can be based on the vehicle's sensor data.

[0003] Gaussian processes (GP) can be used, in particular, as a probabilistic method for non-parametric regression. In embedded systems, for example, there are state-of-the-art hardware accelerators that allow the expected value of a Gaussian process prediction to be calculated efficiently.

[0004] The calculation and memory requirements of the prediction scale particularly with O(M), where M is the number of nodes. The number of nodes can be a hyperparameter and freely chosen for the specific application, whereby the number of nodes can influence the accuracy of the approximation. The nodes are, in particular, located in the same space as the data inputs used. There are also approaches, for example, in which the nodes are located in a different space.

[0005] For the exact calculation of the variance of the prediction, the memory and runtime requirements can be in O(M 2) and their calculation is therefore not supported by current hardware accelerators. Standard algorithms for calculating the variance of the GP prediction scale in particular with O(M 2 ) and are therefore not suitable for embedded systems. For example, there are applications that require uncertainty prediction. One example is safety-critical applications. For this, a second model, a so-called watchdog, is often trained to decide whether the AI ​​model is trustworthy. However, this can lead to problems if the GP and the watchdog assess uncertainty differently. Therefore, it may be better to use the GP variance directly. Disclosure of the invention

[0006] The subject matter of the invention is a method having the features of claim 1, a computer program having the features of claim 8, a device having the features of claim 9, and a computer-readable storage medium having the features of claim 10. Further features and details of the invention emerge from the respective subclaims, the description, and the drawings. Features and details described in connection with the method according to the invention naturally also apply in connection with the computer program according to the invention, the device according to the invention, and the computer-readable storage medium according to the invention, and vice versa, so that with regard to the disclosure of the individual aspects of the invention, reference is or can always be made to each other.

[0007] The invention particularly relates to a method for processing a variance of a Gaussian process prediction of an embedded system, comprising the following steps: - Providing sensor data from the embedded system, - Performing the Gaussian process prediction, whereby an expected value for a target variable is determined based on the sensor data, - Determining at least one sample, preferably at least one Monte Carlo sample, based on the sensor data in order to determine a variance for the determined expected value using the at least one sample, - Classifying the Gaussian process prediction based on an analysis of the determined variance, wherein at least one class of the classification classifies an accuracy and / or inaccuracy of the Gaussian process prediction and preferably classifies the Gaussian process prediction as inaccurate, - Initiate an action depending on the classification of the Gaussian process prediction.

[0008] An embedded system can in particular be a computer or microcomputer, for example in a vehicle, which is integrated into a technical context. The computer or microcomputer can, for example, perform monitoring, control or regulation functions or can be responsible for a form of data or signal processing. A Gaussian process prediction preferably refers to the estimation or prediction of the value of a function or process at one or more unobserved points based on the available support points. A Gaussian process is in particular a probabilistic model that is used to estimate functions. The Gaussian process models a function as a distribution over possible functions and can advantageously provide both an estimate of the function's course and information about the uncertainty of the estimate.Monte Carlo samples are, in particular, a type of sampling used in Monte Carlo simulation. Monte Carlo simulation is preferably a method that uses random numbers to generate numerical results and solve complex problems that may be difficult or impossible to solve. In the context of the present invention, Monte Carlo samples may be examples of functions drawn from the Gaussian process. Each sample may represent a possible function that could explain the given data. By drawing and analyzing many of these samples, an understanding can advantageously be gained of which functions are likely, or more accurate, and which are not. The method can advantageously be used to assess whether the Gaussian process prediction is trustworthy or not, which is preferably expressed by classifying it as inaccurate.In addition, the use of the sample can advantageously reduce the computational effort required.

[0009] It is also conceivable that the sensor data result from a measurement of at least one virtual sensor, with the target variable being a measured variable predicted by the Gaussian process prediction based on the sensor data. Virtual or indirect sensors are preferably software functions with which variables that are not or only with difficulty measurable can be derived or estimated from other, measurable or available sensor data and information. A virtual sensor is therefore not a real sensor, but rather a dependency simulation of representative measured variables on a target variable. Alternatively, the sensor data can also result from a measurement of a real sensor.

[0010] Gaussian process prediction can be performed using a variety of initial functions. These initial functions can be so-called prior functions, which reflect initial knowledge or assumptions about the functions before evidence-based updates are made using the observed data.

[0011] The method may further comprise the following steps: - Providing support points in the sensor data, - Dragging function values ​​of the support points, - Selecting a function from the multitude of initial functions, - Determining a deviation between the function values ​​of the support points and a result of an application of the selected function in order to provide a correction term for the selected function based on the deviation, - Perform the Gaussian process prediction using the correction term and the selected function.

[0012] The vertices can be data points in the sensor data that are used to predict the Gaussian process. The vertices are typically located in the same space as the sensor data, but do not have to be selected from the set of data points in the sensor data. For example, there are also approaches in which the vertices are located in a different space. The number of vertices can vary depending on the application. More vertices can enable a more accurate estimate, but also require more computing power. Fewer vertices can lead to a smoother prediction, but may also capture less detail.

[0013] The preceding steps can be performed for each of the at least one sample. Furthermore, the step of performing the Gaussian process prediction can be performed by the embedded system, and the remaining steps can be performed by an external data processing device. By splitting the steps, the required computing power of the embedded system can be advantageously reduced.

[0014] According to an advantageous development of the invention, it can be provided that, within the scope of the analysis, an exceeding of a defined threshold of the variance is assessed, wherein the exceeding of the defined threshold is specific for an interpolation or an extrapolation, wherein, if an extrapolation is present, the Gaussian process prediction is classified as inaccurate. By recognizing whether an interpolation or an extrapolation is present, it can be determined, for example, in simplified terms, whether the sensor data are close to the training data with which the Gaussian process prediction was trained. If the sensor data are far from the training data, it is therefore particularly likely that the Gaussian process prediction is unreliable or inaccurate.

[0015] Preferably, the measure may comprise at least one of the following steps: - Rejecting the Gaussian process prediction, - Raise an alarm to alert a user that the Gaussian prediction is classified as inaccurate.

[0016] If the Gaussian process prediction is rejected, a new measurement of sensor data can be triggered, for example, or a backup model can be switched to. The alarm can, for example, be intended for a vehicle driver, allowing the driver to initiate appropriate action or contact a repair or service center.

[0017] It is conceivable that performing the Gaussian process prediction further includes the following step: - Performing an approximation using a linear model in order to reduce the computational effort required to determine the variance for the expected value.

[0018] The Gaussian process function of the Gaussian process prediction can be represented by the linear model f(⋅)=∑l=1lwlϕ(⋅)l, w∈N(0,1) approximated. This interpretation allows, in particular, to efficiently determine an approximate solution for a corresponding expected value and variance prediction.

[0019] Furthermore, it is conceivable that the Gaussian process prediction is performed using a machine learning model, whereby the machine learning model is based on a training that includes the following steps: - Providing training data, wherein the training data represents sensor data resulting from a measurement of at least one sensor, - Providing support points in the training data to provide reference points for training the machine learning model, - Training the machine learning model, whereby the machine learning model is trained in such a way that the machine learning model determines a function to determine the expected value of the target variable based on the training data.

[0020] Preferably, within the scope of the invention, it can be provided that the method is applied to a vehicle, robot, or technical system, wherein the embedded system is arranged in the vehicle, robot, or technical system. The vehicle can be designed, for example, as a motor vehicle, passenger vehicle, and / or autonomous vehicle. The vehicle can have a vehicle device, for example, for providing an autonomous driving function, and / or a driver assistance system. The vehicle device can be designed to control the vehicle at least partially automatically, and / or to accelerate, decelerate, and / or steer.

[0021] The invention also relates to a computer program, in particular a computer program product, comprising instructions that, when executed by a computer, cause the computer to carry out the method according to the invention. Thus, the computer program according to the invention provides the same advantages as those described in detail with reference to a method according to the invention.

[0022] The invention also relates to a data processing device configured to carry out the method according to the invention. The device can be, for example, a computer that executes the computer program according to the invention. The computer can have at least one processor for executing the computer program. A non-volatile data memory can also be provided, in which the computer program is stored and from which the computer program can be read by the processor for execution.

[0023] The invention may also provide a computer-readable storage medium that has the computer program according to the invention and / or includes instructions that, when executed by a computer, cause the computer to carry out the method according to the invention. The storage medium is designed, for example, as a data storage device such as a hard disk and / or a non-volatile memory and / or a memory card. The storage medium can, for example, be integrated into the computer.

[0024] Furthermore, the method according to the invention can also be implemented as a computer-implemented method.

[0025] Further advantages, features, and details of the invention will become apparent from the following description, which describes embodiments of the invention in detail with reference to the drawings. The features mentioned in the claims and in the description may be essential to the invention individually or in any combination. They show: Fig. 1 a schematic visualization of a method, a device, a storage medium and a computer program according to embodiments of the invention, Fig. 2 a schematic representation of a vehicle, an embedded system and a sensor according to embodiments of the invention.

[0026] In Fig. 1, a method 100, a device 10, a storage medium 15 and a computer program 20 according to embodiments of the invention are schematically shown.

[0027] In a first step 101 of the method 100 according to an embodiment, sensor data of the embedded system 1 is provided. In a second step 102 of the embodiment, the Gaussian process prediction is performed, wherein an expected value for a target variable is determined based on the sensor data. In a third step 103, at least one sample is determined based on the sensor data in order to determine a variance for the expected value using the at least one sample. The sample is preferably a Monte Carlo sample. In a fourth step 104, the Gaussian process prediction is classified based on an analysis of the determined variance, wherein at least one class of the classification classifies the Gaussian process prediction as inaccurate. In a fifth step 105, a measure is initiated depending on the classification of the Gaussian process prediction.

[0028] In Fig. Figure 2 schematically shows a vehicle 3, an embedded system 1, and a sensor 2. In one embodiment, the embedded system 1 is arranged in the vehicle 3 and is preferably connected to the sensor 2. The sensor 2 can be a virtual sensor 2.

[0029] One aspect of this invention relates in particular to an algorithm for embedded systems that calculates the variance of a Gaussian process (GP) in O(M). The algorithm is known, for example, from the work in [1], with the references listed in square brackets at the end of this description. One aspect of the present invention is in particular the implementation of the algorithm in an embedded system.

[0030] As mentioned above, one aspect of the present invention is to calculate the variance of a GP prediction with a computational and memory requirement of O(M). This preferably allows for the simple application of GP models for safety-critical applications in embedded systems. This advantageously allows the system to react appropriately if the variance prediction is too high.

[0031] The exact computational and memory requirements for a variance prediction are in particular O(M 2 ). The following describes an approximate method that can calculate the variance from Monte Carlo samples. Drawing a Monte Carlo sample can advantageously scale to O(M).

[0032] For the implementation of a possible method of the invention, there can be two target systems: a first system that can efficiently calculate the mean (or expected value) of a GP prediction as a black-box function, and a second system that can perform a set of suitable basic operations in a freely selectable order.

[0033] Calculating the variance can advantageously allow for a statement about whether the GP is operating in interpolation (close to the training data) or extrapolation (outside the training data). In the latter case, a reaction can be triggered, as the prediction is no longer considered trustworthy.

[0034] For example, in the current state of the art, a watchdog is often used to determine whether the GP is in interpolation or extrapolation. However, this can be problematic because the mean prediction and the variance prediction are decoupled. This can lead to the following problems: For example, the watchdog is already in extrapolation while the GP is still in interpolation, which triggers a reaction too early and can cause a false alarm. In another case, the watchdog is still in interpolation while the GP is already in extrapolation, which can lead to an untrusted prediction being used, which in turn can create a security risk.

[0035] In particular, the invention demonstrates how an algorithm for determining the variance of the GP [1] can be calculated in embedded systems. This allows an appropriate reaction to be triggered when the uncertainty of the prediction is too large.

[0036] The invention is particularly relevant in the context of “virtual sensors” when the output of the GPs is used for control, regulation or monitoring applications (e.g. on-board diagnostics).

[0037] In one embodiment of the present invention, the following setup can be assumed: Given a training data set Dtrain=(xn,yn)n=1N, where xn∈ℝDx the input and y n ∈ ℝ is the output of the nth sample. One goal is preferably to find a function f:ℝDx→ℝ ℝ to learn which input x n an output y n Furthermore, we can assume a probabilistic setting in which the function ƒ~GP(0,k) has a GP prior with zero mean. The kernel function is preferably given by k: ℝDx×ℝDx→ℝ. The bases are particularly characterized by Xind=(xm)m=1M and their probabilistic outputs Y ind ~ N(µ ind ,Σ ind ), where µ ind ∈ ℝ M , Σ ind ∈ ℝ M×M The number of support points M can be a hyperparameter and can be freely chosen by the user. It can be a goal of the algorithm to find the most suitable solution for a new input x. ∗ the Mean E[y ∗ ] and the variance Var[y ∗ ] of the GP prediction efficiently.

[0038] Standard prediction: The standard formulas of sparse GP prediction are according to [2]: E[y*]=K(x*,Xind)α,Var[y*]=K(x*,x*)−K(x*,Xind)Ω(Xind,x*) where α = K(X ind , X ind ) -1 µ ind and Ω = K(X ind , Xi nd ) -1 (K(X ind ,X ind ) + Σ ind )K(X ind ,X ind ) -1 . The terms α and Ω can be precalculated since they do not depend on the test point x ∗depend on each other. After caching, the memory and runtime complexity are around E[y ∗ ] or Var[y ∗ ] to calculate O(M) or O(M 2 This may be one reason why current hardware accelerators only provide mean prediction as a function. Variance prediction can be too resource-intensive.

[0039] Interpretation as linear regression: Using the kernel trick according to [3], the kernel function k(·,·) can be represented as the inner product in a reproducing kernel Hilbert space (RKHS) H k(x,x')=〈ψ(x),ψ(x')〉H, where ψ:ℝDx→H can be a mapping into a (potentially infinite) feature space. For many core functions, the inner product can be further than 〈ψ(x),ψ(x')〉H≈ϕ(x)Tϕ(x') be approximated, where ψ:ℝDx→ℝDL is preferably a finite mapping. This is possible, for example, for all stationary kernels, especially for the RBF kernel.

[0040] The GP function can then be described by the linear model f(⋅)=∑l=1lwlϕ(⋅)l, w∈N(0,1) This interpretation of the GP allows, in particular, to efficiently determine an approximate solution for the mean and variance prediction. The a-posteriori distribution p(w|D train ) to then calculate the prediction probability p(y ∗ |x ∗ ) = ∫ p(y ∗ |w,x ∗ )p(w|D train )dw to determine.

[0041] The approximation quality can be determined from the size N of the training data set D as follows: train dependent: The larger the data set, the worse the approximation works.

[0042] In particular, we will show how Matheron's Rule can be used to efficiently calculate the mean and variance of a GP prediction [1].

[0043] In particular, Matheron's Rule states that instead of taking a sample directly from the posterior, a sample can also be taken from the prior and then appropriately corrected: y*|yind=fprior(x*)+k(x*,Xind)K(Xind,Xind)−1(yind−fprior(Xind)), where y ind ~ N(µ ind , Σ ind ) and ƒ prior ∼GP(0, k(·,·)).

[0044] This allows a Monte Carlo sample to be generated from the GP Posterior as follows: 1. First, the support points can be drawn: y ind ~ N(µ ind , Σ ind ), 2. A function can then be extracted from the prior: ƒ prior ∼GP(0,k(·,·)), 3. Then β = K(X ind ,X ind ) -1 (Y ind - ƒ prior (X ind)) can be determined, 4. For the test point x ∗ , the Monte Carlo prediction can then be calculated: y ∗ = ƒ prior (x ∗ ) + k(x ∗ ,X ind )β

[0045] By taking S Monte Carlo samples, the variance of the GP prediction can be determined.

[0046] This calculation can be advantageous in several respects: In particular, steps 1-3 only need to be calculated once (offline). Preferably, the only step that needs to be calculated in the embedded system is step 4.

[0047] Step 2 requires in particular the extraction of a function from the GP prior. This may only be possible to a limited extent if the standard GP formulas are used. However, it is preferably simple if the Gaussian process is approximated as a linear regression, as explained above: After the weights w and the feature mapping ϕ have been extracted, the function f(⋅)=∑l=1Lwlϕ(⋅)l can be evaluated on any input in O(L). Step 4 preferably consists of two parts: the prior and the correction term. The calculation rule for the correction term is identical to the standard mean prediction and scales in O(M). Special hardware accelerators can be used for this calculation.

[0048] The following example shows how the algorithm can be used on an embedded system.

[0049] For example, in offline mode, the following calculations cannot be performed on the target hardware.

[0050] Inputs can be a kernel function: k(·,·), Inducing inputs: X ind , a distribution over inducing outputs: y ind ~ N(µ ind , Σ ind ) and a number of Monte Carlo samples: p.

[0051] For example, a pseudo-code of the calculations is shown below:

[0052] In online mode, the following calculations can be performed on the target hardware.

[0053] Inputs can be a kernel function: k(·,·), Inducing inputs: X ind , a test input x ∗ , a number of Monte Carlo samples: S, a mapping (ϕ(s))s=1S and weights (w(s))s=1S,(β(s))s=1S be.

[0054] According to one exemplary embodiment, Case 1 is an embedded system that can calculate the mean (or expected value) of a GP prediction as a black-box function. In other words, this is an embedded system that only needs to be supplemented with the variance calculation.

[0055] The calculation is described below using an example pseudo-code: Memory: O (SL + MD + SM) Runtime: O (SL + SMD)

[0056] According to one embodiment, Case 2 is an embedded system that can perform a set of suitable basic operations in a freely selectable order. In other words, this is, in particular, an embedded system that calculates or can calculate all the calculations of the present invention itself.

[0057] The calculation is described below using pseudo-code: Memory: O (SL + MD + SM) Duration: O (SL + MD + SM)

[0058] In the context of the present invention, it is particularly assumed that a GP with a mean value, or expected value, of zero (zero-mean) is used: f∼GP(0,k).

[0059] In possible further applications, a Gaussian process with a non-zero mean, or expectation, can be used, ƒ~GP(m,k), where the mean function m(⋅):ℝDx→ℝ for example, comes from domain knowledge.

[0060] The mean or expected value prediction can then preferably be expressed by the formula y*=m(x*)+k(x*,Xind)K(Xind,Xind)−1(yind−m(x*)) calculate.

[0061] In hybrid modeling (combining AI models with prior physical knowledge), this type of model is also called a "Delta Model": The GP can correct the prediction of the physical model. If the test input x ∗ far from X ind is, k(x * ,X ind ) preferably approaches zero, and the physical model takes over. At the same time, the variance of the forecast may increase.

[0062] It can be seen that this formula is very similar to the Monte Carlo prediction formula given above, where m(x * ) the role of ƒ prior (x ∗ ). This may allow the algorithm to be interpreted as a hybrid model with a stochastic physical model, where the linear approximation of the GP prior takes the role of the physical model.

[0063] The above explanation of the embodiments describes the present invention exclusively by way of examples. Of course, individual features of the embodiments can be freely combined with one another, provided they are technically feasible, without departing from the scope of the present invention. [1] Wilson, James, et al. “Efficiently sampling functions from Gaussian process posteriors.” International Conference on Machine Learning. 2020. [2] Titsias, Michaelis. “Variational learning of inducing variables in sparse Gaussian processes” Artificial Intelligence and Statistics, 2009. [3] Schölkopf Bernhard, et al. “Learning with Kernels: Support Vector Machines, Regularization, Optimization and Beyond.” MIT press, 2001. QUOTES CONTAINED IN THE DESCRIPTION

[0000] This list of documents submitted by the applicant was generated automatically and is included solely for the convenience of the reader. This list is not part of the German patent or utility model application. The DPMA assumes no liability for any errors or omissions. Cited non-patent literature

[0000] Wilson, James, et al. “Efficiently sampling functions from Gaussian process posteriors.” International Conference on Machine Learning. 2020

[0063] Titsias, Michaelis. “Variational learning of inducing variables in sparse Gaussian processes” Artificial Intelligence and Statistics, 2009

[0063] Schölkopf Bernhard, et al. “Learning with Kernels: Support Vector Machines, Regularization, Optimization and Beyond.” MIT press, 2001

[0063]

Claims

[1] A method (100) for processing a variance of a Gaussian process prediction of an embedded system (1), comprising the following steps: - Providing (101) sensor data of the embedded system (1), - performing (102) the Gaussian process prediction, wherein an expected value for a target variable is determined on the basis of the sensor data, - determining (103) at least one sample, preferably at least one Monte Carlo sample, on the basis of the sensor data in order to determine a variance for the determined expected value using the at least one sample, - classifying (104) the Gaussian process prediction on the basis of an analysis of the determined variance, wherein at least one class of the classification classifies an inaccuracy of the Gaussian process prediction, - Initiating (105) an action depending on the classification of the Gaussian process prediction. [2] Method (100) according to claim 1, characterized by that the sensor data result from a measurement of at least one virtual sensor (2), wherein the target variable is a measured variable which is predicted by the Gaussian process prediction on the basis of the sensor data, wherein the Gaussian process prediction is carried out using a plurality of initial functions, the method further comprising the following steps: - Providing support points in the sensor data, - Dragging function values ​​of the support points, - Selecting a function from the multitude of initial functions, - Determining a deviation between the function values ​​of the support points and a result of an application of the selected function in order to provide a correction term for the selected function based on the deviation, - performing the Gaussian process prediction using the correction term and the selected function, wherein the preceding steps are performed for each of the at least one sample, preferably wherein the step of performing the Gaussian process prediction is performed by the embedded system and the remaining steps are performed by an external data processing device. [3] Method (100) according to one of the preceding claims, characterized by that, within the framework of the analysis, an exceeding of a defined threshold of the variance is assessed, whereby the exceeding of the defined threshold is specific for an interpolation or an extrapolation, whereby in the presence of extrapolation the Gaussian process prediction is classified as inaccurate. [4] Method (100) according to one of the preceding claims, characterized by that the measure includes at least one of the following steps: - Rejecting the Gaussian process prediction, - Raise an alarm to alert a user that the Gaussian prediction is classified as inaccurate. [5] Method (100) according to one of the preceding claims, characterized by that performing (102) the Gaussian process prediction further comprises the following step: - Performing an approximation using a linear model in order to reduce the computational effort required to determine the variance for the expected value. [6] Method (100) according to one of the preceding claims, characterized by that the Gaussian process prediction is performed using a machine learning model, wherein the machine learning model is based on a training comprising the following steps: - providing training data, wherein the training data represent sensor data resulting from a measurement of at least one sensor (2), - Providing support points in the training data to provide reference points for training the machine learning model, - Training the machine learning model, whereby the machine learning model is trained in such a way that the machine learning model determines a function to determine the expected value of the target variable based on the training data. [7] Method (100) according to one of the preceding claims, characterized by that the method (100) is applied to a vehicle (3), wherein the embedded system (1) is arranged in the vehicle (3). [8] Computer program (20) comprising instructions which, when the computer program (20) is executed by a computer (10), cause the computer (10) to carry out the method (100) according to one of the preceding claims. [9] Device (10) for data processing, which is arranged to carry out the method (100) according to one of claims 1 to 7. [10] A computer-readable storage medium (15) comprising instructions which, when executed by a computer (10), cause the computer (10) to carry out the steps of the method (100) according to any one of claims 1 to 7.

Images