Training procedure for a driver assistance system

Abstract

The invention relates to a training method for a driver assistance system, comprising: a. receiving sensor data with an initial dimension in an input module, wherein the sensor data is decomposed into small time intervals; b. using an encoder, creating compressed data with a dimension reduced compared to the initial dimension by extracting information from the sensor data decomposed into small time intervals; and d. using a decoder, reconstructing the compressed data based on the sensor data, wherein the reconstructed data has a dimension equal to the initial dimension, wherein the encoder and the decoder are part of a variational autoencoder. The training method is characterized in particular by the fact that in a step c. between step b. and step d.The following steps are performed: - using kernel density estimation, the compressed data is estimated and sorted according to probability densities; - based on the probability densities, areas with a lower density compared to other areas are selected; - the decoder is reprogrammed with the data from the low-density areas. The training method proposed here for a driver assistance system significantly improves the recognition of rare traffic situations.

Description

The invention relates to a training method for a driver assistance system. Industrial experience has shown that advanced driver assistance systems (ADAS) and highly automated driving systems rely on the detection of rare traffic situations, known as corner cases, to ensure vehicle safety. These corner cases are infrequently occurring traffic situations unknown to the system, such as bulky aircraft parts on a trailer or a fallen safety beacon. Detecting such corner cases is crucial for the vehicle to maneuver as safely as a human driver. Previous approaches to corner case detection are based on analyzing sensor data and identifying anomalies within that data. However, these methods are often inaccurate and cannot reliably detect all possible corner cases. Training an autoencoder involves searching for ϕ ∈ Φ, θ ∈ Θ for which L(ϕ,θ) is as small as possible. An autoencoder is often trained using one of the many backpropagation methods (conjugate gradient, gradient descent, and others). Although this method is often very effective, there are fundamental problems with training neural networks with hidden layers. Once errors are backpropagated to the first few layers, they become insignificant. This means that the network almost always learns to learn the average of the training data. Although there are advanced backpropagation methods (such as the conjugate gradient method) that partially address this problem, this approach ultimately results in slow learning and poor performance. DE 10 2019 206 720 A1 discloses a method and a device for monitoring an AI module which forms part of a processing chain of a semi-automatic or automatic driving function of a vehicle. To improve performance, initial weights are currently used that already roughly correspond to the desired result. This is called pretraining. In a pretraining technique developed by Geoffrey Hinton for training multilayer autoencoders, adjacent layers are treated as a restricted Boltzmann machine to achieve a good approximation, and then backpropagation is used for fine-tuning. The known methods for detecting edge cases have several limitations. They are often unable to identify the rarest and most unusual traffic situations because they are based on predefined rules and models that do not cover all possible scenarios. Furthermore, they are frequently unable to process sensor data in a way that enables accurate and reliable detection of edge cases. This leads to reduced performance and robustness of driver assistance systems and highly automated driving systems. Based on this, the present invention aims to overcome, at least partially, the disadvantages known from the prior art. The features of the invention are defined in the independent claims, for which advantageous embodiments are shown in the dependent claims. The features of the claims can be combined in any technically meaningful way, whereby the explanations in the following description and features from the figures, which comprise supplementary embodiments of the invention, can also be used. The invention relates to a training method for a driver assistance system, comprising at least the following steps performed in the stated order by a computing device: a. Receiving sensor data with an initial dimension in an input module, wherein the sensor data is decomposed into small time intervals; b. by means of an encoder, creating compressed data with a dimension reduced compared to the initial dimension by extracting information from the sensor data decomposed into small time intervals; and d. by means of a decoder, reconstructing the compressed data based on the sensor data, wherein the reconstructed data has a dimension equal to the initial dimension, the decoder being rewarded or penalized depending on how accurate the reconstructed data is compared to the sensor data, wherein the encoder and the decoder are part of a variational autoencoder. The training procedure is characterized primarily by the fact that in step c, between step b and step d, the following is performed in a bottleneck module: - using kernel density estimation, estimating and sorting the compressed data according to probability densities; - based on the probability densities, selecting areas with a low density compared to other areas; - reparameterizing the decoder with the data from the areas with the low density. Unless explicitly stated otherwise, ordinal numbers used in the preceding and following descriptions serve solely for unambiguous differentiation and do not indicate any order or ranking of the components referred to. An ordinal number greater than one does not necessarily imply the presence of another such component. It should be noted that the method described here involves training an artificial intelligence using a variational autoencoder with an encoder, a bottleneck module, and a decoder, where, unlike a classic autoencoder, the bottleneck module is modifiable, i.e., variable. To clarify the obvious, this training method cannot be executed by a human in a reasonable timeframe, with sufficient reliability, or even with sustained repeatability. It only makes sense as a method that is executed on a computer. The process, in step a, involves receiving sensor data with an initial dimension in an input module. The sensor data is then broken down into small time intervals to enable detailed analysis. These time intervals represent different scenarios captured by the vehicle sensors. The sensor data includes raw data such as camera images and FlexRay signals, as well as processed data. The input module is responsible for converting the sensor data into a computer-readable format, preferably a standardized format, which forms the basis for further processing. The next step of the process involves creating compressed data using an encoder. The encoder is configured to process the sensor data, which has been divided into small time intervals. The initial dimension of the sensor data is reduced to a smaller dimension by extracting relevant information. The encoder transforms the time intervals into latent space embeddings, which represent a compressed version of the original sensor data. This compressed data forms the basis for the subsequent processing steps in the corner case detection process. In step d., the procedure involves reconstructing the compressed data using a decoder. The decoder is configured to convert the compressed data created in the previous step back into a form that corresponds to the initial dimensions. The accuracy of the reconstruction is evaluated by comparing the reconstructed data with the original sensor data. The decoder is rewarded or penalized depending on how accurately the reconstructed data compares to the sensor data. This enables continuous improvement of the reconstruction accuracy and contributes to the robustness of the overall system. In step c, the bottleneck module is used to further process the compressed data. The bottleneck module performs a kernel density estimation to estimate the probability densities of the compressed data and then sorts it. This identifies areas with a lower density compared to other areas. These areas represent the rare and unusual traffic situations that should be recognized as edge cases. The decoder is then reparameterized with the data from the low-density areas (i.e., equipped with appropriately adjusted new parameters) to improve the accuracy of the reconstruction and optimize the detection of edge cases. In a further advantageous embodiment of the training procedure, it is proposed that the distribution of the probability densities be reduced to two or three dimensions, and the correspondingly reduced distribution be visually represented. The probability density distribution is reduced to two or three dimensions to enable a visual representation of the data. This reduction is achieved using appropriate dimensionality reduction techniques, such as principal component analysis (PCA) or t-distributed stochastic neighbor embedding (t-SNE). Visualizing the reduced distribution allows the data to be presented in a human-understandable format. This facilitates data analysis and interpretation by data scientists and other professionals tasked with evaluating system performance and identifying edge cases. The reduced distribution is visually represented using appropriate data visualization techniques. These techniques include, for example, the use of diagrams, graphs, and other visual representations to depict the probability density distribution in a human-understandable way. This visual representation enables data scientists and other professionals to efficiently analyze and interpret the results of kernel density estimation and the identification of corner cases. This contributes to improved system performance and the detection of rare and unusual traffic situations. In a further advantageous embodiment of the training method, it is proposed that a low density is present when 5% or less, preferably 1% or less, of the data exhibit such a low or lower density. Low density is defined as a range in which 5% or less of the data exhibits such a low density. Preferably, low density is present when 1% or less of the data exhibits such a low density. This definition of low density makes it possible to identify the rarest and most unusual traffic situations, which should be recognized as edge cases. If the data set is very large, it is advantageous to select an even smaller proportion of the data with a relatively low density. In one embodiment, a density threshold is defined, and a density is considered low if this threshold is reached or fallen below. The invention described above is explained in detail below against the relevant technical background with reference to the accompanying drawings, which show preferred embodiments. The invention is in no way limited by the purely schematic drawings, although it should be noted that the drawings are not dimensionally accurate and are not suitable for defining size relationships. Figure 1 schematically illustrates the architecture of a variational autoencoder; and Figure 2 schematically illustrates the system architecture for the training method proposed here. Figure 1 schematically shows the architecture of a variational autoencoder 9, as used as module B in the following system architecture. The trapezoid shown on the left is the encoder 3 of the neural network, and the trapezoid shown on the right is the decoder 6 of the neural network. To the left of encoder 3 are the input data or sensor data 1 (x) with an initial dimension 2 (here, for example, 7x1). In the middle, in the bottleneck module 10, are the compressed data 4 (z = e(x)) with a compressed dimension 5 (here, for example, 3x1). On the right are the reconstructed data 7 (X̂ = d(z)) with the reconstructed dimension 8, which is equal to the initial dimension 2 (i.e., here, 7x1). For example, the following condition is sought: Figure 2 schematically shows the system architecture for the training method proposed here, which comprises four modules A to D. It illustrates the interactions of the modules for detecting corner cases in advanced driver assistance systems and highly automated driving systems. Module A is responsible for processing the sensor data 1 acquired in a first step S1. This sensor data 1 comprises signals collected by a motor vehicle using at least one, preferably a multitude of, sensors (represented here by the pictogram of an eye). This sensor data 1 includes, for example, signals provided by FlexRay / CAN BUS in the form of time series (such as radar, LiDAR [Light Detection and Ranging], nano radar, infrared sensor, ultrasonic sensor, and others), as well as raw images from cameras (in the visible light spectrum) and future vectors (data processed by other systems). In other words, any type of sensor and sensor data 1 can be used here to detect edge cases. The resulting time series data is divided into small time segments.A segment of X seconds represents a window or scenario. The size of these windows can vary depending on the use case. Module B is a component described as artificial intelligence, or more precisely, a neural network. Its basic structure is shown in Fig. 1, and its function is described in the accompanying text. In a second step S2, the encoder 3 receives input data (i.e., the sensor data 1), which is preferably provided in a standardized format. The encoder 3 extracts meaningful information from each window (scenario). In this process, each window is transformed into an embedding in a third step S3. An embedding is a point in a hyperspace, which is referred to as latent (hidden, i.e., not comprehensible to humans) space. The embedding represents compressed data 4. However, to improve variations and generalizations in the model, the compressed data 4 is not sent directly to the decoder 6.Instead, in a fourth step S4, a reparameterization trick is used, which consists of selecting a point from the latent space near our embedding. This new embedding, extracted from the distribution, is then provided to Decoder 6. Decoder 6 now receives the compressed data 4, which has been reparameterized accordingly by Encoder 3. Subsequently, Decoder 6 reconstructs the original input using this modified embedding. Decoder 6 is rewarded or penalized based on the accuracy of the reconstruction compared to the original input (the training). The underlying principle is that Encoder 3 learns to extract the most important features from each example, enabling Decoder 6 to reconstruct the original data very accurately. In a fifth step (S5), module C analyzes the embedding generated by module B. Here, it is proposed to use kernel density estimation (KDE), which estimates a probability density function of a dataset without assuming a specific distribution beforehand. It works by applying a kernel (a shape like a bell curve) to each data point and adding or averaging their contributions to obtain a smooth estimate of the overall distribution. This is useful for understanding the underlying structure of the data. KDE thus estimates how many scenarios are similar to each other. Based on this metric, the scenarios are sorted in ascending order according to their respective density values, and the scenarios with the lowest density values ​​(e.g., 1%) are identified to detect the low-density extreme cases (CC).The embeddings generated by Module B and the density estimate generated by Module C are the core components of the training procedure proposed here. All embedding analyses are performed in N-dimensional spaces, where N is greater than 3. For visualization purposes, however, a dimensionality reduction technique is used in a sixth step (S6) to reduce these embeddings to a human-comprehensible scale, for example, by projecting three-dimensional and / or two-dimensional spaces. This reduction facilitates human understanding and supports comprehension of how the classification between corner cases and non-corner cases was determined, i.e., whether the variational autoencoder 9 has been trained correctly. Module D thus displays the corner cases [CC] and non-corner cases [NCC] (high density 12) for human understanding of the scenario analysis. Using the information obtained from Modules B and C, Module D generates diagrams with various data visualization techniques in a seventh step (S7) to further enhance human understanding.This visualization helps data scientists assess the correctness of the system. At the same time, it provides a quick overview of the results. The invention relates to a training method for a driver assistance system, comprising: a. receiving sensor data with an initial dimension in an input module, wherein the sensor data is decomposed into small time intervals; b. using an encoder, creating compressed data with a dimension reduced compared to the initial dimension by extracting information from the sensor data decomposed into small time intervals; and d. using a decoder, reconstructing the compressed data based on the sensor data, wherein the reconstructed data has a dimension equal to the initial dimension, wherein the encoder and the decoder are part of a variational autoencoder. The training method is characterized in particular by the fact that in a step c. between step b. and step d.The following is performed: - using kernel density estimation, estimating and sorting the compressed data according to probability densities; - based on the probability densities, selecting areas with a low density compared to other areas; - reparameterizing the decoder with the data from the areas with the low density. The training procedure proposed here for a driver assistance system significantly improves the recognition of rare traffic situations. Reference symbol list 1 Sensor data 2 Initial dimension 3 Encoder 4 Compressed data 5 Compressed dimension 6 Decoder 7 Reconstructed data 8 Reconstructed dimension 9 Variational autoencoder 10 Bottleneck module 11 Low density (CC) 12 High density (NCC) A Module AB Module BC Module CD Module D S1 First step S2 Second step S3 Third step S4 Fourth step S5 Fifth step S6 Sixth step S7 Seventh step

Claims

Training method for a driver assistance system, comprising at least the following steps performed in the stated order by a computing device: a. Receiving sensor data (1) with an initial dimension (2) in an input module, wherein the sensor data (1) is decomposed into small time intervals; b. by means of an encoder (3), creating compressed data (4) with a dimension (5) reduced compared to the initial dimension (2) by extracting information from the sensor data (1) decomposed into small time intervals; and d.by means of a decoder (6), reconstructing the compressed data (4) based on the sensor data (1), wherein the reconstructed data (7) have a dimension (8) equal to the initial dimension (2), wherein the decoder (6) is rewarded or penalized depending on how accurate the reconstructed data (7) are compared to the sensor data (1), wherein the encoder (3) and the decoder (6) are part of a variational autoencoder (9), characterized in that in a step c. between step b. and step d. in a bottleneck module (10) the following is performed: - by means of a kernel density estimation, estimating and sorting the compressed data (4) according to probability densities; - based on the probability densities, selecting areas with a low density compared to other areas (11); - reparameterizing the decoder (6) with the data (7) from the areas with the low density (11). Training method according to claim 1, wherein the distribution of the probability densities is reduced to two or three dimensions, and the correspondingly reduced distribution is visually represented. Training method according to claim 1 or claim 2, wherein a low density (11) is present when 5% or less, preferably 1% or less, of the data have such a low or lower density (11).

Images