Method for estimating global uncertainty of a neural network

Abstract

A method for estimating a global uncertainty of output data of a computer-implemented main neural network is described, comprising the steps of: determining (102) a first measure quantifying how much a current input data of the main neural network follows the same distribution as the data used for training the main neural network; generating (103) a second measure quantifying a certainty of the main neural network in its own input data-based prediction, in particular by using at least one Bayesian method, e.g. a Bayesian neural network or an ensemble-based method; determining (104) a third measure based on an estimate of a class discrimination feature in the input data and a comparison of the feature with a class activation probability distribution, in particular wherein the class activation probability distribution is created based on the estimated class discrimination feature during training of the main neural network; determining (105) the global uncertainty based on at least two of the first measure, the second measure and the third measure.

Description

Technical Field

[0001] The present invention describes a method for estimating the global uncertainty of output data of a computer-implemented neural network, a computer program, a computer-readable storage device, and an apparatus arranged to perform the method. Background Technology

[0002] Deep neural networks (DNNs) are the latest technology, and their applications in various technological fields are increasing.

[0003] When using deep neural networks for safety-critical tasks, several system security issues arise. A fundamental problem with the security of deep neural network modules is that the error obtained by a deep neural network on a test set can only be generalized to new and unforeseen samples under certain assumptions, assumptions that cannot be satisfied in complex open environments such as autonomous driving. Therefore, to safeguard deep neural network components, a complex security architecture including advanced security measures is required. Summary of the Invention

[0004] This invention describes a method for estimating the global uncertainty of output data of a computer-implemented main neural network, the method comprising the following steps:

[0005] - Determine a first metric (also known as a metric of confidence) that measures the degree to which the current input data of the main neural network follows the same distribution as the data used to train the main neural network;

[0006] - Generate a second measure (also known as introspective uncertainty) that measures the determinism of the main neural network in its own predictions based on the input data, in particular by using at least one Bayesian method, such as a Bayesian neural network or an ensemble-based method to generate the second measure;

[0007] - A third measure (also known as output uncertainty) is determined based on the estimation of class discriminative features in the input data and the comparison of said features with the class activation probability distribution. In particular, the class activation probability distribution is created based on the estimated class discriminative features during the training of the main neural network.

[0008] - Determine the global uncertainty based on at least two of the first, second, and third measures.

[0009] Therefore, the core of this invention is to systematically combine the above three measures into a single global uncertainty, thereby enabling effective protection of the sensing system.

[0010] The main neural network can be a deep neural network, which typically includes at least an input layer, an output layer, and at least one hidden layer. The network can be trained to classify sensor data from one or more sensors. A typical neural network can be designed and trained to detect objects based on images from a camera device.

[0011] Bayesian neural networks are a special type of neural network. They use a distribution of weights instead of scalar weights. This method allows them to output a value for a given set of input data, despite design uncertainties.

[0012] Neural network ensembles are combinations of multiple neural networks that may have different architectures or different weights when using the same architecture. Typically, these multiple neural networks are trained using different training sets. During testing, each network is evaluated separately based on given data, and the different outputs are merged into a single output based on, for example, a voting strategy (majority vote, minority vote, etc.). The uncertainty of the ensemble can be measured by the variance of the individual outputs.

[0013] In another embodiment of the method, the determination of the first metric is achieved by means of an autoencoder, and more particularly by means of a variational autoencoder.

[0014] In another embodiment of the method, the autoencoder includes an encoder part and a decoder part. The encoder part is trained to map input data to a low-dimensional feature space, the so-called latent space, and the decoder part attempts to reconstruct the original higher-dimensional input from the latent space. The confidence metric is based on either the calculation of the probability density values ​​of the input data mapped to the latent space or on the reconstruction error estimated by the autoencoder. In a particular embodiment, the autoencoder is a variational autoencoder.

[0015] In another embodiment of the method, the generation of the second metric is based on Bayesian modeling of the main neural network. In a specific embodiment of the method, the main neural network is modeled as a Bayesian neural network. Bayesian modeling allows uncertainty to be inherently integrated into a particular model (in this case, a neural network).

[0016] In another implementation of this method, the weights of the main neural network are modeled as a distribution. For example, an alternative feasible approach is to use fixed values ​​as weights.

[0017] Using a distribution instead of a fixed value makes it easy to extract a second measure, or so-called introspective uncertainty, from the main neural network.

[0018] In another implementation of this method, the training of the main neural network is performed using variational inference. Compared to other more expensive Bayesian optimization methods (especially Markov chain Monte Carlo), variational inference enables the efficient estimation of the unknown posterior distribution of the weight distribution.

[0019] In another embodiment of the method, the main neural network is modeled as a Bayesian neural network. Variational inference of the Bayesian neural network is performed using numerical integration, wherein the weight distribution is sampled n times and the set of weights for each sample is used once to predict n distinct outputs for a given input. The average of the sampled outputs is then used as the prediction of the main neural network, while the variance of the prediction is used to compute a second metric.

[0020] To perform inference in a Bayesian neural network, it is necessary to marginalize the weight distribution, that is, to integrate it out. This is difficult to do analytically, which is why inference in Bayesian neural networks preferably uses numerical integration, i.e., an approximation of integration over the weight distribution. Alternatively or additionally, analytical computation is also feasible.

[0021] In another embodiment of the method, a second metric is obtained by enhancing the neural network using a Gaussian process.

[0022] The advantage of this approach is that the uncertainty is inherent in the design. Therefore, a second metric can be easily extracted. Furthermore, particularly for classification and regression problems, the training process can be divided into two parts by training the neural network as a feature extractor and the Gaussian process as a regressor separately. This reduces the complexity associated with Bayesian neural networks.

[0023] If a neural network and a Gaussian process are trained together, the method of using a neural network as a feature extractor (feeding its output to a Gaussian process) is also referred to in the literature as deep kernel learning. If the neural network and the Gaussian process are trained separately, another common name is deep Gaussian process.

[0024] In another embodiment of the method, the second metric is generated through the ensemble decision of at least two neural networks, specifically two neural networks with different network architectures or trained using different training datasets. The advantage of this is that training the ensemble does not rely on the expensive optimization methods used in Bayesian methods.

[0025] In another embodiment of the method, the second metric is generated through ensemble decision-making of different machine learning methods, particularly at least one of the methods being a neural network.

[0026] Ensembles can also be built using different types of machine learning methods, such as SVM, random forest, generalized linear models, etc.

[0027] These further implementations open up the possibility of choosing between different methods. The optimal method for generating the second metric can be selected based on the design of the main neural network, the application of the network, and other influencing factors.

[0028] In another implementation of this method, class-discriminating features are estimated by computing scaled and / or unscaled features. A particular approach involves using activation heatmaps. For this method, a weighted gradient class activation map (Grad-Cam) method can be used. This method allows for the automatic detection of whether specific parts of an image have been correctly classified by the neural network. Typically, when using heatmap methods, the results must be manually checked. This method requires no manual intervention.

[0029] In another embodiment of the method, a scaled activation heatmap of the detected object is compared with a class activation heatmap, the class activation heatmap is computed for all detected objects during training, and then the class activation heatmaps are combined by class and / or by pixel to obtain a class activation probability distribution.

[0030] Another advantage is that, while scaling introduces invariance with respect to unwanted values, the evaluation of the resulting heatmap can be done automatically.

[0031] In another embodiment of the method, the first, second, and third measures are compared, and the global uncertainty is set to the value representing the minimum confidence level among the three measures, or the global uncertainty is determined based on the average of the first, second, and third measures or any other type of weighted sum.

[0032] For example, a measure can be standardized to a value between 0 and 1. 0 represents no confidence at all, and 1 represents absolute confidence.

[0033] For the first measure, low confidence is equivalent to a low correspondence between the distribution of the input data and the distribution of the training data, while high confidence is equivalent to a good correspondence.

[0034] For the second metric, confidence can be equated to the confidence level of the main neural network in its own predictions.

[0035] For the third measure, low confidence is equivalent to a large deviation between the estimated class distinguishing features and the class activation probability distribution, where low deviation is equivalent to high confidence.

[0036] In another embodiment of the method, robots that are at least partially automated, particularly vehicles that are at least partially automated, are controlled based on global uncertainty.

[0037] An autonomous or automatically operated vehicle can be defined as a vehicle that operates partially, highly, or fully autonomously. In particular, an autonomous vehicle can be a driverless vehicle that can operate at least partially without human intervention. This vehicle can be, for example, a classic passenger car, minibus, or shuttle bus. The vehicle can also be other types of transportation, such as two-wheeled or three-wheeled vehicles, boats, trains, airplanes, drones, or helicopters. Input data can come from, for example, camera devices, radar, lidar, microphones, ultrasonic sensors, or any other type of sensor.

[0038] In another implementation of the method, the robot's path planning is based on global uncertainty, thereby avoiding high uncertainty regions, particularly those with high uncertainty compared to a predetermined threshold, when controlling the robot, especially by maintaining a predetermined distance from high uncertainty regions.

[0039] High-uncertainty regions can be estimated by comparing the global uncertainty predicted by a neural network based on input data representing the region with a predefined threshold. For example, if the global uncertainty is greater than 0.9, a high level of uncertainty can be reached in the range of 0 to 1, where 0 represents no confidence and 1 represents absolute confidence. The predefined threshold can be adjusted based on multiple influencing factors, such as the robot's speed, the type of area the robot is in, time, and weather.

[0040] If the robot is a vehicle, then different locations can represent different types of roads, such as highways, main roads, pedestrian zones, etc.

[0041] The input data indicating this area may originate, for example, from a sensor that scans the area. Examples include cameras, lidar, radar, or ultrasonic sensors.

[0042] Furthermore, the predetermined distance can be adjusted based on several influencing factors, such as the robot's speed or its position. For example, the slower the robot's speed, the smaller the predetermined distance can be.

[0043] If a robot that is at least partially automated can be controlled by an operator, or in the case of a vehicle that is at least partially automated and controlled by a driver, then if the uncertainty of sensor readings about the environment is too high, or if navigation is not possible or recommended in an area of ​​high uncertainty, then the automated system / robot may further choose to disengage from control and return control to the operator or driver, or adopt a safe state.

[0044] Alternatively, the robot, which is at least partially automated or autonomous, can also be another type of mobile robot, such as a robot that moves by flying, swimming, diving, or walking. For example, the mobile robot could be at least a semi-automatic lawnmower or at least a semi-automatic cleaning robot. In these cases, control signals can also be determined based on global uncertainties predicted by neural networks.

[0045] In another alternative, the at least partially autonomous robot could also be a garden robot that uses imaging sensors and neural networks to determine the species or condition of plants in the environment. Control signals can be determined based on the predicted plant species or determined plant condition and on global uncertainty, thereby applying an amount of chemical reagent corresponding to the determined species or determined condition.

[0046] In other alternatives, at least partially autonomous robots can also be household appliances, particularly washing machines, stoves, ovens, microwave ovens, or dishwashers. Using sensors, such as optical sensors, the condition of objects handled by the appliance can be recorded; for example, in the case of a washing machine, the condition of the clothes inside. A neural network can then be used based on the sensor data to determine the type or state of the object. Control signals can then be determined so that the appliance is controlled according to the determined object type and / or the determined object state, as well as the global uncertainty of this prediction. For example, in the case of a washing machine, control could be based on the material used to make the clothes. This global uncertainty helps achieve the required high safety standards to ensure that no living organisms are present inside the washing machine.

[0047] Additionally, a computer program comprising instructions that, when executed by a computer, cause the computer to perform the method according to the invention is claimed.

[0048] Additionally, protection is claimed for a computer-readable storage device on which a computer program is stored.

[0049] The method described herein also creates an apparatus designed for performing, controlling, or implementing steps in variations of the above-described process. Similarly, this apparatus-based embodiment of the invention can quickly and efficiently solve the task on which the invention is based.

[0050] Therefore, the device may have at least one computing unit for processing signals or data, at least one storage unit for storing signals or data, at least one interface connected to a sensor or actuator for reading sensor signals from the sensor or for outputting data or control signals to the actuator, and / or at least one communication interface for reading or outputting data embedded in a communication protocol. The computing unit may be, for example, a signal processor, a microcontroller, etc., while the storage unit may be flash memory, EEPROM, or magnetic storage. The communication interface may be designed to read or output data wirelessly and / or via a wired connection, whereby a communication interface for reading or outputting wired data may, for example, electrically or optically read or output the data from or to a corresponding data transmission line.

[0051] The device can be an electrical device that processes sensor signals and outputs control and / or data signals based on these signals. The device may have an interface that can be designed according to hardware and / or software. In hardware-based training, the interface can be, for example, part of a so-called system ASIC, which contains various functions of the device. However, interfaces can also be their own integrated circuits, or at least partially composed of discrete components. In software training, the interface can be, for example, a software module that exists on a microcontroller along with other software modules.

[0052] Additionally, it is advantageous to claim protection for a computer program having program code that can be stored on a machine-readable carrier or storage medium (e.g., semiconductor memory, hard disk memory, or optical memory) and used to execute, implement, and / or control method steps according to one of the above embodiments, particularly when the program product or program is executed on a computer or device.

[0053] The proposed method and security architecture can be applied to any system that uses deep neural networks to process one or more sensor inputs to obtain a virtual model of the environment, especially where safety is critical. Use cases could include detecting areas in the environment that cannot be correctly represented with sufficient determinism (e.g., whether someone is at point x in my environment). Attached Figure Description

[0054] Figure 1 A schematic diagram is shown.

[0055] Figure 2 A schematic diagram is shown of a deep neural network module capable of generating different types of uncertainty.

[0056] Figure 3 A schematic diagram illustrates how to use a deep neural network module with uncertainty in an object detection system with early fusion of sensor data. Detailed Implementation

[0057] Over the past few years, artificial intelligence (more precisely, machine learning algorithms) has enabled exciting new applications. A well-known example is the rapid development in the field of autonomous driving. However, the transition from hands-on driver assistance (Levels 1-2) to hands-off autonomous driving (Levels 3-5) is accompanied by fundamental issues of system safety. In particular, the use of deep neural networks (DNNs) for safety-related tasks, such as video-based object detection and classification, has brought entirely new challenges to system safety.

[0058] A fundamental problem with the security of deep neural network components is that the error gained by the deep neural network on the test set can only be generalized to new and unseen samples if the new samples originate from the same distribution as the test data. Furthermore, all samples need to be randomly selected from a "real-world" distribution. In practice, these two requirements are nearly impossible to meet because they would mean:

[0059] 1. The generated training and test datasets must be a perfect representation of the "real world".

[0060] 2. The distribution of the “real world” does not change over time (there is no change in distribution over time).

[0061] Therefore, it is crucial for safe autonomous driving to consider the confidence level of deep neural network predictions in order to indicate when the object detection model is uncertain. For example, if an autonomous vehicle identifies an object ahead as a pedestrian but is unsure of its location, the system can alert the driver to take over the vehicle or slow down as early as possible to avoid a major accident. Since the output probabilities of deep neural networks are not well calibrated, a complex safety architecture incorporating advanced safety measures is needed to safeguard the deep neural network components.

[0062] As mentioned above, several security issues arise when using deep neural networks for safety-critical tasks. A fundamental problem with the security of deep neural network modules is that the errors obtained on the test set can only be generalized to new and unforeseen samples under certain assumptions, which cannot be satisfied in complex open environment applications (such as autonomous driving). Therefore, to safeguard deep neural network components, a complex security architecture including advanced security measures is required.

[0063] In advanced driver assistance systems and autonomous driving, current plausibility checks are often limited to comparing multiple sensor outputs or analyzing input data based on expert knowledge (e.g., identifying challenging environmental conditions like fog or degraded data quality like motion blur). However, the output of a deep neural network relates to the probability distribution of its input data (e.g., the probability distribution of pixel values ​​in the case of video-based object detection), not necessarily to semantic units. Therefore, even for domain experts, understanding possible causes of failure is difficult, or at least partially impossible.

[0064] Therefore, in this application, we propose a sophisticated security architecture that incorporates multiple data-driven plausibility checks and introspection metrics to provide confidence in the predictions of the deep neural network module. The proposed uncertainty-enhanced deep neural network module can replace existing deep neural networks wherever they are used. Possible application areas include image recognition (camera images, radar images, LiDAR images, ultrasound images, and especially combinations thereof), noise classification, and more. They can be used in security applications (home security), automotive, aerospace and aviation applications, and shipping and rail transportation.

[0065] The overall concept behind the proposed security architecture for deep neural network modules is to determine during operation whether the current predictions of the deep neural network can be trusted. To this end, the overall confidence in the predictions is estimated using three separate confidence metrics: a first metric, a second metric, and a third metric. These obtained confidence values ​​(metrics) are then fused together to obtain a global confidence score. This global confidence score can then be used as input to the desired function.

[0066] The specific measures will be described in more detail below:

[0067] The first metric determined indicates whether the current input data (e.g., an image) follows the same distribution as the training dataset. This is necessary because the performance of a deep neural network can only generalize to new data if the new data comes from / follows the same distribution as the training data. Estimating whether new data originates from the training data distribution may require estimating the probability distribution of (potentially) high-dimensional data. This can be achieved, for example, with a standard autoencoder or a variational autoencoder (VAE). These machine learning models consist of an encoder part and a decoder part and can be trained in an unsupervised manner. The encoder part is trained to map the input data to a low-dimensional feature space (also called the latent space). The decoder part then attempts to reconstruct the original high-dimensional input from the latent space. Variational autoencoders can be used in several ways to predict whether new data originates from the same distribution as the data used to train the variational autoencoder. However, the most common approach is based on calculating the probability density value of the new sample in the latent space or using the reconstruction error of the variational autoencoder. A low density value in the latent space indicates a high confidence that the new data does not conform to the training distribution, and this can also be inferred from a large reconstruction error.

[0068] The second metric for generating the proposed security architecture relates to enabling deep neural networks to express confidence in their predictions. This module will be referred to as the introspection metric below. Deep neural networks are typically unable to make statements about the certainty / uncertainty of their predictions. Even when predicted probabilities can be used as values ​​of uncertainty, they are often poorly calibrated. To enable deep neural networks to predict uncertainty, it requires sophisticated additional methods. The most successful approach for this purpose is to leverage Bayesian modeling of neural networks, typically variational inference. Two possible approaches to performing variational inference in deep neural networks are MC Dropout and Bayes-by-Backprop. The general idea behind both approaches is to model the weights of the neural network as distributions rather than fixed values ​​(i.e., point estimates). To obtain the certainty / uncertainty of a prediction (e.g., classification) for a given input (e.g., image data), the weight distribution is sampled n times, and the set of weights from each sample is used once to predict n distinct outputs. The average of the sampled outputs is then used as the model's prediction, while the variance of the predictions is used to compute the uncertainty metric.

[0069] Besides variational inference, introspective uncertainty can also be obtained by attaching methods capable of predicting uncertainties due to design to a base deep neural network. For example, this can be achieved by combining a deep neural network with a Gaussian process.

[0070] Another approach to generating introspective uncertainty is through the ensemble decision-making of multiple independent deep neural networks, which are based on different architectures or have been trained using different training datasets.

[0071] The determined third metric can also be called output uncertainty. This metric helps to understand how and why the network makes certain decisions, and then draws conclusions about their rationality. A useful set of methods for this purpose is visualization methods. They help to understand which features of the input are important for making predictions, thus helping to discover misleading biases and correlations, as well as failure modes of neural networks. For example, weighted gradient class activation mapping (JCM) can localize and visualize class-discriminative features in the input. Such a JCM technique is described in detail, for example, in German patent applications DE102019209462 and DE102019209463.

[0072] For example, scaled activation heatmaps can be computed for all detected objects during training. These are then combined by class and by pixel to obtain a class activation probability distribution. This class activation probability distribution can be used for plausibility checks on new objects. At runtime, the scaled activation heatmaps of detected objects are compared to the class activation probability distribution to determine whether a sample belongs to or follows that distribution. Low likelihood values ​​indicate high output uncertainty.

[0073] For example, gradient-based visualization methods, such as weighted gradient class activation mapping, can be used to compute activation heatmaps for all images and all detected objects of relevant classes in the training dataset, and these heatmaps can be scaled to the corresponding bounding boxes. This is necessary because the size of the detections varies greatly.

[0074] For each class, a scaled activation probability distribution can be computed for each pixel inside and / or outside the bounding box: assuming a normal distribution, a multivariate Gaussian distribution is obtained by calculating the expected value and standard deviation for each pixel. This activation probability distribution for each class is related to the training data and can be used for plausibility checks on detections in new data ("whether the detections in the test image show a deviation from the seen data"). During runtime, the activation of each detected object can be computed, scaled, and compared with the activation probability distribution of the corresponding class.

[0075] This method can be mathematically implemented through the calculation of log-likelihood values. Anomaly detection can be accomplished by defining a threshold for the log-likelihood value. If the value is below this threshold, an anomaly is indicated. Furthermore, the log-likelihood value can be related to values ​​obtained during training and testing, and can be scaled and output as a confidence value. The lower the value, the lower the confidence of the neural network, because the typically used image regions / features are not used in the neural network's predictions.

[0076] In the step of determining global uncertainty, at least two of the first, second, and third measures obtained in the above steps can be combined into a single value that indicates the main neural network's global confidence in its outcome / prediction. This is a crucial insight for other functions / components that accept the output of the deep neural network and use it to make safety-related decisions. It also helps in selecting between different, even potentially contradictory, predictions in systems with sensor-level redundancy.

[0077] Combinations can be performed in several ways. For example, a conservative approach could be taken, allowing the measure with the lowest prediction confidence (or only one of the two measures) to win. In this case, only one probability is sent as the probability of the combination (winner takes all).

[0078] Another approach is to take the average of the different uncertainty values ​​(measures) or any other kind of weighted sum to produce a combined value.

[0079] In both of the above methods, the combination of measures relies on a rule-based approach that incorporates expert knowledge about the individual uncertainty values / measures. Alternatively, a function can be learned that takes at least two of the three measures as input and outputs the combined global uncertainty value. This function can also be a neural network or any other type of statistical method, such as a Gaussian process, a generalized linear model, a random forest, etc. The only requirement of the applied method is that it is used to map the space of at least two of the three measures to the space of the combined uncertainty. This may require a training process for the function that combines the uncertainty measures, which requires additional training data.

[0080] Figure 1 A first embodiment of a method for estimating the global uncertainty of the output data of a computer-implemented main neural network is shown. In this example, the neural network is designed to detect people, particularly facial recognition, for security-related applications. In this case, authorized persons are detected by the neural network, and entry into a specific building is permitted or denied based on the detection result. To estimate the accuracy of the neural network's predictions, the final decision is based on the global uncertainty stated by the neural network. The method for estimating this global uncertainty begins in step 101.

[0081] In step 102, a first metric is determined, which measures the degree to which the current input data of the neural network follows the same distribution as the data used to train the main neural network. This is accomplished using a variational autoencoder. Therefore, the probability density value of the new sample in the latent space is calculated. A low density value in the latent space indicates a high confidence that the new data does not conform to the training distribution; this can also be inferred from large reconstruction errors.

[0082] In step 103, a second metric is generated, which measures the determinism of the neural network in its own predictions based on the input data. Therefore, the neural network is designed as a Bayesian neural network. To obtain the second metric for a given input (e.g., image data of a person and their face), the weight distribution of the network is sampled n times, with each sampled set of weights used once to predict n distinct outputs. The average of the sampled outputs is then used as the model's prediction, while the variance of the predictions is used to compute the second metric.

[0083] In step 104, a third metric is determined based on estimates of class-discriminating features in the input data and comparisons of these features with class activation probability distributions. This is accomplished using a technique called weighted gradient class activation mapping. Therefore, during training, scaled activation heatmaps are computed for all detected objects and faces. These are then combined separately by class and by pixel to obtain class activation probability distributions. These class activation probability distributions are then used for plausibility checks of new objects. At runtime, scaled activation heatmaps of detected objects are compared with class activation probability distributions to determine whether a sample belongs to or follows that distribution. Low likelihood values ​​indicate high output uncertainty.

[0084] In step 105, the global uncertainty is determined based on at least two of the first, second, and third measures. In this example, all three measures are considered, and a conservative approach is chosen. The measure representing the minimum confidence level defines the global uncertainty. In this case, only one of the three measures is sent as the combined probability / global uncertainty.

[0085] The method ends in step 106.

[0086] In the second embodiment, three different confidence metrics (also referred to as the first, second, and third metrics) and global uncertainty are calculated using four different modules (in this case, different software modules), all of which operate on the same vehicle control unit. The overall method is as follows: Figure 1 As shown.

[0087] The neural network module 200 includes a neural network 202 and four modules 210, 220, 230, and 240.

[0088] Module 210 receives input data 201 and determines a first metric 211, which indicates whether the current input data follows the same distribution as the training dataset.

[0089] Module 220 expresses the confidence level of the neural network's predictions (second measure 221).

[0090] Module 230 evaluates the output uncertainty (third measure 231) based on the output data of neural network 202.

[0091] Module 240 determines the global uncertainty 241 based on the first measure 211, the second measure 221 and the third measure 231.

[0092] Therefore, the neural network module 200 receives input data 201 and outputs output data 203 of the neural network 202, as well as the uncertainties of the neural network: a first measure 211, a second measure 221, a third measure 231, and a global uncertainty 241. Depending on the application of this block, only the global uncertainty 241 or a combination of different uncertainties (the first, second, and third measures and the global uncertainty) can be given.

[0093] Figure 3 This demonstrates how to use it in an object detection system with early fusion of sensor data. Figure 2 The neural network module 200 described herein represents a neural network module, particularly a deep neural network module, that additionally outputs one or more uncertain values ​​relative to the output of a neural network.

[0094] 301 represents a sensor block that includes various sensors such as a camera device 311, radar 321, and lidar 331. In other embodiments, different or additional sensors such as microphones or ultrasonic sensors may also be used.

[0095] Different sensors 311, 321, and 331 generate sensor data, such as image 312, radar signal 322, and 3D scan from lidar sensor 331. This sensor data is fused in block 302. In this example, all sensor data 312, 322, and 332 are fed into neural network module 303, which is designed to output a neural network prediction 305 and an uncertainty 304 regarding that prediction 305.

[0096] Such object detection systems are particularly important for driver assistance (DA) or autonomous driving (AD) functions. Due to high safety requirements, understanding the predictive uncertainty of neural networks, especially deep neural networks, is crucial. For example, depending on the magnitude of global uncertainty, subsequent functions may degrade.

Claims

1. A method for estimating the global uncertainty of output data of a computer-implemented main neural network of a perception system for at least a partially automated vehicle, the at least partially automated vehicle being controlled based on the global uncertainty, the method comprising the steps of: Determine (102) a first metric, which measures the degree to which the current input data of the main neural network follows the same distribution as the data used to train the main neural network, the input data being from at least one sensor; Generate (103) a second metric that measures the determinism of the main neural network in its own predictions based on the input data; Based on the estimation of class-discriminating features in the input data and the comparison of said features with the class activation probability distribution, a third metric (104) is determined, wherein the class activation probability distribution is created based on the estimated class-discriminating features during the training of the main neural network, wherein the class-discriminating features are estimated by using an activation heatmap to compute scaled and / or unscaled features, wherein scaled activation heatmaps of detected objects are compared with class activation heatmaps computed for all detected objects during training, and then the class activation heatmaps are combined by class and / or by pixel to obtain the class activation probability distribution; The (105) global uncertainty is determined based on at least two of the first, second, and third measures.

2. The method according to claim 1, wherein, The determination of the first measure (102) is achieved by means of an autoencoder.

3. The method according to claim 2, wherein, An autoencoder includes an encoder part and a decoder part, wherein the encoder part is trained to map input data to a low-dimensional feature space, i.e., a latent space, and the decoder part attempts to reconstruct the original higher-dimensional input from the latent space, wherein a first metric is based on the calculation of the probability density value of the input data mapped to the latent space or on the reconstruction error estimated by the autoencoder.

4. The method according to any one of claims 1-3, wherein, The generation of the second metric (103) is based on Bayesian modeling of the main neural network.

5. The method according to claim 4, wherein, The weights of the main neural network are modeled as a distribution.

6. The method according to claim 5, wherein, The training of the main neural network is performed using variational inference.

7. The method according to claim 6, wherein, The main neural network is modeled as a Bayesian neural network, wherein variational inference of the Bayesian neural network is performed by means of numerical integration, wherein the weight distribution is sampled n times and the set of weights for each sample is used once to predict n distinct outputs for a given input, wherein the average of the sampled outputs is then used as the prediction of the main neural network, and the variance of the prediction is used to compute a second metric.

8. The method according to any one of claims 1-3, wherein, A second measure is obtained by enhancing the neural network using Gaussian processes.

9. The method according to any one of claims 1-3, wherein, The second metric is generated through the ensemble decision-making of at least two neural networks.

10. The method according to any one of claims 1-3, wherein, The second metric is generated through ensemble decision-making using different machine learning methods.

11. The method according to any one of claims 1-3 and 5-7, wherein, Class-discriminating features are estimated by using a weighted gradient class activation mapping method.

12. The method according to any one of claims 1-3 and 5-7, wherein, Compare the first, second, and third measures and set the global uncertainty to the value representing the lowest confidence level among the three measures, or where the global uncertainty is determined based on the average or weighted sum of the first, second, and third measures.

13. The method according to any one of claims 1-3 and 5-7, wherein, Vehicle path planning is based on global uncertainty, thereby avoiding areas of high uncertainty when controlling the vehicle.

14. The method according to claim 1, wherein, A second metric is generated by using at least one Bayesian method or by using an ensemble-based method.

15. The method according to claim 2 or 3, wherein, The autoencoder is a variational autoencoder.

16. The method according to claim 4, wherein, The main neural network is modeled as a Bayesian neural network.

17. The method according to claim 9, wherein, The two neural networks have different network architectures or have been trained using different training datasets.

18. The method according to claim 10, wherein, At least one of the different machine learning methods is a neural network.

19. The method according to claim 13, wherein, Avoid areas of high uncertainty by maintaining a predetermined distance from them.

20. The method according to claim 13, wherein, The high uncertainty region is the region with high uncertainty compared to the predetermined threshold.

21. A computer program product comprising instructions that, when executed by a computer, cause the computer to perform the method according to any one of claims 1-20.

Images