Neural Network Prediction Reliability Assessment via Outlier Detection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Deep neural networks in automated driving systems face challenges in evaluating the reliability of predictions without human-annotated labels, requiring significant computational power, which hinders real-time applications.
Innovation Solution
A method using multiple outlier detection methods to assess the reliability of predictions by determining a class probability score and comparing it to a threshold value, allowing for efficient evaluation of neural network predictions with low computational power.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If multiple outlier detection methods are used to evaluate prediction reliability, then measurement precision is improved, but device complexity increases
Solution Approach 1:
Outlier detection methods are fitted to the neural network output during the training phase, performing preliminary analysis so that during real-time prediction, only simple threshold comparisons are needed, reducing online computational complexity
Solution Approach 2:
A class probability score is introduced as an intermediary metric that translates complex outlier detection results into a single reliable probability value that can be easily compared against threshold values for final prediction reliability assessment
2Reliability
If computational power is increased to evaluate prediction reliability, then reliability is improved, but use of energy increases
Solution Approach 1:
The computationally intensive outlier detection methods are executed during the offline training phase rather than during real-time prediction, shifting energy consumption to when energy availability is less constrained and pre-computing results for faster, lower-energy online evaluation
Solution Approach 2:
Simple threshold comparison operations are used during real-time prediction instead of complex computational models, replacing energy-intensive calculations with low-cost, disposable threshold checks that consume minimal energy
3Speed
If real-time evaluation is implemented, then speed is improved, but measurement precision may worsen
Solution Approach 1:
Complex outlier detection analyses are performed in advance during training to establish reliable threshold values, enabling fast real-time evaluation to maintain speed while the pre-computed thresholds ensure measurement precision is preserved
Solution Approach 2:
The essence of complex outlier detection is captured in simplified threshold values that replicate the reliability assessment function during real-time operation, maintaining accuracy while enabling high-speed evaluation
Data Source
AI summary
A method for assessing the reliability of a prediction made by a deep neural network includes inputting input data to a trained deep neural network, determining a class probability score of the detection, wherein the class probability score is determined by fitting of at least two multiple outlier detection methods, determining whether the input data is or is not an outlier, comparing the class probability score to a threshold value, and outputting that the prediction is or is not reliable. Also disclosed is a system and a computer readable storage medium.