Radar Lidar Confidence Score Calibration for Automated Driving
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current automatic vehicle steering systems face challenges in accurately classifying radar and lidar data due to over-confidence in predictions, noise, interference, and ambiguity in object recognition, leading to potential incorrect actions that could result in damage or injury.
Innovation Solution
A method that uses a classifier to map radar or lidar data to confidence scores, followed by a post-hoc calibration process to adjust these scores for better accuracy, and determines actuation signals based on these calibrated scores to ensure appropriate vehicle actions, while also considering the severity of potential collisions and fusing data from multiple sensors.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If a classifier is trained with one-hot ground truth labels to improve classification speed, then classification speed is improved, but classification accuracy deteriorates due to over-confidence in predictions
Solution Approach 1:
The patent applies preliminary action by performing post-hoc calibration on the classifier's confidence scores after the initial classification. The calibration process adjusts the over-confident predictions by comparing them with actual classification accuracy metrics, thereby correcting the over-confidence issue without retraining the classifier. This maintains the speed advantage while improving accuracy.
2Loss of time
If confidence scores are used to determine vehicle actions to improve response time, then response time is improved, but reliability deteriorates due to over-confidence leading to incorrect actions
Solution Approach 1:
The patent implements feedback by using the calibrated confidence scores (which reflect actual classification accuracy) to determine vehicle actions. The calibration process provides feedback on the true reliability of each classification, allowing the system to make more reliable decisions. This feedback mechanism ensures that actions are taken based on accurate confidence assessments rather than over-confident predictions.
3Ease of operation
If the classifier outputs high confidence scores for all predictions to improve decision-making, then decision-making is simplified, but trustworthiness deteriorates due to mismatch between confidence and actual accuracy
Solution Approach 1:
The patent applies parameter changes by transforming the confidence scores through a calibration function that adjusts them to match actual classification accuracy. Instead of outputting raw high confidence scores, the system transforms these scores into calibrated values that accurately reflect the true reliability. This maintains the simplicity of using confidence scores for decision-making while restoring trustworthiness through parameter transformation.
Data Source
Figure 1
Figure 2a~2b
Figure 3
AI summary
A method (100) for evaluating at least one record of radar or lidar data (2), wherein the data (2) comprises the dependence of at least one measurement quantity that has been derived from reflected radar or lidar radiation on spatial coordinates, the method comprising the steps of: • mapping (110), by means of a given classifier (1), the record of radar or lidar data (2) to a set of confidence scores (3) with respect to classes of a given classification; • processing (120) the set of confidence scores (3), and/or an intermediate product from which the classifier computes confidence scores (3), by means of a post-hoc calibration operation (4) that is configured to match the confidence scores (3) to the classification accuracy (1a) of the classifier (1) with respect to the record of radar or lidar data (2); and • outputting (130) the so-processed set of confidence scores (3), and/or a set of confidence scores (3) that results from the so-processed intermediate product, as an evaluation result (3*) of the record of radar or lidar data (2). A method (200) for training at least one post-hoc calibration operation (4) for use in the method (100).