AI Object Detection Validation Using Map-Based Feature Comparison
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing AI-based systems for object detection in autonomous or semi-autonomous vehicles, particularly in rail transport, lack reliability due to overconfidence in out-of-distribution data, leading to incorrect predictions under varying environmental conditions or sensor malfunctions, which poses a safety risk.
Innovation Solution
A method and device that utilize an auxiliary prediction system trained on the same data as the AI-based system, comparing extracted features with field comparison data from a digital map to determine the reliability of object detection, and adjust sensor weights or operation modes based on validation results.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If AI-based systems are used for object detection in autonomous vehicles, then detection effectiveness is improved, but reliability deteriorates due to overconfidence in out-of-distribution data
Solution Approach 1:
The patent implements a feedback mechanism where the system continuously monitors the distribution of input data and compares it against the training data distribution. When OOD data is detected, the system provides feedback to adjust confidence scores and trigger safety responses. This feedback loop resolves the contradiction by enabling the system to maintain high detection effectiveness while correcting reliability issues through real-time distribution monitoring and adaptive response.
Solution Approach 2:
The patent changes the parameter of confidence score adjustment based on data distribution characteristics. By monitoring statistical parameters of input data (such as feature distribution, image statistics) and comparing them against training data parameters, the system dynamically adjusts confidence levels. This parameter change approach allows the system to maintain effectiveness on known patterns while reducing overconfidence in unfamiliar patterns, thus resolving the reliability issue.
2Measurement precision
If deep learning-based methods are used for environmental perception, then detection accuracy is improved, but system complexity increases
Solution Approach 1:
The patent introduces an intermediary layer between the deep learning detection system and the control system. This intermediary layer includes a distribution monitoring module that acts as a mediator, analyzing data characteristics and providing simplified reliability assessments to the control system. This intermediary structure allows the complex deep learning system to maintain high detection accuracy while the intermediary layer manages and simplifies the complexity for safety-critical decision-making.
Solution Approach 2:
The patent segments the system into distinct functional modules: the deep learning detection module, the distribution monitoring module, and the control module. By segmenting the system, each module can be optimized independently - the deep learning module for detection accuracy and the monitoring module for reliability assessment. This segmentation reduces overall system complexity by making each component more manageable and easier to validate for safety-critical operations.
3Speed
If sensor data is used for object detection, then real-time detection capability is improved, but susceptibility to environmental changes increases
Solution Approach 1:
The patent applies preliminary action by monitoring data distribution characteristics before they affect detection results. The distribution monitoring module analyzes sensor data in advance to identify patterns that indicate OOD conditions (such as unusual lighting, weather, or object characteristics). By detecting these patterns beforehand, the system can take preliminary corrective actions - adjusting confidence scores or triggering safety responses - before incorrect detections are made, thus maintaining detection speed while reducing environmental sensitivity.
Data Source
Figure 1~2
Figure 3~4
Figure 5~6
AI summary
A method for validating the function of an AI-based system (TNN) for detecting, segmenting, and classifying objects in the environment (U) of a means of transport (61) is described. In this method, sensor data (SD) from the environment (U) of the means of transport (61) are acquired, and input data (ED) for the AI-based system (TNN) are generated based on the sensor data (SD). Furthermore, feature data (MD) of static objects are determined based on the input data (SD) or from intermediate layer data (ZSD) of the AI-based system (TNN) using an auxiliary prediction system (APS). The determined feature data (MD) are compared with field comparison data (FVD), which comprises features of static objects in the environment (U). Based on the comparison result (VE), the reliability of the trained neural network (TNN) is determined. A method for generating field comparison data (FVD) is also described.A validation device (50) is also described. Furthermore, a means of transport (61) is described.