Vehicle Perception Diagnostics Using Temporal Sensor Continuity
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Perception systems in autonomous vehicles face challenges in diagnosing faults without ground truth, as it is difficult to determine real-time failures when the vehicle is in motion, and existing methods struggle to differentiate between sensor and processing errors.
Innovation Solution
A system comprising a comparing module, a perception module, and a diagnostics module that calculates temporal continuity of sensor data and perception results, using methods like intersection over union (IoU) pairing, filtering, and Euclidean distance to identify faults, and performs mitigation procedures when faults are detected.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If the vehicle is in motion, then the perception system can operate in real-time, but it becomes difficult to determine ground truth and diagnose faults
Solution Approach 1:
The system performs preliminary actions by collecting and storing sensor data at multiple timestamps before fault diagnosis is needed. This pre-collection of temporal data enables later comparison and fault detection without requiring ground truth during actual operation, thus maintaining real-time performance while enabling accurate diagnostics.
Solution Approach 2:
The system implements feedback by continuously comparing sensor data across different timestamps and using this comparison to detect inconsistencies that indicate faults. The feedback mechanism allows the system to self-diagnose without external ground truth, resolving the contradiction between real-time operation and fault detectability.
2Device complexity
If traditional fault diagnosis methods are used, then the system structure remains simple, but the system cannot differentiate between sensor faults and processing errors
Solution Approach 1:
The system segments the fault diagnosis process into two independent comparison channels: one comparing raw sensor data temporal consistency and another comparing perception results temporal consistency. This segmentation enables precise differentiation between sensor faults (detected in data comparison) and processing errors (detected in results comparison) while maintaining a relatively simple overall system structure.
Solution Approach 2:
The system introduces an intermediary comparison mechanism that acts as a mediator between sensor data and fault diagnosis. By introducing this intermediate comparison layer that operates on both raw data and processed results, the system achieves accurate fault differentiation without significantly increasing structural complexity.
3Measurement precision
If ground truth is used for fault diagnosis, then diagnostic accuracy is high, but additional hardware and systems are required
Solution Approach 1:
The system implements self-service by using its own sensor data and perception results to perform fault diagnosis without requiring external ground truth systems. The temporal comparison mechanism allows the system to self-evaluate its own performance and detect faults using only internally generated data, eliminating the need for additional diagnostic hardware while maintaining high diagnostic accuracy.
Solution Approach 2:
The system applies multi-functionality by using the sensor data and perception processing pipeline for both their primary function (environment perception) and secondary function (self-diagnosis). This universal use of existing components enables accurate fault diagnosis without requiring dedicated diagnostic hardware, as the same sensors and processors serve dual purposes.
Data Source
AI summary
A comparing module receives first data regarding surroundings of a vehicle from a plurality of sensors in the vehicle, receives second data regarding the surroundings from the plurality of sensors after receiving the first data, compares the first data to the second data, and determines a first difference between the first data and the second data based on the comparison of the first data to the second data. A perception module generates a first set of perception results based on the first data, generates a second set of perception results based on the second data, and determines a second difference between the first data and the second data based on the first set of perception results and the second set of perception results. A diagnostics module determines whether one of the sensors or the perception module is faulty based on a combination of the first difference and the second difference.