Vehicle Sensor Validation Using Lidar-Camera Feature Matching
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional sensor verification processes for autonomous vehicles are labor-intensive and costly, and may not detect sensor inaccuracies in a timely manner, leading to potential navigation issues.
Innovation Solution
A computing device configured to continuously monitor and validate sensor data by comparing feature descriptors from lidar and camera images to determine the accuracy of sensor data, including lidar data points and camera poses, allowing for real-time detection of inaccuracies.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If conventional sensor verification processes are used, then sensor calibration can be performed, but the process is labor-intensive and costly
Solution Approach 1:
The system performs self-verification by using the vehicle's own sensors (camera, lidar, pose sensor) to validate each other's data. The processor automatically compares sensor readings without requiring external calibration targets or technician intervention, enabling the system to self-validate its sensor accuracy during normal operation.
Solution Approach 2:
The system continuously monitors sensor data and provides feedback on calibration accuracy. By comparing feature descriptors from camera images with lidar data points and pose sensor information, the system generates feedback signals that indicate whether sensors remain properly calibrated, enabling real-time validation without external intervention.
2Reliability
If conventional sensor verification processes are used, then sensor calibration can be performed, but setup and maintenance of testing environment incurs high costs
Solution Approach 1:
The system uses the vehicle's existing sensor suite and natural environment to perform verification, eliminating the need for expensive external calibration facilities. The processor leverages readily available data from camera, lidar, and pose sensors to validate calibration without requiring specially curated calibration targets or controlled testing environments.
Solution Approach 2:
The verification system serves multiple functions simultaneously: it validates camera calibration, checks lidar geometric accuracy, and verifies pose sensor calibration using a single integrated process. This multi-functional approach eliminates the need for separate verification processes and facilities for each sensor type, reducing overall costs.
3Reliability
If conventional sensor verification processes are used, then sensor calibration can be performed, but sensor inaccuracies cannot be detected in a timely manner
Solution Approach 1:
The system performs continuous verification during vehicle operation rather than periodic offline calibration. The processor continuously compares sensor data and feature descriptors in real-time, enabling immediate detection of calibration drift or sensor inaccuracies as they occur, without requiring the vehicle to be taken out of service for verification.
Solution Approach 2:
The system provides real-time feedback on sensor accuracy by continuously monitoring the consistency between camera, lidar, and pose sensor data. When discrepancies exceed thresholds indicating potential calibration issues, the system immediately generates alerts, enabling timely detection and response to sensor inaccuracies during normal operation.
Data Source
AI summary
In one example, a method is provided that includes receiving lidar data obtained by a lidar device. The lidar data includes a plurality of data points indicative of locations of reflections from an environment of the vehicle. The method includes receiving images of portions of the environment captured by a camera at different times. The method also includes determining locations in the images that correspond to a data point of the plurality of data points. Additionally, the method includes determining feature descriptors for the locations of the images and comparing the feature descriptors to determine that sensor data associated with at least one of the lidar device, the camera, or a pose sensor is accurate or inaccurate.