Sequential Multi-Sensor Calibration Using Lane Marker References
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Autonomous vehicles face challenges in accurately calibrating multiple sensors such as cameras, LiDAR, and Radars to detect lane markers and objects, which is crucial for safe navigation, due to asynchronous sensor data and potential errors in GPS and IMU measurements.
Innovation Solution
A multi-sensor sequential calibration technique that uses a combination of GPS, IMU, and camera data to optimize extrinsic matrices of sensors by associating sensor data with pre-defined road markers, employing factor graphs for non-linear optimization to minimize errors and achieve precise pose transforms between sensors.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If multiple sensors (cameras, LiDAR, Radars) are used for autonomous vehicle navigation, then the capability to detect lane markers and objects is improved, but the complexity of calibrating these sensors increases due to asynchronous sensor data and potential errors in GPS and IMU measurements
Solution Approach 1:
The patent segments the calibration process into multiple sequential steps: first calibrating the camera using lane markers, then using the calibrated camera data to calibrate other sensors. This breaks down the complex multi-sensor calibration into manageable stages, reducing overall system complexity while maintaining accuracy.
Solution Approach 2:
The patent introduces lane markers as an intermediary reference object that facilitates calibration. The lane markers serve as a common reference frame that connects camera observations with vehicle pose information from GPS and IMU, enabling accurate calibration without direct sensor-to-sensor calibration.
2Extent of automation
If GPS and IMU measurements are used for sensor calibration, then the calibration process becomes more automated, but errors in these measurements reduce the precision of lane marker detection and object localization
Solution Approach 1:
The patent implements feedback by using the calibrated camera observations of lane markers to correct and refine the GPS and IMU measurements. The system continuously compares observed lane marker positions with expected positions based on GPS/IMU data, using the difference to adjust the calibration parameters and reduce accumulated errors.
Solution Approach 2:
The patent performs preliminary calibration of the camera using lane markers before using the camera to calibrate other sensors. This preliminary action establishes an accurate reference frame that compensates for GPS and IMU errors, ensuring high precision in subsequent calibration steps.
3Measurement precision
If sequential calibration using factor graphs and non-linear optimization is employed, then extrinsic calibration precision is improved, but the computational time and processing requirements increase
Solution Approach 1:
The patent segments the optimization problem into smaller sub-problems that can be solved sequentially. Instead of optimizing all sensor parameters simultaneously, the system performs separate optimizations for different sensor groups and calibration stages, reducing computational complexity while maintaining precision.
Solution Approach 2:
The patent performs preliminary calculations and pre-processing of sensor data before the main optimization routine. By pre-computing certain transformation matrices and filtering sensor data in advance, the system reduces the computational burden during the actual non-linear optimization, decreasing processing time while preserving accuracy.
Data Source
AI summary
Techniques for performing a sensor calibration using sequential data is disclosed. An example method includes receiving, from a first camera located on a vehicle, a first image comprising at least a portion of a road comprising lane markers, where the first image is obtained by the camera at a first time; obtaining a calculated value of a position of an inertial measurement (IM) device at the first time; obtaining an optimized first extrinsic matrix of the first camera by adjusting a function of a first actual pixel location of a location of a lane marker in the first image and an expected pixel location of the location of the lane marker; and performing autonomous operation of the vehicle using the optimized first extrinsic matrix of the first camera when the vehicle is operated on another road or at another time.