Autonomous Vehicle Sensor Calibration with Camera-LiDAR Fusion
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current sensor calibration techniques for autonomous vehicles require infrastructure, are manual, slow, imprecise, and computationally expensive, leading to potential safety risks and downtime due to the need for bringing the system to a calibrated location.
Innovation Solution
A method using epipolar geometry and lidar data to calibrate sensors on autonomous vehicles without fiducial markers, allowing for real-time correction of misalignment and intrinsic parameters based on image data from multiple cameras, leveraging expectation-maximization algorithms to optimize calibration.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If current calibration techniques using infrastructure are used, then calibration can be performed, but the system requires bringing the vehicle to a calibrated location resulting in downtime and potential safety risks
Solution Approach 1:
The system performs self-calibration using its own sensors (cameras and LIDAR) to capture images of the environment and automatically determine calibration parameters without requiring external infrastructure or manual intervention, thereby eliminating downtime and safety risks associated with transporting the vehicle to calibration locations
Solution Approach 2:
The patent introduces environmental features (natural or artificial objects in the scene) as intermediaries to establish correspondences between sensors' coordinate systems, replacing the need for dedicated calibration infrastructure while enabling accurate calibration through feature matching and geometric relationships
2Measurement precision
If manual calibration with human operators is used, then calibration can be performed, but the process becomes manual, slow, and potentially imprecise
Solution Approach 1:
The patent replaces manual mechanical calibration operations with an automated computational system that uses computer vision algorithms, feature matching, and optimization techniques to automatically determine calibration parameters from captured images, thereby eliminating human operators and significantly increasing calibration speed while maintaining or improving precision
Solution Approach 2:
The system automatically processes captured images, identifies corresponding features across sensors, computes geometric relationships, and determines calibration parameters without human intervention, making the calibration process fully autonomous and dramatically improving productivity
3Adaptability or versatility
If existing calibration techniques that mitigate infrastructure requirements are used, then calibration can be performed without infrastructure, but the process becomes computationally expensive
Solution Approach 1:
The patent performs computational optimization only when calibration is needed rather than continuously, and uses efficient feature matching algorithms that process only relevant environmental features, thereby reducing computational energy consumption while maintaining calibration flexibility and adaptability
Data Source
Figure 1
Figure 2
Figure 3
AI summary
This disclosure is directed to calibrating sensors mounted on an autonomous vehicle 104. First image data 110(1) and second image data 110(2) representing an environment can be captured 102 by first and second cameras 106(1) and 106(2), respectively (and/or a single camera at different points in time). Point pairs comprising a first point in the first image data and a second point in the second image data can be determined 112 and projection errors associated with the points can be determined 134. The operation 112 may determine correspondence between points, e.g., identify the point pairs, using feature matching. A subset of point pairs can be determined 150, e.g., by excluding point pairs with the highest projection error. Calibration data associated with the subset of points can be determined 152 and used to calibrate the cameras without the need for calibration infrastructure. Process 100 uses epipolar geometry to correct for misalignment, e.g., physical misalignment, of cameras on the autonomous vehicle. Lidar data, which does consider the three- dimensional characteristics of the environment (e.g., feature edges) can be used to further constrain the camera sensors, thereby removing any scale ambiguity and/or translational / rotational offsets. In an embodiment, the process may perform several of the operations in parallel, e.g., to solve for the extrinsic calibration data and the intrinsic calibration data from the same image data and at the same time (e.g., a joint optimization).