Vehicle Camera Calibration Using Vanishing Points and Pose Graphs
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing online calibration methods for autonomous driving systems are limited in calibrating a complete vehicle camera system, especially when relying on odometry sensor measurements, which can be inaccurate, and lack general solutions for surround field of view coverage, leading to potential inaccuracies in driver assistance perception due to camera orientation drifts over time.
Innovation Solution
An online calibration system that uses a combination of vehicle sensor measurements and camera perception, where front cameras are calibrated based on the vanishing point of lane lines at high speeds and side and rear cameras are calibrated using a pose graph optimization algorithm when speeds are lower, ensuring accurate orientation adjustments without the need for controlled environments or extensive human labor.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If offline calibration with controlled environment equipment is used, then calibration accuracy is improved, but equipment cost and human labor requirements increase
Solution Approach 1:
The system performs self-calibration using its own camera sensors to detect lane lines and calculate vanishing points, eliminating the need for external calibration equipment. The vehicle uses its perception system to automatically determine camera orientation parameters while driving on normal roads.
Solution Approach 2:
The patent introduces lane lines as an intermediary element that mediates between the camera system and the environment. By detecting lane lines in the natural environment and calculating their vanishing point, the system obtains calibration information without requiring dedicated calibration equipment.
2Measurement precision
If offline calibration is performed at service stations, then initial calibration is achieved, but re-calibration after camera displacement becomes costly and labor-intensive
Solution Approach 1:
The system transitions from static offline calibration to dynamic online calibration. The calibration process occurs continuously while the vehicle is moving, allowing automatic re-calibration after camera displacement without returning to service stations. The vanishing point calculation adapts to the vehicle's current camera orientation in real-time.
3Ease of operation
If odometry-based online calibration methods are used, then calibration can be performed during driving, but measurement accuracy decreases due to accumulated error
Solution Approach 1:
The patent extracts the calibration function from the odometry system and implements it independently through vision-based vanishing point detection. By separating the calibration process from position tracking, the system avoids accumulating odometry errors while maintaining online calibration capability.
4Device complexity
If pure-vision based calibration is applied to complete camera systems, then equipment cost is reduced, but calibration generality and coverage are limited
Solution Approach 1:
The vanishing point-based calibration method is designed to be universally applicable to complete surround-view camera systems. The same principle of detecting lane lines and calculating vanishing points works for calibrating multiple cameras with different fields of view, making the solution generalizable across various camera configurations.
Data Source
AI summary
Aspects of online camera calibration for autonomous driving systems are described herein. The aspects may include a wheel odometer configured to measure a current speed of a moving vehicle, a speed monitor configured to determine that the current speed of the moving vehicle is high or low, and a camera calibrator configured to calibrate, while the vehicle is moving, at least one front camera of the moving vehicle according to a position of a vanishing point of two lane lines when the current speed is high. When the current speed is low and the vehicle is still moving, the camera calibrator may be further configured to calibrate multiple side and rear cameras of the moving vehicle according to a pose graph that includes relative poses of the multiple side and rear cameras.