Stereo-Lidar Self-Calibration for Reliable 3D Depth Mapping
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional 3D vision systems, including lidar and stereo vision systems, face challenges in providing reliable and accurate range information in adverse conditions and suffer from calibration inaccuracies that affect depth determinations, especially in autonomous machines where real-time decision-making is crucial.
Innovation Solution
A combined vision system that automatically calibrates extrinsic geometric parameters between a stereo camera system and a lidar system using simultaneous data from both, enabling self-calibration without external targets, to enhance accuracy and reliability of depth measurements.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If conventional lidar and stereo vision systems are used separately, then each system can operate independently, but calibration inaccuracies between systems affect depth determination reliability
Solution Approach 1:
The patent combines lidar and stereo vision systems into a unified calibration framework where both sensors observe the same scene simultaneously. By merging the depth information from lidar with the stereo disparity data, the system performs joint calibration that leverages the complementary strengths of both sensors, thereby improving depth determination reliability while managing calibration complexity through integrated processing.
Solution Approach 2:
The patent introduces an image processing pipeline that acts as an intermediary between the raw sensor data and the calibration process. The stereo images are processed to extract disparity maps and depth information, which then serve as intermediate representations for comparing with lidar point clouds. This intermediary processing layer facilitates accurate correspondence matching between different sensor modalities, enabling reliable cross-system calibration.
2Loss of time
If stereo vision systems are used to provide real-time depth information for autonomous vehicle control, then reaction time is improved, but measurement accuracy deteriorates in adverse conditions such as rain, snow, fog, or when reflected light is not readily discernable
Solution Approach 1:
The patent performs preliminary calibration of the stereo vision system using lidar data before the autonomous vehicle needs to make critical depth-based decisions. By pre-calibrating the stereo system under various conditions and storing calibration parameters, the system ensures accurate depth measurements are available in real-time during adverse conditions without needing to perform complex calibration computations during the critical reaction window.
Solution Approach 2:
The patent implements a feedback mechanism where lidar depth measurements are continuously compared with stereo vision depth estimates. When discrepancies are detected, the system uses the accurate lidar measurements as ground truth to refine and update the stereo vision calibration parameters. This feedback loop ensures that the stereo system maintains high measurement precision even in adverse conditions by continuously learning from reliable lidar data.
3Measurement precision
If external calibration targets are used to calibrate stereo and lidar systems, then calibration accuracy is improved, but the system cannot perform self-calibration and requires additional equipment
Solution Approach 1:
The patent enables the vision system to perform self-calibration by using the scene itself as the calibration target. The system processes stereo images to identify corresponding features between left and right camera views, computes disparity maps, and uses these along with lidar point clouds to automatically determine and refine extrinsic calibration parameters. This self-service calibration approach eliminates the need for external calibration targets while maintaining adaptability to various operating conditions.
Solution Approach 2:
The patent develops a universal calibration method that works across different scenarios and environments without requiring specialized calibration equipment. The same calibration pipeline processes both structured calibration scenes and unstructured real-world scenes, leveraging the multi-functionality of the stereo-lidar fusion approach to provide consistent calibration accuracy whether using controlled calibration targets or natural scene features.
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
The system provides enhanced point clouds with increased density and accuracy, overcoming limitations of individual systems, ensuring reliable and timely depth data for safe operation of autonomous machines.
Implementation Method 1
Lidar vision systems typically use light (e.g., laser light) reflected from an object to determine a range or distance to the object, based on an elapsed-time measurement for emitted light to reach the object, to reflect from the object, and to return and be detected.
Implementation Method 2
based on an elapsed-time measurement for emitted light to reach the object, to reflect from the object, and to return and be detected
Data Source
AI summary
An automatically calibrated vision system includes: a vision calibration system; a first sensor system that receives first data of a scene captured by a first sensor on a movable machine and outputs a first map based on the first data; a second sensor system that receives second data of the scene captured by a second sensor on the movable machine and outputs a second map based on the second data. The vision calibration system computes calibration data based on the first and second maps, supplies the calibration data to the first sensor system and/or the second sensor system for automatic calibration of the first sensor system and/or the second sensor system, and outputs, to a controller of the movable machine, a calibrated depth map comprised of depth measurements calibrated using the calibration data. The first sensor system and/or the second sensor system automatically perform(s) a self-calibration using the calibration data.