Extrinsic Calibration of Cameras and 2D Lidars
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for extrinsic calibration of imaging sensing devices and 2D LIDARs on transportable apparatus, such as cameras and LIDARs, require accurate measurement or user intervention, which is impractical for robust long-term autonomy and safety-critical systems that need continuous calibration validation.
Innovation Solution
A method that uses a 2D push-broom LIDAR to create a 3D point cloud with trajectory estimates from INS or visual odometry, projecting it into a camera image to generate a laser reflectance image, and employing an edge-based, weighted Sum of Squared Difference objective function for grid-based optimization to determine the SE3 transformation between sensors without artificial targets or human intervention.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If physical measurement of sensor positions is used, then calibration accuracy can be achieved, but the process becomes more complicated due to sensor casings preventing accurate measurement of sensing elements
Solution Approach 1:
The patent introduces a laser reflectance image as an intermediary representation that bridges the LIDAR and camera data. By projecting LIDAR point clouds into a synthetic image space and comparing it with the actual camera image, the system avoids direct physical measurement of sensor elements while achieving accurate calibration through image alignment optimization.
Solution Approach 2:
The patent creates a synthetic laser reflectance image that copies the visual appearance of the scene as captured by the camera, but generated from LIDAR data. This synthetic copy allows for direct comparison and alignment without needing to physically access or measure the sensor elements themselves.
2Measurement precision
If calibration targets are placed in the workspace, then extrinsic calibration can be performed, but continuous assessment of calibration accuracy becomes impractical
Solution Approach 1:
The system performs self-calibration by using the environment itself as the calibration reference. The laser reflectance image synthesizes scene appearance from LIDAR data, and the optimization process automatically aligns this synthetic representation with the camera image, eliminating the need for external calibration targets and enabling continuous self-assessment.
Solution Approach 2:
The calibration process can be performed continuously as the transportable apparatus moves through the environment. By using natural scene features and the optimized alignment metric, the system can repeatedly assess and refine calibration accuracy without requiring periodic placement of calibration targets, enabling continuous validation of calibration state.
3Measurement precision
If user intervention is required for calibration, then calibration can be performed, but robust long-term autonomy is compromised
Solution Approach 1:
The system performs completely automatic self-calibration without requiring user intervention. The optimization algorithm automatically adjusts the extrinsic calibration parameters to maximize the alignment between the laser reflectance image and the camera image, enabling robust long-term autonomous operation.
Solution Approach 2:
The system uses an alignment metric as feedback to guide the optimization process. By continuously evaluating how well the synthetic laser reflectance image matches the actual camera image and adjusting calibration parameters accordingly, the system achieves automatic calibration without user input, enabling autonomous operation.
4Adaptability or versatility
If sensors are mounted on transportable apparatus, then mobility is achieved, but calibration validation after bumps, knocks and vibrations becomes necessary
Solution Approach 1:
The system enables continuous calibration validation by repeatedly performing the image alignment optimization as the apparatus moves through the environment. This continuous assessment detects calibration drift caused by bumps, knocks, and vibrations, allowing the system to maintain reliable calibration despite mobility-related disturbances.
Data Source
Figure 1~3
Figure 2
Figure 4(a)~5
AI summary
A method and system for determining extrinsic calibration parameters for at least one pair of sensing devices mounted on transportable apparatus obtains (202) image data (110)representing images captured by an image generating sensing device (102) of the pair at a series of poses during motion through an environment (120) by transportable apparatus (100). The method obtains (202) data (112) representing a 3D point cloud based on data produced by a 2D LIDAR sensing device (104) of the pair. The method selects (204) an image captured by the image at a particular pose. The method generates (210) a laser reflectance image based on a portion of the point cloud corresponding to the pose. The method computes (212) a metric measuring alignment between the selected image and the corresponding laser reflectance image and uses (214) the metric to determine extrinsic calibration parameters for at least one of the sensing devices.