LiDAR-Camera Calibration via Normal Vector Extraction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Heterogeneous multi-sensor systems, such as those combining LiDAR and cameras, face challenges in calibrating sensors due to differences in temporal or spatial resolution and geometric misalignment, leading to suboptimal decision-making in applications like autonomous vehicles or robotics.
Innovation Solution
A calibration system that extracts normals from calibration patterns in both image frames and point cloud data frames to compute extrinsic calibration parameters, allowing for precise calibration of LiDAR sensors with cameras, even when mounted on vehicles in operational states.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If traditional calibration methods are used for heterogeneous multi-sensor systems, then calibration can be performed, but computational expense is high and precision is reduced due to geometric misalignment and resolution differences
Solution Approach 1:
The patent extracts normals from calibration patterns in both image frames and point cloud frames, isolating the essential geometric features needed for calibration. This extraction approach simplifies the calibration problem by focusing only on the critical normal vectors rather than processing all sensor data points, thereby reducing computational complexity while maintaining calibration precision.
Solution Approach 2:
The patent transforms the calibration problem by computing a transform between normals (3D vectors) rather than directly between raw sensor coordinates. This dimensionality change from point-to-point mapping to normal-to-normal mapping simplifies the mathematical operations required for calibration, reducing computational expense while achieving accurate extrinsic parameter estimation.
2Adaptability or versatility
If sensors are calibrated after mounting on vehicle, then adaptability is improved, but calibration accuracy may be compromised due to operational state variations
Solution Approach 1:
The patent enables the sensor system to perform self-calibration by processing its own sensor data (image frames and point cloud frames) captured during normal operation. The control circuitry extracts normals from the calibration pattern visible to both sensors and computes the transform autonomously, allowing in-situ calibration without requiring external intervention or specialized calibration equipment, thus maintaining both adaptability and accuracy.
Solution Approach 2:
The patent uses feedback from the calibration pattern detection process to iteratively refine the extrinsic parameter estimation. By continuously monitoring the calibration pattern in sensor data and adjusting the transform computation based on the extracted normals, the system achieves accurate calibration even when mounted on the vehicle in operational states, bridging the gap between adaptability and precision.
Data Source
AI summary
A calibration system and method for Light detection and Ranging (LiDAR)-camera calibration is provided. The calibration system receives a plurality of images that includes a calibration pattern and a plurality of point cloud data (PCD) frames that includes the calibration pattern. The calibration system extracts a first normal to a first plane of the calibration pattern in a first PCD frame of the received plurality of PCD frames and further extracts a second normal to a second plane of the calibration pattern in a first image frame of the received plurality of image frames. The calibration system computes a transform between the extracted first normal and the extracted second normal and based on the computed transform, calibrates the LiDAR sensor with the camera.