Lidar Camera Calibration via Reflectance Map 2D-2D Matching
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing online calibration methods for lidar and camera systems in autonomous driving face challenges in accurately registering 2D images with 3D point clouds due to cross-dimensional differences, leading to reduced calibration accuracy and increased computational complexity.
Innovation Solution
The proposed method utilizes a reflectance map projected from lidar point clouds to transform the 2D-3D registration task into a 2D-2D matching problem, employing techniques like wavelet transform-based image denoising and ResNet neural networks to improve calibration accuracy and reduce computational complexity.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If deep learning-based methods extract high-dimensional features from 2D images and 3D point clouds to find matching point pairs, then the calibration method can handle unknown external parameters, but cross-dimensional differences between features remain and computational complexity increases
Solution Approach 1:
The patent introduces a reflectance map as an intermediary representation that transforms 3D point cloud data into a 2D format preserving depth information through reflectance values. This mediator enables direct 2D-2D feature matching between the reflectance map and input image, eliminating cross-dimensional differences while maintaining the ability to handle unknown external parameters. The reflectance map serves as a bridge that reconciles the dimensional mismatch between 2D images and 3D point clouds.
Solution Approach 2:
The patent creates a 2D reflectance map copy of the 3D point cloud data, where each pixel in the reflectance map corresponds to a 3D point in the point cloud. This copying approach allows the system to work with 2D representations throughout the deep learning pipeline while still accessing 3D geometric information through the reflectance values, thereby reducing computational complexity without sacrificing adaptability to unknown external parameters.
2Measurement precision
If traditional 2D-3D registration methods are used to register images with lidar point clouds, then the calibration can be performed, but cross-dimensional differences lead to reduced calibration accuracy
Solution Approach 1:
The patent applies dimensionality change by converting 3D point cloud data into a 2D reflectance map representation. This transformation allows both the input image and the processed lidar data to exist in the same 2D dimensionality, enabling direct pixel-to-pixel correspondence and feature matching. The reflectance map encodes 3D depth information through its grayscale values while maintaining a 2D spatial structure, thus eliminating cross-dimensional differences and improving calibration accuracy.
Data Source
AI summary
A laser radar-camera online calibration method and system based on a reflectance map includes: data preprocessing, generating data that meets the input of the neural network part; inputting the preprocessed data into the trained neural network; the neural network will output the edge point features of the reference image and the reflectance map, that is, the reference image and the reflectance map descriptor; by comparing the similarity of the reference image and the reflectance map descriptor, find the matching relationship between the reference image feature and the reflectance map feature point; use the mapping relationship between the reflectance map and the 3D point cloud to find the matching relationship between the reference image and the point cloud; use the EPnP algorithm to estimate the transformation matrix from the given 2D-3D point pair to complete the calibration process.