4D Radar-Camera Extrinsic Calibration via Adaptive Projection Error
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current 4D millimeter-wave radar systems are insensitive to non-metallic objects, have low resolution, and suffer from multi-path interference, making it difficult for them to independently perform target detection and precise positioning like cameras and lidars.
Innovation Solution
Implementing extrinsic parameter calibration for 4D millimeter-wave radar and camera systems using adaptive projection error, which involves processing pixel data from cameras and 4D point cloud data from radar sensors to apply pose calibration and obtain configuration parameters for accurate fusion.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If 4D millimeter-wave radar is used for target detection and positioning, then speed and 3D contour information are improved, but sensitivity to non-metallic objects and measurement precision deteriorate
Solution Approach 1:
The patent combines 4D millimeter-wave radar and camera into a fused sensor system. The radar provides speed and 3D contour information while the camera compensates for low sensitivity to non-metallic objects and low resolution, achieving complementary advantages through data fusion.
Solution Approach 2:
The calibration system integrates multiple functions: it calibrates extrinsic parameters between radar and camera, performs pose calibration on video frames, and enables the fused system to handle both metallic and non-metallic object detection across different distance ranges.
2Reliability
If fusion of 4D millimeter-wave radar and camera is implemented, then detection capability is improved, but device complexity increases
Solution Approach 1:
The patent performs extrinsic parameter calibration beforehand to establish the spatial relationship between radar and camera. This preliminary calibration includes obtaining configuration parameters and applying pose calibration to video frames, which simplifies subsequent data fusion operations.
Solution Approach 2:
The calibration system acts as an intermediary that establishes the coordinate transformation relationship between radar and camera. By pre-computing extrinsic parameters and configuration parameters, it mediates the integration of data from both sensors, reducing the complexity of real-time fusion.
3Measurement precision
If extrinsic parameter calibration is performed using adaptive projection error, then calibration accuracy is improved, but processing complexity increases
Solution Approach 1:
The patent employs adaptive projection error as a feedback mechanism to iteratively optimize extrinsic parameter calibration. The system projects radar points to camera image space, calculates projection errors, and adjusts calibration parameters to minimize these errors, achieving high accuracy through iterative refinement.
Solution Approach 2:
The calibration process transforms 3D radar point cloud data into 2D camera image space for comparison and error calculation. This dimensional transformation enables the use of 2D image processing techniques to calibrate 3D spatial parameters, simplifying the overall calibration approach.
Data Source
AI summary
An apparatus comprises an interface and a processor. The interface may be configured to receive pixel data representing a first field of view of a camera and four-dimensional (4D) point cloud data representing a second field of view of a 4D imaging radar sensor. The first field of view of the camera and the second field of view of the 4D imaging radar sensor generally overlap. The processor may be configured to process the pixel data and the 4D point cloud data arranged as video frames and perform an extrinsic parameter calibration for the 4D imaging radar sensor and the camera based on adaptive projection error. The extrinsic parameter calibration may comprise applying a pose calibration process to the video frames. The pose calibration process may use (i) a respective translate vector and a respective rotation matrix for each of the 4D imaging radar sensor and the camera and (ii) a radar corner reflector to obtain configuration parameters for the first field of view of the camera and the second field of view of the 4D imaging radar sensor. The pose calibration process generally comprises capturing data for a plurality of positions of the radar corner reflector between a near-distance value and a long-distance value, and modifying a scale of a projection error of the radar corner reflector based on a size of the radar corner reflector in an image captured by the camera.