Camera-LiDAR Orientation Calibration Using 3D Box Projection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for calibrating the orientation of a camera relative to a LIDAR sensor in autonomous vehicles are inadequate, particularly when linear features are lacking, leading to offset errors in projected bounding boxes and increased annotation effort.
Innovation Solution
A method that groups sensor frames based on temporal correlation, projects 3D bounding boxes onto the image plane of an area sensor, calculates an energy term based on coordinate differences, and optimizes the relative sensor orientation to minimize this energy, using neural networks for high-quality annotations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If target-based calibration methods using linear features are used, then calibration accuracy is improved, but the method becomes inapplicable when linear features are lacking in the scene
Solution Approach 1:
The invention extracts calibration information from object bounding boxes instead of relying on linear features. By using the detected objects and their annotated bounding boxes as calibration targets, the method removes the dependency on specific scene features like lane lines, making calibration applicable to diverse scenes without requiring particular linear structures.
Solution Approach 2:
The invention introduces an energy minimization function as an intermediary that bridges 3D bounding box projections and 2D image plane annotations. This energy function quantifies the misalignment between projected 3D boxes and ground truth 2D boxes, enabling automated calibration optimization without manual intervention or dependency on specific scene features.
2Manufacturing precision
If manual calibration correction is performed, then annotation quality is maintained, but the annotation process becomes time-consuming
Solution Approach 1:
The calibration system performs self-calibration by automatically optimizing sensor orientation parameters through energy minimization. The system uses its own annotated data to iteratively improve calibration accuracy without requiring external manual intervention, thereby maintaining high annotation quality while eliminating time-consuming manual correction steps.
Solution Approach 2:
The invention implements a feedback loop where the energy function evaluates the alignment between projected 3D bounding boxes and ground truth 2D boxes, and this evaluation feeds back into optimizing the sensor calibration parameters. This automated feedback mechanism continuously improves calibration accuracy without manual quality checks, reducing annotation time while maintaining precision.
3Extent of automation
If 3D bounding boxes are projected to 2D image plane, then links between 2D and 3D annotations are created, but orientation errors cause offsets between projected and ground truth boxes
Solution Approach 1:
The invention performs preliminary calibration optimization by minimizing the energy function before projecting 3D bounding boxes to the 2D image plane. By pre-optimizing sensor orientation parameters using the energy minimization approach, the system ensures that subsequent projections achieve high alignment accuracy with ground truth annotations, eliminating the need for post-projection correction.
Solution Approach 2:
The invention optimizes sensor calibration parameters (rotation and translation) by changing them to minimize the energy function. This parameter optimization adjusts the transformation between 3D and 2D coordinate systems, ensuring that projected bounding boxes align precisely with ground truth annotations while maintaining automatic linking capabilities.
Data Source
Figure 1~2
Figure 3~4
Figure 5
AI summary
A computer-implemented method for calibrating the relative orientation of a spatial sensor and an area sensor with overlapping measuring zones comprises receiving a plurality of sensor frames of the spatial sensor and the area sensor, grouping sensor frames of the space sensor with sensor frames of the area sensor based on temporal correlation, selecting at least one group of frames comprising a spatial sensor frame and an area sensor frame, receiving annotations for the at least one selected group, wherein the annotations comprise bounding boxes for detected objects, projecting the corners of a three-dimensional bounding box in the spatial sensor frame to an image plane of the area sensor, to produce a projected rectangle, calculating an energy term based on the coordinates of the corners of the projected rectangle and the corresponding bounding box in the area sensor frame, and optimizing the relative sensor orientation by minimizing the energy term.