LIDAR-Camera Sensor Fusion for Autonomous Driving Perception
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current autonomous driving systems face challenges in accurately determining distances and velocities of objects using sensor data, particularly in training and testing scenarios, as existing methods lack realistic and comprehensive data integration from multiple sensors.
Innovation Solution
A method and apparatus for generating and applying combined target information by determining three-dimensional bounding indicators from LIDAR sensors and camera sensors, using a camera matrix to project these indicators, and associating them to create ground truth data for training and testing autonomous driving systems, enabling vehicle control signals.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If multiple sensors (LIDAR and camera) are integrated to generate combined target information, then the accuracy and reliability of autonomous driving perception is improved, but the device complexity and data processing difficulty increase
Solution Approach 1:
The patent combines data from LIDAR sensors (three-dimensional bounding indicators) and camera sensors (camera bounding indicators) into unified target information. The system processes road target information from LIDAR and road image information from cameras, generating a combined target information that integrates both sensor types. This merging approach improves perception reliability by utilizing complementary data from multiple sensors while managing the complexity through systematic data fusion procedures.
2Measurement precision
If three-dimensional bounding indicators from LIDAR are projected to two-dimensional camera coordinates using camera matrix, then the association accuracy between LIDAR and camera objects is improved, but the measurement and processing difficulty increases
Solution Approach 1:
The patent transforms three-dimensional bounding indicators from LIDAR data into two-dimensional coordinates that correspond to camera image space. This dimensionality reduction is achieved through projection using camera matrix parameters, which map 3D points onto the 2D image plane. The process enables accurate association between LIDAR-detected objects and camera-detected objects by expressing both in the same coordinate system, thereby improving measurement precision while managing transformation complexity through established projection geometry.
3Manufacturing precision
If ground truth data is generated by associating projected bounding indicators with camera bounding indicators, then the training data quality for autonomous driving systems is improved, but the data processing time and computational resources increase
Solution Approach 1:
The patent performs preliminary association between LIDAR and camera bounding indicators during the data processing stage to generate ground truth information. By pre-computing the associations and creating labeled training data in advance, the system improves data quality for subsequent model training while reducing real-time processing requirements. The ground truth data is generated offline before being used for training autonomous driving perception systems, thereby separating the computationally intensive association task from time-critical inference operations.
Data Source
AI summary
Disclosed are methods and devices related to autonomous driving. In one aspect, a method is disclosed. The method includes determining three-dimensional bounding indicators for one or more first objects in road target information captured by a light detection and ranging (LIDAR) sensor; determining camera bounding indicators for one or more second objects in road image information captured by a camera sensor; processing the road image information to generate a camera matrix; determining projected bounding indicators from the camera matrix and the three-dimensional bounding indicators; determining, from the projected bounding indicators and the camera bounding indicators, associations between the one or more first objects and the one or more second objects to generate combined target information; and applying, by the autonomous driving system, the combined target information to produce a vehicle control signal.