Camera-LiDAR Fused Object Detection for Autonomous Vehicles
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Real-time prediction of actions by drivers and pedestrians is challenging for semi-autonomous or autonomous vehicles, especially when they break traffic rules, leading to uncertain collision risks due to overly cautious or optimistic predictions.
Innovation Solution
Implementing a camera-LiDAR fused object detection system that matches LiDAR data points with image pixels, using probability distributions and confidence values to accurately detect objects and predict their trajectories, thereby determining collision risks and adjusting vehicle trajectories accordingly.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If LiDAR and camera data are fused for object detection, then object detection accuracy is improved, but system complexity increases
Solution Approach 1:
The patent combines LiDAR point cloud data with camera image data into a unified detection framework. LiDAR provides precise depth and spatial information while camera data provides texture and color information, merging their strengths to achieve more accurate object detection than either sensor alone
Solution Approach 2:
The patent introduces calibration parameters and coordinate transformation mechanisms as intermediaries to bridge the LiDAR and camera data streams. These intermediaries enable accurate spatial alignment and matching between the two different sensor modalities without requiring direct complex interaction between them
2Measurement precision
If probability distribution matching is used to match LiDAR points to image pixels, then matching accuracy is improved, but computational time increases
Solution Approach 1:
The patent transforms the matching problem from direct coordinate comparison to probability distribution comparison. By changing the parameter space from deterministic coordinates to probabilistic distributions, the system achieves more robust matching that accounts for uncertainties in both LiDAR and camera measurements
Solution Approach 2:
The patent computes probability distributions over relevant regions rather than exhaustively processing all possible pixel-point combinations. This partial action approach focuses computational resources on the most likely matches while maintaining accuracy, avoiding the excessive computation that would result from complete enumeration
3Measurement precision
If multiple calibration parameters are used for LiDAR-camera alignment, then alignment precision is improved, but system complexity increases
Solution Approach 1:
The patent divides the calibration parameters into distinct categories: intrinsic camera parameters (focal length, principal point), extrinsic parameters (rotation, translation), and LiDAR parameters. This segmentation allows each parameter type to be calibrated and adjusted independently, managing the overall calibration complexity
Data Source
AI summary
Systems and methods for object detection. The object detection may be used to control an autonomous vehicle. For example, the methods comprise: obtaining, by a computing device, a LiDAR dataset generated by a LiDAR system of the autonomous vehicle; and using, by the computing device, the LiDAR dataset and at least one image to detect an object that is in proximity to the autonomous vehicle. The object is detected by: matching points of the LiDAR dataset to pixels in the at least one image; and detecting the object in a point cloud defined by the LiDAR dataset based on the matching. The object detection may be used to facilitate at least one autonomous driving operation (e.g., autonomous driving operation comprises an object tracking operation, an object trajectory prediction operation, a vehicle trajectory determination operation, and/or a collision avoidance operation).