Multi-Sensor Object Detection Fusion for Autonomous Vehicles
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing object detection systems for autonomous vehicles face challenges due to external and internal constraints affecting data quality from various sensing devices like cameras, LiDAR, and RaDAR, leading to potential miss rates and reduced accuracy, as these devices are influenced by light interference, weather, and internal characteristics such as sensing range and data resolution.
Innovation Solution
An object detection system that integrates data from multiple sensing devices, including RGB cameras, LiDAR, and RaDAR, using a processor to apply different object detection algorithms and environmental parameter calculations to determine bounding boxes, with a framework that normalizes environmental data and applies decision fusion techniques to enhance detection accuracy and reduce false positives.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If data from multiple sensing devices are fused, then object detection accuracy is improved, but system complexity increases
Solution Approach 1:
The system segments the object detection task by applying different detection algorithms to different sensor types (e.g., CNN for camera data, point cloud processing for LiDAR). Each sensor modality is processed independently through specialized algorithms, and results are subsequently fused. This segmentation allows the system to leverage the strengths of each sensor type while managing complexity through modular processing.
Solution Approach 2:
The system merges detection results from multiple sensor types and multiple algorithms through a fusion mechanism. Bounding boxes and detection outcomes from camera, LiDAR, and RaDAR are combined using confidence scoring and spatial alignment, achieving improved detection accuracy by综合利用 complementary information from different sensing modalities.
2Reliability
If multiple object detection algorithms are applied to each sensor data, then detection reliability is improved, but computational cost increases
Solution Approach 1:
The system dynamically selects and applies different numbers of detection algorithms based on sensor type and environmental conditions. For example, camera data may undergo multiple stages of detection (proposal generation, classification, refinement), while LiDAR data uses streamlined processing. This dynamic approach adjusts computational effort to match the reliability needs of each sensing modality.
Solution Approach 2:
Different detection algorithms are applied selectively to different sensor data types based on their specific characteristics. Camera data receives CNN-based object detection, LiDAR data receives point cloud processing algorithms, and RaDAR data receives signal processing methods. Each sensor type gets the locally optimal detection approach, improving reliability without uniformly increasing computational cost across all sensors.
3Measurement precision
If environmental parameters are calculated and normalized, then detection accuracy under varying conditions is improved, but processing time increases
Solution Approach 1:
Environmental parameters such as lighting conditions, weather state, and temperature are calculated and normalized in advance before object detection is performed. These pre-processed environmental contexts are then used to adjust detection thresholds and algorithm parameters, improving accuracy without adding significant processing time during the critical detection phase.
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
The system improves object detection accuracy by integrating data from multiple sensors, accounting for environmental and internal constraints, resulting in enhanced reliability and reduced miss rates, with efficient computational costs and improved performance compared to single-sensor systems.
Implementation Method 1
LiDAR would use modulated infrared (IR) wave to measure the time of flight between the transmitter and the receiver
Implementation Method 2
a RaDAR would use a radio frequency wave to measure the time of flight between the transmitter and the receiver
Data Source
AI summary
In one of the exemplary embodiments, the disclosure is directed to an object detection system including a first type of sensor for generating a first sensor data; a second type of sensor for generating a second sensor data; and a processor coupled to the first type of sensor and the second type of sensor and configured at least for: processing the first sensor data by using a first plurality of object detection algorithms and processing the second sensor data by using a second plurality of object detection algorithms, wherein each of the first plurality of object detection algorithms and each of the second plurality of object detection algorithms include environmental parameters calculated from a plurality of parameter detection algorithms; and determining for each detected object a bounding box resulted from processing the first sensor data and processing the second sensor data.