Stereo ADAS Sensor Fusion for Accurate Depth-Based Object Detection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing advanced driver assist systems (ADAS) face challenges in effectively detecting objects while driving, particularly in enhancing the quality of image information such as depth images, and fusing signals from multiple sensors to improve object detection performance.
Innovation Solution
The proposed ADAS captures video sequences with stereo images, determines depth information using reflected signals, fuses the stereo images and depth information to generate fused information, and employs a method of calibrating the system through training feature extractors, feature pyramid networks, and box predictors using synchronized sensing data to enhance object detection accuracy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If multiple sensors are used to capture images and signals, then object detection accuracy is improved, but system complexity increases
Solution Approach 1:
The patent combines multiple sensors (stereo cameras, radar, LiDAR) into a unified sensor system that captures both image data and depth information simultaneously. The processing circuit integrates signals from these different sensor types to generate fused information, achieving improved object detection accuracy while managing system complexity through unified processing architecture.
Solution Approach 2:
The processing circuit is designed to handle multiple functions: capturing video sequences from stereo cameras, processing radar signals for depth information, fusing these different data types, and performing object detection. This multi-functional approach allows a single system to manage diverse sensor inputs without requiring separate dedicated processing paths for each sensor type.
2Measurement precision
If depth information is determined using reflected signals, then three-dimensional object detection is improved, but processing time increases
Solution Approach 1:
The system determines depth information from reflected signals concurrently with capturing video sequences, rather than sequentially. The processing circuit is configured to process radar/LiDAR signals for depth extraction while simultaneously processing camera feeds, performing preliminary depth calculation that feeds into the object detection algorithm without creating processing bottlenecks.
Solution Approach 2:
The system continuously captures video sequences and continuously determines depth information from reflected signals in an ongoing manner. Both processing streams operate continuously and are fused in real-time, maintaining uninterrupted object detection capability without periodic interruptions or batch processing delays.
3Reliability
If stereo images and depth information are fused, then object detection reliability is improved, but computational complexity increases
Solution Approach 1:
The processing circuit acts as an intermediary that receives both stereo image data and depth information from reflected signals, fuses these heterogeneous data types into unified fused information, and then passes this integrated data to the object detection algorithm. This intermediary fusion step reconciles different data formats and coordinate systems, improving detection reliability while centralizing computational complexity in a dedicated processing module.
4Measurement precision
If the system is calibrated using synchronized sensing data, then detection accuracy is improved, but calibration time increases
Solution Approach 1:
The system performs calibration using synchronized sensing data from all sensors before deploying the object detection algorithm. The processing circuit is configured to execute calibration routines that align coordinate systems and timing offsets between stereo cameras, radar, and LiDAR, storing the calibrated parameters for use during actual operation. This preliminary calibration ensures high detection accuracy without requiring time-consuming calibration during real-time detection.
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
This approach enhances the performance of object detection by effectively fusing multi-sensor data and calibrating the ADAS, leading to improved accuracy and reliability in detecting objects while driving.
Implementation Method 1
determine depth information in the stereo image based on reflected signals received while driving the vehicle
Data Source
AI summary
An advanced driver assist system (ADAS) includes a processing circuit and a memory storing instructions executable by the processing circuit. The processing circuit executes the instructions to cause the ADAS to: obtain, from a vehicle, a video sequence including a plurality of frames captured while driving the vehicle, where each of the frames corresponds to a stereo image including a first viewpoint image and a second viewpoint image; determine depth information in the stereo image based on reflected signals received while driving the vehicle; fuse the stereo image and the depth information to generated fused information, and detect at least one object included in the stereo image based on the fused information.