ADAS Stereo-Depth Fusion for Accurate Object Detection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing advanced driver assist systems (ADAS) face challenges in accurately detecting objects and calibrating depth information, which affects their performance in enhancing driving safety and assistance.
Innovation Solution
The ADAS integrates a processing circuit that captures stereo images and depth information from multiple sensors, fuses this information to generate fused data, and uses a feature pyramid network and box predictor to detect objects, with a calibration method involving training and fine-tuning to enhance object detection accuracy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If depth information is determined based on reflected signals from multiple sensors, then measurement precision of object distance is improved, but device complexity increases due to integration of multiple sensors and calibration requirements
Solution Approach 1:
The patent combines multiple sensors (cameras, LIDAR, radar) into an integrated sensor system that captures data from different modalities. The processing circuit fuses depth information from reflected signals with stereo image data, merging multiple measurement sources to achieve higher precision while managing complexity through unified processing architecture.
Solution Approach 2:
The patent introduces calibration data as an intermediary element that mediates between raw sensor measurements and final depth information. The calibration process establishes transformation relationships between different sensor coordinate systems, enabling accurate fusion of depth data from multiple sensors without requiring direct complex inter-sensor coordination.
2Measurement precision
If stereo images and depth information are fused to generate fused information, then object detection accuracy is improved, but processing time and computational complexity increase
Solution Approach 1:
The patent performs calibration of sensors and preprocessing of depth information before the actual object detection process. Stereo images and depth data are pre-aligned and registered in advance, so that during runtime, the fusion operation requires minimal computation. This preliminary preparation reduces real-time processing time while maintaining high detection accuracy.
Solution Approach 2:
The processing circuit segments the fusion process into distinct stages: depth information extraction from reflected signals, stereo image processing, calibration application, and final fusion. By dividing the complex fusion operation into manageable segments, the system optimizes computational resources at each stage and reduces overall processing time.
3Measurement precision
If calibration is performed using training data sets and fine-tuning, then object detection precision is improved, but loss of time during system setup and maintenance increases
Solution Approach 1:
The patent performs comprehensive calibration using training data sets during system initialization and setup phase. The feature extractor, feature pyramid network, and box predictor are trained and fine-tuned in advance using labeled calibration data. This preliminary calibration establishes optimized detection parameters that reduce the need for frequent recalibration, minimizing time loss during normal operation.
Solution Approach 2:
The system implements feedback mechanisms where detection results are continuously evaluated against ground truth data during calibration. The feature extractor and detection networks are fine-tuned based on feedback from calibration performance metrics. This feedback-driven calibration process optimizes detection precision while establishing efficient calibration protocols that reduce future calibration time requirements.
Data Source
AI summary
An advanced driver assist system (ADAS) includes a processing circuit and a memory storing instructions executable by the processing circuit. The processing circuit executes the instructions to cause the ADAS to: obtain, from a vehicle, a video sequence including a plurality of frames captured while driving the vehicle, where each of the frames corresponds to a stereo image including a first viewpoint image and a second viewpoint image; determine depth information in the stereo image based on reflected signals received while driving the vehicle; fuse the stereo image and the depth information to generated fused information, and detect at least one object included in the stereo image based on the fused information.