Camera-Radar Sensor Fusion With Cross-Training for ADAS Perception
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current autonomous vehicle systems face challenges in achieving accurate environmental sensing due to the limitations of individual sensors, such as cameras being sensitive to lighting and weather, and radars lacking classification capabilities, despite their complementary strengths. Additionally, existing sensor fusion methods do not facilitate interactions among sensors to enhance individual performance.
Innovation Solution
A sensor fusion system that enables data exchange and cross-training between cameras and radars, using camera data as 'ground truth' for radar classification and radar measurements for camera calibration, allowing for continuous improvement and reduced human involvement, enabling accurate object classification and distance/speed estimation.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If camera data is used for object detection and classification, then classification accuracy is improved, but reliability under varying lighting and weather conditions deteriorates
Solution Approach 1:
The patent combines camera and radar sensors into a unified sensor fusion system where camera data provides object classification information and radar data provides distance and velocity measurements. The fusion module integrates these complementary data sources to produce outputs that maintain classification accuracy while improving reliability across varying environmental conditions, as radar is less sensitive to lighting and weather than cameras.
2Measurement precision
If radar is used for distance and speed measurement, then measurement accuracy is improved, but classification capability deteriorates
Solution Approach 1:
The patent merges radar's accurate distance and speed measurements with camera's object classification capabilities through a fusion module. The radar provides precise range and velocity data while the camera supplies object identification information, creating a comprehensive output that includes both measurement accuracy and classification capability that neither sensor could achieve alone.
3Device complexity
If sensors operate independently without interaction, then device complexity is reduced, but sensor performance and fusion results deteriorate
Solution Approach 1:
The patent implements a feedback mechanism where the fusion module's output is used to continuously calibrate and improve individual sensor performances. The system analyzes fused results and feeds this information back to adjust sensor parameters and processing algorithms, creating a self-improving system that enhances measurement precision while maintaining manageable complexity through automated calibration.
Data Source
AI summary
A sensor fusion system for an autonomous vehicle and/or advanced driver assistance systems (ADAS) that combines measurements from multiple independent sensors to generate a better output. The multiple sensors can interact and perform “cross-training” for enhanced sensor performance. The sensor fusion system utilizes data exchange and cross-training between sensors for enhanced sensor performance while constantly improving sensor fusion results. Some exemplary embodiments of the present disclosure provide a system for an autonomous vehicle and/or a vehicle ADAS that combines information from independent sensors in order to accurately distinguish environmental objects and movements. This allows the vehicle to accurately make informed navigation decisions based on the environmental surroundings. The sensor fusion system can exchange ground truth information between the sensors related to a same object, which allows for constant machine learning object classification and constant speed/distance estimation calibration.