Multi-Sensor Fusion for Autonomous Driving in Adverse Weather
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing sensor fusion methods for autonomous driving systems face challenges in integrating camera, lidar, and radar data due to performance degradation in adverse weather conditions and high computational complexity, with limited angular resolution from traditional radar sensors.
Innovation Solution
An early fusion method and architecture that combines camera, high-definition imaging radar, and lidar data using neural networks for object detection and free-space segmentation, reducing computational complexity and improving performance by fusing raw sensor data before feature extraction.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Device complexity
If traditional radar sensors are used for autonomous driving systems, then device complexity is reduced, but angular resolution deteriorates
Solution Approach 1:
The patent combines traditional radar sensors with HD imaging radar sensors in a unified sensor fusion framework. This merging allows the system to maintain the simplicity and reliability of traditional radar while incorporating the high angular resolution capabilities of HD imaging radar, thereby resolving the contradiction between device complexity and measurement precision.
Solution Approach 2:
The sensor fusion architecture processes data from multiple sensor types (camera, lidar, traditional radar, HD imaging radar) through a unified neural network framework. This multi-functional approach allows a single system to handle both the simplicity requirements of traditional radar applications and the high-resolution requirements of HD imaging radar applications.
2Measurement precision
If camera and lidar sensors are used for sensor fusion, then measurement precision is improved, but computational complexity increases
Solution Approach 1:
The patent applies preliminary processing steps to sensor data before feeding it into the neural network, including data formatting, normalization, and feature extraction. This preliminary action reduces the computational burden on the neural network while preserving the high measurement precision benefits of camera and lidar sensors.
Solution Approach 2:
The sensor fusion system selectively processes sensor data based on operational conditions and requirements. Not all sensor data is processed at full detail in all situations - the system applies partial processing where sufficient and excessive processing where needed, optimizing the balance between measurement precision and computational complexity.
3Measurement precision
If high-resolution sensors are used in adverse weather conditions, then measurement precision is improved, but reliability deteriorates
Solution Approach 1:
The neural network-based sensor fusion system continuously evaluates the quality and reliability of input data from different sensors and adjusts processing accordingly. This feedback mechanism allows the system to maintain measurement precision in adverse weather by compensating for sensor limitations through multi-sensor data fusion and adaptive processing.
Solution Approach 2:
The system prepares for adverse weather conditions by having multiple sensor types ready to compensate for each other's limitations. The sensor fusion architecture is designed in advance to handle various weather scenarios, cushioning against reliability deterioration by having redundant sensing capabilities and adaptive processing routines ready before adverse conditions occur.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
Embodiments of the present disclosure relate to a vehicle, an apparatus, a computer-readable data carrier, a computer program, data, and a method for processing sensor data. The method comprises obtaining first sensor data of a first sensor of a first sensor type, obtaining second sensor data of a second sensor of a second sensor type different from the first sensor type, and fusing the first and the second sensor data. As well, the method comprises processing the fused sensor data using one or more neural networks for a computer vision task.