Camera-Radar Fusion Stage Selection for Accurate Vehicle Perception
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current sensor data fusion techniques in vehicles, particularly for automated or assisted driving systems, face challenges in achieving high accuracy due to the limitations of camera and radar sensors under varying lighting conditions, necessitating an improved method for selecting optimal processing stages within neural networks to enhance the quality of fused data.
Innovation Solution
The proposed method involves generating and comparing fused sensor data at different processing states within a neural network, specifically using a feature pyramid network, to select the most accurate processing state for fusing data from camera and radar sensors, allowing for a trade-off between accuracy and model parameters to achieve higher quality fused data.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If sensor data fusion is performed at multiple processing stages of a neural network, then the accuracy of fused data can be improved, but the computational complexity and processing time increase
Solution Approach 1:
The system performs preliminary actions by generating fused sensor data at multiple processing stages in advance, evaluating their accuracies beforehand, and selecting the optimal processing state before final use. This allows the system to prepare multiple candidates and choose the best one, improving accuracy while managing computational resources efficiently.
Solution Approach 2:
The system dynamically adapts the sensor data fusion process by selecting different processing states based on evaluated accuracies. Instead of using a fixed processing stage, the system can switch between different processing states (e.g., before pre-encoding, before encoding, after encoding, or after decoding) depending on which stage provides the optimal accuracy for current conditions.
2Measurement precision
If sensor data fusion is performed at later processing stages with more model parameters, then the accuracy of fused data improves, but the number of model parameters and computational resources required increases
Solution Approach 1:
The system changes parameters by evaluating the accuracy of fused data at different processing stages and selecting the optimal stage based on this evaluation. This allows the system to adapt the number of model parameters and computational resources used, choosing fewer parameters when sufficient accuracy is achieved at earlier stages and more parameters only when necessary for higher accuracy requirements.
3Reliability
If camera and radar sensors are used together, then the complementarity and overall performance improve, but the complexity of data fusion increases
Solution Approach 1:
The system segments the data fusion process by handling camera and radar sensor data through separate processing pipelines that independently process each sensor type through their respective encoding and decoding stages. This segmentation allows for specialized processing of each sensor type while maintaining the ability to fuse their outputs at multiple stages, reducing the complexity of direct integration.
Solution Approach 2:
The system uses an intermediary approach by introducing multiple processing stages (encoding and decoding stages) as intermediate steps between raw sensor data and final fused output. These intermediary stages provide standardized interfaces and transformation layers that facilitate the fusion of heterogeneous camera and radar data, managing complexity through structured intermediate representations.
Data Source
Figure 1
Figure 2a
Figure 2b
AI summary
Embodiments of the present disclosure provide a vehicle, an apparatus, a computer program, and a method (100) for fusing data. The method (100) comprises generating (110) first fused sensor data by fusing the sensor data of the first sensor and the second sensor. The sensor data for the first fused sensor data is in a first processing state. Further, the method (100) comprises generating (120) second fused sensor data by fusing the sensor data of the first sensor and the second sensor. The sensor data for the second fused sensor data is in a second processing state. Also, the method (100) provides for determining (130) an accuracy of the first and the second fused sensor data, and selecting (140), based on the accuracy, one of the first and the second processing state for fusing sensor data of the first and the second sensor.