Radar-Vision Feature Fusion for Accurate Multi-Sensor Matching
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for fusing perception results from different sensors in self-driving vehicles rely heavily on human-set thresholds and rules, leading to potential mismatches and inaccuracies in sensor data fusion, which complicates maintenance and development.
Innovation Solution
A method and apparatus that utilize pre-trained models to determine fusion features based on vision and radar perception data, ensuring accurate matching and fusion of sensor data by employing a target vision feature fusion model and a pre-constructed radar vision feature fusion model, along with tracking and prediction models to enhance accuracy and stability.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of manufacture
If human-set thresholds and rules are used for matching and fusing vision and radar perception targets, then the fusion process can be implemented with simple logic, but the matching accuracy and fusion precision deteriorate due to parameter design limitations
Solution Approach 1:
The patent replaces the mechanical rule-based matching system with a deep learning model (Radar Vision Feature Fusion Model) that automatically learns optimal matching strategies from data. The model substitutes manual threshold-setting and fusion rules with neural network-based feature fusion, enabling the system to adaptively determine matching relationships between radar and vision targets without rigid predefined conditions.
Solution Approach 2:
The patent transforms fixed human-set thresholds into dynamic parameters learned by the deep learning model. Instead of using static matching thresholds determined by engineers, the system employs trainable parameters within the neural network that automatically adjust to optimize matching accuracy based on the characteristics of radar and vision data, thereby resolving the contradiction between implementation simplicity and matching precision.
2Device complexity
If human-set thresholds and rules are used for fusing perception results, then the system structure remains simple, but the fusion result accuracy deteriorates due to inappropriate parameter design
Solution Approach 1:
The patent replaces the mechanical rule-based fusion system with an intelligent deep learning-based fusion model. The Radar Vision Feature Fusion Model substitutes manual fusion rules with automated neural network processing, allowing the system to achieve high fusion accuracy without requiring complex manual parameter tuning or multiple fusion rules.
Solution Approach 2:
The patent enables the fusion system to self-optimize through the deep learning model that automatically learns the optimal fusion strategy from data. The model performs self-adjustment of fusion parameters and weights without external intervention, allowing the system to maintain simplicity while achieving high accuracy through automated learning rather than manual configuration.
3Area of stationary object
If multiple sensors are deployed to increase perception scope, then the perception coverage is improved, but the complexity of fusing multiple perception results deteriorates
Solution Approach 1:
The patent creates a universal deep learning fusion model that can handle multiple sensor types (vision and radar) and multiple targets simultaneously. The Radar Vision Feature Fusion Model serves as a multi-functional system that processes diverse sensor inputs through a unified framework, eliminating the need for separate fusion logic for each sensor combination and reducing overall system complexity.
Solution Approach 2:
The patent transforms the complex multi-sensor fusion problem into a parameter optimization problem solvable by deep learning. By representing sensor data as feature vectors and using learnable parameters in the neural network, the system can handle multiple sensors through a standardized parameter-based approach rather than requiring complex rule-based logic for each sensor combination.
4Ease of operation
If manual parameter design and selection are performed for fusion rules, then the fusion logic remains interpretable, but the maintenance and development difficulty increases due to numerous parameter design requirements
Solution Approach 1:
The patent replaces the manual parameter design and maintenance system with an automated deep learning model. The Radar Vision Feature Fusion Model substitutes human engineers' parameter tuning work with automated learning from data, eliminating the need for manual parameter design, selection, and adjustment while improving maintainability through a single trainable model rather than multiple hand-crafted parameters.
Data Source
AI summary
An embodiment of the present disclosure provides a method and an apparatus for fusing multi-sensor data. The method includes: obtaining a current vision perception result and a current first radar perception result; based on a current vision perception feature corresponding to each current vision perception target in each current vision perception result and a target vision feature fusion model, determining a fusion vision feature corresponding to each current vision perception target; based on the fusion vision feature corresponding to each current vision perception target, a current first radar perception feature corresponding to each current first radar perception target in the current first radar perception result and a pre-constructed radar vision feature fusion model, determining mutually-matched current vision perception targets and current first radar perception targets and corresponding current fusion perception features. In this way, accurate fusion on multi-sensor data can be achieved.