Camera-Radar Fusion Using Tracked Patch Correspondences
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing sensor fusion techniques for automotive applications, such as in ADAS, face challenges in effectively combining the strengths of camera and radar data due to domain differences and inherent ambiguities, particularly in measuring distance and angular position.
Innovation Solution
The method employs affine patch tracking to determine tracked patches in camera data and fuses it with supplemental radar data using equivalent scale ratios, eliminating the need for domain conversion and resolving ambiguities by associating radar information with corresponding patches.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If camera and radar data are fused using traditional domain conversion methods, then sensor fusion is achieved, but domain differences and inherent ambiguities cause measurement inaccuracies
Solution Approach 1:
The patent introduces tracked patches from optical flow as an intermediary element that bridges camera and radar data domains. These patches serve as correspondence markers that enable direct association between radar measurements and camera image regions without requiring complex domain conversions, thereby resolving the technical contradiction by maintaining measurement precision while reducing conversion complexity
Solution Approach 2:
The patent replaces traditional mechanical/domain conversion systems with a direct correspondence-based fusion approach. Instead of converting radar data to camera domain or vice versa through complex transformations, the system uses tracked patches to directly associate measurements across domains, substituting the conversion mechanism with a more efficient correspondence-matching mechanism
2Reliability
If complex domain conversion is performed to fuse camera and radar data, then sensor fusion is achieved, but computational overhead increases
Solution Approach 1:
The patent performs preliminary optical flow tracking to generate tracked patches before the actual sensor fusion operation. By pre-computing these correspondence markers and organizing them in advance, the system eliminates the need for complex runtime conversions during fusion, thereby maintaining high reliability while improving real-time processing speed and reducing computational overhead
3Loss of information
If traditional sensor fusion methods are used, then camera and radar data are combined, but ambiguities in measuring distance and angular position persist
Solution Approach 1:
The patent uses tracked patches as intermediary correspondence markers that explicitly link radar measurements to specific regions in the camera image. This intermediary mechanism resolves ambiguities by providing clear spatial relationships between the two sensor domains, enabling precise determination of distance and angular position without information loss
Data Source
AI summary
A method of performing sensor fusion of data obtained from a camera (51) and a supplemental sensor (52), the method comprising performing patch tracking (55) on image data (53) provided by the camera (51) to determine tracked patches (56), and performing a fusion (57) of the image data (53) obtained from the camera (51) with supplemental data (54) provided by the supplemental sensor (52) based on the tracked patches (56).