Full Velocity Determination Radar Camera Fusion
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Radars are unable to directly measure tangential velocities of points in the environment, limiting their ability to provide full velocities, which are essential for autonomous vehicle operations.
Innovation Solution
A system that uses data from both radars and cameras to calculate full velocities of points detected by the radar, employing a closed-form calculation and a trained neural network to accurately associate radar and camera data, allowing for the determination of full velocities and enabling autonomous or semi-autonomous vehicle operation.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If radar data alone is used for velocity measurement, then radial velocity can be obtained, but full velocity (including tangential component) cannot be determined
Solution Approach 1:
The patent combines radar data (providing radial velocity and depth) with camera data (providing optical flow information) to compute full velocity. The camera captures sequential images to determine optical flow, which contains tangential motion information that radar alone cannot provide. By merging these complementary data sources, the system recovers complete velocity information including both radial and tangential components.
Solution Approach 2:
The patent introduces optical flow computation as an intermediary process that bridges radar and camera data. The optical flow field, derived from sequential camera images, serves as a mediator that provides tangential velocity information. This intermediary allows the system to translate camera observations into velocity components that can be integrated with radar measurements to produce full velocity vectors.
2Measurement precision
If multiple sensors (radar and camera) are integrated to determine full velocity, then measurement completeness improves, but device complexity increases
Solution Approach 1:
The patent segments the velocity determination process into distinct computational stages: (1) radar data processing to obtain radial velocity and depth, (2) camera data processing to compute optical flow, (3) data association between radar points and camera regions, and (4) full velocity computation by combining results. This segmentation allows each subsystem to be processed independently and then integrated, reducing overall system complexity.
Solution Approach 2:
The patent transitions from one-dimensional radial velocity measurement (radar alone) to two-dimensional full velocity vector determination by incorporating the temporal dimension through sequential camera images. The optical flow computation exploits the time dimension to extract tangential motion, adding a new dimension of information that enables complete velocity reconstruction without proportionally increasing hardware complexity.
3Measurement precision
If data association between radar and camera is performed accurately, then full velocity accuracy improves, but computational effort increases
Solution Approach 1:
The patent applies local quality by performing data association only in localized regions around each radar point rather than globally across the entire field of view. For each radar-detected point, the system searches for corresponding camera pixels within a restricted spatial window, reducing the computational search space while maintaining association accuracy. This localized approach preserves precision where needed while minimizing unnecessary computations.
Solution Approach 2:
The patent employs partial action by computing optical flow and performing data association only for regions where velocity information is needed, rather than processing the entire camera image frame. The system selectively processes camera data corresponding to radar-detected objects and their surrounding regions, avoiding unnecessary computational effort in areas without relevant targets while maintaining full velocity accuracy for detected objects.
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
The system provides reliable and accurate full velocities with low computational effort, enabling vehicles to operate autonomously or semi-autonomously by accurately associating radar and camera data, thereby overcoming the limitations of radars in measuring tangential velocities.
Implementation Method 1
Radars can detect radial velocities of points in the environment by taking advantage of the Doppler effect
Implementation Method 2
Radars transmit radio waves and receive reflections of those radio waves to detect physical objects in the environment
Implementation Method 3
Cameras can detect electromagnetic radiation in some range of wavelengths. For example, cameras may detect visible light, infrared radiation, ultraviolet light
Data Source
AI summary
A computer includes a processor and a memory storing instructions executable by the processor to receive radar data including a radar pixel having a radial velocity from a radar; receive camera data including an image frame including camera pixels from a camera; map the radar pixel to the image frame; generate a region of the image frame surrounding the radar pixel; determine association scores for the respective camera pixels in the region; select a first camera pixel of the camera pixels from the region, the first camera pixel having a greatest association score of the association scores; and calculate a full velocity of the radar pixel using the radial velocity of the radar pixel and a first optical flow at the first camera pixel. The association scores indicate a likelihood that the respective camera pixels correspond to a same point in an environment as the radar pixel.