Radar Image Fusion Object Detection Neural Network
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Autonomous vehicles face challenges in accurately detecting and determining various parameters of objects in their surroundings using individual detector systems like cameras and radar, as each system provides limited information, necessitating a fusion of data to obtain comprehensive object details.
Innovation Solution
A method and system that combine image data and radar data using neural networks to generate a fused dataset, allowing for the detection of object parameters such as classification scores, dimensions, observation angles, depth, and velocity vectors, through the use of boundary boxes and multiple heads within the neural network architecture.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of information
If individual detector systems (camera or radar) are used for object detection, then the system complexity is low, but the information completeness and measurement precision are insufficient
Solution Approach 1:
The patent combines camera data and radar data into a unified detection framework. The camera provides high-resolution image data for visual recognition, while radar provides depth and velocity information. These multiple data sources are merged through data association algorithms to create comprehensive object detections with complete parameter sets including position, velocity, and classification.
Solution Approach 2:
The detection system is designed to handle multiple types of objects (pedestrians, vehicles, cyclists, animals) and multiple parameters (position, velocity, depth, classification) using a single integrated framework. The system can detect both static and moving objects, and provides multiple attributes for each detected object, making it universally applicable to various detection needs.
2Measurement precision
If data from multiple detector systems is combined, then the measurement precision and information completeness improve, but the device complexity increases
Solution Approach 1:
The detection process is divided into distinct segments: camera data processing, radar data processing, data association, and parameter estimation. Each detector system processes its own data type independently, and the results are then associated through a unified model. This segmentation allows each component to be optimized independently while maintaining overall system precision.
Solution Approach 2:
The patent introduces a data association module that acts as an intermediary between camera and radar detection results. This intermediary component matches detections from both sensors based on spatial and temporal consistency, resolving ambiguities and combining measurements to achieve high measurement precision without requiring direct complex interaction between the detector systems.
3Measurement precision
If comprehensive object parameters are determined through data fusion, then the navigation accuracy improves, but the processing time and computational complexity increase
Solution Approach 1:
The system performs preliminary processing of camera and radar data independently before fusion, pre-computing features and detection results. The camera extracts visual features and potential object regions in advance, while radar pre-processes point cloud data to identify moving objects. This preliminary action reduces the computational burden during the fusion stage, maintaining accuracy while reducing processing time.
Solution Approach 2:
The patent transforms data from different sensor coordinate systems and formats into a unified parameter representation. Radar data is transformed into the camera coordinate system, and both data types are converted to consistent parameter formats (position, velocity, size) before fusion. This parameter transformation enables efficient comparison and combination of measurements without repeated coordinate transformations during processing.
Data Source
AI summary
A vehicle, system and method of detecting an object. The system includes an image network, a radar network and a head. The image network receives image data and proposes a boundary box from the image data and an object proposal. The radar network receives radar data and the boundary box and generates a fused set of data including the radar data and the image data. The head determines a parameter of the object from the object proposal and the fused set of data.