Video Radar CNN Object Localization
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current Advanced Driver-Assistance Systems (ADAS) face challenges in accurately detecting and localizing objects in a real-world map space using video data alone, lacking efficient integration of multiple information sources for enhanced object detection and collision avoidance.
Innovation Solution
A trainable Convolutional Neural Network (CNN) system that combines image and radar data to detect and localize objects in real-world coordinates, enabling simultaneous object detection, depth estimation, and projection into a 2D map view, thereby overcoming the limitations of prior art by using internal CNN layers as depth layers and applying a MAX function for vertical collapse.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Device complexity
If video data alone is used for object detection in ADAS, then the system complexity is reduced, but the detection accuracy and localization precision in real-world map space deteriorates
Solution Approach 1:
The patent combines video data and radar data into a unified training framework for the CNN. The network is trained on paired video-radar datasets where radar provides accurate depth and position information that complements video's visual features, achieving precise real-world map space localization without requiring complex multi-sensor fusion architecture at runtime
Solution Approach 2:
The patent transforms the detection problem by adding depth estimation as an intermediate dimension. The CNN predicts depth maps from video frames, which are then combined with 2D bounding boxes to generate 3D object positions in real-world map space, effectively bridging the gap between simple video input and precise 3D localization
2Reliability
If multiple sensors are integrated for enhanced object detection, then the detection accuracy improves, but the device complexity and cost increases
Solution Approach 1:
The patent introduces a trained CNN model as an intermediary that processes both video and radar data. During training, the network learns to correlate video features with radar measurements, enabling it to compensate for radar limitations using video information and vice versa, achieving enhanced detection accuracy through software-based fusion rather than complex hardware integration
Solution Approach 2:
The patent creates a virtual representation of the real-world map space by projecting detected objects into a standardized coordinate system. This virtual copy allows for accurate position and velocity estimation without requiring direct integration of multiple sensor coordinate systems, simplifying the overall system architecture
3Ease of manufacture
If traditional object detection methods are used, then the system is easier to implement, but the detection speed and ability to handle multiple scales deteriorates
Solution Approach 1:
The patent extends traditional 2D object detection by incorporating depth estimation as an additional dimension. The CNN outputs both 2D bounding boxes and depth maps, which are combined to create 3D object representations in real-world map space, enabling accurate detection across multiple scales and distances without sacrificing speed
Data Source
AI summary
A computer-implemented method and system are provided. The system includes an image capture device configured to capture image data relative to an ambient environment of a user. The system further includes a processor configured to detect and localize objects, in a real-world map space, from the image data using a trainable object localization Convolutional Neural Network (CNN). The CNN is trained to detect and localize the objects from image and radar pairs that include the image data and radar data for different scenes of a natural environment. The processor is further configured to perform a user-perceptible action responsive to a detection and a localization of an object in an intended path of the user.