Radar-Vision Bounding Boxes for 3D Object and Velocity Estimation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Autonomous vehicles face challenges in accurately drawing 3D bounding boxes for objects in real-time driving environments due to insufficient semantic and local information from neural networks, incomplete object capture by individual cameras, and difficulties in capturing depth and velocity using cameras, making it hard to fuse radar and lidar data with vision data effectively.
Innovation Solution
The system enriches object queries by cross-correlating radar-based and vision-based images, fusing their features to generate improved velocity and three-dimensional object estimation, enhancing the detection process and reducing processing demands.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If neural networks are used to generate bounding boxes from images, then object detection capability is provided, but accuracy is insufficient due to lack of semantic and local information
Solution Approach 1:
The patent merges radar data with vision data by cross-correlating radar-based images with vision-based images to enrich object queries. This combination allows the system to leverage both the semantic information from vision and the precise measurement capabilities from radar, resolving the contradiction between detection capability and accuracy.
Solution Approach 2:
The patent introduces cross-correlation as an intermediary mechanism that processes both radar and vision data. This intermediary process generates enriched object queries that incorporate information from both modalities, enabling accurate bounding box generation while preserving semantic and local information from both sources.
2Quantity of substance
If individual cameras are used to capture objects, then vision data is obtained, but object capture is incomplete and depth/velocity cannot be captured
Solution Approach 1:
The patent combines multiple sensor modalities (radar and vision) to overcome the limitations of individual cameras. Radar provides depth and velocity information that cameras cannot capture, while vision provides semantic information. The cross-correlation process integrates these complementary strengths to achieve complete object capture.
3Measurement precision
If radar and lidar data are fused with vision data, then detection accuracy is improved, but data fusion difficulty increases
Solution Approach 1:
The patent uses cross-correlation as an intermediary processing step that simplifies the data fusion process. By cross-correlating radar-based images with vision-based images to generate enriched object queries, the system creates a unified representation that integrates multiple data sources without requiring complex fusion algorithms.
Data Source
AI summary
A perception system may be used to generate bounding boxes for objects in a vehicle scene. The perception system may receive images and feature maps corresponding to the received images. The perception system may use radar and vision based images to generate one or more bounding boxes for objects in the vehicle scene.