Multi-Azimuth BEV Fusion for Occlusion-Robust Small Object Perception
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Traditional bird's eye view (BEV) perception methods struggle to handle occlusions and accurately capture small objects due to reliance on single viewpoint representations, leading to loss of 3D information and challenges in autonomous driving and robot navigation.
Innovation Solution
Generate multi-azimuth BEV perception by integrating perception sensor data from multiple azimuth angles, using techniques such as voxelization and multi-azimuth fusion of LiDAR and camera data to create a more comprehensive and robust BEV representation.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Device complexity
If single viewpoint representations are used to generate BEV perception, then the transformation process is simple, but the ability to handle occlusions and capture small objects deteriorates
Solution Approach 1:
The patent segments the BEV representation task into multiple viewpoint-specific BEV features (front, rear, left, right viewpoints) rather than using a single unified viewpoint. Each viewpoint processes occlusions and small objects from its specific perspective, and the results are fused to create a comprehensive BEV representation that overcomes the limitations of any single viewpoint.
Solution Approach 2:
The patent adds the viewpoint dimension to the traditional single BEV representation. Instead of generating BEV features from one viewpoint, the system generates BEV features from multiple viewpoints (front, rear, left, right) and fuses them. This multi-dimensional approach preserves 3D information that would be lost in a single viewpoint transformation.
2Productivity
If single viewpoint representations are used to generate BEV perception, then the processing speed is fast, but the accuracy of capturing small objects deteriorates
Solution Approach 1:
The patent divides the object detection task into viewpoint-specific detection tasks. Each viewpoint (front, rear, left, right) independently processes and detects objects including small objects within its field of view. This segmentation allows each viewpoint to focus on specific spatial regions, improving small object detection accuracy while maintaining processing efficiency through parallel computation.
Solution Approach 2:
The patent performs partial detection actions at each viewpoint, where each viewpoint generates BEV features for a specific portion of the environment. By combining these partial detections from multiple viewpoints, the system achieves comprehensive small object detection that is more accurate than a single viewpoint could provide alone.
3Measurement precision
If multi-azimuth projections are generated and fused, then the representation accuracy improves, but the computational complexity increases
Solution Approach 1:
The patent segments the computational workload into four independent viewpoint processing streams (front, rear, left, right). Each stream generates BEV features independently, allowing for parallel computation that reduces overall processing time despite the increased complexity of multi-view processing. The segmentation enables modular implementation and efficient resource utilization.
Solution Approach 2:
The patent merges the BEV features from multiple viewpoints through a fusion process that integrates information from front, rear, left, and right viewpoints into a unified comprehensive BEV representation. This merging step consolidates the computational results while preserving the enhanced accuracy benefits of multi-view processing.
Data Source
AI summary
Certain aspects of the present disclosure provide techniques for bird's eye view perception. A method for multi-azimuth bird's eye view perception by an apparatus comprising: obtaining first perception sensor data, generated by one or more sensors, corresponding to an environment of the apparatus; generating, based on the first perception sensor data, a first plurality of projections of the environment from multiple azimuth perspective angles; extracting a respective feature set from each of the first plurality of projections; fusing the respective feature set of each of the first plurality of projections into a multi-azimuth bird's eye view of the environment; and storing the multi-azimuth bird's eye view in one or more memories.