Multi-Level BEV Feature Fusion for Better Object Detection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for generating bird's eye view (BEV) features neglect many image features, leading to performance degradation in object detection due to concentrating on reference points on the ground surface, thus requiring improved methods to utilize image features effectively.
Innovation Solution
A method for generating enhanced BEV features by utilizing similarity between image features and a BEV grid at predefined levels, involving the production of reference and height BEV features, calculating weights based on their information, and synthesizing these features to create a single enhanced BEV feature using a similarity-based approach.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of manufacture
If backward mapping is used to generate BEV features by projecting only reference points, then the method is simple to implement, but most image features are neglected leading to performance degradation
Solution Approach 1:
The patent segments the BEV space into multiple levels (e.g., ground level and elevated levels) and processes image features at each level separately through backward mapping. This allows comprehensive utilization of image features across different spatial levels while maintaining the simplicity of the backward mapping approach.
Solution Approach 2:
The patent extends the traditional 2D BEV representation into a 3D multi-level BEV structure by adding the height dimension. This dimensional extension enables the system to capture and utilize image features from multiple elevation levels, thereby preventing information loss while maintaining implementation simplicity.
2Device complexity
If reference points are concentrated on the ground surface, then the mapping process is simplified, but many image features affecting object detection are neglected
Solution Approach 1:
The patent segments the mapping process into multiple level-specific mapping operations. Instead of concentrating all reference points on the ground surface, it creates separate reference point sets for different levels (ground level, elevated levels), allowing comprehensive feature capture while keeping each individual mapping operation simple.
Solution Approach 2:
The patent applies different mapping strategies to different spatial regions. Ground-level reference points are used for ground-level features, while elevated reference points are used for elevated features. This local differentiation ensures that each region's specific features are captured appropriately without increasing overall system complexity.
3Productivity
If only a spare feature is employed from image features, then the processing speed is maintained, but object detection performance degrades
Solution Approach 1:
The patent segments feature extraction into multiple parallel streams, each handling a specific level. This segmentation allows the system to process multiple features simultaneously without sequential overhead, maintaining processing speed while improving detection accuracy through comprehensive feature utilization.
Solution Approach 2:
The patent merges features from multiple levels into a unified multi-level BEV representation. By combining ground-level features and elevated features into a single comprehensive feature set, the system achieves both high processing speed (through parallel processing) and high reliability (through comprehensive feature utilization).
Data Source
Figure 1
Figure 2
Figure 3
AI summary
A method performed by an apparatus of a vehicle, the method includes: generating one or more image features from one or more images using an image analysis model; producing a reference BEV feature with reference direction information at a reference level and a height BEV feature with height direction information for each level by mapping the image features to a BEV grid; calculating a weight based on information in the reference BEV feature and the height BEV feature; and generating a single enhanced BEV feature by applying the calculated weight to the reference BEV feature and the height BEV feature.