Multi-View 3D Target Detection With Bird's-Eye Feature Fusion
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing automatic driving systems require separate 3D detection on multiple images and subsequent fusion, leading to low detection efficiency for 3D spatial information around the carrier.
Innovation Solution
Perform feature extraction on images from a multi-camera view, map the features to a unified bird's-eye view space using internal and vehicle parameters, and then perform feature fusion to obtain bird's-eye view fusion features for 3D spatial information.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If separate 3D detection is performed on multiple images followed by fusion, then 3D spatial information can be obtained, but detection efficiency is reduced
Solution Approach 1:
The patent merges the 3D detection and fusion operations into a unified end-to-end process. Instead of performing separate 3D detection on each image followed by fusion, the system performs feature extraction on multi-camera images, maps features to a common bird's-eye view space, and then performs detection in this unified space, thereby combining multiple operations into one efficient pipeline that maintains accuracy while improving efficiency
Solution Approach 2:
The patent performs preliminary feature extraction and feature mapping to a bird's-eye view space before performing the actual 3D detection. This preliminary action of extracting and mapping features from multiple camera views to a unified coordinate system enables the subsequent detection to be performed more efficiently in a single operation rather than requiring multiple separate detection and fusion steps
Data Source
AI summary
Disclosed in embodiments of the present disclosure are a three-dimensional (3D) target detection method and apparatus based on multi-view fusion. In the method, feature extraction is performed on at least one image of a multi-camera view captured by a multi-camera system; the extracted feature data comprising a feature of a target object in a multi-camera view space is mapped to a same bird's-eye view space based on internal parameters and vehicle parameters of the multi-camera system so as to obtain respective feature data corresponding to the at least one image in the bird's-eye view space; a bird's-eye view fusion feature is obtained by means of feature fusion; and target prediction is performed on the target object of the bird's-eye view fusion feature to obtain 3D spatial information of the target object.