Hybrid Bird's-Eye View Perception for Near and Far Feature Detection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing multi-camera automotive vision systems struggle to effectively generate a comprehensive birds-eye view (BEV) of a vehicle's surroundings, leading to incomplete detection of both near and far environmental features, which hinders user understanding and autonomous driving capabilities.
Innovation Solution
A hybrid BEV image generation system that combines a first BEV image, constructed through sensor fusion and voxelization, with a second BEV image generated via Inverse Perspective Mapping (IPM), enhancing feature detection and accuracy by merging these images using a deep learning neural network to create an overhead view.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If a single BEV generation method (either sensor fusion/voxelization or IPM) is used, then the system complexity is reduced, but the detection accuracy and range for both near and far features deteriorate
Solution Approach 1:
The patent combines two different BEV generation methods (sensor fusion/voxelization and IPM) into a hybrid system. The first BEV image is generated through sensor fusion and voxelization, while the second BEV image is generated through IPM. These two images are then merged to create a comprehensive BEV representation that leverages the strengths of both methods, improving detection accuracy for both near and far features.
Solution Approach 2:
The patent segments the BEV generation process into two distinct pathways: one using sensor fusion and voxelization for structured feature extraction, and another using IPM for geometric transformation. Each pathway processes the input camera images independently to produce separate BEV images, which are then combined. This segmentation allows each method to optimize for its strengths while contributing to the overall detection accuracy.
2Measurement precision
If multiple processing modules are combined to improve detection accuracy, then the detection precision improves, but the processing time and computational complexity increase
Solution Approach 1:
The patent applies partial action by selectively processing image data through two different BEV generation pathways only when comprehensive detection is required. The system can operate with either method alone depending on conditions, but combines them when maximum detection accuracy is needed. This approach balances computational resources with detection requirements, avoiding unnecessary processing time when full hybrid processing is not required.
Data Source
AI summary
A method includes obtaining image frames from each camera disposed along a vehicle, where each image frame corresponds to a same timestamp. The method further includes constructing a first birds-eye view (BEV) image from each image frame with a first BEV module and constructing a second BEV image from each image frame by Inverse Perspective Mapping (IPM) with a second BEV module. The first BEV module extracts features of an external environment of the vehicle from each image frame, transforms the features to a three-dimensional space, and projects the three-dimensional space onto an overhead two-dimensional plane. Subsequently, a merging module merges the first and second BEV images to produce a hybrid BEV image. Features of an external environment of the vehicle within the hybrid BEV image are detected by a deep learning neural network and the hybrid BEV image is displayed to a user in the vehicle.