BEV Ray Casting Densification for Sparse ADAS View Transforms
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing camera-based systems for advanced driver-assistance systems (ADAS) face inefficiencies in generating bird's eye view (BEV) images due to sparsity in information density, leading to increased computational burden, latency, and reduced accuracy in automotive perception tasks.
Innovation Solution
The technique involves projecting BEV features outward into discrete feature pixels along rays in the BEV image space and applying convolution operations to diffuse information from neighboring locations into zero-value areas, thereby increasing information density and reducing the need for additional convolution layers.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Device complexity
If BEV features are projected outward into discrete feature pixels along rays in the BEV image space, then the BEV image generation process is simplified, but information density becomes sparse in regions between adjacent rays
Solution Approach 1:
A convolution operation is introduced as an intermediary mechanism to transfer feature information from non-zero-value locations to zero-value locations in the BEV image space. This convolution layer acts as a mediator that diffuses information across the sparse regions between rays, ensuring that downstream machine learning models receive sufficient feature data without requiring complex projection adjustments
Solution Approach 2:
The patent changes the density distribution parameter of the BEV features by applying a convolution operation that redistributes feature information from dense regions (along rays) to sparse regions (between rays). This parameter transformation converts the sparsity pattern from a ray-based distribution to a more uniformly dense distribution, improving information availability for downstream tasks
2Reliability
If machine learning models transport feature information from non-zero-value locations into zero-value locations, then complete BEV feature coverage is achieved, but computational burden and processing time increase
Solution Approach 1:
The convolution operation is performed in advance before the BEV features are input to downstream machine learning models. This preliminary action pre-diffuses feature information into zero-value locations, so that when the data reaches the machine learning model, the feature coverage is already complete and no additional computational effort is needed for feature transport
Solution Approach 2:
The feature transport function is extracted from the downstream machine learning model and placed in a separate convolution operation layer. This extraction allows the machine learning model to focus solely on feature analysis and decision-making tasks, while the convolution layer handles the information distribution task, improving overall system efficiency
3Loss of information
If convolution layers are added after view transform operation to transport feature information, then information density is improved, but model complexity increases
Solution Approach 1:
The convolution operation is applied locally at the BEV feature generation stage, specifically targeting the sparsity issue in the BEV image space. Rather than adding complex global optimization layers throughout the model, a localized convolution operation is applied where the sparsity problem originates, efficiently addressing the information density issue with minimal added complexity
Data Source
AI summary
An apparatus includes a memory for storing image data; and processing circuitry in communication with the memory. The processing circuitry is configured to obtain a plurality of camera images and generate camera features from the plurality of camera images. According to such an example, processing circuitry may be configured to project the camera features from the plurality of camera images into a birds-eye-view (BEV) image space to generate BEV features. In certain examples, processing circuitry is configured to determine a plurality of zero-value locations of the BEV features within the BEV image space and diffuse non-zero BEV features in the BEV image space into the plurality of zero-value locations within the BEV image space to generate increased density BEV features. In at least one example, processing circuitry is configured to output a BEV image having the increased density BEV features.