Vehicle Driving Control Using BEV Map Memory Under Occlusion
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing neural network-based systems struggle to accurately predict and vectorize static map elements in autonomous driving due to occlusion by dynamic objects, which are crucial for safe navigation and path planning.
Innovation Solution
A method and apparatus that utilize multi-view images to extract bird's-eye view (BEV) features and map queries, generate vectorized maps using first memory tokens, and control vehicle driving based on these maps, employing neural networks to predict and vectorize map elements across consecutive time points.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If neural networks are used to predict road geometry and construct high-quality maps, then map quality and autonomous driving safety are improved, but static map elements are repeatedly occluded by dynamic objects on the road
Solution Approach 1:
The system performs preliminary action by predicting and vectorizing static map elements in advance, storing them as memory tokens that can be recalled and used even when occluded by dynamic objects. This allows the system to have map information ready before occlusion occurs, maintaining prediction accuracy despite temporary blocking of visual input.
Solution Approach 2:
The patent introduces memory tokens as an intermediary representation that mediates between the neural network's prediction capability and the actual visual input. These tokens store vectorized map element information and serve as a buffer that compensates for occlusion, allowing the system to maintain accurate map predictions even when direct visual observation is blocked.
2Measurement precision
If multi-view images are processed to extract BEV features and generate vectorized maps, then static map element prediction accuracy is improved, but computational complexity and processing time increase
Solution Approach 1:
The system extracts only the essential vectorized features of map elements (such as lane boundaries, road edges, and intersections) and stores them as compact memory tokens. This extraction process separates the critical geometric information from the full multi-view image data, reducing computational complexity while maintaining prediction accuracy for static map elements.
Solution Approach 2:
The patent transforms the representation parameters of map elements from raw pixel data in multi-view images to vectorized geometric parameters stored as memory tokens. This parameter transformation changes the data format from high-dimensional image pixels to compact vector representations, reducing processing complexity while preserving the essential geometric information needed for accurate prediction.
3Stability of the object's composition
If memory tokens from previously-processed image frames are used to generate vectorized maps, then temporal consistency and map quality are improved, but memory storage requirements and processing overhead increase
Solution Approach 1:
The system creates compact copies of map element information as memory tokens from previously processed frames. Instead of storing entire image frames or full feature maps, the patent generates simplified vectorized copies that capture the essential geometric properties of static map elements. These token copies maintain temporal consistency while occupying minimal storage space.
Solution Approach 2:
The patent segments the memory storage requirement into individual tokens for different map elements (lanes, road boundaries, intersections) rather than storing unified map representations. This segmentation allows the system to store only the necessary vectorized parameters for each map element, reducing overall memory requirements while maintaining temporal consistency for each segmented element.
Data Source
AI summary
Disclosed are a method and apparatus for controlling driving of a vehicle and a vehicle. The method of controlling the driving of the vehicle includes receiving multi-view images including image frames at consecutive time points corresponding to a driving environment of the vehicle, extracting bird's-eye view (BEV) features and map queries respectively corresponding to the consecutive time points for each of the image frames, generating a vectorized map by predicting and vectorizing map elements included in the image frames based on first memory tokens stored in a memory corresponding to queries of previous image frames of the image frames, the BEV features, and the map queries, and controlling the driving of the vehicle based on the vectorized map.