BEV Object Flow Estimation for Autonomous Vehicle Path Planning
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing autonomous vehicle systems face challenges in accurately and efficiently detecting and tracking objects in their environment, particularly due to the computational overheads and limitations of using multiple sensing modalities like lidars and cameras, which can lead to misplacement of objects in bird's-eye view representations and difficulty in classifying unknown objects.
Innovation Solution
Implementing multi-frame temporal aggregation and dense motion estimation using a set of neural networks trained on bird's-eye view features from camera and radar data, enabling robust object classification and tracking, even with sensor dropout scenarios, to enhance object detection and path planning in autonomous vehicles.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If multiple sensing modalities like lidars and cameras are used for object detection, then detection accuracy is improved, but computational overhead increases
Solution Approach 1:
The patent combines data from multiple sensing modalities (cameras, radars, and optionally lidars) into a unified bird's-eye view representation. By merging sensor inputs at the feature level rather than processing them separately, the system achieves comprehensive object detection while reducing redundant computational operations across multiple sensing systems.
Solution Approach 2:
The patent introduces an intermediate bird's-eye view feature representation that acts as a mediator between raw sensor data and final object detection. This intermediate representation consolidates information from multiple sensors into a standardized format, simplifying subsequent processing and reducing computational overhead while maintaining detection accuracy.
2Device complexity
If traditional object detection methods are used, then system simplicity is maintained, but object classification reliability deteriorates in sensor dropout scenarios
Solution Approach 1:
The patent performs preliminary extraction of bird's-eye view features from sensor data before actual object detection and classification occurs. By preparing and storing these intermediate features in advance, the system can quickly perform detection even when sensor data is partially missing or dropped, improving reliability without significantly increasing system complexity.
Solution Approach 2:
The patent implements dynamic feature aggregation that adapts to varying sensor availability. The system dynamically adjusts which sensor inputs are used for feature extraction based on which sensors are currently operational, allowing reliable object classification even when some sensors experience dropout scenarios.
3Measurement precision
If dense motion estimation is performed, then object tracking accuracy is improved, but processing time increases
Solution Approach 1:
The patent segments the bird's-eye view feature space into discrete grids or regions before performing motion estimation. By dividing the continuous space into manageable segments, the system can perform dense motion estimation more efficiently by processing each segment independently, reducing overall processing time while maintaining tracking accuracy.
Solution Approach 2:
The patent applies motion estimation selectively to regions where objects are detected or where motion is likely to occur, rather than performing exhaustive dense estimation across the entire bird's-eye view representation. This partial action approach maintains tracking accuracy for relevant objects while significantly reducing processing time for empty or static regions.
Data Source
AI summary
A method includes obtaining, by a processing device, input data derived from a set of sensors associated with an autonomous vehicle (AV). The input data includes camera data and radar data. The method further includes extracting, by the processing device from the input data, a plurality of sets of bird's-eye view (BEV) features. Each set of BEV features corresponds to a respective timestep. The method further includes generating, by the processing device from the plurality of sets of BEV features, an object flow for at least one object. Generating the object flow includes performing at least one of: multi-frame temporal aggregation or multi-frame dense motion estimation. The method further includes causing, by the processing device, a driving path of the AV to be modified in view of the object flow.