Temporal Feature Map Enrichment for Accurate 3D Object Detection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Challenges exist in accurately determining object characteristics and drawing precise 3D bounding boxes in real-time driving environments due to insufficient semantic and local information obtained by neural networks.
Innovation Solution
The use of earlier-in-time feature maps to enrich later-in-time feature maps in image streams from the same image sensor, enhancing the data available for object detection and trajectory prediction in autonomous vehicles.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If neural networks process image streams in real-time driving environments, then processing speed is improved, but measurement precision of object characteristics deteriorates due to insufficient semantic and local information
Solution Approach 1:
The patent applies preliminary action by generating and storing feature maps from historical image frames before they are needed for current object detection. These pre-computed feature maps containing semantic and local information are then retrieved and integrated with current frame features, enabling fast real-time processing without sacrificing measurement precision since the computationally intensive feature extraction has already been performed on previous frames
Solution Approach 2:
The patent transitions from processing single 2D image frames to utilizing 3D feature maps that incorporate temporal dimension by stacking feature representations from multiple historical frames. This dimensional expansion allows the neural network to access rich semantic and local information across time without increasing processing speed requirements for each individual frame, as the temporal context is already encoded in the extended feature space
2Measurement precision
If more semantic and local information is obtained for object detection, then measurement precision is improved, but device complexity increases due to additional processing requirements
Solution Approach 1:
The patent merges feature maps from multiple historical frames into a unified temporal feature representation that combines semantic and local information. By consolidating these features into integrated feature maps that can be directly fed into the object detection network, the system achieves high measurement precision without proportionally increasing device complexity, as the merging operation leverages existing neural network layers rather than requiring entirely new processing subsystems
Solution Approach 2:
The patent performs preliminary feature extraction and semantic enrichment on historical frames before object detection, pre-computing and storing enriched feature maps that contain both semantic and local information. This preliminary processing distributes computational load over time, allowing the main detection system to operate with reduced complexity while still accessing comprehensive feature information when needed
Data Source
AI summary
A system may be used to determined object characteristics and/or generate bounding boxes for objects in a vehicle scene by enriching later-in-time feature maps using earlier-in-time feature maps. The system may generate a feature map from a received. Using an earlier-in-time feature map, the system may enrich semantic data of the generated feature map to form an enriched feature map. The system may use the enriched feature map to generate one or more object characteristics of an object in the scene.