Neural Network Feature Map Augmentation for Multi-Perspective Scene Perception
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing vehicle perception systems rely on data from a single field of view, limiting their ability to accurately infer scene perception and understand relative locations of objects, especially when multiple perspectives are involved.
Innovation Solution
A perception-network system that provides multiple perspective views of a scene to separate branches of a neural network, generating feature maps and augmenting them to improve inference by sharing bandwidth-efficient complementary feature maps instead of raw inference data.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If multiple perspective views are processed to improve scene perception accuracy, then measurement precision is improved, but device complexity increases
Solution Approach 1:
The neural network is divided into multiple branches, with each branch processing a different perspective view independently to generate its own feature map. This segmentation allows parallel processing of multiple views while maintaining modular architecture, resolving the contradiction by organizing complexity into manageable independent units that can be processed simultaneously.
Solution Approach 2:
The system transitions from processing single-view data to multi-view data by adding a dimensional aspect of perspective diversity. Instead of making one complex processor handle all views, the invention adds another dimension (multiple branches for different views) that enriches the input space, improving measurement precision through enhanced feature representation.
2Loss of information
If feature maps are shared between multiple perception-network devices, then loss of information is reduced, but use of energy increases
Solution Approach 1:
The system extracts and shares only the essential feature maps from the neural network processing, rather than transmitting complete raw sensor data or full inference results. This extraction of critical intermediate representations reduces the volume of data exchanged between devices, lowering energy consumption while preserving the most important perceptual information needed for collaborative scene understanding.
Solution Approach 2:
The invention changes the parameter of data representation from raw sensor data to compressed feature maps. By transforming the data into a more compact and information-dense format through neural network processing, the system reduces transmission requirements and energy consumption while maintaining high information quality for collaborative perception.
Data Source
AI summary
An embodiment of the present disclosure takes the form of a method carried out by a perception-network device. The perception-network device provides a first perspective view of a scene to a first branch of a neural network, and generates a feature map via the first branch based on the first perspective view. The perception-network device augments the generated feature map with features of a complementary feature map generated by a second branch of the neural network provided with a second perspective view of the scene. The perception-network device generates a perception inference via the neural network based on the augmented feature map.