Cross-view attention mechanism for visual perception tasks
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing transformer-based machine learning networks face exponential computational complexity when performing self-attention on large input sequences, limiting the resolution and accuracy of visual perception tasks for spatially distributed image data.
Innovation Solution
Implementing linear transformer layers and fixed embedding sizes to determine cross-view attention, which reduces computational complexity to linear with respect to input image resolution, enabling efficient processing of higher-resolution feature maps.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If transformer-based machine learning networks perform self-attention on large input sequences, then visual perception accuracy is improved, but computational complexity increases exponentially
Solution Approach 1:
The patent divides the computational process into distinct stages: feature extraction from multiple views, projection to reduced embedding dimensions, and cross-view attention computation. This segmentation allows each stage to be optimized independently, reducing overall computational complexity while preserving accuracy
Solution Approach 2:
The patent changes the embedding dimension parameter from its original high-dimensional state to a reduced dimension through projection operations. This parameter change maintains the essential information needed for accurate visual perception while dramatically reducing the computational burden of attention operations
2Measurement precision
If input image resolution is increased, then visual perception accuracy is improved, but computational complexity increases exponentially
Solution Approach 1:
The patent applies parameter changes by reducing the embedding dimension of feature representations. This allows the system to process higher-resolution images efficiently by maintaining only the most salient features, thus improving visual perception accuracy without proportionally increasing computational complexity
Solution Approach 2:
The patent extracts and retains only the most important features from high-resolution images through the projection process. By taking out and keeping only the essential information needed for visual perception tasks, the system achieves high accuracy with reduced computational requirements
Data Source
AI summary
Systems and techniques are provided for processing image data. For example, a process can include obtaining a plurality of input images associated with a plurality of different spatial views. The process can include generating a set of features based on the plurality of input images. The process can include generating a set of projected features based on the set of features, wherein an embedding size associated with the set of projected features is smaller than an embedding size associated with the set of features. The process can include determining a cross-view attention associated with the plurality of different spatial views, the cross-view attention determined using the set of projected features.