Object Tracking Aggregation Network With Hybrid Convolution and Self-Attention
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing object tracking methods face challenges due to object motion blur, fast motion, low resolution, and background clutter, which affect the precision of CNN-based and self-attention-based trackers, particularly in capturing local and global dependencies.
Innovation Solution
A method and system using an aggregation network that combines hybrid convolution and self-attention to process template and search features, employing depthwise and pointwise convolutions to separate spatial and channel features, reduce redundancy, and utilize self-attention to capture long-distance dependencies, followed by a drop-out layer to enhance generalization.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If CNN is used to extract local features, then local information extraction is improved, but global dependency capture deteriorates due to limited receptive field
Solution Approach 1:
The patent merges CNN and self-attention mechanisms into a hybrid architecture where CNN extracts local features and self-attention captures global dependencies. The feature fusion network combines these two types of features, allowing the system to benefit from both local precision and global context without sacrificing either aspect.
2Loss of information
If self-attention is used to capture global dependencies, then long-distance dependency capture is improved, but computational complexity increases due to quadratic time complexity
Solution Approach 1:
The patent segments the feature extraction process into two independent parts: CNN for local features and self-attention for global features. This segmentation allows each component to optimize for its specific function while the overall system benefits from both approaches, managing computational complexity through functional division.
3Measurement precision
If hybrid convolution is used to process features, then feature representation is improved, but processing time increases
Solution Approach 1:
The patent performs preliminary feature extraction using CNN before applying self-attention. This preliminary action organizes the data in advance, allowing the self-attention mechanism to operate more efficiently on pre-processed features, thereby reducing the overall time required for hybrid feature processing.
Data Source
AI summary
Corresponding template features and search features are obtained by convolution operation, and respectively used as input features in aggregation modules. Intermediate features are obtained by performing convolution operation on the input features. The aggregation modules share the same convolution operation, and hybrid convolution in the aggregation module uses a depthwise convolution and a pointwise convolution to separate mixture between space and channel of the intermediate features. Redundancy in spatial and channel features is reduced while increasing receptive field. Self-attention module in the aggregation module learns intermediate features, and adaptively focuses on different regions to capture more global correlations. Output features of the hybrid convolution are added to output features of the self-attention module to pass through a drop-out layer to obtain final output features. The output features aggregate local and global context information. Overfitting of network is alleviated during training, thereby improving generalization ability of tracker.