Two-Stage Feature Aggregator for Scalable Video Face Recognition
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Video subject recognition systems face high computational and storage complexity due to the need to compare and analyze multiple frames, leading to less accurate representations and identification/verification results when using existing aggregation techniques like pooling methods.
Innovation Solution
A video frame processing system with a feature extractor and a two-stage feature aggregator that generates a kernel adapted to the content of feature sets to emphasize desirable features and deemphasize noisy ones, resulting in a more discriminative aggregated feature set for improved recognition accuracy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If multiple pairs of frames are compared and multiple matching results are analyzed to maintain comprehensive information across multiple frames, then recognition accuracy is improved, but computational complexity increases to O(n2)
Solution Approach 1:
The patent segments the video recognition task into frame-level feature extraction followed by temporal aggregation. Instead of comparing all frame pairs (O(n2)), the system extracts features from each frame independently and then aggregates them temporally, dividing the complex problem into manageable segments that reduce computational burden while preserving recognition accuracy.
Solution Approach 2:
The patent introduces a temporal dimension to the feature aggregation process. By stacking feature vectors from multiple frames along the temporal dimension and applying temporal pooling operations, the system transforms the problem from comparing frames in pairs to aggregating features across time, thereby reducing computational complexity from O(n2) to O(n).
2Measurement precision
If sets of features from multiple frames are maintained to comprehensively represent the subject, then recognition accuracy is improved, but storage complexity and indexing requirements increase
Solution Approach 1:
The patent extracts only the essential temporal patterns from video frames through temporal pooling operations. Instead of storing and processing all raw frame features, the system extracts discriminative temporal features by applying pooling operations that capture the most relevant information across frames, thereby reducing storage requirements while maintaining recognition accuracy.
Solution Approach 2:
The patent changes the parameter representation by transforming multiple frame feature vectors into a single aggregated feature vector through temporal pooling. This parameter transformation consolidates temporal information into a compact representation, reducing storage complexity from storing n separate feature sets to storing one aggregated feature set while preserving essential temporal characteristics.
3Device complexity
If average pooling or max pooling techniques are used to aggregate features from multiple frames, then computational complexity is reduced, but representation accuracy deteriorates
Solution Approach 1:
The patent implements dynamic temporal pooling that adapts to the content of video frames. Instead of using fixed average or max pooling operations, the system dynamically determines pooling strategies based on temporal relationships between frames, allowing the aggregation process to adapt to different video scenarios and maintain high representation accuracy while keeping computational complexity low.
Solution Approach 2:
The patent incorporates feedback mechanisms in the temporal aggregation process where the system learns optimal pooling operations from training data. The temporal pooling parameters are adjusted based on feedback from recognition performance, enabling the system to select aggregation strategies that maintain high representation accuracy while preserving computational efficiency.
Data Source
AI summary
In a video frame processing system, a feature extractor generates, based on a plurality of data sets corresponding to a plurality of frames of a video, a plurality of feature sets, respective ones of the feature sets including features extracted from respective ones of the data sets. A first stage of the feature aggregator generates a kernel for a second stage of the feature aggregator. The kernel is adapted to content of the feature sets so as to emphasize desirable ones of the feature sets and deemphasize undesirable ones of the feature sets. In the second stage of the feature aggregator the kernel generated by the first stage is applied to the plurality of feature sets to generate a plurality of significances corresponding to the plurality of feature sets. The feature sets are weighted based on corresponding significances and weighted feature sets are aggregated to generate an aggregated feature set.