Video Object Identification via Spatiotemporal Feature Fusion
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current person re-identification (Re-ID) technologies face challenges in accurately identifying individuals across non-overlapping camera views due to limitations in capturing distance, resolution, pose, occlusion, background, and lighting, which affect the reliability of whole-body identification.
Innovation Solution
A processor-implemented method using a transformer to extract initial feature maps from video images, generating a target feature map through a feature fusion network with self-attention and cross-attention modules, and obtaining a global feature vector to identify objects in videos, thereby improving identification accuracy by simultaneously extracting time and spatial domain features.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If traditional CNN-based Re-ID systems are used, then the system complexity is low, but the identification accuracy deteriorates due to inability to effectively capture temporal and spatial features
Solution Approach 1:
The patent segments the feature extraction process into distinct spatial and temporal components. The spatial feature extraction module processes individual frames to capture appearance information, while the temporal feature extraction module processes sequences of frames to capture motion and evolution patterns. This segmentation allows each module to specialize in one aspect, improving overall identification accuracy while maintaining manageable system complexity through modular design.
Solution Approach 2:
The patent transitions from traditional 2D spatial feature extraction to 4D spatiotemporal feature extraction by adding temporal dimensions. The transformer-based temporal feature extraction module processes video sequences across time, capturing motion patterns and appearance evolution. This dimensional expansion enables the system to leverage both spatial appearance and temporal dynamics, significantly improving identification accuracy in Re-ID tasks.
2Adaptability or versatility
If entire-body identification is used to overcome camera limitations, then face recognition is no longer required, but identification reliability deteriorates due to pose, occlusion, background, and lighting variations
Solution Approach 1:
The patent merges spatial feature extraction from individual frames with temporal feature extraction from video sequences. The spatial module captures appearance information robust to pose and occlusion, while the temporal module captures motion patterns that are invariant to background and lighting changes. By combining these complementary features through the fusion module, the system achieves reliable identification that adapts to various camera limitations.
Solution Approach 2:
The patent utilizes continuous temporal information from video sequences rather than relying on single static frames. The temporal feature extraction module processes continuous video data to capture motion patterns and appearance evolution over time. This continuity provides robustness against transient occlusions, pose changes, and lighting variations, maintaining high identification reliability even when individual frames are degraded.
3Area of stationary object
If multiple cameras with non-overlapping views are used, then coverage area is increased, but identification difficulty increases due to viewpoint changes
Solution Approach 1:
The patent performs preliminary spatial feature extraction from each camera's view independently before temporal fusion. The spatial feature extraction module prepares appearance features from each non-overlapping camera view, normalizing them for cross-camera consistency. This preliminary processing enables the subsequent temporal module to effectively fuse features from multiple viewpoints, making identification across non-overlapping cameras as effective as with overlapping views.
Data Source
AI summary
A processor-implemented method including extracting initial feature maps from respective images extracted from a video, wherein the extracting of the initial features maps is performed using a transformer, generating a target feature map by fusing the initial feature maps using a feature fusion network including one or more layers, and identifying an object in the video based on the target feature map.