Video Action Localization Using Per-Support Attention Matching
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing action recognition systems face challenges in accurately identifying common actions across support videos with varying backgrounds and illumination due to conventional methods that simultaneously attend frames from multiple videos, leading to mismatched frame attention.
Innovation Solution
A neural network apparatus computes attention for each support video separately, determining which frames match the query video by generating attention feature maps, up-sampling, and localizing actions through bounding-box generation and temporal smoothing.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If simultaneous attention methods are used to process multiple support videos, then processing efficiency is maintained, but action localization accuracy deteriorates due to missed frame attention caused by large intra-class variations
Solution Approach 1:
The patent divides the simultaneous processing of multiple support videos into sequential individual processing stages. Each support video is processed separately through the neural network, allowing the system to compute attention for each video independently. This segmentation enables the model to focus on intra-class variations within each video without being overwhelmed by variations across multiple videos simultaneously, thereby improving action localization accuracy while maintaining manageable processing complexity through structured sequential operations.
2Measurement precision
If individual support video processing is used to improve frame attention accuracy, then action localization accuracy improves, but processing time increases
Solution Approach 1:
The patent applies preliminary action by first processing individual support videos to compute accurate attention maps before aggregating results. The system pre-computes attention for each support video separately, identifying relevant frames with high accuracy. This preliminary individual processing ensures that frame attention accuracy is maximized before the final aggregation step, allowing the system to achieve precise action localization without requiring excessive processing time in subsequent stages.
3Measurement precision
If conventional attention methods are used, then computational resources are conserved, but frame matching accuracy deteriorates due to inability to handle intra-class variations
Solution Approach 1:
The patent applies local quality by enhancing the neural network's attention mechanism to specifically address intra-class variations within each support video. The attention computation is tailored to local characteristics of each video, allowing the model to adapt to different backgrounds, illuminations, and conditions individually. This localized attention approach improves frame matching accuracy by focusing computational resources on relevant local features rather than applying uniform processing across all videos, thereby achieving high accuracy with optimized resource utilization.
Data Source
AI summary
Aspects of the disclosure are directed to an apparatus configured to perform common-action localization. In certain aspects, the apparatus may receive a query video comprising a plurality of frames, wherein a first query proposal is determined based on a subset of frames of the plurality of frames, the first query proposal indicative of an action depicted on the subset of frames. In certain aspects, the apparatus may determine a first attendance for a first support video of a plurality of support videos. In certain aspects, the apparatus may determine a second attendance for a second support video of the plurality of support videos after computing the first attendance.