Time Domain Action Detection Using Adjacent Segment Pooling
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current time domain action detection methods in computer vision struggle to accurately identify and segment actions in videos, particularly in unedited long videos, due to limitations in capturing contextual information from adjacent segments, which affects the precision of action integrity assessment.
Innovation Solution
A method that involves obtaining a time domain interval with action instances and adjacent segments, extracting action features, performing pooling processing to obtain a global feature, and determining an action integrity score using time domain pyramid-typed pooling and convolutional neural networks to enhance action category detection and positioning.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If traditional action detection methods are used to process long videos, then the detection speed is maintained, but the accuracy of action integrity identification deteriorates due to inability to capture contextual information from adjacent segments
Solution Approach 1:
The video is divided into multiple segments including the target segment and adjacent segments. Action features are extracted separately for each segment, and then pooled together to form a global feature that captures contextual information across segment boundaries, improving action integrity identification accuracy
Solution Approach 2:
The method extends the detection from a single segment dimension to a multi-segment temporal dimension by incorporating adjacent segments. This dimensional expansion allows the model to capture contextual information before and after the target action, enhancing identification accuracy without proportionally increasing system complexity
2Loss of information
If action features are extracted from multiple video segments including adjacent segments, then the contextual information is improved, but the computational complexity increases
Solution Approach 1:
The method extracts only the necessary action features from adjacent segments rather than processing all video data. By selectively extracting and pooling action features from key temporal regions, the system retains essential contextual information while reducing unnecessary computational overhead
Solution Approach 2:
Instead of processing the entire video or all possible segments, the method applies partial action by focusing computation on the target segment and a limited number of adjacent segments. This selective processing maintains contextual information while avoiding excessive computational energy consumption
3Measurement precision
If pooling processing is performed on action features from multiple segments to obtain global feature, then the action detection precision is improved, but the processing time increases
Solution Approach 1:
The method merges action features from multiple segments through pooling operations to create a comprehensive global feature representation. This consolidation allows the system to leverage contextual information from adjacent segments while maintaining efficient processing through unified feature aggregation rather than separate analyses
Data Source
AI summary
Time domain action detecting methods and systems, electronic devices, and computer storage medium are provided. The method includes: obtaining a time domain interval in a video with an action instance and at least one adjacent segment in the time domain interval; separately extracting action features of at least two video segments in candidate segments, where the candidate segments comprises video segment corresponding to the time domain interval and adjacent segments thereof; pooling the action features of the at least two video segments in the candidate segments, to obtain a global feature of the video segment corresponding to the time domain interval; and determining, based on the global feature, an action integrity score of the video segment corresponding to the time domain interval. The embodiments of the present disclosure benefit accurately determining whether a time domain interval comprises an integral action instance, and improve the accuracy rate of action integrity identification.