Action Localization Using Target Segments Without Optical Flow
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional action localization methods require large models and significant computational resources, limiting their applicability due to high memory and computing demands, and often rely on optical flow data, which hampers real-time performance.
Innovation Solution
An action localization method that identifies target video segments containing objects, performs action recognition on selected frames using a temporal residual neural network, and aggregates results to determine action localization without requiring optical flow data, thereby reducing computational load and improving accuracy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If conventional action localization methods use large models and significant computational resources, then action localization accuracy is improved, but memory capacity and computing power requirements increase, limiting the scope of application
Solution Approach 1:
The patent segments the video processing task into two distinct stages: (1) target object detection to identify video segments containing target objects, and (2) action recognition only on those segmented regions. This segmentation allows the system to apply computationally intensive action recognition models only to relevant portions of the video, rather than processing the entire video with large models, thereby reducing overall memory and computing power requirements while maintaining localization accuracy.
Solution Approach 2:
The patent extracts and removes the dependency on optical flow data from the action localization pipeline. By formulating action localization as a direct classification problem on video segments identified through object detection, the method eliminates the need for computationally expensive optical flow calculations, significantly reducing the computational burden and memory requirements while preserving the core functionality of accurate action localization.
2Reliability
If conventional action localization methods rely on optical flow data, then action recognition capability is enhanced, but real-time performance is hampered due to increased computational load
Solution Approach 1:
The patent extracts and removes the optical flow data dependency from the action localization framework. By directly classifying actions on video segments using object detection results without requiring optical flow computation, the method maintains reliable action recognition capability while eliminating the real-time performance bottleneck caused by optical flow calculations.
Solution Approach 2:
The patent changes the input parameters for action recognition from optical flow data to direct video segment features. This parameter change allows the system to use simpler, faster-to-process visual features while maintaining action recognition reliability, thereby enabling real-time performance that was previously hampered by the computational intensity of optical flow-based methods.
3Measurement precision
If action recognition is performed on the entire video, then comprehensive action detection is achieved, but processing time and computational resources are excessively consumed
Solution Approach 1:
The patent segments the video based on target object detection results, identifying only those portions of the video that contain objects of interest. Action recognition is then performed exclusively on these segmented video regions rather than the entire video. This approach ensures comprehensive action detection coverage for all relevant objects while dramatically reducing processing time by excluding irrelevant video portions from the computationally intensive action recognition step.
Data Source
AI summary
An action localization method, device, electronic equipment, and computer-readable storage medium are provided. The action localization method includes: identifying at least one target video segment containing a target object in a video; acquiring a first action recognition result of at least one image frame in the at least one target video segment and a second action recognition result of the target video segment; and acquiring an action localization result of the video based on the first action recognition result and the second action recognition result.