Compressed Video Action Recognition With Two-Stage Screening
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing neural network-based action recognition solutions for videos require significant computing resources and accuracy is low, especially for short videos, leading to high computational demands and increased manual review costs.
Innovation Solution
A two-stage approach involving a first model for coarse-grained action recognition of compressed video segments using a lightweight neural network and a second model for fine-grained recognition of decoded segments, reducing computational load while maintaining accuracy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If a single heavy neural network model is used for action recognition on compressed video, then recognition accuracy may be improved, but computing power requirements increase significantly
Solution Approach 1:
The video data processing is segmented into two distinct stages: first, a lightweight neural network performs coarse-grained recognition on compressed video segments to identify candidate regions; second, a heavier neural network performs fine-grained recognition on decoded versions of only those candidate segments. This segmentation allows the system to use computational resources efficiently while maintaining high recognition accuracy.
Solution Approach 2:
The lightweight neural network performs preliminary action by conducting coarse-grained recognition on compressed video segments before the heavy model processes any data. This preliminary filtering identifies candidate segments that contain potential actions, allowing the subsequent heavy model to focus only on these relevant segments rather than processing entire videos, thus significantly reducing computing power requirements.
2Measurement precision
If compressed video segments are directly processed by a heavy model, then recognition accuracy improves, but decoding time and computational load increase
Solution Approach 1:
The system applies partial decoding only to candidate segments identified by the lightweight model, rather than decoding entire videos. This partial action approach processes only the necessary portions of video data at high quality, significantly reducing decoding time while maintaining recognition accuracy for the segments that matter most.
3Measurement precision
If full video decoding is performed before recognition, then recognition accuracy improves, but computational resources and processing time increase significantly
Solution Approach 1:
The video processing pipeline is segmented into coarse-grained recognition on compressed segments and fine-grained recognition on decoded candidate segments. This segmentation enables the system to maintain high productivity by avoiding full video decoding, while still achieving high recognition accuracy through selective decoding of only those segments identified as containing potential actions.
Data Source
AI summary
Provided is an action recognition method. The method includes: acquiring grouped video data by grouping original compressed video data; inputting the grouped video data into a first predetermined model, and determining, based on an output result of the first predetermined model, target grouped video data containing an action; acquiring to-be-recognized grouped video data by decoding the target grouped video data; and inputting the to-be-recognized grouped video data into a second predetermined model, and determining, based on an output result of the second predetermined model, a type of an action contained in the to-be-recognized grouped video data.