Compressed Video Action Recognition With Selective Decoding
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing neural network-based action recognition solutions for videos require significant computing resources and are inefficient, particularly for short videos, leading to high CPU and GPU demands and low accuracy, necessitating improved computing efficiency and recognition accuracy.
Innovation Solution
A two-stage approach involving a first model for coarse-grained action recognition on compressed video segments followed by a second model for fine-grained recognition, utilizing pre-trained neural networks to reduce computing power and enhance accuracy, including techniques like feature shift and multi-receptive-field pooling.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If full video decoding and neural network processing is applied to all video segments, then recognition accuracy can be maintained, but computing resources and processing time increase significantly
Solution Approach 1:
The patent divides video processing into two stages: first segmenting video segments at the compressed format level using a first neural network to identify candidate segments containing actions, then decoding only those candidate segments for detailed analysis by a second neural network. This segmentation approach maintains recognition accuracy while significantly reducing the number of segments requiring full decoding and processing.
Solution Approach 2:
The patent performs preliminary screening of video segments in compressed format before full decoding using a first pre-trained neural network. This preliminary action identifies candidate segments that likely contain actions, allowing the system to focus subsequent detailed analysis only on these candidates rather than processing the entire video, thus improving efficiency while maintaining accuracy.
2Measurement precision
If all video segments are decoded for action recognition, then recognition accuracy is maintained, but CPU and GPU computing power requirements increase
Solution Approach 1:
The patent extracts and processes video segments in two distinct phases: first extracting candidate segments from compressed video data using minimal computing resources, then extracting only those specific candidate segments for full decoding. This extraction strategy eliminates the need to decode and process entire videos or non-candidate segments, reducing computing power consumption while preserving recognition accuracy for action-containing segments.
Solution Approach 2:
The patent performs preliminary identification of action-containing segments in compressed format before committing to full decoding. This preliminary action uses the first neural network to screen segments at low computational cost, ensuring that only segments with high probability of containing actions undergo resource-intensive decoding and detailed analysis by the second neural network.
3Quantity of substance
If compressed video format is used for storage and transmission, then storage efficiency improves, but direct processing for action recognition becomes difficult
Solution Approach 1:
The patent performs preliminary processing of compressed video segments using a first neural network that can operate directly on compressed formats without full decoding. This preliminary action extracts motion vectors, pixel differences, and other features from the compressed data structure, enabling action detection in compressed format before selective decoding of candidate segments.
Solution Approach 2:
The patent segments the compressed video into I-frames and P-frames, processing each segment type appropriately. By working with compressed segments rather than requiring full video decoding, the system maintains storage efficiency while enabling action recognition through clever segmentation and selective processing of compressed data structures.
Data Source
Figure 1~3
Figure 4~5
Figure 6~7
AI summary
An action recognition method and apparatus, and a device and a storage medium. The method comprises: performing grouping processing on original compressed video data to obtain grouped video data (101); inputting the grouped video data into a first preset model, and determining target grouped video data, which includes an action, according to an output result of the first preset model (102); decoding the target grouped video data to obtain grouped video data to be recognized (103); and inputting the grouped video data to be recognized into a second preset model, and determining, according to an output result of the second preset model, the type of the action contained in the grouped video data to be recognized (104).