Multi-Scale Skeleton Video Action Recognition via Adaptive Feature Extraction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing video action recognition technologies face challenges in accurately recognizing multi-granularity actions due to the use of one-scale skeleton representations and the propensity for information bypass in typical skeleton-based graph convolutional networks, which affects the accuracy of video action recognition.
Innovation Solution
The method employs content-aware and multiple-scale skeleton 3D-graph convolutional networks, utilizing OpenPose for skeleton extraction, scale-invariant feature transform (SIFT) for feature vector extraction, and linking these vectors with point coordinates to generate a convolutional neural network for accurate video action recognition, enabling adaptive feature extraction and multi-granularity video action recognition.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Device complexity
If one-scale skeleton representation is used for video action recognition, then the system complexity is reduced, but the recognition accuracy for multi-granularity actions deteriorates
Solution Approach 1:
The patent segments the skeleton representation into multiple scales (coarse-grained and fine-grained levels), allowing the system to analyze actions at different granularities simultaneously. This segmentation enables accurate recognition of both overall action patterns and detailed motion characteristics without overwhelming system complexity.
Solution Approach 2:
The patent introduces a scale dimension to the skeleton representation, transforming the traditional single-scale approach into a multi-scale framework. By adding this dimensional aspect, the system can capture actions at multiple granularities, improving recognition accuracy while maintaining manageable complexity through structured processing.
2Speed
If typical skeleton-based graph convolutional networks are used, then the processing speed is improved, but information bypass occurs affecting recognition accuracy
Solution Approach 1:
The patent introduces intermediate processing layers that act as mediators between different skeleton scales and the graph convolutional network. These intermediaries ensure that information from both coarse and fine granularities is properly integrated and transmitted through the network, preventing information bypass while maintaining processing efficiency.
Solution Approach 2:
The patent merges feature representations from multiple skeleton scales before feeding them into the graph convolutional network. This combining approach ensures that comprehensive action information is processed together, preventing information loss while maintaining the speed advantages of convolutional networks through efficient feature fusion.
3Measurement precision
If multi-granularity action recognition is implemented, then the recognition accuracy is improved, but the computational complexity increases
Solution Approach 1:
The patent implements dynamic processing where the level of granularity is adaptively adjusted based on the action being recognized. For simple actions, coarser granularity suffices, reducing computational load. For complex actions, finer granularity is engaged, improving accuracy without unnecessarily increasing complexity for all cases.
Solution Approach 2:
The patent applies different processing qualities to different parts of the skeleton based on their importance to specific actions. Critical joint points receive finer granularity analysis while less critical points use coarser representation, optimizing the balance between recognition accuracy and computational complexity through localized processing quality adjustment.
Data Source
AI summary
A system, method, and computer program product for implementing video action recognition is provided. The method includes receiving a video stream comprising user movement actions. Skeleton points associated with a video representation of a user executing the user movement actions are extracted and categorized with respect to multiple digital levels. Initial visual windows points are generated within video frames and an average movement distance for the group of skeleton points are determined with respect to the video frames. In response, sizes for the visual windows are adjusted and feature vectors are extracted from the group of skeleton points. Point coordinates of the skeleton points are extracted and linked with the feature vectors. A convolutional neural network associated with linking the feature vectors with the point coordinates is generated and the video stream is enabled with respect to video action recognition associated with accurate presentation of the video stream.