Deep Embedded Action Classification for Robust Gesture Control
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current unsupervised action recognition techniques face challenges in distinguishing between actions due to reliance on visual appearance and pose space clustering, leading to poor performance and vulnerability to noise and occlusion, especially in vision-based systems.
Innovation Solution
The system employs deep embedded clustering to map skeletal joint-based pose features into an embedded action space using a recurrent neural network framework with long short-term memory (LSTM) for temporal classification, enhancing action recognition by encoding first- and higher-order dependencies between joint positions and applying Part Affinity Fields for robust pose estimation.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If clustering techniques are applied in image space or pose space for action recognition, then action classification can be performed, but the system suffers from poor performance and vulnerability to noise and occlusion
Solution Approach 1:
The patent transforms the action recognition problem from traditional image space or pose space clustering into a deep embedded feature space. By applying non-linear dimensionality transformation through deep neural networks, the system maps high-dimensional pose features into a lower-dimensional embedded space where action patterns are more separable and robust to noise and occlusion.
Solution Approach 2:
The patent introduces deep embedded clustering as an intermediary layer between raw pose features and action classification. This intermediary transformation layer learns discriminative features that are invariant to noise and occlusion, effectively mediating between the noisy input data and the classification task.
2Measurement precision
If visual appearance features are used for action recognition, then action classification can be performed, but the system achieves poor discrimination between similar actions
Solution Approach 1:
The patent changes the feature representation parameters from traditional visual appearance features to deep embedded pose features. By transforming the parameter space through non-linear mapping and applying deep embedded clustering, the system achieves better action discrimination while maintaining reliability.
3Adaptability or versatility
If traditional clustering methods are used in pose space, then action recognition can be performed, but the system fails to capture temporal dependencies and higher-order joint relationships
Solution Approach 1:
The patent introduces dynamic temporal modeling by applying Long Short-Term Memory (LSTM) networks to the embedded pose features. This allows the system to capture temporal dependencies in action sequences, making the recognition system adaptable to varying action durations and speeds while improving precision through temporal context.
Solution Approach 2:
The patent combines multiple feature types (joint positions, velocities, accelerations) and multiple modeling approaches (deep embedded clustering, LSTM temporal modeling) into a composite action recognition system. This composite approach captures both spatial relationships and temporal dependencies, achieving superior action pattern distinguishability.
Data Source
AI summary
Described is a system for action recognition through application of deep embedded clustering. For each image frame of an input video, the system computes skeletal joint-based pose features representing an action of a human in the image frame. Non-linear mapping of the pose features into an embedded action space is performed. Temporal classification of the action is performed and a set of categorical gesture-based labels is obtained. The set of categorical gesture-based labels is used to control movement of a machine.