2D Skeleton Activity Classification with Contextual RNNs
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for classifying human activities from skeleton data are not robust and require 3D data, which is challenging to extract and computationally intensive, and lack contextual information, making them difficult to train and apply in real-world scenarios.
Innovation Solution
A system using 2D skeleton data processed by a skeleton preprocessor to transform joint positions and velocities, combined with a gesture classifier and action classifier based on recurrent neural networks (RNNs) to identify gestures and actions, leveraging contextual information for improved classification.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If 3D skeleton data is used for activity classification, then measurement precision is improved, but device complexity and computational requirements increase significantly
Solution Approach 1:
The patent uses 2D skeleton data as a simplified copy or representation of 3D skeleton data. Instead of requiring complex 3D depth sensors and processing, the system processes 2D skeleton information from standard RGB cameras, maintaining sufficient accuracy for activity classification while dramatically reducing hardware complexity and computational requirements.
2Productivity
If skeleton data is processed without contextual information, then processing speed is improved, but classification reliability deteriorates
Solution Approach 1:
The system performs preliminary processing of skeleton data to extract and integrate contextual information such as spatial relationships between joints, temporal patterns of movement, and environmental context before classification. This pre-processing enriches the input data with contextual cues that improve classification reliability while maintaining efficient processing through optimized feature extraction pipelines.
3Device complexity
If simple classifiers like SVM or HMM are used, then device complexity is reduced, but adaptability to real-world variations deteriorates
Solution Approach 1:
The patent employs deep neural networks with learnable parameters that automatically adapt to various real-world conditions including different lighting, camera angles, clothing, and human body types. The model parameters are trained on diverse datasets to capture variations in human appearance and behavior, enabling the system to maintain high adaptability while using standard RGB cameras and 2D skeleton data rather than complex specialized hardware.
Data Source
AI summary
This disclosure is directed to an activity classifier system, for classifying human activities using 2D skeleton data. The system includes a skeleton preprocessor that transforms the 2D skeleton data into transformed skeleton data, the transformed skeleton data comprising scaled, relative joint positions and relative joint velocities. It also includes a gesture classifier comprising a first recurrent neural network that receives the transformed skeleton data, and is trained to identify the most probable of a plurality of gestures. There is also an action classifier comprising a second recurrent neural network that receives information from the first recurrent neural networks and is trained to identify the most probable of a plurality of actions.