2D Skeleton Activity Classification Without Specialized 3D Hardware
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for classifying human activities from skeleton data are limited by the need for 3D data, which is difficult to extract and requires specialized hardware, and lack contextual information, making them computationally intensive and less portable across domains.
Innovation Solution
A system using 2D skeleton data processed by a skeleton preprocessor to transform joint positions and velocities, combined with gesture and action classifiers based on recurrent neural networks (RNNs) to identify gestures and actions, leveraging contextual object information.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If 3D skeleton data is used for activity classification, then measurement precision is improved, but device complexity and ease of manufacture worsen due to requiring specialized hardware
Solution Approach 1:
The patent uses 2D skeleton data as a simplified copy or representation of 3D skeleton data. Instead of requiring complex 3D sensing hardware, the system processes 2D skeletal information from standard RGB cameras, maintaining sufficient accuracy for activity classification while dramatically reducing hardware complexity and cost.
2Measurement precision
If 3D skeleton data is used for activity classification, then measurement precision is improved, but ease of manufacture worsens
Solution Approach 1:
The system replaces complex 3D sensing infrastructure with standard RGB cameras that capture 2D skeleton data. This copying approach using readily available hardware significantly improves ease of manufacture and deployment while maintaining adequate measurement precision for the classification task.
3Measurement precision
If contextual information is added to skeleton data, then activity classification accuracy is improved, but loss of time increases due to additional processing
Solution Approach 1:
The system performs preliminary processing of skeleton data by precomputing and storing contextual information such as joint velocities and relative positions during data collection or preprocessing phases. This allows the main classification algorithm to access enriched skeleton data without performing computationally intensive calculations in real-time, thus improving accuracy while minimizing processing time delays.
Data Source
AI summary
An activity classifier system and method that classifies human activities using 2D skeleton data. The system includes a skeleton preprocessor that transforms the 2D skeleton data into transformed skeleton data, the transformed skeleton data comprising scaled, relative joint positions and relative joint velocities. The system also includes a gesture classifier comprising a first recurrent neural network that receives the transformed skeleton data, and is trained to identify the most probable of a plurality of gestures. The system also has an action classifier comprising a second recurrent neural network that receives information from the first recurrent neural networks and is trained to identify the most probable of a plurality of actions.