Action Recognition Using Depth Maps and Skeleton Joints
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current methods for recognizing activities of entities using depth maps face challenges such as noisy data, temporal misalignment, and the inability to distinguish between similar actions, particularly due to reliance on 3D skeleton motion alone.
Innovation Solution
The system employs a depth map acquisition component to obtain depth maps, a skeleton acquisition component to track joint positions, and a local feature descriptor determination component to analyze local occupancy patterns and Fourier Temporal Pyramids, which are used to represent temporal structures and characterize actions robustly, incorporating multiple kernel learning for actionlet ensemble modeling.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Device complexity
If 3D skeleton motion alone is used for action recognition, then the system is simple to implement, but it cannot distinguish between similar actions and has low accuracy
Solution Approach 1:
The patent combines multiple data sources (depth maps, skeleton representations, local feature descriptors) to create a comprehensive action recognition system. The activity determination component integrates these multiple feature types to achieve accurate distinction between similar actions while maintaining reasonable system complexity through unified processing architecture.
Solution Approach 2:
The patent transitions from 2D image data to 3D depth map data, adding the depth dimension to enable better understanding of spatial relationships and action semantics. This dimensional enhancement allows the system to distinguish between similar actions that appear identical in 2D projections but have different three-dimensional characteristics.
2Measurement precision
If depth maps are used for action recognition, then 3D spatial information is captured, but noise and temporal misalignment occur
Solution Approach 1:
The patent segments the action recognition task into multiple independent components: depth map acquisition, skeleton representation extraction, local feature descriptor computation, and activity determination. Each component processes specific features separately, allowing noise in one component to be compensated by others and enabling robust temporal alignment through the skeleton tracking module.
Solution Approach 2:
The skeleton representation acts as an intermediary that bridges depth map data and action recognition. It extracts key pose information from depth maps, providing a normalized representation that is less susceptible to noise and temporal misalignment, while preserving essential spatial relationships needed for accurate action distinction.
3Measurement precision
If multiple feature types are integrated for action recognition, then accuracy is improved, but processing time and computational complexity increase
Solution Approach 1:
The patent performs preliminary extraction and preprocessing of features (depth maps, skeleton representations, local feature descriptors) before the final activity determination. By preparing and organizing these features in advance through structured acquisition components, the system reduces computational burden during the critical decision-making phase, balancing accuracy with processing efficiency.
Data Source
AI summary
A plurality of depth maps corresponding to respective depth measurements determined over a respective plurality of time frames may be obtained. A plurality of skeleton representations respectively corresponding to the respective time frames may be obtained. Each skeleton representation may include joints associated with an observed entity. Local feature descriptors corresponding to the respective time frames may be determined, based on the depth maps and the joints associated with the skeleton representations. An activity recognition associated with the observed entity may be determined, based on the obtained skeleton representations and the determined local feature descriptors.