3D Skeleton Motion Identification via 2D-Depth Fusion
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current human action recognition methods using 2D skeleton point information face challenges in distinguishing human motions due to overlapping projections, and high-resolution 3D sensors require significant resources and time for calculation, while low-resolution sensors lead to inaccurate motion identification.
Innovation Solution
A method and system that capture 2D color or infrared images and corresponding depth images, extract and correct 2D human skeleton point information using size-depth parameters and distortion models, and combine this information to obtain accurate 3D skeleton points for real-time motion identification using a match model.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If high-resolution 3D sensor is used to obtain accurate 3D point cloud information, then motion identification accuracy is improved, but calculation time and resource consumption increase significantly
Solution Approach 1:
The system segments the 3D reconstruction task by separately processing 2D images and depth maps, then combining them through coordinate transformation. This divides the complex high-resolution 3D point cloud generation into manageable stages, reducing computational burden at each step while maintaining accuracy.
Solution Approach 2:
The patent introduces depth maps as an intermediary element between 2D images and 3D point cloud representation. By using depth information to weight and reconstruct 3D points from 2D coordinates, the system achieves accurate motion identification without directly processing full high-resolution 3D point clouds, thus reducing calculation time.
2Productivity
If low-resolution 3D sensor is used to reduce calculation time, then resource consumption is reduced, but measurement accuracy decreases due to background noise
Solution Approach 1:
The system merges 2D image information with depth map information to create enhanced 3D skeleton points. By combining these two data sources, the system compensates for the limitations of low-resolution depth data, maintaining measurement precision while using computationally efficient low-resolution sensors.
Solution Approach 2:
The patent changes the parameter representation by introducing depth weights to the 3D point reconstruction process. Instead of relying solely on resolution, the system adjusts the weighting parameters based on depth information, allowing accurate skeleton point recognition even with low-resolution input data.
3Speed
If 2D skeleton point information is used for human action recognition, then processing speed is improved, but motion discrimination capability deteriorates due to overlapping projections
Solution Approach 1:
The patent transitions from 2D skeleton points to 3D skeleton points by incorporating depth information. This dimensional change adds the Z-axis coordinate information, enabling the system to distinguish motions that overlap in 2D projection while maintaining the processing efficiency of 2D-based methods through selective 3D reconstruction.
Data Source
AI summary
The present invention provides an action recognition method and system thereof. The action recognition method comprises: capturing a 2D image and a depth image at the same time, extracting an 2D information of the human skeleton points from the 2D image and correcting it, mapping the 2D information of the human skeleton points to the depth image to obtain the corresponding depth information with respect to the 2D information of the human skeleton points and combining the corrected 2D information of the human skeleton points and the depth information to obtain the 3D information of the human skeleton points, and finally recognizing an action from a set of 3D information of the human skeleton points during a period of time by a matching model.