3D Gesture Recognition via Indexable Feature Matching
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current gesture recognition technologies face challenges in efficiently recognizing 3D hand gestures due to the high degrees of freedom of human hand kinematics, requiring extensive computational resources and struggling with cluttered backgrounds and self-occlusion, limiting their ability to handle complex gestures in real-time.
Innovation Solution
A method and device for 3D gesture recognition using a large-scale gesture database with an efficient indexing algorithm, enabling fast retrieval of high-dimensional hand motion data, even in noisy conditions, by capturing and processing query images to match them against a vast database of annotated gesture entries, facilitating real-time tracking and recognition.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If 3D hand model based approaches are used to recognize complex hand gestures, then gesture recognition accuracy is improved, but computational complexity and time consumption increase significantly
Solution Approach 1:
The patent segments the complex 27 DoF hand model into multiple independent DoF groups (e.g., wrist rotation, finger movements) that can be processed separately. This segmentation allows the system to handle complex gestures by breaking them down into manageable computational units, reducing overall computational complexity while maintaining recognition accuracy
Solution Approach 2:
The patent transforms the high-dimensional 27 DoF search space into a lower-dimensional representation by projecting hand poses onto 2D image planes and using viewpoint information as an additional dimension. This dimensional transformation reduces the search space from 27 dimensions to a manageable subset, enabling real-time processing of complex gestures
2Adaptability or versatility
If a very large database of hand images is created to cover all characteristic hand images under different views, then gesture recognition coverage is improved, but matching time and computational cost increase
Solution Approach 1:
The patent pre-computes and stores only the essential viewpoint parameters and projection matrices for common camera angles in the database, rather than storing complete 3D hand models for all possible views. This preliminary action reduces database size while maintaining the ability to synthesize novel viewpoints during matching, decreasing search time without sacrificing coverage
Solution Approach 2:
The patent changes the database storage format from storing complete high-resolution 3D hand images for all viewpoints to storing compact parameter sets (viewpoint angles, projection matrices, key landmark positions). This parameter transformation enables rapid database searching and real-time synthesis of hand appearances under different views, reducing both storage requirements and matching time
3Productivity
If appearance based approaches are used for real-time gesture tracking, then real-time performance is improved, but the ability to handle complex hand gestures is limited
Solution Approach 1:
The patent introduces a 3D hand model as an intermediary between the 2D image features and the gesture recognition system. The 3D model serves as a mediator that generates synthetic appearance projections matching the observed 2D features, enabling the system to interpret complex 3D hand gestures from simple 2D image data while maintaining real-time performance
Solution Approach 2:
The patent creates simplified 2D projection copies of the 3D hand model that can be rapidly compared with observed hand images. These copied projections serve as templates for matching, allowing the system to evaluate complex gesture hypotheses in real-time by comparing compact 2D representations rather than processing full 3D models
Data Source
Figure 1
Figure 2A
Figure 2B
AI summary
The disclosure relates to a device and a method in the device for recognizing a 3D gesture. The device is connected to a sensor and has access to a database of gesture images comprising indexable features of normalized gesture images. The indexable features comprise a position and an orientation for each pixel of edge images of the normalized gesture images. The method comprises capturing (110) an image of the 3D gesture via the sensor, normalizing (120) the captured image, deriving (130) indexable features from the normalized captured image, and comparing (140) the derived indexable features with the indexable features of the database using a similarity function. The method also comprises determining (150) a gesture image in the database matching the 3D gesture based on the comparison.