Face-Shoulder Coordinate Transform for Hand Gesture Recognition
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current hand tracking and gesture recognition systems face challenges in efficiently reducing the computational burden and sample collection costs due to person-to-person variations and the need for large datasets, especially when hands are occluded or rotated relative to the camera.
Innovation Solution
The system detects dominant face and shoulder features to establish a face-shoulder coordinate system, transforming hand images into this system, which reduces the feature space dimensions and enables more robust and accurate hand gesture recognition using traditional or deep neural network methods, even when hands are occluded or rotated.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If vision-based hand gesture recognition is used, then natural interaction is achieved, but computational burden increases due to model-based methods requiring 28 degrees of freedom
Solution Approach 1:
The patent extracts and removes unnecessary complexity by eliminating the need for full 3D hand models with 28 degrees of freedom. Instead, it directly processes hand appearance features from images using statistical classifiers, taking out the complex model-based optimization and replacing it with simpler appearance-based feature extraction and classification.
Solution Approach 2:
The patent changes the parameter representation from complex 3D model parameters (28 DOF) to simpler 2D image-based appearance features. This parameter transformation reduces computational burden while maintaining recognition capability, as the system works directly with image pixel data and extracted features rather than complex spatial model parameters.
2Device complexity
If appearance-based methods are used for hand gesture recognition, then computational burden is reduced, but accuracy decreases due to person-to-person variations in hand appearances
Solution Approach 1:
The patent applies preliminary action by performing coordinate transformation and normalization on hand images before classification. It transforms hand images into a standardized coordinate system and applies geometric normalization to account for position, scale, and rotation variations, preparing the data in advance to reduce the impact of person-to-person variations during the actual recognition process.
3Measurement precision
If large amounts of samples with ground truth are collected for training, then gesture recognition accuracy improves, but sample collection and annotation costs increase significantly
Solution Approach 1:
The patent performs preliminary geometric normalization and coordinate transformation on hand images, which standardizes the input data and reduces variability. This preliminary processing makes the classification task easier and more robust, allowing for accurate gesture recognition with fewer training samples, thereby reducing the cost and effort of sample collection and annotation.
4Reliability
If hands are occluded or rotated relative to the camera, then gesture recognition becomes more difficult, but the system should maintain robustness
Solution Approach 1:
The patent applies preliminary geometric normalization that includes correcting for rotation and scaling effects. By transforming hand images into a standardized coordinate system and normalizing their geometric properties before classification, the system compensates for rotation and occlusion effects in advance, making the recognition process more robust without requiring complex models to handle these variations.
Data Source
AI summary
A method for recognizing hand gestures when the hand is held in front of the user's upper body. The method detects dominant features associated with the upper body, including features on the face (eyes and mouth etc.) and the shoulder extremes. Geometrical relations among these features are used to judge whether the face/head has turned to the side relative to the shoulder. If not, a face-shoulder coordinate system is established using these face-shoulder features, and the hand images are transformed into this new coordinate system. This in effect defines hand gestures relative to the user's face and shoulder, thus significantly reducing the dimensions of the feature space for hand gestures. This face-shoulder coordinate transform can be combined either with traditional appearance-based hand gesture recognition methods, or with a conventional or hybrid deep neural network for RGB or RGB-D images.