Hand Gesture Recognition Using Virtual Gesture-Space ROI
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing gesture recognition systems face challenges in complex environments, such as cluttered backgrounds, long distances, low-light conditions, and multiple users, leading to high computational costs and inaccurate detection and recognition of hand gestures.
Innovation Solution
An end-to-end system utilizing a virtual gesture-space defined around a primary user, combined with a trained joint neural network, to detect and track hands, and perform classification, while reducing processing demands through adaptive ROI and image adjustment techniques.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Difficulty of detecting and measuring
If gesture segmentation and recognition is performed on a sequence of digital video frames, then gesture detection capability is provided, but computational cost increases significantly while accuracy remains relatively low
Solution Approach 1:
The patent segments the gesture detection task into two independent parts: hand detection (using a detector that outputs bounding boxes) and gesture classification (using a neural network that processes only the detected hand regions). This segmentation allows each component to be optimized independently, reducing overall computational cost while maintaining detection capability.
Solution Approach 2:
The patent extracts only the relevant hand region from each video frame using the hand detector's bounding box output, and feeds only this extracted region to the gesture classification neural network. This extraction eliminates the need to process the entire video frame sequence through the neural network, significantly reducing computational cost while preserving gesture recognition accuracy.
2Adaptability or versatility
If gesture detection is performed on a sequence of video frames in complex real-life environments, then detection coverage is expanded, but false positives and false negatives increase
Solution Approach 1:
The patent introduces an intermediary hand detector that acts as a mediator between the video input and the gesture classification neural network. This detector first identifies potential hand regions and provides bounding boxes, which then serve as precise inputs to the neural network. This intermediary step filters out false detections from complex backgrounds before they reach the classification stage, reducing false positives while maintaining broad detection coverage.
3Measurement precision
If high resolution frames are used to detect smaller hands at farther distances, then detection accuracy for distant hands improves, but computational cost increases significantly
Solution Approach 1:
The patent applies partial action by processing only the hand region within the bounding box rather than the entire high-resolution frame. The hand detector first identifies the relevant region, and then only this cropped region is fed to the neural network for classification. This partial processing approach maintains detection accuracy for distant hands while significantly reducing the computational cost compared to processing the full high-resolution frame.
4Device complexity
If a bounding box from hand detection is used as input to gesture classification, then detection flow is simplified, but the bounding box size may not match the expected input size for the classifier
Solution Approach 1:
The patent applies parameter changes by using the hand detector's bounding box output (which may have varying dimensions) as input to the gesture classification neural network. The neural network is designed to accept this variable-sized input and process it accordingly, or the bounding box is adjusted to match expected input dimensions. This parameter adaptation maintains simplified detection flow while ensuring input compatibility with the classifier.
Data Source
Figure 1
Figure 2
Figure 3A~3H
AI summary
Methods and systems for gesture-based control of a device are described. A virtual gesture-space is determined in a received input frame. The virtual gesture-space is associated with a primary user from a ranked user list of users. The received input frame is processed in only the virtual gesture-space, to detect and track a hand. Using a hand bounding box generated by detecting and tracking the hand, gesture classification is performed to determine a gesture input associated with the hand. A command input associated with the determined gesture input is processed. The device may be a smart television, a smart phone, a tablet, etc.