Pinch Gap Occlusion Gating for Reliable XR Gesture Input
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing hand gesture recognition systems in extended reality (XR) often mistakenly activate input actions when a user's hand is holding an object, due to the hand's pose resembling an intended input gesture, particularly with palm-up or palm-down positions.
Innovation Solution
The system analyzes the relationship between the thumb and index finger to determine the visibility of a pinch gap, using this as a key indicator to differentiate between intentional and unintentional gestures, and employs occlusion characteristics to predict whether the hand is self-occluded or occluded by an object, thereby filtering out unintentional inputs without requiring additional object detection.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If hand gesture recognition systems use hand pose to detect input gestures, then gesture-based interaction is enabled, but unintentional gestures are mistakenly activated when holding objects
Solution Approach 1:
The system segments the hand into multiple regions (palm, fingers, thumb) and analyzes the pose of each segment independently. By examining the specific configuration of the thumb and index finger relative to other hand segments, the system can distinguish between intentional pinch gestures and unintentional poses while holding objects.
Solution Approach 2:
The system applies different analysis criteria to different parts of the hand. Specifically, it examines the local geometry of the thumb and index finger relationship, looking at the angle and position of these segments to determine whether they form a genuine pinch gesture or are merely positioned that way while holding an object.
2Measurement precision
If the system analyzes thumb and index finger relationship to detect pinch gestures, then gesture recognition is improved, but the complexity of gesture analysis increases
Solution Approach 1:
The system extracts and focuses analysis on the most discriminative features - the relationship between the thumb and index finger - rather than analyzing the entire hand pose. By isolating this specific geometric relationship, the system achieves high recognition precision without needing to process all hand segments, reducing overall computational complexity.
Solution Approach 2:
The system performs a partial analysis by focusing only on the critical thumb-index finger geometry rather than a complete hand pose analysis. This selective approach provides sufficient precision for gesture recognition while significantly reducing the computational burden compared to analyzing all hand segments and their relationships.
Data Source
AI summary
Enabling gesture recognition and input based on hand tracking data and occlusion information is described. Hand tracking data is obtained of a hand performing an input gesture while the hand is in a first interface state. The technique includes determining pinch gap characteristics and occlusion characteristics of the index finger. A hand is determined to either be in an object-occlusion detection state or an object-occlusion un-detection state based on the occlusion characteristics of the index finger and the first interface state. A gesture signal is adjusted to affect an action corresponding to the input gesture based on whether the hand is determined to be in the object-occlusion detection state or the object-occlusion un-detection state.