Hand Gesture Input Gating Using Self-Occlusion Detection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing hand gesture recognition systems in extended reality (XR) often inaccurately detect input gestures when a user's hand is holding an object, leading to unintentional activation of user interface components or actions.
Innovation Solution
Distinguish between self-occlusion and object occlusion of hand joints by using occlusion scores and hand pose geometry to determine the intentionality of hand gestures, thereby suppressing unintentional input actions.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If hand gesture recognition is performed using basic hand pose detection, then gesture input capability is provided, but unintentional gestures are misdetected when hand is occluded by objects
Solution Approach 1:
The patent segments the hand tracking data into multiple components: joint locations, occlusion scores for each joint, and bone segment information. By dividing the hand model into individual bones and joints, the system can analyze occlusion patterns at each segment level to determine whether occlusion is caused by self-hand or external objects, thereby improving gesture detection accuracy while maintaining reliability.
Solution Approach 2:
The patent introduces an intermediary analysis layer that processes occlusion scores and hand pose geometry between the raw sensor data and gesture recognition. This intermediary layer calculates whether each occluded joint is self-occluded or object-occluded by analyzing the spatial relationships between bone segments, serving as a mediator that filters out false positive gestures caused by object occlusion.
2Reliability
If occlusion detection is added to distinguish self-occlusion from object occlusion, then unintentional gesture detection improves, but system complexity increases
Solution Approach 1:
The patent extracts only the necessary occlusion information from the sensor data - specifically, occlusion scores for each joint and basic hand pose geometry. By taking out only the essential occlusion metrics rather than processing complete image data or all possible hand parameters, the system achieves reliable intentional gesture recognition while keeping computational complexity manageable.
Solution Approach 2:
The patent applies local quality analysis by evaluating occlusion characteristics at each individual joint level rather than treating the hand as a single unit. Each joint's occlusion score and self-occlusion status is determined independently based on local bone segment geometry, allowing the system to achieve high reliability in gesture recognition without requiring complex global analysis of the entire hand structure.
Data Source
AI summary
Enabling gesture recognition and input based on hand tracking data and occlusion information is described. A determination is made as to whether a hand or a portion of a hand is occluded by a physical object or by the hand itself, and filters and consolidate the occlusion scores for each portion of the hand to determine whether to invoke or dismiss an input action associated with an input gesture. In doing so, hand tracking data can be used to obtain occlusion data and pose data from which input gesture invocation and gating can be implemented.