XR Gesture State Machine Using Gaze to Reject Unintentional Input

Resolve Bottlenecks,
Find Innovative Solutions
Generate Solutions

Solution Overview

Problem

Existing techniques for determining hand poses in Extended Reality (XR) environments are resource-intensive and lack accuracy in distinguishing intentional gestures from unintentional ones.

Innovation Solution

A method that utilizes standard joint position tracking and gaze information to determine hand poses, without requiring specialized computer vision algorithms, by using spatial relationships between the hand and head, and applying gaze criteria to refine gesture detection.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Measurement precision

If specialized computer vision algorithms are used to determine hand poses, then measurement precision is improved, but device complexity and resource consumption increase

Engineering Contradiction:
Improvehand pose detection accuracyVSAvoidsystem complexity
Core Design Contradiction:
Measurement precisionVSDevice complexity

Solution Approach 1:

The patent extracts only the necessary components for hand pose detection: standard joint position tracking and gaze information. By removing specialized computer vision algorithms and focusing on spatial relationships between hand and head, the system achieves sufficient accuracy without the complexity of advanced vision systems.

Inventive Principle:
Principle #2Taking out (Extraction)

Solution Approach 2:

The patent uses universally available tracking data (joint positions and gaze information) that can be obtained from standard sensors. This multi-functional approach allows the same data to serve both pose detection and gesture intention determination, eliminating the need for specialized algorithms.

Inventive Principle:
Principle #6Universality (Multi-functionality)

2Reliability

If comprehensive gesture detection criteria are applied, then reliability is improved, but processing time increases

Engineering Contradiction:
Improvegesture detection accuracyVSAvoidprocessing time
Core Design Contradiction:
ReliabilityVSLoss of time

Solution Approach 1:

The patent segments gesture detection into distinct criteria: hand pose determination and gaze criterion satisfaction. Each criterion can be evaluated independently and in parallel, reducing processing time while maintaining comprehensive reliability through multiple checks.

Inventive Principle:
Principle #1Segmentation

Solution Approach 2:

The system preliminarily determines hand pose using simple spatial relationships before applying the gaze criterion. This staged approach allows quick rejection of obvious non-gestures while reserving more complex analysis for cases where the hand pose is ambiguous or potentially intentional.

Inventive Principle:
Principle #10Preliminary action

Data Source

PatentUS20250355502A1State Machine and Rejection Criterion for UI Gesture Invocation
Publication Date: 2025.11.20 APPLE INC
  • US20250355502A1 patent drawing
  • US20250355502A1 patent drawing
  • US20250355502A1 patent drawing

AI summary

Input gestures having a particular palm orientation are detected based on geometric characteristics of a hand relative to a head. Gaze information is used to determine a hand gesture state. The gesture state refers to a palm-up gesture or a palm-flip gesture. A hand orientation state machine is used to determine a hand orientation state based on the geometric characteristics. A gesture detection state machine is used to determine a hand gesture based on a hand orientation state and the gaze vector. An action is invoked based on the hand gesture state.