Mixed Reality Gaze and Gesture Control for Handle-Free Calibration

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Existing mixed reality devices require the use of handles for interaction, which can be inconvenient and limit the user experience during calibration processes.

Innovation Solution

The method employs an electronic device with a display generation component, an eye tracking device, and a gesture detection device, allowing users to control virtual objects in a mixed reality environment by tracking their gaze and detecting hand gestures, eliminating the need for physical handles.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Ease of operation

If physical handles are used for interaction in mixed reality devices, then device complexity is reduced and ease of operation is improved, but user experience is limited and calibration efficiency is reduced

Engineering Contradiction:

Improveease of operationVSAvoidcalibration efficiency

Core Design Contradiction:

Ease of operationVSProductivity

Solution Approach 1:

The patent replaces mechanical handle-based interaction with an optical tracking system using eye tracking devices and gesture detection. The eye tracking device captures eye movement data to determine gaze direction, while the gesture detection device detects hand movements, substituting the mechanical handle with a non-contact optical and motion sensing system that improves both ease of operation and calibration efficiency.

Inventive Principle:

Principle #28Mechanics substitution (Replace mechanical system)

2Ease of operation

If physical handles are required for interaction, then device complexity is minimized, but user experience and interaction intuitiveness deteriorate

Engineering Contradiction:

Improveinteraction intuitivenessVSAvoiddevice complexity

Core Design Contradiction:

Ease of operationVSDevice complexity

Solution Approach 1:

The patent implements a multi-functional input system where the eye tracking device serves multiple purposes: detecting gaze direction for object selection, tracking user attention, and enabling hands-free operation. The gesture detection device complements this by detecting hand movements for additional control capabilities. This universal approach to input detection improves interaction intuitiveness while the modular architecture manages device complexity.

Inventive Principle:

Principle #6Universality (Multi-functionality)

Solution Approach 2:

The patent introduces visual indication objects as intermediaries between the user's gaze and the target objects in the mixed reality environment. These visual indicators provide clear feedback about which object is currently selected or being targeted, mediating the interaction between the eye tracking system and the calibration process, thereby improving interaction intuitiveness.

Inventive Principle:

Principle #24Intermediary (Mediator)

3Ease of operation

If handles are used for calibration processes, then ease of operation is maintained, but user experience and convenience deteriorate

Engineering Contradiction:

Improveease of operationVSAvoidcalibration convenience

Core Design Contradiction:

Ease of operationVSProductivity

Solution Approach 1:

The patent enables hands-free operation through eye tracking technology, allowing users to perform calibration tasks without physical handles. The eye tracking device automatically tracks gaze direction and the system autonomously determines which object is being targeted, reducing the need for manual handle manipulation and significantly improving calibration convenience and user experience.

Inventive Principle:

Principle #25Self-service

Applied Scientific Principles

This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.

Function Achieved in This Case

This approach enhances user experience by providing a more intuitive and handle-free interaction method, improving the efficiency and convenience of calibrating objects in mixed reality environments.

Implementation Method 1

detecting, through the eye tracking device, that a gaze of the user points to a target second object of the second objects

Methodology Applied

Scientific EffectEye tracking:

Implementation Method 2

detecting a hand gesture of the user through the gesture detection device

Methodology Applied

Scientific EffectGesture detection:

Data Source

PatentUS20250138644A1Method and apparatus for controlling electronic device, electronic device, and storage medium

Publication Date: 2025.05.01 BEIJING ZITIAO NETWORK TECH CO LTD

AI summary

A method includes: displaying a physical environment of a user through the display generation component; generating at least one second object on a first object in the physical environment in response to a preset processing flow; detecting, through the eye tracking device, that a gaze of the user points to a target second object of the second objects to generate a visual indication object; detecting a hand gesture of the user through the gesture detection device, and controlling the target second object based on the hand gesture and the visual indication object; and determining attribute information of the target second object in a case that a demand of the user is satisfied by adjusting a fitting degree between the target second object and the first object based on the hand gesture.