Virtual Object Arrangement with Eye-Hand Tracking for Faster Interaction

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Existing methods for interacting with augmented and virtual reality environments are cumbersome, inefficient, and create a significant cognitive burden on users, often requiring multiple inputs and leading to energy wastage, particularly in battery-operated devices.

Innovation Solution

The system employs computer systems with improved user interfaces that reduce the number and complexity of user inputs by providing intuitive interactions through touch-sensitive displays, eye-tracking, hand-tracking, and spatial arrangement of virtual objects, enhancing efficiency and reducing power consumption.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Productivity

If conventional user interfaces are used in augmented reality environments, then basic interaction functionality is provided, but user interaction efficiency deteriorates and cognitive burden increases due to cumbersome and complex manipulation requirements

Engineering Contradiction:

Improveuser interaction efficiencyVSAvoidease of interaction

Core Design Contradiction:

ProductivityVSEase of operation

Solution Approach 1:

The system automatically determines user intent by analyzing eye gaze direction and hand pose configurations, eliminating the need for users to explicitly select or confirm virtual objects. The interface serves itself by interpreting natural human behaviors directly as interaction commands, thereby improving efficiency while maintaining ease of operation.

Inventive Principle:

Principle #25Self-service

Solution Approach 2:

The patent replaces traditional mechanical input devices (buttons, switches, explicit selection mechanisms) with optical (eye-tracking) and positional (hand-tracking) sensing systems. This substitution enables more efficient and intuitive interaction by capturing natural human behaviors rather than requiring users to manipulate physical or virtual controls.

Inventive Principle:

Principle #28Mechanics substitution (Replace mechanical system)

2Reliability

If multiple input steps are required to achieve desired outcomes in augmented reality, then precise control is achieved, but interaction time increases and energy consumption rises

Engineering Contradiction:

Improvecontrol precisionVSAvoidinteraction time

Core Design Contradiction:

ReliabilityVSLoss of time

Solution Approach 1:

The system performs preliminary analysis of eye gaze and hand pose data to predict user intent before an explicit selection or activation occurs. By pre-processing and interpreting multiple input modalities simultaneously, the system reduces the number of sequential steps required while maintaining reliable control through multi-factor verification.

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

The patent combines eye-tracking and hand-tracking data into a unified interaction framework, merging multiple input modalities into a single coherent intent determination process. This integration allows the system to achieve precise control through combined signals while reducing overall interaction time compared to sequential processing of each modality separately.

Inventive Principle:

Principle #5Merging (Combining)

3Measurement precision

If complex manipulation sequences are used for virtual objects, then precise positioning and control are achieved, but user cognitive burden increases and error probability rises

Engineering Contradiction:

Improvevirtual object positioning precisionVSAvoidinterface complexity

Core Design Contradiction:

Measurement precisionVSDevice complexity

Solution Approach 1:

The system continuously monitors eye gaze direction and hand pose configurations, providing real-time feedback by highlighting potential target virtual objects based on detected user intent. This feedback mechanism guides users through the interaction process, maintaining precise positioning capability while reducing cognitive burden through intuitive visual cues that confirm system understanding of user goals.

Inventive Principle:

Principle #23Feedback

4Adaptability or versatility

If traditional input methods are used in battery-operated augmented reality devices, then comprehensive interaction control is provided, but power consumption increases due to prolonged interaction times

Engineering Contradiction:

Improveinteraction control capabilityVSAvoidbattery power consumption

Core Design Contradiction:

Adaptability or versatilityVSUse of energy by moving object

Solution Approach 1:

The patent replaces power-intensive traditional input methods (touchscreen interactions, physical controller manipulation) with passive sensing technologies that capture natural eye movements and hand positions. These optical and positional tracking systems consume less power while enabling comprehensive interaction control, thereby extending battery life without sacrificing adaptability.

Inventive Principle:

Principle #28Mechanics substitution (Replace mechanical system)

Data Source

PatentUS20250278166A1Devices, methods, and graphical user interfaces for presenting virtual objects in virtual environments

Publication Date: 2025.09.04 APPLE INC

AI summary

In some embodiments, an electronic device updates the spatial arrangement of one or more virtual objects in a three-dimensional environment. In some embodiments, an electronic device updates the positions of multiple virtual objects together. In some embodiments, an electronic device displays objects in a three-dimensional environment based on an estimated location of a floor in the three-dimensional environment. In some embodiments, an electronic device moves (e.g., repositions) objects in a three-dimensional environment.