Gaze-Based 3D Object Manipulation via Remote Input
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current user interfaces for virtual and augmented reality environments are cumbersome, inefficient, and complex, requiring excessive inputs and leading to a cognitive burden on users, particularly in battery-operated devices where energy efficiency is a concern.
Innovation Solution
The implementation of a computer system with improved input mechanisms that utilize a remote input device and gaze detection to control object selection and movement in three-dimensional environments, reducing the number and complexity of inputs needed through the integration of touch-sensitive surfaces, eye-tracking, and hand-tracking components.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If conventional input devices (controllers, joysticks, touch surfaces) are used to interact with virtual reality environments, then users can manipulate virtual objects, but the interaction becomes cumbersome, complex, and cognitively burdensome
Solution Approach 1:
The patent replaces mechanical input devices (controllers, joysticks, touch surfaces) with a camera-based visual interaction system. Users select and manipulate virtual objects by pointing the camera at them and using gaze detection, eliminating the need for physical controllers and complex input mechanisms while simplifying the interaction model.
Solution Approach 2:
The patent introduces a camera as an intermediary device between the user and the virtual environment. The camera captures real-world views and enables object selection through visual pointing, serving as a mediator that translates natural human pointing gestures into virtual environment commands without requiring direct physical interaction with complex controllers.
2Productivity
If multiple input steps are required to achieve desired outcomes in augmented reality environments, then precise control is possible, but interaction time increases and energy consumption rises
Solution Approach 1:
The patent implements gaze detection that pre-selects virtual objects based on where the user is looking, before any explicit selection input is given. This preliminary action of automatic object selection based on gaze direction eliminates the need for manual navigation through multiple menus or repeated input steps, significantly reducing interaction time and energy consumption.
Solution Approach 2:
The system performs automatic object selection and manipulation based on the user's natural gaze and pointing direction, making the system serve itself by interpreting user intent without requiring explicit step-by-step commands. This self-service approach reduces the number of interaction steps needed while maintaining precise control.
3Ease of operation
If conventional touch-sensitive display interfaces are used, then users can interact with virtual elements, but the interface becomes cluttered and requires excessive inputs
Solution Approach 1:
The patent moves the interaction interface from the two-dimensional touch screen plane into the three-dimensional real-world space captured by the camera. Virtual objects are selected and manipulated by pointing the camera at them in the real world, adding a spatial dimension to the interaction and eliminating the need for cluttered on-screen controls and menus.
Data Source
AI summary
While a view of an environment is visible via the display generation component of a computer system, a first motion input that includes movement of the remote input device in a physical environment is detected. In response to detecting the first motion input and in accordance with a determination that a gaze detected by the computer system was directed to a first object when the first motion input was detected, the first object is moved in the environment in accordance with the first motion input. In response to detecting the first motion input and in accordance with a determination that the gaze detected by the computer system was not directed to the first object when the first motion input was detected, the computer system forgoes moving the first object in the environment in accordance with the first motion input.