XR Digital Assistant Gaze Activation for Low-Input Interaction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing digital assistant interactions in extended reality environments are inefficient and require multiple user inputs, leading to increased cognitive burden and power consumption.
Innovation Solution
The system detects user gaze at persistent objects in an XR environment to initiate a digital assistant session, displaying animations to confirm initiation, and modifies display states based on gaze input to efficiently handle speech inputs, reducing unnecessary interactions and power usage.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If traditional digital assistant interactions are used in XR environments, then multiple user inputs are required to initiate and control the assistant, but this increases cognitive burden and power consumption
Solution Approach 1:
The system performs preliminary actions by continuously monitoring user gaze and displaying persistent objects in the XR environment before the user needs to interact. The digital assistant object is pre-positioned and animated to indicate readiness, so when the user looks at it, the assistant is already prepared to receive commands, eliminating the need for multiple activation steps and reducing overall interaction time and energy consumption
Solution Approach 2:
The digital assistant uses gaze detection to automatically determine when it should be active and when it should be in listening mode, serving itself based on user attention rather than requiring explicit user commands to activate or deactivate. The system self-regulates its power consumption by entering low-power states when not needed and activating only when the user's gaze indicates interest
2Reliability
If multiple user inputs are required to initiate digital assistant session, then the system can ensure proper activation, but this increases cognitive burden on the user
Solution Approach 1:
The system provides continuous visual feedback through animations of the persistent object in the XR environment, showing the user that the digital assistant is ready to receive commands. The object changes its display state based on user gaze detection, providing immediate feedback that guides the user through the interaction without requiring them to remember complex activation sequences or multiple input steps
Solution Approach 2:
The patent replaces complex mechanical interaction sequences (multiple button presses, voice commands, and confirmations) with a simpler optical mechanism - gaze detection. By using eye tracking to automatically trigger assistant activation, the system eliminates the need for users to perform multiple deliberate inputs, reducing cognitive burden while maintaining reliable activation through automatic gaze-based triggering
3Adaptability or versatility
If the digital assistant object is displayed persistently in XR environment, then the assistant is always available, but this increases device complexity
Solution Approach 1:
The persistent object is not static but dynamically responds to user gaze through animations and display state changes. When the user looks at the object, it activates and transitions to listening mode; when the user looks away, it returns to a dormant state. This dynamic behavior provides continuous availability while managing complexity through conditional logic based on gaze detection rather than requiring the assistant to be fully active all the time
4Productivity
If gaze detection is used to detect user intent, then interaction efficiency is improved, but this increases measurement precision requirements
Solution Approach 1:
The system uses gaze detection as a partial indicator of user intent rather than requiring precise and complete measurement of all user intentions. By combining gaze detection with persistent object display and animations, the system achieves sufficient accuracy for reliable interaction without needing perfectly precise gaze tracking. The excessive action of continuous gaze monitoring provides robust data even when individual gaze measurements are not perfectly accurate
Data Source
AI summary
An example process includes: while displaying a portion of an extended reality (XR) environment representing a current field of view of a user: detecting a user gaze at a first object displayed in the XR environment, where the first object is persistent in the current field of view of the XR environment; in response to detecting the user gaze at the first object, expanding the first object into a list of objects including a second object representing a digital assistant; detecting a user gaze at the second object; in accordance with detecting the user gaze at the second object, displaying a first animation of the second object indicating that a digital assistant session is initiated; receiving a first audio input from the user; and displaying a second animation of the second object indicating that the digital assistant is actively listening to the user.