Extended Reality Digital Assistant Indicators Anchored to External Devices
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing digital assistant interactions in extended reality environments require explicit user triggers, which can be burdensome and interrupt the user experience, and do not efficiently utilize the spatial awareness of physical environments.
Innovation Solution
A method where a digital assistant indicator is displayed at a known physical location, allowing interactions without explicit triggers by anchoring the indicator to the location of an external device, enabling intuitive and efficient user interactions.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If explicit user triggers are required to initiate digital assistant interactions, then the assistant can be activated on demand, but user interaction becomes burdensome and interrupts XR experience
Solution Approach 1:
The system performs preliminary actions by continuously monitoring user pose and gaze direction before interaction is needed. The digital assistant proactively determines when to activate based on pre-detected user orientation toward the external device, eliminating the need for explicit triggers at the moment of activation.
Solution Approach 2:
The digital assistant performs self-service by autonomously detecting user intent through pose and gaze analysis. The system serves itself by automatically initiating interactions based on spatial awareness without requiring user-initiated trigger commands, reducing operational burden.
2Ease of operation
If digital assistant indicator is displayed at arbitrary location, then display flexibility is maintained, but spatial awareness and intuitive interaction are reduced
Solution Approach 1:
The system applies local quality by anchoring the digital assistant indicator to a specific spatial location corresponding to the external device's physical position. This localized placement provides intuitive spatial awareness while maintaining display flexibility through proper coordinate transformation in the XR environment.
Solution Approach 2:
The system uses an intermediary coordinate transformation mechanism that maps the external device's physical location to the XR display space. This intermediary mapping enables the indicator to appear at the correct spatial position while maintaining the flexibility of the XR display system.
3Speed
If continuous monitoring for interaction is performed, then interaction responsiveness is improved, but power consumption increases
Solution Approach 1:
The system implements periodic action by monitoring user pose and gaze at defined intervals rather than continuously. This periodic sampling maintains adequate interaction responsiveness while significantly reducing power consumption compared to continuous monitoring.
Solution Approach 2:
The digital assistant performs self-service by utilizing existing sensor data (pose and gaze tracking) that may already be collected for other XR functions. This eliminates the need for separate continuous monitoring infrastructure, reducing additional power consumption while maintaining responsiveness.
Data Source
Figure 1A
Figure 1B
Figure 2
AI summary
An example process includes while displaying, on a display, an extended reality (XR) environment: receiving a user input; sampling, with a microphone, a user speech input; in accordance with a determination that the user input satisfies a criterion for initiating a digital assistant, initiating the digital assistant, including: displaying, within the XR environment, a digital assistant indicator at a first location of the XR environment; and while displaying the digital assistant indicator at the first location, providing, by the digital assistant, a response to the user speech input; after providing the response, ceasing to display the digital assistant indicator at the first location; and in accordance with ceasing to display the digital assistant indicator at the first location, displaying the digital assistant indicator at a second location of the XR environment, the second location corresponding to a physical location of an external electronic device implementing a second digital assistant.