Private Digital Assistant Audio Routing in XR Copresence
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing digital assistant technologies struggle to accurately and efficiently initiate private interactions during copresence sessions, as they often transmit user requests and responses to all participants, compromising privacy and disrupting the extended reality experience.
Innovation Solution
The method involves displaying an extended reality environment, sampling audio inputs, and determining whether they are intended for a digital assistant on an external device. If intended for a private interaction, the digital assistant provides an audible response without transmitting it over the shared communication channel for the copresence session.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If digital assistant audio responses are transmitted over the shared communication channel during copresence sessions, then all participants can hear the response, but user privacy is compromised and other users are disrupted
Solution Approach 1:
The audio output is segmented into private and shared components. The system separates the digital assistant response into a private audio path (local speaker only) and a shared communication path (transmitted to other users), allowing selective delivery based on privacy requirements
Solution Approach 2:
Different quality attributes are applied to different parts of the audio output. The private response receives high-fidelity local playback through the device speaker, while the shared transmission uses compressed audio suitable for network delivery, optimizing both privacy and resource usage
2Measurement precision
If the digital assistant continuously monitors audio inputs during copresence sessions, then it can accurately determine intent, but battery life is reduced due to increased processing
Solution Approach 1:
The audio monitoring operates periodically rather than continuously. The system activates audio processing only during copresence sessions when needed, and uses periodic sampling to detect audio inputs, reducing overall processing time and energy consumption while maintaining detection accuracy
Solution Approach 2:
The device uses its own local resources to process and respond to audio inputs during copresence sessions, reducing the need for continuous cloud communication and associated energy consumption
3Loss of information
If the digital assistant transmits all audio responses to external devices, then communication completeness is maintained, but unnecessary data transmission increases network usage and latency
Solution Approach 1:
The system extracts only the necessary audio responses from the complete set of digital assistant outputs. By identifying which responses are intended for private local playback versus shared transmission, it removes unnecessary network traffic and reduces latency for essential communications
Data Source
AI summary
An example process includes: displaying, on a display of an electronic device, an extended reality (XR) environment corresponding to a copresence session including the electronic device and a second electronic device; while displaying the XR environment: sampling, with a microphone of the electronic device, a first audio input; determining whether the first audio input is intended for a first digital assistant operating on an external electronic device; and in accordance with a determination that the first audio input is intended for the first digital assistant: causing the first digital assistant to provide an audible response to the first audio input, where the audible response is not transmitted to the second electronic device over a shared communication channel for the copresence session.