Automated Assistant Handoff for Continued Multi-Device Conversations
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Determining which device to process audio data for continued conversations with automated assistants across multiple devices in an ecosystem is challenging, leading to inefficiencies and resource wastage.
Innovation Solution
Implementing a system where devices in an ecosystem communicate to determine user presence and context, allowing the nearest device to process subsequent audio data, thereby optimizing resource usage and ensuring seamless continuation of conversations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of energy
If the first device continues to process all subsequent audio data, then conversation context is maintained, but computational resources are wasted when the user is near another device
Solution Approach 1:
The system performs preliminary actions by notifying other devices in the ecosystem about an active conversation before the user actually moves. This allows other devices to prepare for potential audio processing by checking if the user is nearby, so when the user does move, the handoff can occur immediately without interrupting the conversation flow.
Solution Approach 2:
The system implements feedback by continuously monitoring user presence through sensor data from multiple devices. When another device detects the user is nearby, it provides feedback to the first device, triggering a handoff of audio processing responsibilities. This feedback loop ensures context is maintained while optimizing resource usage.
2Measurement precision
If multiple devices process sensor data simultaneously to detect user presence, then the nearest device can be identified, but device complexity increases
Solution Approach 1:
The automated assistant is implemented as a universal service that can execute on multiple different device types within the ecosystem. Rather than requiring specialized hardware or complex device-specific logic, the same assistant instance handles conversations across smartphones, smart speakers, tablets, and other devices, simplifying the overall system architecture.
Solution Approach 2:
The system uses an intermediary notification mechanism where the first device notifies other devices about active conversations, and sensor data processing is coordinated through this notification system. This intermediary approach simplifies coordination by providing a clear trigger event rather than requiring continuous complex inter-device communication protocols.
3Productivity
If the first device ceases processing audio data immediately upon detecting user movement, then resource efficiency improves, but conversation continuity may be interrupted
Solution Approach 1:
The system performs preliminary actions by notifying other devices about active conversations before the user actually moves. This allows other devices to prepare for potential audio processing by checking if the user is nearby, so when the user does move, the handoff can occur immediately without interrupting the conversation flow.
Solution Approach 2:
The system ensures continuity of useful action by maintaining the conversation context on the first device while simultaneously preparing the second device to take over. The notification and sensor checking process occurs in parallel, ensuring that when the handoff occurs, there is no interruption in the conversation flow and the user experience remains seamless.
Data Source
AI summary
Implementations related to facilitating continued conversations of a user with an automated assistant when the user changes locations relative to one or more devices in an ecosystem of linked assistant devices. The user initially invokes a first device and provides a request, which is processed by the first device. The first device provides a notification to one or more other devices in the ecosystem to indicate that the user is likely to issue a further assistant request. The first device processes subsequent audio data to determine whether the subsequent audio data includes a further assistant request. The one or more other notified devices process device-specific sensor data to determine whether the user is co-present with the one of the other devices. If the user presence is detected, an indication is provided to the first device, causing the first device to cease processing subsequent audio data. Further, the co-present device starts to process subsequent audio data.