Automated Assistant Arbitration via Timestamp Comparison
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In environments with multiple electronic devices capable of interacting with automated assistants, there is a challenge in determining which device should respond to a user's utterance when multiple devices are invoked simultaneously.
Innovation Solution
The solution involves each electronic device emitting output encoded with a timestamp indicating when it detected the user's utterance. Devices then compare these timestamps to determine which should respond and which should defer, with techniques like altering local timestamps and using contextual cues to prioritize or deprioritize devices.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If multiple electronic devices are equipped with automated assistant functionality to improve accessibility and user convenience, then the system becomes more versatile and user-friendly, but conflicts arise when multiple devices detect the same user utterance simultaneously, leading to response conflicts and user confusion
Solution Approach 1:
The patent introduces a mediator device that receives invocation detections from multiple electronic devices and coordinates the response arbitration. The mediator device collects timestamp data from all devices, determines which device should respond based on earliest detection time, and notifies the selected device to execute the automated assistant function. This intermediary approach resolves conflicts between multiple devices without requiring complex peer-to-peer communication protocols.
Solution Approach 2:
The patent replaces traditional mechanical arbitration methods (such as waiting for device conflicts or using complex priority-based systems) with a timestamp-based temporal ordering mechanism. Each device records the exact time it detected the invocation, and the mediator uses these timestamps to deterministically select the device with the earliest detection time. This substitution of temporal ordering for mechanical arbitration simplifies the system while improving reliability.
2Ease of operation
If electronic devices use predefined hot words or wake words to invoke automated assistants, then the activation process is simple and user-friendly, but multiple devices may simultaneously detect the same hot word, causing response conflicts
Solution Approach 1:
The mediator device serves as a central coordinator that receives invocation detections from multiple electronic devices and manages the arbitration process. Instead of each device implementing complex arbitration logic independently, the mediator consolidates the arbitration function, collecting timestamp data from all devices and determining which device should respond. This approach maintains ease of operation for users while managing device complexity centrally.
Solution Approach 2:
The patent uses timestamp copying as a simple mechanism to track invocation detection times. Each device creates a timestamp record of when it detected the hot word and transmits this copied time information to the mediator. This copying approach provides a simple, lightweight mechanism for arbitration that doesn't require complex communication protocols or additional hardware, maintaining ease of operation while resolving conflicts.
3Speed
If all electronic devices in the environment continuously monitor for user utterances, then the system responds quickly to user commands, but energy consumption increases and false detections may occur
Solution Approach 1:
The patent implements preliminary action by having devices monitor for specific invocation patterns (hot words or wake words) rather than continuously analyzing all audio input. Devices perform preliminary detection of these trigger phrases and only fully activate the automated assistant when the pattern is recognized. This preliminary monitoring approach maintains quick response times for valid invocations while reducing energy consumption compared to continuous full-analysis monitoring.
Solution Approach 2:
The mediator device performs self-service by autonomously managing the arbitration process without requiring user intervention. When multiple devices detect an invocation, the mediator automatically compares timestamps, determines the earliest detecting device, and notifies it to execute the response. This self-service arbitration reduces the need for user actions to resolve conflicts and maintains efficient system operation without increasing individual device power consumption.
Data Source
Figure 1
Figure 2A~2B
Figure 3
AI summary
Techniques described herein are directed to arbitrating between multiple potentially-responsive, automated-assistant capable electronic devices to determine which should respond to the user's utterance, and/or which should defer to other electronic device(s). In various implementations, a spoken utterance of a user may be detected at a microphone of a first electronic device, a spoken utterance provided by a user. Sound(s) emitted by additional electronic device(s) may also be detected at the microphone. Each of the sound(s) may encode a timestamp corresponding to detection of the spoken utterance at a respective electronic device. Timestamp(s) may be extracted from the sound(s) and compared to a local timestamp corresponding to detection of the spoken utterance at the first electronic device. Based on the comparison, the first electronic device may either invoke an automated assistant locally or defer to one of the additional electronic devices.