Automated Voice Assistant Failure Detection Using Cached Audio
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In situations where multiple automated assistants are present on the same device or nearby devices, a user's request may be incorrectly directed to an assistant that fails to handle it, leading to suboptimal or inaccurate results.
Innovation Solution
An automated assistant can detect failures in other assistants and offer to handle the user's request, either automatically or upon user request, by caching and processing the user's input to provide a suitable response.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If multiple automated assistants are run on the same device or nearby devices, then user choice and specialization are improved, but request fulfillment reliability deteriorates when users direct requests to the wrong assistant
Solution Approach 1:
The patent implements a feedback mechanism where the selected automated assistant reports back to the router about the success or failure of request fulfillment. When a request fails, the router receives this feedback and can re-route the request to a different assistant, thereby maintaining reliability even when users initially select the wrong assistant.
Solution Approach 2:
The patent introduces a router as an intermediary component that sits between the user and multiple automated assistants. The router intelligently directs requests to the appropriate assistant based on various criteria, and can also act as a mediator to forward failed requests from one assistant to another, ensuring that the user's request ultimately gets fulfilled regardless of initial selection.
2Measurement precision
If automated assistants continuously process all audio data, then response accuracy is improved, but resource consumption and privacy risks increase
Solution Approach 1:
The patent applies preliminary action by implementing hotword detection before full speech processing. The system first checks for the presence of a hotword in the audio stream, and only if detected, proceeds to perform the more resource-intensive speech-to-text conversion and natural language processing. This preliminary filtering step significantly reduces resource consumption while maintaining response accuracy for relevant queries.
Data Source
Figure 1A
Figure 1B
Figure 2
AI summary
Techniques are described herein for detecting and handling failures in other automated assistants. A method includes: executing a first automated assistant in an inactive state at least in part on a computing device operated by a user; while in the inactive state, determining, by the first automated assistant, that a second automated assistant failed to fulfill a request of the user; in response to determining that the second automated assistant failed to fulfill the request of the user, the first automated assistant processing cached audio data that captures a spoken utterance of the user comprising the request that the second automated assistant failed to fulfill, or features of the cached audio data, to determine a response that fulfills the request of the user; and providing, by the first automated assistant to the user, the response that fulfills the request of the user.