Proactive Assistant Action Caching for Offline Utterance Resolution
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing client-server approaches for voice-based user interactions in automated assistants require continuous online connectivity, consume significant bandwidth, and exhibit latency due to remote processing, which can be problematic in offline or low-bandwidth situations.
Innovation Solution
Implement proactive caching of assistant action content at the client device for on-device resolution of spoken or typed utterances, using locally stored proactive cache entries that are prefetched based on device attributes and anticipated user needs, enabling quick responses without live server communication.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If a client-server approach is used for voice-based user interactions, then complex speech recognition and semantic analysis functionality can be implemented, but the client requires continuous online connectivity and exhibits significant latency
Solution Approach 1:
The patent applies preliminary action by prefetching and caching assistant action content (AAC) entries at the client device before they are actually needed. The system predicts which content the user will request based on current context (location, time, previously played content) and pre-loads this content into local cache, so that when the user actually makes a request, the response can be provided immediately from local storage without waiting for server communication.
2Measurement precision
If a client-server approach is used for voice-based user interactions, then complex processing functionality is available, but significant bandwidth is consumed due to transmission of high-bandwidth audio data
Solution Approach 1:
The patent applies the extraction principle by removing the need to transmit audio data to the server for certain types of queries. Instead of sending the entire audio stream to the remote system, the patent extracts only the necessary information by matching spoken utterances against a local database of pre-recorded assistant action content entries, thereby eliminating bandwidth-consuming transmissions for these cases.
Solution Approach 2:
The system performs preliminary action by pre-processing and storing assistant action content entries locally at the client device. This includes pre-recording various responses and pre-generating metadata that enables quick local matching against user queries, so that common queries can be resolved without any network transmission.
3Measurement precision
If continuous online connectivity is required for remote processing, then complex speech recognition can be performed, but the system cannot operate in offline or low-bandwidth situations
Solution Approach 1:
The patent applies preliminary action by pre-loading assistant action content entries into local cache at the client device before offline operation is needed. The system predicts which content will be needed based on context (current location, time of day, previously played content) and pre-loads this content, enabling the device to operate fully offline for these predicted queries without requiring continuous server connectivity.
Solution Approach 2:
The patent applies local quality by implementing a local database of assistant action content entries with associated metadata at the client device. This local storage enables the system to provide speech recognition and response functionality without requiring continuous connection to the remote server, with different levels of functionality available locally versus remotely.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
Implementations can reduce the time required to obtain responses from an automated assistant through proactive caching, locally at a client device, of proactive assistant cache entries - and through on-device utilization of the proactive assistant cache entries. Different proactive cache entries can be provided to different client devices, and various implementations relate to technique(s) utilized in determining which proactive cache entries to provide to which client devices. In some of those implementations, in determining which proactive cache entries to provide (proactively or in response to a request) to a given client device, a remote system selects, from a superset of candidate proactive cache entries, a subset of the cache entries for providing to the given client device.