Automated Assistant Data Sync via Hotword-Based Device Arbitration
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
As automated assistants become decentralized across individual client devices, user data becomes siloed, leading to inconsistent experiences and outdated information, especially when devices are not connected to the internet or lack cloud accounts, and sensitive data is not stored in the cloud.
Innovation Solution
Implement cross-device data synchronization through simultaneous hotword triggers, allowing seamless sharing of personalized models and preferences across devices in a fully on-device scenario, enabling a consistent user experience and synchronizing data in a peer-to-peer manner, even without central cloud connections.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If automated assistants are decentralized across individual client devices, then user privacy is preserved and resource consumption is reduced, but user data becomes siloed leading to inconsistent experiences and outdated information
Solution Approach 1:
The patent introduces a cloud-based account system as an intermediary that stores user data centrally while allowing decentralized devices to access synchronized data. The account acts as a mediator between privacy-preserving local processing and consistent data availability across devices, resolving the contradiction by enabling data sharing without requiring continuous cloud connectivity for all operations.
Solution Approach 2:
The system segments data management into local and cloud components: sensitive processing occurs locally on devices to preserve privacy, while data synchronization and consistency are managed through the cloud account. This segmentation allows each component to optimize for its specific function - local processing for privacy and cloud storage for data consistency.
2Use of energy by moving object
If devices operate without cloud connections or accounts, then resource consumption is reduced and privacy is maintained, but data becomes outdated and synchronization cannot occur
Solution Approach 1:
The system implements periodic synchronization where devices connect to the cloud account intermittently to update data, rather than maintaining continuous connections. This allows devices to operate independently and conserve resources most of the time, while periodically refreshing data to maintain consistency without requiring constant cloud connectivity.
3Reliability
If data is not stored in the cloud, then user privacy is protected, but cross-device data synchronization cannot occur
Solution Approach 1:
The cloud account serves as an intermediary storage layer that enables cross-device synchronization while maintaining privacy. Sensitive data processing occurs locally on devices, but the account stores essential user data that can be synchronized across devices. This intermediary approach allows synchronization capability without requiring all data to be stored in the cloud.
4Ease of operation
If automated assistants process all spoken utterances, then user experience is improved, but resource consumption increases and privacy is compromised
Solution Approach 1:
The system applies different processing qualities to different types of audio data: hotword detection is performed continuously with high sensitivity to ensure quick activation, while full speech processing is performed only when hotwords are detected. This local quality differentiation optimizes resource usage by intensive processing only when necessary, while maintaining responsive user experience for actual commands.
Data Source
Figure 1A
Figure 1B
Figure 2
AI summary
Techniques are described herein for cross-device data synchronization. A method includes: executing a first instance of an automated assistant at least in part on a first computing device; receiving audio data that captures a spoken utterance; processing the audio data using a machine learning model to generate a predicted output that indicates a probability of one or more hotwords being present in the audio data; determining that the predicted output satisfies a threshold; in response to determining that the predicted output satisfies the threshold, performing arbitration with at least one other computing device that is executing at least in part at least one other instance of the automated assistant; and in response to performing arbitration with the at least one other computing device, initiating synchronization of user data or configuration data between the first instance of the automated assistant and the at least one other instance of the automated assistant.