Dynamic Hot Words for Low-Power Assistant Invocation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing automated assistants often require cumbersome long tail hot words for invocation, leading to inefficient use of computing resources and power, especially in noisy environments, and may inadvertently activate due to standardized hot words.
Innovation Solution
Implementing dynamic hot words that are context-specific, activated based on user proximity, recognition, or group membership, allowing for enhanced invocation without extensive speech-to-text processing, and caching responsive actions for quicker execution.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Use of energy by moving object
If the automated assistant operates in limited hot word listening state, then power consumption and computing resources are conserved, but the assistant cannot respond to natural language queries without predefined hot words
Solution Approach 1:
The patent implements dynamic hot word expansion where the set of monitored hot words changes based on contextual signals. The system transitions from a static limited hot word list to a dynamic expanded hot word list when contextual conditions are met, allowing the assistant to adapt its listening behavior to balance power consumption and responsiveness
Solution Approach 2:
The system changes the parameter of hot word vocabulary size based on contextual signals. When contextual conditions are satisfied (e.g., user proximity detected, recognized user, or group membership), the system expands the hot word list from a minimal set to a larger context-specific set, enabling natural language queries without full speech-to-text processing
2Ease of operation
If standardized hot words are used for invocation, then the system is simple to operate, but the assistant may inadvertently activate due to false positives in noisy environments
Solution Approach 1:
The patent applies different hot word sets to different contextual situations. Instead of using a single standardized hot word list everywhere, the system uses context-specific hot word expansions tailored to particular users, locations, or scenarios, reducing false activations while maintaining ease of use for authorized users
Solution Approach 2:
The system uses contextual signals (proximity sensors, user recognition, group membership) as feedback mechanisms to dynamically adjust the hot word monitoring behavior. This feedback loop allows the system to distinguish between genuine user intent and ambient noise, reducing inadvertent activation while preserving simple invocation for legitimate users
3Adaptability or versatility
If continuous speech-to-text processing is performed, then the assistant can respond to any user utterance, but power consumption and computing resources are significantly increased
Solution Approach 1:
The patent implements partial speech processing by monitoring only an expanded set of context-specific hot words rather than performing full speech-to-text processing on all utterances. This partial action approach provides selective responsiveness to natural language queries while avoiding the excessive computing resource consumption of continuous full STT processing
Solution Approach 2:
The system segments the speech processing task into two levels: (1) monitoring an expanded but still limited set of context-specific hot words using lightweight on-device models, and (2) performing full speech-to-text processing only when a hot word is detected. This segmentation reduces overall computing resource consumption while maintaining the ability to respond to relevant queries
Data Source
Figure 1
Figure 2
Figure 3A~3B
AI summary
Techniques are described herein for enabling the use of "dynamic" or "context-specific" hot words to invoke an automated assistant. In various implementations, an automated assistant may be executed in a default listening state at least in part on a user's computing device(s). While in the default listening state, audio data captured by microphone(s) may be monitored for default hot words. Detection of the default hot word(s) transitions of the automated assistant into a speech recognition state. Sensor signal(s) generated by hardware sensor(s) integral with the computing device(s) may be detected and analyzed to determine an attribute of the user. Based on the analysis, the automated assistant may transition into an enhanced listening state in which the audio data may be monitored for enhanced hot word(s). Detection of enhanced hot word(s) triggers the automated assistant to perform a responsive action without requiring detection of default hot word(s).