Custom Wake-Up Word Detection With Local Phoneme Adaptation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current wake-up word detection systems for digital assistants lack personalization and require extensive training on thousands of samples from various speakers to achieve accuracy, limiting their adaptability to individual users.
Innovation Solution
A customizable keyword spotting system that allows users to assign their own wake-up word by registering it locally without server assistance, using a phoneme recognizer model trained with a dataset like TIMIT, and employing pronunciation augmentation and pruning to improve detection accuracy through short-term and long-term adaptation.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If a predefined universal wake-up word is used, then keyword spotting accuracy can be maintained through training on thousands of samples, but personalization and adaptability to individual users are lost
Solution Approach 1:
The system dynamically adapts the wake-up word detection model to each user's speech characteristics through short-term adaptation during initial interaction and long-term adaptation over time. The phoneme recognizer model is continuously refined using user-specific audio data, transforming a static universal model into a dynamic personalized model that evolves with each user interaction.
Solution Approach 2:
The system applies local quality by customizing the wake-up word detection parameters and model weights specific to each user's speech patterns, accent, and pronunciation characteristics. Instead of using a one-size-fits-all approach, the system tailors the detection algorithm to match individual user qualities, enabling accurate recognition of personalized wake-up words.
2Measurement precision
If extensive training data from multiple speakers is collected, then keyword spotting accuracy improves, but the system complexity and data processing requirements increase
Solution Approach 1:
The system performs self-service by automatically collecting, processing, and adapting to user-specific speech patterns without requiring manual intervention or extensive external training data. The phoneme recognizer model autonomously refines itself using audio data from individual users during their initial interactions and ongoing usage, eliminating the need for complex centralized training pipelines.
Solution Approach 2:
The training process is segmented into short-term adaptation phase during initial user interaction and long-term adaptation phase over time. This segmentation allows the system to achieve sufficient precision for each user without requiring processing of thousands of samples from multiple speakers, reducing overall system complexity while maintaining detection accuracy.
3Reliability
If a server-based validation system is used for keyword registration, then keyword spotting accuracy can be validated, but system response time and dependency on external servers increase
Solution Approach 1:
The system extracts and performs keyword registration and validation locally on the device using the phoneme recognizer model, eliminating the need for server-based validation. The wake-up word detection algorithm is deployed on-edge, enabling the system to register and validate keywords independently without network dependency, thus reducing response time while maintaining validation reliability.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
A wake-up word for a digital assistant (12) may be specified by a user (202) to trigger the digital assistant to respond to the wake-up word, with the user providing (204) one or more initial pronunciations of the wake-up word. The wake-up word may be unique, or at least not determined beforehand by a device manufacturer or developer of the digital assistant. The initial pronunciation(s) of the keyword may then be augmented (214) with other potential pronunciations of the wake-up word that might be provided in the future, and those other potential pronunciations may then be pruned (214) down to a threshold number of other potential pronunciations. One or more recordings of the initial pronunciation(s) of the wake-up may then be used to train (218) a phoneme recognizer model to better recognize future instances of the wake-up word being spoken by the user or another person using the initial pronunciation or other potential pronunciations.