Custom Hotword Model Training with Pre-trained Speech Embeddings
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing voice-enabled devices face challenges in efficiently detecting custom hotwords due to the need for large training datasets and the inflexibility of traditional hotword detection models, which limits quick customization and accurate on-device processing.
Innovation Solution
A method and system for training a custom hotword model using a pre-trained speech embedding model, combined with a head model, that leverages synthesized and human-generated speech data to adapt to custom hotwords, allowing efficient on-device training with reduced data requirements.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If traditional hotword detection models are used, then robust hotword detection for pre-set hotwords is achieved, but hotword customization is limited and requires large training datasets
Solution Approach 1:
The system performs preliminary action by pre-training the speech embedding model on a large corpus of speech data before actual hotword detection. This pre-training establishes a robust foundation for speech representation that can be adapted to custom hotwords with minimal additional training data, resolving the contradiction between customization capability and data requirements
Solution Approach 2:
The hotword detection model is segmented into two independent components: a pre-trained speech embedding model and a custom hotword detection model. This segmentation allows the embedding model to learn general speech patterns from large datasets while the detection model adapts to specific custom hotwords with small datasets, enabling both robust detection and customization
2Reliability
If large sets of training data are used, then robust hotword detection is generated, but training time and computational resources increase
Solution Approach 1:
The speech embedding model is pre-trained in advance on extensive speech data to establish robust speech representations. This preliminary training separates the time-consuming learning of general speech patterns from the rapid adaptation to specific hotwords, maintaining high detection accuracy while reducing actual training time
Solution Approach 2:
The pre-trained speech embedding model serves itself by providing robust speech representations that enable the custom hotword detection model to achieve high accuracy with minimal training data, eliminating the need for extensive re-training when adapting to new hotwords
3Measurement precision
If custom hotword models are trained from scratch, then detection accuracy for custom hotwords is achieved, but the process lacks flexibility and requires extensive re-training
Solution Approach 1:
The system performs preliminary action by pre-training the speech embedding model on diverse speech data before custom hotword detection. This establishes a robust foundation that enables rapid adaptation to custom hotwords with high accuracy, eliminating the need to train from scratch and providing both precision and flexibility
Solution Approach 2:
The pre-trained speech embedding model serves multiple functions: it provides robust speech representations for any custom hotword, enables rapid adaptation, and maintains high detection accuracy. This universal model eliminates the need for separate training processes for different hotwords, achieving both precision and adaptability
Data Source
AI summary
A method of training a custom hotword model includes receiving a first set of training audio samples. The method also includes generating, using a speech embedding model configured to receive the first set of training audio samples as input, a corresponding hotword embedding representative of a custom hotword for each training audio sample of the first set of training audio samples. The speech embedding model is pre-trained on a different set of training audio samples with a greater number of training audio samples than the first set of training audio samples. The method further includes training the custom hotword model to detect a presence of the custom hotword in audio data. The custom hotword model is configured to receive, as input, each corresponding hotword embedding and to classify, as output, each corresponding hotword embedding as corresponding to the custom hotword.