Streaming Keyword Spotting With Enrollment Embeddings for Custom Phrases
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing keyword detection models struggle to accurately detect user-defined keywords due to limited training data, leading to inefficiencies and degraded user experiences, especially for users with rare or unseen voice characteristics.
Innovation Solution
A training process that involves determining keyword enrollment and test embeddings for user-defined keywords, using a conformer encoder, and employing speaker verification to adapt the keyword detection model for personalized keyword detection, thereby enhancing accuracy for individual users.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If the number of customized keywords is increased to meet user personalization demands, then user adaptability is improved, but keyword detection complexity increases
Solution Approach 1:
The patent segments the keyword detection task into two distinct stages: enrollment phase (where custom keywords are learned and stored as embeddings) and detection phase (where stored embeddings are compared against incoming audio). This segmentation allows the system to handle multiple custom keywords efficiently by organizing them in a structured embedding space rather than processing them as a monolithic complex pattern recognition problem.
Solution Approach 2:
The patent creates a simplified representation (embedding) of each custom keyword during the enrollment phase, capturing its essential acoustic characteristics. During detection, the system compares incoming audio against these pre-computed embeddings rather than re-analyzing all keyword possibilities from scratch. This copying approach reduces computational complexity while maintaining accuracy for multiple personalized keywords.
2Measurement precision
If traditional keyword detection models are used with limited training data, then device complexity is reduced, but detection precision deteriorates
Solution Approach 1:
The patent performs preliminary action during the enrollment phase by capturing and storing acoustic embeddings of custom keywords before actual detection occurs. This pre-computation of keyword representations allows the detection phase to proceed with simpler, faster embedding comparison operations rather than complex real-time pattern matching, thereby improving detection precision without proportionally increasing operational complexity.
Solution Approach 2:
The patent transforms the keyword detection problem from one requiring complex pattern recognition with limited data to one involving embedding similarity comparison. By changing the representation parameters from raw audio waveforms to compressed embedding vectors, the system achieves higher detection precision with more efficient computational requirements, effectively resolving the trade-off between precision and complexity.
3Reliability
If more audio data samples are collected for training, then keyword detection reliability is improved, but processing time increases
Solution Approach 1:
The patent extracts the essential acoustic characteristics from large sets of audio data samples during enrollment, distilling them into compact embedding representations. This extraction process filters out redundant information while preserving the critical features needed for reliable keyword detection. During operational detection, the system works only with these compressed embeddings rather than processing the full audio datasets, thereby maintaining high reliability while significantly reducing processing time.
Data Source
AI summary
A method includes receiving a plurality of sets of utterances. Each respective set of utterances includes audio data samples of a corresponding utterance different than the corresponding utterance of each other set of utterances of the plurality of sets of utterances. For a respective one of the sets of utterances, the method includes determining a keyword enrollment embedding for an enrollment subset of the audio data samples of the respective one of the sets of utterances and determining a corresponding matching keyword test embedding for each respective audio data sample of a test subset of the audio data samples of the respective one of the sets of utterances. The method also includes determining a corresponding nonmatching keyword test embedding for each respective audio data sample of each of the other sets of utterances. The method also includes training a keyword detection model to detect a presence of a custom keyword.