Streaming Keyphrase Detection with Speaker Verification
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional keyphrase detection systems require retraining to support new phrases, are prone to false triggering and rejection, and necessitate predefined invocation phrases, limiting user interaction flexibility and efficiency.
Innovation Solution
The implementation of a streaming keyphrase detection system that customizes keyphrases without retraining models, utilizing a combination of speaker separation, text-independent speaker identification, automatic speech recognition, and adaptive noise cancellation models to accurately detect user-defined keyphrases with reduced latency and false error rates.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If conventional keyword spotting models are used to detect predefined phrases, then the system can recognize specific keyphrases, but the system requires retraining to support new keyphrases and cannot be easily customized
Solution Approach 1:
The system segments the keyphrase detection task into two independent components: speaker verification (using TI-SID model) and text recognition (using ASR model). This segmentation allows each component to be optimized independently, enabling keyphrase customization without retraining the entire system. The speaker verification component handles speaker identification while the ASR component handles text transcription, and their independent operation enables flexible keyphrase customization.
Solution Approach 2:
The ASR model serves multiple functions: it transcribes spoken input into text and simultaneously enables keyphrase detection by comparing the transcription against customized keyphrases. This multi-functionality eliminates the need for separate keyphrase detection models for each custom phrase, allowing the system to support unlimited keyphrase customization without increasing model complexity or retraining requirements.
2Reliability
If conventional keyphrase detection is used, then the system can trigger automated assistant, but the system is prone to false triggering and false rejection
Solution Approach 1:
The system introduces text transcription as an intermediary step between speech input and keyphrase detection. The ASR model converts spoken input into text, which then serves as the basis for keyphrase matching. This intermediary representation enables more reliable detection by allowing precise string matching against customized keyphrases, reducing false positives from acoustic similarities and false negatives from noisy speech recognition.
3Ease of operation
If predefined invocation phrases are required, then the system can invoke automated assistant, but user interaction flexibility is limited
Solution Approach 1:
The system transforms the static, predefined keyphrase set into a dynamic, user-customizable collection. Users can define their own keyphrases that reflect their speech patterns and preferences, and the system adapts by comparing ASR transcriptions against these personalized keyphrases. This dynamic customization capability significantly improves ease of operation while maintaining detection reliability.
4Productivity
If NLU processing is used to determine assistant commands, then the system can understand natural language, but processing latency increases
Solution Approach 1:
The system performs preliminary keyphrase detection and matching before full NLU processing. By first checking whether the ASR transcription matches a customized keyphrase, the system can quickly identify direct commands without requiring complete NLU analysis. This preliminary filtering step reduces processing latency for common commands while maintaining full NLU capability for more complex queries.
Data Source
AI summary
Techniques disclosed herein are directed towards streaming keyphrase detection which can be customized to detect one or more particular keyphrases, without requiring retraining of any model(s) for those particular keyphrase(s). Many implementations include processing audio data using a speaker separation model to generate separated audio data which isolates an utterance spoken by a human speaker from one or more additional sounds not spoken by the human speaker, and processing the separated audio data using a text independent speaker identification model to determine whether a verified and/or registered user spoke a spoken utterance captured in the audio data. Various implementations include processing the audio data and/or the separated audio data using an automatic speech recognition model to generate a text representation of the utterance. Additionally or alternatively, the text representation of the utterance can be processed to determine whether at least a portion of the text representation of the utterance captures a particular keyphrase. When the system determines the registered and/or verified user spoke the utterance and the system determines the text representation of the utterance captures the particular keyphrase, the system can cause a computing device to perform one or more actions corresponding to the particular keyphrase.