Multidomain Endpointer Model for Speech Detection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing natural language processing systems face challenges in accurately determining the end of a user's speech, leading to inefficiencies and resource wastage due to incorrect endpointing, particularly in differentiating between long-form speech and short-form voice queries, which affects voice activity detection and end-of-query detection.
Innovation Solution
A multidomain endpointer model using a unified Convolutional, Long Short-Term Memory, Deep Neural Network (CLDNN) with multitask learning, jointly training for voice activity detection and end-of-query classification, shares hidden layers across domains but uses separate output layers to differentiate between domain-specific targets, leveraging acoustic cues to predict the end of speech.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If separate endpointer models are used for long-form speech and short-form voice queries, then domain-specific detection accuracy is improved, but device complexity and resource consumption increase
Solution Approach 1:
The patent implements a universal endpointer model that can handle both long-form speech and short-form voice queries through a single unified architecture. The model uses domain indication inputs to adapt its behavior to different speech types, eliminating the need for separate models while maintaining detection accuracy across domains.
Solution Approach 2:
The endpointer model dynamically adjusts its detection strategy based on the indicated domain. When processing long-form speech, it applies different endpointing criteria compared to short-form queries, allowing the same model structure to adapt its behavior to match domain-specific requirements without requiring separate static models.
2Measurement precision
If separate endpointer models are used for long-form speech and short-form voice queries, then domain-specific detection accuracy is improved, but training time and computational resources increase
Solution Approach 1:
The patent merges the training processes for long-form speech and short-form voice queries into a single unified training procedure. The model is trained simultaneously on both domains using combined training data with domain indications, reducing total training time compared to separate training while achieving comparable or better performance through cross-domain learning.
3Ease of operation
If traditional voice activity detection is used, then implementation simplicity is maintained, but endpointing accuracy deteriorates due to inability to differentiate speech types
Solution Approach 1:
The patent introduces domain indication inputs as an intermediary mechanism that provides contextual information about speech type to the endpointer model. This additional input layer enables the model to differentiate between long-form and short-form speech without fundamentally complicating the overall system architecture, maintaining ease of implementation while improving accuracy.
Data Source
AI summary
A method for training an endpointer model includes short-form speech utterances and long-form speech utterances. The method also includes providing a short-form speech utterance as input to a shared neural network, the shared neural network configured to learn shared hidden representations suitable for both voice activity detection (VAD) and end-of-query (EOQ) detection. The method also includes generating, using a VAD classifier, a sequence of predicted VAD labels and determining a VAD loss by comparing the sequence of predicted VAD labels to a corresponding sequence of reference VAD labels. The method also includes, generating, using an EOQ classifier, a sequence of predicted EOQ labels and determining an EOQ loss by comparing the sequence of predicted EOQ labels to a corresponding sequence of reference EOQ labels. The method also includes training, using a cross-entropy criterion, the endpointer model based on the VAD loss and the EOQ loss.