ASR Context Biasing via Text Injection for Unseen Named Entities
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Deep learning-based automatic speech recognition (ASR) models tend to overfit training data, leading to difficulties in generalizing unseen data, especially when the training data is not extensive, and struggle with accurately recognizing named entities like contact names, song names, or location names that are infrequently included in training datasets.
Innovation Solution
A training process that injects contextually biased data into the ASR model by using unspoken textual utterances paired with carrier phrases, generating higher order textual feature representations, and training the model on these data pairs to improve recognition of unseen entities without increasing word error rates or latency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If ASR models use conventional biasing techniques to increase probability of transcribing particular words or phrases, then recognition accuracy for specific entities improves, but word error rate and latency degradation occur
Solution Approach 1:
The patent introduces a context encoder as an intermediary component that processes context information separately and generates context embeddings. These context embeddings are then integrated with acoustic features through a fusion layer, allowing the model to incorporate contextual bias without directly manipulating transcription probabilities. This intermediary approach enables accurate recognition of specific entities while maintaining overall system reliability and avoiding the degradation issues of conventional biasing techniques.
Solution Approach 2:
The patent segments the ASR model into distinct components: an acoustic model for processing speech input, a context encoder for processing contextual information, and a fusion layer for integrating these features. This segmentation allows the context encoder to independently process and inject contextual bias without affecting the core acoustic recognition capabilities, thereby improving accuracy for specific entities while maintaining low word error rates and latency.
2Adaptability or versatility
If ASR models are trained on extensive training data to improve generalization, then ability to recognize unseen data improves, but training data availability becomes a limiting factor for rare entities
Solution Approach 1:
The patent applies preliminary action by pre-processing context information through the context encoder to extract meaningful contextual features and generate context embeddings before they are needed for recognition. This pre-extraction of contextual features allows the model to effectively recognize rare entities like contact names, song names, or location names even when these entities are not present in the training data, as the contextual bias is prepared in advance through the encoder.
Solution Approach 2:
The context encoder serves as an intermediary that bridges the gap between limited training data and the need to recognize unseen entities. By encoding contextual information separately and fusing it with acoustic features, the system can generalize to rare entities without requiring extensive training examples, effectively overcoming the limitation of training data availability.
3Adaptability or versatility
If the number of biasing phrases increases to cover more entities, then recognition coverage improves, but word error rate and latency degradation worsen
Solution Approach 1:
The patent extracts contextual information from the input and processes it separately through the context encoder, which generates compact context embeddings. This extraction approach allows the model to incorporate bias for multiple entities without explicitly enumerating numerous biasing phrases, thereby improving recognition coverage while avoiding the computational overhead and latency associated with processing large numbers of biasing phrases through conventional methods.
Data Source
AI summary
A method includes receiving context biasing data that includes a set of unspoken textual utterances corresponding to a particular context. The method also includes obtaining a list of carrier phrases associated with the particular context. For each respective unspoken textual utterance, the method includes generating a corresponding training data pair that includes the respective unspoken textual utterance and a carrier phrase. For each respective training data pair, the method includes tokenizing the respective training data pair into a sequence of sub-word units, generating a first higher order textual feature representation for a corresponding sub-word unit, receiving the first higher order textual feature representation, and generating a first probability distribution over possible text units. The method also includes training a speech recognition model based on the first probability distribution over possible text units.