ASR Context Biasing for Personalized and Trending Terms
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing speech recognition systems struggle to accurately recognize personalized terms and trending entities that are not included in their training data, such as contact names and device names, leading to reduced accuracy in spoken language understanding.
Innovation Solution
Incorporating a language model (LM) to generate contextual information for automatic speech recognition (ASR) using attention-based biasing layers, which enriches the ASR model's hidden representations with external knowledge, enhancing its ability to recognize personalized and trending terms without the need for retraining.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If speech recognition systems use traditional training data approaches, then they can recognize common terms, but they fail to accurately recognize personalized terms and trending entities not included in training data
Solution Approach 1:
The system performs preliminary action by using a language model to generate context information about personalized terms and trending entities before the speech recognition process begins. This pre-generated context is then integrated into the ASR model to improve recognition accuracy without requiring retraining on new data.
Solution Approach 2:
A language model acts as an intermediary between the available training data and the speech recognition system. The LM generates contextual information that bridges the gap between static training data and dynamic personalized/trending terms, enabling the ASR model to recognize terms it was not explicitly trained on.
2Measurement precision
If speech recognition systems are retrained to recognize new terms, then recognition accuracy improves, but computational resources and time increase
Solution Approach 1:
The system performs preliminary action by using a language model to generate context information about personalized terms and trending entities before the speech recognition process begins. This pre-generated context is then integrated into the ASR model to improve recognition accuracy without requiring retraining on new data.
Solution Approach 2:
The patent substitutes the mechanical retraining process with a more efficient approach using language models. Instead of retraining the entire ASR model (which is computationally expensive and time-consuming), the system uses an LM to generate context that can be integrated into the existing model, replacing the need for full retraining while achieving similar or better results.
3Measurement precision
If speech recognition systems process more context information, then recognition of personalized terms improves, but computational complexity increases
Solution Approach 1:
The system extracts only the most relevant context information generated by the language model and integrates it into the ASR model. Rather than processing all possible context, the system selectively extracts and uses only the necessary contextual features, reducing computational complexity while maintaining recognition accuracy.
Solution Approach 2:
The patent applies local quality by enhancing specific parts of the ASR model that deal with personalized and trending terms, rather than uniformly increasing complexity across the entire system. The context information from the language model is targeted specifically at improving recognition of these particular term types.
Data Source
AI summary
Techniques for ASR processing using language model (LM)-generated context are described. A LM is prompted to generate words that are relevant for/may be included in a future user input. The prompt to the LM can include words from user interaction history, dialog history, dialog topic, user preferences, etc. The information included in the prompt may focus on rare or unique words rather than words that the ASR model is already confident in recognizing. The techniques can be plugged into an existing/pretrained ASR model and can be used with any existing/pretrained LM, thus saving resources needed to implement and maintain the components.