Trie-Based Deep Biasing for ASR Word Error Rate Reduction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing end-to-end automatic speech recognition (ASR) systems face challenges in generalizing to open-domain scenarios, scaling with large biasing lists, and performing well on rare long-tail words, with previous solutions either not being modular or requiring re-training.
Innovation Solution
The integration of shallow fusion, trie-based deep biasing, and neural network language model contextualization, which allows the ASR model to access biasing information implicitly and leverage unpaired text data without the need for strong context prefixes or domain-specific patterns, resulting in a lightweight and modular system.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If existing contextual biasing approaches are used, then some improvement in recognition accuracy may be achieved, but the system lacks modularity and requires re-training for modifications
Solution Approach 1:
The system is divided into distinct modular components: the ASR model, the language model, and the biasing module. Each component can be independently trained, modified, and updated without affecting the others, eliminating the need for full system re-training when making modifications.
Solution Approach 2:
A biasing module acts as an intermediary between the language model and the ASR model. This mediator processes contextual information and generates bias embeddings that are integrated into the ASR pipeline, allowing flexible modifications without re-training the core ASR model.
2Measurement precision
If strong context prefixes or domain-specific patterns are required, then recognition accuracy for specific domains improves, but the system cannot generalize to open-domain scenarios
Solution Approach 1:
The language model is trained on diverse, unpaired text data across multiple domains, enabling it to generate contextual biases that work effectively for both domain-specific and open-domain scenarios. This universal approach eliminates the need for domain-specific adaptations.
Solution Approach 2:
The system dynamically adjusts bias embeddings based on the input context and previously emitted tokens, rather than relying on fixed domain-specific parameters. This allows the system to adapt to different domains and contexts without re-training.
3Measurement precision
If the ASR model is trained extensively for every user scenario, then prediction accuracy for custom vocabularies improves, but the system becomes cumbersome and requires frequent re-training
Solution Approach 1:
The language model is pre-trained on extensive unpaired text data to learn general linguistic patterns and contextual relationships. This preliminary training provides a strong foundation that works across multiple scenarios without requiring user-specific re-training.
Solution Approach 2:
The system automatically adapts to different user scenarios by leveraging the pre-trained language model's ability to generate contextual biases from unpaired text data, eliminating the need for manual re-training for each user or scenario.
4Device complexity
If traditional hybrid systems are used, then modularity is maintained, but performance on rare long-tail words deteriorates
Solution Approach 1:
The system merges the strengths of hybrid systems (modularity) with end-to-end approaches (performance on rare words) by integrating a pre-trained language model with the ASR pipeline. The language model provides contextual biases that significantly improve rare word recognition while maintaining system modularity.
Data Source
AI summary
In one embodiment, a method includes receiving a user's utterance comprising a word in a custom vocabulary list of the user, generating a previous token to represent a previous audio portion of the utterance, and generating a current token to represent a current audio portion of the utterance by generating a bias embedding by using the previous token to query a trie of wordpieces representing the custom vocabulary list, generating first probabilities of respective first candidate tokens likely uttered in the current audio portion based on the bias embedding and the current audio portion, generating second probabilities of respective second candidate tokens likely uttered after the previous token based on the previous token and the bias embedding, and generating the current token to represent the current audio portion of the utterance based on the first probabilities of the first candidate tokens and the second probabilities of the second candidate tokens.