Rare-Word Language Model Selection for Streaming Speech Recognition
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional end-to-end automatic speech recognition (ASR) models struggle with recognizing rare words and long-tail proper nouns due to limited training data and lack of future context, particularly in streaming applications, leading to high failure rates.
Innovation Solution
A single E2E ASR model combined with an on-device neural language model, utilizing a data selection pipeline to train a sufficient subset of data, and a cascaded encoder structure that enables both streaming and non-streaming modes, improving rare word recognition through an external language model.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If a single E2E ASR model is used for streaming recognition, then recognition speed is improved, but rare word recognition accuracy deteriorates
Solution Approach 1:
The system segments the recognition task into two parts: a streaming E2E ASR model for real-time transcription and an external language model for rare word enhancement. The E2E model processes audio streams quickly while the external LM selectively improves rare word recognition based on confidence scores, resolving the contradiction between speed and accuracy.
Solution Approach 2:
An external language model acts as an intermediary between the E2E ASR model and the final output. It receives transcripts from the E2E model, identifies low-confidence rare words, and applies language model scores to improve recognition accuracy without significantly impacting the overall recognition speed.
2Productivity
If training data is limited to available transcripts, then training efficiency is improved, but rare word coverage deteriorates
Solution Approach 1:
The system performs preliminary action by pre-training an external language model on large-scale text corpora containing rare words and long-tail proper nouns before deployment. This pre-training ensures comprehensive rare word coverage is available when needed, without compromising the efficiency of the main ASR training process.
Solution Approach 2:
The system changes the training parameters by using different data sources for different model components. The E2E model trains on audio-transcript pairs for efficiency, while the external language model trains on large-scale text corpora to maximize rare word coverage, allowing each component to optimize for its specific purpose.
3Device complexity
If no external language model is used, then system complexity is reduced, but rare word recognition fails
Solution Approach 1:
Instead of uniformly increasing system complexity, the invention applies local quality by adding an external language model component specifically targeted at rare word enhancement. The system maintains simple E2E architecture for common words while introducing complexity only where needed for rare word recognition, optimizing the complexity-accuracy tradeoff.
Data Source
Figure 1A
Figure 1B
Figure 2
AI summary
A method of training a language model (206) for rare-word speech recognition includes obtaining a set of training text samples (422), and obtaining a set of training utterances (452) used for training a speech recognition model. Each training utterance in the plurality of training utterances includes audio data (454) corresponding to an utterance and a corresponding transcription (456) of the utterance. The method also includes applying rare word filtering on the set of training text samples to identify a subset of rare-word training text samples (432) that include words that do not appear in the transcriptions from the set of training utterances or appear in the transcriptions from the set of training utterances less than a threshold number of times. The method further includes training the external language model on the transcriptions from the set of training utterances and the identified subset of rare-word training text samples.