Dynamic Language Model Splicing for Speech Recognition Accuracy
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Hybrid Hidden Markov Model Deep Neural Network Automatic Speech Recognition (HMM-DNN ASR) systems face a tradeoff between transcription accuracy and efficiency due to the use of fixed-sized vocabularies, where adding domain-specific words improves accuracy but increases memory and decoding time requirements.
Innovation Solution
A dynamically generated language model, such as a Finite State Transducer (FST), is spliced into a static language model on a per-utterance or per-ASR session basis, allowing for the inclusion of Out-Of-Vocabulary (OOV) domain-specific words or speech hints, using placeholder arcs that can be replaced with dynamically added words or phrases during decoding.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If the vocabulary size is increased to include domain-specific words, then transcription accuracy is improved, but memory requirements and decoding time increase significantly
Solution Approach 1:
The vocabulary is segmented into a base vocabulary (included in the static language model) and domain-specific vocabulary (added dynamically at decoding time). This segmentation allows the system to maintain a compact base model while selectively incorporating domain-specific words only when needed, avoiding the memory and computational overhead of including all domain words in the base model.
Solution Approach 2:
The language model transitions from a static fixed vocabulary to a dynamic vocabulary that can be modified at decoding time. Domain-specific words are added to the vocabulary dynamically based on the specific transcription task, allowing the model to adapt its vocabulary size and composition to match the requirements of each utterance or session.
2Measurement precision
If domain-specific words are added to the vocabulary, then transcription accuracy for domain data is improved, but the efficiency and cost of the ASR system deteriorate
Solution Approach 1:
Placeholder arcs are prepared in advance in the static language model's Finite State Transducer (FST) structure. These placeholders represent potential domain-specific words but do not consume full memory or computational resources until actual domain words are inserted at decoding time. This preliminary setup enables rapid insertion of domain-specific vocabulary without requiring pre-compilation of the entire extended vocabulary.
Solution Approach 2:
The system changes the vocabulary parameter dynamically at decoding time rather than maintaining a fixed large vocabulary. By adjusting the vocabulary composition based on the specific domain requirements of each transcription task, the system achieves high accuracy for domain-specific data while maintaining efficient decoding speed for general purposes.
3Productivity
If a static language model with fixed vocabulary is used, then decoding efficiency is maintained, but the ability to handle Out-Of-Vocabulary tokens deteriorates
Solution Approach 1:
Placeholder arcs serve as intermediaries between the static language model and dynamic domain-specific vocabulary. These placeholders are embedded in the FST structure of the static model and act as insertion points for domain-specific words. The placeholders maintain the structural integrity and decoding efficiency of the static model while enabling the incorporation of OOV tokens through dynamic word insertion at decoding time.
Data Source
AI summary
Presented herein are techniques for augmenting a speech recognition engine. According to the disclosed techniques, audio data is obtained as part of an automatic speech recognition session. Speech hints are also obtained as part of the automatic speech recognition session. A dynamic language model is generated from the speech hints for use during the automatic speech recognition session. A combined language model is then generated from the dynamic language model and a static language model. Finally, the audio data is converted to text using the combined language model as part of the automatic speech recognition session.