ASR Language Model Training With Pseudo-Random Encoder Features
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional automatic speech recognition (ASR) models struggle with accurately recognizing rare words, proper names, and user-specific vocabularies due to limited training data and computational constraints, particularly in mobile devices, and often require paired text and audio data for effective end-to-end training.
Innovation Solution
Training the internal language model of an ASR model using pseudo-random encoder variables generated from training text samples without corresponding audio data, leveraging a sequence transduction model to improve recognition of rare words and user-specific vocabularies.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If conventional ASR models are trained with paired audio-text data, then speech recognition accuracy is improved, but data requirements and computational complexity increase
Solution Approach 1:
The patent uses text samples as copies or substitutes for paired audio-text training data. By processing text samples through the encoder and decoder without requiring actual audio recordings, the system creates a simplified training approach that maintains recognition accuracy while eliminating the need for large quantities of paired audio data
Solution Approach 2:
The patent extracts and isolates the text processing component from the full audio processing pipeline. By removing the audio input requirement and working directly with text samples through the encoder-decoder architecture, the system reduces data requirements while preserving the core speech recognition functionality
2Adaptability or versatility
If ASR models are trained on mobile devices with computational constraints, then deployment flexibility is improved, but model training complexity and resource requirements worsen
Solution Approach 1:
The patent employs text samples as simplified proxies for full audio training data, enabling model training on resource-constrained mobile devices. This copying approach reduces the computational burden while maintaining the ability to train effective ASR models in decentralized environments
Solution Approach 2:
The patent segments the training process into independent text-sample-based iterations that can be performed separately on mobile devices. By breaking down the complex audio processing into discrete text processing steps through the encoder-decoder framework, the system enables distributed training across multiple devices with limited resources
3Reliability
If paired audio-text data is used for end-to-end training, then model performance is improved, but data preparation time and processing overhead increase
Solution Approach 1:
The patent extracts the essential text processing function from the complete audio-text pairing process. By removing the audio data collection, synchronization, and preprocessing steps while retaining the core encoder-decoder text processing capability, the system maintains model performance while eliminating time-consuming data preparation overhead
Solution Approach 2:
The patent uses text samples as direct copies of the target output without requiring corresponding audio recordings. This approach eliminates the time-intensive process of collecting, aligning, and preprocessing paired audio-text data while preserving the ability to train high-performance ASR models through text-only training iterations
Data Source
AI summary
A method includes obtaining a training text sample, the training text sample not paired with corresponding audio data, and generating a sequence of pseudo-random encoder variables. The method also includes processing, using a decoder of a sequence transduction model, the sequence of pseudo-random encoder variables to predict a probability distribution over possible output labels. The method further includes determining a loss based metric based on the training text sample and the predicted probability distribution over possible output labels, and training the decoder based on the loss metric.