Iterative Adaptive Language Model Training for Speech Recognition
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The manual transcription of speech utterances for developing classification models in speech recognition applications is burdensome and increases development expenses, as it requires significant human effort and time.
Innovation Solution
A training system that employs an iterative process to generate an adaptive language model using multiple sets of training data, where speech utterances are recognized in each iteration to create an adaptive language model for subsequent iterations, eliminating the need for manual transcription and improving model accuracy over time.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If manual transcription is used for speech utterances, then transcription accuracy is improved, but development time and cost increase
Solution Approach 1:
The training data is divided into multiple subsets that are processed in iterative batches. Each subset is used to train a language model component, which is then combined with previous models to form an improved overall model. This segmentation allows automated processing of large volumes of speech data without requiring manual transcription of the entire dataset.
Solution Approach 2:
The system uses automatically transcribed speech data from prior iterations to train improved language models, which then transcribe new speech data with higher accuracy. The system serves itself by using its own output from previous iterations as input for subsequent iterations, eliminating the need for continuous manual transcription while progressively improving accuracy.
2Reliability
If manual transcription is used for all training data, then model training quality is improved, but development expense increases
Solution Approach 1:
A small initial subset of speech data is manually transcribed to create a foundational language model. This preliminary manual transcription provides the base quality needed to automatically transcribe subsequent larger datasets, thereby limiting manual effort to only the essential initial training data while maintaining model training quality.
Solution Approach 2:
The system creates multiple copies and variations of the training data through iterative processing, where each iteration uses the transcribed data from previous iterations to generate improved language models. This allows the system to effectively multiply the value of the initial manually transcribed data without incurring additional manual transcription costs.
3Device complexity
If a single language model is used, then device complexity is reduced, but adaptability to different speech patterns decreases
Solution Approach 1:
The language model is designed to be dynamic and adaptive, evolving through multiple iterations where it incorporates patterns learned from different speech subsets. The model structure allows it to adapt its parameters and characteristics based on the specific speech patterns encountered in each training iteration, enabling versatility without requiring multiple separate fixed models.
Solution Approach 2:
A single language model is designed to perform multiple functions by processing different subsets of training data and adapting to various speech patterns, accents, and domains. Through iterative training with diverse data subsets, the universal model achieves the adaptability of multiple specialized models while maintaining a unified structure that reduces overall system complexity.
Data Source
AI summary
A training system for a speech recognition application is disclosed. In embodiments described, the training system is used to train a classification model or language model. The classification model is trained using an adaptive language model generated by an iterative training process. In embodiments described, the training data is recognized by the speech recognition component and the recognized text is used to create the adaptive language model which is used for speech recognition in a following training iteration.