Multilingual ASR Acoustic Model Transliteration Augmentation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Low resource settings pose challenges for training multilingual acoustic models in speech recognition systems, resulting in high word error rates despite attempts at data augmentation and manual intervention.
Innovation Solution
A computer-implemented method involving training a multilingual network on low resource data, performing transliteration, applying a filtering metric to select enhanced data for retraining, and augmenting the training data to improve model accuracy and diversity.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If data augmentation is applied to clean speech using traditional methods, then training data quantity increases, but word error rate remains relatively high
Solution Approach 1:
The patent introduces transliterated data as an intermediary between original speech data and training data. By converting speech to text and back to speech through transliteration, the system creates augmented training samples that preserve acoustic characteristics while increasing data diversity, thereby improving both quantity and quality of training data
Solution Approach 2:
The patent applies transliteration operations that change the linguistic parameters of training data. By transforming speech through different script systems and back, the system generates varied training examples that maintain acoustic integrity while introducing useful variability, leading to lower word error rates
2Adaptability or versatility
If more transcribed data from other languages is used to build multilingual acoustic models, then model versatility improves, but processing complexity increases
Solution Approach 1:
The patent segments the multilingual training process into distinct stages: original data processing, transliteration generation, filtering, and augmented data integration. This segmentation allows each component to be optimized independently, managing complexity while maintaining versatility
Solution Approach 2:
The patent performs preliminary transliteration and filtering operations before final model training. By pre-processing data through transliteration pipelines and applying filtering metrics in advance, the system reduces the complexity of the main training process while ensuring data quality
3Adaptability or versatility
If transliteration and data augmentation are performed on low resource training data, then data diversity increases, but computational resources required increase
Solution Approach 1:
The patent applies partial transliteration and augmentation only to the most critical portions of training data that benefit most from this processing. By selectively applying transliteration rather than to all data uniformly, the system achieves sufficient data diversity while controlling computational resource consumption
Solution Approach 2:
The patent creates copies of original training data through transliteration processes. By generating transliterated versions of key training examples and using these copies for augmented training, the system increases data diversity without requiring proportional increases in computational resources for original data processing
Data Source
AI summary
A computer-implemented method of building a multilingual acoustic model for automatic speech recognition in a low resource setting includes training a multilingual network on a set of training languages with an original transcribed training data to create a baseline multilingual acoustic model. Transliteration of transcribed training data is performed by processing through the multilingual network a plurality of multilingual data types from the set of languages, and outputting a pool of transliterated data. A filtering metric is applied to the pool of transliterated data output to select one or more portions of the transliterated data for retraining of the acoustic model. Data augmentation is performed by adding one or more selected portions of the output transliterated data back to the original transcribed training data to update training data. The training of a new multilingual acoustic model through the multilingual network is performed using the updated training data.