RNN-T Training Sequence for Balanced English Dialect Recognition
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing RNN-T models for speech recognition are inefficient for real-time voice transcription due to non-streaming capabilities and are not conducive to on-device hosting due to large search graphs, making them vulnerable to latency and reliability issues in remote connections.
Innovation Solution
A method for organizing training data sequences by positioning similar sentences with different dialects closely while imposing a weak constraint on audio length, using a similarity-score dependent penalty to control sentence variety, facilitating the construction of a global English model that balances dialects and reduces biased text prediction.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If RNN-T models use large search graphs for accurate speech recognition, then recognition accuracy is improved, but the model size increases making on-device hosting difficult
Solution Approach 1:
The patent extracts and removes unnecessary elements from the search graph to reduce model size while preserving essential recognition capabilities. By selectively removing redundant states and transitions, the model can achieve accurate speech recognition with a compact representation suitable for on-device deployment.
Solution Approach 2:
The patent applies local quality by optimizing different parts of the search graph with different levels of detail. Frequently accessed or critical paths in the search graph are maintained with high precision, while less critical areas are simplified or approximated, achieving a balance between accuracy and model size for on-device hosting.
2Measurement precision
If RNN-T models process entire input sequences before generating output, then recognition accuracy is improved, but real-time streaming capability is lost
Solution Approach 1:
The patent segments the input sequence processing into smaller, manageable chunks that can be processed incrementally. By dividing the sequence into frames or segments and processing them in order, the model can generate intermediate outputs during processing, enabling real-time streaming while maintaining overall recognition accuracy through proper segmentation and contextual integration.
Solution Approach 2:
The patent applies preliminary action by pre-processing or pre-computing certain aspects of the input sequence before full processing begins. This may include preliminary feature extraction, preliminary alignment, or pre-computation of certain model components, allowing the main processing to proceed more quickly and enable real-time output generation.
3Ease of manufacture
If training data uses random or length-based sorting, then training simplicity is maintained, but dialect balance and model performance degrade
Solution Approach 1:
The patent applies preliminary action by pre-sorting or pre-organizing the training data according to dialect characteristics and sentence similarity before the actual training process begins. This preliminary organization ensures that the training data is balanced across different dialects and structured to maximize learning efficiency, improving model performance without significantly increasing training complexity.
Solution Approach 2:
The patent changes the sorting parameter from simple random or length-based criteria to a more sophisticated similarity-based metric that considers dialect characteristics. By using sentence similarity as the sorting parameter, the training data is automatically balanced across dialects while maintaining a relatively simple sorting process that doesn't significantly complicate the training pipeline.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
A computer-implemented method for preparing training data for a speech recognition model is provided including obtaining a plurality of audio data sets, each audio data set having a different acoustic feature and sorting sentences from the plurality of audio data sets so that similar sentences from different audio data sets are positioned closely, while imposing a weak constraint on audio length, to train the speech recognition model.