Speech Recognition Training Data Sorting for Phoneme Duration Balance
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing speech recognition systems, particularly those using recurrent neural network transducers (RNN-T), face challenges in efficiently training data sequences due to imbalanced phoneme durations, leading to acoustically difficult utterances and latency issues, especially in real-time applications.
Innovation Solution
A method is introduced to sort training data for RNN-T models by calculating an evenness score based on the variance of phoneme durations, dividing phonemes into three hidden states, and sorting sentences accordingly to improve data organization and acoustical ease.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If training data is processed without sorting by phoneme duration evenness, then training process is simpler and faster to initialize, but training efficiency deteriorates and model convergence slows down due to imbalanced phoneme durations
Solution Approach 1:
The patent applies preliminary action by sorting training sentences according to phoneme duration evenness scores before the actual training process begins. This pre-processing step calculates duration statistics, computes evenness scores for each sentence, and arranges sentences in optimal order beforehand, so that when training starts, the model immediately benefits from balanced phoneme duration distribution without encountering the computational overhead during training itself.
Solution Approach 2:
The patent changes the parameter of data organization by transforming unsorted or randomly sorted sentences into systematically sorted sentences based on phoneme duration evenness. This parameter change from random to structured ordering based on acoustic properties directly improves training efficiency by ensuring balanced representation of different phoneme durations throughout the training process.
2Speed
If real-time voice transcription is implemented without optimized data sorting, then system responsiveness is maintained, but latency increases and computational demands exceed acceptable thresholds
Solution Approach 1:
For real-time transcription, the patent applies preliminary action by pre-sorting the training data according to phoneme duration evenness before deployment. This ensures that when the model processes live speech input, it encounters phonemes in an optimal sequence that minimizes computational latency, allowing real-time performance without sacrificing speed.
Solution Approach 2:
The patent optimizes the parameter of data sequence organization by changing from unoptimized ordering to phoneme duration-based sorting. This parameter change reduces the computational time required for each transcription task while maintaining real-time responsiveness, effectively reducing latency without compromising processing speed.
3Adaptability or versatility
If phoneme durations are imbalanced in training sequences, then data diversity is maintained, but acoustic difficulty increases making training more challenging
Solution Approach 1:
The patent addresses acoustic difficulty by changing the parameter of sentence ordering based on phoneme duration evenness scores. This parameter change ensures that sentences with more balanced phoneme durations are prioritized, reducing the acoustic difficulty of training sequences while still maintaining data diversity through the inclusion of various phoneme types and speech patterns.
Solution Approach 2:
The patent applies local quality by optimizing specific local characteristics (phoneme duration evenness within each sentence) rather than changing the overall data composition. By improving the local quality of individual sentences through selective sorting, the overall acoustic difficulty of training is reduced while preserving the global diversity of the dataset.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
A computer-implemented method for preparing training data for a speech recognition model is provided including obtaining a plurality of sentences from a corpus, dividing each phoneme in each sentence of the plurality of sentences into three hidden states, calculating, for each sentence of the plurality of sentences, a score based on a variation in duration of the three hidden states of each phoneme in the sentence, and sorting the plurality of sentences by using the calculated scores.