AI TTS Phoneme Duration Control for Intelligible Speech Timing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional AI text-to-speech systems struggle to maintain intelligibility of generated speech at higher and lower speaking rates compared to regular rates, leading to unintelligible output when attempting to match the time duration of output speech with the original speech.
Innovation Solution
A total duration-aware model is trained to predict frame durations for phonemes, using a target output speech time duration as an input, employing techniques like MaskGIT-style decoding to generate output speech with high clarity and speaker similarity across varying speaking rates.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If conventional AI TTS systems generate speech at higher or lower speaking rates, then the system can adjust speaking speed, but the intelligibility of generated output speech deteriorates
Solution Approach 1:
The patent applies parameter changes by modifying the duration parameters of individual phonemes based on the target speech rate. The model adjusts phoneme duration probabilities dynamically according to whether the target rate is higher or lower than regular speaking rate, enabling the system to maintain intelligibility across different speaking speeds by optimizing temporal parameters for each phoneme segment.
2Duration of action of moving object
If conventional AI TTS systems attempt to match output speech duration with original speech duration, then time alignment is achieved, but speech intelligibility deteriorates
Solution Approach 1:
The patent segments the speech into individual phonemes and applies different duration control strategies to each phoneme segment. Instead of uniformly stretching or compressing the entire speech utterance to match target duration, the model independently adjusts the duration of each phoneme based on its acoustic properties and the target speaking rate, thereby maintaining intelligibility while achieving time alignment.
Solution Approach 2:
The model dynamically changes the duration parameter of each phoneme based on the target speech rate. When the target rate differs from regular speaking rate, the system recalculates optimal phoneme durations to preserve acoustic characteristics and intelligibility, rather than applying uniform time scaling that would degrade speech quality.
3Manufacturing precision
If the AI duration model predicts frame durations for all phonemes, then complete duration control is achieved, but training complexity increases
Solution Approach 1:
The patent applies local quality by treating different phonemes with different duration prediction strategies based on their specific acoustic characteristics. The model identifies which phonemes require precise duration control and prioritizes training for those phonemes, rather than uniformly applying the same training intensity to all phonemes. This selective approach maintains high duration control precision for critical phonemes while reducing overall training complexity.
Data Source
AI summary
Systems and methods are provided for training and using a total duration-aware (TDA) model to control the duration of speech utterances by a text-to-speech computing system when converting text into speech. During use, text to be converted into speech and target output speech time duration are used as inputs into the TDA model. The text is then tokenized into phonemes, and the TDA model predicts frame durations for each phoneme. The TDA model is trained on phonemes derived from text, corresponding actual frame durations for the phonemes, and a target output speech time duration. The TDA model masks a subset of the actual frame durations, and generates predicted frame durations for the subset. A loss between the actual and predicted frame durations is calculated, and used to adjust parameters of the TDA model to control future generation of predicted frame durations.