Parametric Vocoder Parameter Prediction from Prosodic Features
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing text-to-speech (TTS) models, particularly parametric vocoders, struggle to effectively model prosody, leading to synthesized speech that lacks expressiveness as they require additional vocoder parameters beyond prosodic features like pitch, energy, and phoneme duration.
Innovation Solution
A method and system that predict parametric vocoder parameters from prosodic features using a combination of a prosody model and a neural network vocoder model, where the prosody model predicts initial prosodic features and the vocoder model generates the remaining required parameters, such as Mel-cepstrum coefficients and voicing components, to drive the parametric vocoder for improved prosody in synthesized speech.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If traditional concatenative and parametric synthesis models are used, then intelligible speech can be generated, but the synthesized speech lacks expressiveness due to ineffective prosody modeling
Solution Approach 1:
The system segments the speech synthesis process into distinct modules: a prosody model that generates prosodic features (duration, pitch contour, energy contour) and a vocoder model that converts these features into vocoder parameters. This segmentation allows each module to specialize in specific aspects of speech synthesis, with the prosody model focusing on expressive prosodic features and the vocoder model handling the conversion to final speech parameters, thereby resolving the contradiction between maintaining intelligibility and improving expressiveness.
2Ease of manufacture
If variational autoencoders are used to predict prosodic features, then prosody modeling is improved, but the predicted features are insufficient for driving parametric vocoders which require additional vocoder parameters
Solution Approach 1:
The prosody model performs preliminary action by predicting prosodic features (duration, pitch contour, energy contour) in advance, which are then used as input to the vocoder model. The vocoder model subsequently generates the additional vocoder parameters (Mel-cepstrum coefficients, aperiodicity, voicing) that are necessary for driving the parametric vocoder. This preliminary action approach ensures that all necessary parameters are prepared before final speech synthesis, resolving the information gap between prosody prediction and vocoder requirements.
3Ease of operation
If prosodic features alone are provided to a parametric vocoder, then the system is simple to operate, but the vocoder cannot generate speech with proper prosody due to insufficient parameters
Solution Approach 1:
The vocoder model serves as an intermediary between the prosody model and the parametric vocoder. It receives prosodic features from the prosody model and transforms them into complete vocoder parameters by predicting additional features (Mel-cepstrum coefficients, aperiodicity, voicing) that the parametric vocoder requires. This intermediary role maintains system simplicity while ensuring prosody accuracy, as the vocoder model automatically generates all necessary parameters without requiring complex manual configuration.
Data Source
AI summary
A method for predicting parametric vocoder parameter includes receiving a text utterance having one or more words, each word having one or more syllables, and each syllable having one or more phonemes. The method also includes receiving, as input to a vocoder model, prosodic features that represent an intended prosody for the text utterance and a linguistic specification. The prosodic features include a duration, pitch contour, and energy contour for the text utterance, while the linguistic specification includes sentence-level linguistic features, word-level linguistic features for each word, syllable-level linguistic features for each syllable, and phoneme-level linguistic features for each phoneme. The method also includes predicting vocoder parameters based on the prosodic features and the linguistic specification. The method also includes providing the predicted vocoder parameters and the prosodic features to a parametric vocoder configured to generate a synthesized speech representation of the text utterance having the intended prosody.