End-to-End Text-to-Speech Synthesis Using Attention Weights
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing text-to-speech (TTS) synthesis systems face challenges such as low robustness, unnatural-sounding speech, complexity in model training, and high latency during inference, leading to suboptimal performance in generating human-like speech.
Innovation Solution
An apparatus and method for end-to-end TTS synthesis using a trained machine-learning model that maps phonemes to states using an encoder sub-model, estimates target durations, determines attention weights, and maps states to audio data using a decoder sub-model, thereby generating an audio waveform representing speech.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If traditional multi-stage TTS systems are used, then speech synthesis can be achieved, but the system complexity increases and training becomes difficult
Solution Approach 1:
The patent combines multiple independent models (phoneme duration prediction, pitch prediction, spectral envelope prediction, and vocoder) into a single integrated TTS system. This unified architecture reduces training complexity by eliminating the need to train and coordinate multiple separate models, while maintaining comprehensive speech synthesis functionality through the coordinated work of encoder, duration predictor, pitch predictor, spectral predictor, and vocoder components.
2Reliability
If traditional TTS systems with multiple intermediate steps are used, then speech synthesis can be achieved, but the quality becomes unnatural
Solution Approach 1:
The patent employs an encoder to extract comprehensive features from input text before the main synthesis process. This preliminary feature extraction captures semantic, phonetic, and prosodic information in advance, enabling the subsequent duration predictor, pitch predictor, and spectral predictor to generate more natural speech parameters without requiring complex intermediate processing steps.
3Productivity
If fast inference is implemented in TTS systems, then latency is reduced, but speech quality may deteriorate
Solution Approach 1:
The patent segments the speech synthesis process into distinct functional modules: encoder for feature extraction, duration predictor for timing control, pitch predictor for prosody, spectral predictor for acoustic features, and vocoder for waveform generation. This segmentation enables parallel processing of different speech parameters, accelerating inference while maintaining quality through specialized optimization of each module.
Solution Approach 2:
The encoder performs preliminary feature extraction from the input text, generating comprehensive representations that include phonetic, semantic, and prosodic information. This pre-processing step prepares optimized feature vectors that can be quickly processed by the subsequent prediction modules, reducing overall inference latency without compromising speech quality.
Data Source
AI summary
An apparatus for end-to-end text-to-speech synthesis is provided. The apparatus comprises input interface circuitry configured to receive first input data indicative of a phoneme and second input data indicative of a first target duration for the phoneme. The apparatus further comprises processing circuitry configured to, using a trained machine-learning model, map the phoneme to a state using an encoder sub-model of the trained machine-learning model, estimate a second target duration for the phoneme based on the state and determine an attention weight based on the first target duration and the second target duration. The processing circuitry is further configured to map the state to audio data based on the attention weight using a decoder sub-model of the trained machine-learning model, wherein the audio data are indicative of an audio waveform representing speech.