Parallel Neural Text-to-Speech Speed-Up
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Traditional text-to-speech systems based on autoregressive models are slow at synthesis due to their sequential nature, despite advancements in neural network architectures like WaveNet and ClariNet, which still rely on autoregressive components for predicting frame-level acoustic features.
Innovation Solution
A non-autoregressive text-to-spectrogram model, referred to as ParaNet, is introduced, which is fully convolutional and converts text to mel spectrogram iteratively refining attention alignment in a layer-by-layer manner, combined with an inverse autoregressive flow-based neural vocoder for parallel waveform synthesis, enabling a single feed-forward pass for speech generation.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If autoregressive models are used for text-to-speech synthesis, then speech quality can be maintained, but synthesis speed is slow due to sequential processing
Solution Approach 1:
The model segments the speech synthesis process into multiple parallel streams, each handling different aspects of speech generation simultaneously. This allows the system to process multiple speech features in parallel rather than sequentially, achieving speed-up while maintaining quality
Solution Approach 2:
The patent transitions from sequential autoregressive processing to parallel processing by introducing a non-autoregressive architecture that operates in a different computational dimension. This enables all time steps to be processed simultaneously through a single feed-forward pass, resolving the speed-efficiency contradiction
2Speed
If non-autoregressive parallel processing is implemented, then synthesis speed improves, but attention alignment accuracy may deteriorate
Solution Approach 1:
The model incorporates feedback mechanisms where the output of one parallel stream informs and refines the attention alignment for other streams. This iterative refinement within the parallel architecture maintains attention accuracy while preserving the speed benefits of non-autoregressive processing
Solution Approach 2:
The patent introduces intermediary attention alignment mechanisms that mediate between the parallel processing streams, ensuring that attention distributions remain accurate despite the lack of sequential dependencies. These intermediaries coordinate the parallel streams to maintain precision
Data Source
AI summary
Presented herein are embodiments of a non-autoregressive sequence-to-sequence model that converts text to an audio representation. Embodiment are fully convolutional, and a tested embodiment obtained about 46.7 times speed-up over a prior model at synthesis while maintaining comparable speech quality using a WaveNet vocoder. Interestingly, a tested embodiment also has fewer attention errors than the autoregressive model on challenging test sentences. In one or more embodiments, the first fully parallel neural text-to-speech system was built by applying the inverse autoregressive flow (IAF) as the parallel neural vocoder. System embodiments can synthesize speech from text through a single feed-forward pass. Also disclosed herein are embodiments of a novel approach to train the IAF from scratch as a generative model for raw waveform, which avoids the need for distillation from a separately trained WaveNet.