Non-autoregressive Speech Synthesis Using Multiple Decoders
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing autoregressive speech synthesis technologies are slow due to sequential output generation and suffer from reduced sound magnitude and improper end-of-speech detection, degrading sound quality and generation speed.
Innovation Solution
A deep neural network-based non-autoregressive speech synthesis system using multiple decoders generates a speech feature vector sequence from a template with temporal information, allowing simultaneous synthesis of all frames and maintaining consistent sound magnitude through attention mechanisms and convolutional networks.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of information
If autoregressive flow is used for speech synthesis, then information delivery is efficient, but generation speed is slow because outputs need to be sequentially generated
Solution Approach 1:
The speech synthesis process is segmented into multiple independent decoding stages. Each decoder processes different portions of the speech feature vector simultaneously, breaking the sequential dependency while maintaining information flow integrity. This allows parallel computation of speech features that would otherwise require sequential autoregressive generation.
Solution Approach 2:
The patent introduces a temporal dimension through template data that provides time-position information for each speech feature. By adding this temporal dimension, the system can determine which speech features to generate and in what order, enabling non-autoregressive parallel generation while preserving the sequential structure needed for coherent speech output.
2Ease of operation
If autoregressive speech synthesis is used, then speech can be generated step-by-step, but sound magnitude is reduced toward the rear side and end-of-speech detection becomes difficult
Solution Approach 1:
Template data is prepared in advance containing time-position information for all speech features. This preliminary action allows the system to know the total speech duration and structure before generation begins, enabling consistent sound magnitude across all speech features and accurate end-of-speech detection without the degradation problems of autoregressive methods.
3Productivity
If non-autoregressive method is used for speech synthesis, then generation speed is improved, but sound quality degrades due to inability to use previous outputs as inputs
Solution Approach 1:
Template data acts as an intermediary that carries time-position information between the input text and the speech feature generation. This intermediary enables non-autoregressive parallel generation while maintaining the structural relationships needed for high-quality speech synthesis, bridging the gap between speed and quality.
Solution Approach 2:
The patent replaces the mechanical sequential dependency of autoregressive methods with a data-driven approach using template data. Instead of relying on previous outputs as inputs, the system uses pre-computed temporal information to guide parallel feature generation, substituting the sequential mechanical process with a more efficient data-directed approach.
Data Source
AI summary
Proposed are a deep neural network-based non-autoregressive voice synthesizing method and a system therefor. A deep neural network-based non-autoregressive voice synthesizing system according to an embodiment may comprise: a voice feature vector column synthesizing unit which constitutes a non-recursive deep neural network based on multiple decoders, and gradually produces a voice feature vector column through the multiple decoders from a template including temporal information of a voice; and a voice reconstituting unit which transforms the voice feature vector column into voice data, wherein the voice feature vector column synthesizing unit produces a template input, and produces a voice feature vector column by adding, to the template input, sentence data refined through an attention mechanism.