Adversarial Feedforward Speech Synthesis for Realistic Fast Audio
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing neural network models for generating audio data, such as autoregressive and invertible feedforward networks, are computationally expensive and require extensive resources, while existing feedforward networks lack the ability to generate realistic audio samples without explicitly modeling data distributions.
Innovation Solution
A feedforward generative neural network trained adversarially using both conditional and unconditional discriminators, with dilated convolutional layers, to generate high-quality audio samples in a single pass, reducing computational requirements and improving realism.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If autoregressive neural networks are used to generate audio data, then the quality of generated audio samples is improved, but the computational complexity and time required increase significantly
Solution Approach 1:
The patent inverts the traditional approach by using a feedforward network (non-autoregressive) instead of an autoregressive network, achieving both high audio quality and reduced computational complexity through the adversarial training framework with conditional discriminators
Solution Approach 2:
The patent changes the network architecture parameters from autoregressive to feedforward structure, and introduces conditional discriminators with specific parameter configurations (window sizes, dilation rates) to maintain audio quality while reducing computational burden
2Manufacturing precision
If invertible feedforward neural networks are used to generate speech signals, then the realism of generated audio is improved, but the training complexity and resource requirements increase
Solution Approach 1:
The patent extracts the essential function of generating realistic audio from the complex invertible feedforward network framework, achieving the same realism goal through a simpler feedforward network combined with conditional adversarial training, removing unnecessary distillation and invertibility requirements
Solution Approach 2:
The patent uses a computationally lighter feedforward network architecture that can be trained efficiently with adversarial loss, replacing the resource-intensive invertible network approach while maintaining audio realism through the discriminator-guided training process
3Productivity
If feedforward neural networks are used to generate audio, then the generation speed is improved, but the ability to generate realistic audio samples without explicit data distribution modeling is lost
Solution Approach 1:
The patent introduces conditional discriminators as intermediary components that guide the feedforward network to generate realistic audio samples. The discriminators act as mediators between the generator and the audio quality requirement, enabling single-pass generation without explicit data distribution modeling
Solution Approach 2:
The adversarial training framework implements feedback mechanisms where discriminators evaluate generated audio samples and provide gradient feedback to the generator, enabling the feedforward network to learn realistic audio generation patterns while maintaining fast single-pass generation capability
Data Source
Figure 1
Figure 2
Figure 3
AI summary
Methods, systems, and apparatus, including computer programs encoded on computer storage media, for generating output audio examples using a generative neural network. One of the methods includes obtaining a training conditioning text input; processing a training generative input comprising the training conditioning text input using a feedforward generative neural network to generate a training audio output; processing the training audio output using each of a plurality of discriminators, wherein the plurality of discriminators comprises one or more conditional discriminators and one or more unconditional discriminators; determining a first combined prediction by combining the respective predictions of the plurality of discriminators; and determining an update to current values of a plurality of generative parameters of the feedforward generative neural network to increase a first error in the first combined prediction.