Speech Synthesis Using Complex Neural Networks
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current speech synthesis technologies, such as WaveNet, are slow due to their autoregressive nature and the need to generate numerous sampling points, leading to complex and inefficient processes.
Innovation Solution
A computer-implemented method for speech synthesis using a complex neural network that includes a downsampling and upsampling network, processing Mel spectra to generate both real and imaginary components, which are then combined using inverse short-time Fourier transforms to produce synthetic speech, reducing complexity and improving efficiency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If WaveNet autoregressive model is used for speech synthesis, then speech quality is improved, but synthesis speed becomes slow
Solution Approach 1:
The speech spectrum is segmented into multiple frequency bands (e.g., critical bands or mel bands), and each band is processed independently through separate neural network branches. This allows parallel computation of different frequency components, significantly speeding up synthesis while maintaining overall speech quality through the reconstruction of all bands.
Solution Approach 2:
The patent transforms the speech signal from the time domain to the frequency domain using Fourier transform, enabling processing in a different dimension. By operating on spectral coefficients rather than raw waveforms, the system achieves faster computation while preserving speech quality through frequency-domain manipulation and inverse transformation.
2Manufacturing precision
If a large number of speech sampling points are generated, then speech quality is improved, but synthesis complexity increases
Solution Approach 1:
The speech spectrum is divided into multiple frequency bands, with each band processed by dedicated neural network branches. This segmentation allows the system to generate spectral coefficients for all bands simultaneously through parallel computation, reducing the complexity of generating individual sampling points while maintaining high speech quality through comprehensive frequency coverage.
Solution Approach 2:
The patent replaces the traditional time-domain waveform generation approach with frequency-domain spectral synthesis. Instead of sequentially generating individual speech sampling points through autoregressive modeling, the system directly synthesizes spectral coefficients and transforms them to time-domain speech, significantly reducing computational complexity while preserving speech quality.
3Reliability
If traditional autoregressive methods are used, then speech synthesis is achieved, but processing time increases
Solution Approach 1:
The patent employs periodic Fourier transform operations to convert between time-domain and frequency-domain representations. By periodically transforming the speech signal to frequency domain for processing and then back to time domain, the system achieves efficient parallel computation while maintaining speech synthesis reliability through the mathematical equivalence of the transformations.
Solution Approach 2:
The patent substitutes the sequential autoregressive generation mechanism with a parallel frequency-domain processing system. By replacing the step-by-step sampling point generation with direct spectral coefficient synthesis followed by inverse Fourier transform, the system dramatically reduces processing time while maintaining speech synthesis capability through the preserved spectral information.
Data Source
AI summary
A computer-implemented method for speech synthesis, a computer device, and a non-transitory computer readable storage medium are provided. The method includes: obtaining a speech text to be synthesized; obtaining a Mel spectrum corresponding to the speech text to be synthesized according to the speech text to be synthesized; inputting the Mel spectrum into a complex neural network, and obtaining a complex spectrum corresponding to the speech text to be synthesized, wherein the complex spectrum comprises real component information and imaginary component information; and obtaining a synthetic speech corresponding to the speech text to be synthesized, according to the complex spectrum. The method can efficiently and simply complete speech synthesis.