Neural Vocoder Subband Generation for Real-Time Speech Synthesis
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing neural vocoder methods struggle with high computational costs and are unable to perform real-time operations due to the need for multiple forward propagations, limiting their speed and applicability.
Innovation Solution
A method involving an intermediate representation generation process using a combination of probability distribution models to simultaneously generate multiple subband signals, allowing for the generation of speech waveforms at high speed by reducing the number of forward propagations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If a neural vocoder based on WaveNet is used to synthesize speech, then the quality of synthesized speech is comparable to natural speech, but the processing speed becomes too slow for real-time operation due to large amount of calculation
Solution Approach 1:
The speech signal is divided into subband signals using pseudo-QMF, and the neural network processes these subband signals instead of the full speech waveform. This segmentation reduces the computational burden per sample while maintaining speech quality, enabling real-time operation.
Solution Approach 2:
The invention changes the prediction target from speech waveform samples to subband signal samples. By transforming the problem into a different dimensional space (subband domain), the number of forward propagations required is reduced from 1 per speech sample to 1/B per subband sample, where B is the number of subbands, achieving speedup of B times or more.
2Speed
If the number of forward propagations is reduced by using subband signals, then the processing speed increases, but the sequence length must be shortened to 1/B which may affect prediction accuracy
Solution Approach 1:
The invention performs preliminary action by dividing the speech signal into subband signals before neural network processing. This pre-processing step enables the model to capture temporal dependencies more efficiently in the subband domain, maintaining effective sequence length coverage while reducing the number of forward propagations needed.
Solution Approach 2:
Multiple subband signals across different frequency bands are processed simultaneously through parallel neural network paths. By merging the processing of multiple subbands in one forward propagation, the effective sequence length is preserved across all bands while achieving speedup through reduced propagation count.
Data Source
AI summary
A generation device generates intermediate representation information of a subband signal based on an acoustic feature value of a speech waveform, and simultaneously generates a plurality of subband signals corresponding to a plurality of different times and a plurality of different bands by inputting the intermediate representation information to a plurality of probability distribution generation models that outputs information on subband signals corresponding to times and bands allocated respectively, the plurality of probability distribution generation models corresponding to the number of channels of the subband signals and the number of samples to be simultaneously generated; and generates the speech waveform based on the plurality of subband signals.