Speech Synthesis Model Training Using Discrete Bit Stream Prediction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing speech synthesis models require complex vocoders to restore speech signals from lossy Mel spectrum features, leading to high model complexity, long training times, and low training efficiency.
Innovation Solution
A method for training a speech synthesis model that includes a speech decoding model and an acoustic model, where the model is trained by performing speech bit stream prediction and decoding, and updating parameters based on differences between synthesized and standard speeches.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If a complex vocoder is used to restore speech signal from lossy Mel spectrum, then speech synthesis quality is improved, but model complexity increases and training efficiency decreases
Solution Approach 1:
The patent changes the fundamental parameter being modeled from continuous Mel spectrum to discrete speech bit stream. This parameter transformation allows the use of simpler decoding models while maintaining synthesis quality, as the discrete bit stream representation preserves critical speech information more efficiently than lossy continuous spectrum representation.
Solution Approach 2:
Instead of directly restoring the continuous speech signal from lossy Mel spectrum, the patent uses a speech coding/decoding approach where the speech bit stream acts as an efficient intermediate representation. The decoder reconstructs the speech signal from this compact bit stream, achieving high quality with lower complexity than direct vocoder approaches.
2Manufacturing precision
If a complex vocoder is used to restore speech signal, then speech synthesis quality is improved, but training time increases
Solution Approach 1:
By transforming the modeling target from continuous Mel spectrum to discrete speech bit stream, the patent enables more efficient training convergence. The discrete nature of bit streams provides clearer gradient signals and faster optimization compared to the lossy continuous representation, significantly reducing training time while maintaining quality.
3Manufacturing precision
If a complex vocoder is used to restore speech signal, then speech synthesis quality is improved, but training efficiency decreases
Solution Approach 1:
The patent transforms the output parameter from continuous Mel spectrum to discrete speech bit stream, which fundamentally improves training efficiency. The discrete bit stream representation reduces computational complexity during training, enables faster convergence, and improves resource utilization while preserving speech quality through efficient speech coding principles.
Data Source
AI summary
This application provides a method for training a speech synthesis model, a speech synthesis method, an apparatus, an electronic device, a computer-readable storage medium, and a computer program product. The method for training a speech synthesis model includes: obtaining a text sample and a standard speech corresponding to the text sample; performing speech bit stream prediction on the text sample by using the speech synthesis model, to obtain a speech bit stream corresponding to the text sample; decoding the speech bit stream by using the speech synthesis model, to obtain a synthesized speech corresponding to the text sample; and updating a model parameter of the speech synthesis model based on a difference between the synthesized speech and the standard speech.