Speech Synthesis Style Feature Extraction via Residual Modeling
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional speech synthesis solutions face challenges in accurately extracting style features from mel-frequency cepstrum, which includes both style and semantic features, leading to complex modeling and reduced accuracy due to the need for manual annotation and additional style vector modules.
Innovation Solution
A speech synthesis system utilizing an average speech model, residual model, and projection layer, where the residual between predicted and target speech data is used to generate a style feature vector, eliminating the need for manual annotation and reducing model complexity by avoiding the use of mel-frequency cepstrum.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If mel-frequency cepstrum is used as input variable to extract speech features, then speech feature extraction can be performed, but the extraction quality is degraded because the mel-frequency cepstrum includes both speech features and semantic features
Solution Approach 1:
The patent extracts only the speech feature component from the mel-frequency cepstrum by using it as a condition to generate embedded vectors, while excluding the semantic feature component. This is achieved by processing the mel-frequency cepstrum through a neural network model that separates speech characteristics from semantic information, allowing high-quality speech feature extraction without semantic interference.
2Measurement precision
If manual annotation and additional style vector modules are added to achieve accurate style feature extraction, then speech synthesis quality can be improved, but model complexity increases
Solution Approach 1:
The patent merges the style feature extraction function into the existing speech synthesis model by generating embedded vectors that capture style characteristics directly from the mel-frequency cepstrum during normal operation. This integration eliminates the need for separate manual annotation processes and additional style vector modules, achieving accurate style feature extraction while maintaining model simplicity.
3Reliability
If conventional speech synthesis methods are used with mel-frequency cepstrum, then speech synthesis can be performed, but training time increases due to complex modeling requirements
Solution Approach 1:
The patent performs preliminary extraction of style features by generating embedded vectors from the mel-frequency cepstrum before the main synthesis process. These pre-extracted embedded vectors capture style characteristics in advance, allowing the synthesis model to focus on integrating them with text features rather than learning style features from scratch during training, thereby reducing training time while maintaining high speech quality.
Data Source
Figure 1~2
Figure 3~4
Figure 5~6
AI summary
The present application relates to a voice synthesis method, a model training method and device, and a computer device, wherein the method comprises: obtaining linguistic data to be processed; encoding the linguistic data to obtain linguistic encoded data; obtaining an embedded vector for voice feature conversion; wherein the embedded vector is generated according to a residual between reference synthesized voice data corresponding to the same reference linguistic data and the reference voice data; decoding the linguistic encoded data according to the embedded vector to obtain a target synthesized voice data through the voice feature conversion. The solution of the present application can avoid the problem of affecting synthesized voice quality due to semantic features in the log-Mel spectrum.