Neural Text-to-Speech Voice Adaptation Without Speaker Re-Training
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing text-to-speech synthesis methods require significant data and re-learning to accommodate new speakers, limiting their ability to generate natural speech for unknown speakers.
Innovation Solution
A text-to-speech synthesis method and apparatus that extracts and adjusts articulatory features of a new speaker without extensive data collection, using a single artificial neural network model to generate speech reflecting the speaker's characteristics.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If deep learning-based text-to-speech synthesis is used to improve speech quality and naturalness, then the computational complexity and processing time increase significantly
Solution Approach 1:
The speech synthesis process is divided into multiple independent stages: acoustic model processing, vocabulary processing, pronunciation processing, and graph processing. Each stage handles specific computations separately, allowing for optimized resource allocation and parallel processing, thereby reducing overall computational complexity while maintaining speech quality.
Solution Approach 2:
Speech data is pre-processed offline to generate pronunciation dictionaries, acoustic models, and graph structures before actual synthesis. This preliminary action moves heavy computational work away from real-time synthesis, reducing processing time and complexity during actual speech generation while preserving deep learning-based quality.
2Measurement precision
If deep learning models are trained to achieve high speech synthesis quality, then the training time and computational resources required increase
Solution Approach 1:
The training process is segmented into multiple tasks: acoustic model training, vocabulary processing, pronunciation dictionary generation, and graph construction. Each segment can be trained independently and in parallel, significantly reducing total training time while achieving high synthesis quality through the combined system.
Solution Approach 2:
Extensive preprocessing of speech data, vocabulary, and pronunciation rules is performed before model training begins. This preliminary action prepares optimized data structures and reduces the dimensionality of the training problem, allowing deep learning models to converge faster while maintaining high speech synthesis quality.
3Speed
If conventional text-to-speech methods are used to reduce processing time, then the speech naturalness and quality deteriorate
Solution Approach 1:
Speech data is pre-processed offline to generate optimized pronunciation dictionaries, acoustic models, and graph structures. This preliminary action moves heavy computational work away from real-time synthesis, enabling fast processing speed while maintaining the speech naturalness achieved through deep learning-based models.
Solution Approach 2:
The synthesis process is segmented into lightweight stages that can execute quickly: vocabulary lookup, pronunciation retrieval, graph traversal, and acoustic synthesis. This segmentation enables rapid processing while each stage contributes to maintaining high speech naturalness through deep learning-based components.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
A text-to-speech synthesis method using machine learning, the text-to-speech synthesis method is disclosed. The method includes generating a single artificial neural network text-to-speech synthesis model by performing machine learning based on a plurality of learning texts and speech data corresponding to the plurality of learning texts, receiving an input text, receiving an articulatory feature of a speaker, generating output speech data for the input text reflecting the articulatory feature of the speaker by inputting the articulatory feature of the speaker to the single artificial neural network text-to-speech synthesis model.