Neural Text-to-Speech Adaptation Using Articulatory Feature Embeddings
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing text-to-speech synthesis methods require significant data and re-learning to accommodate new speakers, limiting their ability to generate natural and flexible speech responses.
Innovation Solution
A single artificial neural network-based text-to-speech synthesis model that extracts and utilizes articulatory features, such as prosody, emotion, and voice tone, from a speaker's speech sample to generate output speech data without extensive re-learning, allowing for quick adaptation to new speakers.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If a conventional text-to-speech synthesis method is used, then speech can be generated from text, but significant data and re-learning are required to accommodate new speakers
Solution Approach 1:
The system performs preliminary extraction of articulatory features from speech samples during the model training phase, storing these features for rapid application during inference. This preliminary action eliminates the need for time-consuming re-training when accommodating new speakers, as the features can be directly applied to generate speech for new speakers immediately.
Solution Approach 2:
The invention extracts articulatory features (such as prosody, pitch, and timing characteristics) from speech samples as separable components. By taking out these features from the complete speech signal and representing them as embedding vectors, the system can apply them independently to new speakers without requiring full re-learning of the TTS model, thus reducing adaptation time.
2Manufacturing precision
If extensive data collection and re-training are performed for new speakers, then speech synthesis accuracy improves, but the process becomes time-consuming and complex
Solution Approach 1:
The system extracts articulatory features as embedding vectors from speech samples, separating the speaker-specific characteristics from the general TTS model. This extraction allows the system to maintain high speech synthesis accuracy by applying speaker-specific embeddings without requiring complex re-training processes, thereby reducing operational complexity while preserving precision.
Solution Approach 2:
The invention changes the parameter representation from requiring full model re-training to using embedding vectors that encode speaker-specific articulatory features. By transforming the problem from re-learning weights to applying feature embeddings, the system maintains synthesis accuracy while dramatically simplifying the adaptation process for new speakers.
3Device complexity
If a single artificial neural network model is used, then the system structure is simplified, but the ability to capture diverse speaker characteristics is reduced
Solution Approach 1:
The single artificial neural network model is designed with multi-functionality by incorporating an articulatory feature extraction module and embedding layer that can handle diverse speaker characteristics. This universal model structure can adapt to different speakers by applying their specific articulatory embeddings, maintaining simplicity while capturing diverse speaker traits through the flexible embedding mechanism.
Solution Approach 2:
The articulatory feature embedding acts as an intermediary between the single TTS model and diverse speaker characteristics. Instead of requiring multiple specialized models, the embedding vectors serve as mediators that convey speaker-specific information to the universal model, enabling it to capture diverse characteristics while maintaining a simplified single-model structure.
Data Source
AI summary
A text-to-speech synthesis method using machine learning, the text-to-speech synthesis method is disclosed. The method includes generating a single artificial neural network text-to-speech synthesis model by performing machine learning based on a plurality of learning texts and speech data corresponding to the plurality of learning texts, receiving an input text, receiving an articulatory feature of a speaker, generating output speech data for the input text reflecting the articulatory feature of the speaker by inputting the articulatory feature of the speaker to the single artificial neural network text-to-speech synthesis model.