Conversational AI TTS Adapters for Reduced Speaker Training
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional text-to-speech (TTS) machine learning models require significant amounts of speech data and computational resources to re-train for new speakers, and this process can degrade performance for previously trained speakers.
Innovation Solution
Insert adapters, or additional network layers, into a TTS model and update their parameters during training while keeping the base model's parameters fixed, allowing customization for new speakers with less data and resources, and enabling activation or deactivation based on the speaker.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If the TTS machine learning model is re-trained for a new speaker, then the model can be customized for the new speaker, but a large amount of speech data and computational resources are required
Solution Approach 1:
The patent segments the TTS model into a base model and adapter components. The adapter is a separate, trainable module that can be customized for new speakers without retraining the entire base model. This segmentation allows the system to maintain the pre-trained base model while adding speaker-specific customization through the adapter, significantly reducing the amount of speech data and computational resources needed.
Solution Approach 2:
The patent extracts the speaker-specific customization functionality into a separate adapter module. Instead of retraining the entire TTS model for each new speaker, the adapter extracts and learns only the necessary speaker-specific characteristics from the speech data. This extraction approach minimizes the quantity of speech data required while maintaining customization capability.
2Adaptability or versatility
If the TTS machine learning model is re-trained for a new speaker, then the model can be customized for the new speaker, but substantial time and compute resources are needed
Solution Approach 1:
By segmenting the model into a frozen base model and a trainable adapter, the patent reduces training time significantly. Only the adapter parameters need to be updated during training, rather than the entire base model. This segmentation approach maintains customization capability while dramatically reducing the time and compute resources required for training.
Solution Approach 2:
The patent applies parameter changes by updating only the adapter parameters while keeping the base model parameters fixed. This selective parameter updating approach allows the system to adapt to new speakers quickly without the computational burden of retraining the entire model, thus reducing training time and resource consumption.
3Adaptability or versatility
If the TTS machine learning model is re-trained for a new speaker, then the model can be customized for the new speaker, but the performance for previous speakers is degraded
Solution Approach 1:
The patent segments the model into a base model that maintains original speaker performance and an adapter that handles new speaker customization. The base model parameters remain fixed and unchanged during adapter training, ensuring that the performance for previous speakers is preserved. The adapter independently learns new speaker characteristics without interfering with the base model's existing performance.
Solution Approach 2:
The patent extracts speaker-specific customization into a separate adapter module that operates independently from the base model. This extraction ensures that training data for new speakers does not affect the base model's performance on previous speakers. The adapter handles all customization requirements while the base model maintains its original performance characteristics.
4Adaptability or versatility
If the TTS machine learning model is re-trained for a new speaker, then the model can be customized for the new speaker, but the re-training requires using original training audio data
Solution Approach 1:
The patent segments the training process into base model training (completed beforehand) and adapter training (current process). The adapter training only requires speech data from the new speaker, not the original training audio data. This segmentation dramatically reduces the quantity of training data needed while maintaining the ability to customize for new speakers.
Solution Approach 2:
The patent extracts the training data requirement from the entire base model and applies it only to the adapter. The adapter can be trained using a subset of speech data specific to the new speaker, eliminating the need to re-use the original training audio data. This extraction approach reduces the quantity of training data while preserving customization capability.
Data Source
AI summary
In various examples, one or more text-to-speech machine learning models may be customized or adapted to accommodate new or additional speakers or speaker voices without requiring a full re-training of the models. For example, a base model may be trained on a set of one or more speakers and, after training or deployment, the model may be adapted to support one or more other speakers. To do this, one or more additional layers (e.g., adapter layers) may be added to the model, and the model may be re-trained or updated—e.g., by freezing parameters of the base model while updating parameters of the adapter layers—to generate an adapted model that can support the one or more original speakers of the base model in addition to the one or more additional speakers corresponding to the adapter layers.