Text-to-Speech Acoustic Control Without Retraining
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing text-to-speech (TTS) systems lack the ability to generate speech with fine-grained control over acoustic characteristics, requiring extensive training and large datasets for modifications.
Innovation Solution
A two-stage workflow using a pre-trained model and a controllable model to modify speech signals, allowing users to adjust acoustic features through a user interface, with a controllable model trained on outputs from the pre-trained model.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If TTS systems are configured to generate speech signals with different characteristics through training, then speech quality and diversity are improved, but training time and computational resources increase
Solution Approach 1:
The system performs preliminary training of the TTS model on general speech characteristics in advance. During operation, users can modify speech characteristics by adjusting control parameters (pitch, energy, phoneme duration) without additional training, thereby resolving the contradiction between speech diversity and training time
Solution Approach 2:
The invention changes physical parameters of the speech signal (pitch, energy, phoneme duration) through control vectors that modify the acoustic characteristics of generated speech. This allows diverse speech characteristics to be achieved by parameter adjustment rather than retraining the model
2Adaptability or versatility
If TTS systems are configured to generate speech signals with different characteristics, then speech adaptability is improved, but device complexity increases
Solution Approach 1:
The system uses control vectors to modify acoustic parameters (pitch, energy, phoneme duration) of the speech signal. This approach achieves speech characteristic control through parameter adjustment rather than complex system restructuring, resolving the contradiction between adaptability and system complexity
3Ease of operation
If users can modify acoustic features through interface, then ease of operation is improved, but processing time increases
Solution Approach 1:
The system allows users to modify speech characteristics by adjusting control parameters through a graphical interface. The TTS model processes these parameter adjustments in real-time without requiring additional training or complex computations, thereby maintaining both ease of operation and efficient processing time
Data Source
Figure 1
Figure 2
Figure 3(a)
AI summary
A method of modifying a speech signal generated by a text-to-speech synthesiser, the method comprising: • receiving a text signal; • generating a speech signal from the text signal; • deriving a control feature vector, wherein the control feature vector represents modifications to the speech signal; • inputting the control feature vector in the text-to-speech synthesiser, wherein the text-to-speech synthesiser is configured to generate a modified speech signal using the control feature vector; and • outputting the modified speech signal.