Speech Synthesis System with Adjustable Utterance Length via Phoneme Duration Prediction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing speech synthesis technologies struggle to generate natural speech within a desired utterance length, due to difficulties in accurately predicting phoneme durations, especially in autoregressive and non-autoregressive systems.
Innovation Solution
A speech synthesis method and system that predicts phoneme durations using a speech mask and a text to be synthesized, allowing for the regulation of utterance length by up-sampling phonemes according to predicted durations, thereby generating speech of a desired length.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If an autoregressive speech synthesis system is used, then speech generation follows sequential token dependency, but the utterance length cannot be effectively regulated
Solution Approach 1:
The speech synthesis process is segmented into independent phoneme duration predictions rather than sequential token generation. Each phoneme's duration is predicted independently based on the input text and speech mask, allowing parallel processing and effective utterance length control through the mask configuration.
Solution Approach 2:
The speech mask is prepared in advance with the desired utterance length configuration before synthesis begins. This preliminary action defines the target duration structure, enabling the system to regulate the final speech output length without being constrained by sequential generation limitations.
2Productivity
If a non-autoregressive speech synthesis system is used, then speech generation speed improves, but phoneme duration prediction accuracy decreases
Solution Approach 1:
The speech mask acts as an intermediary that guides the phoneme duration prediction process. It provides target duration information that mediates between the fast non-autoregressive generation and accurate duration prediction, allowing the system to achieve both speed and precision simultaneously.
Solution Approach 2:
The system uses the speech mask as a feedback reference during phoneme duration prediction. The predicted durations are compared against the mask-defined target durations, and the prediction model is trained to minimize the difference, thereby improving accuracy while maintaining fast non-autoregressive generation.
3Adaptability or versatility
If phoneme duration prediction is performed without speech mask guidance, then generation flexibility increases, but utterance length control accuracy decreases
Solution Approach 1:
The speech mask introduces a controllable parameter (desired utterance length) that changes the generation process. By adjusting the mask length parameter, the system can precisely control the output speech duration while maintaining flexibility in handling different input texts and speech styles through the same mask mechanism.
Data Source
AI summary
There is provided a speech synthesis system and method with an adjustable utterance length. The speech synthesis method according to an embodiment predicts a duration of each phoneme corresponding to a speech mask from the speech mask and a text to be synthesized with the speech mask, encodes the text to be synthesized and extracts a text sequence which is expressed by feature information of the text, generates a speech frame sequence by regulating a length of each phoneme of the text sequence according to the predicted duration of each phoneme corresponding to the speech mask, and synthesizes a speech from the generated speech frame sequence. Accordingly, a length of a speech to be synthesized can be freely regulated as a user desires by regulating a length of a speech mask.