Spontaneous TTS Synthesis Using Fluency Reference Factors
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional text-to-speech (TTS) systems generate speech that is not spontaneous and lacks the natural fluency characteristics of human speech, such as pauses and repetitions, making them unsuitable for conversational scenarios.
Innovation Solution
Implementing a fluency control mechanism in TTS systems using a fluency reference factor that includes fluency adjustment information and level representation to generate acoustic features that simulate human-like spontaneity, incorporating elements like filled and silent pauses, and word repetitions.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If conventional end-to-end TTS systems generate speech based on text input, then speech waveform can be produced efficiently, but the speech lacks natural fluency characteristics such as pauses and repetitions
Solution Approach 1:
The fluency reference factor is segmented into multiple independent components: pause insertion predictions, repetition predictions, and duration predictions. Each component is generated by separate prediction models and processed independently before being combined to control the synthesis process, allowing modular optimization and maintenance while achieving natural fluency
Solution Approach 2:
A fluency reference factor is introduced as an intermediary element between the text input and acoustic feature generation. This intermediate representation encapsulates fluency characteristics (pauses, repetitions, durations) and guides the synthesis process without requiring direct modification of the core TTS architecture, thus improving naturalness while maintaining system efficiency
2Reliability
If fluency adjustment information is predicted for each phrase boundary, then speech fluency can be controlled, but computational complexity increases
Solution Approach 1:
Instead of applying complex fluency control to every possible phrase boundary, the system selectively applies fluency adjustments only where prediction models identify actual pause or repetition needs. This partial action approach reduces unnecessary computational overhead while maintaining accurate fluency control where required
Solution Approach 2:
Fluency characteristics (pauses, repetitions, durations) are predicted in advance during a preprocessing stage before acoustic feature generation. By performing these predictions preliminarily and caching the fluency reference factor, the system avoids repeated heavy computations during real-time synthesis, reducing overall computational power requirements
Data Source
Figure 1~2
Figure 3
Figure 4
AI summary
The present disclosure provides methods and apparatuses for spontaneous text-to-speech (TTS) synthesis. A target text may be obtained. A fluency reference factor may be determined based at least on the target text. An acoustic feature corresponding to the target text may be generated with the fluency reference factor. A speech waveform corresponding to the target text may be generated based on the acoustic feature.