Text-to-Speech Synthesis with Asynchronous Prosodic Phrase Processing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing text-to-speech synthesis systems suffer from long first packet delays due to sequential processing of text normalization, phoneme prediction, and prosody prediction, which increases with the number of words and degrades user experience.
Innovation Solution
Implementing a multi-core, multi-threaded architecture with a thread pool for asynchronous processing of prosodic phrases, allowing simultaneous processing of different phrases through a connected queue of threads for text normalization, phoneme prediction, and prosody prediction, enabling continuous audio playback.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Device complexity
If sequential processing of text normalization, phoneme prediction, and prosody prediction is used, then processing order is simple to implement, but first packet delay increases and processing efficiency deteriorates
Solution Approach 1:
The patent segments the text-to-speech processing pipeline into three independent modules: text normalization, phoneme prediction, and prosody prediction. Each module processes different prosodic phrases independently and simultaneously, rather than sequentially. This segmentation allows parallel processing of multiple phrases across different modules, significantly reducing the first packet delay while maintaining implementation simplicity through modular design.
Solution Approach 2:
The system performs preliminary actions by pre-processing and preparing multiple prosodic phrases in advance through the thread pool. The text normalization module prepares normalized text for multiple phrases simultaneously, and the phoneme prediction module pre-generates phoneme sequences for upcoming phrases. This preliminary action ensures that when audio playback is needed, the processed data is already ready, reducing waiting time and first packet delay.
2Ease of manufacture
If sequential processing is used, then system implementation is straightforward, but processing speed and productivity decrease
Solution Approach 1:
The patent implements continuous useful action through a thread pool architecture where multiple threads continuously process different prosodic phrases in parallel. While one thread is processing text normalization for a current phrase, another thread simultaneously performs phoneme prediction for a previous phrase, and a third thread handles prosody prediction for an earlier phrase. This continuous parallel processing maximizes productivity and speech synthesis speed while maintaining straightforward implementation through standard multi-threading techniques.
3Quantity of substance
If the number of words in text increases, then more content is processed, but first packet delay increases proportionally
Solution Approach 1:
The patent divides the input text into multiple prosodic phrases and processes them in parallel through independent modules. When text volume increases, the system segments it into more phrases and distributes them across multiple threads and modules for simultaneous processing. This prevents first packet delay from increasing proportionally with text volume, as the parallel processing capacity scales with the number of phrases.
Solution Approach 2:
The patent transitions from one-dimensional sequential processing to multi-dimensional parallel processing by introducing thread pools and modular architecture. Instead of processing phrases one after another in a single timeline, the system creates multiple processing dimensions where different phrases are processed simultaneously in different threads and modules, effectively reducing the time dimension for first packet delivery regardless of text volume.
Data Source
AI summary
A text-to-speech synthesis method, an electronic device, and a computer-readable storage medium are provided. The method includes: obtaining prosodic pause features of an input text by performing a prosodic pause prediction processing on the input text, and dividing the input text into a plurality of prosodic phrases according to the prosodic pause features; synthesizing short sentence audios according to the prosodic phrases by performing a streamed speech synthesis processing on each of the prosodic phrases in the input text in a manner of asynchronous processing of a thread pool; and performing an audio playback operation of the input text according to the short sentence audios corresponding to the first prosodic phrase of the input text, in response to synthesizing the short sentence audio corresponding to the first prosodic phrase of the input text.