Streaming Text TTS Punctuation Model Latency
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional text-to-speech (TTS) systems struggle to generate natural-sounding speech from text input that lacks grammatical punctuation, such as streaming text from texting applications.
Innovation Solution
The integration of a punctuation model into the TTS system, which applies machine learning techniques to add grammatical punctuation to unpunctuated input text, enabling natural-sounding speech synthesis.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If conventional TTS systems process streaming text without punctuation, then processing speed is maintained, but speech naturalness deteriorates
Solution Approach 1:
The patent segments the streaming text into sub-strings at punctuation boundaries, allowing the TTS system to process and synthesize speech for each segment independently and sequentially. This segmentation enables natural speech pauses and intonation while maintaining continuous processing flow, resolving the contradiction between speech naturalness and processing latency.
Solution Approach 2:
The system performs preliminary punctuation insertion into the streaming text before TTS synthesis processing. By adding punctuation marks in advance based on learned patterns from training data, the system prepares the text for more natural speech synthesis without waiting for complete text input, thus improving speech naturalness while minimizing processing delays.
2Measurement precision
If the TTS system waits for complete text input before synthesis, then processing accuracy is improved, but real-time performance deteriorates
Solution Approach 1:
The system performs preliminary punctuation insertion into the streaming text before TTS synthesis processing. By adding punctuation marks in advance based on learned patterns from training data, the system prepares the text for more natural speech synthesis without waiting for complete text input, thus improving speech naturalness while minimizing processing delays.
Solution Approach 2:
The TTS system continuously processes streaming text as it arrives, synthesizing speech for completed segments without interruption. The system maintains continuous operation by processing text sub-strings sequentially as punctuation boundaries are identified, ensuring both real-time performance and synthesis accuracy through uninterrupted processing flow.
3Productivity
If the TTS system processes text in large chunks, then processing efficiency is improved, but speech intonation and pauses deteriorate
Solution Approach 1:
The patent segments the streaming text into sub-strings at punctuation boundaries, allowing the TTS system to process and synthesize speech for each segment independently and sequentially. This segmentation enables natural speech pauses and intonation while maintaining continuous processing flow, resolving the contradiction between speech naturalness and processing latency.
Data Source
Figure 1
Figure 2
Figure 3A~3B
AI summary
A method and system is disclosed for speech synthesis of streaming text. At a text-to- speech ("ITS) system., a real-time streaming text string having a starting point and an ending point may be received, and a first sub-string comprising a first portion of the text string received from an initial point to a first trigger point may be accumulated. The initial point is no earlier than the starting point and is prior to the first trigger point, and the first trigger point is no further than the ending point. A punctuation model of the ITS system may be applied to the first sub-string to generate a pre-processed first sub-string comprising the first sub-string with added grammatical punctuation as determined by the punctuation model. TTS synthesis processing may be applied to at least the pre-processed first sub-string to generate first synthesized speech, and audio play out of the first synthesized speech produced.