Streaming Audio Generation with Dynamic Playback Synchronization
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing audio processing technologies face challenges in minimizing audio playing delays, particularly when text output by a language model needs to be converted into audio, due to unstable text generation speed and network transmission delays.
Innovation Solution
An audio generation method that involves receiving first streaming text, converting it into first audio, determining a target time point based on the number of characters or interval duration of second streaming text, and converting the second streaming text into second audio within a duration range defined by the unplayed duration of the first audio and the playing interval duration.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If text is accumulated before audio conversion, then audio conversion quality is improved, but audio playing delay increases
Solution Approach 1:
The patent applies preliminary action by predicting the end position of the current audio before it is fully played. The system calculates the predicted end position based on current playback progress and converts subsequent text to audio starting from this predicted point, rather than waiting for the audio to actually end. This allows overlapping of text conversion and audio playback processes, reducing the overall delay while maintaining conversion quality.
Solution Approach 2:
The patent implements continuity of useful action by ensuring that text-to-audio conversion continues without interruption during audio playback. The system maintains the conversion process running in parallel with playback, so that when one audio segment ends, the next is already prepared or being prepared, eliminating gaps and reducing total delay while maintaining continuous high-quality conversion.
2Adaptability or versatility
If text generation speed is unstable, then language model flexibility is maintained, but audio playback continuity deteriorates
Solution Approach 1:
The patent applies feedback by continuously monitoring the actual end position of played audio and comparing it with the predicted end position. Based on this feedback, the system dynamically adjusts the starting position for converting the next text segment to audio. This closed-loop control ensures that audio playback remains continuous and smooth even when text generation speed varies, maintaining stability while preserving the language model's flexibility.
3Manufacturing precision
If text is segmented into sentences for conversion, then audio conversion accuracy is improved, but processing time increases
Solution Approach 1:
The patent applies preliminary action by performing sentence segmentation and audio conversion on multiple text segments in advance, before they are actually needed for playback. The system divides the input text into sentences ahead of time and converts them to audio segments, storing them for later playback. This eliminates the need to segment and convert text in real-time during playback, improving both accuracy and processing efficiency.
Data Source
AI summary
The audio generation method includes: receiving first streaming text, and converting the first streaming text into first audio; receiving second streaming text located after the first streaming text, and determining a target time point during receiving of the second streaming text based on a number of characters of the received second streaming text or interval duration between receiving of adjacent characters; obtaining audio duration of the first audio after the target time point as unplayed duration of the first audio; and converting, starting from the target time point, the second streaming text into second audio within a duration range defined by the unplayed duration and playing interval duration, where the playing interval duration represents a maximum time interval between an end time point of the first audio and a start time point of the second audio.