Virtual Agent Voiceover Caching for Low-Latency Speech
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional voiceover media content systems face challenges in integrating machine-learned language models with vocal synthesizers, leading to time-intensive processes, latency issues, and inconsistent intonation in voiceover tracks due to text-based output methods.
Innovation Solution
A media system that leverages a machine-learned language model to generate adaptive speech, parses speech into speech units, and uses a cache database to retrieve or generate audio bytes for these units, combining them to create coherent voiceover tracks.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of time
If the language model outputs text in a stream manner and feeds individual words into the vocal synthesizer, then latency is reduced, but the voiceover track becomes disjointed with inconsistent intonation
Solution Approach 1:
The patent segments the speech text into speech units that represent natural breaks in human speech (phrases, sentence clauses, or full sentences) rather than processing individual words. This segmentation allows the system to process smaller, more manageable units that maintain intonation coherence while reducing overall latency compared to processing complete blocks of text.
Solution Approach 2:
The patent implements partial action by processing speech units incrementally as they become available from the language model, rather than waiting for the complete text block. The system processes each speech unit independently through hashing and caching, allowing overlapping processing of multiple units simultaneously, which reduces latency while maintaining quality through selective processing of natural speech breaks.
2Manufacturing precision
If the language model outputs the entire block of text before generating audio, then the voiceover track is coherent, but latency increases significantly
Solution Approach 1:
The patent implements preliminary action by pre-computing and caching audio bytes for speech units that have been previously generated. When a speech unit is requested, the system first checks the cache database for a matching hash before initiating vocal synthesis. This preliminary caching mechanism reduces latency for repeated speech units while maintaining coherence through consistent audio generation.
Solution Approach 2:
The patent divides the complete text block into smaller speech units representing natural speech breaks. This segmentation enables the system to process and cache individual units independently, allowing parallel processing and reducing the overall time required to generate complete voiceover tracks compared to processing the entire block sequentially.
3Manufacturing precision
If vocal synthesis is performed in real-time without caching, then memory usage is reduced, but the process becomes time-intensive
Solution Approach 1:
The patent implements preliminary action by pre-generating and caching audio bytes for speech units after their first synthesis. The cache database stores hashed speech units with their corresponding audio bytes, allowing subsequent requests for identical speech units to be fulfilled immediately without re-synthesis. This preliminary caching significantly reduces generation time for repeated content while maintaining high audio quality.
Solution Approach 2:
The patent creates copies of audio bytes for cached speech units and stores them in the cache database. When the same speech unit is encountered again, the system retrieves the cached copy instead of performing time-intensive vocal synthesis again. This copying mechanism maintains exact audio quality while dramatically reducing processing time for repeated speech units.
Data Source
AI summary
A system adaptively generates a virtual experience inclusive of a virtual agent. The system receives speech text from a constrained machine-learned language model configured to provide adaptive speech for the agent. The system parses the speech text into a plurality of speech units, wherein a speech unit is an atomic unit representative of natural breaks in human speech. The system applies a hashing function to each speech unit to determine a corresponding hash. The system, for each hash, queries a cache database to identify whether the cache database includes a cached hash that matches the queried hash. Responsive to identifying a matching hash to a first queried hash, the system retrieves a first audio byte stored with the matching hash. The system generates a voiceover track for the virtual agent with the first audio byte for presentation to a user.