Audio-Text Segment Mapping for Accurate TTS Synchronization
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing text-to-speech (TTS) technologies face challenges in generating high-tone-quality audio due to limited operation capabilities of client devices, leading to mismatches between displayed text and read content, especially when different text formats are used for display and reading.
Innovation Solution
A synchronization method and apparatus that determines first text segments for audio conversion and second text segments for display, establishing mapping relationships to synchronize audio and text segments, allowing for high-tone-quality audio generation on a server and improved listening and reading flexibility.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If TTS processing is performed at the server to obtain high-tone-quality audio, then audio quality is improved, but text synchronization accuracy deteriorates due to using different text formats for display and reading
Solution Approach 1:
The patent segments the text into multiple text segments, where some segments are used for display and others for reading. By dividing the text processing into different segments with different purposes, the system can maintain high audio quality from server-side TTS while achieving accurate text synchronization through separate processing paths for display text and reading text.
Solution Approach 2:
The patent introduces an intermediary text segment that serves as a bridge between the display text and the audio content. This intermediary segment allows the system to map audio segments to the appropriate display text segments, resolving the synchronization issue caused by using different text formats for display and reading purposes.
2Adaptability or versatility
If different text formats are used for display and reading, then reading flexibility is improved, but text matching accuracy deteriorates
Solution Approach 1:
The patent divides the text into multiple segments with different formats optimized for specific purposes: display text segments for visual presentation and reading text segments for audio conversion. This segmentation allows each segment to be formatted appropriately for its intended use while maintaining overall text matching accuracy through systematic mapping between segments.
Solution Approach 2:
The patent applies different text formatting qualities to different parts of the text based on their intended use. Display text segments are formatted for visual readability, while reading text segments are formatted for accurate audio synthesis. This local differentiation of text quality allows optimal formatting for each purpose while maintaining overall synchronization accuracy.
Data Source
AI summary
Provided are a synchronization method and apparatus for audio and text, a device, and a medium. The method includes: determining a plurality of first text segments for audio conversion and a second text for reading display, in which the plurality of first text segments and the second text are from an initial text; converting the plurality of first text segments into audio segments, to obtain a first mapping relationship between the first text segments and the audio segments; performing matching on the first text segments and the second text, to obtain a second mapping relationship between the first text segments and second text segments in the second text; determining the second text segment synchronized with each of the audio segments based on the first mapping relationship and the second mapping relationship.