Text Alignment Using Phoneme Similarity and Segmentation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current methods for aligning video/audio and reference text are inefficient due to reliance on speech recognition accuracy and calculation-intensive phoneme-based forced alignment, which fail in environments with low recognition rates or error-containing texts, especially in languages with accents and dialects.
Innovation Solution
A method that aligns texts at the word level based on phoneme similarity, using a hierarchical approach that first segments text into paragraphs and then aligns using phoneme similarity, reducing reliance on speech recognition accuracy and calculation load.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If phoneme-based forced alignment is used to align video/audio and reference text, then alignment accuracy is improved, but computational complexity and calculation load increase significantly
Solution Approach 1:
The patent segments the alignment process into multiple stages: first performing sentence-level alignment using dynamic programming, then performing word-level alignment only on identified mismatched regions. This segmentation avoids the need for complete phoneme-based forced alignment while maintaining accuracy in critical areas, thereby reducing overall computational complexity.
Solution Approach 2:
The patent applies different alignment strategies to different parts of the text-audio data. High-accuracy phoneme-level alignment is applied only to mismatched regions identified by the initial sentence-level alignment, while matched regions are left unchanged. This local quality approach ensures alignment accuracy where needed while minimizing unnecessary computational effort in already-matched areas.
2Extent of automation
If speech recognition engine is used to decode audio data before alignment, then alignment can be performed, but alignment effect deteriorates when recognition rate is low or errors exist in reference text
Solution Approach 1:
The patent introduces an intermediary dynamic programming-based sentence level alignment process between the speech recognition output and the final word-level alignment. This intermediary step identifies matched and mismatched regions without relying on perfect speech recognition, allowing the system to handle low recognition rates and reference text errors while maintaining alignment reliability.
Solution Approach 2:
The patent performs partial alignment by focusing word-level alignment only on mismatched regions identified by the sentence-level alignment, rather than attempting to align the entire text-audio pair. This partial action approach reduces the impact of speech recognition errors on the overall alignment while still achieving reliable results in critical areas.
3Measurement precision
If dynamic programming algorithm is used for sentence level alignment, then character maximum match is achieved, but alignment result is affected by speech recognition accuracy
Solution Approach 1:
The patent segments the alignment process into sentence-level alignment using dynamic programming, followed by word-level alignment on mismatched regions. This segmentation allows the dynamic programming to establish a reliable framework at the sentence level while the subsequent word-level refinement handles discrepancies caused by speech recognition errors, improving overall reliability.
Solution Approach 2:
The patent applies word-level alignment partially only to mismatched regions identified by the dynamic programming sentence-level alignment, rather than re-aligning the entire text. This partial action maintains the reliable sentence-level framework while correcting only the necessary discrepancies, preserving both character match accuracy and alignment reliability.
Data Source
AI summary
A method and apparatus for aligning texts. The method includes acquiring a target text and a reference text and aligning the target text and the reference text at word level based on phoneme similarity. The method can be applied to automatically archiving a multimedia resource and a method of automatically searching a multimedia resource.