Text Time Annotation Using Fundamental Frequency Alignment
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for annotating the playing start-end time periods of characters in lyrics are inaccurate due to misalignment of phonemes when users sing incorrectly, leading to poor user experience and low efficiency in manual editing.
Innovation Solution
A method that utilizes fundamental frequency and short-time energy analysis to accurately determine the playing start-end time periods of text units in audio, segmenting the audio into sub-segments if necessary, and aligning these units with the corresponding text to improve annotation accuracy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Extent of automation
If phoneme alignment method is used to annotate playing start-end time periods, then annotation can be performed automatically, but annotation accuracy deteriorates when users sing incorrectly
Solution Approach 1:
The patent changes the parameter basis from phoneme sequence matching to fundamental frequency analysis. By extracting fundamental frequency features from audio and comparing them with reference frequencies corresponding to text units, the system achieves accurate time annotation even when phonemes are misaligned due to incorrect singing. This parameter transformation resolves the contradiction by making the annotation robust to phoneme variations while maintaining automatic operation.
2Ease of manufacture
If phoneme alignment is performed sequentially, then processing is systematic, but errors propagate to subsequent phonemes when misalignment occurs
Solution Approach 1:
The patent creates a reference fundamental frequency sequence from the audio signal and compares it with pre-defined reference frequencies associated with text units. This copying and comparison approach allows independent verification of each text unit's time period without sequential dependency, preventing error propagation while maintaining systematic processing through structured comparison protocols.
3Measurement precision
If manual editing is used to correct annotation errors, then annotation accuracy can be improved, but processing efficiency deteriorates
Solution Approach 1:
The patent implements self-service by using fundamental frequency analysis to automatically identify and annotate text units with their correct time periods. The system serves itself by extracting features from the audio signal, comparing them with reference data, and generating accurate annotations without requiring manual intervention, thereby maintaining both high accuracy and processing efficiency.
Data Source
Figure 1
Figure 2~3
Figure 4~6
AI summary
A text time annotation method and apparatus, an electronic device, and a readable storage medium are provided. The method includes: receiving an annotation request, obtaining a playing start-end time period of each text unit in target audio based on a fundamental frequency of the target audio, annotating a playing start-end time period of each text unit in a text based on the playing start-end time period of each text unit in the target audio, to obtain an annotation file; and outputting the annotation file. A moment between two adjacent text units in a text corresponding to the target audio is recognized by using the fundamental frequency of the target audio, that is, the two adjacent text units may be distinguished, and the text units are further aligned. Text units are not aligned in a current manner of recognizing the text units, thereby avoiding a problem that time annotation of a text cannot be performed because a text unit in target audio is inconsistent with a text unit in a text corresponding to the target audio, so that accuracy of time annotation of the text is high.