Audio Caption Alignment via Audio Segmentation and Feature Concatenation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Automatic caption alignment for long audios is challenging due to the excessive machine resources required, leading to inefficiencies in feature extraction and alignment accuracy.
Innovation Solution
The method involves slicing long audios into shorter segments for feature extraction, concatenating feature information when the audio duration is within a certain threshold, and generating caption information based on the target caption text and audio feature information, thereby optimizing resource usage and alignment accuracy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If long audio is processed directly for feature extraction and caption alignment, then alignment accuracy can be maintained, but machine resources are excessively occupied leading to low processing efficiency
Solution Approach 1:
The patent divides long audio into multiple short audio segments based on duration thresholds. When audio duration exceeds the first preset duration, it is segmented into multiple short audios. Each segment is processed independently for feature extraction, reducing the computational burden on machine resources while maintaining alignment accuracy through subsequent merging of alignment results.
2Quantity of substance
If long audio is sliced into multiple short audios for feature extraction, then machine resource usage is reduced, but alignment accuracy may deteriorate due to multiple processing steps
Solution Approach 1:
The patent performs preliminary feature extraction on short audio segments before merging them. By extracting features from manageable segments first and then combining the alignment results, the system avoids the computational complexity of processing entire long audio at once while preserving alignment accuracy through the structured merging process.
3Loss of information
If feature extraction is performed on the entire target audio, then comprehensive audio features are obtained, but the process consumes excessive machine resources and time
Solution Approach 1:
The patent segments long audio into short audios and performs feature extraction on each segment separately. This approach maintains comprehensive audio feature coverage by processing all segments and merging their features, while significantly reducing the time and resource consumption compared to processing the entire long audio in one operation.
Data Source
AI summary
The disclosure relates to an audio caption alignment method and apparatus, a medium, and an electronic device. The method includes: obtaining a target audio and a target caption text of the target audio; obtaining a plurality of first target audios by slicing the target audio according to a slicing duration in a case that a duration of the target audio is greater than a first preset duration; determining first audio feature information of each of the first target audios; obtaining target audio feature information of the target audio by concatenating all of the first audio feature information in a case that the duration of the target audio is less than or equal to a second preset duration, where the second preset duration is greater than the first preset duration; and generating caption information corresponding to the target audio according to the target caption text and the target audio feature information.