Multi-Track Subtitle Generation for Accurate Speech Recognition
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing subtitle generation methods for audio-video files suffer from inaccurate recognition due to audio tracks interfering with each other, leading to unclear and inaccurate subtitles.
Innovation Solution
Perform independent speech recognition on each audio track of an audio-video file to obtain text fragments, preprocess these fragments to avoid overlap, and merge them based on a time axis to generate accurate subtitles.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If overall speech recognition is performed on audio data from all audio tracks simultaneously, then processing efficiency is improved, but recognition accuracy deteriorates due to audio track interference
Solution Approach 1:
The patent segments the audio processing task by performing speech recognition on each audio track separately rather than processing all tracks simultaneously. This segmentation eliminates mutual interference between audio tracks while maintaining efficient processing through systematic handling of each track individually
Solution Approach 2:
The patent introduces time-axis-based subtitle generation as an intermediary mechanism that integrates recognition results from multiple audio tracks. By using time-axis alignment and subtitle merging, the system combines results from separate track recognitions while avoiding direct audio interference
2Adaptability or versatility
If multiple audio tracks are processed together, then comprehensive subtitle coverage is improved, but subtitle accuracy deteriorates due to overlapping audio data
Solution Approach 1:
The patent segments subtitle generation into separate processes for each audio track, ensuring that each track's subtitles are generated independently with high accuracy. The comprehensive coverage is then achieved by merging these separately generated subtitles based on time-axis alignment
Solution Approach 2:
The patent transitions from processing audio tracks in the audio domain to organizing subtitles in the time domain. By using time-axis-based alignment and sorting, the system integrates subtitles from multiple tracks while maintaining accuracy, effectively moving the integration problem to a different dimensional space
Data Source
Figure 1~2
Figure 3~4
Figure 5~7
AI summary
Provided are a subtitle generation method and apparatus, and a device and a storage medium. The method comprises: when a subtitle generation trigger operation for at least one audio track in a target audio and video file is received, respectively performing speech recognition on audio data on each audio track in the at least one audio track, so as to obtain a text fragment corresponding to each audio track; and then, generating subtitles for the target audio and video file on the basis of the text fragment corresponding to each audio track. In comparison with performing overall speech recognition for audio data on all audio tracks, the embodiments of the present disclosure are aimed at respectively performing independent speech recognition on audio data on each audio track, such that audio tracks are prevented from affecting one another, and a more accurate speech recognition result can be obtained, thereby improving the accuracy of subtitles generated on the basis of the speech recognition result.