Subtitle Generation Using Syntactic Analysis and Audio Timestamps
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current subtitle generation methods for videos often fail to control the length of single subtitles effectively, leading to poor typesetting and display time, which affects user experience, as they rely on punctuation recovery and do not consider the semantic continuity and audio duration adequately.
Innovation Solution
A method that extracts audio data from videos, performs speech recognition, and uses syntactic analysis to segment text data into segments based on pronunciation objects and timestamp information, merging these segments to achieve smooth semantics and a preset subtitle length, thereby controlling the display duration and reducing ambiguity.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of manufacture
If punctuation recovery is used to segment text for subtitles, then the process is simple, but the subtitle length cannot be controlled and display time is poor
Solution Approach 1:
The patent segments the text data into multiple text segments based on multiple segmentation positions obtained through syntactic analysis, then merges these segments into subtitle segments. This multi-stage segmentation approach enables precise control over subtitle length while maintaining reasonable processing complexity.
Solution Approach 2:
The patent dynamically adjusts the segmentation and merging process by considering both semantic continuity and audio duration. The system adaptively determines where to split and merge text segments based on real-time analysis of semantic boundaries and corresponding audio time lengths, achieving optimal subtitle length control.
2Ease of operation
If punctuation recovery is used for text segmentation, then processing is straightforward, but semantic continuity and audio duration are not considered
Solution Approach 1:
The patent adds a new dimension to text segmentation by incorporating audio duration information alongside semantic analysis. Instead of relying solely on punctuation, the system considers the time length of corresponding audio segments, creating a two-dimensional segmentation criterion that preserves semantic continuity while accounting for audio timing.
Solution Approach 2:
The patent uses syntactic analysis as an intermediary tool to obtain multiple segmentation positions, which then serve as candidate boundaries for text segmentation. This intermediary step enables the system to identify semantically meaningful boundaries without relying on punctuation, preserving semantic continuity.
3Manufacturing precision
If multiple segmentation positions are used based on syntactic analysis, then subtitle length can be controlled, but processing complexity increases
Solution Approach 1:
The patent performs preliminary syntactic analysis to obtain multiple segmentation positions before the actual text segmentation and merging process. By pre-identifying candidate segmentation points based on syntactic structures, the system reduces the complexity of the subsequent merging process while maintaining precise control over subtitle length.
4Reliability
If text segments are merged based on semantics and timestamp information, then subtitle quality improves, but processing time increases
Solution Approach 1:
The patent segments the text data into multiple text segments based on multiple segmentation positions obtained through syntactic analysis, then merges these segments into subtitle segments. This multi-stage segmentation approach enables precise control over subtitle length while maintaining reasonable processing complexity.
Solution Approach 2:
The patent dynamically adjusts the segmentation and merging process by considering both semantic continuity and audio duration. The system adaptively determines where to split and merge text segments based on real-time analysis of semantic boundaries and corresponding audio time lengths, achieving optimal subtitle length control.
Data Source
AI summary
The present disclosure relates to a subtitle generation method, a subtitle generation apparatus, an electronic device, a storage medium and a program. The method includes: extracting audio data from a video to be processed, and performing speech recognition on the audio data to acquire text data corresponding to the audio data; acquiring a plurality of segmentation positions of the text data determined based on syntactic analysis, and acquiring pronunciation object information and timestamp information of audio segments corresponding to characters comprised in the text data; segmenting the text data to acquire a plurality of text segments according to the plurality of segmentation positions, the pronunciation object information and the timestamp information of the audio segments corresponding to the characters, wherein audio segments corresponding to characters in a text segment belong to a same pronunciation object, and a duration of a blank segment in the audio segments corresponding to the text segment is less than a preset duration; merging the plurality of text segments according to semantics of the text segments and the timestamp information of the audio segments corresponding to the characters to acquire a plurality of merged segments which have smooth semantics and meet a preset requirement of single subtitle length; and generating subtitle data corresponding to the video to be processed according to the plurality of merged segments.