Long-Form Audio-Text Alignment Using Segmented ASR Matching
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing audio-text alignment techniques face high computational complexity and error rates when aligning long audio data to its transcript, particularly due to quadratic complexity and memory consumption, making them inefficient and inaccurate.
Innovation Solution
An audio-text alignment system that uses an automatic speech recognition model to align long audio sessions with ground-truth text transcripts by segmenting audio, transcribing speech, and matching text segments, reducing complexity to linear time and memory usage through group-based comparisons.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If existing forced alignment algorithms are used to align audio data to transcripts, then alignment can be achieved, but computational complexity increases quadratically with audio length
Solution Approach 1:
The patent segments the audio data into multiple smaller audio segments and processes them independently. Each audio segment is aligned with corresponding text segments, avoiding the need to process the entire long audio transcript simultaneously. This segmentation reduces the computational complexity from quadratic to linear with respect to audio length.
2Measurement precision
If existing alignment techniques process entire audio sequences, then complete alignment is achieved, but memory consumption increases quadratically
Solution Approach 1:
The system divides the audio transcript into segments and processes them independently, maintaining only a sliding window of recent alignments in memory rather than storing all possible pairwise comparisons. This reduces memory requirements from quadratic to linear with respect to audio length.
Solution Approach 2:
The patent performs preliminary processing by segmenting the audio and pre-computing alignments for smaller segments before assembling the final alignment. This preliminary action reduces the memory burden during the main alignment process.
3Measurement precision
If whole sequence matching is performed to align audio to transcript, then comprehensive alignment is achieved, but processing time increases significantly
Solution Approach 1:
The audio is divided into segments that are processed in parallel or sequence, with each segment aligned independently. This segmentation enables processing time to scale linearly with audio length rather than quadratically, as each segment can be handled with fixed computational resources.
Solution Approach 2:
The system performs preliminary segmentation and pre-processing of audio and text data before the main alignment process. This preliminary action prepares the data in advance, reducing the time required for the actual alignment computation.
Data Source
AI summary
Methods, systems, and apparatus, including computer programs encoded on computer storage media, for performing long-form audio-text alignment. One of the methods includes: receiving audio data and a ground-truth text transcript of the audio data to be aligned with the audio data; dividing the audio data into a plurality of audio segments; each of the plurality of audio segments: processing the audio segment using an automatic speech recognition (ASR) model to generate a machine transcript of the audio segment; identifying, from the ground-truth text transcript, a matching portion of the ground-truth text transcript that matches the machine transcript of the audio segment; and generating audio-text alignment data that defines a correspondence between audio in the audio segment and text in the matching portion.