Timing-Label Refinement for Speech Training Data Chunking
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing AI training data alignment methods struggle with efficiently matching audio file chunks to their corresponding transcript portions, particularly when transcripts lack timing information and contain non-auditory information or multiple languages, leading to inefficiencies in training AI models like ASR.
Innovation Solution
A multistage alignment process involving an alignment module that assigns timing labels to transcript units, using techniques like Levenshtein distance and CTC algorithms to align predicted and ground truth tokens, followed by a second stage to refine these alignments, ensuring accurate matching of audio and transcript chunks.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of manufacture
If transcripts are broken into portions without timing information, then training data preparation becomes simpler, but alignment between audio chunks and transcript portions becomes inaccurate
Solution Approach 1:
The system performs preliminary alignment between the full audio file and full transcript before breaking them into chunks. This preliminary action establishes timing relationships and correspondence between audio segments and transcript portions, enabling accurate chunking without losing alignment information.
Solution Approach 2:
The system segments both the audio file and transcript into corresponding chunks based on the preliminary alignment. By dividing both components simultaneously using the established alignment relationships, the system maintains accurate pairing between audio chunks and transcript portions while enabling efficient training data preparation.
2Adaptability or versatility
If alignment systems handle multiple languages and non-auditory information, then versatility improves, but alignment accuracy deteriorates due to non-matching portions
Solution Approach 1:
The system extracts and identifies non-auditory information (such as headers, paragraph numbers, or text without audio correspondence) from the transcript and handles them separately from the audio-aligned portions. This extraction allows the alignment algorithm to focus on matching only the audio-corresponding transcript segments, maintaining accuracy while accommodating diverse content types.
Data Source
AI summary
Artificial intelligence (AI) training data includes pairs of known input and output. Training an AI model includes generating an output from the model, comparing it against a known output, and modifying the model parameters until the model generates an output close to the known output. For speech recognition AI models, the training data includes pairs of audio files and corresponding transcripts. Dividing the audio files into chunks can be beneficial for the training of a speech recognition AI model. However, transcripts may not include timing or alignment data to find a corresponding transcript portion when dividing an audio file. A first stage alignment can generate timing labels for the speech tokens in a transcript. When dividing an audio file into audio file chunks, the timing labels can be used to find corresponding transcript portions to audio file chunks. A second stage alignment can improve the accuracy of the timing labels.