Speech Corpus Re-Segmentation for Accurate ASR Training Data
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing ASR systems face challenges in generating accurate and scalable training data due to the lack of high-quality conversational and spontaneous speech, leading to degraded performance, as current methods are resource-intensive and require human curation to improve data quality and accuracy.
Innovation Solution
An end-to-end automated method for re-segmentation and re-transcription of audio data, using machine learning models to generate high-quality audio-text utterances by segmenting and aligning speech with text, and performing final segmentation based on silence periods, to create variable audio segments with superior transcription quality.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If automated techniques are used to collect training data, then productivity is improved, but manufacturing precision deteriorates due to lack of control over data quality and accuracy
Solution Approach 1:
The patent segments the transcription process into multiple stages: initial transcription of long audio files, segmentation into text portions based on punctuation, generation of segment-level timestamps, audio segmentation into chunks, re-transcription of chunks, and final segmentation based on silence periods. This multi-stage segmentation approach enables automated processing while maintaining quality control at each step.
Solution Approach 2:
The patent introduces intermediate representations and models as mediators: initial transcription models convert long audio to text, ML models generate segment-level timestamps, silence detection identifies boundaries, and re-transcription models process segmented audio. These intermediary components enable automated quality control without requiring human intervention.
2Manufacturing precision
If human curation is used to improve data quality, then manufacturing precision is improved, but productivity deteriorates due to time and resource intensity
Solution Approach 1:
The system performs self-service quality control through automated silence detection, segment-level timestamp generation, and re-transcription of segmented audio chunks. The ML models automatically identify transcription errors and generate corrected versions without human intervention, enabling the system to self-curate its own training data at scale.
Solution Approach 2:
The patent implements feedback loops where re-transcription models process segmented audio and generate corrected transcriptions, which are then evaluated and integrated into the training corpus. The system uses alignment confidence scores and silence period analysis to feedback on transcription quality, automatically iteratively improving data quality without human curation.
3Productivity
If existing automated methods are used, then productivity is improved, but reliability deteriorates due to mixed languages, incorrectly identified languages, and sounds rather than natural speech
Solution Approach 1:
The patent performs preliminary actions to ensure data reliability: language identification and filtering are performed on the full audio corpus before transcription, silence periods are detected and used as segmentation boundaries, and segment-level timestamps are generated to ensure proper alignment. These preliminary quality control measures prevent unreliable data from entering the training corpus.
Solution Approach 2:
The system applies local quality control measures at different stages: initial language identification and filtering, segment-level timestamp generation with confidence scoring, silence-based segmentation to isolate natural speech from background sounds, and re-transcription of specific audio chunks. Each stage applies quality control locally to its specific data portion, ensuring overall reliability.
4Productivity
If the audio corpus is processed without segmentation, then productivity is improved, but manufacturing precision deteriorates due to inability to provide scalability across multiple data collections
Solution Approach 1:
The patent segments audio files into manageable chunks based on silence periods and punctuation marks, enabling parallel processing of multiple segments while maintaining high transcription quality. The segmented approach allows the system to scale across multiple data collections by processing segments independently and combining results, achieving both productivity and precision.
Solution Approach 2:
The patent introduces temporal dimensionality through segment-level timestamps and word-level timestamps, transforming the transcription process from a single-pass operation to a multi-dimensional alignment process. This enables the system to scale across multiple data collections by processing segments in parallel while maintaining precise temporal alignment and transcription quality through the added temporal dimension.
Data Source
AI summary
A method includes obtaining initial transcription for input natural speech; performing segmentation of initial transcription into text portions, based on punctuation marks in initial transcription; determining segment-level timestamps for text portions based on the input natural speech; performing audio segmentation on input natural speech, by cutting input natural speech based on segment-level timestamps, to obtain audio chunks; generating transcription portions for each of the audio chunks; merging transcription portions to form re-transcription; determining word-level timestamps for re-transcription, by aligning input natural speech against re-transcription; calculating silence time periods, each corresponding to silence between each two adjacent words of input natural speech, based on word-level timestamps; performing a final segmentation on input natural speech and re-transcription, based on silence time periods, to generate final audio segments and corresponding final transcription portions. The final audio segments and corresponding final transcription portions may be included in training dataset for training a model.