ASR Text Segmentation Training for Capitalization and Punctuation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing automatic speech recognition (ASR) systems produce outputs that lack punctuation and capitalization, leading to readability issues, as conventional text segmenters and capitalizers trained on written text fail to accurately handle ASR outputs due to the absence of punctuation and mid-sentence segments.
Innovation Solution
A multistage training process for a text segmenter and capitalizer machine learning models, utilizing speech transcriptions with segmented boundary information to mimic ASR outputs, enabling accurate segmentation and capitalization by training the segmenter with VAD-like data and transforming training data to resemble real-world ASR scenarios.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If conventional text segmenters and capitalizers trained on written text are used, then the system can process written text, but it fails to accurately handle ASR outputs due to absence of punctuation and mid-sentence segments
Solution Approach 1:
The training data is transformed by removing punctuation and capitalization markers to change the data parameters, making ASR outputs resemble written text without punctuation. This allows the model to learn segmentation and capitalization patterns from ASR data that mimics the structure of punctuation-less written text, resolving the contradiction between handling ASR outputs and maintaining accuracy.
Solution Approach 2:
The system creates synthetic training data by copying and transforming existing written text to resemble ASR outputs (removing punctuation, normalizing capitalization). This copied transformed data is then used to train the segmenter and capitalizer, enabling them to accurately handle ASR outputs while maintaining the reliability learned from structured training data.
2Reliability
If ASR outputs are used directly for training, then the model can learn from real ASR data, but the lack of punctuation and mid-sentence segments makes accurate training difficult
Solution Approach 1:
The system performs preliminary transformation on training data by removing punctuation and normalizing capitalization before using it for training. This preliminary action prepares the data in advance to resemble ASR outputs, making the training process easier while maintaining the ability to learn accurate capitalization and punctuation patterns.
Solution Approach 2:
The transformed training data acts as an intermediary between the raw ASR outputs and the model training process. By preprocessing the data to remove punctuation and normalize formatting, the intermediary transformed data bridges the gap between real ASR data and the training requirements, making accurate training possible without direct manipulation of raw ASR outputs.
3Productivity
If text is segmented and capitalized without proper training data transformation, then processing is fast, but the outputs lack readability and cannot be processed by written text-based systems
Solution Approach 1:
The model learns by transforming training data parameters (removing punctuation, normalizing capitalization) to match ASR output characteristics. This parameter transformation enables the model to produce high-quality capitalization and punctuation that matches written text standards while maintaining fast processing speeds through efficient machine learning inference.
Data Source
AI summary
Systems and techniques that facilitate ASR capitalization and punctuation are provided. For example, one or more embodiments described herein can comprise a system, which can comprise a memory that can store computer executable components. The system can also comprise a processor, operably coupled to the memory that can execute the computer executable components stored in memory. The components can comprise a segmenter training component that trains a segmenter machine learning submodel with a set of speech transcriptions and segment boundary information of the speech transcriptions; the segmenter machine learning submodel that generates text segment boundary information of a text comprising a plurality of sentences; and a capitalizer training component that transforms the text into a plurality of segments based on the text segment boundary information and trains a capitalizer machine learning submodel of the text segmenter and capitalizer machine learning model with the plurality of segments.