ASR Normalization Technique for Accurate WER Measurement
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing automatic speech recognition (ASR) systems face inefficiencies due to language-specific issues that complicate the generation of accurate word error rate (WER) measures, leading to the production of inefficient language models, as they fail to account for valid textual variants of terms.
Innovation Solution
A computer-implemented technique normalizes reference and ASR output transcriptions by identifying and standardizing valid textual variants, allowing for the development of a processor-efficient and memory-efficient language model by assigning the same textual form to textually distinct strings considered acceptable variations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If traditional ASR evaluation methods are used without normalization, then the evaluation process is simpler, but the WER measure accuracy deteriorates due to language-specific issues and failure to account for valid textual variants
Solution Approach 1:
The patent applies preliminary action by performing text normalization on both reference transcriptions and ASR output results before computing the WER measure. This preprocessing step converts textually distinct but semantically equivalent terms into a standardized form, ensuring that valid variants are properly recognized as matches. The normalization process handles language-specific issues, compound terms, and spelling variations in advance, so that the subsequent WER calculation operates on normalized data, improving accuracy without requiring complex adjustments during the evaluation phase.
2Productivity
If language-specific issues are not addressed, then the development process is faster, but the language model efficiency deteriorates
Solution Approach 1:
The patent applies preliminary action by incorporating normalization into the language model development process before training and evaluation. By pre-processing the training data to normalize textual variants, the system ensures that the language model learns from consistent representations of equivalent terms. This preliminary normalization step prevents the model from treating valid variants as errors, thereby improving language model efficiency and reducing the need for lengthy post-processing corrections.
3Measurement precision
If textually distinct variants are treated as different terms, then the textual forms are more diverse, but the WER measure flags false positives reducing accuracy
Solution Approach 1:
The patent applies parameter changes by transforming the textual form parameter of terms through normalization. The system changes the representation of textually distinct variants into a standardized form while preserving their semantic meaning. This parameter transformation allows the WER measure to accurately distinguish between genuine errors and valid variants, as equivalent terms are converted to the same normalized representation. The normalization process maintains adaptability by recognizing multiple valid forms while ensuring consistent evaluation through standardized transformation rules.
Data Source
AI summary
A computer-implemented technique identifies terms in an original reference transcription and original ASR output results that are considered valid variants of each other, even though these terms have different textual forms. Based on this finding, the technique produces a normalized reference transcription and normalized ASR output results in which valid variants are assigned the same textual form. In some implementations, the technique uses the normalized text to develop a model for an ASR system. For example, the technique may generate a word error rate (WER) measure by comparing the normalized reference transcription with the normalized ASR output results, and use the WER measure as guidance in developing the model. Some aspects of the technique involve identifying occasions in which a term can be properly split into component parts. Other aspects can identify other ways in which two terms may vary in spelling, but nonetheless remain valid variants.