ASR Confidence Score Calibration for Transcription Accuracy
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing automatic speech recognition (ASR) systems face challenges in producing accurate transcriptions, especially with multiple users who have not been trained, resulting in low accuracy and difficulty in determining conversational rules, leading to suboptimal confidence scores and outputs for client information retrieval systems.
Innovation Solution
A method is introduced to process ASR outputs by collecting statistics to generate an alternate output sequence with improved confidence scores and corrections, using a combination of Generalized Hidden Markov and parametric models to supplement or replace original outputs, allowing for more accurate and meaningful text generation.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If a large vocabulary continuous speech recognizer is used to transcribe spoken words to text, then the system can handle diverse speech content, but the transcription accuracy deteriorates when used with untrained users and low grade audio signals
Solution Approach 1:
The system uses confidence scores from the ASR system as feedback to identify potentially incorrect transcriptions. These confidence scores are then used to generate alternative word suggestions by consulting a dictionary, allowing the system to self-correct and improve accuracy without requiring retraining for different users or audio qualities
Solution Approach 2:
A confidence score calibration model acts as an intermediary between the ASR system and the final text output. This model processes the raw confidence scores and transforms them into calibrated confidence scores that better reflect actual transcription accuracy, enabling more reliable text generation for untrained users
2Measurement precision
If context information is used to improve transcription accuracy for untrained users, then recognition accuracy improves, but the complexity of determining conversational rules increases
Solution Approach 1:
The system automatically generates context information from the transcribed text itself without requiring external annotation or complex rule specifications. The confidence score calibration model processes the ASR output and automatically determines which words need correction based on the contextual information already present in the transcription, eliminating the need for manual rule definition
3Loss of information
If confidence scores are provided by the ASR system, then additional information is available for information retrieval systems, but the confidence scores lack meaning and reliability without calibration
Solution Approach 1:
The system transforms the raw confidence scores into calibrated confidence scores by applying a calibration model that adjusts the parameters of the confidence score distribution. This parameter transformation preserves the information richness while significantly improving the reliability and meaningfulness of the confidence scores for downstream information retrieval systems
Data Source
AI summary
Outputs of an automatic probabilistic event detection system, such as a fact extraction system, a speech-to-text engine or an automatic character recognition system, are matched with comparable results produced manually or by a different system. This comparison allows statistical modeling of the run-time behavior of the event detection system. This model can subsequently be used to give supplemental or replacement data for an output sequence of the system. In particular, the model can effectively calibrate the system for use with data of a particular statistical nature.