ASR Confidence Classifier Using Word Embeddings
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Automated speech recognition (ASR) systems face higher false alarm rates when dealing with non-native language speakers, leading to inaccurate confidence scoring, where the confidence score for incorrect words can be overly high, causing applications to mistakenly recognize out-of-grammar utterances as in-grammar.
Innovation Solution
The enhancement of ASR confidence classifiers by incorporating word embedding confidence features, such as character and phone pronunciation embeddings, into the baseline confidence features to create a more robust feature vector, which is then used to generate a confidence score, trained with labeled examples to improve accuracy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If baseline confidence features are used for ASR confidence scoring, then the system works well for native language speakers, but false alarm rates increase for non-native speakers with different accents
Solution Approach 1:
The patent combines baseline confidence features with word embedding confidence features to create a hybrid confidence scoring system. The word embedding features capture semantic information about whether recognized words make sense in context, which complements the acoustic matching information from baseline features. This merging of different feature types resolves the contradiction by providing more robust confidence scoring that works across different accents and languages.
Solution Approach 2:
The patent introduces word embedding features as an intermediary layer between acoustic features and confidence scoring. These embeddings serve as a mediator that captures linguistic knowledge about word usage and context, helping to distinguish between correct recognitions and false alarms. The intermediary embedding layer translates acoustic information into semantic meaning, resolving the accuracy-reliability contradiction.
2Reliability
If confidence scoring is improved for non-native speakers, then false alarm rates decrease, but system complexity increases
Solution Approach 1:
The patent segments the confidence scoring problem into two independent feature sets: baseline confidence features from acoustic modeling and word embedding confidence features from linguistic modeling. By dividing the complex task into separate feature extraction components, the system can improve reliability without overwhelming complexity. Each segment can be optimized independently and combined through simple feature concatenation.
Solution Approach 2:
The patent adds a new dimension to the confidence scoring space by incorporating word embedding features. Instead of relying solely on the traditional acoustic feature dimension, the system extends into a semantic dimension using pre-trained word embeddings. This dimensional extension improves reliability by providing additional discriminatory information while maintaining manageable complexity through the use of established embedding techniques.
Data Source
AI summary
A method of enhancing an automated speech recognition confidence classifier includes receiving a set of baseline confidence features from one or more decoded words, deriving word embedding confidence features from the baseline confidence features, joining the baseline confidence features with word embedding confidence features to create a feature vector, and executing the confidence classifier to generate a confidence score, wherein the confidence classifier is trained with a set of training examples having labeled features corresponding to the feature vector.