Transformer Speech Recognition Confidence-Based Label Calibration
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In speech recognition systems, incorrect labels in time-series data lead to performance deterioration due to error propagation, especially when using transformer-based models with end-to-end methods, as they rely on supervised learning and can generate corrupted pseudo labels, particularly in semi-supervised learning scenarios.
Innovation Solution
A confidence-based filtering method is employed to identify and correct incorrect labels within the transformer-based speech recognition model by calculating transition probabilities, self-attention probabilities, and source-attention probabilities, allowing for the replacement of incorrect labels at decoder time steps, thereby improving model performance.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If supervised learning is used with transformer-based models for speech recognition, then the model can achieve end-to-end speech recognition performance, but incorrect labels in the training data cause error propagation and deteriorate model performance
Solution Approach 1:
The patent applies preliminary action by pre-processing the training data to identify and correct incorrect labels before model training. The system calculates confidence scores for each training sample based on label consistency with speech content, and corrects or removes samples with low confidence scores before they can propagate errors to the model
Solution Approach 2:
The patent introduces an intermediary confidence calculation mechanism that acts as a mediator between the training data and the model. This intermediary layer evaluates the reliability of each training sample by comparing label information with speech content features, and only high-confidence samples are used for training, thus preventing error propagation
2Quantity of substance
If semi-supervised learning is used to generate pseudo labels, then more training data can be utilized, but the generated pseudo labels may be corrupted and further propagate errors
Solution Approach 1:
The patent implements feedback by using the trained model to evaluate its own predictions and identify uncertain cases. Samples where the model shows low confidence or high uncertainty are flagged for re-labeling or exclusion, creating a feedback loop that continuously improves data quality and prevents corrupted pseudo-labels from degrading performance
Solution Approach 2:
The patent applies parameter changes by dynamically adjusting the confidence threshold and data selection criteria based on model performance metrics. As the model improves, the system adjusts parameters to be more selective about which pseudo-labels are accepted, thereby maintaining high reliability even as training data volume increases
Data Source
AI summary
Proposed are a speech recognition system and method for automatically calibrating a data label. A speech recognition method for automatically calibrating a data label according to an embodiment may comprise the steps of: performing confidence-based filtering to find the location of occurrence of a wrong label in time-series speech data, in which a correct label and the wrong label are temporally mixed, by using a transformer-based speech recognition model; and after performing filtering, replacing a label at a decoder time step, which has been determined to be a wrong label by the location of occurrence of the wrong label, so as to improve the performance of the transformer-based speech recognition model, wherein the step of performing confidence-based filtering to find the location of occurrence of the wrong label in the time-series speech data comprises finding and calibrating the wrong label using the confidence obtained by using a transition probability between labels at every decoder time step.