Speech Recognition Token Masking for Context-Balanced Decoding
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current end-to-end speech recognition models have limited text information modeling capabilities due to their shallow decoder structures, leading to suboptimal utilization of massive text corpora and recognition accuracy issues.
Innovation Solution
Generate first prediction information based on context information, followed by independent second prediction information, create mask information to prune non-matching candidate tokens, and update the first prediction information using the mask, ultimately generating a speech recognition result.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If end-to-end speech recognition models use shallow decoder structures, then the model structure is simple and easy to implement, but the text information modeling capability is limited and recognition accuracy is suboptimal
Solution Approach 1:
The patent segments the prediction process into two independent parts: first prediction information based on context information and second prediction information independent of context. This segmentation allows the model to leverage context when beneficial while maintaining the ability to process speech independently, thereby improving recognition accuracy without requiring an excessively complex decoder structure.
Solution Approach 2:
The patent introduces mask information that modifies the probability distribution of candidate tokens by setting probabilities to zero for tokens that do not match the target speech content. This parameter change approach allows the model to improve recognition accuracy by filtering out incorrect tokens while maintaining a relatively simple base architecture.
2Measurement precision
If the model relies excessively on context information, then text information modeling is enhanced, but the model becomes overly dependent on context and loses independent speech recognition capability
Solution Approach 1:
The patent divides the prediction process into two independent components: first prediction information that utilizes context information and second prediction information that processes speech independently without context. This segmentation ensures the model maintains both context-dependent and context-independent capabilities, preventing over-reliance on context while improving overall recognition accuracy.
Solution Approach 2:
The patent creates a unified prediction mechanism that can operate in multiple modes: utilizing context information when available and beneficial, and operating independently when context is unavailable or less relevant. This multi-functionality allows the model to adapt to different speech recognition scenarios, improving both accuracy and versatility.
3Measurement precision
If the model processes all candidate tokens equally, then the processing is simple, but the recognition accuracy is reduced due to inclusion of non-matching tokens
Solution Approach 1:
The patent applies mask information that changes the probability parameters of candidate tokens by setting probabilities to zero for tokens that do not match the target speech content. This parameter modification approach improves recognition accuracy by eliminating non-matching tokens while maintaining relatively simple processing through the masking operation.
Solution Approach 2:
The patent extracts and removes non-matching candidate tokens from the prediction results through the mask information mechanism. By taking out incorrect tokens while retaining correct ones, the model improves recognition accuracy without requiring complex additional processing steps.
Data Source
AI summary
Embodiments of the disclosure relates to a method, an apparatus, a device and a storage medium for speech recognition. An example method provided herein includes: generating first prediction information for target speech content by using a speech recognition model based on context information; generating second prediction information for the target speech content by using the speech recognition model, the second prediction information being independent of the context information; generating mask information based on a probability of a set of candidate tokens indicated by the second prediction information, the mask information indicating that at least one candidate token in the set of candidate tokens does not match the target speech content; updating the first prediction information by using the mask information; and generating a speech recognition result for the target speech content based on the first prediction information.