Speech Recognition Model Training Fusion Probability
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
End-to-end speech recognition models face challenges in accurately predicting text from speech due to reliance on beam search methods that only consider acoustic information, leading to suboptimal recognition accuracy.
Innovation Solution
A method that combines acoustic and language models to calculate fusion probabilities for candidate texts, selects the most likely texts based on these probabilities, and trains the model using a loss function derived from standard and predicted texts to improve recognition accuracy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of manufacture
If beam search is used for decoding in end-to-end speech recognition models, then the model training process is simplified, but the recognition accuracy deteriorates due to reliance on acoustic information only
Solution Approach 1:
The patent merges acoustic decoding model and language model into a unified decoding process. The acoustic decoding model processes acoustic features to generate candidate texts, while the language model processes these candidate texts to generate language probabilities. Both models work together in sequence, combining acoustic information with language information to improve recognition accuracy while maintaining the end-to-end architecture's training simplicity.
Solution Approach 2:
The patent introduces an intermediary processing step between acoustic decoding and final text output. The language model acts as an intermediary that receives candidate texts from the acoustic decoding model and transforms them into language-probability-weighted predictions. This intermediary layer enables the system to leverage language constraints without complicating the overall training process.
2Device complexity
If only acoustic information is used for decoding, then the decoding process is simple, but the prediction accuracy deteriorates
Solution Approach 1:
The patent segments the decoding process into two distinct stages: acoustic decoding stage where the acoustic decoding model generates candidate texts from acoustic features, and language decoding stage where the language model refines these candidates by computing language probabilities. This segmentation allows each stage to focus on specific information types while maintaining overall system simplicity.
Solution Approach 2:
The patent adds another dimension to the decoding process by introducing language information alongside acoustic information. Instead of a single-dimensional acoustic-only decoding, the system now operates in a two-dimensional space combining acoustic features and language probabilities, enabling more accurate predictions through multi-dimensional information integration.
Data Source
AI summary
A method for training a speech recognition model, a device and a storage medium, which relate to the field of computer technologies, and particularly to the fields of speech recognition technologies, deep learning technologies, or the like, are disclosed. The method for training a speech recognition model includes: obtaining a fusion probability of each of at least one candidate text corresponding to a speech based on an acoustic decoding model and a language model; selecting a preset number of one or more candidate texts based on the fusion probability of each of the at least one candidate text, and determining a predicted text based on the preset number of one or more candidate texts; and obtaining a loss function based on the predicted text and a standard text corresponding to the speech, and training the speech recognition model based on the loss function.