Speech Recognition Model Training With Reference-Model Decoding
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional speech recognition model training methods based on end-to-end decoding are inefficient due to lengthy decoding processes, affecting recognition accuracy.
Innovation Solution
A reinforcement learning approach is employed to train the speech recognition model by generating a speech feature sequence, processing it with a language model to generate probability information, obtaining recognized texts using a reference model, determining a training loss, and adjusting model parameters based on this loss.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If end-to-end decoding is used for training speech recognition models, then the model can perform complete speech recognition tasks, but the decoding process becomes lengthy and training efficiency deteriorates
Solution Approach 1:
The patent segments the speech recognition system into separate components: an encoding model that extracts speech features independently, and a language model that performs decoding separately. This segmentation allows the encoding and decoding processes to be optimized independently, reducing the overall decoding time and improving training efficiency while maintaining complete speech recognition functionality.
Solution Approach 2:
The encoding model performs preliminary extraction of speech features before the language model performs decoding. By pre-processing the speech input to obtain compact feature representations, the system reduces the amount of data that needs to be processed during decoding, thereby shortening the decoding process time and improving overall training efficiency.
2Manufacturing precision
If conventional end-to-end decoding is used, then the model structure is simple, but recognition accuracy deteriorates due to inefficient training
Solution Approach 1:
The patent divides the speech recognition model into two specialized sub-models: an encoding model for feature extraction and a language model for decoding. This segmentation allows each sub-model to be trained independently on optimized data representations, improving recognition accuracy while the modular structure keeps the overall system manageable despite increased complexity.
Solution Approach 2:
The patent introduces an intermediary interface between the encoding model and language model that processes speech features into a standardized representation format. This intermediary layer enables efficient information transfer between models and allows for optimized training of each component, thereby improving recognition accuracy without requiring the entire end-to-end system to be retrained.
3Reliability
If lengthy decoding processes are used for training, then complete speech recognition can be achieved, but training time increases significantly
Solution Approach 1:
The encoding model performs preliminary feature extraction and representation learning before the language model performs decoding. This pre-processing step creates optimized feature representations that reduce the computational burden during decoding, allowing complete speech recognition to be achieved with shorter training durations by avoiding redundant processing of already-encoded speech information.
Solution Approach 2:
The patent implements a feedback mechanism where the language model's decoding performance is used to refine the encoding model's feature extraction. By iteratively optimizing the encoding features based on decoding feedback, the system achieves complete and accurate speech recognition with reduced training time, as each iteration builds upon improved representations rather than requiring complete reprocessing of the entire speech signal.
Data Source
AI summary
Embodiments of the disclosure relate to a method, apparatus, device and storage medium for training a speech recognition model that includes an encoding model and a language model. An example method includes: generating, with the encoding model, a speech feature sequence of a speech sample; processing, with the language model, the speech feature sequence to generate probability information; providing the speech feature sequence to a reference model corresponding to the language model, to obtain a set of recognized texts corresponding to the speech sample; determining a training loss based on the probability information, the set of recognized texts, and a labeled text corresponding to the speech sample; and adjusting parameters of the speech recognition model based on the training loss.