Attention-Based E2E ASR Training Large Margin Optimization
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current Minimum Bayes Risk (MBR) training methods for automatic speech recognition (ASR) are inefficient due to exponential growth in search space, leading to suboptimal performance compared to Maximum a Posterior (MAP) decoding, despite generating multiple hypotheses during training.
Innovation Solution
An attention-based end-to-end (E2E) ASR training method that performs cross-entropy training, beam searching, and determines character-based and word-based gradients to maximize the distance between a reference sequence and the best hypothesis, using backpropagation to update the model, thereby focusing on a sequence-wise large margin training scheme.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If MBR training is used to minimize expected risk on output hypotheses, then recognition accuracy is improved, but decoding efficiency deteriorates due to exponential growth in search space
Solution Approach 1:
The patent extracts only the necessary component (one-best hypothesis) from the complete set of hypotheses generated by beam search, rather than processing all hypotheses in the n-best list. This extraction approach maintains recognition accuracy by focusing on the most probable hypothesis while avoiding the computational burden of processing the entire hypothesis space.
Solution Approach 2:
Instead of following the conventional MBR approach of processing multiple hypotheses to find the optimal output, the patent inverts the approach by using a simplified loss function that directly optimizes based on the one-best hypothesis from beam search. This inversion maintains the benefit of multiple hypothesis generation while avoiding the computational complexity of full MBR decoding.
2Productivity
If n-best list or confusion network based methods are used to improve MBR decoding efficiency, then decoding speed is improved, but implementation complexity increases
Solution Approach 1:
The patent extracts only the one-best hypothesis from the beam search results, discarding the need for complex n-best list processing or confusion network construction. This extraction simplifies the implementation while maintaining the efficiency benefits of beam search.
Solution Approach 2:
The patent uses a simple, computationally inexpensive loss function that can be quickly computed and discarded after each training iteration, replacing the need for complex, long-lived data structures like confusion networks. This approach prioritizes computational efficiency over structural complexity.
3Productivity
If beam search decoding based on MAP is used for evaluation, then decoding efficiency is improved, but training-performance mismatch occurs
Solution Approach 1:
The patent changes the optimization parameter from expected risk minimization (MBR) to a simplified margin-based loss function that directly optimizes for the one-best hypothesis performance. This parameter change aligns the training objective with the MAP decoding evaluation, eliminating the performance mismatch while maintaining decoding efficiency.
Data Source
AI summary
A method of attention-based end-to-end (E2E) automatic speech recognition (ASR) training, includes performing cross-entropy training of a model, based on one or more input features of a speech signal, performing beam searching of the model of which the cross-entropy training is performed, to generate an n-best hypotheses list of output hypotheses, and determining a one-best hypothesis among the generated n-best hypotheses list. The method further includes determining a character-based gradient and a word-based gradient, based on the model of which the cross-entropy training is performed and a loss function in which a distance between a reference sequence and the determined one-best hypothesis is maximized, and performing backpropagation of the determined character-based gradient and the determined word-based gradient to the model, to update the model.