Input-Feeding Architecture for Attention-Based Speech Recognition
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing end-to-end speech recognition systems, particularly attention-based systems like LAS architecture, face limitations in improving speech recognition performance due to suboptimal input feeding mechanisms, leading to higher word error rates.
Innovation Solution
The proposed method enhances the LAS speech recognition system by implementing an input-feeding architecture that generates encoder hidden states, computes attention weights, and updates the decoder's hidden state using a compatibility score, resulting in improved alignment and context vector calculations, which are then used to generate more accurate label predictions.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If the conventional LAS architecture is used with limited input feeding (only embedded prediction and context vector), then the system structure is simpler, but the speech recognition performance and alignment accuracy deteriorate
Solution Approach 1:
The patent applies preliminary action by feeding the full attentional hidden state information from the previous time step into the current decoder as input, before the actual decoding operation occurs. This pre-provision of comprehensive contextual information enables the decoder to make more accurate predictions and alignments from the outset, improving speech recognition accuracy without requiring structural changes to the overall architecture.
Solution Approach 2:
The patent implements feedback by creating a recursive loop where the attentional hidden state from the previous time step is fed back into the current decoder input. This feedback mechanism allows the system to continuously refine its predictions by incorporating accumulated contextual information from previous processing steps, progressively improving alignment accuracy and speech recognition performance.
2Measurement precision
If only embedded prediction information is fed to the decoder, then the computational load is reduced, but the alignment between encoder and decoder deteriorates
Solution Approach 1:
The system performs preliminary computation by calculating and storing the full attentional hidden state information from the previous time step, making this comprehensive information available before the current decoding operation. This preliminary preparation enables more accurate alignment calculations without requiring redundant computations during the actual decoding process.
Solution Approach 2:
The attentional hidden state serves as an intermediary that bridges the encoder and decoder components. By feeding this intermediate representation that captures the essential alignment information from previous steps, the system enables more precise encoder-decoder alignment without requiring direct complex interactions between all encoder and decoder elements.
3Loss of information
If the full attentional hidden state information is fed to the decoder, then the prediction accuracy is improved, but the information loss from previous steps is reduced
Solution Approach 1:
The feedback mechanism recursively incorporates the full attentional hidden state from previous time steps into the current decoder input, ensuring that contextual information is preserved and accumulated over time. This feedback loop prevents information loss by continuously reintroducing relevant historical context into the active processing stream.
Solution Approach 2:
The attentional hidden state serves multiple functions simultaneously: it acts as the decoder output from the previous time step, serves as the basis for computing attention weights, and functions as the comprehensive input for the current decoder. This multi-functionality maximizes the utility of the computed information without requiring separate processing paths.
Data Source
AI summary
Methods and apparatuses are provided for performing end-to-end speech recognition training performed by at least one processor. The method includes receiving, by the at least one processor, one or more input speech frames, generating, by the at least one processor, a sequence of encoder hidden states by transforming the input speech frames, computing, by the at least one processor, attention weights based on each of the sequence of encoder hidden states and a current decoder hidden state, performing, by the at least one processor, a decoding operation based on a previous embedded label prediction information and a previous attentional hidden state information generated based on the attention weights; and generating a current embedded label prediction information based on a result of the decoding operation and the attention weights.