Deep RNN Speech Recognition Using Stacked LSTM and Bidirectional Networks
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current speech recognition technologies using neural networks, such as RNN-HMM hybrids, do not perform as well as deep feedforward networks, and Long Short-term Memory (LSTM) networks have had limited impact on speech recognition.
Innovation Solution
A deep recurrent neural network (RNN) system is proposed, which includes multiple stacked recurrent hidden layers, specifically implemented using Long Short-term Memory (LSTM) and bidirectional RNNs, to enhance speech recognition performance.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If RNN-HMM hybrid systems are used for speech recognition, then the system can leverage the structured probabilistic modeling of HMMs, but the recognition performance does not reach the level of deep feedforward networks
Solution Approach 1:
The patent merges HMMs and RNNs into a unified end-to-end trainable hybrid system where the RNN processes acoustic features and the HMM performs sequence decoding, allowing both components to work together while enabling gradient-based optimization of the entire system, thereby achieving high recognition accuracy without sacrificing the probabilistic strengths of HMMs
Solution Approach 2:
The system segments the speech recognition task into distinct functional components: feature extraction, acoustic modeling via RNN, and sequence decoding via HMM, allowing each component to be optimized independently while contributing to overall system performance
2Loss of information
If Long Short-term Memory (LSTM) networks are used, then the model can capture long-range dependencies, but the impact on speech recognition performance has been limited
Solution Approach 1:
The patent introduces an intermediary connection between the RNN acoustic model and HMM decoder, using a soft alignment mechanism that allows the LSTM to capture long-range dependencies while maintaining compatibility with the HMM decoding framework, thereby enabling both long-term context utilization and high recognition accuracy
3Reliability
If deep feedforward networks are used, then high recognition accuracy is achieved, but the system cannot effectively model temporal dynamics and sequential dependencies in speech
Solution Approach 1:
The patent replaces static deep feedforward network weights with dynamic RNN weights that evolve over time based on input sequences, allowing the system to adapt to temporal patterns in speech while maintaining the representational power of deep networks through multiple hidden layers
Data Source
AI summary
Deep recurrent neural networks applied to speech recognition. The deep recurrent neural networks (RNNs) are preferably implemented by stacked long short-term memory bidirectional RNNs. The RNNs are trained using end-to-end training with suitable regularisation.