Neural Network State Tying for Speech Recognition
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional neural network-based speech recognition systems rely on hybrid infrastructure that combines neural networks with Gaussian Mixture Models (GMM) and Hidden Markov Models (HMM), leading to a mismatched feature set and model family, which limits their effectiveness and adaptability.
Innovation Solution
The approach involves training a neural network using context-dependent states derived from context-independent states, directly optimizing state tying on activation vectors without relying on GMM/HMM systems, using decision trees and K-means clustering to define tied states, and integrating these states into a purely neural network-based infrastructure for speech recognition.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If a hybrid infrastructure combining neural networks with GMM and HMM is used, then speech recognition functionality is provided, but mismatched feature sets and model families limit effectiveness and adaptability
Solution Approach 1:
The patent merges the neural network with GMM/HMM systems by having the neural network output parameters that directly control the GMM/HMM components. The neural network processes acoustic features and generates state probabilities that are fed into the HMM framework, creating a unified hybrid architecture that leverages the pattern recognition capabilities of neural networks while maintaining the temporal modeling strengths of HMMs.
Solution Approach 2:
The neural network is designed to perform multiple functions within the hybrid system: it serves as both a feature extractor from raw acoustic signals and a probability estimator for HMM states. This multi-functional approach allows a single neural network component to support both the feature representation and the state classification tasks, improving overall system efficiency and adaptability.
2Ease of operation
If conventional hybrid systems are used, then speech recognition is enabled, but the mismatched feature set reduces recognition effectiveness
Solution Approach 1:
The patent transforms the feature representation by having the neural network output parameters specifically tailored for GMM/HMM consumption. The neural network converts raw acoustic features into a parameter space that is optimally matched for HMM state estimation, effectively bridging the feature mismatch problem through parameter transformation rather than requiring feature set alignment.
3Extent of automation
If neural networks are integrated with GMM/HMM, then speech recognition functionality is achieved, but dependence on external systems increases complexity
Solution Approach 1:
The neural network performs preliminary processing of acoustic features before they are passed to the GMM/HMM system. By pre-computing feature extractions and generating initial state probability estimates, the neural network reduces the computational burden on the subsequent HMM decoding stage, effectively simplifying the overall system architecture while maintaining automated recognition functionality.
Data Source
AI summary
The technology described herein can be embodied in a method that includes receiving an audio signal encoding a portion of an utterance, and providing, to a first neural network, data corresponding to the audio signal. The method also includes generating, by a processor, data representing a transcription for the utterance based on an output of the first neural network. The first neural network is trained using features of multiple context-dependent states, the context-dependent states being derived from a plurality of context-independent states provided by a second neural network.