Speech Recognition Device Using End-to-End Neural Network and WFST Decoder
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional speech recognition methods using End-to-End neural networks face challenges in achieving high accuracy due to the need for forced modifications and the difficulty in combining neural network outputs with language models, leading to suboptimal recognition performance.
Innovation Solution
A speech recognition device employing an End-to-End neural network with a posterior probability calculator and a decoder using a Weighted Finite-State Transducer (WFST) framework, which integrates word and sub-word level language models to calculate the highest probability word sequence from observed acoustic features, eliminating the need for forced modifications and improving recognition accuracy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If conventional DNN-HMM hybrid method is used to calculate output probability, then speech recognition accuracy is improved compared to GMM, but the neural network output must be modified by applying Bayes' theorem which complicates the decoding process
Solution Approach 1:
The patent extracts and removes the Bayes' theorem modification step from the decoding process. By using an End-to-End neural network that directly outputs the desired probability distribution without requiring conversion through Bayes' theorem, the complex modification step is taken out of the system, simplifying the decoding process while maintaining high accuracy.
Solution Approach 2:
The patent segments the speech recognition system into distinct components: an End-to-End neural network for direct probability calculation, a separate language model for contextual information, and a simplified decoder. This segmentation allows each component to perform its specific function without the need for complex interactions and modifications between them.
2Device complexity
If End-to-End neural network is used to directly represent posterior probability, then the need for forced modification is eliminated, but the difficulty in combining neural network outputs with language models remains
Solution Approach 1:
The patent merges the End-to-End neural network output with the language model through a unified probability calculation framework. The neural network provides acoustic likelihoods while the language model provides contextual probabilities, and these are combined in a coherent probabilistic manner that maintains both the direct output capability of the neural network and the contextual awareness of the language model.
Solution Approach 2:
The patent introduces a probabilistic framework as an intermediary between the End-to-End neural network and the language model. This intermediary layer harmonizes the outputs from both sources, allowing them to work together effectively without requiring forced modifications to either component.
3Adaptability or versatility
If DNN output representing posterior probability is used in conventional HMM framework, then the framework compatibility is maintained, but the output does not fit the conventional framework that employs output probability
Solution Approach 1:
Instead of forcing the neural network output to fit the conventional HMM framework by applying Bayes' theorem, the patent inverts the approach by using an End-to-End neural network that directly outputs the desired probability distribution. This reversal eliminates the need for framework forcing and allows the system to work in the most natural and accurate manner.
Data Source
AI summary
A speech recognition device includes: an acoustic model based on an End-to-End neural network responsive to an observed sequence formed of prescribed acoustic features obtained from a speech signal by feature extracting unit, for calculating probability of the observed sequence being a certain symbol sequence; and a decoder responsive to a symbol sequence candidate, for decoding a speech signal by a WFST based on a posterior probability of each of word sequences corresponding to the symbol sequence candidate, probabilities calculated by the acoustic model for symbol sequences selected based on an observed sequence, and a posterior probability of each of the plurality of symbol sequences.