Hybrid DNN Architecture for Speech Recognition Efficiency
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current automatic speech recognition (ASR) technologies are computationally intensive and require efficient training methods that can utilize available resources effectively, particularly in processing speech using neural networks.
Innovation Solution
The implementation of hybrid frame, phone, diphone, morpheme, and word-level Deep Neural Networks (DNNs) for model training, which involves aligning training data with phonemes, diphones, and words, normalizing features, and training separate DNN models for each level to improve recognition accuracy and resource efficiency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If traditional frame-level DNNs are used for speech recognition, then the system can process speech data, but the computational complexity is high and resource consumption is excessive
Solution Approach 1:
The patent segments the speech recognition task into multiple hierarchical levels: frame-level DNN for basic acoustic unit recognition, phoneme-level DNN for phoneme identification, diphone-level DNN for transition modeling, and word-level DNN for final recognition. Each level processes specific features and provides confidence scores that are combined to achieve accurate speech recognition with reduced computational burden at each stage
2Measurement precision
If multiple DNN models are trained at different levels (frame, phoneme, diphone, morpheme, word), then recognition accuracy is improved, but the training complexity and resource requirements increase
Solution Approach 1:
The training process is segmented into distinct stages corresponding to different linguistic levels. Each DNN model is trained independently on specific feature sets (e.g., frame-level on acoustic features, phoneme-level on phoneme-aligned features), allowing parallel training and reducing the complexity of training a single monolithic model while achieving high accuracy through hierarchical feature integration
3Reliability
If confidence scores from multiple DNN levels are combined, then recognition reliability is improved, but the processing time and computational overhead increase
Solution Approach 1:
The system performs preliminary processing at each hierarchical level, where frame-level DNN quickly identifies acoustic units and generates initial confidence scores. These preliminary results are then used by higher-level DNNs (phoneme, diphone, word levels) which refine the recognition and provide additional confidence scores. The scores from all levels are combined to produce the final reliable recognition result, with each level building upon the previous preliminary results
Data Source
AI summary
A hybrid frame, phone, diphone, morpheme, and word-level Deep Neural Networks (DNN) in model training and applications-is based on training a regular ASR system, which can be based on Gaussian Mixture Models (GMM) or DNN. All the training data (in the format of features) are aligned with the transcripts in terms of phonemes and words with the timing information and new features are formed in terms of phonemes, diphones, morphemes, and up to words. Regular ASR produces a result lattice with timing information for each word. A feature is then extracted and sent to the word-level DNN for scoring Phoneme features are sent to corresponding DNNs for training. Scores are combined to form the word level scores, a rescored lattice and a new recognition result.