Neural Network Training Using Segmented Labeled Sequence Data
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional neural network training methods for speech recognition are inefficient due to high computational complexity and memory usage, especially in resource-constrained devices like mobile phones, and often result in high latency and sensitivity to alignment errors in labeled data.
Innovation Solution
A computer-implemented method for training neural networks using segmented, labeled data that employs a forward-backward pass mechanism with Region of Target (ROT) information to restrict output paths, compute modified variables, and update weights and biases adaptively using momentum and Root Mean Square (RMS) algorithms, reducing computational complexity and ensuring efficient training without latency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If conventional neural network training methods are used for speech recognition, then the system can achieve phoneme recognition capability, but the computational complexity and memory usage become excessively high
Solution Approach 1:
The patent segments the training data into distinct frames with explicit temporal labels indicating which frames correspond to which phonemes. This segmentation allows the neural network to process smaller, labeled portions of data independently, reducing the computational burden of processing entire sequences while maintaining recognition accuracy through the structured temporal information.
2Reliability
If conventional neural network training methods are used for speech recognition, then the system can achieve phoneme recognition capability, but the memory usage becomes excessively high
Solution Approach 1:
By dividing the training data into discrete frames with associated phoneme labels, the patent enables more efficient memory management. The segmented structure allows the system to process and discard frames individually rather than maintaining entire sequences in memory, significantly reducing memory requirements while preserving the temporal relationships necessary for accurate phoneme recognition.
3Reliability
If traditional RNN combined with HMM is used to label input sequences, then the system can handle sequential data, but the latency increases and resource efficiency decreases
Solution Approach 1:
The patent applies preliminary labeling to the training data during the training phase, where each frame is pre-assigned phoneme labels indicating which phoneme is present during that frame. This preliminary action eliminates the need for complex post-processing and decoding steps during inference, significantly reducing latency while maintaining the ability to process sequential speech data effectively.
4Reliability
If conventional training methods are used, then the neural network can be trained for phoneme recognition, but the training process is slow and convergence is slow
Solution Approach 1:
By pre-labeling the training frames with phoneme information before training begins, the patent provides the neural network with structured supervision signals from the start. This preliminary action eliminates the need for complex iterative alignment procedures during training, allowing the network to learn more efficiently and converge faster while maintaining phoneme recognition accuracy.
Data Source
AI summary
Classification training systems and methods include a neural network for classification of input data, a training dataset providing segmented labeled training data, and a classification training module operable to train the neural network using the training data. A forward pass processing module is operable to generate neural network outputs for the training data using weights and bias for the neural network, and a backward pass processing module is operable to update the weights and biases in a backward pass, including obtaining Region of Target (ROT) information from the training data, generate a forward-backward masking based on the ROT information, the forward-backward masking placing at least one restriction on a neural network output path, compute modified forward and backward variables based on the neural network outputs and the forward-backward masking, and update the weights and biases.