Frame-Level Normalization for Speech Recognition Neural Networks
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Traditional batch normalization methods are not suitable for time-series data like speech signals, leading to mismatched learning and recognition processes, and require extensive training data for individualization, making them inconvenient and time-consuming.
Innovation Solution
A processor-implemented method that applies frame-level normalization to neural networks by calculating and updating statistical values such as averages and variances for each frame of a speech sequence, allowing for real-time processing and adaptation to individual users.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If typical batch normalization is performed using image data, then normalization can be applied to batch data, but it cannot be performed using time-series data such as speech signals because portions of the time-series data are highly associated with each other based on a lapse of time
Solution Approach 1:
The patent applies dynamics by transitioning from static batch-level normalization to dynamic frame-level normalization. The statistical values (mean and variance) are calculated and updated for each individual frame rather than for the entire batch at once, allowing the normalization to adapt to the temporal dynamics of speech signals where each frame is highly associated with previous frames.
Solution Approach 2:
The patent segments the batch normalization process into frame-level operations. Instead of processing the entire batch as a single unit, the normalization is applied independently to each frame by calculating statistical values for that specific frame and its predecessors, enabling time-series data to be processed effectively.
2Measurement precision
If a great amount of training data is collected to train the entire network for individualizing speech recognition to a particular user, then recognition accuracy for that user improves, but the training process becomes inconvenient and time-consuming
Solution Approach 1:
The patent applies local quality by performing normalization at the frame level rather than requiring global retraining of the entire network. Each frame is normalized using its own statistical values, allowing the system to adapt to individual users through incremental frame-by-frame processing rather than requiring extensive batch training data.
Solution Approach 2:
The patent implements preliminary action by pre-calculating and storing statistical values for each frame during the recognition process. These pre-computed statistical values enable rapid normalization without requiring real-time computation or extensive training data collection, thus reducing training time while maintaining user-specific accuracy.
3Ease of manufacture
If typical batch normalization is used in the learning process, then the learning process can be enabled, but the learning result does not match the actual recognition process because batch-unit processing cannot be enabled in real-time recognition
Solution Approach 1:
The patent achieves universality by designing a normalization approach that functions effectively in both learning and recognition phases. By calculating statistical values for each frame independently, the same normalization mechanism can be applied during training (learning) and during actual real-time recognition, ensuring consistency between the two processes without requiring batch-unit processing.
Data Source
AI summary
A processor-implemented speech recognition method includes: applying, to an input layer of a neural network, a frame of a speech sequence; obtaining an output of a hidden layer of the neural network corresponding to the frame; calculating a statistical value of at least one previous output of the hidden layer corresponding to at least one previous frame of the speech sequence; normalizing the output based on the statistical value; applying the normalized output to a subsequent layer of the neural network; and recognizing the speech sequence based on the applying of the normalized output.