RNN-T Audio Training with Gradient-Based Pruning for Lower Memory
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing RNN-T neural network training methods for audio processing are computationally inefficient and memory-intensive, leading to slow training speeds and reduced practicability.
Innovation Solution
A method involving encoding training audio data, predicting text labels, jointing the results, pruning based on gradient information, and adjusting network parameters to reduce memory usage and calculation power, enabling faster training.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If traditional RNN-T training methods are used, then audio processing accuracy is maintained, but memory consumption is high and training speed is slow
Solution Approach 1:
The patent segments the joint training process into separate encoder training and prediction network training stages. The encoder is trained first using CTC loss, then the prediction network is trained separately using RNN-T loss. This segmentation reduces the memory footprint by avoiding simultaneous computation of all components, thereby improving training speed while maintaining processing accuracy.
Solution Approach 2:
The patent extracts and removes redundant computational operations from the traditional RNN-T training process. Specifically, it eliminates the need to compute and store the full joint probability distribution matrix by using a simplified loss computation approach that only retains necessary gradient information, significantly reducing memory consumption.
2Measurement precision
If RNN-T operations are performed, then audio processing accuracy is improved, but calculation complexity increases and training becomes slower
Solution Approach 1:
The patent divides the complex RNN-T training process into distinct phases: encoder pre-training with CTC loss, and prediction network training with RNN-T loss. This segmentation simplifies each individual training stage by focusing on specific objectives, reducing calculation complexity while preserving the accuracy benefits of full RNN-T training.
Solution Approach 2:
The patent changes the training parameters and loss functions used at different stages. By using CTC loss for encoder pre-training instead of full RNN-T loss, and then using RNN-T loss only for the prediction network, the patent reduces computational complexity while maintaining the accuracy improvements that RNN-T provides.
Data Source
Figure 1~2
Figure 3~5
Figure 6
AI summary
The present disclosure provides a method and apparatus for training a neural network, and a method and apparatus for audio processing. The method includes: encoding (S101) training audio data input to an encoder network to obtain a first encoding result, and predicting a text label input to a prediction network to obtain a first prediction result; jointing (S102) the first encoding result with the first prediction result to obtain a first joint result; pruning (S 103) the first encoding result and the first prediction result according to the first joint result to obtain a second encoding result and a second prediction result; performing (S 104) a j oint processing on the second encoding result and the second prediction result input to a joiner network to obtain a second joint result, and adjusting network parameters of the encoder network, the prediction network and the joiner network according to the second j oint result.