Per-epoch Data Augmentation for Acoustic Model Training
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing speech analytics systems struggle to perform effectively in real-world conditions due to mismatches in noise, reverberation, echo, level, and equalization, which are not adequately addressed by conventional multi-style training methods.
Innovation Solution
The proposed method involves augmenting clean speech training data with noise, reverberation, and echo, and varying the level and equalization during the training process, while performing data augmentation in the feature domain within the training loop, allowing for more diverse and realistic training scenarios.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If conventional multi-style training is used, then training process is simple, but system performance in real-world conditions deteriorates due to mismatch in noise, reverberation, echo, level, and equalization
Solution Approach 1:
The patent implements dynamic data augmentation where training parameters (noise type, reverberation level, echo amount) are varied randomly during each training epoch rather than using fixed augmentation settings. This allows the model to learn robust features across diverse acoustic conditions without requiring manual curation of multiple training datasets, resolving the contradiction between training simplicity and real-world performance.
Solution Approach 2:
The patent systematically varies multiple acoustic parameters including noise signal-to-background ratio, reverberation time, echo level, and frequency equalization settings during training. By changing these parameters dynamically across epochs, the system achieves high reliability in real-world conditions while maintaining a unified training pipeline, avoiding the complexity of managing multiple fixed training configurations.
2Adaptability or versatility
If data augmentation is performed in feature domain within training loop, then training efficiency and adaptability improve, but computational cost increases
Solution Approach 1:
The patent pre-computes and stores acoustic feature transformations (such as noise addition matrices, reverberation impulse responses, and equalization filters) before the training loop begins. During training, these pre-computed features are applied to the input data through efficient matrix operations rather than performing complex acoustic modeling in real-time, significantly reducing per-epoch computational cost while maintaining high adaptability to diverse acoustic conditions.
Solution Approach 2:
The patent replaces complex physical acoustic modeling with linear algebra operations in the feature domain. Instead of simulating full acoustic wave propagation, the system uses matrix multiplications and convolutions to apply noise, reverberation, and echo effects to feature representations, achieving the same adaptive training效果 with much lower computational cost and higher training efficiency.
Data Source
Figure 1
Figure 1A
Figure 1B
AI summary
In some embodiments, methods and systems for training an acoustic model, where the training includes a training loop (including at least one epoch) following a data preparation phase. During the training loop, training data are augmented to generate augmented training data. During each epoch of the training loop, at least some of the augmented training data is used to train the model. The augmented training data used during each epoch may be generated by differently augmenting (e.g., augmenting using a different set of augmentation parameters) at least some of the training data. In some embodiments, the augmentation is performed in the frequency domain, with the training data organized into frequency bands. The acoustic model may be of a type employed (when trained) to perform speech analytics (e.g., wakeword detection, voice activity detection, speech recognition, or speaker recognition) and/or noise suppression.