Audiographic Image Augmentation for Audio ML
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing data augmentation techniques for machine learning models in audio processing are computationally slow and require additional data sources, making them inefficient and complex, especially for automatic speech recognition tasks.
Innovation Solution
The method involves applying augmentation techniques directly to audiographic images that represent audio signals, such as spectrograms, rather than raw audio data, to generate augmented training data, which can be processed more efficiently and without additional data sources, allowing for online augmentation during model training.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If data augmentation is performed on raw audio data, then additional training data is generated, but the process becomes computationally slow and complex
Solution Approach 1:
The patent introduces spectrograms as an intermediary representation between raw audio data and the machine learning model. Instead of augmenting raw audio directly, the system converts audio to spectrograms and performs augmentation operations (such as adding noise, time stretching, pitch shifting) on the spectrogram domain. This intermediary approach simplifies the augmentation process and improves computational efficiency while still generating diverse training data.
Solution Approach 2:
The patent replaces the mechanical audio processing operations with equivalent operations in the spectrogram domain. For example, instead of physically manipulating audio waveforms, the system applies transformations to the visual spectrogram representation, which can be processed more efficiently by image processing algorithms and parallel computation frameworks.
2Adaptability or versatility
If data augmentation is performed on raw audio data, then training data diversity is improved, but additional data sources are required
Solution Approach 1:
The spectrogram serves as a self-contained intermediary that captures the essential features of audio data. By performing all augmentation operations on spectrograms derived from the original audio, the system eliminates the need for additional external data sources such as separate noise recordings or audio databases, while still achieving diverse training data.
Solution Approach 2:
The system uses the original audio data itself to generate all necessary augmentation variations. Through spectrogram-based operations like adding synthetic noise, applying time-frequency transformations, and modifying spectral characteristics, the system generates diverse training data without requiring external data sources, making the system self-sufficient.
3Productivity
If data augmentation is performed offline in advance, then model training efficiency is improved, but the ability to adapt to new data is reduced
Solution Approach 1:
The patent enables dynamic data augmentation where spectrogram transformations can be applied on-the-fly during model training rather than being fixed in advance. This allows the system to adapt augmentation parameters and strategies based on the current training state and data characteristics, improving both efficiency and adaptability.
Solution Approach 2:
The system performs preliminary conversion of audio to spectrograms, which can then be rapidly augmented during training. This preliminary processing step enables efficient online augmentation because the spectrogram representation is computationally lightweight and can be transformed quickly, allowing the model to adapt to new data patterns while maintaining training efficiency.
Data Source
AI summary
Generally, the present disclosure is directed to systems and methods that generate augmented training data for machine-learned models via application of one or more augmentation techniques to audiographic images that visually represent audio signals. In particular, the present disclosure provides a number of novel augmentation operations which can be performed directly upon the audiographic image (e.g., as opposed to the raw audio data) to generate augmented training data that results in improved model performance. As an example, the audiographic images can be or include one or more spectrograms or filter bank sequences.