Segmented Audio Enhancement for Noise and Reverberation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Amateur video recordings often suffer from low quality audio due to microphone distortion, heavy background noise, and reverberation, making the audio difficult to understand or uncomfortable to listen to.
Innovation Solution
A data-driven audio enhancement system that classifies audio into different time segments using deep neural networks to apply content-specific processing algorithms, enhancing speech, music, and noise segments, and recombines them to improve intelligibility and clarity.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If deep neural networks are used to classify and process audio segments, then audio quality and intelligibility are improved, but device complexity and computational resources increase
Solution Approach 1:
The audio signal is divided into multiple time segments that are classified and processed separately. The system identifies different audio content types (speech, music, noise) in different segments and applies appropriate processing to each, improving overall audio quality while managing computational complexity through localized processing.
Solution Approach 2:
Different processing algorithms are applied to different segments of the audio signal based on their content classification. Speech segments receive enhancement processing, music segments are preserved with minimal processing, and noise segments are suppressed, optimizing quality for each local segment rather than applying uniform processing.
2Measurement precision
If content-specific processing algorithms are applied to different audio segments, then speech clarity and intelligibility are enhanced, but processing time and computational load increase
Solution Approach 1:
The system performs preliminary classification of audio segments into different content types (speech, music, noise) before applying processing algorithms. This preliminary action enables efficient routing of segments to appropriate processing pipelines, reducing overall processing time by avoiding unnecessary processing of non-speech segments.
Solution Approach 2:
The system applies full enhancement processing only to speech segments that require it, while applying minimal or no processing to music and noise segments. This partial action approach focuses computational resources where they are most needed, reducing total processing time while maintaining speech clarity.
Data Source
Figure 1
Figure 2
Figure 3A~3B
AI summary
Systems and methods are disclosed for audio enhancement. For example, methods may include accessing audio data; determining a window of audio samples based on the audio data; inputting the window of audio samples to a classifier to obtain a classification, in which the classifier includes a neural network and the classification takes a value from a set of multiple classes of audio; selecting, based on the classification, an audio enhancement network from a set of multiple audio enhancement networks; applying the selected audio enhancement network to the window of audio samples to obtain an enhanced audio segment, in which the selected audio enhancement network includes a neural network that has been trained using audio signals of a type associated with the classification; and storing, playing, or transmitting an enhanced audio signal based on the enhanced audio segment.