Speech Enhancement Aggressiveness Control for Natural Denoising
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing denoising techniques face a tradeoff between noise reduction and speech preservation, particularly when applied to diverse audio content such as dialog and music, leading to suboptimal results in either excessive noise or inadequate speech preservation.
Innovation Solution
Implementing an aggressiveness control parameter to manage the balance between speech preservation and noise reduction by adjusting the training and post-processing of machine learning models, including modifying training samples, target denoising masks, and model architecture, and using perceptual transformations based on human cochlear processing.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If aggressive speech enhancement processing is applied to improve speech intelligibility in noisy environments, then speech clarity is improved, but naturalness and quality of the speech deteriorate
Solution Approach 1:
The system dynamically adjusts the degree of enhancement processing applied to different frequency components and time segments of speech based on noise conditions and speech characteristics. The enhancement amount is varied adaptively rather than applying uniform processing, allowing optimization of both intelligibility and naturalness in different contexts.
Solution Approach 2:
The system changes multiple processing parameters including enhancement strength, spectral shaping, and temporal smoothing to balance intelligibility improvement with naturalness preservation. By adjusting these parameters based on speech quality metrics and noise levels, the system resolves the contradiction between aggressive enhancement and natural speech quality.
2Measurement precision
If speech enhancement processing is applied to improve speech quality, then speech clarity is improved, but musical sounds and background audio content are distorted or lost
Solution Approach 1:
The system applies enhancement processing selectively to speech frequency regions while preserving non-speech audio content in other frequency bands. By identifying speech segments and applying enhancement only where speech is present, the system improves speech clarity without distorting musical sounds or background audio that fall outside the speech spectrum.
Solution Approach 2:
The system uses an audio content analysis intermediary that identifies and classifies different audio components (speech, music, background sounds). This intermediary guides the enhancement processing to target only speech portions, preventing distortion of other audio content while still achieving speech clarity improvement.
3Object-affected harmful factors
If traditional speech enhancement methods are used to reduce noise, then noise suppression is achieved, but speech sounds unnatural and quality degrades
Solution Approach 1:
The system dynamically adapts the noise suppression strength based on the estimated noise level and speech characteristics. In high-noise conditions, stronger suppression is applied, while in lower-noise conditions, suppression is reduced to preserve naturalness. This dynamic adjustment resolves the contradiction between noise suppression effectiveness and speech quality preservation.
Solution Approach 2:
The system uses feedback from speech quality evaluation and naturalness metrics to adjust the enhancement and noise suppression parameters. When speech sounds unnatural, the system reduces processing intensity or adjusts spectral shaping to restore naturalness, thereby maintaining speech quality while still achieving noise suppression.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
A method for performing denoising on audio signals is provided. In some implementations, the method involves determining an aggressiveness control parameter value that modulates a degree of speech preservation to be applied. In some implementations, the method involves obtaining a training set of training samples, a training sample having a noisy audio signal and a target denoising mask. In some implementations, the method involves training a machine learning model, wherein the trained machine learning model is usable to take, as an input, a noisy test audio signal and generate a corresponding denoised test audio signal, and wherein the aggressiveness control parameter value is used for: 1) generating a frequency domain representation of the noisy audio signals included in the training set; 2) modifying the target denoising masks; 3) determining an architecture of the machine learning model; or 4) determining a loss during training of the machine learning model.