Speech Enhancement Masking to Mitigate Over-Suppression
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing machine learning methods for speech enhancement suffer from speech over-suppression, leading to speech distortion, discontinuity, and loss of unvoiced sounds or non-noise signals like laughter and applause, especially in non-stationary noise or low SNR conditions.
Innovation Solution
A system is trained with a non-linear, asymmetric loss function to penalize speech over-suppression more than under-suppression, using machine learning models to generate masks that adjust for speech presence and detect laughter or applause, applying these masks to enhance audio quality by reducing noise and preserving non-noise signals.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Object-affected harmful factors
If traditional machine learning methods are used for speech enhancement, then noise reduction is improved, but speech over-suppression occurs leading to distortion and loss of non-noise signals
Solution Approach 1:
The patent applies asymmetry by using an asymmetric loss function that penalizes speech over-suppression more heavily than under-suppression. This asymmetric penalty structure allows the system to aggressively reduce noise while maintaining speech and non-noise signals, resolving the contradiction between noise reduction and signal preservation.
Solution Approach 2:
The patent implements dynamics through adaptive mask adjustment mechanisms that dynamically modify suppression levels based on signal characteristics. The system continuously adapts the masking function to preserve speech and non-noise signals while reducing noise, allowing flexible response to varying audio conditions.
2Object-affected harmful factors
If aggressive noise suppression is applied, then noise reduction is improved, but voice naturalness deteriorates due to sharp decay and discontinuity
Solution Approach 1:
The system uses dynamic mask adjustment that adapts suppression levels in real-time based on voice characteristics. This dynamic approach prevents sharp decay and maintains voice naturalness while still achieving aggressive noise reduction where appropriate.
Solution Approach 2:
The patent incorporates feedback mechanisms where the asymmetric loss function continuously monitors speech quality and adjusts suppression levels accordingly. This feedback loop ensures that noise reduction does not compromise voice naturalness, maintaining stability in voice composition.
3Measurement precision
If speech enhancement is optimized for clarity, then speech retention is improved, but other non-noise signals like laughter and applause are suppressed
Solution Approach 1:
The patent applies universality by designing a mask adjustment system that handles multiple signal types (speech, laughter, applause) through a unified asymmetric loss function. This multi-functional approach preserves all non-noise signals while maintaining speech detection accuracy, avoiding selective suppression.
Solution Approach 2:
The asymmetric loss function creates different penalty structures for different types of signal suppression, allowing the system to preserve diverse non-noise signals while maintaining speech clarity. The asymmetric design recognizes that suppressing any non-noise signal is more harmful than suppressing noise.
Data Source
Figure 1~2
Figure 3~4
Figure 5
AI summary
A system for mitigating over-suppression of speech and other non-noise signals is disclosed. In some embodiments, a system is programmed to train a first machine learning model for speech detection or enhancement using a non-linear, asymmetric loss function that penalizes speech over-suppression more than speech under-suppression. The first machine learning model is configured to receive an audio signal and generate a mask indicating an amount of speech present in the audio signal. The mask can be adjusted to remedy sharp voice decay resulting from speech over-suppression. The system is also programmed to train a second machine learning model for laughter or applause detection. The system is further programmed to improve the quality of a new audio signal by applying an adjusted mask to the new audio signal except for the portions of the audio signal that have been identified as corresponding to laughter or applause.