Multi-Pass Speech Enhancement With DNN Feedback Noise Suppression
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing neural network architectures for speech enhancement in single-channel audio signals require significant processing power and memory, making them unsuitable for low-power edge devices like battery-powered headsets or earbuds, and often result in speech distortion or noise leakage.
Innovation Solution
A multi-pass neural network system that combines a deep neural network (DNN) with a statistical signal processor (SSP) to iteratively refine speech signals, using a feedback loop to recursively filter residual noise, reducing the need for additional memory and computational resources.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If larger neural networks are used for speech enhancement, then speech enhancement accuracy is improved, but processing power and memory requirements increase
Solution Approach 1:
The patent segments the speech enhancement task into multiple passes, where each pass processes the audio signal through the same neural network model. Instead of using one large neural network, the system applies a smaller network iteratively multiple times (e.g., 2-5 passes), achieving comparable accuracy to larger networks while reducing memory requirements and power consumption.
Solution Approach 2:
The patent implements continuous refinement by feeding the output of each pass back as input to the next pass. This iterative process continuously improves speech enhancement quality without requiring additional model complexity, maintaining accuracy while optimizing resource usage through repeated application of the same computational structure.
2Measurement precision
If larger neural networks are used for speech enhancement, then speech enhancement accuracy is improved, but device complexity increases
Solution Approach 1:
The patent divides the enhancement task into temporal segments (multiple passes) rather than increasing spatial complexity (network size). Each pass uses the same relatively simple network structure, but the iterative temporal processing achieves cumulative improvement in accuracy, effectively trading temporal computation for spatial simplicity.
Solution Approach 2:
The system applies periodic refinement through multiple discrete passes, where each pass periodically processes the signal to further reduce noise. This periodic application of the same transformation achieves progressive improvement without requiring increasingly complex network architectures.
3Use of energy by moving object
If smaller neural networks are used to reduce resource consumption, then processing power and memory requirements are reduced, but speech enhancement accuracy deteriorates
Solution Approach 1:
The patent compensates for the limited capability of smaller networks by applying them continuously across multiple passes. Each pass provides incremental improvement, and the cumulative effect of repeated application achieves accuracy levels comparable to larger single-pass networks, effectively using temporal repetition to overcome spatial limitations.
Solution Approach 2:
The system implements feedback by feeding the enhanced output from each pass back as input to the next pass. This feedback loop allows the smaller network to iteratively refine its output, progressively improving speech enhancement quality without requiring the network to be larger or more complex in a single transformation.
Data Source
AI summary
This disclosure provides methods, devices, and systems for audio signal processing. The present implementations more specifically relate to multi-pass neural networks configured for speech enhancement. In some aspects, a speech enhancement system may include a deep neural network (DNN) and a statistical signal processor (SSP). The DNN is configured to receive an input audio signal and infer a speech signal representing a speech component of the input audio signal based on a neural network model. The SSP is configured to further denoise the speech signal output by the DNN based on one or more statistical signal processing operations. In some implementations, the denoised speech signal may be fed back into the DNN (as an input audio signal) for further speech enhancement. As such, the speech enhancement system may recursively filter or suppress residual noise in the speech signal over a number of passes or iterations of a feedback loop.