Deep-Learning Echo Cancellation for Virtual Spatial Audio
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Echoes during communication in virtual spatial sound systems affect the auditory experience of users, particularly in scenarios where speakers in different rooms communicate and virtual sound directions change with speaker movement, leading to reverberation and degraded realism.
Innovation Solution
An echo cancellation method using deep learning, involving obtaining microphone signals from far-end and near-end rooms, transforming them into compressed complex number spectra, inputting these spectra into a trained neural network model for echo cancellation, and performing inverse short-time Fourier transform to recover clear near-end speech signals.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Illumination intensity
If virtual spatial sound system reconstructs speaker information to different directions, then auditory realism is improved, but echo and reverberation increase
Solution Approach 1:
The patent uses the far-end microphone signal (which contains the echo path information) as a reference signal to train the neural network model. By converting the harmful echo into useful training data, the system learns to identify and remove echo components from near-end speech signals, thereby improving auditory realism while eliminating the harmful reverberation effects
Solution Approach 2:
The patent introduces a trained neural network model as an intermediary between the microphone signals and the output speech signal. This intermediary processes the near-end speech signal by removing echo components based on patterns learned during training, thus separating the desired speech from the harmful reverberation while preserving the virtual spatial sound effects
2Measurement precision
If deep learning model processes compressed complex number spectra, then echo cancellation performance is improved, but computational complexity increases
Solution Approach 1:
The patent performs compression of the complex number spectra before inputting them to the neural network model. This preliminary compression reduces the dimensionality of the input data, making the subsequent deep learning processing more computationally efficient while preserving the essential features needed for accurate echo cancellation
Solution Approach 2:
The patent replaces traditional signal processing methods with a trained neural network model that operates on compressed complex number spectra. This substitution enables the system to achieve high echo cancellation performance through learned patterns rather than complex mechanical filtering operations, improving both accuracy and computational efficiency
Data Source
AI summary
The present application provides an echo cancellation method based on deep learning, a device, and a readable storage medium. A far-end microphone signal corresponding to a far-end room is obtained, and a near-end microphone signal corresponding to a near-end room is obtained; the far-end microphone signal is used as a reference signal, a first compressed complex number spectrum corresponding to the reference signal is obtained, and a second compressed complex number spectrum corresponding to the near-end microphone signal is obtained; the first compressed complex number spectrum and the second compressed complex number spectrum are input to a trained neural network model for echo cancellation, and a near-end speech compressed complex number spectrum is output; and inverse short-time Fourier transform is performed on the near-end speech compressed complex number spectrum to obtain a clear near-end speech signal.