Unified AEC and PNS Machine Learning Model for Audio Clarity
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Online audio and video conferencing platforms face challenges with acoustic echoes and background noise, which degrade audio quality and raise privacy concerns due to the inability of current systems to effectively distinguish between the target speaker and interfering speakers.
Innovation Solution
A machine learning model trained for both acoustic echo cancellation (AEC) and personalized noise suppression (PNS) is used, which extracts speaker embedding vectors to filter out interfering audio sources, allowing for real-time processing and effective removal of echoes and background noise without requiring separate models for each user.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Object-affected harmful factors
If unconditional Speech Enhancement models are used to remove background noise, then background noise is reduced, but speech from interfering speakers is incorrectly removed along with the target speaker's speech
Solution Approach 1:
The patent applies local quality by transitioning from unconditional speech enhancement that treats all speech uniformly to personalized speech enhancement that applies different processing to different speakers. The system uses speaker embedding vectors to identify and distinguish the target speaker from interfering speakers, applying noise suppression selectively only to background noise while preserving the target speaker's speech and removing only the speech of interfering speakers. This localized approach to quality improvement resolves the contradiction by preventing information loss while reducing background noise.
2Reliability
If separate models are used for AEC and PNS, then each function can be optimized independently, but computational resources and model size increase
Solution Approach 1:
The patent merges the AEC and PNS functions into a single unified model that performs both echo cancellation and personalized noise suppression simultaneously. This combined model processes the audio signal through integrated layers that handle both functions, reducing the total computational resources and model size compared to using separate models for each function while maintaining optimized performance for both AEC and PNS through the unified architecture.
3Measurement precision
If speaker embedding vectors are extracted and used for personalized noise suppression, then the ability to distinguish target speaker from interfering speakers improves, but processing time and computational complexity increase
Solution Approach 1:
The patent applies preliminary action by extracting and storing speaker embedding vectors during an enrollment phase before the actual communication occurs. These pre-computed embedding vectors are stored in a database and can be quickly retrieved and matched during real-time communication, avoiding the need to perform complex speaker identification and embedding extraction during the actual speech processing, thus reducing processing time while maintaining high speaker identification accuracy.
Data Source
AI summary
A data processing system implements receiving a far-end signal associated with a first computing device participating in an online communication session and receiving a near-end signal associated with a second computing device participating in the online communication session. The near-end signal includes speech of a target speaker, a first interfering speaker, and an echo signal. The system further implements providing the far-end signal, the near-end signal, and an indication of the target speaker as an input to a machine learning model. The machine learning model trained to analyze the far-end signal and the near-end signal to perform personalized noise suppression (PNS) to remove speech from one or more interfering speakers and acoustic echo cancellation (AEC) to remove echoes. The model is trained to output an audio signal comprising speech of the target speaker. The system obtains the audio signal comprising the speech of the target speaker from the model.