Speaker Anonymization in Automatic Speech Recognition
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing automatic speech recognition (ASR) systems face challenges in balancing accuracy and user privacy, particularly when performing ASR on-device due to resource constraints and when transmitting raw audio data or embedding vectors to servers, which can compromise user data privacy.
Innovation Solution
A flexible gradient reversal based speaker anonymization framework that generates privacy-enhanced embedding vectors on the client device, anonymizing speaker information without increasing the word error rate, and uses these vectors for accurate ASR on the server, ensuring user privacy by removing identifiable features like prosody and pitch.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If speaker anonymization is performed using traditional methods (feature perturbation, voice normalisation, utterance slicing, differential pitch anonymization), then user privacy is protected, but computational cost increases and device resources are consumed
Solution Approach 1:
The patent extracts speaker identity information from the audio embedding by separating it into speaker-specific features and content-specific features. The speaker anonymization module removes or obfuscates the speaker identity components while preserving the linguistic content, achieving privacy protection without requiring full voice conversion or speech synthesis operations.
Solution Approach 2:
The audio embedding is segmented into distinct components: speaker identity information and linguistic content. This segmentation allows selective processing where only the speaker identity portion is anonymized while the content portion remains intact, reducing computational overhead compared to anonymizing the entire audio signal.
2Measurement precision
If raw audio data or embedding vectors are transmitted to servers for ASR processing, then ASR accuracy is improved, but user data privacy is compromised
Solution Approach 1:
The patent introduces an intermediary processing step where speaker anonymization is performed on the embedding vectors before transmission to the server. This intermediary module acts as a bridge between the client device and server, removing speaker identity information while preserving linguistic content, thus enabling accurate ASR processing on the server without exposing private user data.
3Reliability
If speaker anonymization is performed using neural-based speech synthesizers or voice converters, then speaker identity information is removed, but device resources are exceeded and computational expense increases
Solution Approach 1:
The patent employs a lightweight speaker anonymization module that operates directly on embedding vectors rather than requiring full speech synthesis or voice conversion models. This disposable-like approach processes only the essential speaker identity components in the embedding space, achieving anonymization with minimal computational resources and model complexity.
4Reliability
If domain adversarial training is used to learn speaker-invariant representations, then speaker anonymization is achieved, but ASR performance is reduced
Solution Approach 1:
The patent applies partial anonymization by selectively removing only the speaker identity components from the embedding while preserving the linguistic content components. This partial action approach avoids the excessive anonymization that would remove all speaker-related information including prosody and intonation that carry linguistic meaning, thus maintaining ASR performance while achieving privacy protection.
Data Source
AI summary
Broadly speaking, the present disclosure relates to a computer-implemented method for training a machine learning, ML, automatic speech recognition, ASR, model. The method comprises injecting a speaker anonymiser, which is configured to cause the ML ASR model to generate anonymised acoustic embeddings for the ML ASR model, at one or more layers of the ML ASR model, and suitably training the ML ASR model including the speaker anonymiser on audio data comprising an utterance with one or more words to be recognised. Correspondingly, there is also described a computer implemented method for performing automatic speech recognition using the trained ML ASR model and system for training/inference thereof.