Emotion-Obfuscating Audio for Voice Assistant Privacy
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Smart speaker voice assistants raise privacy concerns due to unauthorized surveillance of users' emotional data through speech emotion recognition, which can be exploited for targeted advertising and law enforcement, lacking effective methods to protect emotional information without disrupting speech transcription.
Innovation Solution
The DARE-GP system uses constrained genetic programming to generate universal adversarial audio perturbations (EONs) that mask emotional content in speech while preserving transcription, applicable to closed, off-the-shelf smart speakers without requiring access to the SER classifier.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of information
If speech emotion recognition is used to analyze user data, then emotional information can be extracted for targeted advertising and law enforcement, but user privacy regarding emotional data is compromised
Solution Approach 1:
The patent converts the harmful effect of emotion recognition into a beneficial privacy protection mechanism by generating adversarial noise that deliberately misleads SER systems. The same technical capability that enables emotion extraction is turned against itself to protect user privacy, causing the SER system to incorrectly classify emotions while preserving speech transcription accuracy.
Solution Approach 2:
The patent introduces adversarial noise as an intermediary between the user's speech and the SER system. This noise acts as a mediator that interferes with the SER system's ability to extract emotional information while allowing the speech transcription system to function normally, thus protecting privacy without completely blocking information flow.
2Loss of information
If adversarial noise is added to mask emotional content, then speech transcription accuracy is preserved, but the noise may interfere with speech signal quality
Solution Approach 1:
The adversarial noise is designed to affect only specific local characteristics of the speech signal that are relevant to emotion recognition, such as spectral features and prosody patterns. The noise is tailored to target emotion-related features while leaving the phonetic and lexical content intact, thus preserving speech transcription accuracy while masking emotional information.
Solution Approach 2:
The patent modifies specific parameters of the speech signal by adding adversarial noise that changes spectral characteristics, frequency distribution, and temporal patterns associated with emotional expression. These parameter changes are carefully controlled to affect only emotion-related features while maintaining the intelligibility and transcription accuracy of the speech.
3Productivity
If universal adversarial perturbations are generated for multiple users, then privacy protection is scalable, but the perturbations must work across diverse speech patterns and emotions
Solution Approach 1:
The patent creates a universal adversarial noise pattern that functions across multiple users, speech patterns, and emotional states. The noise is designed to be user-agnostic and emotion-agnostic, targeting fundamental acoustic features that are common to all human speech rather than user-specific characteristics, enabling scalable privacy protection without requiring individualized customization.
Solution Approach 2:
The adversarial noise is pre-generated and trained offline using diverse speech data from multiple users and emotional states. This preliminary training allows the system to learn robust, generalizable noise patterns that work across different users without requiring real-time adaptation, thus achieving scalability while maintaining cross-user effectiveness.
Data Source
AI summary
Methods and devices for masking emotional information of a user population, e.g., from a voice assistant device. For instance, the device may comprise one or more processor; a microphone; and a speaker. The one or more processor is configured to: listen, using the microphone, to multiple samples of speech by the user population; iteratively create emotionally obfuscating noises, using the multiple samples of speech, to determine a final emotionally obfuscating noise; recognize, using the microphone, a wake word of the voice assistant device; and generate the emotionally obfuscating noise over utterances by a user.