Speaker De-Identification via Adversarial Pitch Perturbation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing speaker identification systems are vulnerable to adversarial attacks using gradient-based techniques that introduce noticeable noise and can be detected by humans, and are targeted, requiring knowledge of the system's characteristics.
Innovation Solution
Implementing adversarial perturbation methods that use small perturbations and a combination of gradient-based and non-gradient-based optimizations to shift voice samples across decision boundaries without significantly altering sound perception, allowing for speaker de-identification without targeting specific systems.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If gradient-based adversarial perturbation techniques are used to protect against speaker identification, then speaker de-identification is achieved, but noticeable noise is introduced into the audio samples that can be detected by human listeners
Solution Approach 1:
The patent changes the parameter of perturbation magnitude from gradient-based large steps to non-gradient-based small steps, thereby achieving de-identification while minimizing audible noise. The system iteratively applies small perturbations that accumulate to cross the decision boundary without introducing noticeable distortion to human listeners.
Solution Approach 2:
The patent applies partial action by using only the necessary minimum perturbation to cross the decision boundary. Instead of applying large gradient-based perturbations, the system uses multiple small non-gradient-based perturbations that just suffice to change the classification outcome, thereby minimizing the harmful noise while achieving the de-identification goal.
2Reliability
If gradient-based adversarial perturbation techniques are used, then speaker de-identification is achieved, but the audio samples are transformed significantly so that they sound distorted to human listeners
Solution Approach 1:
The patent changes the perturbation parameter from large gradient-based steps to small non-gradient-based steps, maintaining audio quality while achieving de-identification. The small incremental perturbations preserve the natural characteristics of the audio sample.
Solution Approach 2:
The patent applies beforehand cushioning by using multiple small perturbations instead of one large perturbation. This gradual approach cushions the transformation process, preventing sudden drastic changes that would distort the audio quality while still achieving the ultimate goal of crossing the decision boundary for de-identification.
3Reliability
If gradient-based adversarial perturbation techniques are used, then speaker de-identification is achieved, but the technique is limited to one type of identification system because gradients cannot be estimated without knowing the system characteristics
Solution Approach 1:
The patent applies universality by using non-gradient-based perturbation techniques that do not require knowledge of the specific identification system's internal characteristics. The system can estimate gradients through multiple queries to the target system, making the de-identification approach universally applicable to different speaker identification systems without needing system-specific information.
Solution Approach 2:
The patent uses an intermediary approach by introducing a gradient estimation mechanism that queries the target identification system multiple times to approximate the gradient direction. This intermediary process allows the system to adapt to different identification systems without requiring direct access to their internal parameters, thereby achieving versatility across multiple systems.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
Provided is a method for enabling speaker de-identification in public audio data by leveraging adversarial perturbation. The method may include receiving audio data associated with at least one voice sample. One or more of the voice sample(s) may be perturbed toward an edge of a decision boundary of at least one classifier model. One pitch of each voice sample may be perturbed to shift each voice sample across the decision boundary of the at least one classifier model to provide at least one de-identified voice sample. A media file with the at least one de-identified voice sample may be encoded. A system and computer program product are also disclosed.