Speaker-Invariant Speech Feature Extraction via Adversarial Training
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Machine learning-based speech feature extraction techniques capture both content and speaker-related information, making it difficult to develop secure speech recognition systems that protect speaker identity.
Innovation Solution
Training neural feature extraction systems to produce speech features that are invariant to speaker information using data perturbations and loss function constraints, such as speaker conversion, pitch flattening, and vocal tract length normalization, along with adversarial loss functions to discourage learning of speaker information.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If machine learning-based speech feature extraction techniques are used to capture comprehensive speech information, then the accuracy for target domain tasks is improved, but speaker-related information is captured making the system insecure
Solution Approach 1:
The patent extracts and removes speaker-related information from the speech features while retaining content information. This is achieved through adversarial training where a speaker verification module attempts to identify speaker identity from the features, and the feature extraction module is trained to minimize this speaker identification capability while maintaining speech recognition accuracy.
Solution Approach 2:
The patent changes the training parameters and objectives of the feature extraction system. Instead of training to capture all speech information, the system uses a composite loss function that includes both speech recognition loss and speaker verification loss (with opposite gradients), effectively changing the optimization parameters to produce speaker-invariant features.
2Loss of information
If speaker-related information is captured in speech features, then more complete speech representation is achieved, but security of the speech recognition system deteriorates
Solution Approach 1:
The patent implements a feedback mechanism where the output features are fed back to a speaker verification module, and the verification results are used to adjust the feature extraction process. The adversarial loss function creates a feedback loop that continuously refines the features to be less informative about speaker identity while preserving speech content.
Solution Approach 2:
The patent applies preliminary anti-action by training the feature extraction system to counteract speaker identification from the outset. The adversarial training setup creates a preemptive defense where the system is explicitly trained to prevent speaker information leakage before any security breach can occur.
3Device complexity
If traditional speech feature extraction is used, then simpler neural architectures can be employed, but speaker identity can be extracted from the features reducing security
Solution Approach 1:
The patent introduces an intermediary adversarial training process between the feature extraction module and the downstream tasks. This intermediary mechanism (the speaker verification module used for adversarial training) mediates the feature extraction process to ensure speaker information is removed while maintaining the simplicity of downstream neural architectures.
Data Source
AI summary
A method, computer program product, and computing system for secure speech feature extraction. A speech signal comprising content information and speaker information is received and a component of the speaker information is altered to generate an augmented voice signal. In a first neural network, first embeddings of the received voice signal are generated. In a second neural network, second embeddings of the received voice signal having minimized speaker information based on the augmented voice signal are generated. The second neural network is trained to generate the second embeddings to be similar to the first embeddings generated by the first neural network.