Synthetic Speech Detection Using Clustered Embedding Boundaries
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Sophisticated deep learning models for voice generation produce highly realistic synthetic speech, making it difficult for speaker recognition systems to distinguish between real and spoofed voices, leading to potential malicious uses and the need for constant model retraining due to rapid advancements in spoofing technologies.
Innovation Solution
A computer-based system using multiple encoders to extract different types of embeddings, clustering these embeddings into archetypal groups, and defining decision boundaries for each cluster to identify anomalies, allowing for adaptable and robust detection of synthetic speech by classifying deviations from natural speech patterns.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If sophisticated deep learning models are used for voice generation, then the realism of synthetic speech is improved, but the ability of speaker recognition systems to distinguish real from spoofed voice deteriorates
Solution Approach 1:
The patent segments the speaker recognition task by creating multiple specialized encoder models, each trained to detect specific types of spoofing attacks. Instead of using a single general-purpose model, the system divides the detection task into multiple specialized components that can identify different artifacts introduced by various synthesis methods, thereby maintaining recognition accuracy despite improving synthetic speech realism.
Solution Approach 2:
The patent introduces an intermediary verification layer between the speaker recognition system and the synthetic speech input. This intermediary consists of the ensemble of encoder models that analyze acoustic artifacts and generate confidence scores, acting as a mediator that prevents spoofed speech from being misclassified as genuine without compromising the system's ability to recognize legitimate speakers.
2Measurement precision
If speaker recognition systems are made more sophisticated to detect spoofing, then the detection accuracy is improved, but the system complexity increases
Solution Approach 1:
The system segments the detection task into multiple independent encoder models, each specializing in detecting specific spoofing artifacts. This segmentation allows each model to be relatively simple and focused, while the ensemble collectively achieves high detection accuracy. The modular structure also simplifies training and deployment compared to a single complex model.
Solution Approach 2:
The patent changes the parameters of the detection system by using multiple encoders with different training configurations and thresholds. Each encoder can be tuned to detect specific types of synthetic speech, and their outputs are combined through voting or averaging. This parameter-based approach allows flexible adjustment of detection sensitivity without fundamentally increasing system complexity.
3Reliability
If multiple encoders are used to extract different types of embeddings, then the detection robustness is improved, but the computational cost increases
Solution Approach 1:
The patent segments the feature extraction process into multiple specialized encoders, each focusing on specific acoustic characteristics. This segmentation allows for efficient computation by dedicating each encoder to a specific type of analysis, rather than using a single large model that processes all features. The computational load is distributed across multiple smaller, specialized models.
Solution Approach 2:
The system employs partial action by using multiple encoders that each perform a subset of the full analysis. Not all encoders need to process every input, and the system can selectively apply different encoders based on the specific detection needs. This partial application reduces overall computational cost while maintaining robustness through the collective output of the encoder ensemble.
Data Source
AI summary
System and method for detecting synthetic speech may include, using a processor: in a training phase: generating an embedding for each of a plurality of bona fide speech samples by providing each of the plurality of bona fide speech samples to one or more encoders; clustering the embeddings into a plurality of clusters; and determining a decision boundary for each of the plurality of clusters; during runtime: generating an embedding for an examined speech sample by providing the examined speech sample to the one or more encoders; assigning the examined speech sample to selected cluster of the plurality of clusters; and determining that the examined speech sample includes synthetic speech based on a location of the embedding of the examined speech sample with relation to the decision boundary in the selected cluster.