Synthetic Speech Detection Using Distilled Streaming Student Models
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing synthesis speech detection models, such as XLS-R, are too large to be deployed on computing resource-constrained devices like smartphones and require significant time for speech data detection, hindering real-time detection of deepfake voices.
Innovation Solution
A lightweight synthesis speech detection model is trained using knowledge distillation from a larger teacher model, with fewer layers and configured for real-time operation on devices with limited resources, utilizing a streaming method with shorter speech data segments and robust to local changes through a sliding window approach.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If a large model such as XLS-R is used for synthesis speech detection, then detection accuracy is improved, but the model size becomes too large to be deployed on edge devices with limited computing resources
Solution Approach 1:
The patent segments the large XLS-R teacher model into multiple smaller student models with different complexity levels. Each student model is trained through knowledge distillation to capture specific aspects of synthesis speech detection, enabling deployment on devices with varying computing resources while maintaining detection accuracy through ensemble usage.
Solution Approach 2:
The patent creates multiple student model copies that are initialized from the teacher model's parameters. These student models are lighter versions that replicate the teacher model's detection capabilities through knowledge distillation, allowing deployment on resource-constrained devices while preserving the original model's detection performance.
2Measurement precision
If existing models process speech data for accurate detection, then detection accuracy is improved, but the processing time becomes too long to achieve real-time detection
Solution Approach 1:
The patent segments speech data into overlapping short-time frames and processes them through multiple student models in parallel. This segmentation enables real-time processing by breaking down long speech sequences into manageable chunks that can be detected simultaneously, reducing overall processing time while maintaining accuracy through ensemble aggregation.
Solution Approach 2:
The patent uses multiple student models with varying levels of processing depth. Some student models perform lighter processing for quick detection, while others provide more thorough analysis. This partial action approach enables real-time detection by using the lighter models for time-critical cases, reducing average processing time while maintaining accuracy when needed.
3Measurement precision
If speech data is processed in longer segments for accurate detection, then detection accuracy is improved, but the real-time response capability deteriorates
Solution Approach 1:
The patent segments both the speech data and the model architecture. Speech data is divided into overlapping short-time frames that can be processed quickly, while the model is segmented into multiple student models of varying complexity. This dual segmentation enables real-time response by processing short segments through appropriate student models, with results aggregated to maintain detection accuracy.
Solution Approach 2:
The patent dynamically selects which student model to use based on the specific detection task and available resources. For time-critical real-time detection, lighter student models are selected for faster processing. For cases where accuracy is prioritized and resources permit, heavier student models are used. This dynamic adaptation balances real-time response speed with detection accuracy.
Data Source
AI summary
The present disclosure may include a method of training a synthesis speech detection model performed by one or more processors including generating a student model initialized by using a teacher model and at least part of one or more teacher model layers included in the teacher model, dividing learning speech data into a piece or pieces of learning speech section data, detecting a voice activity of first learning speech section data among the piece or pieces of learning speech section data, inputting the first learning speech data into the teacher model and the student model, generating first teacher data from the teacher model and first student data from the student model, and training the student model based on the first teacher data and the first student data.