Dual-Track Fake Voice Detection Using Mel Difference Attention
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing fake voice detection methods struggle with accuracy, robustness, and generalization in the face of evolving deepfake technologies, particularly in complex voice situations, necessitating improved methods for detecting synthetic voices.
Innovation Solution
A dual-track differential modeling approach is employed, involving pre-training a voice conversion model to convert single-track audio to stereo, extracting and enhancing Mel spectrogram features, and using dual-branch feature extractors and attention pooling to fuse spatial attention maps for accurate detection.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If robust feature extraction-based fake voice detection is used, then detection capability is improved, but accuracy and robustness deteriorate in the face of evolving deepfake technologies
Solution Approach 1:
The patent segments the voice detection process into dual-track parallel processing (original track and converted track), where each track independently extracts features and contributes to the final detection decision. This segmentation allows the system to capture different characteristics of fake voices through multiple processing paths, improving both accuracy and robustness against evolving deepfake techniques.
Solution Approach 2:
The patent introduces a new dimension by converting single-track audio to dual-track stereo representation. This dimensional transformation enables the system to analyze voice artifacts from multiple perspectives simultaneously, enhancing detection precision without sacrificing reliability across different deepfake scenarios.
2Measurement precision
If complex model designs are used to improve fake voice detection, then detection performance is improved, but generalization to new deepfake scenarios deteriorates
Solution Approach 1:
The dual-track detection model serves multiple functions simultaneously: it processes both original and converted audio tracks, extracts diverse features, and adapts to various deepfake scenarios through a unified architecture. This universal design enables the system to maintain high detection performance while generalizing effectively to new and evolving deepfake techniques without requiring scenario-specific modifications.
3Measurement precision
If fine-grained feature analysis is performed, then detection accuracy is improved, but computational complexity increases causing fine-grained differences to disappear in deep neural networks
Solution Approach 1:
The patent extracts and emphasizes fine-grained differential features by computing absolute differences between original and converted track features. This extraction process isolates subtle artifacts introduced by deepfake processing, making them prominent enough to be detected accurately without being lost in the computational complexity of the deep neural network.
Data Source
AI summary
A fake voice detection method based on dual-track differential modeling includes: converting, by a pre-trained single/dual-track voice conversion model, a single-track audio into a stereo; extracting left and right-track Mel spectrogram features; performing texture enhancement on absolute values of differences between an original single-track Mel spectrogram feature and each of the left and right-track Mel spectrogram features; inputting texture enhancement results into left and right dual-branch feature extractors respectively to acquire a final attention map through feature extraction, processing and fusion; and inputting the final attention map into an attention pooling layer and a final binary classification layer to acquire a fake voice detection result. The fake voice detection method achieves fake voice detection by means of dual-track differential modeling, fine-grained texture enhancement, and multi-head attention feature fusion, improving the accuracy, transferability, and generalization of fake voice detection.