Deepfake Music Detection Using Voice Separation and Vocoder Cues
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current deepfake detection methodologies for singing voices in the music industry are inadequate for effective copyright protection, as the sophistication of deepfake technology poses significant risks of copyright infringement.
Innovation Solution
A deepfake music detection apparatus and method utilizing a variational auto encoder-generative adversarial network model and a neural vocoder detection model to extract sound features and determine deepfake probabilities through voice separation and neural vocoder processing, employing spectral envelope, temporal dynamics, pitch, and vocal tract features, with a multilayer perceptron for final determination.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of manufacture
If deepfake technology is used to generate singing voices, then voice generation capability is improved, but copyright protection reliability deteriorates
Solution Approach 1:
The patent applies preliminary action by extracting and storing voiceprints from original singing voices before deepfake generation occurs. The voiceprint extraction unit captures unique vocal characteristics in advance, creating a reference database that enables later verification of whether generated voices match authorized performers, thus preventing copyright infringement before it can cause harm.
Solution Approach 2:
The patent replaces traditional mechanical voice verification methods with neural network-based biometric analysis. The voiceprint extraction and comparison processes use deep learning models to analyze spectral features, temporal dynamics, and vocal tract characteristics, substituting physical acoustic analysis with intelligent algorithmic verification to achieve more reliable copyright protection.
2Measurement precision
If voiceprint extraction features are increased in number, then detection accuracy is improved, but processing time increases
Solution Approach 1:
The patent segments the voiceprint extraction process into four distinct feature groups: spectral envelope features (MFCCs, spectral centroid), temporal dynamics features (delta, spectral flux), pitch and harmonic frequency features (F0, HNR), and vocal tract features (formant frequencies). This segmentation allows parallel processing of different feature types, maintaining comprehensive analysis while enabling optimized computation for each feature category.
Solution Approach 2:
The patent dynamically adjusts the number of MFCC coefficients extracted based on the specific characteristics of the input voice signal. For voices with rich spectral information, more coefficients are extracted to capture subtle characteristics, while for simpler voices, fewer coefficients suffice. This adaptive parameter adjustment maintains high detection accuracy while minimizing unnecessary processing overhead.
Data Source
AI summary
A deepfake music detection apparatus according to the present disclosure includes an input unit which receives audio data, a feature extracting unit which extracts sound features from the audio data, a voice separation detecting unit which acquires a voice separation probability which is a probability of performing voice separation processing on voices included in the audio data, from the sound feature, a neural vocoder detecting unit which acquires a neural vocoder probability which is a probability of generating voices included in the audio data through a neural vocoder, and a deepfake determining unit which determines whether the audio data is deepfake using the voice separation probability and the neural vocoder probability.