Call Audio Deepfake Detection Using Voice Liveness Cues
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing automatic speaker verification systems are vulnerable to deepfake attacks, which use machine-generated speech to bypass voice biometrics, posing a threat to secure systems and information access.
Innovation Solution
Implement a system that combines speech-phrase repetition detection, background change detection, and passive voice-liveness detection, using machine-learning architectures to identify unnatural delays, NLP processes, and human feedback for adaptive model updates to detect deepfake audio in call conversations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If traditional automatic speaker verification systems are used, then authentication speed and ease of operation are improved, but vulnerability to deepfake attacks increases
Solution Approach 1:
The verification process is divided into multiple independent analysis components: temporal feature analysis (response delays), speech pattern analysis (phrase repetition), background analysis (change detection), and acoustic feature analysis. Each component processes different aspects of the audio signal independently, then their results are combined to make the final authentication decision, preventing deepfake attacks from exploiting a single verification point.
Solution Approach 2:
A machine learning architecture acts as an intermediary between the audio input and the authentication decision. This intermediary analyzes multiple features including temporal delays, speech patterns, background changes, and acoustic characteristics, then integrates these analyses to determine authenticity, thereby blocking deepfake attacks without compromising user experience.
2Measurement precision
If multiple detection methods are combined to improve deepfake detection accuracy, then detection precision is improved, but device complexity increases
Solution Approach 1:
Multiple detection methods are merged into a single integrated machine learning architecture that processes temporal features, speech patterns, background changes, and acoustic characteristics simultaneously. The architecture combines these diverse detection approaches into one unified system, achieving high detection accuracy while managing complexity through integrated processing rather than separate independent systems.
Solution Approach 2:
The machine learning architecture performs multiple functions within a single system: it analyzes temporal response delays, detects speech pattern anomalies, identifies background changes, and evaluates acoustic features. This multi-functional approach achieves comprehensive deepfake detection without requiring separate specialized systems for each detection method.
3Speed
If real-time analysis of multiple features is performed, then detection speed is improved, but computational energy consumption increases
Solution Approach 1:
The system performs preliminary analysis of audio features as they are being received, preparing temporal, spectral, and contextual features in advance before the final authentication decision is required. This preliminary processing allows the machine learning model to make rapid decisions without intensive real-time computation during the critical authentication moment, reducing peak energy consumption while maintaining detection speed.
Data Source
AI summary
Disclosed are systems and methods including software processes executed by a server that detect audio-based synthetic speech (“deepfakes”) in a call conversation. The server applies an NLP engine to transcribe call audio and analyze the text for anomalous patterns to detect synthetic speech. Additionally or alternatively, the server executes a voice “liveness” detection system for detecting machine speech, such as synthetic speech or replayed speech. The system performs phrase repetition detection, background change detection, and passive voice liveness detection in call audio signals to detect liveness of a speech utterance. An automated model update module allows the liveness detection model to adapt to new types of presentation attacks, based on the human provided feedback.