Call Audio Deepfake Detection Using Background Change Cues
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing automatic speaker verification systems are vulnerable to deepfake attacks, which use machine-generated speech to bypass voice biometrics and gain unauthorized access to call center systems, necessitating improved means for detecting fraudulent audio-based deepfake technology and enhancing voice biometric systems with voice liveness detection.
Innovation Solution
Systems and methods for detecting deepfake audio in call conversations using speech-phrase repetition detection, background change detection, and passive voice-liveness detection, combined with an automated model update module that adapts to new presentation attacks based on human feedback, employing machine-learning architectures for temporal feature analysis and Natural Language Processing (NLP) to classify speech as human or machine-generated.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If traditional voice biometric systems are used for authentication, then authentication speed is improved, but vulnerability to deepfake attacks increases
Solution Approach 1:
The authentication process is segmented into multiple independent verification stages: initial voice biometric verification followed by deepfake detection analysis. The system divides the audio stream into segments for analyzing response delays, background noise changes, and speech patterns separately, then combines results for final authentication decision.
Solution Approach 2:
The patent introduces an intermediary deepfake detection system that sits between the traditional voice biometric authentication and the final access decision. This intermediary layer analyzes temporal features and background characteristics without blocking the fast authentication flow for legitimate users, while catching deepfake attacks.
2Reliability
If deepfake detection analysis is added to voice biometric systems, then resistance to deepfake attacks is improved, but system complexity increases
Solution Approach 1:
The system performs preliminary analysis of temporal features and background noise during the normal voice authentication process. By preparing and pre-processing these features as the audio stream arrives, rather than analyzing everything after authentication, the system adds detection capability without significantly increasing processing complexity or delay.
Solution Approach 2:
The deepfake detection module uses the same audio input and processing infrastructure as the traditional voice biometric system. Multiple verification functions (voice matching, response delay analysis, background noise detection) share common computational resources and data structures, reducing overall system complexity despite multiple security layers.
3Measurement precision
If response delay analysis is performed to detect deepfakes, then detection accuracy is improved, but processing time increases
Solution Approach 1:
The system continuously measures response delays between sequential speech segments throughout the entire conversation, rather than pausing to perform batch analysis. This continuous measurement approach maintains constant detection capability without adding discrete processing steps that would increase overall authentication time.
Solution Approach 2:
The temporal feature analysis uses the naturally occurring speech segments and their inherent timing information from the conversation itself. The system extracts response delay metrics from the existing audio stream without requiring additional test inputs or extended interaction time, allowing accurate detection within normal conversation flow.
Data Source
AI summary
Disclosed are systems and methods including software processes executed by a server that detect audio-based synthetic speech (“deepfakes”) in a call conversation. The server applies an NLP engine to transcribe call audio and analyze the text for anomalous patterns to detect synthetic speech. Additionally or alternatively, the server executes a voice “liveness” detection system for detecting machine speech, such as synthetic speech or replayed speech. The system performs phrase repetition detection, background change detection, and passive voice liveness detection in call audio signals to detect liveness of a speech utterance. An automated model update module allows the liveness detection model to adapt to new types of presentation attacks, based on the human provided feedback.