Voice-to-Text Call Clustering for Privacy-Preserving Fraud Detection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Users are vulnerable to phishing attacks through fraudulent calls, which can compromise their sensitive information, and existing call verification methods often rely on crowdsourced data, posing privacy risks and lacking real-time analysis.
Innovation Solution
A system that integrates a local security agent with a cloud-based service to analyze call context, using machine learning to detect fraudulent calls by converting audio to text, clustering similar calls, and applying shared fraud profiles for real-time classification, with local and cloud-based models ensuring privacy through anonymization.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If crowdsourced data is used for call verification, then detection coverage is improved, but user privacy is compromised
Solution Approach 1:
The system segments the fraud detection process into local processing (audio-to-text conversion, feature extraction) and cloud processing (model training, pattern matching). Only anonymized features and metadata are transmitted to the cloud, while raw audio data remains locally processed. This segmentation allows crowd-sourced collaboration without exposing user privacy.
Solution Approach 2:
The patent introduces an intermediary anonymization layer that transforms sensitive audio data into feature vectors and metadata before cloud transmission. This intermediary process extracts essential fraud-detection features (tone, pitch, keywords) while removing personally identifiable information, enabling crowd-sourced analysis without direct exposure of user data.
2Measurement precision
If real-time audio analysis is performed, then fraud detection accuracy is improved, but processing time and energy consumption increase
Solution Approach 1:
The system performs preliminary actions by pre-converting audio to text and extracting key features (tone, pitch, keywords) during the call itself, before final fraud determination. This preliminary processing prepares data for rapid cloud-based pattern matching, reducing the time required for real-time analysis while maintaining high accuracy.
Solution Approach 2:
The patent applies partial action by focusing analysis on specific fraud-relevant features (certain keywords, tone patterns, pitch variations) rather than processing entire audio streams in detail. This selective analysis achieves sufficient detection accuracy while significantly reducing processing time and energy consumption compared to comprehensive audio analysis.
3Loss of information
If local security agent processes all audio data, then privacy is protected, but device computational load increases
Solution Approach 1:
The local security agent extracts only essential features (audio-to-text conversion, tone, pitch, keywords) and metadata from raw audio data, then transmits these extracted features to the cloud for heavy processing. This extraction approach protects privacy by keeping raw audio locally while offloading computationally intensive fraud detection algorithms to cloud infrastructure.
Solution Approach 2:
Instead of transmitting or processing raw audio data locally, the system creates and transmits copies of processed features (text transcripts, feature vectors, metadata). These feature copies contain sufficient information for fraud detection while being computationally lightweight and privacy-preserving, reducing device computational load significantly.
Data Source
AI summary
A computer-implemented system and method for preventing fraudulent call activity includes detecting a plurality of voice calls from different phone numbers; converting audio content of the calls to text; clustering the calls based on similarity of the converted text and voice characteristics; assigning a shared fraud profile to the clustered calls; and using the shared fraud profile to classify future calls.