Hierarchical Clustering for Low False Positive Fraud Detection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Behavior modeling-based anomaly detection systems often generate excessive alerts and false positives due to the complexity of analyzing large volumes of data from enterprise networks, making it difficult to identify genuine anomalies in a timely and effective manner.
Innovation Solution
Implementing a hierarchical clustering approach to analyze temporal behavior data at multiple scales, aggregating anomaly scores across different levels to reduce false positives and focus on significant deviations from normal behavior.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If traditional anomaly detection models analyze temporal behavior data at a single level, then they can detect individual anomalous behaviors, but they generate excessive false positives and require too many alerts to be investigated
Solution Approach 1:
The patent segments the anomaly detection process into multiple hierarchical levels (individual user level, department level, enterprise level). Each level analyzes behavior patterns at its specific scope, allowing the system to distinguish between local anomalies and genuine cross-boundary threats. This segmentation reduces false positives by filtering out anomalies that are normal at higher levels while maintaining detection sensitivity at each level.
Solution Approach 2:
The patent adds a hierarchical dimension to the anomaly detection analysis by examining behavior data at multiple organizational levels simultaneously. Instead of a single flat analysis layer, the system creates a multi-dimensional view where anomalies are evaluated in context of their hierarchical position, enabling better discrimination between true threats and false alarms through aggregate score computation across levels.
2Reliability
If the system processes 100 Billions of events per year from enterprise networks, then it can detect comprehensive anomalies, but the data volume cannot fit into one single machine for traditional in-memory analytics
Solution Approach 1:
The patent segments the massive dataset processing across multiple machines in a distributed computing environment. Each machine processes a partition of the hierarchical data (e.g., specific departments or user groups), computing local anomaly scores independently. This segmentation enables the system to handle 100 Billions of events per year by distributing the computational load while maintaining the ability to aggregate results for enterprise-wide anomaly detection.
Solution Approach 2:
The patent transitions from single-machine in-memory analytics to a distributed multi-dimensional processing architecture. By organizing computation across multiple machines with hierarchical data partitioning, the system achieves both comprehensive detection coverage and scalable infrastructure that can accommodate massive event volumes without requiring a single large computational system.
3Measurement precision
If behavioral modeling analyzes detailed user behavior patterns, then it can identify specific anomalous actions, but it generates too many alerts that cannot be investigated in a timely and effective manner
Solution Approach 1:
The patent segments anomaly detection into hierarchical levels where detailed behavioral analysis occurs at lower levels (individual users) but final alert generation is filtered through higher levels (departments, enterprise). This segmentation allows precise behavioral modeling to identify potential anomalies while the hierarchical aggregation filters out false positives, reducing the number of alerts requiring investigation and enabling timely response to genuine threats.
Solution Approach 2:
The patent applies different analysis depths at different hierarchical levels. At the user level, detailed behavioral patterns are analyzed with high precision to detect potential anomalies. At higher organizational levels, the analysis focuses on aggregate patterns and cross-boundary deviations. This local quality approach ensures detailed detection where needed while reducing overall alert volume by filtering out anomalies that don't persist across hierarchical levels.
Data Source
AI summary
Techniques to detect fraud through behavioral analysis with low false positives are disclosed. In various embodiments, resource access data indicating for each resource in a set of resources respective usage data for each of one or more users of the resource is received. Hierarchical clustering analysis is performed to determine at each of two or more hierarchical levels a set of one or more clusters of users, resources, or both. A level-specific anomaly score is computed at each of said two or more hierarchical levels. The level-specific anomaly scores are aggregated across said two or more hierarchical levels to determine an aggregate anomaly score. The aggregate anomaly score to determine whether an anomaly has been detected.