Dynamic Graph Anomaly Detection via Community Segmentation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current anomaly detection techniques in dynamic graphs are inefficient, particularly when dealing with large datasets like enterprise or social networks, as they often operate at the vertex level, requiring excessive processing and lacking a scalable framework for community-level or evolutionary path-level anomaly detection.
Innovation Solution
A dynamic anomaly detection framework that aggregates machine-generated event log data into graph snapshots, using a massively parallel processing database to identify anomalous communities and evolutionary paths through modified Jaccard coefficients and community tracking links, enabling flexible and adaptive anomaly detection at various levels of granularity.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If anomaly detection is performed at the vertex level in dynamic graphs, then detection granularity is fine, but processing complexity and time increase excessively
Solution Approach 1:
The patent segments the graph data into multiple snapshots at different time points and divides anomaly detection into two levels: community-level detection across snapshots and vertex-level detection within communities. This segmentation reduces the overall processing complexity by hierarchically filtering anomalies rather than examining every vertex in every snapshot simultaneously.
Solution Approach 2:
The patent introduces the time dimension by creating multiple graph snapshots and adding the community dimension as an intermediate layer between vertex-level details and graph-level overview. This multi-dimensional approach allows efficient filtering at community level before detailed vertex-level analysis, reducing processing time while maintaining detection precision.
2Measurement precision
If anomaly detection is performed at the vertex level, then detection detail is high, but scalability to large datasets is poor
Solution Approach 1:
The system segments large graph datasets into multiple time-based snapshots and further segments vertices into communities. This hierarchical segmentation enables parallel processing of different snapshots and communities, improving scalability while maintaining vertex-level detection capability where needed.
Solution Approach 2:
The patent applies partial action by performing comprehensive vertex-level anomaly detection only on suspicious vertices identified at the community level, rather than examining every vertex in detail. This selective approach maintains high detection detail for anomalies while improving overall scalability.
3Productivity
If community-level anomaly detection is implemented, then processing efficiency improves, but detection precision may be reduced
Solution Approach 1:
The patent implements a two-tiered segmentation strategy: first detecting anomalies at the community level across graph snapshots, then performing detailed vertex-level analysis on suspicious communities. This ensures both processing efficiency at the community level and high precision at the vertex level for confirmed anomalies.
Solution Approach 2:
Communities serve as an intermediary layer between graph-level overview and vertex-level detail. The community-level detection acts as a mediator that filters and prioritizes data before detailed vertex analysis, maintaining both efficiency and precision through this intermediate processing stage.
4Adaptability or versatility
If multiple levels of anomaly detection are supported, then detection flexibility increases, but system complexity increases
Solution Approach 1:
The system segments anomaly detection into distinct hierarchical levels (community-level and vertex-level) that can be independently configured and executed. This segmentation allows flexible combination of detection levels based on specific needs while managing complexity through modular, layered architecture.
Solution Approach 2:
The patent implements dynamic multi-level detection where the system can adaptively adjust which detection levels to apply based on data characteristics and user requirements. This dynamic approach provides flexibility while managing complexity by activating only necessary detection levels for each scenario.
Data Source
AI summary
Machine generated event log data which includes events occurring over a window of time is received where each event includes a first node, a second node, and a timestamp. The events are aggregated into a plurality of aggregated graph snapshots. Communities within the plurality of aggregated graph snapshots are identified and community tracking links are determined between communities in the plurality of aggregated graph snapshots. A community that has an anomalous evolution in the plurality of aggregated graph snapshots compared to the evolution of other communities is identified based at least in part on the community tracking links. The communities are displayed where the display includes the community tracking links and identifies the community that has the anomalous evolution.