Streaming Graph Anomaly Detection via Epoch Embedding
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Detecting sudden appearances or disappearances of dense subgraphs in time-evolving graphs in real-time is computationally challenging due to the continuous arrival of new edges and vertices, requiring efficient methods to process changes and identify anomalies.
Innovation Solution
The method involves extracting graph summaries by embedding each epoch graph into a multi-dimensional space, generating K-dimensional summary vectors, and applying anomaly detection algorithms like robust random cut forest to identify outliers, allowing for real-time detection of dense subgraph anomalies.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If real-time anomaly detection is performed on streaming graphs with continuous data arrival, then detection speed and timeliness are improved, but computational complexity and processing difficulty increase
Solution Approach 1:
The patent segments the continuous streaming graph data into discrete epoch graphs based on time windows or data volume thresholds. Each epoch graph represents a snapshot of the streaming graph at a specific time point, transforming the continuous detection problem into discrete batch processing. This segmentation reduces computational complexity by limiting the data scope for each anomaly detection operation while maintaining real-time detection capability through sequential processing of epochs.
Solution Approach 2:
The patent extracts graph summaries from epoch graphs by computing condensed representations such as node degree distributions, edge density metrics, and subgraph frequency statistics. These extracted features capture the essential structural characteristics of the streaming graph without requiring processing of the entire graph data. The extraction process transforms complex graph data into manageable feature vectors that can be efficiently analyzed for anomalies.
2Measurement precision
If dense subgraph anomalies are detected in real-time streaming data, then measurement precision is improved, but processing time increases
Solution Approach 1:
The patent performs preliminary actions by pre-computing graph summaries and structural features for each epoch graph before anomaly detection. Graph embeddings are generated in advance, and baseline statistical metrics are calculated for normal graph structures. When new data arrives, the system compares against these pre-computed references, significantly reducing the time required for actual anomaly detection while maintaining high precision through comprehensive feature analysis.
Solution Approach 2:
The patent transforms graph data into multi-dimensional feature spaces through graph embedding techniques. Instead of analyzing raw graph structures directly, the system projects graph nodes and subgraphs into K-dimensional vectors that capture structural relationships. This dimensionality transformation enables efficient anomaly detection by converting complex graph comparison problems into simpler distance or density calculations in vector space, reducing processing time while preserving detection precision.
Data Source
AI summary
Techniques for detecting anomalies in streaming graph data are described. For example, an embedding technique of generating a multi-dimensional vector of summations of each weighted edge found in both a random source bounding proper subset and a random destination bounding proper subset associated with a dimension of the epoch graph is detailed. Anomaly detection is performed on the generated multi-dimensional vectors.