Anomaly Detection Using Relationship Graphs and Threshold Segmentation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Anomaly detection in systems like online social networks and computer networks faces challenges in setting thresholds for alert generation, leading to either missed anomalies (false negatives) or excessive false positives, due to the difficulty in distinguishing between anomalous behavior of individual entities and groups.
Innovation Solution
A system that constructs a graph representing entities and their relationships, computes single-entity and subgraph anomaly scores, and generates alerts based on predefined thresholds, with a higher threshold for individual entities and a lower threshold for groups, to reduce false positives while conserving computing resources.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Device complexity
If a single threshold is used for anomaly detection in both individual entities and groups, then the system is simple to implement, but it leads to either missed anomalies (false negatives) or excessive false positives
Solution Approach 1:
The patent segments the anomaly detection process into two distinct pathways: one for individual entities (single-entity anomaly scores) and one for groups of entities (subgraph anomaly scores). Each pathway has its own optimized threshold, allowing the system to achieve high detection accuracy without the complexity of a single universal threshold.
Solution Approach 2:
The patent applies different threshold criteria to different levels of the detection hierarchy. Individual entities use one threshold criterion while groups use another, allowing each level to be optimized for its specific detection needs rather than applying a one-size-fits-all approach.
2Reliability
If the system computes anomaly scores for all possible subgraphs, then it can detect all group anomalies, but it consumes excessive computing resources
Solution Approach 1:
The patent performs preliminary filtering by first computing single-entity anomaly scores and identifying entities that exceed a threshold. Only subgraphs containing these pre-identified anomalous entities are then evaluated for subgraph anomaly scores, significantly reducing the number of subgraphs that need full evaluation while maintaining detection completeness.
Solution Approach 2:
The patent extracts and focuses computational effort only on relevant subgraphs—those containing entities that have already been identified as potentially anomalous. This extraction approach eliminates the need to compute scores for all possible subgraphs, thereby conserving computing resources while maintaining reliability.
3Measurement precision
If the system uses a lower threshold for group anomalies, then it reduces false positives, but it may miss subtle individual anomalies
Solution Approach 1:
The patent segments detection into two independent but complementary processes: individual entity detection with one threshold criterion, and group detection with another threshold criterion. This segmentation ensures that neither false positives nor false negatives dominate, as each level operates with optimized parameters for its specific detection goals.
Solution Approach 2:
The patent changes the threshold parameter differently for individual entities versus groups. By adjusting the threshold criterion based on the detection level (individual vs. group), the system optimizes performance for each level while maintaining overall detection reliability.
Data Source
AI summary
An anomaly-detection system that gathers information relating to the relationships between entities and represents these relationships in a graph that interconnects each pair of related entities. The graph may represent a computer network, in which each node corresponds to a respective device in the network and each edge between two nodes indicates that the devices represented by the nodes exchanged communication with one another in the past. the system monitors each of the entities in the graph, by continually computing a single-entity anomaly score (SEAS) for the entity. If the SEAS exceeds a first threshold the system generates an alert. Otherwise, the system checks whether the SEAS exceeds a second, lower threshold. If so, the system computes a subgraph anomaly score (SAS) for the entity's subgraph. If the SAS exceeds a SAS threshold, an alert is generated. By computing the SAS in this manner resources are conserved.