Anomalous Segment Detection in Network Metrics via Graph Ranking
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current anomaly detection algorithms in data networks fail to identify significant anomalous segments within large datasets, as they consider anomalies in relation to the entire dataset rather than individual segments, leading to missed detections when anomalies offset each other upon aggregation.
Innovation Solution
A processing device identifies and ranks anomalous segments in a metrics dataset by generating a data graph with weighted edges representing similarity and relationship to the larger dataset, using algorithms like PageRank to highlight higher-ranked segments that deviate from the overall trend.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If anomaly detection is performed on the entire aggregated metrics dataset, then the analysis covers all data comprehensively, but significant anomalous segments may be missed when anomalies offset each other
Solution Approach 1:
The patent divides the metrics dataset into multiple segments based on common features (e.g., source, device type, geographic location) and performs anomaly detection on each segment separately. This segmentation allows the system to identify anomalous segments that would be obscured in aggregated analysis, where opposite anomalies (e.g., increased visits from one source, decreased visits from another) would cancel each other out. The segmentation principle directly resolves the contradiction by preserving segment-level anomaly information while maintaining comprehensive coverage.
2Measurement precision
If the metrics dataset is divided into multiple segments for analysis, then segment-level anomalies can be detected, but the complexity of the analysis process increases
Solution Approach 1:
The system segments the metrics dataset by common features (source, device type, geographic location, etc.) and performs anomaly detection on each segment independently. This segmentation approach enables precise identification of anomalous segments while managing complexity through systematic organization of the analysis process.
Solution Approach 2:
The patent applies different analysis approaches to different segments based on their local characteristics. Each segment is evaluated for anomaly using its own baseline and statistical properties, allowing the system to adapt to local patterns rather than applying a uniform analysis across all data. This local quality approach improves detection precision while keeping the overall system manageable.
3Reliability
If all segments are analyzed in detail to identify anomalies, then comprehensive anomaly detection is achieved, but the processing time and computational resources increase
Solution Approach 1:
The patent implements a two-stage approach: first, a quick screening phase that identifies potentially anomalous segments using simplified metrics; second, a detailed analysis phase that applies comprehensive anomaly detection only to segments flagged as potentially anomalous. This partial action principle allows the system to maintain high reliability by thoroughly analyzing suspicious segments while reducing overall processing time by avoiding exhaustive analysis of all segments.
Solution Approach 2:
The system dynamically adjusts analysis parameters such as anomaly thresholds, segmentation granularity, and detection sensitivity based on the characteristics of each segment and the overall data distribution. By changing parameters adaptively, the system achieves reliable anomaly detection while optimizing processing efficiency for different data conditions.
Data Source
AI summary
In some embodiments, a processor accesses a metrics dataset, which includes metrics whose values indicate data network activity. The metrics dataset has segments. Each segment is a respective subset of the data items having a common feature. The processor identifies anomalous segments in the metrics dataset. Each anomalous segment has a segment trend that is different from a trend associated with the larger metrics dataset. The processor generates a data graph that includes nodes, which represent anomalous segments, and edges connecting the nodes. The processor applies weights to the edges. Each weight indicates (i) a similarity between a pair of anomalous segments represented by the nodes connected by the weighted edge and (ii) a relationship between the anomalous segments and the metrics dataset. The processor ranks the anomalous segments based on the applied weights and selects one or more segments with sufficiently high ranks.