Network Anomaly Detection via Correlation Matrix Shifts
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing anomaly detection methods in computer networks face challenges such as high-dimensional complexity, dynamic behavior changes, and the difficulty in differentiating between noise and relevant anomalies, especially in environments like Low-Power and Lossy Networks (LLNs) where environmental changes and node constraints complicate routing and quality of service.
Innovation Solution
The approach involves analyzing changes in correlation matrices using spectral analytics, locality-sensitive hashing, and persistent homology-based methods to detect anomalies in multi-dimensional time series data, allowing for the identification of anomalies by examining shifts in correlation structures rather than individual metric changes, thereby avoiding the Curse of Dimensionality.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If anomaly detection analyzes individual metric changes in high-dimensional spaces, then detection coverage increases, but the Curse of Dimensionality makes it difficult to differentiate between noise and relevant anomalies
Solution Approach 1:
The patent extracts the essential characteristic of system behavior by analyzing correlation matrices rather than individual metrics. By focusing on the relationships between metrics (correlation structures) rather than each metric independently, the method extracts the meaningful signal from high-dimensional data while filtering out noise, thus resolving the Curse of Dimensionality problem
Solution Approach 2:
The patent transforms the analysis from the original high-dimensional metric space to a new dimension of correlation relationships. By computing correlation matrices and analyzing their spectral properties (eigenvalues, eigenvectors), the method projects high-dimensional data into a lower-dimensional spectral space where anomaly detection becomes feasible and effective
2Adaptability or versatility
If anomaly detection models behavior in complex network environments with dynamic changes, then adaptability improves, but differentiating between environmental noise and actual anomalies becomes more difficult
Solution Approach 1:
The patent performs preliminary action by establishing baseline correlation structures during normal operating conditions. The system learns and stores the typical correlation patterns among metrics before anomalies occur, creating a reference model that enables later comparison and detection of deviations without confusion from environmental noise
Solution Approach 2:
The patent implements feedback by continuously monitoring correlation matrices and comparing them against baseline patterns. The system provides feedback on detected anomalies and adapts to genuine behavior changes while filtering out noise, creating a self-adjusting detection mechanism that improves over time
Data Source
AI summary
In one embodiment, a device in a network receives a first plurality of measurements for network metrics captured during a first time period. The device determines a first set of correlations between the network metrics using the first plurality of measurements captured during the first time period. The device receives a second plurality of measurements for the network metrics captured during a second time period. The device determines a second set of correlations between the network metrics using the second plurality of measurements captured during the second time period. The device identifies a difference between the first and second sets of correlations between the network metrics as a network anomaly.