Network Anomaly Detection via Sketching and PCA
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Network administrators face challenges in diagnosing and defending against network anomalies in real-time, particularly in large-scale enterprise networks with high data rates and complex configurations, requiring effective monitoring and anomaly localization.
Innovation Solution
A network management station employs sketching and principal component analysis (PCA) to actively monitor administrator-configurable parameters, mathematically identify network anomalies, and automatically mitigate their causes by utilizing sketching for random projections and PCA for eigenvalue decomposition to detect variability and isolate anomalies.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If traditional network monitoring methods are used to monitor network parameters in real-time, then network administrators can detect anomalies, but the system cannot effectively localize the root cause of anomalies in large-scale networks with high data rates
Solution Approach 1:
The patent segments the network monitoring problem by dividing the network into multiple regions or zones, and applies independent monitoring and analysis to each segment. This allows the system to handle large-scale networks by breaking them down into manageable parts, improving anomaly localization precision without being overwhelmed by overall network complexity.
Solution Approach 2:
The patent introduces a new dimension of analysis by applying principal component analysis (PCA) to transform network parameter data into a different dimensional space. This dimensional transformation enables the system to identify patterns and anomalies that are not apparent in the original parameter space, improving detection precision while managing the complexity of high-data-rate networks.
2Reliability
If comprehensive network parameter monitoring is implemented to detect all types of anomalies, then network security is improved, but the computational complexity and processing time increase significantly
Solution Approach 1:
The patent implements preliminary action by pre-calculating baseline network behavior patterns and establishing normal parameter ranges before anomalies occur. This allows the monitoring system to quickly compare current parameters against pre-established baselines, enabling rapid anomaly detection without requiring complex real-time analysis of all network parameters, thus improving security while reducing detection time.
Solution Approach 2:
The patent replaces traditional mechanical/manual anomaly detection methods with automated statistical analysis and machine learning algorithms. By substituting manual monitoring with automated PCA-based analysis, the system can process comprehensive network parameters in real-time, improving both security coverage and detection speed without proportional increases in computational burden.
3Productivity
If real-time anomaly detection and mitigation is implemented, then network security and optimization are improved, but the system complexity and resource requirements increase
Solution Approach 1:
The patent implements self-service by enabling the network monitoring system to automatically detect anomalies, diagnose root causes, and initiate mitigation actions without human intervention. The system uses automated rule engines and decision algorithms to respond to detected anomalies, improving network optimization efficiency while managing complexity through automation rather than requiring complex manual intervention procedures.
Solution Approach 2:
The patent incorporates feedback mechanisms where the results of anomaly detection and mitigation actions are fed back into the monitoring system to continuously refine detection algorithms and update baseline parameters. This feedback loop improves productivity over time by making the system increasingly accurate and efficient, while the automated nature of the feedback process manages system complexity through iterative learning rather than requiring complex manual tuning.
Data Source
AI summary
Autonomous diagnosis and mitigation of network anomalies may include creating a plurality of sketch matrices wherein each sketch matrix corresponds to an individual hashing function and each row in each sketch matrix corresponds to an array of hashed parameters of interest from multiple network devices for a given period of time, the parameters of interest being configurable by an administrator. A principal components analysis (PCA) input matrix is created for each of the sketch matrices by computing an entropy value for each element in the sketch matrices, and principal components analysis (PCA) is performed on each of the PCA input matrices to heuristically detect a network anomaly in real time.