Network Anomaly Detection via Dynamic Threshold Adjustment
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current network anomaly detection systems, such as IDS and data harvesters, face challenges in promptly identifying potential attacks or anomalies due to fixed threshold settings, resource-intensive detailed analysis, and inability to effectively utilize data harvester information in real-time, leading to false alarms and delayed responses.
Innovation Solution
A statistical approach that dynamically adjusts threshold alert levels based on observed network activity over a pre-set time period, using a statistics-based profile to estimate the probability of anomalous behavior and alert users only when thresholds are breached, incorporating Bayes' Rule for probability calculations and percentiling to prioritize alerts.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If fixed threshold levels are used for anomaly detection, then the system is simple to operate, but false alarms increase during high-usage periods and potential attacks are missed during low-usage periods
Solution Approach 1:
The patent implements dynamic threshold adjustment by continuously learning normal network behavior patterns and adapting thresholds based on observed traffic characteristics. The system transitions from static, manually-configured thresholds to dynamic, automatically-adjusted thresholds that respond to changing network conditions, thereby maintaining high detection accuracy across varying usage periods without increasing operational complexity
Solution Approach 2:
The system changes the parameter of threshold values from fixed to variable, allowing thresholds to adapt based on learned network behavior patterns. By modifying threshold parameters dynamically according to observed traffic statistics and anomaly scores, the system resolves the contradiction between operational simplicity and detection reliability
2Measurement precision
If detailed analysis of data contents is performed, then detection precision is improved, but resource consumption increases significantly
Solution Approach 1:
The patent segments the detection process into multiple stages: initial lightweight monitoring of network traffic, intermediate analysis of suspicious patterns, and detailed examination only of confirmed anomalies. This hierarchical segmentation allows the system to maintain high precision for critical detections while minimizing resource consumption by applying detailed analysis only when necessary
Solution Approach 2:
The system applies partial analysis to most traffic (monitoring key parameters without deep inspection) and reserves excessive/detailed analysis only for suspicious packets. This selective approach achieves high detection precision for attacks while consuming minimal processing resources for the majority of normal traffic
3Area of stationary object
If data harvester collects extensive network information, then the breadth of monitoring is improved, but the ability to provide timely anomaly alerts is reduced due to lack of real-time evaluation
Solution Approach 1:
The patent implements real-time feedback loops where the data harvester continuously monitors network traffic, the system immediately evaluates collected data against learned patterns, and alerts are generated promptly when anomalies are detected. This closed-loop feedback mechanism enables the system to utilize extensive network information while maintaining rapid detection and response times
Solution Approach 2:
The system performs preliminary analysis of network data as it is collected, evaluating potential anomalies in real-time rather than waiting for batch processing. By conducting preliminary evaluation immediately upon data collection, the system maintains both broad monitoring coverage and timely anomaly detection
Data Source
Figure 1
Figure 2~3
Figure 4~6
AI summary
An anomaly detection method and system determine network status by monitoring network activity. A statistics based profile for said network over a period is generated to analyze potentially anomalous network activity to determine if said network activity is anomalous by comparing current activity against the profile. Using the profile as a reference, the anomaly detection system and process estimate and prioritize potentially anomalous network activity based on the probability that the behavior is anomalous. The level of severity that the anomaly detection process uses to determine if an alarm is needed is based on comparing user-adjustable thresholds to the current probability. If the threshold has been breached, the user is alerted, subject to other quality checks. After a reporting cycle concludes, the anomaly detection system and process recompiles the statistics based profile to take into account the information observed in the previous reporting cycle.