Web Server Anomaly Detection Using Short- and Long-Term Traffic Trends
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing network security systems struggle to accurately detect anomalies in network traffic due to the bursty nature of data, leading to a high number of false positives and unnecessary resource consumption in remediation efforts.
Innovation Solution
The system employs exponential smoothing and trend analysis to identify anomalies by comparing short-term and long-term trends in network traffic, using bucketization and exponential smoothing algorithms to reduce the impact of noise and improve detection accuracy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If anomaly detection is performed on high-resolution time-series network traffic data, then detection precision is improved, but false positives increase due to noise in short bursts
Solution Approach 1:
The patent segments network traffic analysis into two distinct time-scale components: short-term trends (capturing immediate traffic patterns) and long-term trends (capturing baseline behavior). By separating these time scales, the system can compare them to identify anomalies without being misled by short-term noise or bursts that would otherwise be indistinguishable from genuine anomalies at high resolution.
Solution Approach 2:
The patent transforms the single-dimensional time-series anomaly detection problem into a multi-dimensional analysis by introducing the time-scale dimension. Instead of analyzing traffic at a single resolution, the system analyzes traffic across multiple time scales (short-term and long-term), allowing anomalies to be identified as deviations from the long-term trend that are confirmed by short-term pattern analysis, thereby filtering out false positives.
2Reliability
If comprehensive anomaly detection is implemented, then network security is improved, but computing resources are consumed by processing high-resolution time-series data
Solution Approach 1:
The patent segments the computationally intensive anomaly detection task into two separate trend analyses: short-term trend detection and long-term trend detection. Each segment processes data at appropriate resolutions and time scales, avoiding the need to process entire high-resolution time-series datasets comprehensively, thereby reducing overall computing resource consumption while maintaining security effectiveness.
Solution Approach 2:
The patent extracts only the essential trend components (short-term and long-term trends) from the full high-resolution network traffic data, rather than processing all raw data points. By extracting and analyzing only these trend components, the system achieves effective anomaly detection with significantly reduced computational requirements compared to comprehensive high-resolution analysis.
3Difficulty of detecting and measuring
If high time resolution is used for network traffic data, then anomaly detection capability is improved, but noise susceptibility increases making short bursts harder to distinguish
Solution Approach 1:
The patent segments the traffic analysis into short-term and long-term trend components, where the long-term trend captures the baseline pattern and the short-term trend captures immediate variations. By segmenting the analysis this way, genuine anomalies appear as significant deviations from the long-term trend that are confirmed in the short-term view, while noise and short bursts appear as temporary fluctuations that do not sustain across both time scales, effectively separating signal from noise.
Solution Approach 2:
The patent adds the time-scale dimension to the analysis, examining traffic patterns across multiple temporal resolutions simultaneously. This multi-scale temporal analysis allows the system to distinguish between transient noise (which appears differently across time scales) and genuine anomalies (which show consistent patterns across scales), thereby improving signal-to-noise ratio without sacrificing detection capability.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
A method described herein involves various operations directed toward network security. The operations include accessing transaction data describing network traffic associated with a web server during an interval. Based on a count of new transactions involving an online entity during the interval according to the transaction data, a short-term trend is determined for the online entity. The operations further include applying exponential smoothing to a history of transactions of the online entity to compute a long-term trend for the online entity. Based on a comparison between the short-term trend and the long-term trend for the online entity, an anomaly is detected with respect to the online entity in the network traffic associated with the web server. Responsive to detecting the anomaly, an access control is implemented between the online entity and the web server.