Sliding Window Outlier Detection for Server Transaction Times
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Performance monitoring systems face challenges in distinguishing between sudden or unexpected changes in server transaction times and shifts in normal expected values, especially in dynamic environments with varying workloads and contextual changes.
Innovation Solution
A method and system for detecting outliers in server transaction times using a sliding window approach, where data points are added to a window and outliers are identified based on predetermined thresholds, allowing for the establishment of new windows when excessive outliers are detected, thereby adjusting the baseline for normal values.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If a fixed baseline is used for outlier detection, then the detection rule is simple, but it cannot adapt to dynamic workloads and contextual changes causing false alerts
Solution Approach 1:
The patent implements a sliding window approach that dynamically updates the baseline for outlier detection. Instead of using a fixed baseline, the system maintains a rolling window of recent data points (e.g., last 10-30 minutes) and calculates statistics (mean, standard deviation) from this moving window. This allows the baseline to adapt automatically to changing workloads and contextual conditions while maintaining a relatively simple detection rule: compare new data points against the current sliding window statistics.
2Measurement precision
If all data points are included in the baseline calculation, then the baseline is comprehensive, but it includes outliers that skew the normal value range
Solution Approach 1:
The patent extracts and removes identified outliers from the sliding window before calculating the baseline statistics. When an outlier is detected (data point falling outside the confidence interval), it is excluded from the set of data points used to compute the mean and standard deviation. This ensures that the baseline reflects only normal operational patterns and not distorted by extreme values, improving both the precision of the baseline and the reliability of subsequent outlier detections.
3Reliability
If the outlier threshold is set strictly, then false positives are reduced, but legitimate performance issues may be missed
Solution Approach 1:
The patent uses statistical parameters (mean and standard deviation) derived from the sliding window to dynamically determine the outlier threshold. Instead of using a fixed threshold, the confidence interval is calculated as mean ± (z-score × standard deviation), where the standard deviation adapts to the variability in the recent data. This allows the detection sensitivity to adjust automatically: when variability is high, the threshold widens to avoid false positives; when variability is low, the threshold becomes more sensitive to detect subtle anomalies.
Data Source
AI summary
Described embodiments provide systems and methods for detecting outliers on a series of data. A device receives a plurality of data points and adds a received data point to a first window of data comprising at least a predetermined number of received data points from the plurality of data points, responsive to detecting that the received data point is not an outlier from the first window of data. The device detects that one or more next data points of the received plurality of data points are outliers from the first window of data and determines that a count of the one or more next data points that are outliers exceeds a predetermined threshold. In response, the device establishes a replacement window of data.