Anomaly Detection for Non-Stationary Data via Adaptive Segmentation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Complex systems like electronic commerce systems generate vast amounts of non-stationary data, making it challenging for human operators to determine their overall health and detect anomalies, which can have negative business impacts.
Innovation Solution
An automated anomaly detection system that continuously monitors data from data-generating processes, using a data-adaptive approach to identify deviations from statistical equilibrium, even in non-stationary data environments.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If automated anomaly detection is implemented for non-stationary data, then detection accuracy is improved, but system complexity increases
Solution Approach 1:
The patent segments the anomaly detection process into distinct modules: a non-stationarity detection module that identifies whether data is non-stationary, and separate anomaly detection paths for stationary and non-stationary data. This segmentation allows the system to handle complexity selectively rather than universally, improving detection accuracy for non-stationary data while managing overall system complexity through modular design.
Solution Approach 2:
The patent implements dynamic adaptability by detecting the non-stationary nature of data and adjusting the detection approach accordingly. The system dynamically selects between different detection methodologies based on the observed characteristics of the input data, allowing it to adapt to changing data conditions without requiring a completely complex system for all possible scenarios.
2Reliability
If data-adaptive approach is used to handle non-stationarity, then false detections are minimized, but computational requirements increase
Solution Approach 1:
The patent performs preliminary detection of non-stationarity characteristics before applying the full anomaly detection algorithm. By identifying non-stationary data early in the process, the system can prepare appropriate detection parameters and methods in advance, reducing false detections while avoiding unnecessary computational overhead for data that doesn't require complex handling.
Solution Approach 2:
The patent changes detection parameters based on the non-stationary characteristics of the data. When non-stationarity is detected, the system adjusts its detection thresholds, time windows, and statistical methods to match the data's changing properties. This parameter adaptation improves reliability by matching the detection sensitivity to the actual data conditions without requiring consistently high computational resources.
Data Source
AI summary
A method of detecting anomalies in a time series is disclosed. A training time series corresponding to a process is extracted from an initial time series corresponding to the process, the training time series including a subset of the initial time series. Outlier data points in the training time series are modified based on predetermined acceptability criteria. A plurality of prediction methods are trained using the training time series. An actual data point corresponding to the initial time series is received. The plurality of prediction methods are used to determine a set of predicted data points corresponding to the actual data point. It is determined whether the actual data point is anomalous based on a calculation of whether each of the set of predicted data points is statistically different from the actual data point.