Nonparametric CUSUM Algorithm for Anomaly Detection in Non-Stationary Systems
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing change-point detection algorithms, such as the classic cumulative sum (cusum) algorithm, are inadequate for complex systems with non-stationary data and numerous performance metrics, as they require parametric assumptions and manual oversight, and are not computationally efficient for handling large datasets.
Innovation Solution
A modified cusum algorithm that uses non-parametric methods to detect anomalous events in complex systems by initializing dynamic thresholds from historical data, simulating data streams, and performing real-time cumulative sum analysis to automate the detection of anomalies, while accounting for non-stationarity and periodic fluctuations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If classic cusum algorithm is used for change-point detection, then the algorithm can detect shifts from targeted mean, but it requires parametric assumptions and manual oversight which reduces adaptability and increases complexity
Solution Approach 1:
The patent transforms the classic parametric cusum algorithm into a non-parametric version by changing the fundamental parameters and assumptions. Instead of requiring normal distribution assumptions and manual threshold setting, the algorithm uses empirical distribution functions and automated threshold determination based on historical data characteristics, making it adaptable to complex systems with unknown or non-normal distributions
Solution Approach 2:
The algorithm implements self-service by automatically determining thresholds and detecting change-points without requiring manual oversight or expert intervention. The system uses historical data to automatically calibrate parameters, set thresholds, and identify anomalous events, eliminating the need for continuous manual adjustment and oversight
2Measurement precision
If manual oversight is used for threshold setting and anomaly detection, then detection accuracy can be maintained, but productivity decreases due to time-consuming manual processes
Solution Approach 1:
The algorithm performs preliminary action by pre-processing historical data to establish baseline characteristics, automatically determining thresholds and parameters before real-time monitoring begins. This preliminary setup phase automates what would traditionally require manual expert analysis, enabling rapid real-time detection without ongoing manual intervention while maintaining high accuracy
Solution Approach 2:
The patent replaces the mechanical process of manual threshold setting and anomaly review with an automated computational system. The algorithm uses mathematical transformations and empirical distribution functions to automatically determine thresholds and detect change-points, substituting human manual processes with automated mechanical computation that maintains accuracy while dramatically increasing productivity
3Productivity
If parametric methods are used for data analysis, then computational efficiency can be maintained, but the methods become unsuitable for complex systems with non-stationary data and diverse metrics
Solution Approach 1:
The algorithm implements dynamics by adapting to non-stationary data characteristics rather than assuming fixed parametric forms. It uses empirical distribution functions that can evolve with changing data patterns and automatically adjusts to non-stationary conditions, making the system dynamically adaptable to complex systems with time-varying statistics while maintaining computational efficiency through algorithmic optimizations
Data Source
AI summary
According to a feature of the present disclosure, a method is provided for the determination of anomalous events in complex systems, such as problems, inefficiencies, and failures, and a tool is provided for the detection of these events. Many complex systems are non-stationary or experience periodic fluctuations or spikes in values that are outside of normal ranges, but constitute normal behavior nevertheless. The method accounts for both non-stationarity, as well as fluctuations and spikes. Additional novel features include both a threshold setting initialization method and a regression method for the determination of the start points and end points of events.