Dynamic Change Point Detection for Streaming Data Anomalies
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing computing systems struggle to efficiently analyze ever-changing data streams and detect anomalies in real-time, leading to potential system incidents and increased costs for incident resolution.
Innovation Solution
A computer-implemented method for dynamic change point and anomaly detection in live streaming data, using previously collected time series data to identify non-anomalous blips, anomalous spikes, and changes in normal, with user-defined and default parameters for real-time analysis.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of time
If real-time analysis of live streaming data is performed to detect anomalies promptly, then incident resolution time is reduced, but computational complexity and resource consumption increase
Solution Approach 1:
The patent segments the data stream into fixed-time windows and performs change point detection on each window independently. This divides the complex real-time analysis task into smaller, manageable segments that can be processed efficiently with less computational overhead while maintaining real-time detection capability.
Solution Approach 2:
The patent pre-calculates statistical parameters (mean, standard deviation) for each time window before performing anomaly detection. By preparing these baseline metrics in advance, the system reduces the computational burden during actual real-time analysis, enabling faster incident detection without excessive resource consumption.
2Measurement precision
If change point detection is performed on every data point in the time series, then measurement precision of anomalies is improved, but processing speed decreases
Solution Approach 1:
The patent divides the continuous time series into discrete time windows and performs change point detection at window boundaries rather than on every data point. This segmentation maintains measurement precision by capturing significant changes while dramatically improving processing speed through reduced computation frequency.
Solution Approach 2:
The patent applies change point detection selectively at critical points (time window boundaries) rather than continuously on all data points. This partial action approach achieves sufficient anomaly detection precision for incident identification while maintaining high processing speed for real-time operation.
3Reliability
If multiple parameters (minimum size, penalty, change point detection, anomaly parameters) are configured for accurate anomaly identification, then reliability of detection is improved, but ease of operation decreases
Solution Approach 1:
The patent implements dynamic parameter adjustment where the minimum size parameter and penalty parameter are automatically adapted based on the characteristics of the data stream and detection results. This dynamic configuration maintains high detection reliability while reducing the burden on operators to manually tune multiple parameters, as the system self-adjusts to optimal settings.
Data Source
AI summary
A computer-implemented method for processing live streaming data includes performing a change point detection on a time series of data using one or more previously collected time series of data, an identification comprising: identifying one or more non-anomalous blips in the time series of data based on the change point detection, identifying one or more anomalous spikes in the time series of data based on the change point detection, and identifying one or more changes in normal in the time series of data based on the change point detection, displaying the one or more changes in normal from the time series of data based on the identification, and displaying the one or more anomalous spikes from the time series of data based on the identification.