Incomplete Matrix Profiles for Real-Time Time Series Anomaly Detection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional matrix profile methods for anomaly detection in time series data are computationally intensive and inefficient, particularly for high-frequency data, leading to slow detection and outdated insights, and do not scale well for granularities smaller than one hour.
Innovation Solution
Anomaly detection process based on determining incomplete matrix profiles, which iterates over time series values and computes incomplete matrix profiles using a historical lookback parameter, reducing computational complexity to O(n) and enabling fast, efficient detection of anomalies at low granularities.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If conventional matrix profile methods are used for anomaly detection, then detection accuracy is maintained, but computational complexity becomes excessively high and detection speed decreases
Solution Approach 1:
The patent segments the complete matrix profile computation into incomplete matrix profiles by dividing the time series into overlapping segments and computing matrix profiles only for each segment independently. This segmentation reduces the overall computational complexity from O(n^2) for complete matrix profiles to approximately O(k*n) for incomplete matrix profiles, where k is the number of segments, while maintaining anomaly detection accuracy through the overlapping segment design.
Solution Approach 2:
The patent applies partial action by computing only the necessary portion of matrix profiles (incomplete matrix profiles) rather than complete matrix profiles for the entire time series. By calculating matrix profiles only for segmented portions of the data with overlap, the system achieves anomaly detection with reduced computational complexity O(k*n) compared to the full O(n^2) complexity of conventional methods.
2Reliability
If conventional matrix profile methods are used, then comprehensive anomaly detection is achieved, but processing time increases and real-time detection becomes infeasible
Solution Approach 1:
The time series data is divided into multiple overlapping segments, allowing parallel computation of incomplete matrix profiles for each segment. This segmentation enables the system to process large volumes of high-frequency data efficiently, reducing detection processing time from hours to minutes while maintaining comprehensive anomaly detection through the overlapping segment strategy that ensures no anomalies are missed at segment boundaries.
Solution Approach 2:
By computing incomplete matrix profiles for segmented portions rather than complete matrix profiles for the entire dataset, the patent reduces processing time significantly. The overlapping segments provide sufficient coverage for comprehensive detection while keeping individual segment computations small enough to be executed quickly, enabling near-real-time anomaly detection.
3Measurement precision
If complete matrix profiles are computed for high-frequency data, then detection accuracy is maintained, but the method does not scale for granularities smaller than one hour
Solution Approach 1:
The patent segments high-frequency time series data into manageable overlapping segments, enabling the computation of incomplete matrix profiles that scale linearly with data volume. This segmentation approach allows the system to handle granularities smaller than one hour (e.g., minute-level or second-level data) by processing segments in parallel, achieving both high detection accuracy and scalability that complete matrix profile methods cannot provide.
4Measurement precision
If batch-based anomaly detection is used, then thorough analysis is achieved, but detection results become outdated and fail to provide timely insights
Solution Approach 1:
The patent segments the time series into overlapping segments that can be processed incrementally rather than waiting for complete batch data accumulation. This allows the system to detect anomalies as they occur in each segment while maintaining thorough analysis through the overlapping design, providing timely detection results without requiring lengthy batch processing waits.
Data Source
AI summary
Various embodiments described herein support or provide for anomaly detection in time series data (e.g., comprising measurement values for a metric of a monitored system) based on one or more incomplete matrix profiles.