Anomaly Detection in Time Series Using Deep Learning Feature Vectors
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional anomaly detection methods, such as linear statistical methods, are inadequate for complex, distributed, and unpredictable systems like microservice application platforms due to non-stationarity, multiple seasonality frequencies, and high noise-to-signal ratios, requiring significant user intervention and manual parameter settings.
Innovation Solution
A deep learning-based approach that combines signal, contextual, and temporal features into a feature vector, using unsupervised and supervised machine learning algorithms, including recurrent neural networks and convolutional neural networks, to automatically detect anomalies by comparing current time series to a pre-stored baseline model, minimizing user involvement and improving detection accuracy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If linear statistical methods (ARIMA, Holt-Winters, STL) are used for anomaly detection, then the detection works well for simple stationary time series with single seasonality, but it fails for complex non-stationary time series with multiple seasonality frequencies and high noise ratios
Solution Approach 1:
The patent transforms the time series data through multiple parameter changes including differencing to achieve stationarity, log transformations to stabilize variance, and seasonal decomposition to separate multiple frequency components. These parameter transformations enable the application of statistical methods to complex non-stationary data that would otherwise be incompatible with traditional anomaly detection approaches
Solution Approach 2:
The patent combines multiple statistical methods (ARIMA for trend, seasonal decomposition for periodic patterns, and outlier detection algorithms) into a composite anomaly detection system. This composite approach integrates the strengths of individual methods to handle the multiple seasonality frequencies and non-stationarity characteristics of complex distributed systems
2Ease of manufacture
If manual parameter setting is used in conventional anomaly detection methods, then the methods can be applied to simple time series, but it requires significant user intervention and cannot handle complex systems automatically
Solution Approach 1:
The patent implements automatic parameter selection and model configuration through data-driven methods. The system automatically determines optimal decomposition parameters, selects appropriate statistical models, and configures detection thresholds based on the characteristics of the input time series data, eliminating the need for manual parameter setting by domain experts
Solution Approach 2:
The patent performs preliminary data preprocessing including stationarity testing, seasonality detection, and noise characterization before applying anomaly detection. This preliminary analysis automatically configures the detection parameters based on the actual data characteristics, enabling the system to adapt to different complex systems without manual intervention
Data Source
Figure 1
Figure 2
Figure 3
AI summary
The present disclosure relates to the field of data processing, and more particularly to an apparatus and method for detecting an anomaly among successive events by using deep learning technologies, as well as to a corresponding computer program product. The present disclosure allows the anomaly to be detected in a time series quickly and automatically even if the time series is characterized by non-stationarity, multiple seasonality frequencies, multiple underlying distribution, and a high noise to signal ratio. For this purpose, the present disclosure suggests retrieving different salient features from the time series, which are then combined in a feature vector. The feature vector is further analyzed for its correspondence to a pre-stored baseline model characterizing a normal class. The time series is associated with the anomaly if there is a lack of correspondence between the feature vector of the time series and the baseline model.