Periodicity Analysis for Heterogeneous Logs
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Traditional periodicity analysis methods struggle with complicated and noisy temporal data, particularly when data sampling is uneven or at a low rate, and are not effective in detecting hidden periodic behaviors in machine-generated logs.
Innovation Solution
The Category Model Based Periodicity Analysis (CMBPA) system transforms heterogeneous logs into multiple time series, builds a category model for each periodic event type, and applies it to live data to detect anomalies, using a fitness score, category center, and error bounds to identify statistically periodic patterns.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If traditional periodicity analysis methods (FFT, auto-correlation) are used, then measurement precision is improved for evenly sampled data, but applicability deteriorates for unevenly sampled data and low sampling rates
Solution Approach 1:
The patent transforms the time series data from uneven sampling intervals to evenly spaced intervals by resampling at a uniform rate. This parameter transformation allows traditional spectral analysis methods (FFT, auto-correlation) to be applied effectively, resolving the contradiction between measurement precision and adaptability to uneven sampling.
2Ease of operation
If statistical analysis techniques are applied to single time series, then analysis simplicity is improved, but detection capability deteriorates for noisy and oscillating periodic behaviors
Solution Approach 1:
The patent segments the single complex time series into multiple simpler time series based on different event types or characteristics. Each segmented series is then analyzed separately using statistical techniques, improving both the simplicity of analysis and the reliability of detecting periodic behaviors by reducing noise and oscillation effects in each segment.
Data Source
AI summary
Systems and methods are disclosed for detecting periodic event behaviors from machine generated logging by: capturing heterogeneous log messages, each log message including a time stamp and text content with one or more fields; recognizing log formats from log messages; transforming the text content into a set of time series data, one time series for each log format; during a training phase, analyzing the set of time series data and building a category model for each periodic event type in heterogeneous logs; and during live operation, applying the category model to a stream of time series data from live heterogeneous log messages and generating a flag on a time series data point violating the category model and generating an alarm report for the corresponding log message.