Time Series Shape Matching for Scalable Log Correlation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional log analytics tools struggle to efficiently scale and analyze vast amounts of data from large numbers of computing systems and applications, leading to inefficient resource usage and difficulty in identifying correlated behaviors within complex systems.
Innovation Solution
A log analytics system that uses shape identification values to characterize time series datasets, allowing for efficient comparison and correlation of log records across distributed systems, thereby facilitating rapid triaging and root cause analysis by identifying messages with similar occurrence patterns.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If conventional log analytics tools analyze vast amounts of data from large numbers of computing systems, then comprehensive system monitoring is achieved, but resource usage becomes inefficient and scaling becomes difficult
Solution Approach 1:
The patent extracts and compares only the essential shape characteristics of time series signals using shape identification values, rather than analyzing entire log datasets. This selective extraction of key features enables efficient comparison and correlation identification without processing voluminous log data, thereby maintaining comprehensive monitoring capability while dramatically improving analysis efficiency and reducing resource consumption
Solution Approach 2:
The patent transforms time series signals into shape identification values by changing the parameter representation from raw data points to condensed shape descriptors. This parameter transformation allows for rapid comparison and correlation analysis of log patterns across distributed systems, solving the scalability problem while preserving the ability to detect correlated behaviors
2Measurement precision
If conventional tools attempt to identify correlated behaviors in complex systems, then root cause analysis capability is improved, but the time and effort required for triaging increases
Solution Approach 1:
The patent performs preliminary action by pre-computing and storing shape identification values for time series signals. When correlation analysis is needed, these pre-computed values enable rapid comparison and identification of correlated log patterns without requiring time-consuming real-time processing of entire log datasets, thus maintaining high correlation detection accuracy while significantly reducing triaging time
Solution Approach 2:
The patent uses shape identification values as simplified copies or representations of the actual time series signals. These compact representations capture the essential shape characteristics needed for correlation detection, allowing rapid comparison and analysis without handling the full complexity and volume of original log data, thereby reducing analysis time while preserving correlation identification capability
Data Source
AI summary
Some embodiments relate to analyzing log records. A method may include determining a first shape identification value that characterizes a shape described by data points of a first time series signal that represents a time distribution of timestamps of a first plurality of messages. For each message among a second plurality of messages, the method may also include determining a shape identification value for the message that characterizes a shape described by data points of a corresponding time series signal that represents a time distribution of timestamps of a plurality of instances of the message. The method may further include determining that a shape identification value, from among the shape identification values for the second plurality of messages, is the same as the first shape identification value and, in response to the determining, providing information identifying the corresponding message to a user interface.