Event Correlation via Feature Linkage Analysis
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Enterprise environments face challenges in analyzing extensive log files to detect errors and anomalies due to the volume of data, requiring efficient methods to extract relevant insights and correlations.
Innovation Solution
An event correlation system that mines log files to generate directed cyclic graphs, compares feature sets across log files, and determines linkage strength and time lapse metrics to identify correlations and anomalies, providing a quantitative measure of event relationships.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of information
If log content analytics is applied to analyze heterogeneous computer-generated log files, then relevant insights can be discovered and extracted in a rationalized and structured form, but the extensive volume of log files makes sifting through the data daunting
Solution Approach 1:
The system segments log files into multiple data sources and processes them individually through feature extraction and correlation analysis. Each log file is treated as a separate trace sequence that can be independently analyzed and correlated with other logs, making the overwhelming volume of data manageable through systematic division and processing
Solution Approach 2:
The patent introduces an intermediary correlation analysis layer that sits between raw log data and final insights. This intermediary layer extracts features, determines linkage strengths, and establishes correlations between events across different log files, transforming the daunting raw data into structured insights through a manageable intermediate representation
2Reliability
If feature extraction and correlation analysis are performed across multiple log files, then event relationships and anomalies can be identified, but the complexity of processing and comparing feature sets increases
Solution Approach 1:
The system transforms complex log data into standardized feature parameters (linkage strength, time lapse, correlation metrics) that can be uniformly compared across different log files. By changing the representation of data into these standardized parameters, the complexity of comparing heterogeneous log files is reduced while maintaining detection reliability
Solution Approach 2:
The patent applies partial action by focusing correlation analysis on specific feature sets and linkage criteria rather than analyzing every possible aspect of log data. This selective approach to correlation analysis reduces processing complexity while still identifying significant anomalies and event relationships
Data Source
AI summary
Event correlation may include identifying a feature set for each log file of a plurality of log files, and extracting the feature set for each event of a plurality of events in each log file of the plurality of log files. Event correlation may further include determining a plurality of trace event pairs linkage strength values for an event from a first log file of the plurality of log files and a plurality of events from a second log file of the plurality of log files. The trace event pairs linkage strength values may represent an overlap of the feature set for the event from the first log file and the feature set for each of the plurality of events from the second log file.