Log Message Correlation System for Incident Management
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The challenge lies in managing and reviewing vast volumes of log messages from computer systems, which are often unstructured and varied in format, making it labor-intensive and costly to identify incidents and ensure regulatory compliance, especially given the requirement to archive these messages for extended periods.
Innovation Solution
A system and method that pre-parses disparate log messages and compares them to incident descriptions, creating an incident case with workflow steps when correlations are found, allowing for automated review, archiving, and compliance reporting, thereby reducing manual intervention and resource requirements.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If log messages are archived for extended periods to meet regulatory requirements, then compliance is improved, but storage volume and associated costs increase significantly
Solution Approach 1:
The patent extracts only the essential elements from log messages for long-term archiving. Instead of storing complete log files, the system identifies and extracts key incident-relevant data points, storing only these extracted elements in the archive while maintaining full compliance capabilities with reduced storage requirements.
Solution Approach 2:
The system performs preliminary parsing and analysis of log messages before archiving. By pre-processing log data to identify and structure incident-relevant information in advance, the system prepares data for efficient long-term storage and future compliance queries without needing to retain all original log details.
2Measurement precision
If manual review of log messages is performed to identify incidents, then detection accuracy is improved, but labor time and resources increase massively
Solution Approach 1:
The patent introduces an intermediary automated analysis system that acts as a mediator between raw log messages and human reviewers. This intermediary layer pre-processes logs, identifies potential incidents, and presents curated incident candidates to human analysts, maintaining detection accuracy while dramatically reducing the volume of manual review required.
Solution Approach 2:
The system performs preliminary automated analysis of log messages to identify potential incidents before human review. By pre-processing logs to detect patterns, anomalies, and incident signatures, the system prepares incident candidates in advance, allowing human reviewers to focus only on confirmed incidents rather than reviewing every log message from scratch.
3Adaptability or versatility
If diverse log message formats from multiple vendors are accepted, then system compatibility is improved, but processing complexity increases due to freeform variations
Solution Approach 1:
The patent applies parameter changes by transforming diverse log message formats into a standardized internal representation. The system dynamically adjusts parsing parameters and patterns based on the specific vendor and format being processed, converting freeform logs into structured data with consistent fields and schemas that simplify downstream processing while maintaining compatibility with multiple vendors.
Solution Approach 2:
The system segments the complex task of processing diverse log formats into modular format-specific parsers. Each vendor's log format is handled by a dedicated parsing module that extracts standardized fields, allowing the system to maintain high compatibility while managing processing complexity through organized, reusable components rather than monolithic processing logic.
Data Source
AI summary
Systems and methods for correlating log messages into actionable incidents. Some embodiments implement a method which includes comparing a plurality of disparate log messages to a plurality of incident descriptions. The disparate log messages can be parsed. When the messages correlate with an incident description an incident case can be created. Workflow steps can be associated with the incident case and output along with the incident case. Additional disparate log messages can be compared to the incident expressions and, when additional messages correlate with the correlated incident description, the incident case can be adjusted. In some embodiments, the adjustment can include adding workflow steps to the incident case. Results of various workflow steps can be monitored and adjustments can be made accordingly. In some embodiments, the results can include out-of-bounds activities.