Log Analysis Apparatus for Automated Event Extraction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing log analysis methods require prior knowledge of log generation rules and are not effective in extracting correlated log information from various machines without user input, limiting their applicability and efficiency.
Innovation Solution
A log analysis apparatus that stores templates for log messages, determines matching templates, extracts event candidates based on temporal correlation, and identifies frequently occurring template sequences as events, enabling the extraction of highly correlated logs without user-provided generation rules.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If prior knowledge of log generation rules is required for log analysis, then analysis accuracy can be improved, but the complexity of operation and ease of use deteriorate
Solution Approach 1:
The log analysis apparatus automatically extracts templates and identifies event patterns from log data without requiring user input of generation rules. The system performs self-learning by analyzing log message structures, extracting templates based on occurrence frequencies, and automatically identifying event sequences, thereby eliminating the need for manual rule configuration while maintaining high analysis accuracy
Solution Approach 2:
The system changes the parameter of template occurrence frequency to identify event patterns. By monitoring how frequently templates occur together in specific sequences and using this frequency information to automatically define events, the system transforms the approach from rule-based to pattern-based analysis, improving ease of operation while preserving measurement precision
2Measurement precision
If manual input and preparation of templates is required, then template accuracy can be improved, but the productivity and time consumption deteriorate
Solution Approach 1:
The system performs preliminary automatic template extraction from log data before actual analysis begins. By pre-processing log messages to extract templates based on their structural patterns and occurrence frequencies, the system prepares accurate templates automatically, eliminating manual preparation time while ensuring high matching accuracy through systematic pattern recognition
Solution Approach 2:
The system replaces the mechanical manual process of template creation with an automated computational process. The template extraction mechanism uses algorithmic analysis of log message structures and occurrence patterns to generate templates automatically, substituting human manual input with machine-based pattern recognition and frequency analysis, thereby dramatically improving productivity without sacrificing template accuracy
3Speed
If log analysis is performed without considering temporal correlation, then processing speed can be improved, but the reliability of event detection deteriorates
Solution Approach 1:
The system segments log analysis into template-level processing and event-level processing. At the template level, individual log messages are processed independently for fast matching. At the event level, temporal correlation is considered by analyzing sequences of templates occurring within defined time windows. This segmentation allows most processing to occur at high speed while reliability is enhanced through temporal pattern recognition for event detection
Solution Approach 2:
The system applies partial temporal correlation analysis by considering time factors only for event pattern identification rather than for every individual log processing operation. By applying temporal analysis selectively to sequence pattern matching rather than to all processing steps, the system maintains high processing speed for individual logs while ensuring reliable event detection through temporal correlation when needed
Data Source
AI summary
Techniques for extracting a set of highly correlated logs without user's prior knowledge of generation rules of logs generated by a machine. A log analysis apparatus includes: a template storage unit configured to store templates for log messages; a template determination unit configured to determine templates corresponding to log messages provided from a to-be-monitored host from the stored templates; an event candidate extraction unit configured to detect templates determined for log messages provided from the same host from the determined templates and extract a sequence of the detected templates as an event candidate; and an event extraction unit configured to extract a sequence of frequently occurring templates as an event from the extracted event candidate.