Log Analysis System Pattern Extraction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing technologies fail to effectively extract a normal pattern from a normal log in information processing systems without pre-provided information for grouping messages in units of process, making it difficult to analyze system faults.
Innovation Solution
A log analysis system that divides log files into contiguous time series groups, extracts patterns formed by contiguous events, and associates them with frequency information, selecting patterns based on inclusion relationships and frequency to output meaningful patterns without pre-provided grouping information.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If the administrator analyzes the entire log in detail, then the accuracy of fault cause identification is improved, but the time and computational resources required increase significantly
Solution Approach 1:
The patent segments the huge log into multiple divided logs based on time periods or other criteria. This allows the administrator to analyze only relevant portions of the log rather than the entire log, significantly reducing analysis time while maintaining the ability to identify fault causes accurately when needed.
Solution Approach 2:
The patent extracts cause messages that are directly related to fault occurrence from the huge log using pattern matching and frequency analysis. By extracting only the relevant cause messages and their surrounding context, the system enables focused analysis that maintains accuracy while reducing the volume of data that requires detailed examination.
2Productivity
If the administrator identifies cause messages from the huge log, then the fault analysis efficiency is improved, but the difficulty of accurately identifying relevant patterns increases
Solution Approach 1:
The patent uses frequency information as feedback to identify cause messages. By analyzing how frequently certain message patterns occur in the log and comparing this to normal operation patterns, the system automatically highlights anomalies that are likely to be cause messages, reducing the difficulty of identification while maintaining efficiency.
Solution Approach 2:
The patent replaces manual pattern identification with automated computer-based analysis using pattern matching algorithms and frequency analysis. This substitution of mechanical/manual processes with automated computational methods reduces both the difficulty of identifying patterns and maintains high productivity in fault analysis.
3Measurement precision
If pre-provided grouping information is used to extract normal patterns, then the pattern extraction accuracy is improved, but the system complexity and preparation requirements increase
Solution Approach 1:
The patent enables the log analysis system to automatically extract normal patterns from logs without requiring pre-provided grouping information. The system performs self-service by autonomously analyzing log data, identifying message patterns, and determining normal operation patterns through frequency analysis, thereby eliminating the need for complex external configuration while maintaining extraction accuracy.
Data Source
AI summary
An objective of the present invention is to appropriately extract, or assist in extracting, a normal operation pattern from a normal operation log. A log file including information wherein a plurality of log records is ordered in time series is segmented into a plurality of log groups according to prescribed rules. A plurality of patterns which are configured by a plurality of events which are contiguous in time series is extracted from the plurality of log groups, and the patterns are associated with frequency information which represents how many log records the pattern is extracted from. The pattern to be outputted from the plurality of patterns is selected on the basis of an inclusion relation between the plurality of events which are contiguous in time series and configure a given pattern and the plurality of events which are contiguous in time series and configure other patterns, and on the basis of the frequency information which is associated with the patterns.