Log Record Analysis Using Automatic Template Parsing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current log record analysis methods require generating regular expressions for each log record layout and language, which is time-consuming and costly, and often result in excessive logging that increases storage needs and problem resolution time, as well as making it difficult to identify the source of issues due to numerous irrelevant log records.
Innovation Solution
The method automatically generates parsing instructions using regular expressions to extract log data items based on log record templates, eliminating the need for human intervention and reducing the complexity of handling different log record layouts and languages, thereby identifying the dominant log record template and taking appropriate actions.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If regular expressions are generated for each log record layout and language manually, then log record analysis accuracy is improved, but time consumption and cost increase significantly
Solution Approach 1:
The system performs self-service by automatically generating regular expressions and parsing instructions for log record analysis without requiring manual human intervention. The log analysis device autonomously processes different log record layouts and languages, eliminating the time-consuming manual creation of analysis rules while maintaining high accuracy.
Solution Approach 2:
The patent replaces the mechanical manual process of creating regular expressions with an automated computational system. The log analysis device uses algorithms and machine learning to generate parsing rules automatically, substituting human manual work with intelligent automation that is both faster and more consistent.
2Loss of information
If excessive logging is performed to capture all possible information, then information completeness is improved, but storage needs and problem resolution time increase
Solution Approach 1:
The system extracts only the relevant log records that correspond to the identified dominant template pattern, separating useful information from irrelevant data. By focusing analysis on the majority template type, the system retrieves critical information without storing or processing excessive log data, thus reducing storage requirements while maintaining information completeness for problem resolution.
Solution Approach 2:
Instead of performing complete analysis on all log records equally, the system applies partial action by concentrating resources on analyzing the dominant log template type that represents the majority of records. This selective approach captures sufficient information for most problems without the overhead of exhaustive analysis of every single log record.
3Adaptability or versatility
If multiple different log record templates are processed equally, then coverage of all log types is improved, but complexity of analysis increases and source identification becomes difficult
Solution Approach 1:
The system applies local quality by treating different log template types differently based on their prevalence. Instead of uniform processing, it identifies the dominant template type and focuses analysis resources there, applying different analysis strategies to different template categories based on their local characteristics and importance.
Solution Approach 2:
The system changes the parameter of analysis focus by dynamically identifying which log template type is most prevalent and adjusting the analysis parameters accordingly. By modifying the focus parameter from equal distribution to dominance-based selection, it simplifies the analysis process while maintaining adaptability to handle multiple log types when needed.
Data Source
AI summary
Log record analysis based on log record templates is disclosed. A plurality of log records that comprise log data generated by one or more logging entities over a period of time is accessed. Each log record of the plurality of log records corresponds to one log record template of a plurality of different log record templates. The log records are analyzed to determine a particular log record template of the plurality of different log record templates to which a majority of the plurality of log records corresponds. An action is taken that is at least partially based on the particular log record template.