Log Annotating via Documentation Templates
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing log message analytics struggle due to the unstructured and dynamic nature of log messages, making template mining ineffective for system diagnosis and anomaly detection.
Innovation Solution
A method and system for annotating logs by processing log documentation to create templates with constants and variables, using natural language processing and machine learning techniques to identify and label variables in log entries, enabling accurate annotation and pattern recognition.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If template mining is performed directly on unstructured log messages, then the process is simple and fast, but the effectiveness and accuracy of anomaly detection deteriorate
Solution Approach 1:
The patent applies preliminary action by using natural language processing to process log documentation and extract variable definitions, data types, and descriptions before performing template mining on log messages. This preprocessing step creates a structured reference that guides the subsequent template matching and annotation process, improving effectiveness without significantly increasing overall system complexity
Solution Approach 2:
The patent introduces an intermediary layer consisting of extracted variable information (names, types, descriptions) from log documentation that mediates between the unstructured log messages and the template mining process. This intermediary structure enables accurate variable identification and template matching while maintaining a manageable process complexity
2Measurement precision
If log messages are processed without structured templates, then the processing speed is fast, but the precision of variable identification and annotation deteriorates
Solution Approach 1:
The system performs preliminary extraction of variable information (names, types, descriptions) from log documentation before log message processing. This advance preparation creates a reference structure that enables precise variable identification during log processing without requiring complex real-time analysis, thus maintaining processing efficiency while improving precision
Solution Approach 2:
The patent changes the parameter state of log messages by transforming unstructured text into structured annotated data with identified variables, types, and descriptions. This parameter transformation enables precise measurement and classification of log contents while the use of pre-extracted templates keeps the time cost manageable
3Reliability
If dynamic log contents are analyzed without documentation processing, then the adaptability to different workloads is high, but the accuracy of system diagnosis deteriorates
Solution Approach 1:
The patent creates a universal solution by processing log documentation to extract variable information that can be applied across different log messages, workloads, and time periods. The extracted templates and variable definitions serve as a multi-functional reference that improves diagnostic accuracy for diverse log contents while maintaining adaptability through the flexible matching process
Solution Approach 2:
The patent introduces an intermediary layer of extracted variable information from documentation that mediates between diverse dynamic log contents and the diagnosis process. This intermediary structure provides consistent variable identification and classification across different workloads and time periods, improving diagnostic accuracy while adapting to various log formats and contents
Data Source
AI summary
Embodiments of the invention are directed to annotating a log based on processing log documentation. Aspects include obtaining the log having a plurality of entries. Aspects also include creating a set of log entry templates by processing the log documentation associated with the log, wherein each log entry template includes one or more constants and one or more variables. Aspects further include annotating each of the plurality of entries based on the set of templates, wherein the annotating includes labeling each value of the one or more variables with a variable name.