Log Entry Template Generation for Automated Knowledge Graph Construction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Manual analysis of log files in industrial plants is error-prone, time-consuming, and expensive due to the variety of log file structures and formats, and existing automated methods require expert knowledge and are inefficient.
Innovation Solution
A computer-implemented method using unsupervised clustering to generate triples from log entries, where templates with variable and fixed parts are created and applied to classify similar messages, extracting key/value pairs without human intervention, resulting in reliable and efficient data for knowledge graphs.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If manual analysis of log files is performed by experts, then relevant information can be extracted from diverse log structures, but the process becomes time-consuming, error-prone, and expensive
Solution Approach 1:
The system performs automated self-analysis of log files using unsupervised clustering algorithms and template generation, eliminating the need for expert manual intervention. The method automatically identifies patterns, generates templates, and extracts information from diverse log structures without human input, thereby reducing time consumption while maintaining reliability through algorithmic consistency
Solution Approach 2:
The invention transforms the analysis approach by changing from manual expert parameters to automated computational parameters. It uses unsupervised clustering with similarity measures to dynamically generate templates and extract key/value pairs, converting subjective expert judgment into objective algorithmic processing that is both faster and more reliable
2Productivity
If automated information extraction is performed using regular expressions, then processing speed increases, but the approach requires expert knowledge for pattern definition and is error-prone
Solution Approach 1:
The system performs preliminary unsupervised clustering on log entries to automatically generate templates before information extraction. This preliminary action creates reusable patterns from the data itself, eliminating the need for experts to manually define regular expressions. The templates are then applied to extract key/value pairs, maintaining high processing speed while reducing complexity
Solution Approach 2:
The invention introduces templates as an intermediary between raw log entries and extracted information. These templates are automatically generated through unsupervised clustering and serve as mediators that simplify the extraction process. Instead of directly applying complex regular expressions, the system uses these intermediate templates to guide information extraction, reducing both complexity and error rates
3Reliability
If IncMap is used for mapping log entries to ontology concepts, then structured mapping can be achieved, but the ontology must be defined beforehand in a time-consuming manner
Solution Approach 1:
The invention inverts the traditional mapping approach by not starting with a pre-defined ontology. Instead, it performs unsupervised clustering on log entries to automatically discover patterns and generate templates from the data itself. This inversion eliminates the time-consuming ontology definition step while maintaining mapping consistency through the algorithmic template generation process
4Adaptability or versatility
If diverse log file structures and formats are processed manually, then comprehensive analysis is possible, but the effort and cost increase significantly
Solution Approach 1:
The system achieves universality by using unsupervised clustering that can handle diverse log file structures and formats without requiring format-specific processing. The same clustering algorithm and template generation process works across different log types, making the system multi-functional and adaptable to various formats while maintaining ease of implementation through a unified approach
Data Source
AI summary
A computer-implemented method, computer program product, and a technical system for generating triples includingproviding a plurality of log entries from respective log files, wherein each log entry of the plurality of log entries includes at least one text message,generating at least one template based on the plurality of log entries using unsupervised clustering, wherein the at least one template includes at least one variable part and at least one fixed part,assigning each log entry of the plurality of log entries to one respective template based on the generated at least one template using a similarity measure,extracting the corresponding at least one variable and at least one fixed part of each text message of the plurality of text messages as key/value pairs using the respective assigned at least one template based on the plurality of log entries, andproviding the text messages, keys and values as triples.