Industrial Plant Log Analysis Using Semantic Event Graphs
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The manual analysis of log files in industrial plants is infeasible due to the large volume of information and the complexity of interpreting system-level events, which requires extensive training for domain experts and is time-consuming and costly, especially when dealing with different subsystems and unstructured text messages.
Innovation Solution
A method that automatically interprets log files by determining blocks of co-occurring log messages, annotating them with semantic metadata, and representing them in a graph structure for event detection, using techniques like Frequency Pattern Trees and graph convolutional neural networks to identify patterns and anomalies.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If manual review of log files is performed, then detailed analysis of log entries is possible, but the process becomes infeasible due to large volume of information
Solution Approach 1:
The patent segments log files into meaningful blocks by identifying co-occurring log messages that belong together. This segmentation transforms the overwhelming volume of individual log entries into manageable blocks, each representing a coherent event or process state, making manual review feasible while preserving detailed analysis capability.
Solution Approach 2:
The patent introduces an intermediary processing layer that automatically clusters and annotates log messages before presentation to domain experts. This intermediary system performs preliminary analysis by grouping related messages and adding semantic metadata, reducing the volume of information experts need to review while maintaining analytical depth.
2Measurement precision
If domain experts manually inspect log files to find anomalies, then detailed interpretation is possible, but extensive training is required and the process is time-consuming
Solution Approach 1:
The patent performs preliminary actions by automatically clustering log messages into coherent blocks and annotating them with semantic metadata before presentation to domain experts. This pre-processing work includes identifying co-occurring messages, determining message types, and highlighting potential anomalies, so experts receive pre-organized information ready for interpretation.
Solution Approach 2:
The system enables self-service by automatically performing tasks that would otherwise require expert intervention, such as clustering messages, annotating semantic metadata, and identifying potential anomalies. This allows the system to serve itself in preparing log data, reducing the time experts need to spend on preliminary analysis while preserving their ability to perform detailed interpretation.
3Reliability
If regular expressions are manually crafted to match log entries, then specific patterns can be detected, but the formulation effort is time-consuming and not scalable
Solution Approach 1:
The system performs self-service by automatically learning clustering rules from log data without requiring manual crafting of regular expressions. The system analyzes co-occurring messages and derives clustering patterns autonomously, making the process scalable to different log file types while maintaining reliable pattern detection through data-driven rule generation.
4Productivity
If existing log monitoring tools are used, then basic pattern detection is possible, but complex patterns and hierarchical dependencies are not considered
Solution Approach 1:
The patent segments log analysis into hierarchical levels: individual messages, co-occurring message blocks, and higher-level event patterns. This multi-level segmentation enables the system to capture both basic patterns at the message level and complex hierarchical dependencies across multiple blocks, overcoming the limitations of flat pattern matching in existing tools.
Solution Approach 2:
The patent adds temporal and hierarchical dimensions to log analysis by clustering messages that co-occur in time and organizing them into nested blocks. This dimensional enhancement transforms simple pattern matching into multi-dimensional pattern recognition, enabling detection of complex sequences and dependencies that span multiple log entries while maintaining basic detection capabilities.
Data Source
AI summary
Device, Computing Platform and Method of Analyzing Log Files of an Industrial Plant are disclosed. The method including: determining at least one block in log entries of the log files, wherein the log entries includes one or more log messages and wherein the block represents co-occurring log messages; annotating the co-occurring log messages of the block using semantic metadata, wherein the semantic metadata defines one or more message types for the co-occurring log messages, wherein the semantic metadata is indicative of at least one of a start action, an end action, a source, an anomaly, a cause and an inspect action; generating a coherent representation for the block by representing the co-occurring log messages in a graph based on the semantic metadata; and enabling detection of at least one event in the block based on a comparison the coherent representation with template representations of predefined events associated with the industrial plant.