Multi-Sequence Transformer for Industrial Log Analysis
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Industrial machines generate massive, unordered, unlabeled, and unstructured log files that are difficult for human operators to analyze, leading to inefficiencies in identifying the root cause of machine errors.
Innovation Solution
A multi-sequence transformer is trained on log file databases to predict normal behavior and detect abnormal behavior in industrial machines by analyzing the deviation between predicted and actual log file tokens.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If log files are designed to be overinclusive to record all states of all components, then the completeness of error troubleshooting information is improved, but the data size and complexity of log files increase massively
Solution Approach 1:
The patent extracts only the relevant tokens from the massive log files that are necessary for troubleshooting. The transformer model processes the complete log files but outputs only the subset of tokens that contain error-related information, effectively separating the useful signal from the overwhelming noise of irrelevant data.
Solution Approach 2:
The transformer model serves as an intermediary between the overinclusive log files and the troubleshooting process. It translates the massive, complex log file data into a condensed form by predicting and identifying relevant tokens, making the information manageable and actionable for operators.
2Measurement precision
If supervised learning is used to train machine learning models on log files, then the accuracy of error detection is improved, but the requirement for labeled log files increases which are scarce and cumbersome to obtain
Solution Approach 1:
Instead of using labeled log files to train the model to detect errors, the patent inverts the approach by using unlabeled log files to train the model to predict normal operation. Error detection then becomes identifying deviations from this learned normal behavior, eliminating the need for scarce labeled error data.
Solution Approach 2:
The system uses the abundant unlabeled log files that machines generate naturally to train itself. The transformer model learns from the self-generated data without requiring external labeling efforts, making the training process self-sufficient and scalable.
3Measurement precision
If human operators manually analyze log files to determine the cause of machine errors, then the accuracy of error diagnosis is improved, but the time required for troubleshooting increases significantly
Solution Approach 1:
The patent replaces the mechanical process of human operators manually analyzing log files with an automated transformer model. The model processes the log data and identifies relevant tokens automatically, substituting human cognitive effort with machine learning computation that operates faster and consistently.
4Reliability
If the mapping between components and log files is made comprehensive to cover all relationships, then the completeness of component tracking is improved, but the complexity of interrelationships between components and log file tokens increases
Solution Approach 1:
The transformer model extracts and identifies only the relevant component-token relationships from the comprehensive but complex mapping. By predicting which tokens are relevant to errors, the system effectively filters the complex interrelationships down to the essential connections needed for troubleshooting.
Data Source
AI summary
A multi-sequence transformer predict N tokens, in parallel, for log files in industrial machines. Token patterns derived from N log file token sequences logged by N respective loggers may be input into N respective intra sequence multi-head self-attention layers identifying patterns among tokens within the same log file token sequence generated by the same logger. Token patterns derived from a combination of the N log file token sequences may be input into a same inter sequence multi-head self-attention layer identifying patterns among tokens across multiple different sequences generated by multiple different loggers. N softmax layers may be generated of N distinct probability distributions that each candidate tokens is a next token in each of the N respective log file token sequences. A plurality of N next tokens may be predicted, in parallel, to co-occur in the plurality of N respective sequences of log file tokens.