Automation Log Classification for Rare and Anomalous Events
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing system automation tools struggle to differentiate between rare and anomalous events in complex computing environments, leading to difficulties in identifying errors and potential security vulnerabilities, especially in z-architecture systems and large cloud computing data centers.
Innovation Solution
A computer-implemented method and system utilizing a combination of long short-term memory (LSTM) and 1-class support vector machine (1C-SVM) machine-learning models to classify log entries, predicting the likelihood and unprecedented nature of next log entries, enabling differentiation between normal, rare, and anomalous events.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If conventional system automation tools are used to analyze log entries, then basic log monitoring is possible, but the ability to differentiate between rare and anomalous events deteriorates
Solution Approach 1:
The patent segments the event classification task into three distinct categories (normal, rare, anomalous) and uses separate machine learning models for each classification level. The first ML model classifies events as normal or uncommon, while the second ML model specifically identifies anomalous events, allowing for precise differentiation without overwhelming complexity
Solution Approach 2:
The patent introduces machine learning models as intermediary components between raw log data and human operators. These models process and interpret log entries, providing classified outputs that help operators understand system state without requiring them to manually analyze complex log patterns
2Productivity
If manual analysis of log entries is performed, then detailed inspection is possible, but operator workload and time consumption increase
Solution Approach 1:
The system performs self-service by automatically analyzing log entries and classifying events without requiring manual intervention. The machine learning models continuously process log data, detect patterns, and provide classifications that enable automated responses, freeing operators from routine monitoring tasks
Solution Approach 2:
The patent implements feedback mechanisms where the system continuously monitors log entries, compares them against learned patterns, and provides real-time classifications. This feedback loop allows the system to adapt to changing system behavior and maintain high detection efficiency over time
3Reliability
If false-positive alarms are generated, then anomaly detection sensitivity improves, but system reliability deteriorates
Solution Approach 1:
The patent applies local quality by treating different event types with different classification criteria. The first ML model uses specific features to distinguish normal from rare events, while the second ML model applies different features to identify true anomalies. This differentiated approach reduces false positives by avoiding overly sensitive detection across all event types
Data Source
AI summary
A computer-implemented method, system, and computer program product for classifying a sequence of log entries of a computing system may be provided. The method may include pre-processing the log entries. The method may also include predicting, as a first output of a first trained machine-learning system, a likelihood of a particular next log entry after the window. The method may also include, predicting, as a second output of a second trained machine-learning system, whether the next log entry is unprecedented. The method may also include combining the first output and the second output for determining a classification of the sequence of log entries.