Volatile Event Type Detection in Distributed System Logs
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In large distributed computing systems, identifying relevant information within enormous event logs for detecting operational anomalies and managing systems is challenging due to the vast volume of event messages generated, which can lead to data loss and downtime.
Innovation Solution
The method involves identifying volatile event types in event-log files, determining their frequency over a historical period, and displaying event messages in a graphical user interface to narrow the search for potential sources of problems by analyzing frequency changes in time intervals and sub-intervals.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If all event messages in enormous event logs are analyzed to detect operational anomalies, then detection completeness is improved, but analysis time and system complexity increase significantly
Solution Approach 1:
The patent segments the enormous event logs by dividing them into manageable time intervals (e.g., hourly, daily segments). This allows the system to analyze specific time windows rather than processing all historical data, significantly reducing analysis time while maintaining detection effectiveness for recent anomalies.
Solution Approach 2:
The patent extracts and identifies volatile event types that exhibit unexpected frequency increases from the bulk event logs. By focusing analysis only on these volatile event types rather than all event messages, the system achieves efficient anomaly detection without processing the entire log volume.
2Reliability
If all event messages in enormous event logs are analyzed to detect operational anomalies, then detection completeness is improved, but system complexity increases significantly
Solution Approach 1:
The system complexity is reduced by segmenting the analysis process into distinct stages: volatile event type identification, frequency determination across time intervals, and anomaly detection. This modular approach simplifies the overall system architecture compared to analyzing all event messages comprehensively.
Solution Approach 2:
The patent extracts only the volatile event types that are relevant to anomaly detection, filtering out the vast majority of normal event messages. This extraction process simplifies the data set that requires complex analysis, reducing the computational burden and system complexity.
3Measurement precision
If volatile event types are identified and frequency analysis is performed across multiple time intervals, then anomaly detection precision is improved, but processing time increases
Solution Approach 1:
The patent implements periodic action by analyzing event frequencies at regular time intervals (e.g., hourly, daily) rather than continuously processing all data. This periodic sampling approach maintains detection precision for significant anomalies while reducing overall processing time compared to continuous analysis.
Solution Approach 2:
The system performs partial analysis by focusing only on volatile event types with unexpected frequency increases rather than analyzing all event messages in detail. This partial action approach achieves sufficient detection precision for critical anomalies without the excessive processing time required for comprehensive analysis.
Data Source
AI summary
Methods and systems to narrow a search for potential sources of problems in a distributed computing system are described. A volatile event type of event messages recorded in an event-log file is identified. The volatile event type is an event type that may have unexpectedly increased in frequency over an observation time window. An historical period of time may be selected to search for potential sources of the volatile event type. Frequencies of event messages in the event-log file with the same event type as the volatile event type are determined for time intervals of the historical period of time. A time interval of the historical period of time with a largest increase in frequency of event messages is identified. A list of event messages of the event-log file in a selected sub-time interval of the sub-time intervals of the time interval are displayed in a graphical user interface.