Log Event Relevance Detection via Deep Learning
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Information Technology companies face challenges in delivering high availability and rapid recovery of computing devices due to inefficient event logging, especially in systems with limited non-volatile storage, where fixed alert levels obscure the root cause of failures and lead to unnecessary event logging, consuming valuable storage space.
Innovation Solution
Implementing a deep learning approach using Recurrent Neural Networks to dynamically determine the relevance of log events for storage, prioritizing events based on context and available buffer space, and ignoring irrelevant events to optimize storage usage and diagnose root cause errors.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If fixed alert levels are used for event logging, then event classification is simplified, but root cause diagnosis precision deteriorates due to irrelevant events being logged
Solution Approach 1:
The patent transforms the static fixed alert level classification into a dynamic context-aware classification system. The deep learning model continuously analyzes event sequences and adjusts classification based on current system state and historical context, allowing the logging behavior to adapt dynamically to different failure scenarios while maintaining diagnostic precision.
Solution Approach 2:
The system changes the parameters used for event classification from simple fixed alert levels to complex multi-dimensional features including event sequences, system state, and contextual information. This parameter transformation enables the deep learning model to distinguish relevant from irrelevant events effectively.
2Loss of information
If all events are logged to ensure comprehensive diagnosis information, then diagnostic completeness is improved, but storage space is consumed excessively leading to key events being overwritten
Solution Approach 1:
The deep learning model extracts only the essential and relevant events from the complete event stream for storage in limited non-volatile memory. By identifying and separating key diagnostic events from irrelevant ones through contextual analysis, the system preserves critical information while minimizing storage consumption.
Solution Approach 2:
The patent introduces a buffer as an intermediary between the event stream and non-volatile storage. The deep learning model processes events through this buffer, allowing for temporary storage and intelligent selection of which events to persist to non-volatile memory, thereby preventing key events from being overwritten while managing limited storage resources.
3Measurement precision
If context-aware dynamic logging is implemented, then root cause diagnosis precision is improved, but system complexity increases due to deep learning model requirements
Solution Approach 1:
The deep learning model operates autonomously to perform event classification and selection without requiring complex external control systems. The model self-manages the logging decisions based on patterns learned from training data, reducing the need for additional system complexity while maintaining high diagnostic precision.
4Productivity
If deep learning model is used for event classification, then event logging efficiency is improved, but computational resources are consumed increased
Solution Approach 1:
The deep learning model is trained offline in advance on comprehensive failure scenarios and event patterns. This preliminary training allows the model to be deployed with pre-learned knowledge, reducing the computational burden during actual operation while maintaining high logging efficiency and accurate event classification.
Data Source
AI summary
Examples disclosed herein relate to log events for root cause error diagnosis. A deep learning model including parameters that are trained to identify relevance of log event information to diagnose root cause errors associated with a computing device is used. In one example, log event information is received. The log event information is processed according to the deep learning model to determine, for each log event, whether to store the log event information in a buffer based on relevance of the log event information to diagnosis of root cause errors.