Abnormal Log Detection with Dynamic Diagnostic Collection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing log systems fail to provide sufficient information for developers to diagnose issues due to the overwhelming volume of DEBUG messages and the disregard of INFO messages, leading to inefficient problem-solving in software systems.
Innovation Solution
A computer-implemented method that predicts time intervals using trained models to identify suspected abnormal logs, collects additional diagnostic information during abnormal log periods, and returns to normal collection rates when the log is no longer abnormal, utilizing models like regression and sequence models to enhance accuracy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If DEBUG messages are enabled to provide detailed diagnostic information, then measurement precision is improved, but productivity deteriorates due to overwhelming volume of messages impacting system performance
Solution Approach 1:
The system dynamically adjusts the log collection rate based on the abnormal log detection state. When an abnormal log is detected, the collection rate increases to capture detailed diagnostic information. When no abnormal logs are present, the collection rate decreases to minimal levels, thereby maintaining system performance while ensuring diagnostic quality when needed.
Solution Approach 2:
The patent changes the parameter of log collection rate from a static fixed value to a dynamic variable that adjusts based on system state. The collection rate is modified according to whether abnormal logs are detected, allowing the system to optimize between diagnostic precision and performance by varying this parameter in response to conditions.
2Loss of information
If INFO messages are collected to provide comprehensive system operation records, then loss of information is reduced, but productivity deteriorates due to processing overhead of voluminous messages
Solution Approach 1:
Instead of continuously collecting all INFO messages (excessive action), the system collects INFO messages partially and selectively - only when abnormal logs are detected. This partial action approach ensures that critical information is captured without the continuous processing overhead of collecting all INFO messages during normal operation.
Solution Approach 2:
The system implements periodic adjustment of the collection rate based on detection of abnormal logs. During normal operation, INFO message collection occurs at minimal intervals. When abnormal logs are detected, the collection frequency increases periodically to capture relevant information, then returns to minimal frequency after the issue is resolved.
3Reliability
If additional diagnostic information is collected continuously to ensure rapid problem detection, then reliability is improved, but loss of energy deteriorates due to continuous high-rate information collection
Solution Approach 1:
The system performs preliminary detection of abnormal logs using a first collection rate before initiating intensive diagnostic information collection. This preliminary action allows the system to identify potential issues early and only then activate the higher energy-consuming detailed collection mode, ensuring reliability while minimizing unnecessary energy consumption during normal operation.
Solution Approach 2:
The system uses feedback from abnormal log detection to control the diagnostic information collection rate. When abnormal logs are detected, the feedback triggers increased collection rate to ensure reliable problem detection. When no abnormal logs are present, the feedback reduces the collection rate to minimal levels, optimizing energy usage while maintaining detection capability when needed.
Data Source
AI summary
An approach for collecting diagnostic information, associated with a software system, based on identifying an abnormal log. The approach predicts a first time interval associated with a current log message based on one or more trained models and a first previous log message. The approach determines if a log is a suspected abnormal log based on the time interval. The approach, responsive to the log being a suspected abnormal log, executes actions of collecting additional diagnostic information based on a baseline diagnostic information collection rate, determines if the log is no longer a suspected abnormal log based at least on a second time interval. The approach, responsive to the log no longer being a suspected abnormal log, returns to the baseline diagnostic information collection rate.