Log Variation Analysis for Early Anomaly Detection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing log analysis techniques require prior knowledge and experience to identify meaningful log information, and they fail to detect advance signs of failures or changes in log generation effectively, especially in complex and massive log data sets.
Innovation Solution
A monitoring device information analyzing device and method that calculates log variation amounts before and after a specific event, using a log variation amount calculation unit and verification unit, to extract advance logs without requiring prior knowledge of log message meanings, by referencing past log messages and statistical methods.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If prior knowledge and experience are used to identify meaningful log information, then analysis accuracy is improved, but the complexity of operation increases and requires specialized expertise
Solution Approach 1:
The system performs self-service by automatically analyzing log data without requiring user expertise. The log analysis unit autonomously identifies meaningful patterns and generates analysis results, eliminating the need for operators to have specialized knowledge of log message meanings or system internals.
Solution Approach 2:
The patent replaces the mechanical approach of manual log analysis (requiring human expertise) with an automated information processing system. The log analysis unit uses computational methods to substitute human analytical capabilities, transforming the manual process into an automated information processing task.
2Reliability
If all log information is monitored on a daily basis, then completeness of monitoring is improved, but the time and resources required increase significantly
Solution Approach 1:
The system extracts only the essential and meaningful log information from the vast amount of daily log data. The log analysis unit identifies and extracts significant patterns, events, and anomalies, separating useful information from the overwhelming volume of routine log entries that would otherwise require full monitoring attention.
Solution Approach 2:
The patent changes the monitoring parameter from comprehensive daily review of all logs to targeted analysis based on significance thresholds and patterns. By changing how logs are analyzed (from quantity-based to quality-based monitoring), the system maintains reliability while reducing time consumption.
3Loss of information
If log data is collected and indexed without selective analysis, then data availability is improved, but the ability to find meaningful information decreases
Solution Approach 1:
The system performs preliminary action by pre-analyzing and categorizing log data into meaningful groups and patterns. The log analysis unit prepares the data in advance by identifying significant events, trends, and anomalies, so that when queries are made, the information is already organized and ready for efficient retrieval rather than requiring exhaustive searching.
Data Source
AI summary
A monitoring device information analyzing device calculates a log variation amount from a log variation amount of a monitoring target device h and a log variation amount of the monitoring target device on another day, by referring to a template information storage unit storing a time at which a log message arises, the log message arising in the past, a template that is associated with an ID of the monitoring target device, and a template ID of the template to calculate the log variation amount of the monitoring target device and to obtain the log variation amount of the monitoring target device on the other day at the same time zone; obtains the log variation amount that is to be a sample; executes verification according to a statistical method, and determines the sample as an advance log, if a test statistic is greater than a predetermined threshold value.