Anomaly Detection Using Time-Series Log Distributions
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing anomaly detection methods fail to accurately detect system anomalies due to variations in log output quantity over time and differences in distribution characteristics across devices, as they rely on averaged distributions that hide unique features of each device and aggregate unit.
Innovation Solution
An anomaly detection method that acquires and utilizes time-series distributions of log output quantities for each device and aggregate unit, selecting a reference distribution similar to the analysis target distribution to enhance detection accuracy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If an average state of logs from multiple devices is used as a reference for anomaly detection, then the reference becomes more generalizable, but the unique distribution features of each device are hidden and detection accuracy decreases
Solution Approach 1:
The patent segments the anomaly detection process by creating separate distribution models for each device and each aggregate unit rather than using a single averaged reference. This segmentation preserves the unique characteristics of each device while still allowing the system to handle multiple devices, resolving the contradiction between generalizability and detection accuracy.
2Quantity of substance
If logs are aggregated over different time ranges, then more data is available for analysis, but the time-varying characteristics of log output quantity are lost
Solution Approach 1:
The patent applies dynamics by creating time-series distributions that capture how log output quantities vary over different time ranges. Instead of using static averaged references, the system generates distributions for each time unit (hourly, daily, etc.) that adapt to the time-varying characteristics of log output, preserving temporal patterns while utilizing available data.
3Device complexity
If a single theoretical distribution is used for anomaly detection, then the detection process is simpler, but it cannot capture the diverse distribution patterns across different devices and time units
Solution Approach 1:
The patent applies local quality by generating specific distribution models tailored to each device and each aggregate unit's characteristics rather than forcing a single theoretical distribution on all data. Each local context (device-time unit combination) gets its own distribution model that reflects its unique patterns, improving adaptability while maintaining manageable complexity through automated model selection.
Data Source
AI summary
The present invention provides an anomaly detection method, an anomaly detection system, and an anomaly detection program that can detect an anomaly at high accuracy by using log output quantity distributions generated for to different aggregate units and different devices. An anomaly detection system according to one example embodiment of the present invention has: a reference distribution, which is a time-series distribution of a log output quantity acquisition unit that acquires a plurality of distributions generated for each device that outputs logs and for each unit of a time range in which logs are aggregated; and an anomaly detection unit that detects an anomaly by using the plurality of distributions.