Log-Based Anomaly Detection Configuration With Multi-Detector Tuning
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for detecting anomalous operating states in computer systems are resource-intensive and require manual adjustments for each detector, limiting their ability to identify various types of anomalies and leading to high time expenditure and false positives.
Innovation Solution
A method that configures a detection system by creating log records, assigning values to keys, specifying detectors with detection criteria, and adjusting configuration parameters to recognize anomalous states using multiple detector types, including stability and threshold checks to reduce false positives.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If pairwise comparison methods are used to detect anomalous states, then detection capability is achieved, but resource requirements and time expenditure increase significantly
Solution Approach 1:
The patent segments the anomaly detection process into two distinct phases: an offline learning phase where normal behavior patterns are captured and stored as profiles, and an online detection phase where only deviation from these profiles is checked. This segmentation eliminates the need for pairwise comparison of all log lines, reducing computational complexity from O(n²) to O(n), thereby resolving the contradiction between detection reliability and processing efficiency.
2Adaptability or versatility
If multiple detectors of different types are used to cover all anomaly types, then detection coverage improves, but device complexity and configuration difficulty increase
Solution Approach 1:
The patent creates a universal detector framework that can handle multiple anomaly detection tasks through a single configurable component. The detector uses programmable key selection parameters and configurable threshold values that can be adjusted via API calls, allowing the same detector infrastructure to adapt to different anomaly types without requiring separate specialized detectors for each case. This universality reduces device complexity while maintaining comprehensive detection coverage.
Solution Approach 2:
The patent enables flexible adaptation of detector behavior through parameter changes rather than structural modifications. By allowing configuration of key selection parameters, threshold values, and detection criteria through software interfaces, the system can detect different anomaly types by changing parameters rather than deploying multiple complex detector types, thereby reducing overall system complexity while maintaining versatility.
3Measurement precision
If detectors are configured to be sensitive to all possible anomalies, then detection accuracy improves, but false positives increase
Solution Approach 1:
The patent performs preliminary action by capturing and storing the normal operating state profiles during an offline learning phase before actual anomaly detection begins. These profiles establish baseline expectations for normal behavior, allowing the detector to distinguish between genuine anomalies and normal variations. This preliminary characterization of normal states enables the system to be sensitive to anomalies while maintaining low false positive rates, as deviations are measured against pre-established norms rather than requiring overly sensitive real-time thresholds.
Data Source
Figure 1
AI summary
Method for configuring a detection system for characterizing anomalous operating states of a computer system (1), wherein - the computer system (1) creates logs during the operation of the computer system by generating a log record in the form of a log line (P;P1, P2, P3) is created, extracting a number of values from the log line and assigning each of these values to a key, - specifying a number of detectors for the detection system, - optionally selecting several detectors of different types, - specifying a number of normal log lines created during a normal operating state, and - specifying a number of detectors configured, based on their assigned method for adjusting configuration parameters, such that the specified normal log lines or a specified minimum proportion of these specified log lines meet the detection criterion.;