Log Analysis Abstraction Level Adjustment via Variable Distribution
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing log analysis techniques face challenges in accurately and efficiently analyzing logs due to the inability to determine optional character strings as similar formats, leading to difficulties in setting the appropriate level of abstraction, which results in inefficient and inaccurate analysis.
Innovation Solution
A log analysis device and method that generates a new abstraction log by replacing variables with log values based on their distribution, allowing for the determination of whether to perform value-to-variable conversion and replacement, thereby adjusting the level of abstraction for efficient analysis.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If the level of abstraction of a format is set high (with high ratio of variables), then the number of log candidates increases, but the desired log gets buried and analysis accuracy decreases
Solution Approach 1:
The patent implements dynamic adjustment of abstraction levels by allowing the system to automatically determine the appropriate level of abstraction for each log format based on the distribution characteristics of log values. The abstraction level is not fixed but can be dynamically optimized to balance between matching coverage and analysis accuracy.
Solution Approach 2:
The system changes the parameter of abstraction level by replacing variables with log values based on their distribution. When a variable shows limited distribution (e.g., appears in only a few different forms), it is replaced with concrete log values, thereby lowering the abstraction level for that specific position and improving analysis accuracy without losing too much adaptability.
2Measurement precision
If the level of abstraction of a format is set low (with low ratio of variables), then analysis accuracy improves, but many patterns are required and analysis efficiency decreases
Solution Approach 1:
The system dynamically adjusts the abstraction level parameter by replacing variables with log values based on distribution analysis. This selective replacement reduces the number of patterns needed while maintaining analysis accuracy, as only positions with limited log value diversity are concretized.
Solution Approach 2:
Instead of completely concretizing all variables (which would require many patterns and reduce efficiency), the system applies partial action by selectively replacing only those variables whose log values show limited distribution. This partial concretization achieves sufficient accuracy without the overhead of fully low-level abstraction.
3Measurement precision
If optional character strings (user name, server name, file name) are treated as different formats, then format matching becomes more specific, but the ability to handle them uniquely as similar formats is lost
Solution Approach 1:
The system changes the treatment of optional character strings by analyzing the distribution of log values associated with variables representing these strings. When such variables show limited distribution (e.g., the same username appears repeatedly), the system replaces the variable with the concrete log value, enabling unique handling of similar optional strings while maintaining format matching capability.
Solution Approach 2:
The handling of optional character strings is made dynamic rather than static. The system automatically determines whether to treat a variable as a general placeholder or replace it with a specific log value based on the observed distribution of log values, allowing flexible adaptation to different log formats and optional string patterns.
Data Source
AI summary
A generation unit that generates a new abstraction log based on the distribution of log values corresponding to a variable in an abstraction log is included. The abstraction log includes at least one or more variables. The variable includes at least one or more log values. The new abstraction log is obtained by replacing the variable in the abstraction log with the log value.