Dynamic Log Message Prioritization Under Storage Contention
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing logging systems lack an efficient and dynamic mechanism to manage log messages based on their importance and the system's capacity, leading to potential storage bottlenecks and reduced system throughput due to contentions and resource limitations.
Innovation Solution
A logging manager system that evaluates log messages and the logging system using multiple factors, including data source, age, queries, redundancy, size, and trends, to dynamically decide on the storage or discard of log messages, optimizing storage based on the current system load and message importance.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If all log messages are stored in the logging volume, then complete logging coverage is achieved, but storage capacity is exceeded and system throughput decreases due to contentions
Solution Approach 1:
The patent applies local quality by evaluating each log message individually based on its specific characteristics (importance, redundancy, age, size) and making storage decisions on a per-message basis. This allows the system to treat different log messages differently, storing only those that meet the importance threshold while discarding less critical ones, thereby maintaining logging coverage for important events while reducing overall storage pressure and improving system throughput.
2Productivity
If log messages are filtered to reduce storage volume, then system throughput improves, but important log messages may be discarded
Solution Approach 1:
The patent employs parameter changes by dynamically adjusting the importance threshold based on the current state of the logging system (storage capacity, system load, contention levels). This allows the filtering criteria to adapt to changing conditions, ensuring that when storage capacity is available, more messages are retained, and when capacity is constrained, less important messages are filtered out. This dynamic parameter adjustment maintains system throughput while preserving important log messages.
3Productivity
If multiple evaluation factors are considered for each log message, then storage optimization improves, but system complexity increases
Solution Approach 1:
The patent applies segmentation by breaking down the evaluation process into distinct, independent factors (importance, redundancy, age, size) that can be assessed separately for each log message. This modular approach to evaluation allows the system to consider multiple dimensions of log message quality without creating a monolithic complex system, as each factor can be computed and weighted independently, simplifying the overall evaluation architecture while achieving comprehensive storage optimization.
Data Source
AI summary
Methods, computer program products, and systems are presented. The method computer program products, and systems can include, for instance: generating log messages from one or more data source; evaluating one or more log message produced from the generating, and outputting, in dependence on the evaluating the one or more log message, a log message rating of the one or more log message; evaluating a logging system, and outputting, in dependence on the evaluating the logging system, a logging system rating of the logging system, wherein the logging system includes a logging volume for storing log messages; comparing the log message rating to the logging system rating; and producing an action decision impacting a count of log messages stored in the storage volume in dependence on the comparing of the log message rating, and the logging system rating.