Reactive Log Spooling for Storage Optimization
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Information handling systems face challenges in managing and storing large volumes of log messages, which include irrelevant data, leading to increased bandwidth, computational resources, and storage space requirements, making it difficult to track production issues efficiently.
Innovation Solution
An information handling system that stores logs in a volatile buffer and monitors them in real-time to detect triggering events, storing debug-level logs in non-volatile storage while discarding or persisting logs based on predetermined criteria, thereby optimizing resource usage and reducing noise.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of information
If all log messages are stored in non-volatile storage, then complete log data is preserved for analysis, but storage space requirements increase significantly
Solution Approach 1:
The patent extracts only the relevant portion of log data (debug-level logs during triggering events) from the complete log stream and stores only that in non-volatile storage. The irrelevant logs are discarded, thereby resolving the contradiction between preserving complete log data and minimizing storage space requirements.
Solution Approach 2:
The patent applies different quality standards to different portions of log data: debug-level logs during triggering events are preserved with high fidelity in non-volatile storage, while other logs are discarded. This local differentiation of data quality resolves the contradiction by preserving only the locally relevant information.
2Loss of information
If debug-level logs are stored during all operations, then complete troubleshooting information is available, but bandwidth and computational resources are wasted
Solution Approach 1:
The system performs preliminary action by storing debug-level logs in a volatile buffer before they are needed, during normal operations when they are not required. When a triggering event occurs, the pre-stored logs are immediately available for analysis. This resolves the contradiction by preparing data in advance without continuously consuming resources.
Solution Approach 2:
The patent implements periodic action by storing debug-level logs only during specific periods (triggering events) rather than continuously. The system monitors for triggering events and stores logs periodically when these events occur, thereby reducing bandwidth and computational resource consumption while ensuring troubleshooting information is captured when needed.
3Loss of information
If logs are stored in real-time without filtering, then all production issues are captured, but noise from irrelevant data increases
Solution Approach 1:
The patent extracts only the relevant log messages associated with triggering events from the complete log stream. By taking out only the pertinent information and discarding irrelevant logs, the system resolves the contradiction between capturing all production issues and reducing data noise.
Solution Approach 2:
The patent applies different quality standards to different log messages: debug-level logs during triggering events are preserved with high fidelity, while other logs are discarded. This local differentiation ensures that production issues are tracked with high quality while minimizing noise from irrelevant data.
4Loss of information
If volatile buffer retains logs for extended periods, then more log data is available for analysis, but memory resources are consumed
Solution Approach 1:
The system performs preliminary action by storing log data in the volatile buffer before it is needed, during normal operations when memory resources are available. When a triggering event occurs, the pre-stored logs are immediately available for analysis without requiring extended retention periods, thereby resolving the contradiction between data availability and memory resource consumption.
Solution Approach 2:
The patent implements periodic action by retaining logs in the volatile buffer only for specific periods (until a triggering event occurs or the buffer is full) rather than continuously. This periodic retention strategy ensures log data is available when needed while minimizing memory resource consumption during normal operations.
Data Source
AI summary
An information handling system stores logs in a volatile log buffer for a pre-determined time, and monitors the logs in the log buffer in real-time. The system parses the logs to detect a triggering event, wherein the triggering event is one of an error event, a fatal event, or a critical event. In response to detecting the triggering event, the system stores debug level logs in a non-volatile storage device. If the predetermined time is reached, then the system determines whether to discard the logs or to store the logs in the non-volatile storage device. In response to a determination to discard the logs, the logs in the log buffer may be discarded.