Hybrid Log Viewer Memory Management via Placeholder Detection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In distributed computing systems, managing log messages from multiple nodes to ensure timely processing and prevent disruptions due to paused log sources, existing methods fail to efficiently handle placeholder messages and maintain message order.
Innovation Solution
A method is implemented where a log filter list is stored in memory, identifying log filters executed across nodes, and log messages are processed by removing the least recent message, detecting placeholder messages from paused sources, and discarding them, while selecting non-placeholder messages for further processing.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If log messages are collected from multiple nodes in real-time, then processing timeliness is improved, but memory usage increases
Solution Approach 1:
The patent extracts and removes the least recent log message from the data structure when memory capacity is reached, ensuring that only necessary log messages are retained in memory at any given time. This extraction principle directly addresses the memory usage issue while maintaining real-time processing capability.
Solution Approach 2:
The patent changes the state of log messages by identifying and removing those with the least recent timestamps. By dynamically adjusting which messages are retained based on timestamp parameters, the system optimizes memory usage while preserving the most relevant log data for real-time analysis.
2Stability of the object's composition
If placeholder messages from paused log sources are processed, then message order is maintained, but processing efficiency decreases
Solution Approach 1:
The patent extracts and removes placeholder messages from the data structure upon detection, preventing them from consuming processing resources. This extraction approach maintains message order for valid logs while eliminating the efficiency penalty associated with processing placeholder entries.
Solution Approach 2:
The patent discards placeholder messages that are generated by paused log sources, recovering processing efficiency by preventing these non-informative messages from consuming system resources. This discarding principle allows the system to maintain stability for valid messages while improving overall productivity.
3Loss of information
If all log messages are retained in the data structure, then data completeness is improved, but memory consumption increases
Solution Approach 1:
The patent systematically extracts and removes the least recent log message from the data structure when memory capacity is reached, ensuring that data completeness is maintained for the most relevant logs while preventing unbounded memory growth. This extraction mechanism balances information retention with resource constraints.
Solution Approach 2:
The patent implements periodic removal of least recent messages based on timestamp comparison, creating a cyclical process that continuously maintains an optimal balance between data completeness and memory consumption. This periodic action ensures the system adapts to changing memory conditions while preserving essential log data.
Data Source
AI summary
A method is disclosed, comprising: storing, in a memory of a first computing device, a log filter list that identifiers a plurality of log filters, each of the plurality of log filters being executed on a respective one of a plurality of nodes in a distributed computing system; removing a first log message from a data structure that is stored in the memory of the first computing device, the first log message being obtained by the first computing device from a first one of the plurality of log filters that is associated with a first log source; detecting whether the first log message includes a placeholder message that is generated by the first log filter while the first log source is in a paused state; and selecting the first log message for further processing, when the first log message does not include a placeholder message.