Multi-threaded Log Storage with Thread Reference Cache
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Storing large volumes of multi-threaded log data consumes significant computing resources, with cloud repositories being particularly expensive and resource-intensive, and existing solutions do not efficiently manage relevant versus irrelevant data.
Innovation Solution
A method and system that categorize multi-threaded log data using markers and thread identifiers, where logs are stored in a map data structure with a thread reference cache, log caches, and repositories based on relevance, with mechanisms to communicate relevant data to high-performance repositories and less relevant data to lower-performance repositories, and manage cache sizes and retention based on time and size limits.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If all multi-threaded log data is stored in cloud repositories, then data security and accessibility are improved, but computing resources and costs increase significantly
Solution Approach 1:
The patent segments log data into different categories using markers and thread identifiers, storing frequently accessed logs in local cache and less frequently accessed logs in cloud repository. This segmentation reduces the volume of data stored in expensive cloud infrastructure while maintaining security for critical data.
Solution Approach 2:
The patent implements local quality by creating thread-specific log caches that store logs locally for frequently accessed threads, while only storing logs for less frequently accessed threads in the cloud. This ensures high-performance access for critical operations while reducing overall cloud storage requirements.
2Duration of action of stationary object
If large volumes of log data are stored in cloud repositories, then data retention is improved, but resource consumption and costs increase
Solution Approach 1:
The patent extracts only the necessary log data from multi-threaded operations, using markers to identify and extract only relevant logs based on category and thread identifier. This extraction approach ensures adequate data retention for analysis while minimizing the total volume of data that requires expensive cloud storage and associated power consumption.
3Loss of information
If all log data is retained for comprehensive analysis, then analytical completeness is improved, but processing time and resource usage increase
Solution Approach 1:
The patent performs preliminary action by pre-categorizing logs with markers and thread identifiers during data generation, and pre-storing frequently accessed logs in local cache. This preliminary organization enables rapid retrieval and analysis of relevant logs without requiring processing of entire log datasets, significantly reducing processing time while maintaining analytical completeness for targeted investigations.
Data Source
Figure 1A
Figure 1B
Figure 1C
AI summary
Systems, methods, and computer program products are provided for efficiently storing multi-threaded log data. A method includes receiving multi-threaded log data comprising logs, markers, and thread identifiers. For each respective log, the respective thread identifier is set as a most recently used item in a thread reference cache. A respective log cache in a map data structure is determined based on the respective thread identifier. The respective log is added to the respective log cache. Whether to communicate the respective log and/or the respective log cache to a first repository is determined based on the respective marker. The respective log is communicated to a second repository. Whether to remove an oldest log from the log cache is determined based on a log cache size limit and/or a time limit. Whether to remove a least recently used log cache is determined based on at least one map data structure size limit.