Log File Aggregation with Size Thresholds
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The increasing number of log files from client computing devices poses challenges in efficiently managing and storing them at data centers, making it difficult to quickly locate and manage the multitude of log files.
Innovation Solution
The method involves aggregating multiple log files into a single aggregated log file, setting a size limit for the aggregated file, and transmitting it when the limit is reached, while also filtering out log files that exceed predetermined size or frequency thresholds.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If multiple individual log files are stored separately at data centers, then complete activity records are preserved, but file management complexity and storage overhead increase significantly
Solution Approach 1:
The patent merges multiple individual log files into a single consolidated log file that contains activity records from multiple client devices. This consolidation reduces the number of separate files that need to be managed at data centers while preserving the complete activity records. The consolidated log file is transmitted as a single entity, simplifying storage and management operations.
2Loss of information
If all log files are transmitted to data centers, then complete activity data is available for analysis, but transmission time and network bandwidth consumption increase
Solution Approach 1:
The patent combines multiple log files into a single consolidated log file before transmission, which reduces the total number of transmission operations required. This merging approach maintains complete activity data while decreasing transmission time by sending one consolidated file instead of multiple separate files.
Solution Approach 2:
The system performs preliminary aggregation of log files at the client device before transmission to the data center. By consolidating logs in advance, the system reduces the transmission burden and network bandwidth consumption while ensuring all activity records are captured and transmitted together.
3Productivity
If log files are aggregated into fewer files, then storage and management efficiency improve, but file size increases which may impact transmission and processing
Solution Approach 1:
The patent segments the consolidated log file into multiple partitions or chunks based on size thresholds. Each partition contains a portion of the aggregated log data, allowing the system to manage large volumes of log information in smaller, more manageable units. This segmentation maintains storage efficiency while avoiding the drawbacks of excessively large single files.
Data Source
AI summary
Methods, devices and computer program products facilitate the storage, access and management of log files that are associated with particular client devices. The log files provide a record of user or client device activities that are periodically sent to a data backup center. A dedicated log file server facilitates the processing and storage of an increasingly large number of log files that are generated by new and existing client devices. A storage server pre-processes the received log files to facilitate the processing and storage of the log files by the log file server. This Abstract is provided for the sole purpose of complying with the Abstract requirement rules. This Abstract is submitted with the explicit understanding that it will not be used to interpret or to limit the scope or the meaning of the claims.