Large Event Log Replay via MapReduce Segmentation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing log replay technologies face deviations and bottlenecks when processing large event logs, leading to inefficiencies in processing speed and operation performance.
Innovation Solution
A method and system that divide large event logs into smaller partitions using a distributed application program interface (API) web service and the MapReduce algorithm, enabling faster processing and distribution of event logs for improved replay performance.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If large event logs are processed using traditional log replay technology, then complete event log processing is achieved, but processing speed decreases and bottlenecks occur
Solution Approach 1:
The patent divides a large event log into multiple smaller partitions or segments. Each partition is processed independently and in parallel, which eliminates bottlenecks and significantly improves processing speed. The segmentation allows the system to handle large volumes of event logs by breaking them down into manageable chunks that can be distributed across multiple processing units.
2Productivity
If distributed API web service with MapReduce algorithm is used to divide event logs, then processing efficiency improves, but system complexity increases
Solution Approach 1:
The patent employs a distributed API web service that implements the MapReduce algorithm, providing a universal framework for processing event logs. This multi-functional system handles both the division of event logs into partitions and the coordination of parallel processing, reducing the need for separate specialized components and managing complexity through a unified approach.
3Speed
If event logs are divided into small partitions, then processing bottlenecks are eliminated, but data management complexity increases
Solution Approach 1:
The patent introduces a distributed API web service as an intermediary layer between the event log storage and the processing units. This mediator handles the complexity of dividing event logs into partitions, managing the distribution of these partitions to multiple processing units, and coordinating the aggregation of results, thereby simplifying the overall data management while maintaining high processing speeds.
Data Source
AI summary
A large event log replay method and system are provided. The large event log replay method includes counting event logs generated in a process before a next process performed subsequent to the process and maintaining a log file that records the event logs in a storage system, verifying a log file corresponding to a process included in an access command for a log replay page from the storage system in response to the access command being generated, and generating a plurality of split log files by dividing the verified log file into a set size and acquiring the plurality of split log files from the storage system.