Hash-Based Event Merging for Pipeline Throughput
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Event processing systems face inefficiencies in handling large volumes of computing events, as traditional methods like increasing hardware resources do not significantly improve performance, leading to difficulties in accommodating sudden increases in event processing loads.
Innovation Solution
The system identifies and merges equivalent computing events in an execution pipeline using a hash function to eliminate duplicate processing, thereby reducing the event load without requiring additional resources, by selecting a representative event from each group and removing the others from the pipeline.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If the number of machines is increased to share the processing burden, then the system can accommodate more computing events, but the performance of a single processing machine does not improve and additional hardware is required
Solution Approach 1:
The patent merges equivalent computing events into a single representative event using hash-based grouping. By identifying events with identical hash values and consolidating them, the system reduces the total number of events that need processing, thereby increasing effective processing capacity without adding hardware resources.
Solution Approach 2:
The patent changes the parameter of event representation by selecting a representative event from each equivalence group. Instead of processing multiple identical events, the system transforms the event set into a reduced set where each group is represented by one event, changing the density and distribution parameters of events in the execution pipeline.
2Reliability
If duplicate computing events are processed individually, then all events are handled, but processing efficiency decreases due to redundant operations
Solution Approach 1:
The patent merges equivalent computing events into a single representative event while preserving the completeness of processing. By using hash-based identification and grouping, the system ensures that all equivalent events are captured and represented, maintaining processing reliability while eliminating redundant operations through consolidation.
Solution Approach 2:
The patent creates a representative copy of each group of equivalent events. Instead of processing multiple identical events, the system generates a single representative event that captures the essential processing logic, thereby maintaining the effect of processing all events while significantly reducing the actual number of processing operations.
3Reliability
If the execution pipeline processes all computing events, then no events are lost, but the pipeline becomes congested with duplicate events
Solution Approach 1:
The patent extracts duplicate events from the execution pipeline by identifying equivalent events through hash comparison and removing redundant entries. By taking out duplicate events and retaining only representative events from each equivalence group, the system reduces pipeline congestion while maintaining processing completeness through the representative events that remain.
Solution Approach 2:
The patent changes the density parameter of events in the execution pipeline by consolidating multiple equivalent events into single representatives. This parameter change reduces the total event count and improves pipeline throughput while maintaining the essential processing coverage through strategic selection of representative events.
Data Source
AI summary
Systems and methods of increasing the efficiency of an event processing system are disclosed. In some example embodiments, a computer-implemented method comprises identifying a first group of computing events from a plurality of computing events in an execution pipeline based on a similarity between the computing events of the first group, and merging the computing events of the first group into a single representative computing event of the first group in the execution pipeline. In some example embodiments, the identifying of the first group of computing events comprises calculating a corresponding identifier using a hash function on at least one attribute of the corresponding computing event for each one of the plurality of computing events in the execution pipeline, and determining that the computing events of the first group have corresponding identifiers that match each other.