Distributed Event Counting via Segmented Aggregation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current systems face limitations in processing and counting event messages due to memory capacity constraints, high event rates, and geographical dispersion, which complicates aggregation and analysis.
Innovation Solution
A distributed system architecture that aggregates event attributes across multiple nodes using distinct hash functions for storage and merging, allowing for efficient processing and rule evaluation to detect system conditions and generate reports.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Device complexity
If event counting is performed at a single node, then processing and analysis can be centralized, but memory capacity and processing capacity limitations are encountered
Solution Approach 1:
The patent divides the centralized event counting function into multiple distributed nodes. Each node maintains local counters for event attributes, segmenting the overall counting task across multiple memory spaces. This eliminates the single-node memory capacity bottleneck while maintaining centralized analysis capability through subsequent aggregation of intermediate results.
2Device complexity
If event counting is performed at a single node, then processing can be centralized, but processing capacity is insufficient for high event rates
Solution Approach 1:
The patent segments the event processing workload across multiple distributed nodes, each handling a portion of the event stream independently. This parallel processing architecture enables the system to handle high event rates that would overwhelm a single node, while intermediate results are subsequently aggregated to maintain centralized analysis capability.
3Device complexity
If event aggregation is performed at a single node, then analysis can be centralized, but bandwidth and storage limitations make it infeasible for geographically disparate locations
Solution Approach 1:
The patent implements segmentation by performing event aggregation locally at distributed nodes rather than centralizing all aggregation at a single location. Each node aggregates events from its local region and stores intermediate results, eliminating the need to transmit all raw events across the network. This significantly reduces bandwidth consumption while maintaining the ability to perform centralized analysis through aggregation of intermediate results.
4Device complexity
If event aggregation is performed at a single node, then analysis can be centralized, but storage limitations prevent handling large data volumes
Solution Approach 1:
The patent segments the storage function across multiple distributed nodes, with each node maintaining local storage for aggregated event attributes. This distributes the storage burden away from a single centralized location, enabling the system to handle large data volumes that would exceed the storage capacity of any single node while maintaining centralized analysis capability.
Data Source
AI summary
In a method and system for aggregating event information, events are received at a first plurality of nodes in a distributed system. For the events received at each node aggregated attribute information is determined in accordance with two or more rules and stored in distinct first tables, each table storing aggregated attribute information for a respective rule of the two or more rules. At each node of the first plurality of nodes, the two or more distinct first tables are transmitted to a respective node of a second set of nodes in the distributed system. At each node of the second set of nodes, two or more distinct second tables are generated by merging the aggregated attribute information in the tables transmitted to the node. Each rule of the two or more rules is evaluating using the aggregated attribute information obtained from a corresponding table of the second tables.