Logical Partition Mapping for Event Data Queue Load Balancing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The uneven distribution of event data across multiple queues leads to bottlenecks and delayed reading of event data, which can result in delayed detection of malicious events and inefficient operations of reader services.
Innovation Solution
A processor determines logical partitions for event data based on partition keys and stores the data in multiple queues. The processor updates the mapping of logical partitions to queues based on load balances, transferring logical partitions between queues to distribute the load more evenly.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If event data is stored in multiple queues based on user identifiers, then data organization and retrieval efficiency are improved, but load distribution becomes uneven causing bottlenecks
Solution Approach 1:
The patent segments event data into different logical partitions based on user identifiers and maps these partitions to multiple queues. This segmentation allows efficient retrieval of user-specific data while distributing the load across multiple queues to prevent bottlenecks, directly resolving the contradiction between retrieval efficiency and load distribution.
Solution Approach 2:
The patent dynamically adjusts the mapping parameters between logical partitions and queues based on load conditions. By changing the mapping configuration in response to system state, the patent maintains both efficient retrieval performance and balanced load distribution across queues.
2Speed
If all event data for a user is stored in a single queue, then data retrieval for that user is efficient, but reader services experience delays when that queue has high load
Solution Approach 1:
The patent introduces a new dimension of organization by creating logical partitions that map to multiple queues rather than a single queue. This dimensional change allows user data to be accessed across multiple queues simultaneously, maintaining fast retrieval speed while reducing delays caused by single-queue bottlenecks.
3Ease of operation
If event data is partitioned by user identifier, then user-specific data access is optimized, but malicious events from single users can overload specific queues
Solution Approach 1:
The patent segments user data across multiple queues through logical partitions, so that even if a single user generates malicious events, the load is distributed across multiple queues rather than concentrated in one. This maintains easy user-specific data access while preventing queue overload from malicious activities.
Data Source
AI summary
According to examples, an apparatus may include a processor and a memory on which are stored machine-readable instructions that, when executed by the processor, may cause the processor to receive event data to be placed in a plurality of queues. The processor may determine logical partitions for the received event data and may store the event data in the plurality of queues based on the determined logical partitions. A first queue of the plurality of queues may be mapped to a first logical partition range and a second queue of the plurality of queues may be mapped to a second logical partition range. Based on respective loads at the plurality of queues, the processor may update a mapping of the determined logical partitions to the plurality of queues by transferring logical partitions between the first queue and the second queue.