Sharded In-Memory Timer Queues for Low-Latency Expiration Processing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing techniques for managing user-specific timers at scale face challenges with high latency and network requests, requiring complex and error-prone solutions for handling large numbers of timers, especially in distributed systems.
Innovation Solution
A distributed in-memory timer service that supports multi-tenancy, using a durable in-memory database and append-only logs to efficiently manage timers, ensuring fault tolerance and minimizing duplicate sends, with unique consumers handling sharded time buckets to process expirations even during outages.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If existing techniques for storing and processing unique per user timers are used, then timer functionality is achieved, but large amounts of network requests and high storage IOPs are required
Solution Approach 1:
The system segments timers into sharded time buckets organized by time window (e.g., minute, hour) and shard index. This segmentation allows the system to process only relevant timer batches instead of scanning all timers, reducing network requests and storage IOPs while maintaining high processing throughput.
Solution Approach 2:
The system performs preliminary organization of timers into sharded time buckets during timer creation, pre-partitioning the data structure before expiration events occur. This preliminary structuring enables efficient batch processing at expiration time without requiring full system scans, reducing real-time network and I/O overhead.
2Quantity of substance
If existing techniques for timers at scale are used, then timer storage is achieved, but high storage IOPs are required
Solution Approach 1:
The timer storage system is segmented into sharded time buckets distributed across multiple storage units. Each bucket handles a specific time window and shard range, allowing parallel storage operations and reducing the IOP burden on any single storage unit while supporting large quantities of timers.
Solution Approach 2:
Timers are preliminarily organized into sharded time buckets at creation time, establishing a pre-partitioned storage structure. This preliminary organization enables efficient batch storage operations and reduces real-time IOP requirements by avoiding full-system scans and allowing targeted writes to specific shards.
3Adaptability or versatility
If distributed timer processing is implemented, then scalability is improved, but system complexity increases
Solution Approach 1:
The distributed system uses sharded time buckets that partition timer data across multiple nodes based on time windows and shard indices. This segmentation enables horizontal scalability by allowing individual nodes to be added or removed without affecting the entire system, while the consistent sharding logic maintains processing correctness across the distributed architecture.
Solution Approach 2:
The sharded time bucket structure serves multiple functions simultaneously: it provides data partitioning for scalability, enables efficient expiration processing by time window, supports sharding for load distribution, and maintains consistency across distributed nodes. This multi-functionality reduces overall system complexity by consolidating multiple concerns into a single unified structure.
Data Source
AI summary
Disclosed are systems and methods for implementing distributed in-memory timer queues. A plurality of queues is distributed across a plurality of nodes. The plurality of queues includes a first queue. The first queue corresponds to current timer expirations for a tenant, time bucket, and shard. A sorted set includes minute references to unacknowledged timers for a tenant and shard. A plurality of processes is created. Each of the plurality of processes is configured to be a unique consumer of the distributed plurality of queues. A first consumer group and a second consumer group are created within each of the plurality of processes. The first consumer group corresponds to the first queue. The second consumer group corresponds to the sorted set. The plurality of processes is used to proportionally consume the first queue with the first consumer group and the sorted set with the second consumer group.