Distributed Timer Service with Sharded Buckets for Low-Latency Scale
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing techniques for managing large-scale timers face challenges in achieving low latency and high scalability due to high network requests and storage IOPs, leading to complex and error-prone solutions for handling billions of active timers.
Innovation Solution
A distributed timer service with a wait condition primitive supports multi-tenancy, using a durable in-memory database and append-only logs to efficiently manage timers, ensuring at-least once delivery and fault tolerance, with features like sharded time buckets and consumer groups for even distribution and recovery from failures.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If existing techniques for storing and processing unique per user timers are used, then timer functionality can be provided, but network requests and storage IOPs increase significantly
Solution Approach 1:
The patent segments the timer service into multiple consumer groups that process timers in parallel batches. Instead of handling all timers through a single sequential process, the system divides timer processing across multiple consumers, each handling a portion of the timer workload. This segmentation reduces the IOP burden on storage systems and distributes network request handling across multiple service instances, thereby improving overall efficiency while maintaining reliable timer delivery.
2Reliability
If existing techniques for storing and processing unique per user timers are used, then timer functionality can be provided, but storage IOPs increase significantly
Solution Approach 1:
The patent segments the timer service into multiple consumer groups that process timers in parallel batches. Instead of handling all timers through a single sequential process, the system divides timer processing across multiple consumers, each handling a portion of the timer workload. This segmentation reduces the IOP burden on storage systems and distributes network request handling across multiple service instances, thereby improving overall efficiency while maintaining reliable timer delivery.
3Productivity
If high scale timer processing is implemented, then more timers can be handled, but latency increases
Solution Approach 1:
The patent implements preliminary action by pre-positioning timer data in distributed caches before the actual timer firing time. Consumer groups proactively load and prepare timer batches in advance, so when timers need to be processed, the data is already available in memory rather than requiring real-time storage access. This preliminary preparation reduces latency during high-scale timer processing while maintaining the ability to handle large volumes of timers.
Data Source
AI summary
Disclosed are systems and methods for implementing timers as a service. A request to create a timer is received. The request corresponds to a wait state condition associated with an event in a data stream. The request specifies an endpoint for a payload associated with an expiration of the timer. The timer is created based on the wait state condition. The creating of the timer includes publishing information pertaining to the timer in a database. The timer is consumed. The consuming of the timer includes processing the information pertaining to the timer. Based on the expiration of the timer, the payload associated with the expiration of the timer is sent to the specified endpoint.