Event Fan-Out Queue for High-Volume Subscription Delivery
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional request-response and publish-subscribe models are inefficient for delivering event content to multiple users, leading to increased computing resource usage and network activity, particularly in messaging platforms where real-time interactions are required, such as alerting users about ongoing conversations.
Innovation Solution
A system that initiates a subscription session for users to topics, utilizing a channel server to send events directly to clients without continuous requests, and employs an event fan-out queue to distribute events efficiently among multiple channel servers hosting subscriptions, with features like session timers to manage idle clients and conserve resources.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If a conventional publish-subscribe model is used where all channel servers receive all events, then all subscribers can be served, but excessive writing to individual event queues occurs when many channel servers host subscriptions to the same topic
Solution Approach 1:
The system implements a threshold-based mechanism where events are selectively written to fan-out queues only when the number of channel servers hosting subscriptions to a topic exceeds a predetermined threshold. This partial action approach avoids the excessive writing to all event queues when the number of channel servers is below the threshold, thereby reducing computing resource usage while ensuring proper event distribution when scaling occurs.
2Reliability
If events are written to all channel server event queues regardless of subscription count, then all subscribers receive events, but redundant writes occur when few channel servers are involved
Solution Approach 1:
The system introduces an intermediary mechanism (the threshold-based fan-out queue system) that sits between the event publisher and channel servers. This intermediary intelligently routes events only when necessary (when channel server count exceeds threshold), reducing redundant writes while maintaining complete event delivery to all subscribed channel servers through the fan-out queue mechanism.
3Adaptability or versatility
If session timers are not used, then all subscriptions are maintained, but resources are consumed by idle clients
Solution Approach 1:
The system implements periodic session timers that are reset upon receiving events or user activity. When a session timer expires without being reset, the subscription is terminated. This periodic action mechanism maintains flexibility in subscription management while conserving resources by automatically cleaning up idle client subscriptions, balancing adaptability with resource efficiency.
Data Source
AI summary
Methods, systems, and apparatus, including computer programs encoded on computer storage media, for delivering event content. One of the methods includes initiating, for one or more client devices, a subscription to a topic; receiving an event relevant to a topic; determining a number of channel servers that correspond to the topic, wherein each channel server is associated with one or more client devices; determining that the number of channel servers that correspond to the topic is greater than a threshold number of channel servers; writing the received event from the event processor to an event fan-out queue, in response to the determination that the number of channel servers that correspond to the topic is greater than the threshold number of channel servers; reading the received event from the event fan-out queue; and sending the received event to the one or more client devices associated with each respective channel server.