Cloud Event Update Scalability via Segmentation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing cloud-based collaboration platforms face scalability bottlenecks when updating multiple users due to the difficulty in storing event entries within a designated time frame, especially as the number of collaborators increases, leading to potential failures in updating clients with events such as renaming or editing actions.
Innovation Solution
The system divides event updates into smaller chunks, allowing each chunk to be stored within a designated time frame, and incorporates mechanisms to retry only failed chunks, ensuring successful updates and maintaining scalability, robustness, and availability for large numbers of collaborators.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If the system updates all collaborators with event entries simultaneously, then all clients receive timely notifications, but the system fails to store event entries within the designated time frame when the number of collaborators is large
Solution Approach 1:
The system divides the list of event entries into multiple sub-lists (chunks), where each sub-list contains a manageable number of events. These sub-lists are then stored separately in the database, allowing parallel processing and reducing the time required to update all collaborators while ensuring each chunk is processed within the designated time frame
2Loss of time
If the system divides event updates into smaller chunks, then the system can store events within the designated time frame, but the complexity of managing multiple sub-lists and retries increases
Solution Approach 1:
The system implements a retry mechanism that monitors which sub-lists fail to store within the designated time frame. Failed sub-lists are identified and retried separately, while successful updates are confirmed. This feedback loop automates the management of multiple sub-lists, reducing the perceived complexity while ensuring reliable delivery
3Device complexity
If the system processes all event entries in a single batch, then the implementation is simpler, but the system cannot scale to handle large numbers of collaborators
Solution Approach 1:
The system segments the collaborator update process into manageable sub-lists, each representing a portion of the total collaborator base. This segmentation enables the system to scale to large numbers of collaborators by processing multiple sub-lists in parallel, while maintaining relatively simple processing logic for each individual sub-list
Solution Approach 2:
The system introduces a new dimension of organization by grouping collaborators into sub-lists based on event relevance rather than processing all collaborators uniformly. This dimensional change allows the system to scale horizontally by adding more sub-lists as needed, improving productivity without proportionally increasing overall system complexity
Data Source
AI summary
Techniques are disclosed for improving scalability in a system which incrementally updates remote clients with events that occurred in a cloud-enabled platform. In one embodiment, a method comprises, in response to an action from a user in the cloud-enabled platform, determining a list of events to update one or more collaborators of the user about the action. The method further comprises separating the list of events into a plurality of sub-lists of events such that each sub-list of events can be stored in a database within a designated amount of time, and further comprises storing the plurality of sub-lists of events into the database to be read by the one or more collaborators. Among other advantages, embodiments disclosed herein provide enhancement in scalability, robustness and availability for cloud-based collaboration platforms with large numbers of collaborators by incorporating mechanisms to divide-and-conquer the workload of event updates in such platforms.