Optimistic Concurrency for Distributed Event Assignment
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Large computerized systems face inefficiencies in managing event tracking due to excessive computational resource usage and latency when dealing with a vast number of events across multiple machines, leading to potential delays and resource limitations.
Innovation Solution
Implementing an optimistic concurrency database to allow event tracking machines to compete for event assignments, enabling each machine to claim responsibility for multiple events and allowing concurrent redundancy, thereby optimizing resource allocation and reducing latency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If a single event tracking machine is used to track all events, then comprehensive event coverage is achieved, but computational resource usage and latency increase excessively
Solution Approach 1:
The system divides the event tracking workload across multiple worker machines, each responsible for tracking specific subsets of events. This segmentation allows the system to maintain comprehensive event coverage while reducing the computational burden and latency on any single machine, as each machine processes only its assigned portion of events
2Loss of time
If multiple event tracking machines are used to reduce latency, then processing speed improves, but resource management complexity and overhead increase
Solution Approach 1:
Worker machines autonomously compete to claim events for tracking without requiring centralized assignment or coordination. Each worker machine independently attempts to insert itself into event tracking responsibilities, and the optimistic concurrency control mechanism automatically resolves conflicts. This self-service approach eliminates the need for complex resource management overhead while maintaining low latency through parallel processing
3Productivity
If multiple worker machines compete for event assignments, then resource allocation efficiency improves, but database conflict and assignment failure rate increase
Solution Approach 1:
The system performs preliminary checks before finalizing event assignments by first verifying that no other worker machine has claimed the event. The optimistic concurrency control mechanism allows worker machines to proceed with assignments based on the assumption that no conflicts exist, and only resolves conflicts when actual conflicts are detected during the assignment process
Solution Approach 2:
The system converts the potential harm of database conflicts into a beneficial mechanism for ensuring assignment uniqueness. When conflicts do occur during concurrent assignment attempts, the optimistic concurrency control detects these conflicts and prevents duplicate assignments, thereby ensuring that each event is tracked by exactly one worker machine while still allowing high levels of concurrent competition
Data Source
AI summary
A method performed by a service management device includes (a) receiving from a plurality of worker machines, a plurality of requests, each request from a worker machine indicating a respective work item to be assigned to that worker machine, (b) in response to receiving each request, performing an assignment attempt operation configured to (i) succeed and insert a reference to the worker machine that issued that request in a database entry associated with the respective work item if no other reference to a worker machine has yet been inserted in the database entry and (ii) otherwise report failure, and (c) returning a reply to the worker machine that issued each request indicating whether the assignment attempt operation succeeded, that worker machine configured to (i) process the respective work item in response to a successful reply and (ii) refrain from processing the respective work item in response to a failed reply.