Stateful Distributed Streaming Queue for Event-Based Task Processing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional task processing solutions face challenges in achieving elasticity, load-balancing, and scalability to meet stringent service level agreements (SLAs) due to difficulties in obtaining accurate compute processing times, handling compute resource heterogeneity, and providing fault tolerance for stateful computations and streaming events.
Innovation Solution
A platform and language agnostic event-based task processing module utilizing a distributed queue to store tasks, which updates them at runtime events, enabling stateful computation, fault tolerance, and dynamic resource management for improved elasticity and scalability.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If centralized schedulers are used to assign batches of work to workers, then task distribution can be achieved, but accurate information regarding compute processing times is difficult to obtain and the system becomes complex
Solution Approach 1:
The patent extracts the scheduling logic from a centralized scheduler and distributes it to individual workers. Each worker independently manages its own task queue and makes scheduling decisions locally, eliminating the need for a complex centralized scheduling system while maintaining efficient task distribution
Solution Approach 2:
Workers autonomously retrieve tasks from their local queues and execute them without requiring centralized coordination. The system enables self-service by allowing each worker to independently manage its own workload and make scheduling decisions based on local conditions
2Productivity
If optimal task distribution is attempted to account for performance variations and compute resource heterogeneity, then productivity can be improved, but the problem becomes NP complete and computationally intractable
Solution Approach 1:
Instead of attempting global optimization across the entire system, the patent applies local quality by allowing each worker to optimize its own task execution based on its specific hardware characteristics and performance profile. This local optimization approach avoids the computational intractability of global optimization while still improving overall system productivity
Solution Approach 2:
The patent segments the scheduling problem into independent sub-problems, one for each worker. By dividing the monolithic scheduling decision into smaller, localized decisions made by individual workers, the system avoids the NP-complete complexity of centralized optimization while achieving near-optimal performance through distributed decision-making
3Reliability
If centralized schedulers are used to meet stringent SLAs, then processing latency can be controlled, but fault tolerance is limited and the scheduler must be resilient in case of failures
Solution Approach 1:
The patent implements beforehand cushioning by maintaining persistent task queues that survive worker failures. Tasks are queued in a durable storage medium before execution, so if a worker fails, the tasks remain in the queue ready to be picked up by another worker, providing automatic fault tolerance without complex recovery mechanisms
Solution Approach 2:
The patent extracts the fault tolerance requirement from the centralized scheduler and implements it at the worker level through local task queues. Each worker independently manages its own task queue with persistence capabilities, eliminating the need for a complex centralized fault tolerance system while maintaining SLA compliance
4Device complexity
If decentralized queuing solutions are used with multiple stateless schedulers, then system complexity is reduced, but stateful computation is not supported and fault tolerance remains limited
Solution Approach 1:
The patent applies preliminary action by pre-configuring workers with local task queues and persistence mechanisms before execution begins. This preliminary setup enables workers to maintain state locally and continue execution even after failures, providing stateful computation support without requiring complex centralized state management
Data Source
AI summary
Various methods, apparatuses/systems, and media for event-based task processing are disclosed. The system includes a processor and a memory operatively connected to the processor via a communication interface. The processor defines a stateful distributed stream queue as a streaming queue which holds stateful computational requests generated at a stream of events, wherein each request contains a context and a collection of queue tasks processed by workers dedicated to the queue; runs the stateful distributed stream queue on a cloud infrastructure; implements the stateful computational requests, by utilizing the stateful distributed stream queue, at the stream of events; and clusters the queue tasks into affinity sub-queues by implementing configurable classification criteria to indicate that their computation is sped up on particular workers, while workers process tasks specific to their affinity; and implements abstractions for submitting and automatically re-computing the queue tasks while maintaining queue state between streaming events.