Deadline-Based Thread Scheduling for Cloud Latency SLAs
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Managing stringent latency needs in large-scale distributed systems is challenging, particularly in cloud computing environments where services must process requests from hundreds of thousands or millions of clients concurrently.
Innovation Solution
The implementation of operating system-level scheduling algorithms that use response initiation time targets (RITTs) based on latency service level agreements (SLAs) to determine the order in which runnable threads are selected for execution at back-end service nodes.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If conventional scheduling algorithms are used to process client requests in cloud computing environments, then the system can handle large numbers of concurrent requests, but latency service level agreements cannot be met
Solution Approach 1:
The patent changes the scheduling parameter from conventional FIFO or priority-based scheduling to deadline-based scheduling where each task is assigned a response initiation time target (RITT) derived from latency SLAs. This parameter change enables the scheduler to prioritize tasks based on their deadline urgency, ensuring latency SLA compliance while maintaining high request processing throughput through efficient deadline-aware thread selection.
2Reliability
If computing resources are increased to meet latency SLAs, then latency requirements can be satisfied, but resource costs increase
Solution Approach 1:
The patent applies preliminary action by pre-calculating response initiation time targets (RITTs) for tasks based on their latency SLAs before task execution begins. This allows the scheduling system to proactively prioritize tasks that are approaching their deadlines, ensuring latency SLA compliance through intelligent scheduling rather than through brute-force increases in computing resources.
3Productivity
If more threads are scheduled for execution to handle concurrent requests, then request processing capacity increases, but latency SLAs may not be met due to scheduling overhead
Solution Approach 1:
The patent changes the thread scheduling parameter to include deadline urgency information in the scheduling decision process. By modifying the scheduler to select runnable threads based on their associated task deadlines rather than conventional scheduling criteria, the system can handle high request volumes while maintaining latency SLA compliance through deadline-driven thread selection.
Data Source
AI summary
A response initiation time target based at least in part on a service level agreement is assigned to a task requested from a network-accessible service. A deadline parameter of a thread identified to perform sub-tasks of the task at a service logic implementation node is set based on the target. The thread is scheduled for execution using an operating system thread scheduling algorithm which selects threads for execution based on their deadline parameters. After a first sub-task of thread is completed, a response is sent to the requester of the task.