Reinforcement Learning Scheduling for Distributed Multi-Tiered Computing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
As computing environments grow in complexity, efficiently provisioning applications across diverse devices and domains in a distributed multi-tiered computing environment becomes challenging, particularly in managing service level agreements and optimizing resource utilization.
Innovation Solution
The method involves decomposing a service dependency graph, assigning normalized compute and network units, generating a Q-table using reinforcement Q-learning, calculating critical and max learned paths, and determining earliest and latest start times for tasks to generate scheduling assignments.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If traditional scheduling methods are used in complex distributed environments, then implementation is simpler, but scheduling efficiency and resource utilization deteriorate
Solution Approach 1:
The service dependency graph is decomposed into multiple subgraphs, each representing a manageable portion of the scheduling problem. This segmentation allows the complex scheduling task to be divided into smaller, more tractable sub-problems that can be solved independently and then combined, improving scheduling efficiency without overwhelming the system.
Solution Approach 2:
The system performs preliminary actions by pre-calculating and storing scheduling decisions in a Q-table through reinforcement learning. When scheduling decisions are needed, the system can quickly retrieve pre-computed information from the Q-table rather than performing complex calculations in real-time, significantly improving scheduling efficiency while the complexity is managed during the offline learning phase.
2Productivity
If reinforcement learning is used to optimize scheduling, then resource allocation improves, but computational overhead increases
Solution Approach 1:
The reinforcement learning Q-table is constructed in advance through offline learning, storing optimal or near-optimal scheduling decisions for various service dependency scenarios. During actual scheduling operations, the system retrieves pre-computed recommendations from the Q-table rather than performing intensive reinforcement learning calculations, thereby improving resource utilization while minimizing real-time computational overhead.
Solution Approach 2:
The system uses partial action by selecting only the most relevant portions of the Q-table that apply to the current scheduling context, rather than computing or storing all possible scheduling scenarios. This selective approach improves resource utilization for the specific tasks at hand while reducing the overall computational overhead associated with maintaining and querying the complete Q-table.
3Manufacturing precision
If detailed service dependency analysis is performed, then scheduling accuracy improves, but processing time increases
Solution Approach 1:
The system performs preliminary analysis of service dependencies during the Q-table construction phase, storing the results of detailed dependency analysis in advance. When actual scheduling is needed, the system retrieves pre-analyzed dependency information from the Q-table, achieving high scheduling accuracy without incurring the processing time penalty of performing detailed analysis in real-time.
Solution Approach 2:
The service dependency graph is segmented into smaller subgraphs, allowing detailed analysis to be performed on manageable portions rather than the entire system at once. This segmentation enables accurate scheduling decisions for each subgraph while reducing the overall processing time compared to analyzing the complete dependency graph in a single operation.
Data Source
AI summary
Techniques described herein relate to a method for managing a distributed multi-tiered computing (DMC) environment. The method includes decomposing, by a local controller associated with an DMC domain, a service dependency graph associated with a scheduling job; assigning normalized compute units and normalized network units to tasks included in the service dependency graph; generating a Q-table using the service dependency graph and reinforcement Q-learning; calculating a critical path and a max learned path using the Q-table and the service dependency graph; calculating the earliest start time and the latest start time for each task using the service dependency graph and the max learned path to obtain a plurality of earliest start time and latest start time pairs for each task; and generating scheduling assignments using the plurality of earliest start time and latest start time pairs for each task.