Cloud resource scheduling method and device and readable storage medium

Abstract

The embodiment of the invention discloses a cloud resource scheduling method and device and a readable storage medium. Wherein a to-be-scheduled task set is received, and a single calculation task in the task set has task deadline and task value density attributes; a task sorting model based on the Markov decision process is constructed, a to-be-scheduled task set is input into the task sorting model, a processor serves as an intelligent agent, the to-be-scheduled task set serves as an environment, the task sorting model is processed through a reinforcement learning algorithm in combination with a model prediction control method, and an optimal task sequence is obtained; and establishing a load balance model based on multi-dimensional resource evaluation, inputting the optimal task sequence into the load balance model, calculating a load balance degree index according to the load data, and adjusting task allocation among the processors according to the load balance degree index. According to the embodiment of the invention, the task sorting and resource allocation in the cloud computing environment are optimized by utilizing the synergistic effect of reinforcement learning and model prediction control, and the load balance and resource utilization rate are improved.

Description

Technical Field

[0001] Applications generally relate to the field of artificial intelligence technology, and in particular to a cloud resource scheduling method, device, and readable storage medium. Background Technology

[0002] In cloud computing environments, task scheduling is a core factor affecting system performance. Traditional scheduling methods are mainly based on static rules or heuristic algorithms, such as first-come, first-served (FFS) and shortest-job-first (SJF) strategies. While these methods are effective in simple scenarios, they have gradually revealed significant limitations in dynamically changing cloud environments, especially when dealing with highly volatile and uncertain tasks. They often cannot adapt to changes in system state in real time, resulting in poor performance in queuing latency, load balancing, and resource utilization.

[0003] In recent years, reinforcement learning techniques have been introduced into the field of cloud resource scheduling, enabling dynamic task scheduling through deep neural networks. These methods can adjust task priorities based on real-time system states, improving upon the shortcomings of traditional methods to some extent. However, existing reinforcement learning methods still have significant drawbacks: training deep neural networks requires substantial data and computational resources, resulting in high training costs; deterministic policy gradient methods converge slowly when dealing with high-dimensional continuous action spaces, impacting algorithm efficiency; and existing solutions often focus on a single optimization objective, lacking a holistic consideration of the entire scheduling process.

[0004] As the complexity of cloud computing environments continues to increase, task scheduling models need to possess stronger adaptability and intelligent management capabilities. Especially when dealing with scenarios such as sudden task requests and fluctuations in resource demand, existing methods struggle to achieve an ideal balance between scheduling accuracy, real-time performance, and resource utilization. Therefore, there is an urgent need for an intelligent scheduling method that can comprehensively consider task characteristics, system status, and resource conditions to achieve efficient management and optimized utilization of cloud computing resources. Summary of the Invention

[0005] This disclosure addresses some of the shortcomings mentioned in the background art by providing a cloud resource scheduling method, device, and medium.

[0006] In a first aspect, embodiments of this disclosure provide a cloud resource scheduling method, comprising the following steps: S1. Receive a set of tasks to be scheduled, wherein each computational task in the task set has a task deadline and a task value density attribute. S2. Construct a task ranking model based on Markov decision process. Input the set of tasks to be scheduled into the task ranking model. The processor acts as the agent and the set of tasks to be scheduled acts as the environment. Process the task ranking model through reinforcement learning algorithm combined with model predictive control method to obtain the optimal task sequence. S3. Establish a load balancing model based on multi-dimensional resource evaluation, input the optimal task sequence into the load balancing model, calculate the load balance index based on the load data, and adjust the task allocation among the processors according to the load balance index.

[0007] In one embodiment of the first aspect, step S2 further includes: Using the set of tasks to be scheduled as input, a state space containing multi-dimensional features of the tasks is constructed. Each state in the state space includes the resource requirement features of the task, the time constraint attributes of the task, and the value density parameters of the task. Based on the output of the state space, a corresponding action space is constructed, where a single action in the action space corresponds to the operation of selecting the next task to be scheduled from the current task queue.

[0008] In one embodiment of the first aspect, it further includes: A global priority table is established based on the task deadline and the task value density. A unique priority number is assigned to each task according to a preset priority rule, wherein the priority rule takes the task value density as the primary sorting criterion and the task deadline as the secondary sorting criterion. A composite reward function is constructed based on the global priority table.

[0009] In one embodiment of the first aspect, constructing the composite reward function based on the global priority table further includes: Query the global priority table and obtain the priority numbers of the current task and the selected task; Compare the priority number of the selected task with the priority number of the current task; If the priority number of the selected task is not higher than the priority number of the current task, a positive reward signal is output. If the priority number of the selected task is higher than the priority number of the current task, a negative reward signal is output.

[0010] In one embodiment of the first aspect, it further includes: The system state changes at multiple future time steps are predicted using model predictive control methods, and the state prediction results are output. Based on the state prediction results and reward signals, the reinforcement learning agent selects the optimal action and outputs the action selection decision. The state-action value function is iteratively updated using the Q-learning algorithm, and a converged Q-value table is output. The optimal task sequence is generated based on the Q-value table.

[0011] In one embodiment of the first aspect, step S3 further includes: The set of processor performance parameters, including CPU utilization, memory utilization, and network bandwidth utilization, are used as input to the load balancing model. By assigning corresponding weight coefficients to individual performance parameters, a weighted multidimensional feature representation is obtained; The multidimensional feature representation is mapped to a multidimensional space to obtain the feature point distribution of the processor in the multidimensional space; The location of the load's center of gravity is calculated based on the distribution of the feature points. The load balance index is obtained by calculating the distance from a single feature point to the load center of gravity.

[0012] In one embodiment of the first aspect, the method further includes setting a load balancing threshold, triggering task reallocation when the load balancing index exceeds the threshold.

[0013] In one embodiment of the first aspect, it further includes: Based on the load balance index, the execution order of tasks in the optimal task sequence is adjusted, and tasks are migrated from overloaded processors to lightly loaded processors.

[0014] In a second aspect, embodiments of this disclosure provide an electronic device including a memory and a processor, wherein the memory stores a program that runs on the processor, and the processor executes the steps of the cloud resource scheduling method as described in the first aspect when running the program. In a third aspect, embodiments of this disclosure provide a computer-readable storage medium having computer instructions stored thereon, which, when executed, perform the steps of the cloud resource scheduling method as described in the first aspect.

[0015] This disclosure proposes a cloud resource scheduling method, device, and medium, which have the following beneficial effects: (1) In terms of technical improvement, this disclosure constructs a task scheduling process with Markov characteristics by combining reinforcement learning with model predictive control strategies, realizing intelligent training and optimization of task priority order. This method can effectively minimize task queuing time and significantly improve the accuracy and efficiency of task sorting compared with traditional static priority rules, better adapting to the dynamic characteristics of cloud computing environments. At the same time, by introducing a dynamic task allocation mechanism based on processor load analysis and using innovative methods such as load balance evaluation and center of gravity location calculation, a balanced distribution of task load among processors is achieved, thereby greatly improving the system's task processing efficiency and overall load balancing performance. This optimization strategy, which organically combines the two stages of task sorting and task allocation, comprehensively improves the system's scheduling performance, meeting the strict requirements of task deadlines while improving the processor's service quality and CPU utilization.

[0016] (2) In terms of commercial value, this disclosure significantly improves the utilization rate of cloud resources and the overall performance of the system through an intelligent resource scheduling mechanism, effectively reducing resource waste and energy costs. The optimized task scheduling process brings about a significant improvement in task processing efficiency and a reduction in response time, thereby improving service quality and user experience, and meeting the market demand for efficient and stable cloud services. These technological advantages are further transformed into competitive advantages for enterprises, enhancing their market competitiveness by reducing operating costs and improving resource utilization, and providing technical support for achieving higher economic benefits and market share. Attached Figure Description

[0017] Figure 1 This is a schematic diagram of a cloud resource scheduling method. Figure 2 This is a schematic diagram illustrating the working principle of reinforcement learning in an embodiment of this disclosure. Detailed Implementation

[0018] The present application / disclosure will now be described in further detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the present application / disclosure and are not intended to limit the scope of the present application / disclosure. Furthermore, it should be noted that, for ease of description, only the parts relevant to the present application / disclosure are shown in the accompanying drawings, not the entire structure.

[0019] Example 1 Figure 1 To illustrate the cloud resource scheduling method flowchart disclosed herein, the following will refer to... Figure 1 This document provides a detailed description of the cloud resource scheduling method disclosed herein.

[0020] In step 101, the cloud computing task scheduling environment and parameter initialization configuration are established.

[0021] In one embodiment, a cloud computing environment comprising multiple computing nodes is first constructed, each node being configured with processor, memory, and network resources. A task queue is defined to receive and cache pending computing tasks. Each computing task includes a task identifier, resource requirement characteristics, deadline attributes, and value density parameters. The parameters of the reinforcement learning module are initialized, including the learning rate, discount factor, and exploration rate parameters.

[0022] In step 102, a task ranking model based on Markov decision process is constructed. The set of tasks to be scheduled is input into the task ranking model, the processor is used as the agent, the set of tasks to be scheduled is used as the environment, and the task ranking model is processed by reinforcement learning algorithm combined with model predictive control method to obtain the optimal task sequence.

[0023] In this embodiment, task ordering is a crucial factor in determining system performance within the cloud resource task scheduling system. The main objective of this step is to determine the optimal execution order in the task queue using intelligent algorithms, thereby minimizing task queuing time and satisfying various task constraints. Considering the inherently NP-hard nature of the task scheduling problem, traditional optimization methods struggle to obtain satisfactory solutions within a reasonable timeframe. Therefore, this disclosure employs a combination of reinforcement learning and model predictive control for training and testing.

[0024] In one embodiment, each task is considered as an independent state, where the state space S contains all possible task queue configurations. Each state s t Each ∈S contains complete information about the currently scheduled tasks, including but not limited to important features such as computational resource requirements, memory requirements, deadline constraints, and value density. The action space A defines all possible task selection operations, i.e., the behavior of selecting the next task to execute from the current set of tasks to be scheduled. A reward function is designed, which constructs a priority evaluation mechanism based on the two dimensions of task deadline and value density.

[0025] refer to Figure 2 The reinforcement learning agent acts as the decision-making core, continuously interacting with the task scheduling environment. At each decision time t, the agent observes the current system state S. t Select and execute action A based on a specific strategy. t Subsequently, environmental feedback provides an immediate reward R t And transition to the new state S t+1 Therefore, the continuous interaction between the agent and the task scheduling environment produces the following sequence:

[0026] The state-action-reward sequence formed by this interaction process fully records the decision-making history, providing a data foundation for strategy optimization.

[0027] This task scheduling process constructs a typical Markov decision process. In this framework, the core objective of the agent is to maximize its cumulative reward by selecting a series of actions. The entire process begins with the initial interaction between the agent and the environment: based on its current state, the agent selects the action to be performed according to its state transition strategy; after receiving the action, the environment generates a reward signal as feedback and updates its own state to prepare for the next round of interaction.

[0028] The core of Markov decision processes lies in their "no aftereffects," meaning that the future state of the system depends only on the current state and the action to be performed, and is independent of the historical state sequence. In the specific context of task scheduling, this property is mapped as follows: each task is treated as a state, containing all the relevant information for that task needed to make a scheduling decision. Therefore, the entire task scheduling process is successfully transformed into a learning process with Markov properties. In this model, the processor is defined as an agent, the task scheduling queue constitutes its learning environment, and each action of selecting a task from the queue for scheduling is defined as an action. The rewards generated by these actions are used to continuously train and optimize the agent's decision-making strategy.

[0029] In one embodiment, the Markov decision process quintuple parameters are defined. for: S: Set of environmental states, i.e., action space, state at time t. ; A: The set of optional actions. That is, the action space, the actions to be executed at time t. ; P: State transition probability, i.e., in the current state Next action Then, transition to the next state. The probability, ; R: The set of immediate rewards, i.e., in the current state Next action Then, transition to the next state. The reward ; Discount factor Primarily responsible for calculating future discount cumulative rewards .

[0030] The goal of reinforcement learning can be understood as solving the optimal value function, which is represented by Equations 1 and 2.

[0031] Formula 1 Formula 2 Reinforcement learning algorithms make decisions by iteratively updating a Q-value table. After multiple iterations, the Q-value table gradually converges to a stable state. Based on the converged Q-value table, the system sorts tasks according to their Q-values, thereby generating the optimal task execution sequence.

[0032] Given that task deadline and task value density are two different evaluation criteria, it is difficult to effectively unify them through simple arithmetic operations or weighted merging. Therefore, a comprehensive global priority lookup table was designed based on deadline and value density.

[0033] Table 1 Priority Table Design:

[0034] As shown in Table 1, a comprehensive global priority table is established based on the task deadline D (including specific values ​​such as D1, D2, etc.) and task value density V (including specific values ​​such as V1, V2, etc.). Value density (V) is the primary priority; when value densities are the same, an earlier deadline (D) has a higher priority. Specifically, at each decision step, the agent selects the next task based on the current state. The algorithm consults Table 1 and compares the priority number of the selected task with the priority number of the current task: if the priority number of the selected task is less than or equal to the priority number of the current task, it indicates that the selection conforms to the global priority order, and a positive reward is given; otherwise, a negative reward is given.

[0035] In this embodiment, Model Predictive Control (MPC) is introduced. This technique optimizes control actions by minimizing a cost function, thereby improving controller performance. Theoretically, Deep Reinforcement Learning (DRL) can be further employed to improve the predictive model in MPC, achieving direct optimization of control actions.

[0036] In one embodiment, combined with the MPC predictive control method, the overall task ranking algorithm can be specifically executed as follows: Input: learning rate α∈(0,1), decay factor γ∈(0,1), policy π: ε-greedy, prediction range H p Threshold μ. State set S, action set A, priority table List pro Reward value.

[0037] S1021, initialize the value Q for all states and actions, initialize the Q value to 0 for the terminal state, and initialize λ and ε. S1022, utilizing Strategy based on List pro Table selects action ; S1023, Take action To obtain the next state And calculate the reward value based on the reward function. ; S1024, Order (Jump to step 1023). S1025, Update Q value:

[0038] Q( , )←Q( , ) +

[0039] Step 1026: Output the result after all Q values ​​converge. ; If S n If the current state is S, then record |dQ|, calculate the average of the last 10 values ​​|dQ|, and if mean(|dQ|) < μ, then terminate the learning process.

[0040] Select the trajectory with the maximum value to obtain the action sequence. , ,..., and output

[0041] The agent continuously observes its current state. Select and execute an action Rewards for taking actions in the current state In addition, considering the learning rate With loss factor The formula for updating the entire table is: Formula 3 In step 103, a load balancing model based on multi-dimensional resource evaluation is established. The optimal task sequence is input into the load balancing model, the load balance index is calculated based on the load data, and the task allocation among the processors is adjusted according to the load balance index.

[0042] In this embodiment, in actual operation, if a simple random allocation strategy is adopted, it is very easy to cause uneven processor load: some processors are overloaded due to too many tasks and cannot process new tasks within the specified time; at the same time, other processors are idle due to insufficient task allocation, resulting in a serious waste of computing resources. To address this critical problem, this disclosure proposes to accurately monitor the processor load status by quantitatively evaluating the system's load balance. In a multiprocessor system architecture, the randomness of task allocation will significantly exacerbate the load imbalance phenomenon. Therefore, it is necessary to establish a scientific load evaluation system and corresponding dynamic scheduling strategy. By monitoring the load status of each processor in real time and dynamically adjusting the allocation ratio of tasks among the processors based on the load balance evaluation results, it is possible to ensure that the overall system load tends to be balanced and maximize resource utilization.

[0043] In one embodiment, it is assumed that the system contains n computing nodes (processors), and the operating state of each node is characterized by m key performance parameters. These parameters are assigned different weights according to their importance in task execution, forming a weight vector K=[k1,k2,...,k m ], where each weight component satisfies the normalization constraint ∑k i =1. The weight allocation process needs to comprehensively consider task characteristics and system objectives. For example, computationally intensive tasks should be given a higher weight for CPU utilization, while data-intensive tasks should be given a higher weight for storage I / O performance.

[0044] Formula 4 0 ,0 ,...,0

[0045] In Formula 3 above, each computation node is mapped to a feature point in the m-dimensional resource state space, and the spatial coordinates of the j-th node are represented as Point. j ={p j1 , p j2 , ..., p jm}, where each coordinate component p ji This represents the normalized metric for the node in the i-th performance dimension. These metrics are standardized to ensure that 0 ≤ p ji ≤k i The constraints ensure that the values ​​of each dimension not only reflect the actual resource status but also demonstrate the importance of that dimension in the overall assessment. This representation method transforms complex load conditions into quantifiable spatial geometric relationships.

[0046] In one embodiment, to assess the overall system load, it is necessary to calculate the centroid positions of all processor feature points in m-dimensional space. The centroid coordinates are P = (P1, P2, ..., P...). m It is determined by the arithmetic mean of the parameters in each dimension. The specific calculation formula is as follows: 1= 2= ,..., m= Formula 5 Based on the centroid location, the load balance index L is further calculated. This index is defined as the average Euclidean distance from all processor feature points to the centroid location, i.e.: Formula 6 In this embodiment, the load balance index L provides a quantitative assessment of the system load distribution. When the characteristic points of each processor are closely clustered around the center of gravity, the L value is small, indicating a balanced system load distribution. As the dispersion of the characteristic point distribution increases, the L value increases accordingly, reflecting a decrease in the system load balance. In particular, when the system experiences an extreme load distribution, i.e., half of the processors are idle while the other half are fully loaded, the characteristic point distribution is most dispersed, the L value reaches its maximum, the system load balance is worst, and CPU utilization drops to its lowest level.

[0047] In one embodiment, a dynamic task allocation and adjustment mechanism is established based on the load balance index. This mechanism continuously monitors changes in the load balance L, and automatically triggers a task reallocation process when the L value exceeds a preset threshold. The reallocation process comprehensively considers task characteristics and the real-time load status of the processors, and uses an intelligent task migration algorithm to dynamically adjust some tasks on overloaded processors to lightly loaded processors, thereby making the overall system load distribution more balanced.

[0048] In this embodiment, the dynamic load balancing mechanism brings several performance improvements. First, by optimizing the task allocation strategy, it significantly improves the overall CPU utilization of the system, avoiding a situation where resource idleness and overload coexist. Second, by balancing the load of each processor, it effectively reduces the missed task deadline rate, improving the system's real-time performance. Furthermore, this mechanism can adapt to changes in processor size, maintaining good load balancing characteristics even when the system is expanded, overcoming the task clustering problem that occurs in traditional random allocation methods.

[0049] The entire task allocation process is seamlessly integrated with the earlier task sequencing phase, together forming a complete cloud resource scheduling solution. Through quantitative evaluation and dynamic adjustment of load balancing, the system achieves efficient utilization of computing resources while ensuring optimal task execution order, ultimately maximizing system throughput and minimizing response time.

[0050] This disclosure provides a cloud resource scheduling method, device, and medium to address the problems of slow task deadlines and poor communication in existing task scheduling methods for intelligent production lines. Scheduling is divided into two stages: task sorting and task allocation. The aim is to minimize queuing time in the task sorting stage and utilize reinforcement learning methods to obtain the task order with the shortest queue time. In the task allocation stage, more accurate scheduling decisions are made by analyzing processor load.

[0051] In a second aspect, embodiments of this disclosure provide an electronic device including a memory and a processor, wherein the memory stores a program that runs on the processor, and the processor executes the steps of the cloud resource scheduling method as described in the first aspect when running the program. In a third aspect, embodiments of this disclosure provide a computer-readable storage medium having computer instructions stored thereon, which, when executed, perform the steps of the cloud resource scheduling method as described in the first aspect.

Claims

1. A cloud resource scheduling method, characterized in that, Includes the following steps: S1. Receive a set of tasks to be scheduled, wherein each computational task in the task set has a task deadline and a task value density attribute. S2. Construct a task ranking model based on Markov decision process. Input the set of tasks to be scheduled into the task ranking model. The processor acts as the agent and the set of tasks to be scheduled acts as the environment. Process the task ranking model through reinforcement learning algorithm combined with model predictive control method to obtain the optimal task sequence. S3. Establish a load balancing model based on multi-dimensional resource evaluation, input the optimal task sequence into the load balancing model, calculate the load balance index based on the load data, and adjust the task allocation among the processors according to the load balance index.

2. The cloud resource scheduling method according to claim 1, characterized in that, Step S2 further includes: Using the set of tasks to be scheduled as input, a state space containing multi-dimensional features of the tasks is constructed. Each state in the state space includes the resource requirement features of the task, the time constraint attributes of the task, and the value density parameters of the task. Based on the output of the state space, a corresponding action space is constructed, where a single action in the action space corresponds to the operation of selecting the next task to be scheduled from the current task queue.

3. The cloud resource scheduling method according to claim 1, characterized in that, Further includes: A global priority table is established based on the task deadline and the task value density. A unique priority number is assigned to each task according to a preset priority rule, wherein the priority rule takes the task value density as the primary sorting criterion and the task deadline as the secondary sorting criterion. A composite reward function is constructed based on the global priority table.

4. The cloud resource scheduling method according to claim 3, characterized in that, The construction of the composite reward function based on the global priority table further includes: Query the global priority table and obtain the priority numbers of the current task and the selected task; Compare the priority number of the selected task with the priority number of the current task; If the priority number of the selected task is not higher than the priority number of the current task, a positive reward signal is output. If the priority number of the selected task is higher than the priority number of the current task, a negative reward signal is output.

5. The cloud resource scheduling method according to claim 4, characterized in that, Also includes: The system state changes at multiple future time steps are predicted using model predictive control methods, and the state prediction results are output. Based on the state prediction results and reward signals, the reinforcement learning agent selects the optimal action and outputs the action selection decision. The state-action value function is iteratively updated using the Q-learning algorithm, and a converged Q-value table is output. The optimal task sequence is generated based on the Q-value table.

6. The cloud resource scheduling method according to claim 1, characterized in that, Step S3 further includes: The set of processor performance parameters, including CPU utilization, memory utilization, and network bandwidth utilization, are used as input to the load balancing model. By assigning corresponding weight coefficients to individual performance parameters, a weighted multidimensional feature representation is obtained; The multidimensional feature representation is mapped to a multidimensional space to obtain the feature point distribution of the processor in the multidimensional space; The location of the load's center of gravity is calculated based on the distribution of the feature points. The load balance index is obtained by calculating the distance from a single feature point to the load center of gravity.

7. The cloud resource scheduling method according to claim 1 or 6, characterized in that, It also includes setting a load balancing threshold, which triggers task redistribution when the load balancing index exceeds the threshold.

8. The cloud resource scheduling method according to claim 7, characterized in that, Further includes: Based on the load balance index, the execution order of tasks in the optimal task sequence is adjusted, and tasks are migrated from overloaded processors to lightly loaded processors.

9. An electronic device, characterized in that, The system includes a memory and a processor, wherein the memory stores a program that runs on the processor, and the processor executes a cloud resource scheduling method according to any one of claims 1-8 when running the program.

10. A computer-readable storage medium storing computer instructions thereon, characterized in that, The computer instructions execute a cloud resource scheduling method according to any one of claims 1-8.

Images