A method and apparatus for scheduling heterogeneous computing resources based on multi-objective optimization

Abstract

This application belongs to the field of parallel task scheduling technology, specifically disclosing a method and apparatus for scheduling heterogeneous computing resources based on multi-objective optimization. The method includes: selecting at least two performance indicators as optimization objectives from two dimensions—task and resource—to establish a multi-objective optimization model for heterogeneous computing resource scheduling; transforming the computation task request process into a computation task waiting model based on queuing theory; constructing a multi-task adaptive scheduling model based on reinforcement learning based on observed resource states; constructing a Markov decision process model for the multi-task-oriented heterogeneous computing resource scheduling problem and resource mapping process based on the multi-objective optimization model and the computation task waiting model; and realizing adaptive multi-task heterogeneous computing resource scheduling based on the Markov decision process model and the multi-task adaptive scheduling model. The embodiments of this application solve the problem that scheduling methods for homogeneous computing resources are difficult to adapt to heterogeneous computing resource scheduling.

Description

Technical Field

[0001] This application belongs to the field of parallel task scheduling technology, and more specifically, relates to a method and apparatus for scheduling heterogeneous computing resources based on multi-objective optimization. Background Technology

[0002] In recent years, the explosive growth in data scale, algorithm complexity, and computing power demands, along with the emergence of various coprocessors, has driven the development of computer architectures towards heterogeneous parallelism. Faced with a large number of concurrent task demands from users, the existing architectures feature a heterogeneous coexistence of general-purpose processors and coprocessors. Traditional resource scheduling methods struggle to fully leverage the computing power advantages of different types of processors, leading to numerous performance issues such as high heterogeneous resource scheduling overhead, low task throughput, and low resource utilization. Furthermore, unilaterally satisfying a single performance metric usually means sacrificing other performance metrics; therefore, balancing various performance metrics to maximize utility has become a key focus in the field of computing resource scheduling in recent years.

[0003] In related technologies, the scheduling problem of complex parallel tasks mostly utilizes the Directed Acyclic Graph (DAG) model. This model uses a DAG to represent parallel tasks, thus eliminating the limitations of other models in terms of the number of subtasks and the order of multiple tasks. DAG-based parallel task scheduling methods are mainly implemented in three ways: decomposition-based scheduling, global scheduling, and federated scheduling. Decomposition-based scheduling decomposes the task into a set of sequential subtasks and then schedules them using traditional multiprocessor scheduling algorithms. Global scheduling performs schedulability analysis on global tasks and then performs load balancing scheduling, dynamically allocating tasks to processors during runtime. Federated scheduling executes tasks based on characteristics such as utilization, with high-utilization tasks running independently on dedicated processors, and other tasks running sequentially on a single processor.

[0004] However, most of the scheduling algorithms mentioned above are designed for parallel systems based on homogeneous processors, and it is difficult to achieve efficient mapping between heterogeneous processors and tasks, which leads to many performance problems such as low execution efficiency and insufficient resource sharing. Summary of the Invention

[0005] To address the shortcomings of related technologies, this application provides a heterogeneous computing resource scheduling method and apparatus based on multi-objective optimization, aiming to solve the problem that homogeneous computing resource scheduling methods are difficult to adapt to heterogeneous computing resource scheduling.

[0006] In a first aspect, embodiments of this application provide a heterogeneous computing resource scheduling method based on multi-objective optimization, comprising:

[0007] A multi-objective optimization model for heterogeneous computing resource scheduling is established by selecting at least two performance indicators as optimization objectives from the two dimensions of tasks and resources; the computing task request process is transformed into a computing task waiting model based on queuing theory; and a multi-task adaptive scheduling model based on reinforcement learning is constructed based on the observed resource status.

[0008] Based on the multi-objective optimization model and the computational task waiting model, the heterogeneous computing resource scheduling problem and resource mapping process for multi-task applications are constructed as a Markov decision process model.

[0009] Based on the Markov decision process model and a trained multi-task adaptive scheduling model, adaptive multi-task heterogeneous computing resource scheduling is achieved.

[0010] Secondly, embodiments of this application also provide a heterogeneous computing resource scheduling device based on multi-objective optimization, comprising:

[0011] The first building block includes:

[0012] The first construction submodule is used to select at least two performance indicators as optimization objectives from the two dimensions of tasks and resources, and to establish a multi-objective optimization model for heterogeneous computing resource scheduling.

[0013] The second construction submodule is used to transform the computation task request process into a computation task waiting model based on queuing theory.

[0014] The third construction submodule is used to build a multi-task adaptive scheduling model based on reinforcement learning based on the observed resource status.

[0015] The second building module is used to construct the heterogeneous computing resource scheduling problem and resource mapping process for multi-task applications into a Markov decision process model based on a multi-objective optimization model and a computing task waiting model.

[0016] The scheduling module is used to achieve adaptive scheduling of heterogeneous computing resources for multiple tasks based on the Markov decision process model and a trained multi-task adaptive scheduling model.

[0017] Thirdly, embodiments of this application also provide an electronic device, including: at least one memory for storing a program; and at least one processor for executing the program stored in the memory. When the program stored in the memory is executed, the processor is used to execute the method described in the first aspect or any possible implementation of the first aspect.

[0018] Fourthly, embodiments of this application also provide a computer-readable storage medium storing a computer program that, when run on a processor, causes the processor to perform the method described in the first aspect or any possible implementation thereof.

[0019] Fifthly, embodiments of this application also provide a computer program product that, when run on a processor, causes the processor to perform the method described in the first aspect or any possible implementation thereof.

[0020] The heterogeneous computing resource scheduling method and apparatus based on multi-objective optimization provided in this application realizes adaptive scheduling of heterogeneous computing resources for multiple tasks by constructing a multi-objective optimization model for heterogeneous computing resources, modeling the arrival process of computing request tasks, modeling resource scheduling and resource mapping, and using an improved reinforcement learning model based on resource reservation. This solves the problem that the scheduling method for homogeneous computing resources is difficult to adapt to the scheduling of heterogeneous computing resources. Attached Figure Description

[0021] To more clearly illustrate the technical solutions in this application or related technologies, the accompanying drawings used in the description of the embodiments or related technologies will be briefly introduced below. Obviously, the accompanying drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0022] Figure 1 This is a flowchart illustrating the heterogeneous computing resource scheduling method based on multi-objective optimization provided in an embodiment of this application.

[0023] Figure 2 This is a schematic diagram illustrating the principle of the computational task waiting model provided in the embodiments of this application;

[0024] Figure 3 This is a schematic diagram illustrating the principle of the multi-task adaptive scheduling model provided in the embodiments of this application;

[0025] Figure 4 This is a schematic diagram of the heterogeneous computing resource scheduling device based on multi-objective optimization provided in an embodiment of this application;

[0026] Figure 5 This is a schematic diagram of the structure of the electronic device provided in the embodiments of this application. Detailed Implementation

[0027] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0028] Figure 1 This is a flowchart illustrating the heterogeneous computing resource scheduling method based on multi-objective optimization provided in an embodiment of this application, as shown below. Figure 1As shown, the method includes at least the following steps:

[0029] S101. Select at least two performance indicators as optimization objectives from the two dimensions of tasks and resources, and establish a multi-objective optimization model for heterogeneous computing resource scheduling; transform the computing task request process into a computing task waiting model based on queuing theory; and construct a multi-task adaptive scheduling model based on reinforcement learning based on the observed resource status.

[0030] S102. Based on the multi-objective optimization model and the computational task waiting model, the heterogeneous computing resource scheduling problem and resource mapping process for multi-task applications are constructed into a Markov decision process model.

[0031] S103. Based on the Markov decision process model and the trained multi-task adaptive scheduling model, adaptive multi-task heterogeneous computing resource scheduling is realized.

[0032] Specifically, the construction order of the multi-objective optimization model, the computational task waiting model, and the multi-task adaptive scheduling model in S101 can be adjusted, and this embodiment does not limit this.

[0033] Regarding the establishment of a multi-objective optimization model: When constructing an optimization model for scheduling heterogeneous computing resources, considering only one performance indicator usually means sacrificing other performance indicators. Therefore, it is necessary to balance the performance indicators to maximize the utility.

[0034] In this embodiment of the application, starting from the two dimensions of task and resources, at least two performance indicators are selected as optimization objectives to establish a multi-objective optimization model for heterogeneous computing resource scheduling.

[0035] In some embodiments, establishing a multi-objective optimization model for heterogeneous computing resource scheduling specifically includes:

[0036] Define the constraints for the multi-objective optimization model;

[0037] From the task dimension, task request response time, computing resource scheduling overhead and task throughput are selected as optimization objectives, and a task model is established. The task model represents the service quality of computing tasks.

[0038] From the resource dimension, the utilization rate of computing resources is selected as the optimization objective, and a resource model is established to represent the utilization rate of computing resources.

[0039] Based on the task model and resource model, the objective function of the multi-objective optimization model is determined.

[0040] Specifically, when establishing a multi-objective optimization model for heterogeneous computing resource scheduling, multiple performance indicators such as task request response time, computing resource scheduling overhead, task throughput, and computing resource utilization are comprehensively considered. The various performance indicators are balanced and abstracted into service quality, thus avoiding other performance bottlenecks caused by considering only a single performance indicator.

[0041] The overall resource scheduling system meets the following constraints:

[0042]

[0043] Within the system, various heterogeneous computing resources are categorized into K types, with each type corresponding to a certain number of computing resources. i Let C represent the quantity of the i-th type of resource. K This represents the total capacity of computing resources of type K.

[0044] Under the above constraints, in order to rationally allocate computing tasks and maximize the quality of service and resource utilization, the following objective f(SR) is adopted to construct a multi-objective optimization model for heterogeneous computing resource scheduling. The heterogeneous computing resource scheduling objective model is decomposed into a task model and a resource model, and the parameters α and β are used to achieve the balance between the task model and the resource model.

[0045] Specifically, the objective of heterogeneous computing resource scheduling can be modeled as the following optimization problem, with the objective function being f(SR), specifically expressed as follows:

[0046]

[0047] Among them, SR i This represents the i-th scheduling process assigned to the computation task, QoS(SR) i ) represents the quality of service (QoS) of the computational task during the i-th scheduling process, UR(SR) i Let f(SR) represent the resource utilization rate corresponding to the i-th scheduling process, i.e., f(SR) represents the maximization of the service quality and resource utilization rate of the task. α and β are the weight values ​​of the service quality objective function and the resource utilization objective function, respectively, and satisfy α+β=1.

[0048] The task model represents the quality of service (QS) of a computation task. The QS of a computation task includes various attributes, specifically task request response time, scheduling overhead, and task throughput, and is calculated using the following formula:

[0049]

[0050] Among them, RT(SR) i ) represents the task request response time, DT(SR) i) represents the computational resource scheduling overhead of a computational task, TP(SR) i The ) represents task throughput, and the formula for calculating task throughput is as follows:

[0051]

[0052] in, This indicates the total computation task request. This indicates that the computation task request has been successfully responded to.

[0053] The resource model is the result of modeling computing resource utilization. Specifically, it refers to modeling computing resource utilization as a load based on heterogeneous computing resources. Based on the resource model, heterogeneous computing resource scheduling is implemented to achieve a balanced resource load, ensuring efficient and correct task execution. The load of heterogeneous computing resources reflects the demand for computing resources. Generally, the higher the load capacity of heterogeneous computing resources, the more powerful the computing capabilities, and the more workloads they can handle. However, the load capacity of heterogeneous computing resources cannot exceed their design limits, otherwise it will lead to overheating, frequency throttling, or damage. Therefore, real-time modeling of computing resource load can ensure overall system load balancing.

[0054] The resource model satisfies the following calculation formula:

[0055]

[0056] Where M represents the period for monitoring heterogeneous computing resource cores, h represents the number of heterogeneous computing resource cores, and the computing power of each core is as follows: Then the time spent in the running state during period M are respectively

[0057] Because load provides real-time information about the current system load, it helps the scheduler more accurately assess the system's operating status. By monitoring and analyzing load, the scheduler can make smarter and more optimized scheduling decisions based on the current system state. Secondly, real-time load information can be used to achieve load balancing, an important task scheduling strategy that aims to rationally distribute tasks across various heterogeneous computing resources in the system to avoid resource overload or imbalance. As one type of resource information, load can help the scheduler identify nodes with high loads and schedule new tasks to nodes with lower loads, thereby achieving load balancing and improving the overall system performance and efficiency.

[0058] The heterogeneous computing resource scheduling method provided in this application starts from two dimensions: task and resource. It uses a resource model to ensure the reservation of computing resources for hard real-time tasks and also regulates the balanced use of resources. The task model, on the other hand, addresses the task from the perspective of tasks and ensures the orderly execution of multiple tasks. In particular, when facing hard real-time tasks, the resource reservation method reserves the resources required by real-time tasks, which not only ensures the smooth execution of hard real-time tasks, but also ensures the effective execution of other non-real-time tasks.

[0059] Furthermore, regarding the establishment of a waiting model for computational tasks: Queuing Theory, also known as the theory of stochastic service systems, obtains statistical laws of quantitative indicators such as waiting time, queuing duration, and busy period length through statistical research on the arrival of service objects and service time. Then, based on these laws, the structure of the service system is improved or the service objects are reorganized so that the service system can not only meet the needs of the service objects, but also optimize certain indicators of the system.

[0060] Figure 2 This is a schematic diagram illustrating the principle of the computational task waiting model provided in the embodiments of this application, as shown below. Figure 2 As shown, classical queuing theory is used to model the computational task request process of the system and construct a computational task waiting model. When the number of computational resources is S, the random process of computational task requests follows a Poisson process with an arrival rate of λ, and the completion time of computational task requests follows an exponential distribution with a mean of 1 / μ.

[0061] Then calculate the average length L of the task request queue. q And the average waiting time W for the computation task request q It is expressed as follows:

[0062]

[0063]

[0064]

[0065] Where p0 represents the probability of no computational task, ρ = λ / μ represents the computational service intensity, and ρ s =ρ / s.

[0066] The heterogeneous computing resource scheduling method provided in this application uses a classic queuing model to simulate the computing task request process when constructing the computing task waiting model. Different computing task request processes will have a huge impact on the heterogeneous computing resource scheduling scheme. By using queuing theory to analyze and model the computing task request process of heterogeneous computing resources, a more accurate simulation of the arrival process of multiple task requests is achieved.

[0067] In some embodiments, constructing a multi-task adaptive scheduling model based on reinforcement learning specifically includes:

[0068] A multi-task adaptive scheduling model based on deep Q-learning is constructed. This model is used to produce corresponding action values ​​based on the computing resource status and computing resource scheduling actions of each agent.

[0069] Specifically, Q-learning is a model-free reinforcement learning technique. Q stands for Action-Utility Function, which is used to evaluate the merits of taking a certain action in a specific state and is the agent's memory.

[0070] Q-learning uses a Q-table to store the value pairs of states and actions. Initially, the Q-table is given a random initial value, and then iteratively converges until it stabilizes. The Q-learning algorithm can find an optimal policy in a finite Markov decision process that maximizes the expected overall reward. At each time point t, the agent chooses an action a. t Receive a reward R t At this point, the environment enters a new state s. t+1 Then the Q-value is updated. The core of Q-learning is the value function iteration process, that is:

[0071] Q(s t ,a t )←Q(s t ,a t )+α[r t +γmaxQ(s t+1 ,a t+1 )-Q(s t ,a t )]

[0072] Here, α represents the learning rate, and γ represents the discount factor. Q-learning performs well in most cases with limited state spaces, but as the state and action spaces expand, the storage space required for Q-learning also increases, leading to storage and computational difficulties. Therefore, Deep Q-learning (DQN) is used to solve the problem for scheduling heterogeneous computing resources.

[0073] DQN is a value-based, model-free reinforcement learning algorithm that improves upon the Q-learning algorithm by using a neural network to obtain the Q-values ​​of states and actions. This allows DQN to perform well even with large state and action spaces. The DQN algorithm will be described in detail below.

[0074] Value function networks (VQNs) contain a key concept: value function approximation. An action-state value pair corresponds to a value function Q(s,a). Theoretically, action and state values ​​can be stored in a table, and any Q(s,a) can be retrieved by looking up the table. However, when the number of states and actions is large, the table becomes very large, and the solution and iteration processes become lengthy. Therefore, a function approximation approach is proposed to estimate state-action values; that is, the function takes actions and values ​​as input and outputs the value corresponding to that state-action pair.

[0075]

[0076] By using function approximation, even for some unseen state-action pairs, their Q-values ​​can still be predicted.

[0077] The input parameters of DQN can be divided into two parts: one is a representation of the computing resource state, with a dimension of k+v×(k+1); the other is the computing resource scheduling action, with a dimension of v. The output dimension is 1, and the output value represents the action value corresponding to the computing resource state s and the computing resource scheduling action a.

[0078] Figure 3 This is a schematic diagram illustrating the principle of the multi-task adaptive scheduling model provided in the embodiments of this application, combined with... Figure 3 The heterogeneous computing resource scheduling algorithm provided in this application embodiment is described as follows:

[0079] The first step is to initialize the memory playback space D, which is used to store the traversed data.

[0080] The second step is to determine the number v of heterogeneous computing resource core types, thereby determining the input structure of the neural network. Then, the Q-network parameters w are randomly initialized, and the target Q-network parameters w are also randomly initialized. - =w:

[0081] y1 = relu(w1z1 + b1)

[0082] Q = relu(w²y¹ + b²)

[0083] Where z1 is the input pair of computing resource status and computing resource scheduling action, b1 and b2 are bias vectors, and Q is the final output value.

[0084] The third step is to initialize the initial computing resource state space based on the limitations of computing resources, where the number of computing resources is randomly initialized.

[0085] The fourth step is to place the computing resource request task into the computing request task queue.

[0086] Furthermore, this describes the training process for a multi-task adaptive scheduling model based on deep Q-learning.

[0087] In some embodiments, the deep Q-learning-based multi-task adaptive scheduling model is trained using the following steps:

[0088] Based on the observed computing resource status, a greedy strategy (ε-greedy) is used to generate computing resource scheduling action a. t At the same time, it receives a computing resource scheduling reward r t and new computing resource states t+1 Then the sample group is placed into the memory space D;

[0089] Stochastic batch gradient descent is used to train and update the deep Q-learning-based multi-task adaptive scheduling model until the model converges.

[0090] Specifically, the model training process is as follows:

[0091] The first step is to initialize the memory space D with a capacity of N.

[0092] The second step is to determine the types and quantities of computing resources, v, randomly initialize the neural network parameters w and the Q-network, and initialize the target Q-network parameters w. - =w.

[0093] The third step is to initialize state s1.

[0094] The fourth step is to place the computing resource requests into the computing task request queue and preprocess the computing resource request tasks.

[0095] Fifth step, use the ε-greedy strategy to generate computational resource scheduling action a. t : Randomly select a computational resource scheduling action with probability ε, or select a t =max a Q(s t ,a,w).

[0096] Step 6: Execute computing resource scheduling action a t Obtain computing resource scheduling reward r t And new computing resource states t+1 .

[0097] Step 7, transfer the sample (s) t ,a t ,r t ,s t+1 Place it into memory playback space D.

[0098] Step 8: Randomly select a minibatch (s) from the memory replay space D. j ,a j ,r j ,s j+1 ).

[0099] In the ninth step, if step j+1 is the termination step, let y j =r j ;otherwise,

[0100] Step 10, for (y) j -Q(s t ,a j ,w)) 2 Regarding the use of stochastic batch gradient descent to update w, the target Q-network is updated every C steps. - =w.

[0101] Once the model has converged, the specific action for each step is selected using the greedy algorithm.

[0102] In some embodiments, constructing a reinforcement learning-based multi-task adaptive scheduling model in step S103 further includes:

[0103] The constraints are to minimize the waiting time of the computing task request queue and minimize the consumption of computing resources, and a resource constraint penalty term is added to the computing resource scheduling reward based on the constraints.

[0104] Specifically, in the process of computational resource scheduling, it is necessary to minimize the waiting time of computational task request queues while minimizing computational resource consumption. Considering that scheduling behavior is constrained by the upper limit of specific computational resources, a resource constraint penalty term is added to the computational resource scheduling reward to constrain unreasonable scheduling behavior and reduce such behavior. The updated reward formula is as follows:

[0105]

[0106] Where, r t Represents the reward for computational resource scheduling, QoS (SR) i ) represents the quality of service (QoS) of the computational task during the i-th scheduling process, and the calculation method has been given above; EC i This represents the computational resource consumption for the i-th type of task in a hard real-time environment. This represents the number of devices of type i at time t.

[0107] Here, w1 is a hyperparameter representing the penalty for adding or canceling computation task requests. Frequently adding or canceling computation task requests is not only unnecessary but also leads to computational resource consumption. This is added to the reward system to reduce the frequency of operations by the scheduling system. w2 is another hyperparameter, representing the penalty applied when, during computational resource scheduling, the scheduling action increases the computation task request queue, causing the required physical computational resources to exceed the limit. t The specific definitions are as follows:

[0108]

[0109] Among them, c t c represents the extent to which the required computational resources at time t exceed the upper limit. When the required computational resources are less than the available resources, c... t It is 0.

[0110] In summary, by penalizing unreasonable scheduling behavior through resource constraint penalties, the problem of physical resource constraints in computational resource scheduling is solved.

[0111] The heterogeneous computing resource scheduling method based on multi-objective optimization provided in this application adopts a stochastic batch gradient descent algorithm to update the multi-task adaptive scheduling model based on deep Q-learning when constructing an improved reinforcement learning model based on resource reservation. By reducing the switching cost of the scheduling model's execution scheduling strategy through the reinforcement learning model, the computational amount of the multi-task adaptive scheduling model based on deep Q-learning is reduced, and the convergence speed of the multi-task adaptive scheduling model based on deep Q-learning is improved.

[0112] In some embodiments, regarding the construction of a Markov decision process model: the computational resource scheduling process satisfies the Markov property, that is, for any computational resource scheduling at any time, the next computational resource scheduling state is only related to the current computational resource status and computational task requests, and is not related to the past computational resource utilization and past computational task requests.

[0113] Therefore, the heterogeneous computing resource scheduling problem and resource mapping process for multi-task applications are constructed as a Markov Decision Process (MDP), and described by the following quintuple:

[0114] E = {S, A, π, R, G}

[0115] In this context, the agent resides in environment E, S represents the state space of computing resources, A represents the action space, which is the set of available computing resource scheduling actions for the agent under the computing resource state, π represents the computing resource scheduling policy, which is the mapping function from the computing resource state to the computing resource scheduling action, R represents the computing resource scheduling reward, which is the feedback from the environment to the agent after the agent gives a computing resource scheduling action, and G represents the reward, which is the accumulation of the computing resource scheduling reward over time.

[0116] Computing resource status: Defined as a vector consisting of the quantity of each type of computing device resource and the amount of computing resources consumed by each type of computing device. The complete definition is as follows:

[0117]

[0118] in, Let represent the remaining amount of the i-th type of computing resources at time t, where the total number of computing resources is k; This represents the number of idle computing resources of type l at time t.

[0119] Computational resource scheduling actions: Computational resource scheduling action a t This indicates the creation or destruction of different types of computing task requests. After completing the computing resource scheduling action, the computing resource scheduling system will start from state s at time t. t Transition to the new state s in the next moment t+1 At the same time, the capacity of different types of idle computing resources in the computing resource scheduling system changes, i.e., from... arrive The current remaining computing resources in the computing resource scheduling system are from Change to

[0120] Computational resource scheduling reward: Computational resource scheduling reward r t =r(s t ,a t ,s t+1 ) is in state s t The following uses computational resource scheduling action a t The feedback subsequently obtained from the computing resource scheduling system. The computing resource scheduling reward function defines the reward value received by the computing resource scheduling system, with the ultimate goal of maximizing the overall cumulative reward value, as shown below:

[0121]

[0122] Among them, QoS (SR) i EC represents the quality of service (QoS) of the computational task during the i-th scheduling process. i This represents the computational resource consumption for the i-th type of task in a hard real-time environment. Let represent the number of devices of type i at time t. By penalizing unreasonable scheduling behavior through the aforementioned resource constraint penalty term, the problem of physical resource constraints in computational resource scheduling can be solved.

[0123] Computational resource scheduling strategy: The computational resource scheduling strategy π is the scheduler behavior function, which is a mapping from the state of computational resources to the scheduling actions of computational resources. It is determined by inputting the current resource state s. t The mapping function will output the corresponding computing resource scheduling action 'a'. t During the computational resource scheduling process, the status of computational resources represents the current state of the computational resource scheduling system. When the computational resource scheduling system performs an operation, these indicator values ​​will change, that is, enter the next state.

[0124] The heterogeneous computing resource scheduling method based on multi-objective optimization provided in this application realizes adaptive scheduling of heterogeneous computing resources for multiple tasks by constructing a multi-objective optimization model for heterogeneous computing resources, modeling the arrival process of computing request tasks, modeling resource scheduling and resource mapping, and using an improved reinforcement learning model based on resource reservation. This solves the problems of high scheduling overhead and low resource utilization in heterogeneous computing resource environments.

[0125] Figure 4 This is a schematic diagram of the heterogeneous computing resource scheduling device based on multi-objective optimization provided in the embodiments of this application, as shown below. Figure 4 As shown, the device includes at least:

[0126] The first building module 401 includes:

[0127] The first construction submodule is used to select at least two performance indicators as optimization objectives from the two dimensions of tasks and resources, and to establish a multi-objective optimization model for heterogeneous computing resource scheduling.

[0128] The second construction submodule is used to transform the computation task request process into a computation task waiting model based on queuing theory.

[0129] The third construction submodule is used to build a multi-task adaptive scheduling model based on reinforcement learning based on the observed resource status.

[0130] The second building module 402 is used to construct the heterogeneous computing resource scheduling problem and resource mapping process for multi-task applications into a Markov decision process model based on a multi-objective optimization model and a computing task waiting model.

[0131] The scheduling module 403 is used to achieve adaptive scheduling of heterogeneous computing resources for multiple tasks based on the Markov decision process model and the trained multi-task adaptive scheduling model.

[0132] In some embodiments, the first construction submodule is specifically used for:

[0133] Define the constraints for the multi-objective optimization model;

[0134] From the task dimension, task request response time, computing resource scheduling overhead and task throughput are selected as optimization objectives, and a task model is established. The task model represents the service quality of computing tasks.

[0135] From the resource dimension, the utilization rate of computing resources is selected as the optimization objective, and a resource model is established to represent the utilization rate of computing resources.

[0136] Based on the task model and resource model, the objective function of the multi-objective optimization model is determined.

[0137] In some embodiments, the objective function satisfies the following calculation formula:

[0138]

[0139] Where f(SR) represents the objective function, SR i This represents the i-th scheduling process assigned to the computation task, QoS(SR) i ) represents the quality of service (QoS) of the computational task during the i-th scheduling process, UR(SR) i Let ) represent the resource utilization rate corresponding to the i-th scheduling process, and let α and β represent the weight values ​​and satisfy α+β=1.

[0140] In some embodiments, the third construction submodule is specifically used for:

[0141] A multi-task adaptive scheduling model based on deep Q-learning is constructed. This model is used to produce corresponding action values ​​based on the computing resource status and computing resource scheduling actions of each agent.

[0142] In some embodiments, the reinforcement learning-based multi-task adaptive scheduling model is trained using the following steps:

[0143] Based on the observed computing resource status, a greedy strategy is used to generate computing resource scheduling actions, computing resource scheduling rewards, and new computing resource statuses, and the sample group is placed into the memory space;

[0144] Stochastic batch gradient descent is used to train and update the deep Q-learning-based multi-task adaptive scheduling model until the model converges.

[0145] In some embodiments, a greedy strategy is used to generate computing resource scheduling actions, computing resource scheduling rewards, and new computing resource states, including:

[0146] The greedy strategy is the ε-greedy strategy, which uses ε-greedy to generate computational resource scheduling actions;

[0147] Randomly select a computational resource scheduling action with probability ε, or select the computational resource scheduling action corresponding to the maximum state action value;

[0148] Execute the selected computing resource scheduling action to obtain computing resource scheduling rewards and a new computing resource status.

[0149] In some embodiments, the third construction submodule is also used for:

[0150] The constraints are to minimize the waiting time of the computing task request queue and minimize the consumption of computing resources, and a resource constraint penalty term is added to the computing resource scheduling reward based on the constraints.

[0151] In some embodiments, the heterogeneous computing resource scheduling problem for multi-task applications is constructed as a Markov decision process model, satisfying the following quintuple:

[0152] E = {S, A, π, R, G}

[0153] In this context, the agent resides in environment E, S represents the state space of computing resources, A represents the action space, which is the set of available computing resource scheduling actions for the agent under the computing resource state, π represents the computing resource scheduling policy, which is the mapping function from the computing resource state to the computing resource scheduling action, R represents the computing resource scheduling reward, which is the feedback from the environment to the agent after the agent gives a computing resource scheduling action, and G represents the reward, which is the accumulation of the computing resource scheduling reward over time.

[0154] It is understood that the detailed functional implementation of each of the above units / modules can be found in the description in the aforementioned method embodiments, and will not be repeated here.

[0155] It should be understood that the above-described device is used to execute the methods in the above embodiments. The implementation principle and technical effect of the corresponding program modules in the device are similar to those described in the above methods. The working process of the device can be referred to the corresponding process in the above methods, and will not be repeated here.

[0156] Based on the methods described in the above embodiments, this application provides an electronic device. The device may include at least one memory for storing a program and at least one processor for executing the program stored in the memory. When the program stored in the memory is executed, the processor performs the methods described in the above embodiments.

[0157] Figure 5 This is a schematic diagram of the structure of the electronic device provided in the embodiments of this application, such as... Figure 5As shown, the electronic device may include a processor 501, a communications interface 520, a memory 503, and a communication bus 504. The processor 501, communications interface 502, and memory 503 communicate with each other via the communication bus 504. The processor 501 can call software instructions stored in the memory 503 to execute the methods described in the above embodiments.

[0158] Furthermore, the logical instructions in the aforementioned memory 503 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to related technologies, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods in the various embodiments of this application.

[0159] Based on the methods in the above embodiments, this application provides a computer-readable storage medium storing a computer program that, when run on a processor, causes the processor to execute the methods in the above embodiments.

[0160] Based on the methods in the above embodiments, this application provides a computer program product that, when run on a processor, causes the processor to execute the methods in the above embodiments.

[0161] It is understood that the processor in the embodiments of this application can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. A general-purpose processor can be a microprocessor or any conventional processor.

[0162] The method steps in this application embodiment can be implemented in hardware or by a processor executing software instructions. The software instructions can consist of corresponding software modules, which can be stored in random access memory (RAM), flash memory, read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), registers, hard disks, portable hard disks, CD-ROMs, or any other form of storage medium known in the art. An exemplary storage medium is coupled to the processor, enabling the processor to read information from and write information to the storage medium. Of course, the storage medium can also be a component of the processor. The processor and the storage medium can reside in an ASIC.

[0163] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented, in whole or in part, as a computer program product. A computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the flow or function according to the embodiments of this application is generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in or transmitted through a computer-readable storage medium. The computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid-state disk (SSD)).

[0164] It is understood that the various numerical designations used in the embodiments of this application are merely for the convenience of description and are not intended to limit the scope of the embodiments of this application.

[0165] Those skilled in the art will readily understand that the above are merely preferred embodiments of this application and are not intended to limit this application. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this application should be included within the scope of protection of this application.

Claims

1. A heterogeneous computing resource scheduling method based on multi-objective optimization, characterized in that, include: A multi-objective optimization model for heterogeneous computing resource scheduling is established by selecting at least two performance indicators as optimization objectives from the two dimensions of task and resources. Based on queuing theory, the process of requesting computational tasks is transformed into a model of waiting for computational tasks. Based on the observed resource status, a multi-task adaptive scheduling model based on reinforcement learning is constructed. Based on the multi-objective optimization model and the computing task waiting model, the heterogeneous computing resource scheduling problem and resource mapping process for multi-task applications are constructed as a Markov decision process model. Based on the Markov decision process model and the trained multi-task adaptive scheduling model, the constraint is to minimize the waiting time of the computing task request queue while minimizing the consumption of computing resources. Based on the constraint, a resource constraint penalty term is added to the computing resource scheduling reward to achieve adaptive multi-task heterogeneous computing resource scheduling. The construction of the multi-task adaptive scheduling model based on reinforcement learning includes: constructing a multi-task adaptive scheduling model based on deep Q-learning, wherein the multi-task adaptive scheduling model based on deep Q-learning is used to produce corresponding action values ​​according to the computing resource status and computing resource scheduling actions of each agent.

2. The heterogeneous computing resource scheduling method based on multi-objective optimization according to claim 1, characterized in that, The establishment of a multi-objective optimization model for heterogeneous computing resource scheduling includes: Define the constraints for the multi-objective optimization model; From the task dimension, task request response time, computing resource scheduling overhead and task throughput are selected as optimization objectives, and a task model is established, which represents the service quality of computing tasks. From the resource dimension, the utilization rate of computing resources is selected as the optimization objective, and a resource model is established, which represents the utilization rate of computing resources. Based on the task model and the resource model, the objective function of the multi-objective optimization model is determined.

3. The heterogeneous computing resource scheduling method based on multi-objective optimization according to claim 2, characterized in that, The objective function satisfies the following calculation formula: in, Denotes the objective function, Represents the first task assigned to the computation task. i This scheduling process Indicates the first i The quality of service for tasks is calculated during each scheduling process. Indicates the first i Resource utilization rate corresponding to this scheduling process Represents the weight value and satisfies .

4. The heterogeneous computing resource scheduling method based on multi-objective optimization according to claim 3, characterized in that, The reinforcement learning-based multi-task adaptive scheduling model is trained using the following steps: Based on the observed computing resource status, a greedy strategy is used to generate computing resource scheduling actions, computing resource scheduling rewards, and new computing resource statuses, and the sample group is placed into the memory space; The multi-task adaptive scheduling model based on deep Q-learning is trained and updated using stochastic batch gradient descent until the model converges.

5. The heterogeneous computing resource scheduling method based on multi-objective optimization according to claim 4, characterized in that, The process of generating computing resource scheduling actions, computing resource scheduling rewards, and new computing resource states using a greedy strategy includes: The greedy strategy is as follows: Strategy, Adopt Generate computing resource scheduling actions; by A computational resource scheduling action is randomly selected with probability, or the computational resource scheduling action corresponding to the maximum state action value is selected. Execute the selected computing resource scheduling action to obtain computing resource scheduling rewards and a new computing resource status.

6. The heterogeneous computing resource scheduling method based on multi-objective optimization according to claim 1, characterized in that, The heterogeneous computing resource scheduling problem for multi-task applications is constructed as a Markov decision process model, satisfying the following quintuple: Wherein, the agent is in environment E, S represents the state space of computing resources, and A represents the action space, which is the set of available computing resource scheduling actions of the agent in the state of computing resources. It represents the computing resource scheduling strategy and is a mapping function from the computing resource state to the computing resource scheduling action; This represents the reward for scheduling computing resources, which is the feedback from the environment to the agent after the agent performs a computing resource scheduling action. The reward represents the accumulation of computing resource scheduling rewards over time.

7. A heterogeneous computing resource scheduling device based on multi-objective optimization, characterized in that, include: The first building block includes: The first construction submodule is used to select at least two performance indicators as optimization objectives from the two dimensions of tasks and resources, and to establish a multi-objective optimization model for heterogeneous computing resource scheduling. The second construction submodule is used to transform the computation task request process into a computation task waiting model based on queuing theory. The third construction submodule is used to construct a multi-task adaptive scheduling model based on deep Q-learning based on the observed resource status, and to produce corresponding action values ​​according to the computing resource status and computing resource scheduling actions of each agent through the multi-task adaptive scheduling model. The second construction module is used to construct the heterogeneous computing resource scheduling problem for multiple tasks into a Markov decision process model based on the multi-objective optimization model and the computing task waiting model. The scheduling module is used to minimize the waiting time of the computing task request queue and the computing resource consumption based on the Markov decision process model and the trained multi-task adaptive scheduling model, and to add resource constraint penalties to the computing resource scheduling reward based on the constraints.

8. An electronic device, characterized in that, include: At least one memory for storing computer programs; At least one processor is configured to execute a program stored in the memory, wherein when the program stored in the memory is executed, the processor is configured to perform the method as described in any one of claims 1-6.

Images