Time-varying workload scheduling method based on priority and resource awareness
The cloud server scheduling method addresses inefficiencies in existing workload scheduling by using a classifier, queue scorer, and parallel GAT layer to enhance reinforcement learning, resulting in improved resource utilization and reduced waiting times.
Patent Information
- Application Number
- CN202411010621.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-26
- Publication Date
- 2024-12-06
- Estimated Expiration
- 2044-07-26
AI Technical Summary
Existing methods for time-varying workload scheduling fail to address the efficient allocation of workloads in the cloud fail to efficiently utilize cloud server scheduling fail to efficiently utilize cloud server cluster, and the cloud server scheduling method based on priority and resource awareness, which enhances the efficiency of workload scheduling by using a pre-trained classifier and queue scorer to guide reinforcement learning, and incorporates a parallel GAT layer to capture dimensional and temporal characteristics of the cluster, improving resource utilization and predicting future changes.
The method includes obtaining the cloud server cluster status and workload set, using a classifier to assign labels, constructing a priority queue state matrix, and employing a parallel graph attention layer and gated recurrent unit to extract the hidden state, followed by a reinforcement learning-based policy network to determine scheduling actions, with a reward system to optimize the process.
The method maximizes cluster resource utilization, minimizes resource fragmentation, and reduces workload waiting time, thereby improving scheduling efficiency and meeting user needs effectively.
Smart Images

Figure CN119088539A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of time-varying workload scheduling, and in particular to a time-varying workload scheduling method based on priority and resource perception. Background Art
[0002] Nowadays, many enterprises are migrating their user-generated workloads, such as web requests, application tasks, and data processing jobs, to cloud server clusters to provide services. Cloud Service Providers (CSPs) manage the operation of cloud server clusters, provide users with pay-per-use cloud resource access, and save costs, improve service reliability, and improve service performance for enterprises. Workloads arrive and leave cloud server clusters dynamically in an online manner, and their resource requirements change dynamically over time. The scheduler considers the time-varying resource requirements of workloads and the status of the server cluster, and then allocates workloads to designated servers for processing according to its scheduling strategy, striving to achieve excellent results on key indicators such as cluster resource utilization. This process is called time-varying workload scheduling. Due to the heterogeneity of cloud servers, the dispersed distribution of available resources, and the time-varying characteristics of workloads, the solution of time-varying workload scheduling is somewhat complex.
[0003] At present, exact algorithms, heuristic algorithms, and learning-based methods are commonly used to solve the time-varying workload scheduling problem. Exact algorithms aim to find the optimal solution, but are usually not applicable to large-scale clusters due to computing requirements. Heuristic algorithms are either built based on empirical rules or natural phenomena, or use prior knowledge to model cluster environments and workload resource requirements to design scheduling strategies. They rely too much on manual settings, resulting in a lack of interaction with dynamic environments. Learning-based methods can automatically learn scheduling strategies from a large amount of historical data, among which reinforcement learning (RL) can continuously interact with the dynamic cluster environment, allowing the scheduler to explore various decisions and optimize strategies through reward signals, thus becoming the preferred method for developing adaptive workload scheduling frameworks. However, existing RL-based methods often rely on first-come, first-served or preset priorities when scheduling, which ignores the current state of resource utilization in the server cluster, limits the scheduler's choice of scheduling order, and hinders the discovery of more optimal solutions and the improvement of scheduling efficiency. Another important challenge is that these methods have defects in fully perceiving the time utilization of multi-dimensional resources in the cluster, and cannot accurately predict future changes in cluster status, resulting in potential convergence to suboptimal solutions. Therefore, how to design a time-varying workload scheduling algorithm that can effectively utilize priority and resource awareness becomes a challenge. Summary of the invention
[0004] The purpose of the present invention is to provide a time-varying workload scheduling method based on priority and resource awareness, comprising the following steps:
[0005] 1) Obtain the cloud server cluster status and the workload set W that needs to be scheduled;
[0006] 2) Determine whether the workload set W is successfully scheduled, if so, proceed to step 20), otherwise, proceed to step 3);
[0007] 3) Determine whether the preset time step threshold T is reached, if so, proceed to step 14), otherwise, proceed to step 4);
[0008] 4) Using classifier f c For the current workload w i Assign label c i ;
[0009] 5) Use the queue scorer to evaluate the workload in W and get the workload score
[0010] 6) Construct the priority queue state matrix S with the top K workloads with the highest scores Q ∈Z K×(1+D) ; Among them, the priority queue state matrix S Q ∈Z K×(1+D) The first column of is the assigned label, and the other D columns represent the required resources of the workload;
[0011] 7) Use parallel graph attention layer and gated recurrent unit to extract the hidden state S of the server cluster C ;
[0012] 8) Hidden state S of the connected server cluster C , priority queue state matrix S Q , forming the final state S at time step t t ;
[0013] 9) The final state S t Input into the reinforcement learning-based policy network and output the scheduling action A t ;
[0014] 10) Determine whether the scheduling action is valid, if yes, proceed to step 11), otherwise proceed to step 13);
[0015] 11) Schedule the workload at the head of the priority queue to action A t In the specified server;
[0016] 12) The final state S t Transfer to Cluster environment based on Generate reward R t , regard the scheduling process as trajectory e=1, and record (S t ,A t ,R t ) tuple, return to step 5);
[0017] 13) Increment the time step by 1, put all new workload requests that will be introduced into the cluster into W, and return to step 3).
[0018] 14) Determine whether the preset number of training iterations has been reached, if so, proceed to step 19), otherwise proceed to step 15).
[0019] 15) Initialize the cloud server cluster status and workload request, consistent with step 2);
[0020] 16) Carry out multiple simulation experiments and record the corresponding trajectories e=2,...,N;
[0021] 17) Calculate all trajectory gradients and add all gradients to update the network weights of the policy network and parallel graph attention layer;
[0022] 18) Increase the number of iterations by 1 and return to step 14).
[0023] 19) Initialize the time step to 1 and return to step 2).
[0024] 20) All workloads have been successfully scheduled, and the process ends. If a new workload request arrives at the cluster, return to step 1).
[0025] Furthermore, the cloud server cluster includes J servers, denoted as {s1,...,s j ,...,s J};
[0026] Each server has D resource dimensions;
[0027] For each server j , the resource capacity in the resource dimension d is heterogeneous, denoted as c j,d ;
[0028] At time step t, the amount of available resources of the server in resource dimension d is expressed as
[0029] Furthermore, among the workload set W that needs to be scheduled, workload w i The time-varying resource demand of ∈W is defined as in represents the number of resources requested across D dimensions at the lth execution time step. iRepresents the execution time step.
[0030] Furthermore, the classifier f c The construction steps are as follows:
[0031] a1) Collect a set of historical workloads from the cluster history trace;
[0032] a2) using dynamic time warping algorithm to calculate the distance metric between different workloads;
[0033] Among them, the workload w i and w g The distance measure between As shown below:
[0034]
[0035] Among them, the parameters The workload w i and w g time-varying resource requirements; Time-varying resource requirements The minimum distance measure between ; Time-varying resource requirements The minimum distance measure between Time-varying resource requirements The minimum distance measure between Time-varying resource requirements The minimum distance measure between ; for time-varying resource requirements;
[0036] a3) Clustering workloads using a density-based clustering algorithm based on distance metrics to generate C equivalence classes, where workloads that are densely connected form one equivalence class;
[0037] a4) Use the workload and the corresponding class label as training data to train the classifier f c .
[0038] Furthermore, the method to generate C equivalence class is:
[0039] For workload w i , if its ∈ neighborhood contains at least minPts workloads, then workload w i As a core point;
[0040] If the workload w i It is the core point and It is believed that from w i to w g The path is directly density-reachable; ∈ is the preset distance threshold;
[0041] Given a core point w i ,w g ,w e , if w i to w g and from w g to w e The paths are all directly density-reachable, then w i to w e The path is density-reachable;
[0042] If a core point makes the g to w i 、w g to w e is density-reachable, then w i and w e are defined as density connected.
[0043] Furthermore, the workload w i The fraction q i As shown below:
[0044]
[0045] In the formula, α1, α2, and α3 are constants that control the importance of different components to the score; Indicates that at time step t, server s j The amount of resources available in resource dimension d; represents the workload w i The number of resources requested on the resource dimension d at the first execution time step; t represents the current time step; t i represents the workload w i At time step t i Arrival cluster; N ci Indicates that the class c is currently running in the cluster i The number of workloads; Ι()=0,1 is the indicator function; B(t) represents the servers running in the cluster at time step t.
[0046] Furthermore, the hidden state S of the server cluster is extracted using the parallel graph attention layer and gated recurrent unit. C The steps include:
[0047] b1) Using one-dimensional convolution from each server s j Extracting high-level features from resource usage time series j ,Right now:
[0048] v j =conv(o j ) (4)
[0049] Among them, the parameters M is the number of historical time steps read by the server, in horizontal pixels Indicates server j The resource utilization status within; conv() represents the convolution operation;
[0050] b2) Capture high-level features V = [v1, ..., v j ,...,v J ], and obtain the dimension graph attention state h feat ;
[0051] The time-oriented parallel graph attention layer is used to capture the temporal dependencies in high-level features and obtain the temporal graph attention state h time ;
[0052] Among them, the dimension graph attention state h feat As shown below:
[0053] h feat =[h 1 ,...,h j ,...,h J ](5)
[0054]
[0055] In the formula, σ represents the sigmoid activation function, α jg is the attention score, represents the set of adjacent nodes of node j; h j is the attention state of node j; v g For advanced features;
[0056] Attention score α jg As shown below:
[0057]
[0058]
[0059] In the formula, LeakeyReLU represents the activation function; ω represents the weight parameter; v j 、v g is an advanced feature; e jg is an intermediate parameter; intermediate parameter
[0060] Time series diagram of attention state h time As shown below:
[0061] h time =[h 1 ,...,hm ,...,h M ](9)
[0062]
[0063] In the formula, α mp is the attention score;
[0064] b3) Connection dimension graph attention state h feat , time series diagram attention state h time , and obtain the feature matrix; then, the feature matrix is input into the gated recurrent unit to obtain the hidden state S of the server cluster C ,Right now:
[0065]
[0066] Where GRU stands for gated recurrent unit.
[0067] Further, scheduling action A t The action space is A t =j means assigning the first workload in the priority queue to the jth server; Represents an “invalid” action, meaning that the scheduler refuses to schedule this workload in the current time step.
[0068] Further, reward R t =-P C -P U -P W ;
[0069] Among them, resource contention penalty P C As shown below:
[0070]
[0071] In the formula, K c is a constant that determines the weight of the resource contention penalty;
[0072] Among them, the cross-resource dimension d and server s j The relevant contention score C r (s j ,d) as shown below:
[0073]
[0074] Among them, s j,W Indicates that on the server j The workload set running on i,d is a vector representing the server s j Workload w iResource usage time series along d; r g,d represents the workload w g time series of resource usage along d;
[0075] Underutilization penalty P U As shown below:
[0076]
[0077] Among them, the constant term K u is the scale coefficient;
[0078] Waiting time penalty P W As shown below:
[0079] P W =K w *|W|(15)
[0080] Among them, K w is a constant; |W| represents the total number of workloads to be scheduled in the cluster.
[0081] Further, the steps of calculating all trajectory gradients include:
[0082] c1) Construct the trajectory expression, namely:
[0083]
[0084] In the formula, trace e Indicates trajectory; represents the time step T e Corresponding states, actions, and rewards;
[0085] c2) Calculate the cumulative reward for each step in all trajectories, that is:
[0086]
[0087] Where γ is the decay factor used to calculate the cumulative reward. Time step x = t, ..., T e ; represents the reward corresponding to time step x;
[0088] c3) Calculate the baseline z based on the cumulative reward at each step in all trajectories t ,Right now:
[0089]
[0090] Where N is the number of trajectories;
[0091] c4) Calculate the gradient Δθ of all trajectories based on the baseline, that is:
[0092]
[0093] Where α is the learning rate.
[0094] The technical effect of the present invention is unquestionable, and the present invention solves the problem of time-varying workload scheduling. The present invention generates a priority queue for the incoming workload by introducing a pre-trained classifier and a queue scorer, and makes full use of the priority queue to guide and improve the efficiency of reinforcement learning in solving the problem of time-varying workload scheduling; in order to better perceive cluster resources, a parallel GAT layer is proposed, so that it can effectively capture the dimensional and temporal characteristics of the cluster and accurately predict future changes in cluster status; in addition, a reinforcement learning strategy network training method with a baseline is introduced to enhance training stability and improve training efficiency. The present invention can maximize cluster resource utilization while minimizing cluster resource fragmentation rate, number of servers used, and workload waiting time, better meet user needs, and bring higher benefits to CSPs. BRIEF DESCRIPTION OF THE DRAWINGS
[0095] Figure 1 Flowchart of the overall algorithm for scheduling time-varying workloads. DETAILED DESCRIPTION
[0096] The present invention is further described below in conjunction with the embodiments, but it should not be understood that the above subject matter of the present invention is limited to the following embodiments. Without departing from the above technical ideas of the present invention, various substitutions and changes are made according to the common technical knowledge and customary means in the art, which should all be included in the protection scope of the present invention.
[0097] Embodiment 1:
[0098] See also Figure 1 ,The time-varying workload scheduling method based on priority and resource awareness includes the following steps:
[0099] 1) Obtain the cloud server cluster status and the workload set W that needs to be scheduled;
[0100] 2) Determine whether the workload set W is successfully scheduled, if so, proceed to step 20), otherwise, proceed to step 3);
[0101] 3) Determine whether the preset time step threshold T is reached, if so, proceed to step 14), otherwise, proceed to step 4);
[0102] 4) Using classifier f c For the current workload w i Assign label c i ;
[0103] 5) Use the queue scorer to evaluate the workload in W and get the workload score
[0104] 6) Construct the priority queue state matrix S with the top K workloads with the highest scores Q ∈Z k×(1+D) ; Among them, the priority queue state matrix S Q ∈Z K×(1+D) The first column of is the assigned label, and the other D columns represent the required resources of the workload;
[0105] 7) Use parallel graph attention layer and gated recurrent unit to extract the hidden state S of the server cluster C ;
[0106] 8) Hidden state S of the connected server cluster C , priority queue state matrix S Q , forming the final state S at time step t t ;
[0107] 9) The final state S t Input into the reinforcement learning-based policy network and output the scheduling action A t ;
[0108] 10) Determine whether the scheduling action is valid, if yes, proceed to step 11), otherwise proceed to step 13);
[0109] 11) Schedule the workload at the head of the priority queue to action A t In the specified server;
[0110] 12) The final state S t Transfer to Cluster environment based on Generate reward R t , regard the scheduling process as trajectory e=1, and record (S t ,A t ,R t ) tuple, return to step 5);
[0111] 13) Increment the time step by 1, put all new workload requests that will be introduced into the cluster into W, and return to step 3).
[0112] 14) Determine whether the preset number of training iterations has been reached, if so, proceed to step 19), otherwise proceed to step 15).
[0113] 15) Initialize the cloud server cluster status and workload request, consistent with step 2);
[0114] 16) Carry out multiple simulation experiments and record the corresponding trajectories e=2,...,N;
[0115] 17) Calculate all trajectory gradients and add all gradients to update the network weights of the policy network and parallel graph attention layer;
[0116] 18) Increase the number of iterations by 1 and return to step 14).
[0117] 19) Initialize the time step to 1 and return to step 2).
[0118] 20) All workloads have been successfully scheduled, and the process ends. If a new workload request arrives at the cluster, return to step 1).
[0119] The cloud server cluster consists of J servers, denoted as {s1,...,s j ,...,s J};
[0120] Each server has D resource dimensions;
[0121] For each server j , the resource capacity in the resource dimension d is heterogeneous, denoted as c j,d ;
[0122] At time step t, the amount of available resources of the server in resource dimension d is expressed as
[0123] In the workload set W that needs to be scheduled, workload w i The time-varying resource demand of ∈W is defined as in represents the number of resources requested across D dimensions at the lth execution time step. i Represents the execution time step.
[0124] Classifier f c The construction steps are as follows:
[0125] a1) Collect a set of historical workloads from the cluster history trace;
[0126] a2) using dynamic time warping algorithm to calculate the distance metric between different workloads;
[0127] Among them, the workload w i and w g The distance measure between As shown below:
[0128]
[0129] Among them, the parameters The workload w i and wg time-varying resource requirements; Time-varying resource requirements The minimum distance measure between ; Time-varying resource requirements The minimum distance measure between ; Time-varying resource requirements The minimum distance measure between ; Time-varying resource requirements The minimum distance measure between ; for time-varying resource requirements;
[0130] a3) Clustering workloads using a density-based clustering algorithm based on distance metrics to generate C equivalence classes, where workloads that are densely connected form one equivalence class;
[0131] a4) Use the workload and the corresponding class label as training data to train the classifier f c .
[0132] The method to generate C equivalence class is:
[0133] For workload w i , if its ∈ neighborhood contains at least minPts workloads, then workload w i As a core point;
[0134] If the workload w i It is the core point and It is believed that from w i to w g The path is directly density-reachable; ∈ is the preset distance threshold;
[0135] Given a core point w i ,w g ,w e , if w i to w g and from w g to w e The paths are all directly density-reachable, then w i to w e The path is density-reachable;
[0136] If a core point makes the g to w i 、w h to w e is density-reachable, then w i and w e are defined as density connected.
[0137] Workload i The fraction qi As shown below:
[0138]
[0139] In the formula, α1, α2, and α3 are constants that control the importance of different components to the score; Indicates that at time step t, server s j The amount of resources available in resource dimension d; represents the workload w i The number of resources requested on the resource dimension d at the first execution time step; t represents the current time step; t i represents the workload w i At time step t i Arrival cluster; Indicates that the class c is currently running in the cluster i The number of workloads; Ι()=0,1 is the indicator function; B(t) represents the servers running in the cluster at time step t.
[0140] Use parallel graph attention layers and gated recurrent units to extract the hidden state S of the server cluster C The steps include:
[0141] b1) Using one-dimensional convolution from each server s j Extracting high-level features from resource usage time series j ,Right now:
[0142] v j =conv(o j ) (4)
[0143] Among them, the parameters M is the number of historical time steps read by the server, in horizontal pixels Indicates server j The resource utilization status within; conv() represents the convolution operation;
[0144] b2) Capture high-level features V = [v1, ..., v j ,...,v J ], and obtain the dimension graph attention state h feat ;
[0145] The time-oriented parallel graph attention layer is used to capture the temporal dependencies in high-level features and obtain the temporal graph attention state h time ;
[0146] Among them, the dimension graph attention state h feat As shown below:
[0147] h feat =[h 1 ,...,h j ,...,h J ](5)
[0148]
[0149] In the formula, σ represents the sigmoid activation function, α jg is the attention score, represents the set of adjacent nodes of node j; h j is the attention state of node j; v g For advanced features;
[0150] Attention score α jg As shown below:
[0151]
[0152] In the formula, LeakeyReLU represents the activation function; ω represents the weight parameter; v j 、v g is an advanced feature; e jg is an intermediate parameter; intermediate parameter
[0153] Time series diagram of attention state h time As shown below:
[0154] h time =[h 1 ,...,h m ,...,h M ](9)
[0155]
[0156] In the formula, α mp is the attention score;
[0157] b3) Connection dimension graph attention state h feat , time series diagram attention state h time , and obtain the feature matrix; then, the feature matrix is input into the gated recurrent unit to obtain the hidden state S of the server cluster C ,Right now:
[0158]
[0159] Where GRU stands for gated recurrent unit.
[0160] Scheduling Action A t The action space is A t=j means assigning the first workload in the priority queue to the jth server; Represents an “invalid” action, meaning that the scheduler refuses to schedule this workload in the current time step.
[0161] Reward R t =-P C -P U -P W ;
[0162] Among them, resource contention penalty P C As shown below:
[0163]
[0164] In the formula, K c is a constant that determines the weight of the resource contention penalty;
[0165] Among them, the cross-resource dimension d and server s j The relevant contention score C r (s j ,d) as shown below:
[0166]
[0167] Among them, s j,W Indicates that on the server j The workload set running on i,d is a vector representing the server s j Workload w i Resource usage time series along d; r g,d represents the workload w g time series of resource usage along d;
[0168] Underutilization penalty P U As shown below:
[0169]
[0170] Among them, the constant term K u is the scale coefficient;
[0171] Waiting time penalty P W As shown below:
[0172] P W =K w *|W|(15)
[0173] Among them, K w is a constant; |W| represents the total number of workloads to be scheduled in the cluster.
[0174] The steps to calculate the gradients of all trajectories include:
[0175] c1) Construct the trajectory expression, namely:
[0176]
[0177] In the formula, trace e Indicates trajectory; represents the time step T e Corresponding states, actions, and rewards;
[0178] c2) Calculate the cumulative reward for each step in all trajectories, that is:
[0179]
[0180] Where γ is the decay factor used to calculate the cumulative reward. Time step x = t, ..., T e ; represents the reward corresponding to time step x;
[0181] c3) Calculate the baseline z based on the cumulative reward at each step in all trajectories t ,Right now:
[0182]
[0183] Where N is the number of trajectories;
[0184] c4) Calculate the gradient Δθ of all trajectories based on the baseline, that is:
[0185]
[0186] Where α is the learning rate.
[0187] Embodiment 2:
[0188] The time-varying workload scheduling method based on priority and resource awareness includes the following steps:
[0189] 1) Obtain the cloud server cluster status and the workload set W that needs to be scheduled;
[0190] 2) Determine whether the workload set W is successfully scheduled, if so, proceed to step 20), otherwise, proceed to step 3);
[0191] 3) Determine whether the preset time step threshold T is reached, if so, proceed to step 14), otherwise, proceed to step 4);
[0192] 4) Using classifier f c For the current workload w i Assign label c i ;
[0193] 5) Use the queue scorer to evaluate the workload in W and get the workload score
[0194] 6) Construct the priority queue state matrix S with the top K workloads with the highest scores Q ∈Z K×(1+D) ; Among them, the priority queue state matrix S Q ∈Z K×(1+D) The first column of is the assigned label, and the other D columns represent the required resources of the workload;
[0195] 7) Use parallel graph attention layer and gated recurrent unit to extract the hidden state S of the server cluster C ;
[0196] 8) Hidden state S of the connected server cluster C , priority queue state matrix S Q , forming the final state S at time step t t ;
[0197] 9) The final state S t Input into the reinforcement learning-based policy network and output the scheduling action A t ;
[0198] 10) Determine whether the scheduling action is valid, if yes, proceed to step 11), otherwise proceed to step 13);
[0199] 11) Schedule the workload at the head of the priority queue to action A t In the specified server;
[0200] 12) The final state S t Transfer to Cluster environment based on Generate reward R t , regard the scheduling process as trajectory e=1, and record (S t ,A t ,R t ) tuple, return to step 5);
[0201] 13) Increment the time step by 1, put all new workload requests that will be introduced into the cluster into W, and return to step 3).
[0202] 14) Determine whether the preset number of training iterations has been reached, if so, proceed to step 19), otherwise proceed to step 15).
[0203] 15) Initialize the cloud server cluster status and workload request, consistent with step 2);
[0204] 16) Carry out multiple simulation experiments and record the corresponding trajectories e=2,...,N;
[0205] 17) Calculate all trajectory gradients and add all gradients to update the network weights of the policy network and parallel graph attention layer;
[0206] 18) Increase the number of iterations by 1 and return to step 14).
[0207] 19) Initialize the time step to 1 and return to step 2).
[0208] 20) All workloads have been successfully scheduled, and the process ends. If a new workload request arrives at the cluster, return to step 1).
[0209] Embodiment 3:
[0210] The time-varying workload scheduling method based on priority and resource awareness has the same technical content as that of Example 2. Furthermore, the cloud server cluster includes J servers, denoted as {s1,...,s j ,...,s J};
[0211] Each server has D resource dimensions;
[0212] For each server j , the resource capacity in the resource dimension d is heterogeneous, denoted as c j,d ;
[0213] At time step t, the amount of available resources of the server in resource dimension d is expressed as
[0214] Embodiment 4:
[0215] The time-varying workload scheduling method based on priority and resource awareness has the same technical content as any one of Embodiments 2-3. Furthermore, in the workload set W that needs to be scheduled, the workload w i The time-varying resource demand of ∈W is defined as in represents the number of resources requested across D dimensions at the lth execution time step.
[0216] Embodiment 5:
[0217] The time-varying workload scheduling method based on priority and resource awareness has the same technical content as any one of Embodiments 2-4. Further, the classifier f c The construction steps are as follows:
[0218] a1) Collect a set of historical workloads from the cluster history trace;
[0219] a2) using dynamic time warping algorithm to calculate the distance metric between different workloads;
[0220] Among them, the workload w i and w g The distance metric between them is as follows:
[0221]
[0222] in, The workload w i and w g time-varying resource requirements;
[0223] a3) Clustering workloads using a density-based clustering algorithm based on distance metrics to generate C equivalence classes, where workloads that are densely connected form one equivalence class;
[0224] a4) Use the workload and the corresponding class label as training data to train the classifier f c .
[0225] Embodiment 6:
[0226] The time-varying workload scheduling method based on priority and resource awareness has the same technical content as any one of Embodiments 2-5. Further, the method for generating C equivalence classes is:
[0227] For workload w i , if its ∈ neighborhood contains at least minPts workloads, then workload w i As a core point;
[0228] If the workload w i It is the core point and It is believed that from w i to w g The path is directly density-reachable; ∈ is the preset distance threshold;
[0229] Given a core point w i ,w g ,w e , if w i to w g and from w g to w e The paths are all directly density-reachable, then w i to w e The path is density-reachable;
[0230] If a core point makes the g to w i 、w g to w e is density-reachable, then w i and w eare defined as density connected.
[0231] Embodiment 7:
[0232] The time-varying workload scheduling method based on priority and resource awareness has the same technical content as any one of Embodiments 2-6. Further, the workload w i The fraction q i As shown below:
[0233]
[0234] In the formula, α1, α2, and α3 are constants that control the importance of different components to the score; Indicates that at time step t, server s j The amount of resources available in resource dimension d; represents the workload w i The number of resources requested on the resource dimension d at the first execution time step; t represents the current time step; t i represents the workload w i At time step t i Arrival cluster; Indicates that the class c is currently running in the cluster i The number of workloads; Ι()=0,1 is the indicator function; B(t) represents the servers running in the cluster at time step t.
[0235] Embodiment 8:
[0236] The time-varying workload scheduling method based on priority and resource awareness has the same technical content as any one of Embodiments 2-7. Furthermore, the hidden state S of the server cluster is extracted by using a parallel graph attention layer and a gated recurrent unit. C The steps include:
[0237] b1) Using one-dimensional convolution from each server s j Extracting high-level features from resource usage time series j ,Right now:
[0238] v j =conv(o j ) (4)
[0239] in M is the number of historical time steps read by the server, in horizontal pixels Indicates server j The resource utilization status within
[0240] b2) Capture high-level features V = [v1, ..., v j ,...,v J], and obtain the dimension graph attention state h feat ;
[0241] The time-oriented parallel graph attention layer is used to capture the temporal dependencies in high-level features and obtain the temporal graph attention state h time ;
[0242] Among them, the dimension graph attention state h feat As shown below:
[0243] h feat =[h 1 ,...,h j ,...,h J ](5)
[0244]
[0245] In the formula, σ represents the sigmoid activation function, α jg is the attention score, represents the set of adjacent nodes of node j;
[0246] Attention score α jg As shown below:
[0247]
[0248]
[0249] Time series diagram of attention state h time As shown below:
[0250] h time =[h 1 ,...,h m ,...,h M ](9)
[0251]
[0252] In the formula, α mp is the attention score;
[0253] b3) Connection dimension graph attention state h feat , time series diagram attention state h time , and obtain the feature matrix; then, the feature matrix is input into the gated recurrent unit to obtain the hidden state S of the server cluster C ,Right now:
[0254]
[0255] Where GRU stands for gated recurrent unit.
[0256] Embodiment 9:
[0257] The time-varying workload scheduling method based on priority and resource awareness has the same technical content as any one of Embodiments 2-8. Further, the scheduling action A t The action space is A t =j means assigning the first workload in the priority queue to the jth server; Represents an “invalid” action, meaning that the scheduler refuses to schedule this workload in the current time step.
[0258] Embodiment 10:
[0259] A time-varying workload scheduling method based on priority and resource awareness, the technical content of which is the same as any one of Embodiments 2-9, further, a reward of R t =-P C -P U -P W ;
[0260] Among them, resource contention penalty P C As shown below:
[0261]
[0262] In the formula, K c is a constant that determines the weight of the resource contention penalty;
[0263] Among them, the cross-resource dimension d and server s j The relevant contention score C r (s j ,d) as shown below:
[0264]
[0265] Among them, s j,W Indicates that on the server j The workload set running on i,d is a vector representing the server s j Workload w i time series of resource usage along d;
[0266] Underutilization penalty P U As shown below:
[0267]
[0268] Among them, the constant term K u is the scale coefficient;
[0269] Waiting time penalty P W As shown below:
[0270] P W =K w *|W|(15)
[0271] Among them, K w is a constant; |W| represents the total number of workloads to be scheduled in the cluster.
[0272] Embodiment 11:
[0273] The time-varying workload scheduling method based on priority and resource awareness has the same technical content as any one of Embodiments 2-10. Further, the step of calculating all trajectory gradients includes:
[0274] c1) Construct the trajectory expression, namely:
[0275]
[0276] c2) Calculate the cumulative reward for each step in all trajectories, that is:
[0277]
[0278] Where γ is the decay factor used to calculate the cumulative reward.
[0279] c3) Calculate the baseline based on the cumulative reward at each step in all trajectories, that is:
[0280]
[0281] c4) Calculate the gradient of all trajectories according to the baseline, that is:
[0282]
[0283] Where α is the learning rate.
[0284] Embodiment 12:
[0285] The time-varying workload scheduling method based on priority and resource awareness has the following steps:
[0286] 1) Obtain the cloud server cluster status and the workload W that needs to be scheduled as input. The cloud server cluster {s1,...,s j ,...,s J} means that it consists of J servers, each of which has D resource dimensions, such as CPU, memory, etc. For each server s j , the resource capacity in the resource dimension d is heterogeneous, denoted as c j,d ; At time step t, the amount of available resources in resource dimension d is expressed as For the time step t i The workload w arriving at the clusteri ∈W, it requests n i The execution time step, the corresponding time-varying resource requirement can be defined as in represents the number of resources requested across D dimensions at the lth execution time step.
[0287] 2) Determine whether W has been successfully scheduled, if so, proceed to step 23), otherwise proceed to step 3).
[0288] 3) Determine whether the preset time step threshold T is reached, if yes, proceed to step 14), otherwise proceed to step 4).
[0289] 4) Use a classifier f c Equivalence class mapping is performed on the workloads in W, which helps the scheduler to distinguish and learn the resource usage characteristics between different workloads based on the generated class labels.
[0290] 4.1) To train f offline c , a set of workloads are collected from the cluster history. Then, the Dynamic Time Warping (DTW) algorithm is used to calculate the distance metric between workloads. i and w g The distance metric between is calculated as:
[0291]
[0292] The recursive formula for this calculation is:
[0293]
[0294] in
[0295] 4.2) Then, a density-based spatial clustering algorithm (DBSCAN) is applied to generate C equivalent classes. The algorithm requires two parameters: ∈ and minPts. For each workload w i , if its ∈ neighborhood contains at least minPts workloads, it is designated as a core point. First, if w i It is a core point and It is believed that from w i to w g The path is directly density-reachable. Secondly, given the core point w i ,w g ,w e , if from wi to w g and from w g to w e The paths of w are all directly density-reachable, so i to w e The path of w is considered density-reachable. Third, if a core point makes it possible to g to w i and w g to w e is density-reachable, then w i and w e is defined as density connected. Finally, all density connected points form an equivalence class. After establishing the equivalence class, all workloads and their respective class labels are used as classifier f c training data.
[0296] 4.3) During the scheduling process, the classifier f c A new workload is received and assigned a label c i To generate a refined representation of the workload:
[0297]
[0298] in
[0299] 5) At each time step t, a queue scorer is used to evaluate the workloads in W and rank each workload based on four key parameters: the dominant resources available on all servers in the cluster, the dominant resources required by the workload, the class of workload assignment, and the waiting time. i The fraction q i The calculation is as follows:
[0300]
[0301] The first term in the equation indicates whether the resource dimension d with the most idle resources in the cluster corresponds to the dominant resource required by the workload. If a match is established, the indicator function I() returns a value of 1, otherwise it returns 0, and B(t) represents the server running in the cluster at time step t. The variable N ci Indicates that the class c is currently running in the cluster i The number of workloads.
[0302] 6) Select the top K workloads with the highest scores and construct the priority queue state matrix S Q ∈Z K×(1+D) , where the first column is the assigned label and the subsequent D columns represent the required resources for the workload with the highest score.
[0303] 7) In order to effectively perceive the dimensional and temporal characteristics of resources in the server, a parallel Graph Attention (GAT) layer is designed, and the characteristics of each server are treated as a node in the graph.
[0304] 7.1) First, use one-dimensional convolution to extract j Extracting high-level features from resource usage time series j :
[0305] v j =conv(o j ),
[0306] in M is the number of historical time steps that the server can read, in horizontal pixels Indicates server j The resource utilization status within.
[0307] 7.2) Then V = [v1, ..., v j ,...,v J ] is processed by two GAT layers. The dimension-oriented GAT layer treats it as a complete graph, where each node represents a separate server feature v j , each edge represents the relationship between the corresponding servers. In this way, through the graph attention operation, the relationship between adjacent nodes can be carefully captured. Specifically, for each node v j A sequence vector v j =[v j,1 ,...,v j,m ,...,v j,M ] indicates that there are J nodes in total, and the resulting dimensional graph attention state is:
[0308] h feat =[h 1 ,...,h j ,...,h J ],
[0309]
[0310] Where σ represents the sigmoid activation function, α jg is the attention score, measuring the contribution of node g to node j, represents the set of neighboring nodes of node j. Attention score α jg It can be calculated as:
[0311]
[0312] 7.3) At the same time, a time-oriented GAT layer is used to capture the temporal dependencies in the time series. Specifically, a node v m =[v m,1 ,...,v m,j ,...,v m,J ] represents a feature vector at time step m, and its neighboring nodes contain all other time steps in the current sliding window. All nodes are modeled by a fully connected self-attention operation:
[0313] h time =[h 1 ,...,h m ,...,h M ],
[0314]
[0315] α mp The computation of is similar to the dimension-oriented GAT layer.
[0316] 7.4) Finally, the outputs from the two parallel GAT layers are concatenated to obtain an M×3J matrix, where each row represents a 3J-dimensional feature vector for each time step. This matrix is then fed into a Gated Recurrent Unit (GRU) layer, which is designed to capture the main sequential patterns. The output of the GRU layer represents the hidden state S of the server cluster. C :
[0317]
[0318] 8) Set the cluster status S C and the state S of the priority queue Q Connect them to form the final state S at time step t t .
[0319] 9) The reinforcement learning-based policy network receives the final state S t As input, it generates a scheduling action A according to its own strategy t The action space is A t =j means assigning the first workload in the priority queue to the jth server; Represents an “invalid” action, meaning that the scheduler refuses to schedule this workload in the current time step.
[0320] 10) Determine whether the scheduling action is valid, if so, proceed to step 11), otherwise proceed to step 13).
[0321] 11) Schedule the workload at the head of the priority queue to action A t The specified server.
[0322] 12) Final state S t Transfer to Cluster environment based on Generate reward R t , which consists of the following three parts:
[0323] 12.1) Resource contention penalty:
[0324]
[0325] Specifically, C r (s j ,d) represents the cross-resource dimension d and server s j The associated contention score is calculated as:
[0326]
[0327] Among them, s j,W Indicates that on the server j The workload set running on i,d is a vector representing the server s j Workload w i The time series of resource usage along d. K c is a constant that determines the weight of the resource contention penalty.
[0328] 12.2) Underutilization Penalty:
[0329]
[0330] The constant term K u As a scaling factor, here it is an exponent.
[0331] 12.3) Waiting time penalty:
[0332] P W =K w *|W|.
[0333] K w is a constant and |W| represents the total number of workloads to be scheduled in the cluster.
[0334] Finally, the reward R t It can be expressed as: R t =-P C -P U -P W . Consider the scheduling process as trajectory e=1, and record (S t ,At ,R t ) ancestor, return to step 5).
[0335] 13) The time step is increased by 1, and all new workload requests incoming to the cluster are placed into W, and return to step 3).
[0336] 14) Determine whether the preset number of training iterations has been reached, if so, proceed to step 22), otherwise proceed to step 15).
[0337] 15) Initialize the cloud server cluster status and workload request to be consistent with step 2).
[0338] 16) Carry out multiple simulation experiments and record the corresponding trajectories e=2,...,N, where each trajectory is represented by:
[0339]
[0340] 17) Calculate the cumulative reward for each step in all trajectories:
[0341]
[0342] Where γ is the decay factor used to calculate the cumulative reward.
[0343] 18) Calculate the baseline based on the cumulative reward at each step in all trajectories:
[0344]
[0345] 19) Calculate the gradient of all trajectories according to the baseline:
[0346]
[0347] 20) Add all the obtained gradients and update the network weights of the policy network and parallel GAT layer.
[0348] 21) Increase the number of iterations by 1 and return to step 14).
[0349] 22) Initialize the time step to 1 and return to step 2).
[0350] 23) All workloads have been successfully scheduled, and the process ends. If a new workload request arrives at the cluster, return to step 1).
Claims
1. Priority- and resource-aware time-varying workload scheduling method, It is characterized in that The following steps are involved: 1) Obtain the cloud server cluster status and the workload set W that needs to be scheduled. 2) Determine whether the workload set W is successfully scheduled, if so, proceed to step 20), otherwise, proceed to step 3); 3) Determine whether the preset time step threshold T is reached, if so, proceed to step 14), otherwise, proceed to step 4); 4) Using classifier f c For the current workload w i Assign label c i ; 5) Use the queue scorer to evaluate the workload in W and get the workload score 6) Construct the priority queue state matrix S with the top K workloads with the highest scores Q ∈Z K×(1+D) ; Among them, the priority queue state matrix S Q ∈Z K×(1+D) The first column of is the assigned label, and the other D columns represent the required resources of the workload; 7) Use parallel graph attention layer and gated recurrent unit to extract the hidden state S of the server cluster C ; 8) Hidden state S of the connected server cluster C , priority queue state matrix S Q , forming the final state S at time step t t ; 9) The final state S t Input into the reinforcement learning-based policy network and output the scheduling action A t ; 10) Determine whether the scheduling action is valid, if yes, proceed to step 11), otherwise proceed to step 13); 11) Schedule the workload at the head of the priority queue to action A t In the specified server; 12) The final state S t Transfer to Cluster environment based on Generate reward R t , regard the scheduling process as trajectory e=1, and record (S t ,A t ,R t ) tuple, return to step 5); 13) Increment the time step by 1, put all new workload requests that will be introduced into the cluster into W, and return to step 3). 14) Determine whether the preset number of training iterations has been reached, if so, proceed to step 19), otherwise proceed to step 15). 15) Initialize the cloud server cluster status and workload request, consistent with step 2); 16) Carry out multiple simulation experiments and record the corresponding trajectories e=2,...,N; 17) Calculate all trajectory gradients and add all gradients to update the network weights of the policy network and parallel graph attention layer; 18) Increase the number of iterations by 1 and return to step 14). 19) Initialize the time step to 1 and return to step 2). 20) All workloads have been successfully scheduled, and the process ends. If a new workload request arrives at the cluster, return to step 1).
2. The priority and resource-aware time-varying workload scheduling method according to claim 1, It is characterized in that The cloud server cluster includes J servers, denoted as {s 1 ,...,s j ,...,s J }; Each server has D resource dimensions; For each server j , the resource capacity in the resource dimension d is heterogeneous, denoted as c j,d ; At time step t, the amount of available resources of the server in resource dimension d is expressed as 3. The priority-based and resource-aware time-varying workload scheduling method according to claim 1, It is characterized in that In the workload set W that needs to be scheduled, workload w i The time-varying resource demand of ∈W is defined as in represents the number of resources requested across D dimensions at the lth execution time step; n i Represents the execution time step.
4. The priority-based and resource-aware time-varying workload scheduling method according to claim 1, It is characterized in that Classifier f c The construction steps are as follows: 1) Collect a set of historical workloads from the cluster history trace; 2) Using the dynamic time warping algorithm to calculate the distance metric between different workloads; Among them, the workload w i and w g The distance measure between As shown below: Among them, the parameters The workload w i and w g time-varying resource requirements; Time-varying resource requirements The minimum distance measure between Time-varying resource requirements The minimum distance measure between Time-varying resource requirements The minimum distance measure between Time-varying resource requirements The minimum distance measure between for time-varying resource requirements; 3) Based on the distance metric, the workloads are clustered using a density-based clustering algorithm to generate C equivalence classes, where workloads that are densely connected form one equivalence class; 4) Using the workload and the corresponding class label as training data, the classifier f is trained c .
5. The priority-based and resource-aware time-varying workload scheduling method according to claim 4, It is characterized in that The method to generate C equivalence class is: For workload w i , if its ∈ neighborhood contains at least minPts workloads, then workload w i As a core point; If the workload w i It is the core point and It is believed that from w i to w g The path is directly density-reachable; ∈ is the preset distance threshold; Given a core point w i ,w g ,w e , if w i to w g and from w g to w e The paths are all directly density-reachable, then w i to w e The path is density-reachable; If a core point makes the g to w i 、w g to w e is density-reachable, then w i and w e are defined as density connected.
6. The priority-based and resource-aware time-varying workload scheduling method according to claim 1, Features ,Workload w i The fraction q i As shown below: In the formula, α 1 , α 2 , α 3 is a constant that controls the importance of different components to the score; Indicates that at time step t, server s j The amount of resources available in resource dimension d; represents the workload w i The number of resources requested on the resource dimension d at the first execution time step; t represents the current time step; t i represents the workload w i At time step t i Arrival cluster; Indicates that the class c is currently running in the cluster i The number of workloads; Ι()=0,1 is the indicator function; B(t) represents the servers running in the cluster at time step t.
7. The priority- and resource-aware time-varying workload scheduling method according to claim 1, It is characterized in that Use parallel graph attention layers and gated recurrent units to extract the hidden state S of the server cluster C The steps include: 1) Using one-dimensional convolution from each server s j Extracting high-level features from resource usage time series j ,Right now: v j =conv(o j ) (4) Among them, the parameters M is the number of historical time steps read by the server, in horizontal pixels Indicates server j The resource utilization status within the network; conv() represents the convolution operation; 2) Capture high-level features V = [v 1 ,...,v j ,...,v J ], and obtain the dimension graph attention state h feat ; The time-oriented parallel graph attention layer is used to capture the temporal dependencies in high-level features and obtain the temporal graph attention state h time ; Among them, the dimension graph attention state h feat As shown below: h feat =[h 1 ,...,h j ,...,h J ](5) In the formula, σ represents the sigmoid activation function, α jg is the attention score, represents the set of adjacent nodes of node j; h j is the attention state of node j; v g For advanced features; Attention score α jg As shown below: In the formula, LeakeyReLU represents the activation function; ω represents the weight parameter; v j 、v g is an advanced feature; e jg is an intermediate parameter; intermediate parameter Time series diagram of attention state h time As shown below: h time =[h 1 ,...,h m ,...,h M ](9) In the formula, α mp is the attention score; 3) Connection dimension graph attention state h feat , time series diagram attention state h time , and obtain the feature matrix; then, the feature matrix is input into the gated recurrent unit to obtain the hidden state S of the server cluster C ,Right now: Where GRU stands for gated recurrent unit.
8. The priority-based and resource-aware time-varying workload scheduling method according to claim 1, It is characterized in that Scheduling Action A t The action space is A t =j means assigning the first workload in the priority queue to the jth server; Represents an "invalid" action, meaning that the scheduler refuses to schedule this workload in the current time step.
9. The priority and resource-aware time-varying workload scheduling method according to claim 1, It is characterized in that Reward R t =-P C -P U -P W ; Among them, resource contention penalty P C As shown below: In the formula, K c is a constant that determines the weight of the resource contention penalty; Among them, the cross-resource dimension d and server s j The relevant contention score C r (s j ,d) as shown below: Among them, s j,W Indicates that on the server j The workload set running on i,d Indicates server j Workload w i Resource usage time series along d; r g,d represents the workload w g time series of resource usage along d; Underutilization penalty P U As shown below: Among them, the constant term K u is the scale coefficient; Waiting time penalty P W As shown below: P W =K w *|W|(15) Among them, K w is a constant; |W| represents the total number of workloads to be scheduled in the cluster.
10. The priority-based and resource-aware time-varying workload scheduling method according to claim 1, It is characterized in that The steps to calculate the gradients of all trajectories include: 1) Construct the trajectory expression, namely: In the formula, trace e Indicates trajectory; represents the time step T e Corresponding states, actions, and rewards; 2) Calculate the cumulative reward for each step in all trajectories Right now: Where γ is the decay factor; the time step x = t, ..., T e ; represents the reward corresponding to time step x; 3) Calculate the baseline z based on the cumulative reward at each step in all trajectories t ,Right now: Where N is the number of trajectories; 4) Calculate the gradient Δθ of all trajectories according to the baseline, that is: In the formula, α is the learning rate.
Citation Information
Patent Citations
Docker cloud platform based dynamic-priority-queue-feedback load-balanced resource scheduling method
CN106790726A
Adaptive configuration method for short-time variable big data job cluster scheduling
CN110737529A
Methods and systems for data collection, learning, and streaming of machine signals for analytics and maintenance using the industrial internet of things
CN112703457A
Container cloud resource intelligent scheduling method oriented to LVC simulation
CN117687760A
Reinforcement learning for multi-access traffic management
US20220014963A1
