An accelerated optimization method for data center resource scheduling

Abstract

The present invention discloses an accelerated optimization method for data center resource scheduling, comprising the following steps: step 1, a user submits a task request, all requests enter a queue, and wait for the data center to perform resource allocation; step 2, the data center performs resource allocation according to task attributes through an optimal scheduling target; step 3, using the task attributes as the optimal scheduling target to form an experience knowledge base during late deep reinforcement learning; step 4, based on the experience knowledge base obtained in step 3, through a deep reinforcement learning network, it is applied to the data center resource scheduling task, iteratively obtains the optimal strategy, that is, obtains the optimal Q value and the allocation strategy corresponding to the Q value; step 5, after the optimal strategy is executed, when the computing power between nodes is unbalanced and the computing power of some idle nodes is not fully utilized, dynamic supplementary coding is performed to optimize resource scheduling efficiency. The present invention can maximize resource utilization, efficiently allocate resources, and reduce ineffective carbon emissions.

Description

Technical Field

[0001] The present invention relates to the technical field of data center resource management and scheduling, and in particular to an accelerated optimization method for data center resource scheduling. Background Art

[0002] With the rapid development of big data applications and artificial intelligence in recent years, more and more industries are gradually transforming towards online, networked, and digital operations. The shift to innovative energy-saving and low-carbon energy usage models in data centers will become an inevitable trend in their future development. The core of the dual carbon goals is to reduce ineffective carbon emissions. Currently, the average utilization rate of data center servers remains low, with idle power still accounting for 70% of its peak power. Using energy-saving optimization algorithms to optimize load scheduling can achieve energy savings and effectively reduce carbon emissions.

[0003] Traditional scheduling optimization algorithms have their own performance advantages for specific tasks, but they cannot simultaneously meet the needs of heterogeneous environments. For example, heuristic algorithms are quite limited in optimizing for dynamic and complex environments, and sometimes even have negative effects. Existing research using deep reinforcement learning mostly uses value iteration methods, while few algorithms use policy iteration. Summary of the Invention

[0004] The technical problem to be solved by the present invention is to provide an accelerated optimization method for data center resource scheduling, which can maximize resource utilization, efficiently allocate resources, and reduce ineffective carbon emissions.

[0005] To solve the above technical problems, the present invention provides an accelerated optimization method for data center resource scheduling, comprising the following steps:

[0006] Step 1: The user submits a task request. All requests enter the queue and wait for resource allocation by the data center.

[0007] Step 2: The data center allocates resources based on task attributes through the optimal scheduling goal;

[0008] Step 3: Use task attributes as optimal scheduling targets to form an experience knowledge base for later deep reinforcement learning.

[0009] Step 4: Based on the empirical knowledge base obtained in step 3, the deep reinforcement learning network (DQN) is used to apply it to the data center resource scheduling task, and the optimal strategy is iterated to obtain the optimal Q value and the allocation strategy a corresponding to the Q value.

[0010] Step 5: After the optimal strategy is executed, when the computing power between nodes is unbalanced and the computing power of some idle nodes is not fully utilized, dynamic additional encoding is performed to further optimize resource scheduling efficiency.

[0011] Preferably, in step 3, the optimal scheduling objectives include minimizing the completion time, minimizing the delay parameter, and maximizing the task parallelism.

[0012] Preferably, in step 4, based on the experience knowledge base obtained in step 3, the deep reinforcement learning network DQN is used to apply it to the data center resource scheduling task, and the optimal strategy is iterated, that is, the optimal Q value and the allocation strategy a corresponding to the Q value are obtained. Specifically, a state action value function Q(s, a) is initialized using a convolutional neural network CNN. The Q value includes the task queue information of the resource. At time t in the training process, there will be a state s t , take an action a t , and get a reward r t , then jump to state s t+1 , at this time the data (s t ,a t ,r t ,s t+1 ) is stored in the cache. This set of cache data (s i ,a i ,r i ,s i+1 ) is used to calculate the target value The formula is:

[0013]

[0014] By continuously updating the Q value using state s, action a, and reward r, the problem becomes a regression problem, that is, the target value of the task is obtained by the gradient descent method in machine learning. and with the Allocation strategy corresponding to the value

[0015] Preferably, in step 5, after the optimal strategy is executed, when the computing power between nodes is unbalanced and the computing power of some idle nodes is not fully utilized, dynamic supplementary encoding is performed to further optimize resource scheduling efficiency, specifically including the following steps:

[0016] Step 51: Split the dataset A into k equal parts, denoted as A1, ...A k ;

[0017] Step 52: A1, ...A k The first polynomial encoding of the data subset is performed, denoted as n can be regarded as the number of server working nodes that can be called, that is, to maximize the use of servers, then for working node i, i∈[1,n], where a∈V k is the k-dimensional vector of the polynomial;

[0018] Step 53: For working node i, calculate its result P i ,get

[0019] Step 54: Under normal circumstances, calculate the result P i After that, it is often directly sent back to the decoder for decoding. This solution often cannot maximize the use of working nodes. In order to achieve fast communication transmission, the calculation result P i Continue slicing, divide into m parts, and get P i The column matrix of is:

[0020]

[0021] The entire task is divided into mk subtask blocks, first divided into k parts, and then divided into m parts. In order to achieve the requirement of recovering the calculation results, the subtask block P i,j , j∈[1,m] is encoded again, and the encoding formula is: Among them, i∈[1,n], j∈[1,m], for each work task, mk encoded data blocks will appear for transmission when the calculation is completed, realizing dynamic collaborative computing of node resources in a distributed system.

[0022] Preferably, the computing power between nodes refers to CPU and memory.

[0023] The beneficial effects of the present invention are as follows: Aiming at the scenario of resource waste in task scheduling in data centers, the present invention proposes a deep reinforcement learning model based on policy iteration, which realizes training of better scheduling strategies for different scheduling goals, and facilitates optimization work in complex environments; by introducing an empirical knowledge base strategy during training, the time for training the model from scratch is reduced, and the learning speed is improved; the coding scheme for dynamic addition of idle nodes - double polynomial coding fully utilizes all nodes to significantly reduce the calculation completion time, improves the computing power of all nodes during scheduling, maximizes the task parallelism to ensure that as many tasks as possible are run at the same time, and further optimizes resource scheduling efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Figure 1 Schematic diagram of the method of the present invention.

[0025] Figure 2 This is a schematic diagram of the scheduling management of the present invention.

[0026] Figure 3 Schematic diagram of the dual polynomial encoding method of the present invention. DETAILED DESCRIPTION

[0027] like Figure 1 As shown, an accelerated optimization method for data center resource scheduling includes the following steps:

[0028] Step 1: The user submits a task request. All requests enter the queue and wait for resource allocation by the data center.

[0029] Step 2: The data center allocates resources based on task attributes through the optimal scheduling goal;

[0030] Step 3: Use task attributes as optimal scheduling targets to form an experience knowledge base for later deep reinforcement learning.

[0031] Step 4: Based on the empirical knowledge base obtained in step 3, the deep reinforcement learning network (DQN) is used to apply it to the data center resource scheduling task, and the optimal strategy is iterated to obtain the optimal Q value and the allocation strategy a corresponding to the Q value.

[0032] Step 5: After the optimal strategy is executed, when the computing power between nodes is unbalanced and the computing power of some idle nodes is not fully utilized, dynamic additional encoding is performed to further optimize resource scheduling efficiency.

[0033] like Figure 2 As shown, the dynamic nature of the application and release of shared resources in cloud computing applications makes dynamic scheduling strategies crucial. In data center task processing, resource allocation based on task attributes through scheduling strategies can be considered a decision-making process, ultimately achieving the scheduling goals of minimizing task completion time, minimizing latency parameters, and maximizing task parallelism. This addresses the issues of low idle resource coordination and lagging scheduling policy updates in task scheduling optimization. Steps 3-5 are performed based on the determination of the optimal scheduling goal.

[0034] If the model training in the reinforcement learning model relies entirely on data to start from scratch, the training cycle and convergence effect are not ideal due to the training process and mechanism of random exploration. Therefore, the present invention takes minimizing completion time, minimizing delay parameters, and maximizing task parallelism as scheduling goals, and obtains algorithm strategies formed for these scheduling goals, forming an experience knowledge base for later deep reinforcement learning. The experience knowledge base contains a wide range of prior knowledge to guide the exploration direction of the intelligent agent in the deep reinforcement learning model, thereby reducing the iteration time of reinforcement learning, shortening training time, and improving efficiency.

[0035] The present invention applies the deep reinforcement learning network DQN to the data center resource scheduling task to iterate the optimal strategy, that is, to obtain the optimal Q value and the allocation strategy a corresponding to the Q value.

[0036] Define some letters: state action value function Q(s, a), convolutional neural network CNN, state s (i.e. the current task scheduling queue state), action a (i.e. the allocation strategy adopted), reward r. First, use the convolutional neural network CNN to initialize a state action value function Q(s, a). The Q value contains the task queue information of the resource. At time t in the training process, there will be a state s t , take an action a t , and get a reward r t , then jump to state s t+1 , at this time the data (s t ,a t ,r t ,s t+1 ) is stored in the cache. This set of cache data (s i ,a i ,r i ,s i+1 ) is used to calculate the target value The formula is:

[0037]

[0038] By continuously updating the Q value using state s, action a, and reward r, the problem becomes a regression problem, that is, the target value of the task is obtained by the gradient descent method in machine learning. and with the Allocation strategy corresponding to the value

[0039] Utilizing the encoding acceleration optimization of idle nodes. After the deep reinforcement learning model gives a probability distribution, it is found that after the task scheduling strategy is executed, when the computing power (mainly CPU and memory computing power) between nodes is unbalanced, the computing power of some idle nodes is not fully utilized. To address this phenomenon of collaborative waste in task scheduling, the present invention designs a dynamically supplemented encoding scheme to improve the computing power of all nodes in the data center during task scheduling, fully utilizing all nodes to significantly reduce the calculation completion time, thereby further optimizing resource scheduling efficiency.

[0040] The present invention performs appropriate linear encoding on the input data set to form an appropriate amount of computational redundancy to ensure the reliability of the computation when some CPU and GPU devices fail. It is a dual encoding scheme that encodes the data set and the computation results separately. Figure 3 As shown, the process includes the following:

[0041] Process 1: First, split the dataset A into k equal parts, denoted as A1, ...A k ;

[0042] Process 2: A1, ...A kThe first polynomial encoding of the data subset is performed, denoted as n can be regarded as the number of server working nodes that can be called, that is, maximizing the use of servers. Then for working node i, i∈[1,n], where a∈V k is the k-dimensional vector of the polynomial.

[0043] Process 3, for working node i, calculate its result P i ,get

[0044] Process 4, usually, calculate the result P i After that, it is often directly sent back to the decoder for decoding. This solution often cannot maximize the use of working nodes. In order to achieve fast communication transmission, this patent will calculate the result P i Continue slicing, divide into m parts, and get P i The column matrix of is:

[0045]

[0046] The entire task is divided into mk subtask blocks (first divided into k parts, then divided into m parts). In order to achieve the requirement of recovering the calculation results, the subtask block P i,j , j∈[1,m] is encoded again. The encoding formula is: where i∈[1,n] and j∈[1,m]. For each task, mk encoded data blocks will be available for transmission upon completion of the computation. The encoding scheme of the present invention focuses on fully utilizing the resources of idle nodes and achieving dynamic collaborative computation of node resources in a distributed system.

[0047] To further improve the efficiency of the encoding framework for scheduled computing tasks, local linear encoding results are mapped to a portion of an encoded dataset. This allows each new result to be viewed as a linear combination of the original local results. Sharding large tasks fully utilizes the computing power of idle nodes. Computational tasks are divided into subtask blocks, which are then processed and transmitted to the master node. The master node's decoding work can be performed immediately after the worker nodes have completed their partial tasks. This solution, which fully utilizes the computing power of all nodes, further accelerates the optimization of the scheduling strategy generated by deep reinforcement learning models and maximizes the amount of parallel tasks, achieving comprehensive scheduling optimization from strategy and encoding to communication links.

[0048] When data centers are operating, the efficiency of task scheduling directly impacts their operational efficiency. To address this issue, with the goal of low-carbon operation, we research optimization algorithms and resource allocation techniques for task scheduling. This paper accelerates model training by leveraging an empirical knowledge base, establishes a deep reinforcement learning model to obtain more efficient and accurate scheduling policy information, and designs a coding scheme that utilizes idle nodes to accelerate task scheduling.

[0049] The present invention mainly utilizes the algorithm of policy iteration to improve the efficiency of scheduling strategies in complex environments and increase the policy gradient update speed of the model. At the same time, it introduces an empirical knowledge base to reduce the model training time. During resource scheduling, a coding scheme for dynamically supplementing idle nodes is designed, which greatly reduces the calculation completion time, improves the computing power of distributed nodes, and further improves the resource scheduling efficiency.

Claims

1. An accelerated optimization method for data center resource scheduling, characterized in that: The steps include: Step 1: The user submits a task request. All requests enter the queue and wait for resource allocation by the data center. Step 2: The data center allocates resources based on task attributes through the optimal scheduling goal; Step 3: Use task attributes as optimal scheduling targets to form an experience knowledge base for later deep reinforcement learning. Step 4: Based on the experience knowledge base obtained in step 3, the deep reinforcement learning network DQN is used to apply it to the data center resource scheduling task, and the optimal strategy is iterated, that is, the optimal Q value and the allocation strategy a corresponding to the Q value are obtained; specifically: a state action value function Q(s, a) is initialized using the convolutional neural network CNN. The Q value contains the task queue information of the resource. At time t in the training process, there will be a state s t , take an action a t , and get a reward r t , then jump to state s t+1 , at this time the data (s t ,a t ,r t ,s t+1 ) is stored in the cache, this set of cache data (s t ,a t ,r t ,s t+1 ) is used to calculate the target value The formula is: By continuously updating the Q value using state s, action a, and reward r, the problem becomes a regression problem, that is, the target value of the task is obtained by the gradient descent method in machine learning. and with the Allocation strategy corresponding to the value Step 5: After the optimal strategy is executed, if the computing power between nodes is unbalanced and the computing power of some idle nodes is not fully utilized, dynamic supplementary encoding is performed to further optimize resource scheduling efficiency. The specific steps include the following: Step 51: Split the dataset A into k equal parts, denoted as A1, ...A k ; Step 52: A1, ...A k The first polynomial encoding of the data subset is performed, denoted as n can be regarded as the number of server working nodes that can be called, that is, to maximize the use of servers, then for working node i, i∈[1,n], where a∈V k , is the k-dimensional vector of the polynomial; Step 53: For working node i, calculate its result P i ,get Step 54: calculate the result P i Continue slicing, divide into m parts, and get P i The column matrix of is: The entire task is divided into mk subtask blocks, first divided into k parts, and then divided into m parts. In order to achieve the requirement of recovering the calculation results, the subtask block P i,j , j∈[1,m] is encoded again, and the encoding formula is: Among them, i∈[1,n], j∈[1,m], for each work task, mk encoded data blocks will appear for transmission when the calculation is completed, realizing dynamic collaborative computing of node resources in a distributed system.

2. The accelerated optimization method for data center resource scheduling according to claim 1, characterized in that: In step 3, the optimal scheduling objectives include minimizing the completion time, minimizing the delay parameter, and maximizing the task parallelism.

3. The accelerated optimization method for data center resource scheduling according to claim 1, characterized in that: The computing power between nodes refers to CPU and memory.

Images