Artificial intelligence-based computing power task-resource dynamic combination optimization scheduling method

Through the heterogeneous task-heterogenous resource dynamic combination optimization scheduling method based on artificial intelligence, the problem that the data center computing power task scheduling method cannot reflect the actual operating conditions of large-scale data centers is solved, and efficient power management and system performance are achieved Guaranteed, scalable and generalized.

CN118964027AActive Publication Date: 2024-11-15NORTH CHINA ELECTRIC POWER UNIV +1

Patent Information

Application Number
CN202411095281.3
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-12
Publication Date
2024-11-15
Estimated Expiration
2044-08-12

AI Technical Summary

Technical Problem

The existing data center computing power task scheduling methods are limited by the model scale and computing power, and cannot truly reflect the actual operating conditions of large-scale data centers, resulting in the inability to perform refined management, which has great limitations.

Method used

Adopting the method of heterogeneous tasks-heterogeneous resources dynamic combination optimization scheduling, through refined modeling, constructing traditional Markov decision-making models, and strengthening them in directed acyclic graph neural network-pointer network-soft actor critics Efficient scheduling is carried out under the framework of learning algorithms to realize dynamic combination optimization of heterogeneous tasks-heterogeneous resources.

Benefits of technology

It realizes efficient management of large-scale dynamic computing power task scheduling in heterogeneous data centers, reduces power consumption costs, ensures system performance and user experience, is scalable and generalized, and is suitable for computing power tasks of different scales. Scene.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118964027A_ABST
    Figure CN118964027A_ABST
Patent Text Reader

Abstract

The invention discloses a computing power task-resource dynamic combination optimization scheduling method based on artificial intelligence, which is oriented to a heterogeneous data center environment with servers with different hardware configurations and energy consumption characteristics and considers heterogeneous resource request characteristics and time delay tolerance characteristics of different types of computing power tasks. A heterogeneous computing power task and heterogeneous resource matching optimization model is established; the heterogeneous task-heterogeneous resource dynamic combination optimization Markov decision modeling adaptive to the heterogeneous data center large-scale computing power task scheduling underlying logic is realized; a directed acyclic graph neural network-pointer network-soft actor commentator end-to-end algorithm framework is constructed, so that the problem of massive high-dimensional heterogeneous computing power task characterization is solved, and heterogeneous task-heterogeneous resource dynamic combination optimization online efficient scheduling is realized; the method has expandability and generalization, and the model for training convergence of the small-scale computing power task scene can be expanded and applied to different large-scale computing power task scenes.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention belongs to the field of artificial intelligence technology, and specifically relates to a computing task-resource dynamic combination optimization scheduling method based on artificial intelligence. Background Art

[0002] With the rapid development of artificial intelligence and large language models, data centers, as computing infrastructure, are using more and more energy, and their electricity costs remain high. Data centers can exert unique energy flexibility by scheduling computing tasks, which has the following four advantages:

[0003] (1) Optimizing resource utilization: Data centers usually have a large number of heterogeneous servers and resources, and the utilization of these resources is directly related to the economic benefits of the data center. Through an effective computing task scheduling algorithm, efficient resource utilization can be achieved, resource waste can be minimized, and operating costs can be reduced;

[0004] (2) Optimizing performance: Data centers usually carry a large number of different types of user requests and computing tasks. These computing tasks often have different performance requirements and latency tolerances. By optimizing the computing task scheduling strategy, resources can be reasonably allocated to ensure the performance indicators of key computing tasks, thereby improving the overall performance of the system and user satisfaction.

[0005] (3) Reduce energy consumption: The energy consumption of a data center is an important component of its operating costs. Through a reasonable computing task scheduling algorithm, active computing tasks can be concentrated on some servers, allowing other servers to enter a low-power or shutdown state, thereby reducing energy consumption and environmental impact.

[0006] When optimizing the scheduling of computing tasks in a data center, it is necessary to comprehensively consider the influence of multiple factors in order to formulate an appropriate scheduling strategy to achieve system optimization and efficient operation: (1) Computing task attributes and requirements: The attributes of computing tasks include computing requirements (CPU), memory requirements (I / O), network bandwidth requirements (NET), disk requirements (DISK), etc. In addition, the latency tolerance of computing tasks is also an important factor to consider; (2) Resource heterogeneity: Different servers may have different hardware configurations and performance characteristics; (3) User experience: In addition to energy goals such as energy costs, another goal of computing task scheduling is to optimize system performance and response time to improve user experience and overall system efficiency; (4) Generalization: When the scale of computing tasks changes, the computing task allocation and rebalancing can be dynamically adjusted to ensure efficient operation of the system.

[0007] Existing methods for scheduling computing tasks in data centers are limited by the scale of the model and computing power. They usually simplify the problem by limiting the solution set scale of computing tasks or resources, or making assumptions such as homogenizing computing tasks or resources. However, this simplification makes these methods fail to truly reflect the actual operating conditions of large-scale data centers and cannot be managed in a targeted and refined manner. Therefore, they have great limitations in industrial applications. Summary of the invention

[0008] In response to the above-mentioned deficiencies in the prior art, the present application provides an artificial intelligence-based computing task-resource dynamic combination optimization scheduling method.

[0009] In the first aspect, the present application proposes a method for dynamic combination optimization scheduling of heterogeneous tasks and heterogeneous resources based on artificial intelligence, comprising the following steps:

[0010] Perform fine-scale modeling on the heterogeneous data center environment to obtain a fine-scale model of the heterogeneous data center environment;

[0011] The refined model of heterogeneous data center environment is constructed as a traditional Markov decision model;

[0012] The traditional Markov decision model is optimized to form a two-layer Markov decision model for dynamic combination optimization of heterogeneous tasks and heterogeneous resources that is adapted to the underlying logic of large-scale dynamic computing task scheduling in data centers. Based on the optimized model, efficient scheduling is performed under the framework of directed acyclic graph neural network-pointer network-soft actor critic reinforcement learning algorithm.

[0013] In some embodiments, the performing fine-scale modeling on the heterogeneous data center environment to obtain a fine-scale model of the heterogeneous data center environment includes:

[0014] Based on the energy consumption characteristics of servers with different hardware configurations, we conduct refined modeling of resource occupancy and energy consumption characteristics of different types of servers configured in heterogeneous data centers:

[0015] Create a server collection:

[0016]

[0017] in Indicates The energy consumption type of the server tower;

[0018] The high-dimensional heterogeneous resource parameters corresponding to the server are:

[0019]

[0020] in, Indicates the server type tag. , , and Respectively represent the server's current CPU, memory, disk and network usage parameters. , , , and Both represent the upper limit of server resource parameters;

[0021] According to the random arrival time, execution time, heterogeneous resource requests, internal dependency structure and delay tolerance of different types of computing tasks, the characteristics of heterogeneous computing tasks in the data center are modeled as a directed acyclic graph structure, in which computing tasks are The arrival trajectory is recorded as , a computing task includes: job information , and the dependency information between jobs. The dependency information between jobs is represented as the adjacent edge set of computing tasks. and the adjacency matrix , which provides basic information for the directed acyclic graph neural network operation, recorded as:

[0022]

[0023]

[0024] Among them, Computing tasks No. The high-dimensional heterogeneous features of a job include: CPU occupancy rate , mem occupancy , disk occupancy rate , net occupancy rate , Execution time: and delay tolerance .

[0025] In some embodiments, the step of constructing the refined model of the heterogeneous data center environment into a traditional Markov decision model includes the following state space construction steps:

[0026] The state space construction step comprises:

[0027] DAGNN is used to transform high-dimensional heterogeneous computing task features into low-dimensional homogeneous computing task embedding vectors, and three embedding vectors are obtained, namely, job vector, computing task vector and global vector.

[0028] Among them, the jobs are processed according to the partial order defined by the computing power tasks, and the attention mechanism is used to The features of The operator indicates that for the Layer , output message vector , which means that The same layer of calculation All and homework Directly connected subtasks The job vector A weighted combination of:

[0029]

[0030] Using the associative operator The The job feature vector of the previous layer and the message vector for the job Aggregation to generate updated job vector :

[0031]

[0032] in , and They are the input, past state, and updated state / output of GRU respectively; The initial state is 0;

[0033] go through After layer processing, use the readout operator Root job of calculation Generate computing task vector :

[0034]

[0035] Generate a global vector of all computing tasks:

[0036]

[0037]

[0038] in, and Represents a nonlinear transformation of a vector input;

[0039] The state space defined according to the traditional Markov decision process is as follows:

[0040]

[0041] By the time in the training cycle , computing task trajectory , Server Collection and electricity prices constitute;

[0042] Through the three obtained embedding vectors, the high-dimensional heterogeneous computing task state vector in the traditional Markov decision model is converted into a low-dimensional homogeneous vector, compressing the defined state space into:

[0043]

[0044] .

[0045] In some embodiments, the step of constructing the refined model of the heterogeneous data center environment into a traditional Markov decision model includes the following steps:

[0046] The action space construction step comprises:

[0047] Based on the DAGNN compressed embedding vector, we construct an action space that adapts to the homogeneous input of the pointer network:

[0048]

[0049]

[0050] in, represents the selectable action sequence of the action space of the adaptive pointer network isomorphic input at time t, Represents each job The embedding vector of Indicates that the job Computing tasks The embedding vector of represents the global embedding vector, Indicates server information. Indicates that no job is selected for execution;

[0051] In some embodiments, the step of constructing the refined model of the heterogeneous data center environment into a traditional Markov decision model includes the following state transition construction steps:

[0052] The state transfer construction step includes:

[0053] Given state , the reinforcement learning agent at time step The corresponding actions are:

[0054]

[0055] in, Represents multiple job-server pairs The sequence composed of Indicates that the current job is not executed. If the action sequence selected at a certain time step is all If it is composed, it means that no action is selected in this step;

[0056] When action After being determined, Interact with the environment to get the next state and rewards , this interaction process can be expressed as a mapping function:

[0057]

[0058]

[0059] in, Indicates the resource usage of the corresponding job.

[0060] In some embodiments, the step of constructing the refined model of the heterogeneous data center environment into a traditional Markov decision model includes the following reward feedback construction steps:

[0061] The reward feedback construction step includes: designing a reward function:

[0062]

[0063] in, Indicates the running cycle of the current time step, and the value of each time step is uncertain; Indicates the system performance weight coefficient; , and Represent the energy consumption characteristic functions of different servers respectively. Indicates CPU utilization. represents the memory access count, Indicates the disk read / write rate. Indicates the network read and write rate.

[0064] In some embodiments, the traditional Markov decision model is optimized to form a heterogeneous task-heterogeneous resource dynamic combination optimization double-layer Markov decision model that adapts to the underlying logic of large-scale dynamic computing task scheduling in a data center, and efficient scheduling is performed under the directed acyclic graph neural network-pointer network-soft actor critic reinforcement learning algorithm framework according to the optimized model, including:

[0065] The traditional Markov decision model is improved into a two-layer Markov decision model for dynamic combination optimization of heterogeneous tasks and heterogeneous resources, so as to adapt it to the underlying logic of large-scale dynamic computing task scheduling in data centers;

[0066] The pointer algorithm is embedded in the framework of the soft actor-critic algorithm to solve the problem of dynamic timing arrangement caused by the dynamic combination optimization of heterogeneous tasks and heterogeneous resources. The soft strategy iteration is used to maximize the goal, and the strategy evaluation and strategy improvement are performed alternately in the maximum entropy framework.

[0067] Finally, an end-to-end training framework of directed acyclic graph neural network-pointer network-soft actor critic reinforcement learning algorithm is formed to achieve real-time online decision-making.

[0068] In some embodiments, the traditional Markov decision model is improved into a heterogeneous task-heterogeneous resource dynamic combination optimization double-layer Markov decision model to adapt to the underlying logic of large-scale dynamic computing task scheduling in a data center, including:

[0069] The traditional Markov decision model with N-dimensional actions is reconstructed into a combined optimization Markov decision model containing N 1-dimensional action sequences, and the action value function of the reconstructed combined optimization Markov decision model is defined as:

[0070]

[0071] Among them, the definition ; For different time steps is changing, and the action values ​​of the two models are equal in some states:

[0072]

[0073] in, represents the action-value function of the reconstructed dynamic combinatorial optimization Markov decision model, Represents the action-value function of the traditional Markov decision model.

[0074] In some embodiments, the pointer algorithm is embedded in the soft actor-critic algorithm framework to solve the problem of dynamic timing arrangement caused by dynamic combination optimization of heterogeneous tasks and heterogeneous resources, and soft policy iteration is used to maximize the goal, while alternately performing policy evaluation and policy improvement in the maximum entropy framework, including pointer network soft policy evaluation, pointer network soft policy improvement and pointer network soft policy iteration;

[0075] The pointer algorithm is embedded in the framework of the soft actor-critic algorithm. First, the pointer network encoder is used to encode the feature vector, and then the decoder is combined with the attention calculation method to gradually construct the solution in an autoregressive manner to obtain the conditional probability. :

[0076]

[0077] Among them, given a training pair , represents the strategy sequence to be trained, The encoder input is read in sequence, and finally the vector V storing the input sequence information is encoded. At the same time, the encoder obtains the hidden state of each matching pair during the calculation process. ;

[0078] The decoder decodes the vector V, which reads V and outputs the first hidden layer state. , using the attention mechanism according to The hidden state of each matching pair obtained by the encoder Calculate the probability of each matching pair, select the matching pair with the highest probability as the matching pair in the first step, and the decoder reads the hidden layer output of the previous step and the feature vector of the matching pair, and outputs the current hidden layer state ,according to The probability of selecting each matching pair is calculated by the calculation of each matching pair. If the matching pair selected in a certain step does not meet the resource upper limit constraint, or when an empty action occurs , the pointer network stops outputting.

[0079] The pointer network parameters are optimized using a discrete soft actor-critic algorithm to find a policy that maximizes the discounted return over time:

[0080]

[0081] in, Representation strategy; represents the optimal strategy; Representation strategy The distribution of induced trajectories; Determines the relative importance of the entropy term with respect to the reward, expressed as a temperature parameter; Representation strategy In Status The entropy of time, assuming The dimension of the input sequence for a certain time step, Represents a sequence of actions, , at the end of the time step, the reward the agent receives based on the pointer network decision ;

[0082] The pointer network soft strategy evaluation includes: setting is the dimension of a certain action sequence, Representing action sequence , applied to Bellman operator is convergent:

[0083]

[0084] in:

[0085]

[0086]

[0087] when Time convergence;

[0088] The pointer network soft policy enhancement includes: defining a new policy target as:

[0089]

[0090] definition is the optimizer that maximizes the formula, for each , then and ;

[0091] The pointer network soft strategy iteration includes: starting from any strategy or applying the pointer network soft strategy evaluation and improvement, the strategy sequence converges to , then , .

[0092] In a second aspect, the present application proposes a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the above method are implemented.

[0093] Beneficial effects of the present invention:

[0094] 1. Aiming at the heterogeneous data center environment with servers with different hardware configurations and energy consumption characteristics, considering the different resource request characteristics and latency tolerance characteristics of different types of computing tasks, an optimization model for matching heterogeneous computing tasks with heterogeneous resources is established;

[0095] 2. Implemented Markov decision modeling for dynamic combination optimization of heterogeneous tasks and heterogeneous resources that adapts to the underlying logic of large-scale computing task scheduling in heterogeneous data centers;

[0096] 3. We built an end-to-end algorithm framework of directed acyclic graph neural network-pointer network-soft actor critic (DAGNN-PointerNetwork-SACD), based on which we solved the problem of characterizing massive high-dimensional heterogeneous computing power tasks and realized online efficient scheduling of dynamic combination optimization of heterogeneous tasks and heterogeneous resources;

[0097] 4. It is scalable and generalizable. The model trained and converged in a small-scale computing task scenario can be extended and applied to different large-scale computing task scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0098] Figure 1It is the overall flow chart of the present invention.

[0099] Figure 2 Schematic diagram of the traditional Markov decision model and the combinatorial optimization Markov decision model.

[0100] Figure 3 Schematic diagram of data center electricity costs and violation tasks.

[0101] Figure 4 This is a schematic diagram of the power consumption of data center servers. DETAILED DESCRIPTION

[0102] The exemplary embodiments of the present invention will be described in more detail below with reference to the accompanying drawings. Although the exemplary embodiments of the present invention are shown in the accompanying drawings, it should be understood that the present invention can be implemented in various forms and should not be limited by the embodiments described herein; on the contrary, these embodiments are provided to enable a more thorough understanding of the present invention and to fully convey the scope of the present invention to those skilled in the art.

[0103] The key points of the present invention include:

[0104] (1) We conduct refined modeling of heterogeneous data center environments with servers of different hardware configurations and energy consumption characteristics, consider the different resource request characteristics and latency tolerance characteristics of different types of computing tasks, and establish an optimization model for matching heterogeneous computing tasks with heterogeneous resources.

[0105] (2) A dynamic combination optimization Markov model of heterogeneous tasks and heterogeneous resources was established to adapt to the underlying logic of large-scale computing task scheduling in heterogeneous data centers. The traditional MDP model of N-dimensional actions was reconstructed into a reconstructed combination optimization MDP containing N 1-dimensional action sequences.

[0106] (3) This paper constructs a DAGNN (directed acyclic graph neural network)-PointerNetwork connection framework. DAGNN is used to solve the problem of characterizing massive high-dimensional heterogeneous computing tasks, compressing the high-dimensional computing task features into low-dimensional homogeneous embedding vectors, and forming low-dimensional homogeneous sequence vectors that can be input into PointerNetwork together with heterogeneous server features;

[0107] (4) The proposed pointer network-soft actor-critic (Ptr-SACD) efficient algorithm for the dynamic combinatorial optimization MDP model of adaptive heterogeneous task-heterogeneous resource matching decomposes the combinatorial optimization problem in the new MDP into a sequence-to-sequence solution process, and uses the pointer network to solve the temporal dynamic arrangement problem: the input of the pointer network is all possible combinations of the current time step, and the output is the ranking of the optimal combinations based on the current network strategy decision;

[0108] (5) Finally, a reinforcement learning algorithm framework with DAGNN-Ptr-SACD generalization and end-to-end solution was formed, which is scalable and generalizable. The model trained and converged in small-scale computing task scenarios can be extended and applied to different large-scale computing task scenarios.

[0109] Specifically, in the actual heterogeneous data center computing task scheduling scenario, the number of computing tasks is in the tens of millions. Taking n computing tasks and m servers as an example, the scale of the computing task scheduling problem is , which is an NP-hard combinatorial optimization problem, and from the perspectives of tasks and resources, it has more complex nonlinear constraints:

[0110] For tasks, the execution of a task will be broken down into jobs with different dependencies. The industry and academia usually model it as a directed acyclic topology (DAG). The reasonable execution order of DAG computing tasks is closely related to system performance. With the rapid development of the Internet, data centers need to serve different types of users (such as government agencies, enterprises, private users, etc.), so they also need to handle a variety of DAG computing tasks (such as form queries, machine learning model training, online shopping, etc.). The arrival of these computing tasks is also random. Therefore, the DAG computing tasks under actual working conditions have a high-dimensional heterogeneous topology with different job numbers and different job dependencies. At the same time, the requirements for heterogeneous resources such as CPU, MEM, DISK, NET, etc. are different, and online tasks and offline tasks also have different delay tolerances.

[0111] Regarding resources, in heterogeneous data centers, servers with different hardware configurations have different energy consumption characteristics, and the types of resources that affect their energy consumption are also different. According to statistics, the energy consumption of CPU resources only accounts for about 30%-60% of the total energy of the server. Therefore, considering only the CPU as a single resource will not only cause other resources to violate the resource upper limit constraint and cause the computing task to fail, but also ignore the heterogeneous energy consumption characteristics, resulting in increased costs.

[0112] In summary, during the scheduling process, the scheduler needs to make online and real-time matching decisions between heterogeneous computing tasks and heterogeneous resources for scenarios such as refined heterogeneous resource operation, random arrival of computing tasks, dynamic changes in computing task scale, and real-time clearing of electricity prices. It is also expected to have high generalization capabilities under different fresh states. The scheduling model and algorithm need to take into account both energy costs and the system performance limitations of the data center itself.

[0113] Therefore, in the first aspect, the present application proposes a computing task-resource dynamic combination optimization scheduling method based on artificial intelligence, comprising the following steps:

[0114] S100: Perform fine modeling on the heterogeneous data center environment to obtain a fine model of the heterogeneous data center environment;

[0115] In some embodiments, the performing fine-scale modeling on the heterogeneous data center environment to obtain a fine-scale model of the heterogeneous data center environment includes:

[0116] Refined modeling of heterogeneous servers in data centers:

[0117] Based on the energy consumption characteristics of servers with different hardware configurations, we conduct refined modeling of resource occupancy and energy consumption characteristics of different types of servers configured in heterogeneous data centers;

[0118] Create a server collection:

[0119]

[0120] in Indicates The energy consumption type of the server tower;

[0121] The high-dimensional heterogeneous resource parameters corresponding to the server are:

[0122]

[0123] in, Indicates the server type tag. , , and Respectively represent the server's current CPU, memory, disk and network usage parameters. , , , and Both represent the upper limit of server resource parameters;

[0124] Modeling of massive heterogeneous DAG computing tasks in data centers:

[0125] According to the random arrival time, execution time, heterogeneous resource requests, internal dependency structure and delay tolerance of different types of computing tasks, the characteristics of heterogeneous computing tasks in the data center are modeled as a directed acyclic graph structure, in which computing tasks are The arrival trajectory is recorded as , a computing task includes: job information , and the dependency information between jobs. The dependency information between jobs is represented as the adjacent edge set of computing tasks. and the adjacency matrix , which provides basic information for the directed acyclic graph neural network operation, recorded as:

[0126]

[0127]

[0128] Among them, Computing tasks No. The high-dimensional heterogeneous features of a job include: CPU occupancy rate , mem occupancy , disk occupancy rate , net occupancy rate , Execution time: and delay tolerance ;

[0129] In particular, online computing tasks are based on their characteristics , that is, the delay tolerance is 0. Different offline computing tasks have different delay tolerances, ranging from more than ten minutes to several hours.

[0130] S200: Build a refined model of heterogeneous data center environment into a traditional Markov decision model;

[0131] In some embodiments, the step of constructing the refined model of the heterogeneous data center environment into a traditional Markov decision model includes the following state space construction steps:

[0132] The state space construction step comprises:

[0133] DAGNN is used to transform high-dimensional heterogeneous computing task features into low-dimensional homogeneous computing task embedding vectors, and three embedding vectors are obtained, namely, job vector, computing task vector and global vector.

[0134] Among them, the jobs are processed according to the partial order defined by the computing power tasks, and the attention mechanism is used to The features of The operator indicates that for the Layer , output message vector , which means that The same layer of calculation All and homework Directly connected subtasks The job vector A weighted combination of:

[0135]

[0136] In particular, at level 0, , that is, the characteristics of the job itself; weight coefficient Following the query-key design in the traditional attention mechanism, the previous layer The job vector As a query:

[0137]

[0138] in, and The model parameters are in the additive form in this embodiment, rather than the usual dot product form, which involves fewer parameters.

[0139] Using the associative operator The The job feature vector of the previous layer and the message vector for the job Aggregation to generate updated job vector :

[0140]

[0141] in , and They are the input, past state, and updated state / output of GRU respectively; The initial state is 0. This design is different from most message passing mechanisms that use simple summation or connection for aggregation. In this embodiment, the computing task features are used as input, and the message is used as a hidden state for periodic updates.

[0142] go through After layer processing, use the readout operator Root job of calculation Generate computing task vector :

[0143]

[0144] Generate a global vector of all computing tasks:

[0145]

[0146]

[0147] in, and Represents a nonlinear transformation of a vector input;

[0148] The state space defined according to the traditional Markov decision process is as follows:

[0149]

[0150] Through the three obtained embedding vectors, the high-dimensional heterogeneous computing task vector is converted into a low-dimensional state vector representing the traditional Markov decision model. vector, compressing the defined state space into:

[0151]

[0152]

[0153] in, represents the time in the training cycle; Indicates the current day-ahead market electricity price.

[0154] The main innovation of the state space is to use the advantage of DAGNN in processing partial order structures to compress the features of massive heterogeneous computing tasks into low-dimensional homogeneous computing task embedding vectors, so as to fully characterize the heterogeneous feature information of DAG computing tasks and the dependencies between their internal jobs, and compress the state space to obtain three embedding vectors, namely the partial order aggregation job embedding vector (referred to as the job vector), the computing task pooling embedding vector (referred to as the computing task vector), and the nonlinear change global embedding vector (referred to as the global vector), which can then be combined with the characteristics of heterogeneous resources to form a low-dimensional homogeneous solution of the adaptive pointer network.

[0155] In some embodiments, the step of constructing the refined model of the heterogeneous data center environment into a traditional Markov decision model includes the following steps:

[0156] Among them, the combinatorial optimization problem of matching heterogeneous computing tasks with heterogeneous servers, that is, selecting the best job-server pair from all possible job-server pair sequences to form the optimal sequence, is an NP-hard problem whose scale increases exponentially. In addition, due to the uncertainty of the arrival of computing tasks and the heterogeneity of computing tasks, the input of the policy network, that is, the length of the selectable job-server matching pair sequence at each time step and the optimal sequence length of the decision, are variable. Therefore, the decision of the action can be considered as a special sequence-to-sequence problem.

[0157] Using a pointer network, the encoder is used to encode the input sequence of the combinatorial optimization problem to obtain a feature vector, and then the decoder is combined with the attention calculation method to gradually construct the solution in an autoregressive manner. Autoregression means selecting a task-server pair each time and selecting the next action based on the selected action until a complete solution is constructed.

[0158] The action space construction step comprises:

[0159] Based on the DAGNN compressed embedding vector, we construct an action space that adapts to the homogeneous input of the pointer network:

[0160]

[0161]

[0162] in, represents the selectable action sequence of the action space of the adaptive pointer network isomorphic input at time t, Represents each job The embedding vector of Indicates that the job Computing tasks The embedding vector of represents the global embedding vector, Indicates server information. Indicates that no job is selected for execution;

[0163] The main innovation of the action space: Due to the uncertainty of the arrival of computing tasks and the heterogeneity of computing tasks, the size of the action space / feasible domain that needs to be decided at each time step is different, that is, the length of the sequence of selectable job-server matching pairs and the optimal sequence length of the decision are variable at each time step, resulting in the problem of matching heterogeneous computing tasks with heterogeneous servers being a time-series dynamic permutation problem. This invention models this NP-hard problem whose scale increases exponentially.

[0164] In some embodiments, the step of constructing the refined model of the heterogeneous data center environment into a traditional Markov decision model includes the following state transition construction steps:

[0165] The state transfer construction step includes:

[0166] Given state , the reinforcement learning agent at time step The corresponding actions are:

[0167]

[0168] in, Represents multiple job-server pairs The sequence composed of Indicates that the current job is not executed. If the action sequence selected at a certain time step is all If it is composed, it means that no action is selected in this step;

[0169] When action After being determined, Interact with the environment to get the next state and rewards , this interaction process can be expressed as a mapping function:

[0170]

[0171]

[0172] in, It represents the resource occupation corresponding to the corresponding job, which is constrained and characterized by the operation rules and the corresponding physical rules, so that the feasibility of the action selected in each time step can be guaranteed.

[0173] Innovation of state transfer: from arrive The conversion process has the following randomness: 1. New tasks arrive; 2. Jobs are completed.

[0174] In some embodiments, the step of constructing the refined model of the heterogeneous data center environment into a traditional Markov decision model includes the following reward feedback construction steps:

[0175] At the end of the time step, the agent obtains its reward. The goal of the agent is to optimize the scheduling plan of each task-job to minimize the energy consumption cost of the data center and give full play to the time flexibility by appropriately sacrificing system performance when high electricity prices occur.

[0176] Therefore, the reward function can be designed as follows:

[0177] 1. The server operation energy consumption has a negative cost, which is a certain functional relationship with resource occupation;

[0178] 2. Penalty for violating the latency tolerance constraint of computing tasks;

[0179] The reward feedback construction step includes: designing a reward function:

[0180]

[0181] in, Indicates the running cycle of the current time step, and the value of each time step is uncertain; Indicates the system performance weight coefficient; , and Represent the energy consumption characteristic functions of different servers respectively. Indicates CPU utilization. represents the memory access count, Indicates the disk read / write rate. Indicates the network read and write rate.

[0182] The innovation of the award is that it takes into account both the energy consumption costs of three heterogeneous servers and the performance of the data center.

[0183] S300: Optimize the traditional Markov decision model to form a heterogeneous task-heterogeneous resource dynamic combination optimization double-layer Markov decision model that is adapted to the underlying logic of large-scale dynamic computing task scheduling in the data center, and perform efficient scheduling under the directed acyclic graph neural network-pointer network-soft actor critic reinforcement learning algorithm framework based on the optimized model.

[0184] In some embodiments, the traditional Markov decision model is optimized to form a heterogeneous task-heterogeneous resource dynamic combination optimization double-layer Markov decision model that adapts to the underlying logic of large-scale dynamic computing task scheduling in a data center, and efficient scheduling is performed under the directed acyclic graph neural network-pointer network-soft actor critic reinforcement learning algorithm framework according to the optimized model, including:

[0185] The discrete soft actor-critic algorithm is used to optimize the pointer network parameters in the traditional Markov decision model, forming a heterogeneous task-heterogeneous resource dynamic combination optimization double-layer Markov decision model that adapts to the underlying logic of large-scale dynamic computing task scheduling in data centers;

[0186] Use soft policy iteration to maximize the objective while alternating between policy evaluation and policy improvement within a maximum entropy framework;

[0187] Finally, a directed acyclic graph neural network-pointer network-soft actor critic reinforcement learning algorithm framework is formed, and real-time online decision-making is achieved by optimizing the formed model.

[0188] Among them, the traditional Markov decision model established above cannot adapt to the dynamic combination optimization of heterogeneous tasks and heterogeneous resources with multiple boundary conditions. Therefore, the Markov decision model is further improved to form an end-to-end training framework. A two-layer Markov decision model (reconstructed combinatorial optimization Markov decision model) is established to model the energy cost management problem of large-scale heterogeneous data centers under the scenario of random arrival of massive heterogeneous computing power tasks. Based on this framework, a dynamic combination optimization matching scheme for heterogeneous tasks and heterogeneous resources with multiple boundary conditions is proposed. The Ptr algorithm is embedded in the SACD algorithm to solve the dynamic sequence-sequence decision problem caused by the dynamic combination optimization of heterogeneous tasks and heterogeneous resources. Different strategies are fully explored under the environment of electricity price fluctuations, random arrival of massive heterogeneous computing power tasks and dynamic changes in the state of heterogeneous resources. In order to maximize the goal, soft policy iteration is used. It is necessary to alternately evaluate and improve strategies within the maximum entropy framework, and finally form a DGANN-Ptr-SACD end-to-end training framework to achieve real-time online decision-making.

[0189] In some embodiments, the traditional Markov decision model is improved into a heterogeneous task-heterogeneous resource dynamic combination optimization double-layer Markov decision model to adapt to the underlying logic of large-scale dynamic computing task scheduling in a data center, including:

[0190] Furthermore, due to the heterogeneous and random arrival characteristics of computing tasks, the action decision of the traditional Markov decision model is a time-series dynamic arrangement problem that cannot be solved by existing reinforcement learning algorithms. Therefore, both the model and the algorithm are improved at the same time. In this step, the model is improved first.

[0191] Consider a state Next, action environment, an N-dimensional action at a time step t is regarded as a sequence of N one-dimensional actions, which introduces a state The new MDP is composed of the subscripts of the traditional MDP state Alignment, superscript Represents the reconstruction of combinatorial optimization MDP relative to Therefore is a tuple containing the states from the traditional MDP and reconstruct the history of other states under the combinatorial optimization MDP definition (previously selected operations of series);

[0192] This transformation of the reconstructed combinatorial optimization MDP can be defined by two rules: when all one-dimensional actions are executed, that is, the last step is to calculate 1 step in the N-dimensional environment and receive a new state ,award , and reset the action to ; and in all other previous transitions, append the previously selected action to Win and get 0 reward;

[0193] like Figure 2 As shown, the two Markov decision models are different forms of the same environment;

[0194] The traditional Markov decision model with N-dimensional actions is reconstructed into a combined optimization Markov decision model containing N 1-dimensional action sequences, and the action value function of the reconstructed combined optimization Markov decision model is defined as:

[0195]

[0196] Among them, the definition ; For different time steps is changing, and the action values ​​of the two models are equal in some states:

[0197]

[0198] in, represents the action-value function of the reconstructed dynamic combinatorial optimization Markov decision model, Represents the action-value function of the traditional Markov decision model.

[0199] Among them, is the length of the offline dataset The agent's experience trajectory, for a given time step , and the corresponding actions in the trajectory , define the action is composed of the multi-dimensional actions contained in the one-step decision of the pointer network: Represents the first dimension from the pointer network To Dimensions The action dimension vector.

[0200] In some embodiments, the pointer algorithm is embedded in the soft actor-critic algorithm framework to solve the problem of dynamic timing arrangement caused by dynamic combination optimization of heterogeneous tasks and heterogeneous resources, and soft policy iteration is used to maximize the goal, while alternately performing policy evaluation and policy improvement in the maximum entropy framework, including pointer network soft policy evaluation, pointer network soft policy improvement and pointer network soft policy iteration;

[0201] The pointer algorithm is embedded in the framework of the soft actor-critic algorithm. The feature vector is obtained by encoding using the pointer network encoder. Then the decoder is combined with the attention calculation method to gradually construct the solution in an autoregressive manner:

[0202]

[0203] Among them, given a training pair , represents the strategy sequence to be trained, express, The encoder input is read in sequence, and finally the vector V storing the input sequence information is encoded. At the same time, the encoder obtains the hidden state of each matching pair during the calculation process. ;

[0204] The decoder decodes the vector V, which reads V and outputs the first hidden layer state. , using the attention mechanism according to The hidden state of each matching pair obtained by the encoder Calculate the probability of each matching pair, select the matching pair with the highest probability as the matching pair in the first step, and the decoder reads the hidden layer output of the previous step and the feature vector of the matching pair, and outputs the current hidden layer state ,according to The probability of selecting each matching pair is calculated by the calculation of each matching pair. If the matching pair selected in a certain step does not meet the resource upper limit constraint, or when an empty action occurs , the pointer network stops outputting.

[0205] Among them, the specific formula for calculating the probability of selecting each matching pair is:

[0206]

[0207] in, , and are the learnable parameters of the output model; is the input vector The number of As a pointer to the input element, the attention mechanism value (query) vector represents the The output and The matching degree of the input; softmax transforms the vector (length n) normalized to the output distribution based on the input dictionary;

[0208] The pointer network parameters are optimized using a discrete soft actor-critic algorithm to find a policy that maximizes the discounted return over time:

[0209]

[0210] in, Representation strategy; represents the optimal strategy; Representation strategy The distribution of induced trajectories; Determines the relative importance of the entropy term with respect to the reward, expressed as a temperature parameter; Representation strategy In Status The entropy of time, assuming The dimension of the input sequence for a certain time step, Represents a sequence of actions, , at the end of the time step, the reward the agent receives based on the pointer network decision ;

[0211] The pointer network soft strategy evaluation includes: setting is the dimension of a certain action sequence, Representing action sequence , applied to Bellman operator is convergent:

[0212]

[0213] in:

[0214]

[0215]

[0216] when Time convergence;

[0217] The pointer network soft policy enhancement includes: defining a new policy target as:

[0218]

[0219] definition is the optimizer that maximizes the formula, for each , then and ;

[0220] The pointer network soft strategy iteration includes: starting from any strategy or applying the pointer network soft strategy evaluation and improvement, the strategy sequence converges to , then , .

[0221] In a second aspect, the present application proposes a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the above method are implemented.

[0222] The present invention has the following three advantages:

[0223] (1) The underlying logic of dynamic computing task scheduling for large-scale heterogeneous data centers more accurately reflects the actual situation of dynamic combination and optimization of heterogeneous computing tasks and heterogeneous resources in data centers, which not only reduces electricity costs but also ensures system performance and user experience. That is, the number of tasks that violate delay tolerance is small, such as Figure 3 shown.

[0224] (2) For scenarios such as refined heterogeneous resource operation, random arrival of computing tasks, dynamic changes in computing task scale, and real-time clearing of electricity prices, dynamic combination optimization and matching decisions of large-scale heterogeneous computing tasks and heterogeneous resources can be made online in real time, such as Figure 4 As shown in the figure, the proposed method can make the power consumption of heterogeneous servers adapt to the dynamically changing electricity prices according to their own characteristics: due to the energy consumption characteristics of server 2, the no-load energy consumption is relatively high, so it maintains full-load operation once it is started; while server 1 has average performance and low energy consumption, so it maintains basic operation; server 3 has low no-load energy consumption, but high operating energy consumption, so when the electricity price is high and the computing tasks are small, only a small number of computing tasks are assigned to server 3 for execution, maintaining low power consumption; when the electricity price is low and the computing tasks are large, more computing tasks are assigned to server 3 for execution, so the power consumption of server 3 shows an obvious following trend according to the electricity price and the amount of tasks.

[0225] (3) A DAGNN-Ptr-SACD RL algorithm framework was formed, which is scalable and generalizable, improves the efficiency and accuracy of problem solving, and makes the invention more widely applicable and practical in actual industrial applications.

[0226] The above are only preferred implementations of the present invention. It should be pointed out that a number of modifications and improved technical solutions made by those skilled in the art without departing from the technical solution should also be deemed to fall within the scope of protection required by the claims.

Claims

1. A computing task-resource dynamic combination optimization scheduling method based on artificial intelligence, characterized by: The following steps are involved: Perform fine-scale modeling on the heterogeneous data center environment to obtain a fine-scale model of the heterogeneous data center environment; The refined model of heterogeneous data center environment is constructed as a traditional Markov decision model; The traditional Markov decision model is optimized to form a two-layer Markov decision model for dynamic combination optimization of heterogeneous tasks and heterogeneous resources that is adapted to the underlying logic of large-scale dynamic computing task scheduling in data centers. Based on the optimized model, efficient scheduling is performed under the framework of directed acyclic graph neural network-pointer network-soft actor critic reinforcement learning algorithm.

2. The method according to claim 1, characterized in that: The method of performing fine modeling on the heterogeneous data center environment to obtain a fine model of the heterogeneous data center environment includes: Based on the energy consumption characteristics of servers with different hardware configurations, we conduct refined modeling of resource occupancy and energy consumption characteristics of different types of servers configured in heterogeneous data centers: Create a server collection: in Indicates The energy consumption type of the server tower; The high-dimensional heterogeneous resource parameters corresponding to the server are: in, Indicates the server type tag. , , and Respectively represent the server's current CPU, memory, disk and network usage parameters. , , , and Both represent the upper limit of server resource parameters; According to the random arrival time, execution time, heterogeneous resource requests, internal dependency structure and delay tolerance of different types of computing tasks, the characteristics of heterogeneous computing tasks in the data center are modeled as a directed acyclic graph structure, in which computing tasks are The arrival trajectory is recorded as , a computing task includes: job information , and the dependency information between jobs. The dependency information between jobs is represented as the adjacent edge set of computing tasks. and the adjacency matrix , which provides basic information for the directed acyclic graph neural network operation, recorded as: Among them, Computing tasks No. The high-dimensional heterogeneous features of a job include: CPU occupancy rate , mem occupancy , disk occupancy rate , net occupancy rate , Execution time: and delay tolerance .

3. The method according to claim 2, characterized in that: The method of constructing the refined model of the heterogeneous data center environment into a traditional Markov decision model includes the following steps: The state space construction step comprises: DAGNN is used to transform high-dimensional heterogeneous computing task features into low-dimensional homogeneous computing task embedding vectors, and three embedding vectors are obtained, namely, job vector, computing task vector and global vector. Among them, the jobs are processed according to the partial order defined by the computing power tasks, and the attention mechanism is used to The features of The operator indicates that for the Layer , output message vector , which means that The same layer of calculation All and homework Directly adjacent subtasks The job vector A weighted combination of: Using the associative operator The The job feature vector of the previous layer and the message vector for the job Aggregation to generate updated job vector : in , and They are the input, past state, and updated state / output of GRU respectively; The initial state is 0; go through After layer processing, use the readout operator Root Operation Perform maximum pooling calculation to generate computing power task vector : Generate a global vector of all computing tasks: in, and Represents a nonlinear transformation of a vector input; The state space defined according to the traditional Markov decision process is as follows: By the time in the training cycle , computing task trajectory , Server Collection and electricity prices constitute; Through the three obtained embedding vectors, the high-dimensional heterogeneous computing task state vector in the traditional Markov decision model is converted into a low-dimensional homogeneous vector, compressing the defined state space into: 。 4. The method according to claim 3, characterized in that: The method of constructing the refined model of the heterogeneous data center environment into a traditional Markov decision model includes the following steps of constructing the action space: The action space construction step comprises: Based on the DAGNN compressed embedding vector, we construct an action space that adapts to the homogeneous input of the pointer network: in, represents the selectable action sequence of the action space of the adaptive pointer network isomorphic input at time t, Represents each job The embedding vector of Indicates that the job Computing tasks The embedding vector of represents the global embedding vector, Indicates server information. Indicates that no job execution is selected.

5. The method according to claim 4, characterized in that: The method of constructing the refined model of the heterogeneous data center environment into a traditional Markov decision model includes the following state transfer construction steps: The state transfer construction step includes: Given state , the reinforcement learning agent at time step The corresponding actions are: in, Represents multiple job-server pairs The sequence composed of Indicates that the current job is not executed. If the action sequence selected at a certain time step is all If it is composed, it means that no action is selected in this step; When action After being determined, Interact with the environment to get the next state and rewards , this interaction process can be expressed as a mapping function: in, Indicates the resource usage of the corresponding job.

6. The method according to claim 5, characterized in that: The heterogeneous data center environment refinement model is constructed into a traditional Markov decision model, including the reward feedback construction step: The reward feedback construction step includes: designing a reward function: in, Indicates the running cycle of the current time step, and the value of each time step is uncertain; Indicates the system performance weight coefficient; , and Represent the energy consumption characteristic functions of different servers respectively. Indicates CPU utilization. represents the memory access count, Indicates the disk read / write rate. Indicates the network read and write rate.

7. The method according to claim 6, characterized in that: The traditional Markov decision model is optimized to form a heterogeneous task-heterogeneous resource dynamic combination optimization double-layer Markov decision model that adapts to the underlying logic of large-scale dynamic computing task scheduling in data centers. The optimized model is used to perform efficient scheduling under the directed acyclic graph neural network-pointer network-soft actor critic reinforcement learning algorithm framework, including: The traditional Markov decision model is improved into a two-layer Markov decision model for dynamic combination optimization of heterogeneous tasks and heterogeneous resources, so as to adapt it to the underlying logic of large-scale dynamic computing task scheduling in data centers; The pointer algorithm is embedded in the framework of the soft actor-critic algorithm to solve the problem of dynamic timing arrangement caused by the dynamic combination optimization of heterogeneous tasks and heterogeneous resources. The soft strategy iteration is used to maximize the goal, and the strategy evaluation and strategy improvement are performed alternately in the maximum entropy framework. Finally, an end-to-end training framework of directed acyclic graph neural network-pointer network-soft actor critic reinforcement learning algorithm is formed to achieve real-time online decision-making.

8. The method according to claim 7, characterized in that: The traditional Markov decision model is improved into a two-layer Markov decision model for dynamic combination optimization of heterogeneous tasks and heterogeneous resources to adapt to the underlying logic of large-scale dynamic computing task scheduling in data centers, including: The traditional Markov decision model with N-dimensional actions is reconstructed into a combined optimization Markov decision model containing N 1-dimensional action sequences, and the action value function of the reconstructed combined optimization Markov decision model is defined as: Among them, the definition ; For different time steps is changing, and the action values ​​of the two models are equal in some states: in, represents the action-value function of the reconstructed dynamic combinatorial optimization Markov decision model, Represents the action-value function of the traditional Markov decision model.

9. The method according to claim 8, characterized in that: The pointer algorithm is embedded in the soft actor-critic algorithm framework to solve the problem of dynamic timing arrangement caused by dynamic combination optimization of heterogeneous tasks and heterogeneous resources, and soft policy iteration is used to maximize the goal. At the same time, policy evaluation and policy improvement are alternately performed in the maximum entropy framework, including pointer network soft policy evaluation, pointer network soft policy improvement and pointer network soft policy iteration; The pointer algorithm is embedded in the framework of the soft actor-critic algorithm. First, the pointer network encoder is used to encode the feature vector, and then the decoder is combined with the attention calculation method to gradually construct the solution in an autoregressive manner to obtain the conditional probability. : Among them, given a training pair , represents the strategy sequence to be trained, The input of the encoder is read in sequence, and finally the vector V storing the input sequence information is encoded. At the same time, the encoder obtains the hidden state of each matching pair during the calculation process. ; The decoder decodes the vector V, which reads V and outputs the first hidden layer state. , using the attention mechanism according to The hidden state of each matching pair obtained by the encoder Calculate the probability of each matching pair, select the matching pair with the highest probability as the matching pair in the first step, and the decoder reads the hidden layer output of the previous step and the feature vector of the matching pair, and outputs the current hidden layer state ,according to The probability of selecting each matching pair is calculated and selected when the selected matching pair does not meet the resource upper limit constraint, or when an empty action occurs When , the pointer network stops outputting; The pointer network parameters are optimized using a discrete soft actor-critic algorithm to find a policy that maximizes the discounted return over time: in, Representation strategy; represents the optimal strategy; Representation strategy The distribution of induced trajectories; Determines the relative importance of the entropy term with respect to the reward, expressed as a temperature parameter; Representation strategy In Status The entropy of time, assuming The dimension of the input sequence for a certain time step, Represents a sequence of actions, , at the end of the time step, the reward the agent receives based on the pointer network decision ; The pointer network soft strategy evaluation includes: setting is the dimension of a certain action sequence, Representing action sequence , applied to Bellman operator is convergent: in: when Time convergence; The pointer network soft policy enhancement includes: defining a new policy target as: definition is the optimizer that maximizes the formula, for each , then and ; The pointer network soft strategy iteration includes: starting from any strategy or applying the pointer network soft strategy evaluation and improvement, the strategy sequence converges to , then , .

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 9 are implemented.

Citation Information

Patent Citations

  • Multi-strategy intelligent scheduling method and device oriented to heterogeneous computing power

    CN115237581A

  • Cloud computing heterogeneous task scheduling and container management method based on artificial intelligence

    CN117648174A

  • Deep reinforcement learning cooperative scheduling method and device for heterogeneous computing resources

    CN117909044A

  • Multi-policy intelligent scheduling method and apparatus oriented to heterogeneous computing power

    US20240111586A1

Cited By

  • Carrier rocket software control method and decision making device based on SAC

    CN119292752A

  • Maneuvering edge application scene-oriented computing power resource evaluation method and system

    CN119376955A

  • Decision-making large model-oriented multi-level heterogeneous memory collaborative scheduling method

    CN119576555A

  • Cloud resource dynamic scheduling method based on security reinforcement learning

    CN119576557A

  • Data center energy consumption optimization method based on joint multi-objective optimization model

    CN119620842A