Method and system for multi-data center task scheduling and data routing under network topology

Abstract

The application discloses a network topology multi-data center task scheduling and data routing method and system, belongs to the technical field of cloud computing, and models multi-geographical position data center scheduling under a complex topology network as a Markov decision process, defines states, actions, reward functions and transition functions to reflect changes in task and resource states, simultaneously models a data transmission process as a Boolean linear programming problem, optimizes a transmission starting point and a transmission path of data in a data center network topology, combines minimum energy consumption cost of data transmission with the reward function of MDP, enables an intelligent agent to learn to balance energy consumption cost of task scheduling and data transmission under multiple conditions such as a complex network topology, resource heterogeneity, task heterogeneity, complex task-data dependency and network bandwidth capacity constraints, and realizes overall optimization of data center energy consumption.

Description

Technical Field

[0001] This invention belongs to the field of cloud computing technology, specifically relating to a method and system for task scheduling and data routing in a network topology with multiple data centers. Background Technology

[0002] With the rapid development of cloud computing and the rapid advancement of digital transformation, the energy consumption problem of geographically distributed data centers is becoming increasingly prominent. Data center computing tasks have unique characteristics, allowing for transfer between data centers without relying on power transmission equipment. Therefore, it is possible to fully utilize the differences in electricity prices across multiple geographical locations to reduce energy costs for multi-location data centers. However, this process is also constrained by data distribution and data center network topology. Data is a prerequisite for task execution, while data transmission is limited by network topology and electricity prices. Therefore, task scheduling in geographically distributed data centers requires consideration of the coordinated scheduling of tasks and data.

[0003] Some research has been conducted on the task scheduling problem of data transmission in multi-geographic data centers:

[0004] 1. CN117880292A A Data Center Energy Optimization Method Based on Power-Computing Power-Communication Coordinated Scheduling: This paper proposes a data center energy optimization method based on power-computing power-communication coordinated scheduling. The method solves the model of minimizing the electricity cost of the data center and obtains the optimal scheduling scheme with the goal of minimizing the energy cost of the data center.

[0005] 2. CN118779084A Online Task Scheduling Method and System Considering Data Availability: This paper proposes an online task scheduling method considering data availability among multiple data centers, which achieves collaborative scheduling in scenarios with a large number of tasks and data through reinforcement learning multi-agent systems.

[0006] 3. The paper "Optimizing Network Transfers for Data Analytic Jobs across Geo-Distributed Datacenters" proposes a network transfer optimization method for data analysis tasks across geographically distributed data centers. These tasks generate network traffic between different data centers during execution, and the paper focuses on how to optimize this network traffic using multi-path methods across different countries.

[0007] 4. The paper "Energy and Network Aware Workload Management for Geographically Distributed Data Centers" addresses the issues of minimizing energy costs and allocating workloads across multiple data centers. It proposes a game theory-based workload management framework that makes intelligent scheduling decisions under the conditions of data transmission costs and data center queuing delays to solve the problem of reducing the overall operating costs of data centers.

[0008] 5. The paper "GEODIS: towards the optimization of data locality-aware jobscheduling in geo-distributed data centers" addresses the time optimization problem of data-intensive task scheduling in geo-distributed data centers. It proposes a linear programming approach for task placement and data transmission, and uses the GLPK solver to solve the problem, achieving a 44% reduction in task completion time.

[0009] However, current methods for task scheduling in multi-geographic data centers, considering data transmission, fail to achieve online scheduling of tasks and data to optimize data center energy costs, given the complex network topologies of these data centers. The methods in papers 3-5 typically focus on optimizing data transmission efficiency, neglecting energy consumption considerations for task scheduling and data transmission within the data center. Alternatively, they may only address data transmission between a small number of data centers with simple connections, assuming a simple one-to-one correspondence between data and tasks. Referring to points 1 and 2 above, these methods fail to consider the complex network topologies of data center clusters or the complex many-to-many dependencies between tasks and data, making them unsuitable for real-world physical scenarios. Summary of the Invention

[0010] To address the shortcomings of the existing technologies, this application provides a method and system for multi-datacenter task scheduling and data routing under a network topology.

[0011] The first aspect proposes a multi-datacenter task scheduling and data routing method under a network topology, including the following steps:

[0012] The task data collaborative scheduling problem of multi-geographic data centers in complex topology networks is modeled as a Markov decision process model.

[0013] A policy network is constructed and the Markov decision process model is solved by combining reinforcement learning and Boolean linear programming algorithms to obtain task scheduling results. Based on the task scheduling results, the target transmission location of the data required for various tasks is determined.

[0014] The data transmission process of the multi-geographic data center is modeled as a Boolean linear programming model. The Boolean linear programming model is called to optimize the optimal transmission path and starting point of the data required by various tasks by comprehensively considering electricity price, link energy consumption and capacity constraints. The minimum transmission cost is solved and the minimum data transmission cost solved by the Boolean linear programming model is used as part of the reward, which together with energy consumption cost, task default penalty and task completion benefit form a reward function.

[0015] By training the scheduling network using a strategy optimization method combined with the reward function, joint scheduling optimization of tasks and data can be achieved.

[0016] In some embodiments, modeling the task scheduling problem of multi-geographic data centers as a Markov decision process model includes:

[0017] Define state ,action Reward function and transfer function;

[0018] Define the state include:

[0019] Define information about all tasks that have not been executed at the current time t. , This represents the collection of task-dependent data, denoted as... ,in This represents the Kth piece of data that task j depends on. This refers to the resource usage information for the task. The duration of the task execution. The latest deadline for completing the task. The reward for completing the task. The average minimum weighted distance between all data for the task and each data center is expressed as: ,in This represents the average minimum weighted distance between all data of task j and data center m, and information about all data, including data volume and the presence of data in the data center network. Where d represents the index number of the data. This indicates the amount of data in the data set; This indicates whether the data exists in data center M at time step t; 1 indicates existence, 0 indicates non-existence. The remaining resource status of all data centers is represented as follows: ; Let represent the remaining CPU resources of data center M at time step t. The electricity price for all data centers is expressed as: , This represents the electricity price of data center M at time step t;

[0020] Define the action This includes the action of sequentially assigning tasks to be assigned at the current time t to a data center, including an empty action that does not assign any tasks to any data center. In this case, the task will be marked as unassigned and will continue to be assigned at the next time.

[0021] The reward function is defined as comprising four parts: energy consumption cost, data transmission cost, task execution penalty, and task completion reward.

[0022] Define energy consumption cost It is the negative product of actual energy consumption and electricity price. ,in Let m be the actual energy consumption of data center m at time t. Define the data transmission cost based on the electricity price at time t where data center m is located. Define the penalty for breach of task execution. for This indicates that if a task fails to be completed before the specified deadline as required by the user, a task execution violation penalty will be calculated. Indicates task Penalties for violations, and defined rewards for task completion. The reward for completing the task on time is represented as follows: in, This indicates tasks completed at time t that did not exceed their execution deadline. It is the reward for completing task j;

[0023] The transfer function represents the state. Take action below After transitioning to state The probability is such that the transition function is a deterministic mapping with a probability of 1 at all times.

[0024] In some embodiments, the formula for the average minimum weighted distance between all data and data center m is:

[0025]

[0026] Where src(d) represents the collection of all data centers where data d currently exists. This represents the network distance from candidate data center m to source data center s. This represents the minimum network distance from all source data centers s to candidate data center m for data d.

[0027] In some embodiments, constructing the policy network includes:

[0028] The design strategy network is represented as follows: , indicating the state Select action The probability of allocating tasks to data centers in a data center network is used to make decisions, and the policy network is represented as a probability distribution of action selection for all data centers and empty actions. ;

[0029] To ensure that the cumulative probability of the output is 1, a softmax layer is added before the output. The output probability is expressed as:

[0030] in, This represents the original output of the policy network, indicating the policy network's response to the action. In state The degree of preference is given by exp, which is an exponential function that converts the original scores into positive numbers, thereby constructing a probability distribution. This represents the summation of the exponents of all possible actions, used for normalization to ensure that the sum of the probabilities of the output is 1.

[0031] In some embodiments, the construction of a policy network and the solution of the Markov decision process model by combining reinforcement learning and Boolean linear programming algorithms to obtain task scheduling results, and the determination of the target transmission locations of data required for various tasks based on the task scheduling results, includes the following steps:

[0032] Step S1: For each agent, initialize the network parameters and policy network respectively. and value network in For the parameters of the policy network, For the parameters of the value network, initialize the learning rate and optimizer of each network, initialize the hyperparameters, including the learning rate and pruning parameters, the discount factor and smoothing parameter of the generalized advantage estimation method, and initialize the parameters of the Boolean linear programming solution module;

[0033] Step S2: Environment initialization, establishing a new data center cluster state based on task arrival and data upload status. This includes task information, data location, electricity price, and resource status;

[0034] Step S3: The agent, based on the current policy... Generate actions sequentially Sample the tasks to be assigned at the current moment and choose to assign them to the data center or not;

[0035] Step S4: Perform all actions to obtain a new state. Analyze the target location to which the data required to execute all tasks needs to be transmitted;

[0036] Step S5: Call the BLP solver to obtain the optimal data transmission path and the minimum data transmission energy cost;

[0037] Step S6: Calculate the reward function Reward function =Energy cost +Data transmission cost + Penalty for breach of task +Rewards for completing tasks on time ;

[0038] Step S7: Obtain the state ,action ,award Next time step state and the logarithm of the probability of choosing an action. It is placed in the replay cache as a result of task scheduling.

[0039] In some embodiments, the value network The input to the value network is consistent with that of the policy network, containing all information about the current environment: task queue status, remaining data center resources, electricity price, data distribution, and network topology information. The output of the value network is a scalar used to estimate the expected total reward that can be obtained by executing tasks and scheduling data according to the current policy in the current state. The value network updates its parameters by minimizing the mean squared error between the predicted value and the actual reward. This provides accurate state value assessment for strategy optimization.

[0040] In some embodiments, modeling the data transmission process of the multi-geographic data center as a Boolean linear programming model includes:

[0041] definition Represents a collection of data types. This refers to a collection of data centers, where any one data point is considered a data center. The amount of data is represented as There is data The data center node is represented as ,data The destination to be transmitted is represented as The data center network topology is represented as ,in Indicates the connection lines between data centers;

[0042] Based on three Boolean variables for decision-making: the set of data types, the set of data centers, the data volume, the data center nodes, the transmission endpoint, and the data center network topology: This determines whether data k is transferred from data center i to data center j. This determines whether data center i is the final storage location for data k. It is an auxiliary binary variable used for linearization processing;

[0043] Construct the objective function:

[0044] After linearization, it is represented as:

[0045]

[0046] in, This refers to the power consumption of a router transmitting a unit of data. It is the energy consumption of the line for transmitting a unit of data. and It is a data center , Electricity price, This indicates the amount of data transmitted for data k.

[0047] In some embodiments, the Boolean linear programming model includes:

[0048] First constraint: Flow constraints at the data source, transit point, and endpoint. For the data transmission source, outflow must always be greater than inflow.

[0049]

[0050] For transit points, inflows should equal outflows:

[0051]

[0052] For each endpoint, the inflow must be greater than 1, and it must be a unique endpoint.

[0053] The second constraint is the line capacity constraint for data transmission, ensuring that the total data transmission volume of each line does not exceed its capacity limit.

[0054]

[0055] in This represents the maximum data transfer capacity between data centers i and j;

[0056] Third constraint: Ensure that the required data has an outflow:

[0057]

[0058] Fourth constraint:

[0059] Ensure that data is not transmitted back and forth:

[0060] ;

[0061] Linearization constraints:

[0062] .

[0063] In some embodiments, the joint scheduling optimization of tasks and data by training the scheduling network through a policy optimization method combined with the reward function includes:

[0064] Calculate the advantage estimate for each time step using generalized advantage estimation:

[0065]

[0066] It is TD error. It is a discount factor used to control the importance of future rewards. It is a smoothing parameter that controls the degree of focus on long-term estimates. It is a value estimate of the current state. It is a value estimate of the next state. It is the final advantage estimate used to train the policy network;

[0067] Calculate the probability ratio of the old strategy:

[0068]

[0069] Calculate the loss function of the policy network:

[0070]

[0071] Calculate the value network loss function:

[0072] Update parameters using the Adam optimizer. and Scheduling optimization is achieved by updating parameters.

[0073] Secondly, this application proposes a multi-datacenter task scheduling and data routing system under a network topology, including a datacenter modeling module, a task scheduling module, a data routing module, and a scheduling optimization module;

[0074] The data center modeling module is used to model the task data collaborative scheduling problem of multi-geographical data centers under complex topology networks as a Markov decision process model.

[0075] The task scheduling module is used to construct a policy network and solve the Markov decision process model by combining reinforcement learning and Boolean linear programming algorithms to obtain task scheduling results. Based on the task scheduling results, the target transmission location of the data required for various tasks is determined.

[0076] The data routing module is used to model the data transmission process of the multi-geographic data center as a Boolean linear programming model, call the Boolean linear programming model, comprehensively consider electricity price, link energy consumption and capacity constraints to optimize the optimal transmission path and starting point of the data required by various tasks, and solve for the minimum transmission cost. The minimum data transmission cost solved by the Boolean linear programming model is used as part of the reward, and together with energy consumption cost, task default penalty and task completion benefit, they form a reward function.

[0077] The scheduling optimization module is used to train the scheduling network by combining the strategy optimization method with the reward function to achieve joint scheduling optimization of tasks and data.

[0078] Thirdly, this application proposes an electronic device including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the method described above.

[0079] Fourthly, this application provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the method described above.

[0080] The beneficial effects of this invention are:

[0081] This invention models the scheduling of multi-geographical data centers in complex topology networks, including the setting of states, actions, reward functions, and transition functions in Multidimensional Data Programming (MDP). It proposes Boolean linear programming, making decisions based on optimizing data transmission paths with low data transmission costs. This is combined with MDP, enabling the agent to learn action decisions that simultaneously consider task execution costs and data transmission costs. Under the complex network topology of geographically distributed data centers, it considers the trade-off between task scheduling and data transmission energy costs, achieving overall optimization of data center energy consumption. It also considers a fast online solution method under various complex constraints such as resource heterogeneity, task heterogeneity, data routing, and complex task-data dependencies, realizing an online task scheduling strategy. Attached Figure Description

[0082] Figure 1 This is the overall flowchart of the present invention.

[0083] Figure 2 This is a system principle block diagram of the present invention. Detailed Implementation

[0084] Exemplary embodiments of the invention will now be described in more detail with reference to the accompanying drawings. While exemplary embodiments of the invention are shown in the drawings, it should be understood that the invention may be implemented in various forms and should not be limited to the embodiments set forth herein; rather, these embodiments are provided so that a more thorough understanding of the invention can be achieved and that the full scope of the invention can be conveyed to those skilled in the art.

[0085] The first aspect proposes a multi-datacenter task scheduling and data routing method under network topology, such as... Figure 1 As shown, it includes the following steps:

[0086] S100: Model the task data collaborative scheduling problem of multi-geographical data centers in complex topology networks as a Markov decision process model;

[0087] This invention proposes an online task and data collaborative scheduling method for multi-geographic data centers, comprising: 1) modeling the task scheduling process as a Markov Decision Process (MDP) to characterize the task placement strategy that changes over time, ensuring that dynamic task scheduling can adapt to different data distributions and resource states; and 2) modeling the data transmission process as a Binary Linear Programming (BLP) to optimize data transmission paths and data management, ensuring that the data scheduling process can meet the computational requirements of tasks while minimizing transmission overhead. Through this modeling framework, we can optimize data transmission paths and reduce overall data center energy consumption while ensuring efficient utilization of computing resources, thereby improving task execution efficiency in multi-data center environments.

[0088] In some embodiments, modeling the task scheduling problem of multi-geographic data centers as a Markov decision process model includes:

[0089] Define state ,action Reward function and transfer function;

[0090] Define the state include:

[0091] Define information about all tasks that have not been executed at the current time t. , This represents the collection of task-dependent data, denoted as... ,in This represents the Kth piece of data that task j depends on. This refers to the resource usage information for the task. The duration of the task execution. The latest deadline for completing the task. The reward for completing the task. The average minimum weighted distance between all data for the task and each data center is expressed as: ,in This represents the average minimum weighted distance between all data of task j and data center m, and information about all data, including data volume and the presence of data in the data center network. Where d represents the index number of the data. This indicates the amount of data in the data set; This indicates whether the data exists in data center M at time step t; 1 indicates existence, 0 indicates non-existence. The remaining resource status of all data centers is represented as follows: ; Let represent the remaining CPU resources of data center M at time step t. The electricity price for all data centers is expressed as: , This represents the electricity price of data center M at time step t;

[0092] Define the action This includes the action of sequentially assigning tasks to be assigned at the current time t to a data center, including an empty action that does not assign any tasks to any data center. In this case, the task will be marked as unassigned and will continue to be assigned at the next time.

[0093] The reward function is defined as comprising four parts: energy consumption cost, data transmission cost, task execution penalty, and task completion reward.

[0094] Define energy consumption cost It is the negative product of actual energy consumption and electricity price. ,in Let m be the actual energy consumption of data center m at time t. Define the data transmission cost based on the electricity price at time t where data center m is located. Define the penalty for breach of task execution. for This indicates that if a task fails to be completed before the specified deadline as required by the user, a task execution violation penalty will be calculated. Indicates task Penalties for violations, and defined rewards for task completion. The reward for completing the task on time is represented as follows: in, This indicates tasks completed at time t that did not exceed their execution deadline. It is the reward for completing task j;

[0095] The transfer function represents the state. Take action below After transitioning to state The probability is such that the transition function is a deterministic mapping with a probability of 1 at all times.

[0096] In some embodiments, the formula for the average minimum weighted distance between all data and data center m is:

[0097]

[0098] Where src(d) represents the collection of all data centers where data d currently exists. This represents the network distance from candidate data center m to source data center s. This represents the minimum network distance from all source data centers s to candidate data center m for data d.

[0099] S200: Construct a policy network and solve the Markov decision process model by combining reinforcement learning and Boolean linear programming algorithms to obtain task scheduling results. Based on the task scheduling results, determine the target transmission location of the data required for various tasks.

[0100] In some embodiments, constructing the policy network includes:

[0101] The design strategy network is represented as follows: , indicating the state Select action The probability of allocating tasks to data centers in a data center network is used to make decisions, and the policy network is represented as a probability distribution of action selection for all data centers and empty actions. ;

[0102] To ensure that the cumulative probability of the output is 1, a softmax layer is added before the output. The output probability is expressed as:

[0103] in, This represents the original output of the policy network, indicating the policy network's response to the action. In state The degree of preference is given by exp, which is an exponential function that converts the original scores into positive numbers, thereby constructing a probability distribution. This represents the summation of the exponents of all possible actions, used for normalization to ensure that the sum of the probabilities of the output is 1.

[0104] In some embodiments, the construction of a policy network and the solution of the Markov decision process model by combining reinforcement learning and Boolean linear programming algorithms to obtain task scheduling results, and the determination of the target transmission locations of data required for various tasks based on the task scheduling results, includes the following steps:

[0105] Step S1: For each agent, initialize the network parameters and policy network respectively. and value network in For the parameters of the policy network, For the parameters of the value network, initialize the learning rate and optimizer of each network, initialize hyperparameters including learning rate and pruning parameters, discount factor and smoothing parameters of the generalized advantage estimation method, initialize Boolean linear programming solution module parameters, and build a simulation environment including task queue, data center topology and electricity price simulation. Among them, the generated task data is based on the heavy-tail effect of task generation, the topology is randomly generated using a graph structure based on the scale of cloud service providers such as Google, and the electricity price is also randomly generated according to existing electricity pricing models such as flat pricing, TOU and CPP.

[0106] Step S2: Environment initialization, establishing a new data center cluster state based on task arrival and data upload status. This includes task information, data location, electricity price, and resource status;

[0107] Step S3: The agent, based on the current policy... Generate actions sequentially Sample the tasks to be assigned at the current moment and choose to assign them to the data center or not;

[0108] Step S4: Perform all actions to obtain a new state. Analyze the target location to which the data required to execute all tasks needs to be transmitted;

[0109] Step S5: Call the BLP solver to obtain the optimal data transmission path and the minimum data transmission energy cost;

[0110] Step S6: Calculate the reward function Reward function =Energy cost +Data transmission cost + Penalty for breach of task +Rewards for completing tasks on time ;

[0111] Step S7: Obtain the state ,action ,award Next time step state and the logarithm of the probability of choosing an action. It is placed in the replay cache as a result of task scheduling.

[0112] In some embodiments, the value network The input to the value network is consistent with that of the policy network, containing all information about the current environment: task queue status, remaining data center resources, electricity price, data distribution, and network topology information. The output of the value network is a scalar used to estimate the expected total reward that can be obtained by executing tasks and scheduling data according to the current policy in the current state. The value network updates its parameters by minimizing the mean squared error between the predicted value and the actual reward. This provides accurate state value assessment for strategy optimization.

[0113] S300: The data transmission process of the multi-geographic data center is modeled as a Boolean linear programming model. The Boolean linear programming model is called to comprehensively consider electricity price, link energy consumption and capacity constraints to optimize the optimal transmission path and starting point of the data required by various tasks, and solve for the minimum transmission cost. The minimum data transmission cost solved by the Boolean linear programming model is used as part of the reward, and together with energy consumption cost, task default penalty and task completion benefit, it forms a reward function.

[0114] In some embodiments, to solve the model, an algorithm combining reinforcement learning and Boolean linear programming is designed. The minimum transmission cost solved by the Boolean linear programming is used as part of the reward and perceived by the reinforcement learning. Through continuous learning, a task allocation strategy is learned to optimize the energy cost of data centers under complex network topologies. The process of modeling the data transmission process of the multi-geographical data centers as a Boolean linear programming model includes:

[0115] definition Represents a collection of data types. This refers to a collection of data centers, where any one data point is considered a data center. The amount of data is represented as There is data The data center node is represented as ,data The destination to be transmitted is represented as The data center network topology is represented as ,in Indicates the connection lines between data centers;

[0116] Based on three Boolean variables for decision-making: the set of data types, the set of data centers, the data volume, the data center nodes, the transmission endpoint, and the data center network topology: This determines whether data k is transferred from data center i to data center j. This determines whether data center i is the final storage location for data k. It is an auxiliary binary variable used for linearization processing;

[0117] Construct the objective function:

[0118] After linearization, it is represented as:

[0119]

[0120] in, This refers to the power consumption of a router transmitting a unit of data. It is the energy consumption of the line for transmitting a unit of data. and It is a data center , Electricity price, This indicates the amount of data transmitted for data k.

[0121] In some embodiments, the Boolean linear programming model includes:

[0122] First constraint: Flow constraints at the data source, transit point, and endpoint. For the data transmission source, outflow must always be greater than inflow.

[0123]

[0124] For transit points, inflows should equal outflows:

[0125]

[0126] For each endpoint, the inflow must be greater than 1, and it must be a unique endpoint.

[0127] The second constraint is the line capacity constraint for data transmission, ensuring that the total data transmission volume of each line does not exceed its capacity limit.

[0128]

[0129] in This represents the maximum data transfer capacity between data centers i and j;

[0130] Third constraint: Ensure that the required data has an outflow:

[0131]

[0132] Fourth constraint:

[0133] Ensure that data is not transmitted back and forth:

[0134] ;

[0135] Linearization constraints:

[0136] .

[0137] S400: By combining the reward function with a strategy optimization method to train the scheduling network, joint scheduling optimization of tasks and data is achieved.

[0138] In some embodiments, the joint scheduling optimization of tasks and data by training the scheduling network through a policy optimization method combined with the reward function includes:

[0139] Calculate the advantage estimate for each time step using generalized advantage estimation:

[0140]

[0141] It is TD error. It is a discount factor used to control the importance of future rewards. It is a smoothing parameter that controls the degree of focus on long-term estimates. It is a value estimate of the current state. It is a value estimate of the next state. It is the final advantage estimate used to train the policy network;

[0142] Calculate the probability ratio of the old strategy:

[0143]

[0144] Calculate the loss function of the policy network:

[0145]

[0146] Calculate the value network loss function:

[0147] Update parameters using the Adam optimizer. and Scheduling optimization is achieved by updating parameters.

[0148] Secondly, this application proposes a multi-datacenter task scheduling and data routing system under a network topology, such as... Figure 2 As shown, it includes a data center modeling module, a task scheduling module, a data routing module, and a scheduling optimization module;

[0149] The data center modeling module is used to model the task data collaborative scheduling problem of multi-geographical data centers under complex topology networks as a Markov decision process model.

[0150] The task scheduling module is used to construct a policy network and solve the Markov decision process model by combining reinforcement learning and Boolean linear programming algorithms to obtain task scheduling results. Based on the task scheduling results, the target transmission location of the data required for various tasks is determined.

[0151] The data routing module is used to model the data transmission process of the multi-geographic data center as a Boolean linear programming model, call the Boolean linear programming model, comprehensively consider electricity price, link energy consumption and capacity constraints to optimize the optimal transmission path and starting point of the data required by various tasks, and solve for the minimum transmission cost. The minimum data transmission cost solved by the Boolean linear programming model is used as part of the reward, and together with energy consumption cost, task default penalty and task completion benefit, they form a reward function.

[0152] The scheduling optimization module is used to train the scheduling network by combining the strategy optimization method with the reward function to achieve joint scheduling optimization of tasks and data.

[0153] Thirdly, this application proposes an electronic device including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the method described above.

[0154] Fourthly, this application provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the method described above.

[0155] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is merely an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. Furthermore, the specific names of the functional units and modules are only for easy differentiation and are not intended to limit the scope of protection of this application. The specific working process of the units and modules in the above system can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0156] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0157] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this disclosure.

[0158] In the embodiments provided in this disclosure, it should be understood that the disclosed apparatus / computer devices and methods can be implemented in other ways. For example, the apparatus / computer device embodiments described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. Multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the mutual coupling or direct coupling or communication connection shown or discussed may be through some interfaces, and the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0159] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment, depending on actual needs.

[0160] Furthermore, the functional units in the various embodiments of this disclosure can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0161] If an integrated module / unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments can also be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program may include computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. A computer-readable medium may include: any entity or device capable of carrying computer program code, recording media, USB flash drives, portable hard drives, magnetic disks, optical disks, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc. It should be noted that the content included in a computer-readable medium may be appropriately added to or subtracted according to the requirements of legislation and patent practice in a jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable media may not include electrical carrier signals and telecommunication signals.

[0162] The above are merely preferred embodiments of the present invention. It should be noted that any modifications and improvements made by those skilled in the art without departing from the present technical solution should also be considered to fall within the scope of protection claimed by the present solution.

Claims

1. A method for multi-data center task scheduling and data routing under network topology, characterized in that: The method comprises the following steps: The task data cooperative scheduling problem of the multi-geographical location data center under the complex topology network is modeled as a Markov decision process model; The strategy network is constructed based on reinforcement learning, and the construction of the strategy network comprises: A policy network is represented as: , representing the probability of selecting an action in a state . In a data center network making decisions to allocate tasks to data centers, the policy network is represented as a probability distribution over action selection for all data centers and an empty action ; In order to ensure that the cumulative value of the output probability is 1, a softmax layer is added before the output, and the output probability is expressed as: where, represents the original output of the policy network, represents the preference degree of the policy network to the action in the state , exp is an exponential function, which is used to convert the original score to a positive number, so as to construct a probability distribution, represents the sum of the exponential values of all possible actions, which is used for normalization to ensure that the sum of the probabilities of the output is 1; The Markov decision process model is solved by combining a Boolean linear programming to obtain a task scheduling result, and based on the task scheduling result, a target transmission location of data required by each type of task is determined, comprising the following steps: Step S1: initialize the network parameters for each agent respectively, the policy network and the value network wherein is the parameter of the policy network, is the parameter of the value network, initialize the learning rate and optimizer of each network, initialize the hyperparameters, including the learning rate and the clipping parameter, the discount factor and the smoothing parameter of the generalized advantage estimation method, initialize the Boolean linear programming solution module parameters; Step S2: environment initialization, forming a new data center cluster state according to the arrival of tasks and the uploading of data , including task information, data location, electricity price, and resource conditions; Step S3: the agent assigns the task to the data center according to the current strategy Generate actions in turn Sample the task to be assigned at the current time, and select to assign it to the data center or not Step S4: performing all actions, resulting in a new state analyzing the target location to which the data required for performing all tasks needs to be transmitted; Step S5: calling a BLP solver to obtain an optimal data transmission path and a minimum data transmission energy consumption cost; Step S6: Calculate the reward function : Reward function = Energy cost + Data transmission cost + Penalty for violation of tasks + Reward for timely completion of tasks ; Step S7: Put the resulting state , action , reward , next time step state , and log probability of the selected action into the replay buffer as a result of task scheduling. The data transmission process of the multi-geographical location data center is modeled as a Boolean linear programming model, the optimal transmission path and starting point of data required by each type of task are optimized by comprehensively considering the electricity price, link energy consumption and capacity constraint, and the minimum transmission cost is solved, the minimum data transmission cost solved by the Boolean linear programming model is taken as part of the reward, and the energy consumption cost, task default penalty and task completion benefit jointly constitute a reward function, comprising: Definitions a collection of data types, a collection of data centers, any one data a data volume is represented by a data center node is represented by a data center node is represented by a data center node is represented by a data center node is represented by a data center network topology is represented by wherein a connection line between data centers is represented by Three decision-making Boolean variables are constructed according to the union of data types, the union of data centers, the data volume, the data center node, the end of transmission and the data center network topology: , whether data k is transmitted from data center i to data center j, , whether data center i is the final storage location of data k, is a auxiliary binary variable for linearization processing; The objective function is constructed: After linearization, it is expressed as: wherein, is the router energy consumption to transmit a unit of data, is the line transmission energy consumption to transmit a unit of data, and is the data center , the electricity price, denotes the transmission data volume of data k; The scheduling network is trained by combining the reward function through a strategy optimization method to realize joint scheduling optimization of tasks and data.

2. The method of claim 1, wherein: The task data cooperative scheduling problem of the multi-geographical location data center under the complex topology network is modeled as a Markov decision process model, comprising: define states , actions , reward functions and transition functions; defining the state comprising: Definition of information of all tasks not executed at current time t , represents the set of task dependency data, denoted as , where represents the Kth data that task j depends on, is the resource information occupied by the task, is the execution duration of the task, is the latest deadline for the completion of the task, is the benefit of the completion of the task, is the average minimum weighted distance of all data of the task and each data center, denoted as , where represents the average minimum weighted distance of all data of task j and data center m, the information of all data, including data volume and the existence of data in the data center network , where d represents the index number of the data, represents the data volume of the data; represents whether the data exists in data center M at time step t, 1 for existence and 0 for non-existence; the resource remaining situation of all data centers is represented as: ; represents the remaining cpu resource number of data center M at time step t, the electricity price of all data centers is represented as: , represents the electricity price of data center M at time step t; defining said action comprises the action of assigning in turn the tasks to be assigned at the current time instant t to a certain data center, including the null action of not assigning a task to any data center, so that the task will be marked as not assigned, continuing with the assignment at the next time instant; The reward function is defined, comprising four parts of energy consumption cost, data transmission cost, task execution default penalty and task completion benefit: Definition of energy cost is the actual energy consumption of data center m at time t, where is the actual energy consumption of data center m at time t, is the electricity price of the location of data center m at time t, Definition of data transmission cost , Definition of task execution penalty is , represents the task execution penalty when the task is not completed before the specified deadline according to the user's demand, where represents the task penalty for violation, Definition of task completion benefit is the reward for completing the task on time, which is represented as: where, represents the task completed at time t that does not exceed the deadline for task execution, is the benefit of task j completion for The transition function represents the probability of transitioning to state from state after taking action The transition function is a deterministic mapping, the probability is always 1.

3. The method of claim 2, wherein: The average minimum weighted distance formula of all data and data centers m is: where src(d) denotes the union of all data centers where data d currently resides, denotes the network distance from candidate data center m to source data center s, denotes the minimum network distance from all source data centers s of data d to candidate data center m.

4. The method of claim 3, wherein: The Boolean linear programming model comprises: The first constraint condition is the flow constraint of the data source, transfer point and end point, and for the source point of data transmission, the outflow is greater than the inflow: For a transit point, inflow should equal outflow: For an end point, inflow is greater than 1, and needs to be guaranteed to be a unique end point: Second constraint: line capacity constraint of data transmission, which ensures that the total amount of data transmission of each line does not exceed its capacity limit: wherein denotes the maximum data transmission capacity between data centers i, j; The third constraint condition is to ensure that the required data has outflow: Fourth constraint: To ensure that data is not transmitted back and forth: ； The linear constraint condition is: 。 5. The method of claim 4, wherein: The value network The input of the value network is consistent with the policy network, containing all the information in the current environment state: task queue state, data center remaining resources, electricity price, data distribution, and network topology information. The output of the value network is a scalar, which is used to estimate the expected total reward that can be obtained by executing the task and data scheduling according to the current policy in the current state. The value network updates its parameters by minimizing the mean square error between the predicted value and the actual return , providing accurate state value evaluation for policy optimization.

6. The method of claim 5, wherein: The scheduling network is trained by combining the reward function through a strategy optimization method to realize joint scheduling optimization of tasks and data, comprising: The advantage estimation value of each time step is calculated by using generalized advantage estimation: is the TD error, is a discount factor that controls the importance of future rewards, is a smoothing parameter that controls the extent to which the estimate of the long-term value is focused on, is the value estimate for the current state, is the value estimate for the next state, is the advantage estimate that is ultimately used to train the policy network; The old strategy probability ratio is calculated: Compute policy network loss function: computing a value network loss function: updating the parameters by an Adam optimizer and scheduling optimization by updating the parameters.

7. A multi-data center task scheduling and data routing system under network topology, characterized in that: The data center modeling module, the task scheduling module, the data routing module and the scheduling optimization module are comprised; The data center modeling module is used for modeling the task data cooperative scheduling problem of the multi-geographical location data center under the complex topology network as a Markov decision process model; The task scheduling module is used for constructing a strategy network based on reinforcement learning, and the construction of the strategy network comprises: A policy network is represented as: , representing the probability of selecting an action in a state Making a decision to allocate tasks to data centers in a data center network, the policy network is represented as a probability distribution over action selection for all data centers and an empty action ; In order to ensure that the cumulative value of the output probability is 1, a softmax layer is added before the output, and the output probability is expressed as: where, represents the original output of the policy network, represents the policy network's preference for action at state , exp is an exponential function that serves to convert the raw score to a positive number, thus constructing a probability distribution, represents the sum of the exponential values for all possible actions, used for normalization, ensuring that the sum of the probabilities of the output is 1; The Markov decision process model is solved by combining a Boolean linear programming to obtain a task scheduling result, and based on the task scheduling result, a target transmission location of data required by each type of task is determined, comprising the following steps: Step S1: initialize the network parameters for each agent respectively, the policy network and the value network where is the parameter of the policy network, is the parameter of the value network, initialize the learning rate and optimizer of each network, initialize the hyperparameters, including the learning rate and the clipping parameter, the discount factor and the smoothing parameter of the generalized advantage estimation method, initialize the Boolean linear programming solution module parameters; Step S2: environment initialization, forming a new data center cluster state according to the arrival of tasks and the uploading of data , including task information, data location, electricity price, and resource conditions; Step S3: the agent assigns the task to the data center according to the current strategy Generate actions in turn Sample the task to be assigned at the current time, and select to assign it to the data center or not to assign Step S4: performing all actions, resulting in a new state analyzing the target location to which the data required for performing all tasks needs to be transmitted; Step S5: calling a BLP solver to obtain an optimal data transmission path and a minimum data transmission energy consumption cost; Step S6: Calculate the reward function : Reward function = Energy cost + Data transmission cost + Penalty for violation of tasks + Reward for timely completion of tasks ; Step S7: Obtain the state ,action ,award Next time step state and the logarithm of the probability of choosing an action. It is placed in the replay cache as a task scheduling result; The data routing module is configured to model a data transmission process of the multi-geographical location data center as a Boolean linear programming model, call the Boolean linear programming model, comprehensively consider electricity price, link energy consumption and capacity constraint to optimize optimal transmission paths and starting points of data required by various tasks, and solve a minimum transmission cost, take the minimum data transmission cost solved by the Boolean linear programming model as a part of a reward, and combine an energy consumption cost, a task default penalty and a task completion benefit to form a reward function, and the reward function comprises: Definitions a collection of data types, a collection of data centers, any one data a data volume is represented by a data center node is represented by a data center node is represented by a data center node is represented by a data center node is represented by a data center node is represented by a data center network topology is represented by a data center node is represented by Three decision-making Boolean variables are constructed according to the union of data types, the union of data centers, the data volume, the data center node, the end of transmission and the data center network topology: whether data k is transmitted from data center i to data center j, whether data center i is the final storage location of data k, is a auxiliary binary variable for linearization processing; constructing an objective function: After linearization, it is expressed as: wherein, is the router energy consumption to transmit a unit of data, is the line transmission energy consumption to transmit a unit of data, and is the data center , , denotes the transmission data volume of data k; The scheduling optimization module is configured to train a scheduling network by a strategy optimization method in combination with the reward function, and realize joint scheduling optimization of tasks and data.

Images