Multi-data center online task scheduling method and system considering data availability

Abstract

The present invention discloses a multi-data center online task scheduling method and system considering data availability, and relates to the technical field of data center task scheduling. The task execution module stores multiple tasks received at different times into a processing task queue, and selects part of the tasks in the processing queue at each time to be assigned to the corresponding data center for execution; the data transmission module manages the data of the data center cluster, transmits part of the data in the data center at each time, and updates the data stored in multiple data centers; through a deep reinforcement learning algorithm, the task execution module and the data transmission module are coupled to construct a service provider electricity cost minimization model that meets the task service quality constraints. The present invention schedules unfinished tasks and newly generated data online; processes a large amount of data through deep neural network learning, adapts to the situation of large data scale and multiple time steps, and effectively solves the scenario of large amounts of data and tasks on a large scale.

Description

Technical Field

[0001] The present invention relates to the technical field of data center task scheduling, and in particular to a multi-data center online task scheduling method and system considering data availability. Background Art

[0002] In order to complete the calculation of large-scale data tasks, computing service providers have built multiple data centers. As the scale of data centers expands, the overall electricity consumption increases day by day, and the electricity cost is also increasing. In the context of data centers distributed in multiple locations, data center computing tasks have flexibility in both time and space. Making full use of this flexibility can effectively reduce the electricity cost of data centers. The flexible scheduling of data centers in space has brought about the problem of data availability. The calculation of tasks depends on data. Therefore, transferring computing tasks to data centers in other geographical locations for execution also requires the transfer of the data required for the calculation. For some tasks that rely on large amounts of data, the communication time and electricity costs required to transfer data cannot be ignored. Previous studies have mostly focused on the task scheduling problem of a single data center, without considering the dependency between tasks and data (such as patent cn117648174A, cloud computing heterogeneous task scheduling and container management method based on artificial intelligence). For the few studies that consider the dependency between tasks and data, they are usually solved by establishing operations research models (such as Xiaoping Li, Wei Yu, Rub´en Ruiz, Jie Zhu. Energy-aware cloud workflow applications scheduling with geo-distributed data. IEEE Transactions on Services Computing, 2020, DoI:10. 1109 / TSC .2020.2965106). The task scheduling calculation process of multiple data centers is modeled, and the objective function is to minimize the total cost of the computing service provider (including the electricity cost of data communication and the electricity cost of task calculation). Under the constraints of data center computing resources and communication resources between data centers, the optimal multi-data center task scheduling strategy is found. This method is a global optimal solution obtained when the task arrival status in all time steps and all data are known.

[0003] At present, the task scheduling problem of data centers basically does not take into account the strong dependency between data and tasks, but only schedules the tasks. This scheduling is too idealistic. For the task scheduling problem of data centers that takes data availability into consideration, the optimal multi-data center task scheduling strategy is mainly sought by establishing an operations research model. This method usually obtains the global optimal solution when all the tasks are known and the data is uploaded. This is contrary to the reality that the data center subject is unknown about the future task arrival and data upload. At the same time, as the task scale expands and the time step increases, since the original method obtains the global optimal solution, the computational time of the original method will increase dramatically, and it cannot meet the needs of real large-scale scenarios. Therefore, it is still necessary to propose a scheduling algorithm based on deep reinforcement learning to solve the above problems. Summary of the invention

[0004] The technical problem to be solved by the present invention is that in the past, the optimal multi-data center task scheduling strategy was sought by establishing an operations research model. As the task scale expands and the time step grows, the calculation time will increase sharply, which cannot meet the needs of real large-scale scenarios. The present invention proposes a multi-data center online task scheduling method and system that takes data availability into consideration, which provides a multi-data center task scheduling method based on deep reinforcement learning to solve the above technical problems.

[0005] The present invention is achieved through the following technical solutions:

[0006] The method for online task scheduling of multiple data centers considering data availability includes: a task execution module stores multiple tasks received at different times into a processing task queue, and selects part of the tasks in the processing task queue at each time to be assigned to the corresponding data center for execution; a data transmission module manages the data of a data center cluster, transmits part of the data in the data center at each time, and updates the data stored in multiple data centers; and through a deep reinforcement learning algorithm, the task execution module and the data transmission module are coupled to construct a service provider electricity cost minimization model that meets the task service quality constraints.

[0007] The task execution module selects part of the tasks in the processing task queue at each moment and assigns them to the corresponding data center for execution, including: according to the current moment, the electricity price and remaining available CPU resources of each data center, the data required to execute the current task, whether the task is assigned, the assigned data center, the executable status of the task, the remaining CPU resources required to complete the task, the latest feasible completion time of the task, the status of all currently unfinished tasks, and the migration and storage status of all data; and assigns each task in the processing task queue to the corresponding data center for execution.

[0008] The multi-data center online task scheduling method considering data availability also includes:

[0009] Assume that there is a data centers, each data center is numbered as ;

[0010] The state space of the task execution module is:

[0011] ;

[0012] represents the current time step; Indicates data center electricity price; express CPU resource usage of each data center; Indicates the data number required to execute the task job. Indicates the distribution of tasks, there are +1 possibility, if =0 means the task is not assigned, = Indicates allocation to data center middle; Indicates the executable status of the task. If =0 means the data center where the task is located does not have the data required by the task , the task is not executable; Indicates the remaining amount of CPU resources required to complete the task. Indicates the execution deadline of the task. Represents the collection embedding of all task information at the current moment, Represents the collection embedding of all data information at the current moment.

[0013] The data transmission module selects the data required for task execution and transmits it to the data center, including: selecting the data departure point and the data reception point; selecting the number of bandwidth resources allocated to the data at this moment; assigning a transmission priority to the transmission of each data, and allocating bandwidth resources in descending order according to the transmission priority until all data are allocated, or there are insufficient bandwidth resources or switch resources in the data center network.

[0014] The multi-data center online task scheduling method considering data availability also includes: assuming that there are data centers, each data center is numbered as ;

[0015] The state space of the data transmission module is represented as:

[0016]

[0017] represents the current time step; Indicates data center The electricity price, Indicates the index number of the current data. Indicates the total amount of data contained in the data; Indicates the amount of data remaining to be transmitted at this time; Indicates the bandwidth resource usage in the network topology. * The matrix of Indicates the switch power resource usage of each data center; Indicates the existence of data in each data center, which is a (1, ), 1 means the data exists, 0 means the data does not exist; It is a record of the data transmission direction at the current moment. , j), used to record the current moment data from the data center Transmitted to data center j; j ; Represents the collection embedding of all task information at the current moment, Represents the collection embedding of all data information at the current moment.

[0018] The reward function of the deep reinforcement learning algorithm includes the negative computational cost of task processing in the data center, the negative communication cost of data transfer in the data center, the negative deadline violation penalty and the positive reward for completing the task on time.

[0019] The invention relates to a deep reinforcement learning algorithm, coupling a task execution module and a data transmission module, and constructing a service provider electricity cost minimization model that meets the task service quality constraints, including: forming a new data center cluster state according to the arrival of tasks and the upload of data; the task execution module and the data transmission module generate actions according to their respective strategies; the task execution module and the data transmission module enter the next state according to the strategy-generated actions and obtain corresponding rewards; the task execution module and the data transmission module respectively obtain a piece of trajectory data in the process of interacting with the environment; for the task execution module and the data transmission module, an estimate of the advantage function is calculated based on the current value function; the strategies of the task execution module and the data transmission module are updated; and the value functions of the task execution module and the data transmission module are optimized.

[0020] The task execution module and the data transmission module calculate the estimate of the advantage function based on the current value function, including: using a generalized advantage estimation method to calculate the estimate of the advantage function;

[0021] The updating of the strategies of the task execution module and the data transmission module includes: updating their strategies by maximizing their PPO-truncated objectives;

[0022] The optimization of the value functions of the task execution module and the data transmission module includes: optimizing the value function of each intelligent agent through a mean square error loss function.

[0023] The multi-data center online task scheduling system considering data availability is implemented based on the multi-data center online task scheduling method considering data availability.

[0024] An electronic device comprises a memory, a processor and a computer program running on the processor, wherein the steps of the method are implemented when the processor executes the computer program.

[0025] Compared with the prior art, the present invention has the following advantages and beneficial effects:

[0026] The present invention proposes a multi-data center online task scheduling technology based on a multi-agent reinforcement learning model that considers data availability, and considers executing tasks when data is available; the present invention models data transmission agents and task scheduling agents; the present invention can dynamically change the CPU resource utilization rate of each task at different times and the bandwidth occupancy of each data transfer at each time, and can realize dynamic changes in CPU usage and bandwidth occupancy, thereby improving the flexibility of the scheduling process; the present invention separates task scheduling and data scheduling into two independent agents, and the allocation of tasks can guide the flow direction of data, and the flow of data can also be guided by task allocation, so that the two can influence each other in the learning process and cooperate with each other to obtain the optimal solution; the present invention can realize real-time updating and online scheduling of tasks and data. The present invention uses reinforcement learning and a "model-free" approach to continuously interact with the environment through an intelligent agent to learn a multi-data center task scheduling strategy and a data transmission strategy between multiple data centers. The strategy can observe the current and previous tasks, optimize scheduling based on the current situation, and schedule unfinished tasks and newly generated data online. It processes large amounts of data through deep neural network learning, adapts to large amounts of data scale and multiple time steps, and effectively solves problems on a large scale, thus solving the problem that the previous use of operations research models to seek the optimal multi-data center task scheduling strategy requires assuming that the data center subject knows the future task arrival and data upload status, and cannot perform real-time online scheduling. In addition, as the task scale expands and the time step increases, the computational time increases dramatically, making it impossible to meet the needs of real large-scale scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] In order to more clearly illustrate the technical solutions of the exemplary embodiments of the present invention, the following briefly introduces the drawings required for use in the embodiments. It should be understood that the following drawings only illustrate certain embodiments of the present invention and should not be regarded as limiting the scope. For those of ordinary skill in the art, other relevant drawings can be obtained based on these drawings without creative work. In the drawings:

[0028] Figure 1 The present invention is a flowchart of a multi-data center online task scheduling method considering data availability according to an embodiment of the present invention. DETAILED DESCRIPTION

[0029] In order to make the objectives, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with embodiments and drawings. The exemplary embodiments of the present invention and their description are only used to explain the present invention and are not intended to limit the present invention.

[0030] Example

[0031] like Figure 1 As shown, the embodiment of the present application provides a multi-data center online task scheduling method considering data availability, including: the task execution module stores multiple tasks received at different times into a processing task queue, and selects part of the tasks in the processing task queue at each time and allocates them to the corresponding data center for execution; the data transmission module manages the data of the data center cluster, transmits part of the data in the data center at each time, and updates the data stored in multiple data centers; through a deep reinforcement learning algorithm, the task execution module and the data transmission module are coupled to construct a service provider electricity cost minimization model that meets the task service quality constraints. Since the data transmission module uniformly updates and manages the data of the data center cluster, the task requires the corresponding data in the data center to be executed, so the execution of the task is constrained by the data transmission module.

[0032] The task execution module selects part of the tasks in the processing task queue at each moment and assigns them to the corresponding data center for execution, including: according to the current moment, the electricity price and remaining available CPU resources of each data center, the data required to execute the current task, whether the task is assigned, the assigned data center, the executable status of the task, the remaining CPU resources required to complete the task, the latest feasible completion time of the task, the status of all currently unfinished tasks, and the migration and storage status of all data; and assigns each task in the processing task queue to the corresponding data center for execution.

[0033] The multi-data center online task scheduling method considering data availability also includes:

[0034] Assume that there is a data centers, each data center is numbered as ;

[0035] The state space of the task execution module is:

[0036] ;

[0037] represents the current time step; Indicates data center electricity price; express CPU resource usage of each data center; Indicates the data number required to execute the task job. Indicates the distribution of tasks, there are +1 possibility, if =0 means the task is not assigned, = Indicates allocation to data center middle; Indicates the executable status of the task. If =0 means the data center where the task is located does not have the data required by the task , the task is not executable; Indicates the remaining amount of CPU resources required to complete the task. Indicates the execution deadline of the task. Represents the collection embedding of all task information at the current moment, It represents the set embedding of all data information at the current moment. Among them, the executable status of ready is jointly determined by the data center cluster managed by the data transmission module, which reflects the mutual coupling between the data transmission module and the task execution module.

[0038] The data transmission module selects the data required for task execution and transmits it to the data center, including: selecting the data departure point and the data reception point; selecting the number of bandwidth resources allocated to the data at this moment; assigning a transmission priority to the transmission of each data, and allocating bandwidth resources in descending order according to the transmission priority until all data are allocated, or there are insufficient bandwidth resources or switch resources in the data center network.

[0039] The multi-data center online task scheduling method considering data availability also includes: assuming that there are data centers, each data center is numbered as ;

[0040] The state space of the data transmission module is represented as:

[0041]

[0042] represents the current time step; Indicates data center The electricity price, Indicates the index number of the current data. Indicates the total amount of data contained in the data; Indicates the amount of data remaining to be transmitted at this time; Indicates the bandwidth resource usage in the network topology. * The matrix of Indicates the switch power resource usage of each data center; Indicates the existence of data in each data center, which is a (1, ), 1 means the data exists, 0 means the data does not exist; It is a record of the data transmission direction at the current moment. , j), used to record the current moment data from the data center Transmitted to data center j; j ; Represents the collection embedding of all task information at the current moment, Represents the collection embedding of all data information at the current moment.

[0043] The reward function of the deep reinforcement learning algorithm includes the negative computational cost of task processing in the data center, the negative communication cost of data transfer in the data center, the negative deadline violation penalty and the positive reward for completing the task on time.

[0044] The invention relates to a deep reinforcement learning algorithm, coupling a task execution module and a data transmission module, and constructing a service provider electricity cost minimization model that meets the task service quality constraints, including: forming a new data center cluster state according to the arrival of tasks and the upload of data; the task execution module and the data transmission module generate actions according to their respective strategies; the task execution module and the data transmission module enter the next state according to the strategy-generated actions and obtain corresponding rewards; the task execution module and the data transmission module respectively obtain a piece of trajectory data in the process of interacting with the environment; for the task execution module and the data transmission module, an estimate of the advantage function is calculated based on the current value function; the strategies of the task execution module and the data transmission module are updated; and the value functions of the task execution module and the data transmission module are optimized.

[0045] The task execution module and the data transmission module calculate the estimate of the advantage function based on the current value function, including: using a generalized advantage estimation method to calculate the estimate of the advantage function;

[0046] The updating of the strategies of the task execution module and the data transmission module includes: updating their strategies by maximizing their PPO-truncated objectives;

[0047] The optimization of the value functions of the task execution module and the data transmission module includes: optimizing the value function of each intelligent agent through a mean square error loss function.

[0048] The multiple data centers considered in the present invention belong to the same computing service provider, so the electricity prices, computing resource usage, and bandwidth occupancy in the transmission network of multiple data centers are all known. All tasks arriving at the data center are stored in a unified task queue waiting for scheduling, and the data is uploaded by different users and will be stored in a certain data center after being uploaded. Tasks must exist in the same data center as the data to be processed. Therefore, the present invention constructs two agents, one of which is responsible for selecting the data to be transmitted and the destination of the data transmission, and optimizing and updating the data stored in multiple data centers; the other is responsible for processing tasks waiting for scheduling in the task queue, and selecting tasks to be assigned to data centers suitable for its execution, thereby achieving the minimization of electricity bills paid by the computing service provider while meeting the task service quality QoS constraints. First of all, the present invention proposes to establish the decision-making process of the above-mentioned agent as a deep reinforcement learning model.

[0049] Reinforcement learning studies how an agent can maximize the reward it receives in a complex and uncertain environment. The agent is constantly interacting with the environment. The agent obtains a state in the environment and uses this state to output an action. The environment executes the action taken by the agent and outputs the next state and the reward for the current action.

[0050] To build a reinforcement learning model, it is necessary to define the state, action and reward function in the Markov decision process (MDP). After the MDP model is built, the network structure is characterized by a deep neural network. The MDP model constructed by the present invention is as follows:

[0051] ;

[0052] The state space of the task execution agent module is:

[0053] in, Indicates the distribution of tasks, there are +1 possibility, if =0 means the task is not assigned, =i means assigned to data center middle; Indicates the executable status of the task. If =0 means the data center where the task is located does not have the data required by the job , the task is not executable; Indicates the deadline of the job. Represents the embedding of all job information at the current moment. Represents the set embedding of all data information at the current moment.

[0054] in, , It allows the agent to observe the current moment The electricity prices of each data center and the remaining available CPU resources of each data center are as follows: It allows the agent to observe the data number needed to perform the current task. It allows the agent to observe whether the current task has been assigned and to which data center it has been assigned. The agent observes the environment, , If the data center is assigned to a data center with the number , it means that the task can be executed at this time; The agent can observe the remaining CPU resources required to complete the task. , , , to judge the urgency of the task at this time, so as to make a better decision on the number of CPU resources to be allocated to it. If the task is urgent, relatively more CPU resources can be allocated to speed up the execution of the task. Based on the above information, the agent can already observe the execution and allocation of each task, but in order to help the agent make better decisions, it should also be able to observe the status of all tasks that have arrived but not yet completed, and the migration and storage of all data in the data center cluster. Therefore, all the current unfinished task information and all the current data information are embedded to form a fixed-length feature, which is added to the state space of each task, so that the agent can observe both the execution of the current task and the overall situation.

[0055] The action of the task execution module is divided into two parts. The first part is to assign the task to a certain data center, and the second part is to assign CPU resources to the task. For the first part, the output of the neural network is n-dimensional. After softmaxing the output of the neural network, a data center is selected based on probability and the task is assigned to this data center. For the second part, the neural network outputs a 1-dimensional value, which indicates the number of CPU resources assigned to the task at that moment. If the task is not in an executable state, the second part of the output action is an invalid action, and this part of the result is masked; if the task is in a state where it has been assigned to a data center, the first part of the output action is an invalid action, and this part of the output is masked. For all actions in the second part that are not masked, the resources allocated to the job in each data center are softmaxed according to the data center to which the job is assigned, so that the total number of CPU resources allocated to the task does not exceed the resource upper limit of each data center, expressed as:

[0056] .

[0057] For the data transmission agent, its state space is:

[0058]

[0059] in, Indicates the bandwidth resource usage in the network topology. * The matrix is ​​flattened into a (1, * ) tensor, Indicates the switch power resource usage of each data center, which is a (1, ) tensor; Indicates the existence of data in each data center, which is a (1, ) vector, 1 indicates that the data exists, 0 indicates that the data does not exist, and DT is a record of the data transmission direction at that moment ( , j), used to record the current moment data from the data center Transmitted to data center j; Represents the embedding of all task information at the current moment. Represents the set embedding of all data information at the current moment.

[0060] , , , It allows the agent to observe the current moment The following table contains the electricity price of each data center, the bandwidth resource occupancy in the data center cluster topology network, the resource occupancy of the switches in each data center, and the existence of data in the current data center cluster. , It allows the agent to correspond to a certain number of data and the total amount of data. and It can jointly reflect the direction and progress of the data transmission. Based on the above information, the intelligent agent can already observe the migration of each type of data, but in order to help the intelligent agent make better decisions, it should also be able to observe the distribution and storage of all data in the data center cluster and the status of all tasks that have arrived but have not yet been completed. Therefore, we also embed all the current unfinished task information and all the current data information to form a fixed-length feature, which is added to the state space of each task, so that the intelligent agent can observe the migration and storage of all current data, as well as the execution of tasks, thereby helping the intelligent agent to make better decisions.

[0061] The operation of the data transmission module is divided into four parts. The first part is to select a data departure point from the data center where the data exists. The second part is to select a data receiving point in the data center where the data does not exist. The third part is bandwidth allocation, which selects the number of bandwidth resources allocated to the data at this moment. The fourth part is transmission priority, which assigns a transmission priority to each data transmission and allocates bandwidth resources in descending order according to the priority until all data are allocated at this moment or there are no sufficient bandwidth resources or switch resources in the data center network, which is expressed as:

[0062] .

[0063] The two intelligent agents, the data transmission module and the task execution module, share the same reward function, which consists of four items. The first item is the negative computational cost of the data center for task processing, which can be obtained by multiplying the energy consumption generated by the calculation amount by the time-varying electricity price; the second item is the negative communication cost of data transfer in the data center, which can be obtained by multiplying the energy consumption generated by the switch by the time-varying electricity price; the third item is the negative deadline violation penalty; the fourth item is the positive reward for completing the task before the latest feasible completion time. According to the user's service quality requirements, each task has its own deadline completion time. If the calculation cannot be completed before this time, it will be punished.

[0064] The multi-agent independent proximal strategy optimization algorithm provided by the present invention is as follows:

[0065] Input: For each agent, initialize three network parameters, namely the old policy network , policy network and value network Initialize the learning rate and optimizer of each network, and initialize the hyperparameters, including the clip parameter epsilon.

[0066] For episode

[0067] For

[0068] 1. A new data center cluster status is formed based on the arrival of tasks and the upload of data.

[0069] 2. Each agent generates actions based on its own strategy .

[0070] 3. The agent performs actions based on Get corresponding rewards .

[0071] Each agent obtains its own trajectory data during the interaction with the environment.

[0072] For each agent, the advantage function estimate is calculated based on the current value function using the generalized advantage estimation method

[0073] For each agent, update its policy by maximizing its PPO-truncated objective

[0074] For each agent, its value function is optimized by the mean square error loss function

[0075] End for

[0076] In summary, the embodiments of the present application provide a multi-data center online task scheduling method considering data availability, and a system implemented by the method, which has the following advantages:

[0077] 1. It optimizes the energy consumption of computing resources and communication equipment at the same time. It can dynamically change the amount of CPU resources used by each task at different times, and it can also dynamically change the bandwidth occupied at each moment of the data transmission process, thereby dynamically changing the energy consumption of the switch at each moment;

[0078] 2. The present invention considers the multi-data center task scheduling process as two independent intelligent entities, task scheduling and data transmission, so that the two processes have strong initiative and coordination. It is not that the task guides the placement of data, nor is it a simple data locality (data locality refers to a scheduling method that simply places tasks in the data center that contains the required calculations). Instead, tasks and data influence and cooperate with each other to learn an optimal situation.

[0079] 3. While performing task scheduling, the learning method can be used to select the appropriate bandwidth to transmit the appropriate data, optimize the data configuration between multiple data centers, and alleviate the congestion of data traffic;

[0080] 4. Different from the global optimization of all known tasks and data information in the previous operations research model, this paper proposes a multi-data center online task scheduling technology based on multi-agent reinforcement learning that takes into account data availability. This model will learn a strategy through a large amount of training, which can comprehensively consider the tasks and data generated online, and make real-time decisions on the data transfer and task calculation that should be performed at the moment;

[0081] 5. Traditional operations research models are severely limited in scale and are difficult to handle large-scale data and tasks. The model proposed in this invention uses deep neural networks and can be used in large-scale scenarios to effectively process scenarios with large amounts of data and tasks.

[0082] The specific implementation methods described above further illustrate the objectives, technical solutions and beneficial effects of the present invention in detail. It should be understood that the above description is only a specific implementation method of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A multi-data center online task scheduling method considering data availability, characterized in that: include: The task execution module stores multiple tasks received at different times into a processing task queue, and selects some tasks in the processing task queue at each time and assigns them to the corresponding data center for execution; The data transmission module manages the data of the data center cluster, transmits part of the data in the data center at each moment, and updates the data stored in multiple data centers; Through deep reinforcement learning algorithm, coupling task execution module and data transmission module, a service provider electricity cost minimization model that meets the task service quality constraints is constructed, including: A new data center cluster status is formed based on the arrival of tasks and the upload of data; The task execution module and data transmission module generate actions according to their respective strategies; The task execution module and data transmission module generate actions according to the strategy to enter the next state and obtain corresponding rewards; The task execution module and the data transmission module each obtain a piece of trajectory data during the interaction with the environment; For the task execution module and the data transmission module, an estimate of the advantage function is calculated based on the current value function, including: using a generalized advantage estimation method to calculate an estimate of the advantage function; Updating the strategies of the task execution module and the data transmission module, including: updating their strategies by maximizing their PPO-truncated objectives; Optimizing the value functions of the task execution module and the data transmission module, including: optimizing the value function of each agent through the mean square error loss function; The reward function of the deep reinforcement learning algorithm includes the negative computational cost of task processing in the data center, the negative communication cost of data transfer in the data center, the negative deadline violation penalty and the positive reward for completing the task on time.

2. The multi-data center online task scheduling method considering data availability according to claim 1 is characterized in that: The task execution module selects some tasks in the processing task queue at each moment and allocates them to the corresponding data center for execution, including: According to the current moment, the electricity price and remaining available CPU resources of each data center, the data needed to execute the current task, whether the task has been assigned, the assigned data center, the executable status of the task, the remaining CPU resources required to complete the task, the latest feasible completion time of the task, the status of all currently unfinished tasks, and the migration and storage status of all data; each task in the processing task queue is assigned to the corresponding data center for execution.

3. The multi-data center online task scheduling method considering data availability according to claim 1 is characterized in that: Also includes: Assume that there is a data centers, each data center is numbered as ; The state space of the task execution module is: ； represents the current time step; Indicates data center electricity price; Indicates CPU resource usage of each data center; Indicates the index number of the current data. Indicates the distribution of tasks, there are +1 possibility, if =0 means the task is not assigned, = Indicates allocation to data center middle; Indicates the executable status of the task. If =0 means the data center where the task is located does not have the data required by the task , the task is not executable; Indicates the remaining amount of CPU resources required to complete the task. Indicates the execution deadline of the task. Represents the collection embedding of all task information at the current moment, Represents the collection embedding of all data information at the current moment.

4. The multi-data center online task scheduling method considering data availability according to claim 1 is characterized in that: The data transmission module selects the data required for task execution and transmits it to the data center, including: selecting the data departure point and the data reception point; selecting the number of bandwidth resources allocated to the data at this moment; assigning a transmission priority to the transmission of each data, and allocating bandwidth resources in descending order according to the transmission priority until all data are allocated, or there are insufficient bandwidth resources or switch resources in the data center network.

5. The multi-data center online task scheduling method considering data availability according to claim 1 is characterized in that: Also includes: The state space of the data transmission module is represented as: represents the current time step; Indicates data center The electricity price, Indicates the index number of the current data. Indicates the total amount of data contained in the data; Indicates the amount of data remaining to be transmitted at this time; Indicates the bandwidth resource usage in the network topology. * The matrix of Indicates the switch power resource usage of each data center; Indicates the existence of data in each data center, which is a (1, ), 1 means the data exists, 0 means the data does not exist; It is a record of the data transmission direction at the current moment. , j), used to record the current moment data from the data center Transmitted to data center j; j ; Represents the collection embedding of all task information at the current moment, Represents the collection embedding of all data information at the current moment.

6. A system for online task scheduling in multiple data centers considering data availability, characterized in that: The multi-data center online task scheduling method considering data availability as described in claim 1 is implemented.

7. An electronic device comprising a memory, a processor, and a computer program running on the processor, characterized in that: When the processor executes the computer program, the steps of the multi-data center online task scheduling method considering data availability are implemented as described in any one of claims 1 to 5.

Images