Data center micro-grid computing cooperation double-layer deep reinforcement learning scheduling method

Abstract

The invention discloses a data center micro-grid computing cooperation double-layer deep reinforcement learning scheduling method, and belongs to the technical field of power systems, and the method comprises the steps: firstly, providing a double-layer multi-agent deep reinforcement learning framework for a high renewable energy penetration scene of a geographically distributed data center micro-grid; a double-layer multi-agent deep reinforcement learning framework containing a global layer and a local layer is constructed, the global layer carries out distribution calculation tasks through centralized deep reinforcement learning training, and the local layer optimizes internal energy scheduling through distributed deep reinforcement learning; and cross-layer interaction and rapid convergence are realized in combination with a layered multi-agent double-delay depth deterministic strategy gradient algorithm. Through cooperation of space-time task adjustment and energy storage, the cost of non-renewable energy sources is reduced by 15%-20%, the convergence speed is increased by 5 times compared with the prior art, dynamic data backup is supported, and the reliability and economical efficiency of the system are remarkably improved while the service quality is kept.

Description

Technical Field

[0001] The present invention belongs to the field of power systems, and particularly relates to a power-computing collaborative double-layer deep reinforcement learning scheduling method for a data center microgrid. Background Art

[0002] The complexity of geographically distributed data center microgrids, combined with the complex nature of computing tasks and the high uncertainty associated with renewable energy generation, requires advanced real-time and effective power-computing scheduling methods. While maintaining the quality of service, ensuring the security, economy, and stable operation of the system has become a key challenge in the power and computing management of geographically distributed data center microgrids. Therefore, it is of great significance to develop a power-computing collaborative scheduling method for the time and space control ability of data center microgrids. Summary of the Invention

[0003] To address the above problems, the present invention provides a power-computing collaborative double-layer deep reinforcement learning scheduling method for a data center microgrid, which is applicable to real-time scheduling and cost optimization in high renewable energy penetration scenarios, realizes the power-computing collaborative scheduling of geographically distributed data center microgrids, and ensures the security, economy, and stable operation of the system while maintaining the quality of service.

[0004] The technical solution adopted by the present invention is as follows:

[0005] A power-computing collaborative double-layer deep reinforcement learning scheduling method for a data center microgrid, comprising the following steps:

[0006] S1: Construct a power-computing collaborative scheduling model, task flexibility requirements, and integrated data protection mechanism for a geographically distributed data center microgrid.

[0007] Among them, the power-computing collaborative scheduling model includes a data center energy consumption model, a energy storage system model, a computing task model, and a data center backup data energy consumption model. Among them, the constraints of the energy storage system model include upper and lower limits of charge and discharge power and upper and lower limits of its capacity.

[0008] The task flexibility requirement is that the real-time processing of computing tasks by each data center cannot exceed the maximum computing task volume that all its servers can handle.

[0009] The data protection mechanism is that the data center microgrid will locally save data to prevent data transmission errors.

[0010] S2: Construct a double-layer multi-agent deep reinforcement learning framework containing a global layer and a local layer.

[0011] The global layer adopts a centralized deep reinforcement learning algorithm to allocate computing tasks based on renewable energy prediction and real-time task requirements. The centralized deep reinforcement learning algorithm is that the global layer uses an agent to allocate the computing tasks arriving at the global layer in real time according to the new energy prediction output, the number of active servers, and the real-time remaining computing task volume information of the geographically distributed data center microgrids in the next two hours;

[0012] The local layer adopts a decentralized deep reinforcement learning algorithm to optimize the internal energy management and task scheduling of each data center microgrid while meeting the service quality requirements. The decentralized reinforcement learning algorithm is that each geographically distributed data center microgrid uses an agent to schedule the computing tasks arriving in real time according to the new energy output and the number of active servers in the next two hours;

[0013] S3: Interact between the global layer and the local layer. The global layer allocates computing tasks to each local layer, and the local layer feeds back the uncompleted computing tasks to the global layer to affect the global layer's allocation;

[0014] S4: Guide the global layer to allocate computing tasks to the geographically distributed data center microgrids through a reward function, ensure that the local layer focuses on maximizing the use of local renewable energy within a day, and the local layer minimizes the non-renewable energy cost, so as to achieve the collaborative optimal scheduling of power and computing power of the geographically distributed data center microgrids.

[0015] Furthermore, in step S1, the data center energy consumption model is shown in formulas (1)-(6), the energy storage system model is shown in formulas (7)-(11), and the computing task model is shown in formulas (12)-(20), including the batch processing computing task model and the interactive computing task model. Among them, formulas (12)-(16) are the batch processing computing task model, and among them, formulas (15) and (16) are the constraints of the batch processing computing task model, indicating that the batch processing computing task needs to be completed within the maximum tolerance time and meet the non-negative constraint at any time; formulas (17)-(21) are the interactive computing task model, and among them, formulas (18) and (19) are the constraints of the interactive processing computing task model, indicating that the interactive computing task needs to be completed within the maximum tolerance time and meet the non-negative constraint at any time; the data center backup data energy consumption model is shown in formula (21),

[0016] P i,DC (t) = x i (t)[P i,idle +(PUE i -1)P i,peak +x i (t)(P i,peak -P i,idle )γi (t) + ξ i (1)

[0017] ξ i ≤ P i,DC (t) ≤ M i ·PUE i ·P i,peak + ξ i (2)

[0018]

[0019] 0 ≤ x i (t) ≤ M i (6)

[0020] B i (t + 1) = B i (t) + (u i,ch P i,ch (t)η i,ch - u i,dis P i,dis (t) / η i,dis )Δt (7)

[0021] u i,ch + u i,dis =1, u i,ch ≥ 0, u i,dis ≥ 0 (8)

[0022]

[0023] 0 ≤ P i,dis (t) ≤ p i,dis (10)

[0024]

[0025] Where: P i,DC (t) is the energy consumption of data center i at time t; x i (t) is the number of active servers in data center i at time t; PUE i is the power usage effectiveness of data center i; P i,idle , P i,peak respectively represent the energy consumption of the servers in data center i when idle and when fully loaded; γ i (t) represents the average utilization rate of the servers in data center i at time t; ξ i represents the energy consumption of the infrastructure of data center i; M i is the maximum number of servers in data center i; ω i (t) is the amount of batch processing computing tasks that data center i needs to process at time t; For the data center i to process at time t the batch computing tasks arriving at time t; Let the maximum tolerance time of the batch computing tasks arriving at time t; B i (t) represents the capacity of the energy storage system included in the data center microgrid i at time t; and B i represent the upper and lower limits constraints of the capacity of the energy storage system included in the data center microgrid i; P i,ch (t) and P i,dis (t) represent the charging and discharging power of the energy storage system included in the data center microgrid i at time t; and p i,dis represent the upper and lower limits constraints of the charging and discharging power of the energy storage system included in the data center microgrid i; η i,ch and η i,dis represent the charging and discharging efficiency of the energy storage system included in the data center microgrid i; u i,ch and u i,dis represent the charging and discharging binary variables of the energy storage system included in the data center microgrid i; Δt represents the time interval; Ω is the set of batch computing tasks; J is the type of batch computing tasks; τ j is the arrival time set of the j - type batch computing tasks; the batch computing tasks arriving at time t processed by the data center i jm ; S is the set of data center microgrids; be the initial number of batch computing tasks arriving at the data center i at time t 0 ; be the initial number of interactive computing tasks arriving at the data center i at time t; λ i (t) is the amount of interactive computing tasks processed by the data center i at time t; be the adjustable amount of the number of batch computing tasks arriving at time t processed by the data center i ; Δλ i (t) is the adjustable amount of interactive computing tasks processed by the data center i at time t; be the energy consumption of backing up data by the data center i at time t; be the amount of data backed up by the data center i; α i be the energy consumption coefficient of backing up data by the data center i; R(ω i (t u0 ∣t u0-1 )) be the remaining batch computing tasks of the data center microgrid i arriving at time t at time t u0-1 ; u0

[0026] ​Further, in step S2, a two-layer multi-agent deep reinforcement learning framework including a global layer and a local layer is constructed, as shown in formulas (22)-(25).

[0027]

[0028] In the formula: s global represents the global layer state space; i = 1, …, S represents the state space of data center microgrid i; P i,green (t), i = 1, …, S is the renewable energy output of data center microgrid i at time t;

[0029] R(ω i (t u0 ∣t u0-1 )), i = 1, …, S is the remaining batch processing computing tasks arriving at data center microgrid i at time t u0-1 at time t u0 ; u global represents the global layer action space; i = 1, ..., S represents the action space of data center microgrid i; i = 1, ..., S is the batch processing computing tasks and interactive computing tasks assigned by the global layer to data center microgrid i at time t u0 ; i = 1, ..., S is the output of the N i th non-renewable energy unit of data center microgrid i.

[0030] Further, in step S3, the global layer distributes computing tasks to each local layer, and the local layer feeds back the unfinished computing tasks to the global layer to affect the global layer distribution, as shown in formulas (26)-(28).

[0031]

[0032] R(ω i (t 1 ∣t 0 )) = 0 (28)

[0033] In the formula: T l represents the scheduling time step of the local layer in one day; is the indicator function.

[0034] Further, in step S4, the global layer reward function and the local layer reward function are as shown in formulas (29)-(37).

[0035]

[0036] P i,green(t) = P i,WT (t) + P i,PV (t) (31)

[0037]

[0038]

[0039] Where: r global and i = 1,..., S respectively represent the reward functions of the global layer and the local layer; P i,green (t) represents the renewable energy output at the geographical location of the data center microgrid i at time t; P i,WT (t) and P i,PV (t) respectively represent the wind power output and the photovoltaic power output at the geographical location of the data center microgrid i at time t; and P i,WT respectively represent the upper and lower limit constraints of the wind power output of the data center microgrid i at the geographical location; and P i,PV respectively represent the upper and lower limit constraints of the photovoltaic power output of the data center microgrid i at the geographical location; N i represents the non-renewable energy units included in the data center microgrid i; represents the output of the l i th non-renewable energy unit of the data center microgrid i at time t; and respectively represent the upper and lower limit constraints of the output of the l i th non-renewable energy unit of the data center microgrid i; and respectively represent the increased or decreased output of the fuel unit per unit time;

[0040] Among them, formula (37) is the ramp rate constraint of the l i th non-renewable energy unit of the data center microgrid i.

[0041] Another object of the present invention is achieved by a data center microgrid computer-aided collaborative two-layer deep reinforcement learning scheduling system, including: a memory, a processor, and a computer program stored on the memory and executable on the processor. When the computer program is executed by the processor, it implements a data center microgrid computer-aided collaborative two-layer deep reinforcement learning scheduling method as described above.

[0042] Advantages and beneficial effects of the present invention: The present invention realizes the coordinated scheduling of power and computing power in a geographically distributed data center microgrid, improves the security, reliability, economy and stability of the system while maintaining the quality of service. The present invention reduces the cost of non-renewable energy by 15%-20% through the coordination of spatio-temporal task regulation and energy storage, the convergence speed is increased by 5 times compared with the existing technology, and dynamic data backup is supported. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] Figure 1 is a flowchart of the present invention;

[0044] Figure 2 is a comparison diagram of convergence situations; among them, (a) is a training convergence diagram using the existing scheduling method, and (b) is a training convergence diagram using the scheduling method of the present invention;

[0045] Figure 3 is a cost comparison diagram for power and computing power scheduling of a data center microgrid. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0046] In order to make the objectives, technical solutions and advantages of the present invention clearer, the present invention will be further described in detail below with reference to embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0047] Embodiment 1:

[0048] As Figure 1 shown, a double-layer deep reinforcement learning scheduling method for power and computing coordination in a data center microgrid includes the following steps:

[0049] S1: Construct a power and computing coordination scheduling model, task flexibility requirements and integrated data protection mechanism for a geographically distributed data center microgrid,

[0050] Among them, the power and computing coordination scheduling model includes a data center energy consumption model, an energy storage system model, a computing task model, and a data center backup data energy consumption model. Among them, the constraints of the energy storage system model include upper and lower limits of charge and discharge power and upper and lower limits of its capacity;

[0051] The task flexibility requirement is that the real-time processing of computing tasks by each data center cannot exceed the maximum computing task volume that all its servers can handle;

[0052] The data protection mechanism is that the data center microgrid will locally save data to prevent data transmission errors;

[0053] The data center energy consumption model is shown in formulas (1)-(6), the energy storage system model is shown in formulas (7)-(11), and the computing task model is shown in formulas (12)-(20), including a batch processing computing task model and an interactive computing task model. Among them, formulas (12)-(16) are the batch processing computing task model, and among them, formulas (15) and (16) are the constraints of the batch processing computing task model, indicating that the batch processing computing task needs to be completed within the maximum tolerance time and satisfy the non-negativity constraint at any time; formulas (17)-(21) are the interactive computing task model, and among them, formulas (18) and (19) are the constraints of the interactive processing computing task model, indicating that the interactive computing task needs to be completed within the maximum tolerance time and satisfy the non-negativity constraint at any time; the data center backup data energy consumption model is shown in formula (21).

[0054] P i,DC (t) = x i (t)[P i,idle +(PUE i -1)P i,peak +x i (t)(P i,peak -P i,idle )γ i (t)+ξ i (1)

[0055] ξ i ≤P i,DC (t)≤M i ·PUE i ·P i,peak +ξ i (2)

[0056]

[0057] 0≤x i (t)≤M i (6)

[0058] B i (t + 1) = B i (t)+(u i,ch P i,ch (t)η i,ch -u i,dis P i,dis (t) / η i,dis )Δt (7)

[0059] u i,ch +u i,dis =1,u i,ch ≥0,u i,dis ≥0 (8)

[0060]

[0061] 0 ≤ P i,dis (t) ≤ p i,dis (10)

[0062]

[0063] Where: P i,DC (t) is the energy consumption of data center i at time t; x i (t) is the number of active servers in data center i at time t; PUE i is the power usage effectiveness of data center i; P i,idle 、P i,peak respectively represent the energy consumption of the servers in data center i when idle and when fully loaded; γ i (t) represents the average utilization rate of the servers in data center i at time t; ξ i represents the energy consumption of the infrastructure of data center i; M i is the maximum number of servers in data center i; ω i (t) is the amount of batch processing computing tasks to be processed by data center i at time t; is the batch processing computing task arrived at time by data center i; is the maximum tolerance time of the batch processing computing task arrived at time i (t) represents the capacity of the energy storage system included in data center microgrid i at time t; and B i represent the upper and lower limit constraints of the capacity of the energy storage system included in data center microgrid i; P i,ch (t) and P i,dis (t) represent the charging and discharging power of the energy storage system included in data center microgrid i at time t; and p i,dis represent the upper and lower limit constraints of the charging and discharging power of the energy storage system included in data center microgrid i; η i,ch and η i,dis represent the charging and discharging efficiency of the energy storage system included in data center microgrid i; u i,ch and u i,dis represent the charging and discharging binary variables of the energy storage system included in data center microgrid i; Δt represents the time interval; Ω is the set of batch processing computing tasks; J is the type of batch processing computing tasks; τ j is the arrival time set of the j-th type of batch processing computing tasks; The batch processing computing task arrived at time by data center i; S is the set of data center microgrids; is the initial batch computing task quantity arriving at data center i at time t 0 The number of initial batch computing tasks arriving at data center i at time t; is the initial interactive computing task quantity arriving at data center i at time t; λ i The amount of interactive computing tasks processed by data center i at time t; is the amount of batch computing tasks processed by data center i at time t The adjustable amount of the number of batch computing tasks arriving at time t; Δλ i The adjustable amount of interactive computing tasks processed by data center i at time t; is the energy consumption of data center i for backing up data at time t; is the quantity of data backed up by data center i; α i is the energy consumption coefficient of data center i for backing up data; R(ω i (t u0 ∣t u0-1 )) is the remaining batch computing tasks of data center microgrid i arriving at time t u0-1 at time t u0 at time t

[0064] S2: Construct a two - layer multi - agent deep reinforcement learning framework containing a global layer and a local layer:

[0065] The global layer adopts a centralized deep reinforcement learning algorithm to allocate computing tasks based on renewable energy prediction and real - time task requirements. The centralized deep reinforcement learning algorithm is that the global layer uses one agent to allocate the computing tasks arriving at the global layer in real - time according to the new energy prediction output, the number of active servers, and the real - time remaining computing task quantity information of the geographically distributed data center microgrids in the next two hours;

[0066] The local layer adopts a decentralized deep reinforcement learning algorithm to optimize the internal energy management and task scheduling of each data center microgrid while meeting the service quality requirements. The decentralized reinforcement learning algorithm is that each geographically distributed data center microgrid uses one agent to schedule the computing tasks arriving in real - time according to the new energy output and the number of active servers in the next two hours;

[0067] Among them, constructing a two - layer multi - agent deep reinforcement learning framework containing a global layer and a local layer is shown in formulas (22) - (25),

[0068]

[0069] In the formula: s global represents the global layer state space; i = 1,..., S represents the state space of data center microgrid i; P i,green$P_{ri}(t), i = 1, \ldots, S$ is the renewable energy output of the $i$-th data center microgrid at time $t$;

[0070] $R(\omega$ i (t u0 $\mid t$ u0-1 ))), $i = 1, \ldots, S$ is the remaining batch computing tasks of the $i$-th data center microgrid at time $t$ when arriving at time $t$; u0-1 At time $t$, the remaining batch computing tasks of the $i$-th data center microgrid when arriving at time $t$; u0 $u$ global represents the action space of the global layer; $i = 1, \ldots, S$ represents the action space of the $i$-th data center microgrid; $i = 1, \ldots, S$ is the batch computing tasks and interactive computing tasks assigned to the $i$-th data center microgrid by the global layer at time $t$; u0 At time $t$, the batch computing tasks and interactive computing tasks assigned to the $i$-th data center microgrid by the global layer; $i = 1, \ldots, S$ is the output of the $N$-th i non-renewable energy unit of the $i$-th data center microgrid.

[0071] S3: Interact between the global layer and the local layer. The global layer assigns computing tasks to each local layer, and the local layer feeds back the unfinished computing tasks to the global layer to affect the global layer's assignment; among them, the global layer assigns computing tasks to each local layer, and the local layer feeds back the unfinished computing tasks to the global layer to affect the global layer's assignment, as shown in formulas (26)-(28),

[0072]

[0073] $R(\omega$ i (t 1 $\mid t$ 0 )) = 0 (28)

[0074] In the formula: $T$ l represents the scheduling time step of the local layer in a day; is the indicator function.

[0075] S4: Guide the global layer to assign computing tasks to the geographically distributed data center microgrids through the reward function, ensure that the local layer focuses on maximizing the use of local renewable energy within a day, and the local layer minimizes the non-renewable energy cost, so as to realize the coordinated optimal scheduling of power and computing power of the geographically distributed data center microgrids; among them, the global layer reward function and the local layer reward function are as shown in formulas (29)-(37),

[0076]

[0077] $P$ i,green (t) = $P$ i,WT (t) + $P$ i,PV (t) (31)

[0078]

[0079] where: r global and i = 1, ..., S respectively represent the reward functions of the global layer and the local layer; P i,green (t) represents the renewable energy output at the geographical location of the data center microgrid i at time t; P i,WT (t) and P i,PV (t) respectively represent the wind power output and the photovoltaic power output at the geographical location of the data center microgrid i at time t; and P i,WT respectively represent the upper and lower limit constraints of the wind power output of the data center microgrid i at the geographical location; and P i,PV respectively represent the upper and lower limit constraints of the photovoltaic power output of the data center microgrid i at the geographical location; N i represents the non-renewable energy units included in the data center microgrid i; represents the output of the l i th non-renewable energy unit of the data center microgrid i at time t; and respectively represent the upper and lower limit constraints of the output of the l i th non-renewable energy unit of the data center microgrid i; and respectively represent the increase or decrease in the output of the fuel unit per unit time;

[0080] where formula (37) is the ramp rate constraint of the l i th non-renewable energy unit of the data center microgrid i; Through the above steps, the coordinated optimal scheduling of power and computing power of the geographically distributed data center microgrid can be realized.

[0081] Embodiment 2

[0082] As Figure 2 shown, Global level represents the global layer, Local level represents the local layer; DC1, DC2, DC3 represent three data center microgrids. It can be seen that Figure 2 (a) At least 2000 training rounds are required to reach convergence using the existing algorithm; Figure 2 (b) For the algorithm used in the present invention, only 500 rounds are required to reach convergence. It can be seen that the method used in the present invention is 5 times more efficient than the prior art.

[0083] As Figure 3As shown in the figure, Random means that no optimization algorithm is used for the coordinated scheduling of power computing power in the geographically distributed data center microgrid. Bi-TD3 means that the existing technology is used for the coordinated scheduling algorithm of power computing power in the geographically distributed data center microgrid. Proposed means that the present invention is used for the coordinated scheduling algorithm of power computing power in the geographically distributed data center microgrid. By comparing the costs of power computing power scheduling for the data center microgrid using no optimization scheduling method, the existing method, and the method proposed by the present invention, it can be seen that the method used in the present invention significantly reduces the non-renewable energy cost by 15%-20% compared with not using the optimization scheduling method, and also has certain advantages compared with the existing method.

Claims

1. A data center microgrid computer collaborative double-layer deep reinforcement learning scheduling method, characterized in that: The following steps are involved: S1: Constructing a power computing power collaborative dispatch model, task flexibility requirements, and integrated data protection mechanism for geographically distributed data center microgrids, The electric power computing power collaborative scheduling model includes a data center energy consumption model, an energy storage system model, a computing task model, and a data center backup data energy consumption model, wherein the constraints of the energy storage system model include upper and lower limit constraints on charging and discharging power and upper and lower limit constraints on its capacity; The task flexibility requires that the real-time computing tasks processed by each data center cannot exceed the maximum computing task volume that all its servers can handle; The data protection mechanism described is that the data center microgrid will save the data locally to prevent data transmission errors; S2: Construct a two-layer multi-agent deep reinforcement learning framework with global and local layers: The global layer adopts a centralized deep reinforcement learning algorithm to allocate computing tasks based on renewable energy prediction and real-time task requirements. The centralized deep reinforcement learning algorithm is a deep reinforcement learning algorithm that uses an intelligent agent at the global layer to allocate computing tasks that arrive at the global layer in real time based on the predicted output of new energy sources, the number of active servers, and the real-time remaining computing task volume of the geographically distributed data center microgrid in the next two hours; The local layer adopts a distributed deep reinforcement learning algorithm to optimize the internal energy management and task scheduling of each data center microgrid while meeting the service quality requirements. The distributed reinforcement learning algorithm is an algorithm in which each geographically distributed data center microgrid uses an intelligent agent to schedule the computing tasks arriving in real time according to the new energy output in the next two hours and the number of active servers; S3: The global layer interacts with the local layer. The global layer assigns computing tasks to each local layer. The local layer feeds back the unfinished computing tasks to the global layer to influence the global layer allocation. S4: Through the reward function, the global layer is guided to allocate computing tasks to the geographically distributed data center microgrids, ensuring that the local layer focuses on maximizing the use of local renewable energy within a day and the local layer minimizes the cost of non-renewable energy, thereby achieving coordinated optimization scheduling of power computing power in geographically distributed data center microgrids.

2. According to claim 1, a data center microgrid computer collaborative double-layer deep reinforcement learning scheduling method is characterized in that: In step S1, the data center energy consumption model is as shown in formulas (1)-(6), the energy storage system model is as shown in formulas (7)-(11), and the computing task model is as shown in formulas (12)-(20), including a batch computing task model and an interactive computing task model, wherein formulas (12)-(16) are batch computing task models, wherein formulas (15) and (16) are constraints of the batch computing task model, indicating that the batch computing task must be completed within the maximum tolerance time and satisfy the non-negative constraint at any time; formulas (17)-(21) are interactive computing task models, wherein formulas (18) and (19) are constraints of the interactive computing task model, indicating that the interactive computing task must be completed within the maximum tolerance time and satisfy the non-negative constraint at any time; the data center backup data energy consumption model is as shown in formula (21), P i,DC (t)=x i (t)[P i,idle +(PUE i -1)P i,peak ]+x i (t)(P i,peak -P i,idle )γ i (t)+ξ i (1) ξ i ≤P i,DC (t)≤M i ·PUE i ·P i,peak +ξ i (2) 0≤x i (t)≤M i (6) B i (t+1)=B i (t)+(u i,ch P i,ch (t)η i,ch -u i,dis P i,dis (t) / η i,dis )Δt (7) in i,ch +in i,dis =1,in i,ch ≥0,u i,dis ≥0 (8) 0≤P i,dis (t)≤ p i,dis (10) Where: P i,DC (t) is the energy consumption of data center i at time t; x i (t) is the number of active servers in data center i at time t; PUE i is the energy utilization efficiency of data center i; P i,idle , P i,peak They represent the energy consumption of the server in data center i when idle and when fully loaded; γ i (t) represents the average utilization of the servers in data center i at time t; ξ i represents the energy consumption of the infrastructure of data center i; M i is the maximum number of servers in data center i; ω i (t) is the amount of batch computing tasks that data center i needs to process at time t; Processing of data center i at time t Batch computing tasks that arrive at a certain time; for The maximum tolerable time of the batch computing task arriving at time B i (t) represents the capacity of the energy storage system included in the data center microgrid i at time t; as well as B i represents the upper and lower capacity constraints of the energy storage system included in the data center microgrid i; P i,ch (t) and P i,dis (t) represents the charging and discharging power of the energy storage system included in the data center microgrid i at time t; as well as p i,dis represents the upper and lower limits of the charging and discharging power of the energy storage system contained in the data center microgrid i; η i,ch and η i,dis represents the charging and discharging efficiency of the energy storage system included in the data center microgrid i; u i,ch and u i,dis represents the binary variable of charging and discharging of the energy storage system contained in the data center microgrid i; Δt represents the time interval; Ω is the batch computing task set; J is the batch computing task type; τ j is the arrival time set of the j-th type of batch computing task type; Data center i processing The batch computing tasks that arrive at the time; S is the set of data center microgrids; is the number of initial batch computing tasks arriving at data center i at time t0; is the number of initial interactive computing tasks arriving at data center i at time t; i (t) is the amount of interactive computing tasks processed by data center i at time t; Processing of data center i at time t The adjustable amount of batch computing tasks arriving at a certain time; Δλ i (t) is the adjustable amount of interactive computing tasks processed by data center i at time t; is the energy consumption of backing up data in data center i at time t; The amount of backup data for data center i; α i is the energy consumption coefficient of data center i for backing up data; R(ω i (t u0 ∣t u0-1 )) is t u0-1 Arriving at the data center microgrid i at time t u0 The remaining batch computing tasks at this moment.

3. According to claim 2, a data center microgrid computer collaborative double-layer deep reinforcement learning scheduling method is characterized in that: In step S2, a two-layer multi-agent deep reinforcement learning framework containing a global layer and a local layer is constructed, as shown in formulas (22)-(25). Where: s global represents the global layer state space; represents the state space of data center microgrid i; P i,green (t), i = 1,…, S is the renewable energy output of data center microgrid i at time t; R(ω i (t u0 ∣t u0-1 )),i=1,…,S is t u0-1 Arriving at the data center microgrid i at time t u0 The remaining batch computing tasks at the moment; u global Represents the action space of the global layer; represents the action space of data center microgrid i; For the global layer at t u0 The batch computing tasks and interactive computing tasks are allocated to the data center microgrid i at all times; is the Nth microgrid i in the data center i Output of non-renewable energy units.

4. According to claim 3, a data center microgrid computer collaborative double-layer deep reinforcement learning scheduling method is characterized in that: In step S3, the global layer assigns computing tasks to each local layer, and the local layer feeds back the unfinished computing tasks to the global layer to influence the global layer assignment, as shown in formulas (26)-(28), R(ω i (t1∣t0))=0 (28) Where: T l represents the scheduling time step of one day at the local level; is an indicative function.

5. A data center microgrid computing collaborative double-layer deep reinforcement learning scheduling method according to claim 4, characterized in that: In step S4, the global layer reward function and the local layer reward function are shown in formulas (29)-(37), P i,green (t)=P i,WT (t)+P i,PV (t) (31) Where: r global as well as Represent the reward functions of the global layer and the local layer respectively; P i,green (t) represents the renewable energy output at the geographical location of the data center microgrid i at time t; P i,WT (t) and P i,PV (t) represent the wind power output and photovoltaic power output at the geographical location of the data center microgrid i at time t, respectively; as well as P i,WT They represent the upper and lower limits of wind power output at the geographical location of data center microgrid i; as well as P i,PV They represent the upper and lower limits of photovoltaic output at the geographical location of the data center microgrid i; N i represents the non-renewable energy units included in the data center microgrid i; represents the data center microgrid i at time t i Output of non-renewable energy units; as well as They represent the data center microgrid i and l i The output upper and lower limits of non-renewable energy units are constrained; as well as They respectively represent the increase or decrease in output of the fuel unit per unit time; Wherein, formula (37) is the lth microgrid of data center i i The non-renewable energy units have ramping constraints.

6. A data center microgrid computer collaborative double-layer deep reinforcement learning scheduling system, comprising: A memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: when the computer program is executed by the processor, a data center microgrid computing collaborative two-layer deep reinforcement learning scheduling method as described in any one of claims 1 to 5 is implemented.

Images