Data center energy consumption optimization method based on joint multi-objective optimization model

Abstract

The invention discloses a data center energy consumption optimization method based on a joint multi-objective optimization model, and the method is based on an SAC-Discovery algorithm, and comprises the steps: S1, selecting a server field, calculating the cooling power consumption of different server fields through employing a cooling power consumption model, or employing the cooling power consumption model and a dynamic pricing model, and carrying out the calculation of the cooling power consumption of different server fields; selecting the server field with the minimum cooling power consumption as an optimal server field; s2, selecting a server, calculating server newly-increased power consumption of different servers in the optimal server field by using a server power consumption model, or by using the server power consumption model and a dynamic pricing model, selecting the server with the minimum server newly-increased power consumption as an optimal server, and allocating a task to the optimal server; and S3, temperature regulation and control: calculating the temperature change of the equipment where the data center is located under the natural condition by utilizing a temperature control model and combining the cooling power consumption of the optimal server field and the server power consumption of the optimal server, and carrying out temperature regulation and control.

Description

Technical Field

[0001] The present invention relates to the field of data center power consumption, and in particular to a data center energy consumption optimization method based on a joint multi-objective optimization model. Background Art

[0002] The energy consumption of data centers mainly comes from IT equipment and cooling systems. The heat generated by server operation directly affects the load of the cooling system, and the two are coupled with each other. Energy consumption models based on a single indicator cannot accurately describe the complex interactive relationship between IT equipment and cooling systems.

[0003] Therefore, how to establish a correlation model between cooling power consumption and temperature of micro data centers and maintain the temperature of micro data centers within an appropriate range through effective control strategies is a challenge that needs to be solved urgently.

[0004] We propose a joint load energy consumption model that can accurately predict the computing power consumption and cooling energy consumption of micro data centers. By combining the SAC-DISCRETE deep reinforcement learning algorithm with the experience replay mechanism and introducing a dynamic pricing strategy as an incentive mechanism, we optimize task scheduling and resource allocation, significantly reducing server energy consumption and cooling energy consumption. At the same time, we can achieve dynamic control of the temperature of micro data centers and minimize the energy consumption cost of micro data centers. Summary of the invention

[0005] The purpose of the present invention is to provide a data center energy consumption optimization method based on a joint multi-objective optimization model to solve the problems existing in the above-mentioned prior art.

[0006] In order to solve the above problem, according to a first aspect of the present invention, a data center energy consumption optimization method based on a joint multi-objective optimization model is provided, the method is based on a SAC-Discrete algorithm and includes: S1. Select the server farm Using a cooling power consumption model, or using the cooling power consumption model and a dynamic pricing model, the cooling power consumption of different server farms is calculated, and a server farm with minimum cooling power consumption is selected as the optimal server farm; S2. Select a server Using a server power consumption model, or using a server power consumption model and a dynamic pricing model, calculating the server additional power consumption of different servers in the optimal server farm, selecting a server with minimum server additional power consumption as the optimal server, and assigning tasks to the optimal server; S3. Temperature control By using the temperature control model and combining the cooling power consumption of the optimal server farm and the server power consumption of the optimal server, the temperature change of the equipment for cooling the data center under natural conditions is calculated to perform temperature control.

[0007] Optionally, the method further includes setting a reward value, wherein In step S1, the cooling power consumption model and the dynamic pricing model are used as reward values; In step S2, the server power consumption model and the dynamic pricing model are used as reward values; In step S3, the energy consumption caused by the temperature difference is taken as a reward value.

[0008] Optionally, the cooling power consumption model includes: At time t, the data center increases the cooling power consumption EnergyCool(t) depending on a parameter group, which is defined as: {TaskSize, TaskRer, TaskArray, Apath, RedTem, ServeNum} (1) Among them, TaskSize represents the number of tasks being processed, TaskRer represents the number of resources required for the task, TaskArray represents the distribution of tasks, Apath represents the interference matrix file path, ServeNum represents the number of servers in the current server farm, and RedTem represents the red temperature threshold and is the preset reference temperature.

[0009] Optionally, the cooling power consumption model includes constraints and optimization variables; The optimization variables are: (2) in, Indicates electrical power, represents the cooling coefficient, Indicates the supplied power, Indicates the supply temperature, Indicates frequency, Indicates output power; The constraints are: (3) in, Indicates the supply temperature, Indicates output power; Based on the optimization variables and the constraints, the increased cooling power consumption EnergyCool(t) of the data center at time t is calculated.

[0010] Optionally, the server power consumption model includes: server Server power consumption at time t The computation of depends on a two-element parameter set: (4) in, Indicates the utilization of the server CPU. Indicates server memory utilization; Server at time t The server power consumption is expressed as: (5) The total server power consumption of the data center at time t is expressed as: (6) The increased energy consumption of all servers in the data center at time t is expressed as: (7) Optionally, the F method of the server power consumption model is expressed as follows: (8) Among them, the values ​​of β0, β1, β2, and β4 are 20.5604, 1.3027, –0.0107, and –0.0041, respectively; the values ​​of γ0, γ1, and γ2 are 1.5963, 0.0200, and –0.6236, respectively.

[0011] Optionally, the dynamic pricing model includes a time pricing component and a real-time pricing component. The time pricing component is represented as: (9) The real-time pricing component is represented as: (10) Among them, K represents a threshold, Different charging standards are implemented, and the value range of K in step S1 and step S2 is different; After processing a task, the total server power consumption cost consumed by the data center is expressed as: (11) in, Function and is the input parameter, It is expressed as the task upload time in the tth minute, It is represented as the server power consumption in the tth minute, The total cooling power consumption cost consumed by the data center is expressed as: (12) in, Function and is the input parameter, It is expressed as the task upload time in the tth minute, Expressed as the cooling energy consumption in the tth minute.

[0012] Optionally, the temperature control model includes: The internal temperature of the data center is calculated by the following formula: (13) in, is the coefficient, Indicates the current atmospheric temperature, represents the heat input and is related to the power consumption of the server, Indicates the heat removed by the cooling system and is related to the cooling power consumption.

[0013] Optionally, the method further comprises a reward function, In step S1, the reward function is: (14) In step S2, the reward function is: (15) In step S3, the reward function is: (16) in, Indicates the energy consumption of regulating temperature difference under natural conditions, represents the energy consumption of controlling the temperature difference under the unnamed algorithm, Indicates the reward value for whether the AI ​​controls the temperature within the optimum temperature.

[0014] Optionally, in the SAC-Discrete algorithm, the goal of updating the policy parameters is to maximize a combination of expected cumulative reward and entropy.

[0015] The beneficial effects of the present invention are: Through the SAC-Discrete algorithm, combined with a comprehensive and accurate energy consumption model, and the use of a general type of dynamic pricing strategy and temperature control strategy consisting of time-of-use pricing TOUP (Time-of-Use Pricing) and real-time pricing RTP (Real-Time Pricing), energy consumption can be effectively reduced and dynamic temperature management of micro data center resources can be achieved.

[0016] (1) For micro data centers that process mixed intensive tasks, a comprehensive joint load energy consumption model is constructed, which can accurately model and optimize both server power consumption and cooling power consumption.

[0017] (2) In view of the high coupling between the internal temperature, cooling power consumption and IT power consumption of micro data centers, a correlation model between the cooling system power consumption and the server power consumption and internal temperature is established, and an efficient temperature control strategy is proposed to achieve temperature control of the data center with lower energy consumption.

[0018] (3) In order to optimize the energy consumption of micro data centers, a dynamic task scheduling method based on SAC-Discrete reinforcement learning is proposed. This method can flexibly adjust the energy consumption strategy according to the changes in real-time task load and environmental conditions, thereby improving the overall energy efficiency and realizing dynamic control of the temperature of micro data centers.

[0019] (4) Two effective reward setting methods are designed to minimize energy consumption, control active servers, and achieve load balancing. In the first and second phases, a reward value update method based on a dynamic pricing model is introduced. The real-time pricing components in the two phases are different (different K values). In the third phase, a multi-objective reward function is designed to comprehensively evaluate the temperature control effect and energy consumption performance. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 is a flow chart of a data center energy consumption optimization method based on a joint multi-objective optimization model according to an embodiment of the present invention; Figure 2 is a convergence comparison diagram of a reward value 1 of an experimental example of the present invention; Figure 3 is a convergence comparison diagram of the reward value 2 of an experimental example of the present invention; Figure 4 is a graph showing the change trend of cooling power consumption versus the number of tasks in an experimental example of the present invention; Figure 5 This is a trend diagram of the variation of the server power consumption with the number of tasks in an experimental example of the present invention; Figure 6 is a graph showing the change trend of cooling power consumption cost with the number of tasks in an experimental example of the present invention; Figure 7 It is a trend diagram of the server power consumption cost changing with the number of tasks in an experimental example of the present invention; Figure 8 This is a temperature variation diagram of a micro data center of an experimental example of the present invention; Fig. 9 This is a comparison chart of power consumption in a smile experiment of an experimental example of the present invention. DETAILED DESCRIPTION

[0021] The preferred embodiments of the present invention will be described in detail below in conjunction with the accompanying drawings so that the purpose, features and advantages of the present invention can be more clearly understood. It should be understood that the embodiments shown in the accompanying drawings are not intended to limit the scope of the present invention, but are only intended to illustrate the essential spirit of the technical solution of the present invention.

[0022] In the following description, certain specific details are set forth for the purpose of illustrating the various disclosed embodiments to provide a thorough understanding of the various disclosed embodiments. However, those skilled in the relevant art will recognize that the embodiments may be practiced without one or more of these specific details. In other cases, well-known devices, structures, and techniques associated with the present application may not be shown or described in detail to avoid unnecessarily obscuring the description of the embodiments.

[0023] References throughout the specification to "one embodiment" or "an embodiment" indicate that a particular feature, structure, or characteristic described in connection with the embodiment is included in at least one embodiment. Thus, the appearances of "in one embodiment" or "in an embodiment" in various places throughout the specification are not necessarily all referring to the same embodiment. Furthermore, the particular features, structures, or characteristics may be combined in any manner in one or more embodiments.

[0024] In the following description, in order to clearly show the structure and working mode of the present invention, many directional words will be used for description, but the words "front", "back", "left", "right", "outside", "inside", "outward", "inward", "up", "down", etc. should be understood as convenient terms and should not be understood as restrictive terms.

[0025] In addition, the terms "horizontal", "vertical", "overhanging" and the like do not mean that the components are required to be absolutely horizontal or overhanging, but can be slightly tilted. For example, "horizontal" only means that its direction is more horizontal than "vertical", and does not mean that the structure must be completely horizontal, but can be slightly tilted.

[0026] In the description of this application, it should also be noted that, unless otherwise clearly specified and limited, the terms "set", "install", "connect", and "connect" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection, or it can be indirectly connected through an intermediate medium, or it can be the internal communication of two elements. For ordinary technicians in this field, the specific meanings of the above terms in this application can be understood according to specific circumstances.

[0027] An embodiment of the present invention provides a data center energy consumption optimization method based on a joint multi-objective optimization model, referring to Figure 1 , the method is based on the SAC-Discrete algorithm and includes: S1. Select the server farm Using a cooling power consumption model, or using the cooling power consumption model and a dynamic pricing model, the cooling power consumption of different server farms is calculated, and a server farm with minimum cooling power consumption is selected as the optimal server farm; S2. Select a server Using a server power consumption model, or using a server power consumption model and a dynamic pricing model, calculating the additional server power consumption of different servers in the optimal server farm, selecting a server with the minimum additional server power consumption as the optimal server, and assigning tasks to the optimal server; S3. Temperature control By using the temperature control model and combining the cooling power consumption of the optimal server farm and the server power consumption of the optimal server, the temperature changes of the equipment in the cooling data center under natural conditions are calculated to perform temperature control.

[0028] Specifically, in step S1, the task scheduling algorithm selects the server farm with the minimum cooling power consumption based on the cooling power consumption model and the dynamic pricing model as the reward value. The server farm with the minimum cooling power consumption can be understood as the optimal server farm. After the selection is completed, the system transmits the information of the selected server farm, the cooling energy consumption and the task-related information to step S2. Step S2 receives the server farm selection result of step S1, and assigns the task to the appropriate server based on the server power consumption model and the dynamic pricing model as the reward value. The appropriate server can be understood as the optimal server. The goal of step S2 is to minimize the additional power consumption of the server and transmit the energy consumption to step S3. In step S3, the system combines the energy consumption information and task resource information transmitted in the first two stages to calculate the temperature change of the cooling equipment under natural conditions. Subsequently, the algorithm selects appropriate temperature control, takes the energy consumption caused by the temperature difference as the reward value, and ensures that the temperature of the micro-data center is maintained within the ideal range while reducing energy consumption as much as possible.

[0029] In one embodiment of the present invention, the cooling power consumption model includes the following contents: At time t, the data center increases the cooling power consumption EnergyCool(t) depending on the parameter group, which is defined as: {TaskSize, TaskRer, TaskArray, Apath, RedTem, ServeNum} (1) Among them, TaskSize indicates the number of tasks being processed, TaskRer indicates the number of resources required for the task, TaskArray indicates the distribution of tasks, Apath indicates the interference matrix file path, that is, the degree of interference between servers, ServeNum indicates the number of servers in the current server farm, and RedTem indicates the red temperature threshold and is the preset reference temperature. This temperature is a set reference temperature to ensure that the system operates normally at this temperature.

[0030] Set parameters and variables to simulate the real environment: Fan and cooling parameters: FanNum represents the number of fans, CP represents the specific heat capacity of air, AirDen and FlowSpeed ​​represent the flow rate and density of air. M_intletair represents the air mass per unit time. M_sup_array represents the air mass provided by each fan (cooling unit). M_out represents the output mass of air in g / s. Alpha represents the air heat capacity of each fan in J / g. It describes the heat capacity that each cooling fan can provide. CPU power and CPU thermal capacity: max_freq indicates the maximum frequency of the CPU. P_e_max indicates the CPU power consumption at the maximum frequency. P_e_min indicates the CPU power consumption when idle. c_0 indicates the power constant of the CPU in idle state. c_cpu indicates the thermal capacity constant of the CPU. M_cpu indicates the quality of the CPU.

[0031] Temperature constant: T_red represents the red temperature, which is the maximum temperature allowed by the system. Once this temperature is exceeded, the system will start overheat protection measures. It is usually designed to ensure the safety threshold for stable operation of the server. T_cpu_red: represents the maximum temperature used to relax the CPU temperature. P_out_red: represents the maximum allowed output power. This value represents the maximum energy output allowed by the system at the red temperature.

[0032] Optimization variables (2) in, Indicates electrical power, represents the cooling coefficient, Indicates the supplied power, Indicates the supply temperature, Indicates frequency, Indicates the output power.

[0033] Constraints (3) in, Indicates the supply temperature, Indicates the output power.

[0034] The optimization problem is defined by the Problem class in the cvxpy library, and prob.solve() is used to solve the problem. The solution process is based on the constraints and optimization variables to calculate the additional cooling power consumption of the entire micro data center at time t, expressed as: EnergyCool(t).

[0035] In one embodiment of the present invention, the server power consumption model of the present invention is more suitable for a mixed task data center, and the server power consumption model includes the following contents: server Server power consumption at time t The computation of depends on a two-element parameter set: (4) in, Indicates the utilization of the server CPU. Indicates server memory utilization; The F method (Chinese explanation) of the server power consumption model is expressed as follows: (8) Among them, the values ​​of β0, β1, β2, and β4 are 20.5604, 1.3027, –0.0107, and –0.0041, respectively; the values ​​of γ0, γ1, and γ2 are 1.5963, 0.0200, and –0.6236, respectively.

[0036] Server at time t The server power consumption is expressed as: (5) The total server power consumption of the data center at time t is expressed as: (6) The increased energy consumption of all servers in the data center at time t is expressed as: (7) In one embodiment of the present invention, the dynamic pricing model includes the following: Considering a reasonable and realistic non-flat price model, the dynamic pricing model includes a time pricing component (TOUP) and a real-time pricing component (RTP). TOUP(t) where t = Ttime i , its value depends on the time of day, usually divided into three stages: peak period, flat period, and valley period. The price is highest during the peak period and lowest during the valley period. RTP is a measure of whether the total power consumption generated every minute exceeds the threshold. Once the threshold is exceeded, the price will rise. The purpose of this non-flat price model is to minimize the energy consumption of the data center through dynamic electricity prices.

[0037] The time-based pricing component (TOUP) is expressed as: (9) The real-time pricing component (RTP) is represented as: (10) Among them, K represents a threshold, Implement different charging standards. In step S1 and step S2, the setting range of K value is different, for example, in step S1, K value can be 40-50, and in step S2, K value can be 200-250.

[0038] Different charging standards are implemented, namely two Therefore, in the entire three-stage RP-TS unnamed, after processing a task, the total server energy consumption cost of the entire micro data center is It is expressed as: (11) in, Function and is the input parameter, It is expressed as the task upload time in the tth minute, It is represented as the server power consumption at the tth minute.

[0039] Total cooling energy cost consumed by the entire micro data center It is expressed as: (12) in, Function and is the input parameter, It is expressed as the task upload time in the tth minute, Expressed as the cooling power consumption in the tth minute.

[0040] Price is a function that takes the task energy consumption and task upload time in the tth minute as input parameters. The task upload time will directly affect the values ​​of the time pricing (TOUP) component and the real-time pricing (RTP) component.

[0041] In one embodiment of the present invention, the temperature control model includes the following contents: The internal temperature of the data center is calculated by the following formula: (13) in, is the coefficient, Indicates the current atmospheric temperature, represents the heat input and is related to the power consumption of the server and the number of users, It indicates the heat taken away by the cooling system and is related to the cooling power consumption. The cooling system can be understood as a cooling fan.

[0042] The optimal temperature range is set to [18,27] degrees Celsius (18 to 27 degrees Celsius), which represents the ideal operating temperature for a micro data center. When the actual temperature exceeds this range, a temperature deviation penalty is imposed to push the temperature back to the ideal range.

[0043] Each time a control action is performed, the system will adjust the temperature according to the set temperature step and control direction. For example, when the temperature step is set to 0.5, each adjustment is 0.5 degrees Celsius; the positive or negative sign of the direction indicates whether the temperature is rising or falling. In addition, under natural conditions, the temperature will change naturally with the action of the cooling system. In control mode, the system will actively adjust the temperature based on natural changes to achieve better temperature control effects.

[0044] In one embodiment of the present invention, reinforcement learning can handle complex decision-making problems, learn and optimize strategies through interaction with the environment, and make dynamic adjustments in a changing environment. This is particularly important for handling complex and dynamic energy consumption problems in green micro and small data centers.

[0045] Action Space: The action space can be expressed as . The action space can be expressed as . The action space can be expressed as .

[0046] State Space: , The state space is a shape F M The three-dimensional list of V represents the resource usage of the entire data center.

[0047] In one embodiment of the present invention, Soft Actor-Critic for Discrete (SAC-Discrete) is the full name of the soft action-critic algorithm in discrete action space, which is an offline reinforcement learning algorithm. Through the combination of the actor-critic architecture and the maximum entropy objective, the entropy term is used to guide learning to improve the efficiency and stability of training agents to perform tasks in the environment. SAC-Discrete is divided into 5 networks, an Actor network is responsible for outputting a strategy for selecting an action in the current state; 4 Q-Critic networks are used to estimate the action-value function, which estimates the expected return under a given state and action.

[0048] The goal of SAC-Discrete is to learn a policy and a soft Q function ,in represents the probability of selecting action a in state s, Estimate the expected cumulative reward given state s and action a.

[0049] The maximum entropy objective function not only considers the reward value, but also includes the strategy The entropy term of is used to encourage exploration. SAC-Discrete defines the soft state value function V(s) and the soft Q value function by the following formula.

[0050] Soft state value function: (17) Bellman backup of soft Q function: (18) In discrete action space, SAC-Discrete uses a neural network to parameterize the soft Q function , and is trained by minimizing the following loss function: The policy improvement step updates the policy parameters by minimizing the following objective function: (19) The policy improvement step updates the policy parameters by minimizing the following objective function: (20) The policy network of SAC-Discrete outputs a probability distribution over each possible action, so there is no need to use reparameterization techniques.

[0051] SAC-Discrete can also learn the temperature parameter α through the following objective function: (twenty one) In this way, SAC-Discrete can maintain efficient and stable training in complex reinforcement learning tasks, balance exploration and exploitation, and ultimately find the optimal strategy.

[0052] In one embodiment of the present invention, a reasonable design of a reward value (usually expressed as R) is crucial to the learning of the agent, and R directly affects the learning strategy of the agent. In order to achieve energy consumption optimization of green micro data centers, a reward value based on a realistic price model is designed, which considers how to minimize the energy consumption of the data center in a reasonable and realistic manner. Another reward value is also designed, which is expressed as the energy consumption required to regulate the temperature, and is used to control the temperature of the data center to remain in an appropriate range. In the above context, R represents the reward calculated after the task is successfully executed, which reflects the immediate feedback of the decision made by the agent in the current state.

[0053] This reward design needs to pay special attention to the server status and task status as well as the task upload time, so that the agent can effectively learn and complete the task within the specified time. A reasonable reward function can encourage the agent to take more effective actions, speed up task execution, and avoid unnecessary delays.

[0054] The goal of the first two stages of the three-stage RP-TS processor is to minimize long-term energy consumption by taking a series of actions, and the third stage is to maintain the temperature of the data center within a suitable range. After taking action in the current state st, the system will evolve to a new state s t+1 , and obtains a reward from the environment, that is, the energy cost of taking action at the current moment increases, that is, the current energy cost minus the previous energy cost.

[0055] In steps S1 and S2, the time-of-use pricing component and the scheduled pricing component in the dynamic pricing model are introduced to implement a reward and punishment mechanism for energy consumption. For example, if the price is above a certain threshold per minute, a reward is given; if the price is below the threshold, a penalty is imposed. This approach is designed to incentivize the optimization of energy consumption allocation while reflecting the impact of fluctuations in real-time and time-of-use electricity prices on overall costs.

[0056] In step S1, the reward function after executing the entire task list can be obtained as: (14) In step S2, the reward function after executing the entire task list can be calculated as: (15) Temperature changes directly affect the energy consumption of the data center, so it is hoped that the temperature can always be maintained within a suitable range. In step S3, a multi-objective reward function is designed to comprehensively evaluate the temperature control effect and energy consumption performance. When the temperature is within the most suitable range, the reward function will give positive incentives; when the temperature exceeds this range, it will be punished according to the degree of deviation. The greater the deviation, the heavier the penalty. In addition, the reward function also measures the control efficiency by comparing the temperature difference under the unnamed algorithm and natural control. The smaller the temperature difference of the control, the lower the energy consumption required. Therefore, when the temperature difference of the unnamed algorithm is smaller than that of the natural control, it will receive additional rewards to encourage more efficient temperature management.

[0057] In step S3, the reward function is: (16) in, Indicates the energy consumption of regulating temperature difference under natural conditions, represents the energy consumption of controlling the temperature difference under the unnamed algorithm, Indicates the reward value for whether the AI ​​controls the temperature within the optimum temperature.

[0058] In one embodiment of the present invention, in the SAC-Discrete algorithm, the goal of updating the policy parameters is to maximize the combination of the expected cumulative reward and entropy. The policy parameters are updated using the gradient descent method. ,in is the temperature parameter, Indicates in status Next select action strategy. express About parameters gradient. represents the logarithmic strategy probability.

[0059] (twenty two) The policy parameters are updated as follows: (twenty three) here represents the value of a state-action pair, The term is used to encourage the exploration of the policy by maximizing the entropy of the policy.

[0060] Critic Update: In the SAC-Discrete algorithm, the goal of the critic is to learn a soft Q function . Use mean squared error loss to update the critic parameters . Indicates immediate reward, represents the soft state value function, which is the estimated value of the cumulative reward for the next state, Represents the discount factor.

[0061] The soft state value function V(s)V(s)V(s) is defined as: (twenty four) Target Q value Represents the target value of the current state, that is, the estimated value of the cumulative reward of the current state: (25) TD-error represents the difference between the target Q-value and the estimated Q-value, thus guiding the training of the Q-function: (26) Loss Function It is the loss function of the Q-value function, usually the mean square error (MSE) loss between the target Q-value and the estimated Q-value: (27) Parameter update of the Critic network: (28) in Represents the gradient of the Q function, which is used to update the parameters of the Q function Represents the learning rate. The Critic network uses gradient descent to update parameters.

[0062] Experimental example This experimental example will elaborate on the experimental settings, data sets, evaluation indicators, and compare the proposed joint model with multiple benchmark algorithms. The experiments in this study were conducted on a computer equipped with an Intel® CoreTM i9-9900K CPU, 250GB RAM, and an RTX 3080 graphics card (24 GB). The relevant algorithms were written in Python 3.7 and PyTorch 1.13. All experiments were completed on the same computer.

[0063] Experimental configuration: Through simulation experiments, the complete process of handling mixed tasks in a micro-data center is deeply analyzed. The simulation environment is highly close to reality, covering the server receiving and executing task requests, the operation of the cooling system, and the scheduling algorithm adjusting the temperature. The experiment sets up a micro-data center consisting of 100 servers, which are divided into 10 server farms, each containing 10 servers. To enhance the authenticity of the experiment, two virtual machines are configured on each server, and the task dependencies in the user task model and the resource quotas of the virtual machines and servers are randomly generated. In addition, the task information in the experiment (number of tasks, CPU and memory requirements, indoor temperature, etc.) is derived from real data records to ensure the reliability of the simulation results.

[0064] Dataset: The dataset comes from the log records of multiple micro data centers of a Chinese company. The dataset records the daily resource consumption of the micro data centers, provides task information recorded with timestamps (each time interval is 1 minute), records the size of each task and the CPU cycles it requires, as well as the server CPU and memory load, and the indoor temperature at the corresponding time.

[0065] Evaluation indicators: i) Total cooling energy consumption after executing the entire task list.

[0066]

[0067] Total server energy consumption after executing the entire task list.

[0068]

[0069] ii) Total server energy cost after executing the entire task list.

[0070]

[0071] Cooling energy cost after executing the entire task list.

[0072]

[0073] in, Indicates the upload time of the task.

[0074] Benchmark Methodology: In order to evaluate the performance of the proposed data center energy consumption optimization method based on the joint multi-objective optimization model, six different classical scheduling methods are selected for comparison.

[0075] AC

[29] : The Actor-Critic algorithm is a reinforcement learning algorithm that combines the value function and strategy ideas. The actor is responsible for selecting actions based on the current state, and the critic is responsible for evaluating the quality of the action selected by the actor. The strategy is learned through the interaction between the actor and the critic.

[0076] A2C

[30] : This is a deep reinforcement learning algorithm that combines the actor-critic architecture and asynchronous parallel learning. It allows multiple agents to interact with the environment simultaneously and continuously adjust the parameters of the actor and critic to accelerate the training process of the model.

[0077] DQN

[31] : This is an algorithm that combines deep learning and reinforcement learning to solve decision-making problems in complex environments. It uses a neural network to approximate the Q-value function, thereby evaluating the value of state-action pairs and selecting actions that maximize long-term rewards.

[0078] Dueling DQN

[32] : Dueling DQN is a deep reinforcement learning algorithm that improves the stability and convergence speed of the algorithm by decomposing the Q value into two parts: state value and action advantage, and using two independent networks to estimate these two parts respectively.

[0079] D3QN

[33] : D3QN alleviates the problem of overestimation of Q values ​​in the traditional DQN algorithm by introducing Double DQN and Dueling DQN. It also separates the state value and advantage function through a dual network structure, thereby more accurately evaluating the action value.

[0080] PPO

[34] : As a policy gradient-based algorithm, PPO ensures the stability of the training process by limiting the amplitude of policy updates. At the same time, PPO uses the clipping probability ratio and value function to optimize the strategy and improve the learning efficiency of the algorithm.

[0081] Specific experimental example 1: Analysis of the convergence speed of reward value The number of iterations required for convergence can reflect the learning efficiency of the algorithm. The fewer the number of iterations, the faster the algorithm can find a near-optimal strategy, which is very important for scenarios with limited computing resources or requiring real-time decision-making. Fewer iterations means less computing cost and a more efficient training process. Figure 2 and Figure 3 The reward value convergence of seven reinforcement learning algorithms, including SAC-Discrete, PPO, AC, A2C, DQN, DUE, and D3QN, in the first and second phases on 500 task sets is shown. The experimental results show that when the learning rate , Under the setting of , the SAC-Discrete algorithm shows advantages in reward value, convergence speed and stability. Specifically, the reward value of the SAC-Discrete algorithm is significantly higher than that of other algorithms except PPO. The reason is that the entropy regularization mechanism encourages the diversity of strategies. The convergence speed of the SAC-Discrete algorithm is also very fast. It only takes about 5 rounds of training to reach a stable reward value, which is significantly faster than the A2C algorithm. This is mainly attributed to the entropy optimization mechanism that can quickly focus on high-reward actions. The training curve of the SAC-Discrete algorithm is smooth and the reward value fluctuates little, reflecting the good stability of the algorithm. The reason is that the soft policy update mechanism effectively avoids the problem of over-optimization of the strategy and improves the robustness of the algorithm.

[0082] Specific experimental example 2: model performance analysis The performance of different models is compared through experiments when processing 100, 500, 1000, 1500 and 2000 tasks on 100 servers. The performance indicators include server energy consumption, cooling energy consumption, server energy consumption cost and cooling energy consumption cost.

[0083] 1. Impact of task quantity on cooling energy consumption exist Figure 4 In the simulation, as the number of tasks gradually increases, the total cooling energy consumption of the micro data center also increases. Compared with the other seven algorithms, the SAC-Discrete algorithm has a significant energy consumption advantage. Specifically, when the number of servers remains unchanged and the number of tasks increases from 100 to 2000, the total cooling energy consumption of the SAC-Discrete algorithm is reduced by 20.62%, 25.69%, 32.33%, 31.04%, 36.77%, and 37.64% on average compared with the PPO, AC, A2C, DQN, DUE, and D3QN algorithms.

[0084] The SAC-Discrete algorithm guides policy learning through entropy regularization, encouraging the agent to explore in the early stages, so that the algorithm can quickly adapt to changes in resource requirements when the number of tasks gradually increases, avoid being limited to suboptimal scheduling schemes, and thus optimize cooling energy consumption.

[0085] 2. Impact of the number of tasks on server energy consumption exist Figure 5In the study, as the number of tasks gradually increases, the total server energy consumption of the micro-data center also increases. Compared with the other seven algorithms, the SAC-Discrete algorithm has a significant energy consumption advantage. Specifically, when the number of servers remains unchanged and the number of tasks increases from 100 to 2000, the total server energy consumption of the SAC-Discrete algorithm is reduced by 22.97%, 23.18%, 22.91%, 12.40%, 13.81%, and 14.39% on average compared with the PPO, AC, A2C, DQN, DUE, and D3QN algorithms.

[0086] The reason is that the SAC-Discrete algorithm adopts a soft update strategy and updates the target network parameters with a small step size. Compared with other algorithms, it can gradually approach the optimal strategy in the process of stable learning, thereby reducing volatility and improving the overall optimization effect of energy consumption.

[0087] 3. Impact of the number of tasks on server energy consumption and cooling energy consumption Figure 6 and Figure 7 It shows that as the number of tasks in the micro data center continues to increase, the total cooling energy consumption and total server power consumption show a clear upward trend. Specifically, when the number of servers remains unchanged and the number of tasks increases from 1000 to 2000, the total cooling energy consumption cost of the SAC-DISCRETE algorithm in scheduling tasks is significantly lower than that of other comparison algorithms. Compared with the PPO, AC, A2C, DQN, DUE, and D3QN algorithms, the cooling energy consumption cost of the SAC-Discrete algorithm is reduced by 37.50%, 50.45%, 50.87%, 50.51%, 54.71%, and 56.10% on average, respectively.

[0088] Moreover, the SAC-Discrete algorithm also showed significant advantages in the total server energy cost of scheduling tasks. Compared with the comparison algorithm, the server energy cost of the proposed algorithm was reduced by 50.34%, 46.54%, 45.84%, 29.90%, 32.99% and 32.29% on average, respectively.

[0089] It is worth noting that this cost is calculated based on a dynamic electricity price model. Compared with the improvement in energy consumption, SAC performs better in electricity price optimization. The reason is that in the real-time electricity price strategy, which is updated in minutes to reflect the instantaneous fluctuations in electricity costs, the energy consumption generated by SAC-DISCRETE is lower than a certain critical threshold every minute, so that the electricity price every minute is always lower than this specific value.

[0090] Specific experimental example 3: Data center temperature evaluation based on the number of tasks like Figure 8As shown in the figure, the temperature changes of the micro data center under natural conditions and under the control of seven algorithms are shown. It can be seen that the temperature under the control of the SAC-Discrete scheduling algorithm is always lower than the temperature under natural conditions, and the fluctuation is small, showing higher stability. In addition, except for the PPO algorithm, the overall temperature of the SAC-Discrete algorithm is usually lower than that of other algorithms. Compared with the PPO algorithm, SAC-DISCRETE shows a lower temperature in the early stage, and when it is close to 250 minutes, the temperatures of the two gradually approach each other. The reason is that SAC-DISCRETE uses a discretized action space, which makes the selection of each action more precise and controllable, so as to have stability. It should be noted that the temperature control uses the energy consumption of controlling the temperature as the reward value, ensuring that the temperature of the micro data center is maintained within an appropriate range under the premise of minimizing energy consumption.

[0091] Specific experimental example 4: Data center temperature evaluation based on the number of tasks In the scenario of evaluating 100 servers and 500 tasks, the total cooling power consumption performance difference between the single server power consumption model and the joint load model when scheduling tasks was considered. The different structures of the two models were considered: the single cooling model directly selects the server, which is a two-stage model; while the joint load model first selects the server farm and then selects the specific server, which is a three-stage model. By comparing the total cooling energy consumption of the two, such as Fig. 9 As shown in the figure, during the whole process when the number of tasks increases from 100 to 2000, the cooling power consumption of the three-stage scheduling model based on the joint load model is higher than that of the single server model by the following percentages: 39.85%, 36.56%, 37.64%, 35.67%, and 34.74%.

[0092] The reason is that in the joint load model, the server farm is selected based on the cooling power consumption model first, and the number of servers in each server farm is different, which will directly affect the distribution of the entire task. The selection of server farms and load distribution have an important impact on cooling energy consumption. If the calculation is based only on a single server power consumption model, it will not be possible to select a server farm with an appropriate number of servers, resulting in unreasonable task allocation and ultimately increased energy consumption.

[0093] The preferred embodiments of the present invention have been described in detail above, but it should be understood that after reading the above teachings of the present invention, those skilled in the art may make various changes or modifications to the present invention. These equivalent forms also fall within the scope defined by the appended claims of this application.

Claims

1. A data center energy consumption optimization method based on a joint multi-objective optimization model, characterized in that: The method is based on the SAC-Discrete algorithm and comprises: S1. Select the server farm Using a cooling power consumption model, or using the cooling power consumption model and a dynamic pricing model, the cooling power consumption of different server farms is calculated, and a server farm with minimum cooling power consumption is selected as the optimal server farm; S2. Select a server Using a server power consumption model, or using a server power consumption model and a dynamic pricing model, calculating the server additional power consumption of different servers in the optimal server farm, selecting a server with minimum server additional power consumption as the optimal server, and assigning tasks to the optimal server; S3. Temperature control By using the temperature control model and combining the cooling power consumption of the optimal server farm and the server power consumption of the optimal server, the temperature change of the equipment for cooling the data center under natural conditions is calculated to perform temperature control.

2. The data center energy consumption optimization method based on the joint multi-objective optimization model according to claim 1 is characterized in that: The method also includes setting a reward value, wherein In step S1, the cooling power consumption model and the dynamic pricing model are used as reward values; In step S2, the server power consumption model and the dynamic pricing model are used as reward values; In step S3, the energy consumption caused by the temperature difference is taken as a reward value.

3. The data center energy consumption optimization method based on the joint multi-objective optimization model according to claim 1 is characterized in that: The cooling power consumption model includes: At time t, the data center increases the cooling power consumption EnergyCool(t) depending on a parameter group, which is defined as: {TaskSize, TaskRer, TaskArray, Apath, RedTem, ServeNum} (1) Among them, TaskSize represents the number of tasks being processed, TaskRer represents the number of resources required for the task, TaskArray represents the distribution of tasks, Apath represents the interference matrix file path, ServeNum represents the number of servers in the current server farm, and RedTem represents the red temperature threshold and is the preset reference temperature.

4. The data center energy consumption optimization method based on the joint multi-objective optimization model according to claim 3 is characterized in that: The cooling power consumption model includes constraints and optimization variables; The optimization variables are: （2） in, Indicates electrical power, represents the cooling coefficient, Indicates the supplied power, Indicates the supply temperature, Indicates frequency, Indicates output power; The constraints are: （3） in, Indicates the supply temperature, Indicates output power; Based on the optimization variables and the constraints, the increased cooling power consumption EnergyCool(t) of the data center at time t is calculated.

5. The data center energy consumption optimization method based on the joint multi-objective optimization model according to claim 1 is characterized in that: The server power consumption model includes: server Server power consumption at time t The computation of depends on a two-element parameter set: （4） in, Indicates the utilization of the server CPU. Indicates server memory utilization; Server at time t The server power consumption is expressed as: （5） The total server power consumption of the data center at time t is expressed as: （6） The increased energy consumption of all servers in the data center at time t is expressed as: （7）。 6. The data center energy consumption optimization method based on the joint multi-objective optimization model according to claim 5 is characterized in that: The F method of the server power consumption model is expressed as follows: （8） Among them, the values ​​of β0, β1, β2, and β4 are 20.5604, 1.3027, –0.0107, and –0.0041, respectively; the values ​​of γ0, γ1, and γ2 are 1.5963, 0.0200, and –0.6236, respectively.

7. The data center energy consumption optimization method based on the joint multi-objective optimization model according to claim 1 is characterized in that: The dynamic pricing model includes a time pricing component and a real-time pricing component. The time pricing component is represented as: （9） The real-time pricing component is represented as: （10） Among them, K represents a threshold, Different charging standards are implemented, and the value range of K value in step S1 and step S2 is different; After processing a task, the total server power consumption cost consumed by the data center is expressed as: （11） in, Function and is the input parameter, It is expressed as the task upload time in the tth minute, It is represented as the server power consumption in the tth minute, The total cooling power consumption cost consumed by the data center is expressed as: （12） in, Function and is the input parameter, It is expressed as the task upload time in the tth minute, Expressed as the cooling energy consumption in the tth minute.

8. The data center energy consumption optimization method based on the joint multi-objective optimization model according to claim 1 is characterized in that: The temperature control model includes: The internal temperature of the data center is calculated by the following formula: （13） in, is the coefficient, Indicates the current atmospheric temperature, represents the heat input and is related to the power consumption of the server, Indicates the heat removed by the cooling system and is related to the cooling power consumption.

9. The data center energy consumption optimization method based on the joint multi-objective optimization model according to claim 2 is characterized in that: The method further comprises a reward function, In step S1, the reward function is: （14） In step S2, the reward function is: （15） In step S3, the reward function is: （16） in, Indicates the energy consumption of regulating temperature difference under natural conditions, represents the energy consumption of controlling the temperature difference under the unnamed algorithm, Indicates the reward value for whether the AI ​​controls the temperature within the optimum temperature.

10. The data center energy consumption optimization method based on the joint multi-objective optimization model according to claim 1 is characterized in that: In the SAC-Discrete algorithm, the goal of policy parameter updating is to maximize the combination of expected cumulative reward and entropy.

Images