A data center water cooling unit energy consumption optimization method and system based on DQN

Abstract

The application discloses a data center water cooling unit energy consumption optimization method and system based on DQN, and provides an efficient solution for PUE optimization of a water cooling data center. The application mainly comprises the following steps: collecting massive data by means of Internet of Things equipment, modeling water cooling unit energy consumption based on a neural network, considering normal working conditions and safety boundary conditions of the water cooling unit, converting the water cooling unit energy consumption optimization process into a Markov decision process, and designing a water cooling unit energy consumption optimization algorithm based on the water cooling unit energy consumption model, so that the energy consumption is reduced under the premise of ensuring the safe operation of the data center. The application can significantly reduce the energy consumption of the water cooling unit under the premise of ensuring the normal operation of the water cooling unit and the normal operation of the data center, has the advantages of fast convergence speed and remarkable optimization effect, and can exhibit greater potential in future more complex optimization scenarios.

Description

Technical Field

[0001] This invention belongs to the field of data center energy-saving technology, and relates to deep learning, reinforcement learning technology, and data center PUE optimization scheme design. Specifically, it relates to a method and system for optimizing the energy consumption of data center water-cooled units based on DQN. Background Technology

[0002] The Power Usage Effectiveness (PUE) metric for data centers is defined as the ratio of total energy consumption to the energy consumption of IT equipment. It's worth noting that nearly half of the energy costs in traditional data centers are related to the cooling system, and water-cooling systems are currently the most commonly used cooling system in traditional data centers. Therefore, optimizing the energy consumption of water-cooling equipment is key to reducing the PUE of water-cooled data centers.

[0003] To optimize the energy consumption of water-cooled systems, existing research typically employs the following strategy: first, establishing an energy consumption model for the equipment or system; second, designing optimization strategies based on the established model. The model in the first step is usually built based on thermodynamic principles and empirical data, while the second step generally uses mathematical reasoning to adjust the inputs for energy consumption optimization. For example, Yu et al. published an article in 2008 entitled "Optimization of water-cooled chiller system with load-based speed control," which created thermodynamic analysis models of chillers and cooling towers. These models were used to evaluate the impact of different cooling tower and condensate pump control strategies on cooling system performance. Furthermore, Wang et al. published an article in 2011 entitled "A holistic and optimal approach for data center cooling management," which derived a simplified model based on basic mass and energy balance principles, describing the complex mass and energy flows in data centers using raised floors for air circulation and cooling, and introduced a model predictive controller for energy consumption optimization. However, these methods have limitations in generalization ability and face significant challenges in stability and timeliness.

[0004] In recent years, the development of the Internet of Things (IoT), especially in the field of wireless sensor networks, has fundamentally changed the way we collect data from real data centers. This cutting-edge technology enables us to rapidly accumulate massive amounts of data, creating unprecedented opportunities for data-driven energy consumption modeling and customized optimization strategy design for various devices in data centers. It not only allows us to gain a deeper understanding of the complex operations of data centers but also paves the way for the development of more efficient, adaptive, and finely tuned solutions. Furthermore, with the rapid development of deep learning (DL) and reinforcement learning (RL), Deep Q-Network (DQN), a deep reinforcement learning (DRL) algorithm that combines the advantages of both, holds promise for addressing the challenges faced by traditional optimization algorithms. Therefore, considering the powerful fitting ability of deep learning, the superior optimization capabilities of the DQN algorithm, and the crucial role of water-cooled units in data centers, it is necessary to research a DQN-based energy consumption optimization model and method for water-cooled units to efficiently optimize the PUE of data centers while ensuring their normal operation. Summary of the Invention

[0005] Purpose of the invention: Water-cooled units are the primary energy-consuming equipment in water-cooled data centers. To address the shortcomings of existing technologies, the purpose of this invention is to provide a method and system for optimizing the energy consumption of water-cooled units in data centers based on DQN. This method can reduce the energy consumption of water-cooled units while ensuring the safe operation of the data center, thereby optimizing the PUE of the water-cooled data center. It has the advantages of good optimization effect and fast algorithm convergence.

[0006] Technical Solution: To achieve the above-mentioned objectives, this invention provides a method for optimizing the energy consumption of data center water-cooled units based on DQN, comprising the following steps:

[0007] Data from measurement points of water-cooled unit equipment is acquired using Internet of Things (IoT) technology.

[0008] Based on the acquired equipment measurement data, a water-cooled unit energy consumption model based on neural networks is constructed. The input parameters of the water-cooled unit energy consumption model are the key equipment measurement data, and the output is the normalized power value of the water-cooled unit.

[0009] The energy consumption optimization process of water-cooled units is modeled as a Markov decision process, constructing an energy consumption optimization model for water-cooled units. Based on DQN (Distributed Quantity Optimization), the energy consumption of water-cooled units is optimized to reduce energy consumption, thereby reducing the power efficiency (PUE) of the data center. In the energy consumption optimization model, the state includes the values ​​of the input parameters at specific time points, the action is the adjustment of the cooling water outlet temperature (CWOT), the action space is the adjustable range of CWOT each time, and boundary conditions are set for the evaporator outlet temperature (EOWT) of the water-cooled unit. Rewards include the energy efficiency reduction achieved by transitioning to the next state and the penalty for exceeding the safety boundary in the next state. The EOWT is predicted by a neural network-based EOWT prediction model, whose input parameters are the input parameters of the water-cooled unit energy consumption model excluding EOWT, and whose output is the specific value of EOWT.

[0010] As a preferred approach, considering the state space of CWOT, if the action chosen by the intelligent decision will cause CWOT to exceed its state space, then the next state is set to the nearest CWOT boundary value.

[0011] Preferably, a data-driven approach is used, combining a multilayer perceptron (MLP) in a deep neural network to construct the energy consumption model and EOWT prediction model for the water-cooled unit.

[0012] Preferably, the key equipment measurement data include: chilled water supply pressure, chilled water supply temperature, condenser inlet water temperature, condenser outlet water temperature, evaporator inlet water temperature, evaporator outlet water temperature, cooling water inlet water temperature, cooling water outlet water temperature, cooling water outlet water pressure, compressor suction pressure, compressor suction temperature, compressor discharge pressure, compressor discharge temperature, water-side electric regulating valve opening feedback, refrigeration electric regulating valve opening feedback, and chiller operating percentage.

[0013] Preferably, the reward r in the energy consumption optimization model of the water-cooled unit t It consists of two parts:

[0014] r t =r dec,t +r penal,t

[0015] Where, r dec,t After performing the action, from the current state s t Transition to the next state s t+1 The reward for the achieved energy reduction, r penal,t Indicates the next state s t+1 Penalty for exceeding the safety boundary in EOWT.

[0016] As a preferred option, the reward for reduced energy consumption r dec,t Represented as:

[0017]

[0018] Among them, P t ,P t+1 The energy consumption values ​​for the current state and the next state are obtained by calling the energy consumption model N1 of the water-cooled unit, where λ1 is a pricing factor greater than 1.

[0019] As a preferred option, the penalty term r penal,t Defined as:

[0020]

[0021] in, and Let N2 and N2(s) represent the lower and upper bounds of EOWT, respectively, where λ2 is the penalty pricing factor, and N2(s) are the upper and lower bounds of EOWT. p,t+1 ) represents the EOWT value for the next state, obtained by calling the EOWT prediction model, s p,t+1 This represents the model input.

[0022] As a preferred approach, the Q-Net in the DQN algorithm is designed based on a deep neural network (DNN). The state vector composed of key equipment measurement point data is used as the input of the Q-Net, and the corresponding action value Q(s,a) of taking action a in each state is output. A training network and a target network are introduced to train the parameters of the Q-Net.

[0023] Based on the same inventive concept, this invention provides a data center water-cooled unit energy consumption optimization system based on DQN, comprising:

[0024] The data acquisition module is used to acquire measurement point data of water-cooled unit equipment based on Internet of Things (IoT) technology.

[0025] The water-cooled unit energy consumption modeling module is used to construct a water-cooled unit energy consumption model based on neural networks based on the acquired equipment measurement point data; the input parameters of the water-cooled unit energy consumption model are the key equipment measurement point data, and the output is the normalized value of the water-cooled unit power.

[0026] The EOWT prediction modeling module is used to construct an EOWT prediction model based on neural networks based on the acquired equipment measurement point data. The input parameters of the EOWT prediction model are the input parameters of the water-cooled unit energy consumption model that does not include EOWT, and the output is the specific value of EOWT.

[0027] The energy consumption optimization module is used to model the energy consumption optimization process of water-cooled units as a Markov decision process, construct an energy consumption optimization model for water-cooled units, and implement energy consumption optimization of water-cooled units based on DQN to reduce energy consumption of water-cooled units, thereby achieving the effect of reducing data center PUE. The state in the energy consumption optimization model of water-cooled units includes the value of the input parameters of the energy consumption model of water-cooled units at a specific time point, the action is the adjustment amount of cooling water outlet temperature CWOT, the action space is the adjustable range of CWOT each time, and boundary conditions are set for the evaporator outlet temperature EOWT of water-cooled units. The rewards include the energy efficiency reduction achieved by transitioning to the next state and the penalty for exceeding the safety boundary in the next state.

[0028] Based on the same inventive concept, the present invention provides a computer system including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the computer program is loaded onto the processor, it implements the steps of the DQN-based data center water-cooled unit energy consumption optimization method.

[0029] Beneficial Effects: Compared with existing technologies, this invention has the following beneficial effects: 1. This invention effectively utilizes the advantages of IoT technology, coordinating the collection, processing, and optimization instruction issuance of measurement point data through a Data Center Infrastructure Management (DCIM) system; 2. This invention adopts a data-driven approach to model the energy consumption of water-cooled units, making the model more generalizable and iterative, and solving the problems of insufficient timeliness and stability caused by traditional physical models; 3. This invention models the energy consumption optimization problem of water-cooled units as a Markov decision process problem, considering the safety boundary of equipment operation, making the subsequent design of DQN-based optimization algorithms more interpretable and secure. Experiments show that this invention can significantly reduce the energy consumption of water-cooled units, thereby optimizing the PUE of water-cooled data centers, with the advantages of better optimization effect and faster algorithm convergence; 4. The neural networks designed in this invention all adopt lightweight network models, which can be quickly retrained to adapt to changes in equipment state, with the advantages of low complexity and high flexibility. Attached Figure Description

[0030] Figure 1 This is a PUE optimization framework diagram for a water-cooled data center according to an embodiment of the present invention.

[0031] Figure 2 This is a schematic diagram of an MLP network used for energy consumption prediction of water-cooled units in an embodiment of the present invention.

[0032] Figure 3 This is a schematic diagram of the Q-Net based optimization algorithm of DQN in an embodiment of the present invention.

[0033] Figure 4This is a comparison chart of the rewards of the DQN-based optimization algorithm and the random action algorithm in the examples of this invention.

[0034] Figure 5 This is a comparison chart showing the percentage reduction in energy consumption between the DQN-based optimization algorithm and the random action algorithm in this invention example.

[0035] Figure 6 This is a comparison chart showing the percentage reduction in energy consumption between the DQN-based optimization algorithm and the traditional global optimization algorithm in this invention example.

[0036] Figure 7 This is a comparison chart showing the number of iterations required for convergence between the DQN-based optimization algorithm and the traditional global optimization algorithm in this invention example. Detailed Implementation

[0037] To make the objectives, technical solutions, and advantages of this invention clearer, the embodiments of this invention will be described in detail below with reference to the accompanying drawings. It should be understood that the specific examples described herein are merely illustrative of the invention, but the scope of protection of this invention is not limited to the following embodiments.

[0038] This invention discloses a novel energy consumption optimization method for data center water-cooled units based on Direct Quality Network (DQN), aiming to effectively reduce the power usage effect (PUE) of data centers. First, a PUE optimization framework for water-cooled data centers is designed, and measurement data of water-cooled unit equipment is acquired using IoT technology. Then, based on the acquired equipment measurement data, a water-cooled unit energy consumption model based on a neural network is constructed. The water-cooled unit energy consumption optimization process is modeled as a Markov decision process, and a water-cooled unit energy consumption optimization model is built. Based on DQN, the water-cooled unit energy consumption is optimized to reduce energy consumption, thereby achieving the effect of reducing the data center PUE.

[0039] Figure 1This paper showcases an IoT-enabled PUE optimization framework for water-cooled data centers. In this data center scenario, various sensors are deployed on the equipment, such as temperature, humidity, and pressure sensors on the water-cooled units. These sensors are connected to a sensor network via sensor nodes. The sensor nodes are responsible for sensing and collecting various parameters from the physical environment. The collected data is then aggregated and processed at the aggregation node before being transmitted to the gateway node. After processing by the gateway node, the data is uploaded to the Data Center Infrastructure Management (DCIM) system and stored in databases according to predefined rules: a MySQL database stores all device measurement point names, assigning each a unique UID; an InluxDB database stores time-series data for each device measurement point, which can be retrieved based on the UID corresponding to the device measurement point name. The collected data undergoes several preprocessing operations, including data cleaning, augmentation, feature extraction, and normalization. Next, the processed dataset is used to train a device energy consumption model, which is then deployed in a PUE prediction algorithm. Based on the feedback from the device energy consumption model and the operational safety boundaries of the equipment, an energy consumption optimization algorithm is designed. Finally, the PUE optimization algorithm generates control commands and sends them to the DCIM system, which then adjusts the adjustable parameters of the devices in the real system accordingly. Simultaneously, the real system continuously transmits data through the devices' sensors.

[0040] A water-cooled chiller is the core equipment in a water-cooling system, consisting of a compressor, condenser, evaporator, and control system. The compressor compresses the refrigerant into a high-pressure gas, raising its temperature. The condenser transfers the heat from the high-temperature, high-pressure gas to the cooling medium (usually air or water), cooling it and condensing it into a liquid. The evaporator evaporates the liquid refrigerant into low-temperature, low-pressure vapor, absorbing and carrying away indoor heat. The control system monitors and regulates the chiller's operating status to ensure efficient and stable operation. Through the refrigeration cycle, the water-cooled chiller extracts heat from the air or water, lowering the chilled water outlet temperature. The chilled water then flows to designated locations for heat exchange, achieving cooling.

[0041] This invention considers the crucial role of water-cooled units in the overall energy consumption of water-cooled data centers. It uses water-cooled units as the starting point for PUE optimization, reducing the PUE of the water-cooled data center by lowering the energy consumption of the water-cooled units. Accordingly, based on the PUE optimization framework described in this invention, it is necessary to construct a water-cooled unit energy consumption model, a water-cooled unit energy consumption optimization model, and design a water-cooled unit energy consumption optimization algorithm.

[0042] This invention first constructs an energy consumption model for the water-cooled chiller unit. In this data center scenario, since all water-cooled chillers integrated into the system have the same specifications, a single water-cooled chiller unit can be selected for modeling. However, deriving the energy consumption model solely based on physical formulas presents significant challenges, primarily due to the lack of energy consumption calculation formulas applicable to specific water-cooled chiller models. Therefore, considering this, we aim to leverage the potential of IoT technology and the abundant data collected from real water-cooled chillers to overcome the limitations imposed by the lack of customized formulas through a data-driven approach. This method, based on the utilization of rich real-world data, represents a feasible solution.

[0043] Specifically, we constructed a highly accurate energy consumption model for the chiller unit using a multilayer perceptron (MLP). As shown in Table 1, we selected 16 measurement points of the chiller unit equipment, including cooling water outlet temperature (CWOT), evaporator outlet temperature (EOWT), chilled water supply pressure (CWSP), and cooling water outlet pressure (CWOP), as input parameters for the energy consumption model. We selected the normalized energy consumption value of the chiller unit as the output parameter of the model. The model was trained using an MLP network with three hidden layers and the ReLU activation function. The network architecture is as follows: Figure 2 As shown.

[0044] Table 1 Measurement Points of Water-cooled Unit Equipment

[0045]

[0046] In deep learning, R 2 R-squared is a widely used statistical metric for evaluating the accuracy of a regression model fit. It is typically calculated by dividing the squared difference between predicted and actual values ​​by the total squared difference. It measures the proportion of variation in the dependent (or target) variable that is explained by the model. 2 The value range is [0,1]. If R 2 =0, indicating that the model failed to explain any variation in the target variable; if R = 0, it means that the model failed to explain any variation in the target variable; 2 =1 indicates that the model fully explains all the variations in the target variable. In practical applications, R... 2 The closer the value is to 1, the better the model fit. Tests on the energy consumption model of the water-cooled unit show that for a randomly selected test set with a batch size of 512, R... 2 The performance was stable above 0.97, and the order of magnitude of the loss value was stable at 1e-5. This indicates that the energy consumption model of the water-cooled unit has high performance and can serve as a reliable basis for subsequent energy consumption optimization algorithms for water-cooled units.

[0047] This invention, in its embodiments, models the energy consumption optimization problem of water-cooled units based on Markov decision processes, constructing an energy consumption optimization model for water-cooled units. Given the observed state s... t The intelligent agent will perform action a t Next, the environment provides a corresponding reward r. t The state will then transition to s. t+1 In this invention, the design of states, actions, state transitions, state terminations, and rewards is as follows:

[0048] State: We constructed a 16*1 dimensional array containing the values ​​of the 16 input parameters of the water-cooled unit energy consumption model at a specific time point t0. This array represents the state of the water-cooled unit at t0.

[0049] Action: Considering the actual situation of the water-cooled unit, we choose the cooling water outlet temperature CWOT as the adjustable parameter. That is, we define the action as the adjustment quantity of CWOT, and the action space is the range that CWOT can adjust each time. We assume that the state space of CWOT is... Assuming the action space is Minimum step size is If, according to the agent's policy, the action chosen in the current state would cause the CWOT to go beyond its state space, then we set the next state to the nearest CWOT boundary value.

[0050] State Transition and Termination: Among the many parameters in the energy consumption model of a water-cooled unit, the evaporator outlet water temperature (EOWT) is significantly affected by the condenser water temperature (CWOT) and is a crucial reference parameter for the safe operation of the water-cooled unit. We constructed an EOWT prediction model using an MLP, where the inputs are the 15 input parameters of the water-cooled unit energy consumption model excluding EOWT, and the output is the specific value of EOWT. After each adjustment of CWOT based on the policy, the EOWT prediction model is used to update the value of EOWT. For the termination condition, we limit the maximum number of exploration attempts for the agent to 20 per round. Through algorithm training, we can usually find an optimal solution that meets our expectations within 20 exploration attempts.

[0051] Reward: Reward r t It consists of two parts:

[0052] r t =r dec,t +r penal,t

[0053] Where, r dec,t After performing the action, from the current state s t Transition to the next state s t+1 The reward for the achieved energy reduction, r penal,t Indicates the next state s t+1Penalty for exceeding the safety boundary in EOWT.

[0054] We denote the normalized energy consumption value of the water-cooled unit as P, which can be obtained by calling the water-cooled unit energy consumption model N1:

[0055] P = N1(s)

[0056] r dec,t This can be further expressed as:

[0057]

[0058] Here, λ1 is a pricing factor greater than 1, so the cost of ineffective actions is significantly greater than the benefit of effective actions. This difference ensures that the agent's exploration is more focused and direct.

[0059] Considering the actual operating conditions of water-cooled units, we represent the lower and upper bounds of EOWT as follows: and We created a 15x1 dimensional array s p Let represent the input to the EOWT prediction model. Based on the EOWT prediction model N2 and the boundary conditions, the penalty term is defined as follows:

[0060]

[0061] Where λ2 is the penalty pricing factor.

[0062] After establishing the energy consumption model and optimization model for the water-cooled unit, this embodiment of the invention also designs an energy consumption optimization algorithm for the water-cooled unit based on DQN. In DQN, a parameter ω is introduced, and a neural network is used to fit the action value function Q(s, a):

[0063] f(s, a; ω) = Q(s, a)

[0064] The update method for Q(s, a) is as follows:

[0065] f(s t a t ;ω)←f(s t a t )+α[f Target -f(s t a t ;ω)]

[0066] Where α is the learning rate of the neural network. For the time-difference TD objective, R t Let γ be the reward the agent receives at time t, and γ be the discount factor.

[0067] Q-Net takes a vector of 16 input parameters from a water-cooled unit energy consumption model as input and outputs the action value Q(s, a) corresponding to any action taken in state s. The specific network structure is as follows: Figure 3 As shown, the relevant training parameter settings include: Adam optimization algorithm, ReLU activation function, learning rate of 2e-3, batch size of 128, and discount factor γ of 0.95. It is important to note that including a value function in the TD objective means that the feature values ​​include the output of Q-Net itself. During training, the feature values ​​change with the Q-Net parameters, which will lead to instability in the training process and results. To address this issue, we further introduce two networks: a training network Q-Net(ω) and a target network Q-Net(ω). - Q-Net(ω) is used to calculate f(s). t a t The ω) term updates the parameters using standard gradient descent, while Q-Net(ω) - ) used for calculation The term uses older neural network parameters compared to Q-Net(ω).

[0068] The steps of the DQN-based energy consumption optimization algorithm for water-cooled units used in this embodiment of the invention are as follows:

[0069] Step 1: Initialize Q-Net(ω) with random network parameters;

[0070] Step 2: Copy the same parameter ω - ←ω to initialize the target network Q-Net(ω) - );

[0071] Step 3: Initialize the experience replay pool R;

[0072] Step 4: For each training round, first reset the environment and obtain the initial environment s0; then for each time step, perform the following loop: Based on the current Q-Net(ω), the agent executes action a using a greedy policy. t (Adjusting CWOT); Based on the setting of the Markov decision process environment, obtain the corresponding reward r. t The agent enters the next state s t+1 ; Put (s t a t r t s t+1 This quadruple is placed into the experience replay pool R; when the amount of data in R reaches the critical capacity, N quadruples are randomly sampled from it, according to L(ω)=E[(R t +γmax a f(s t+1 , a; ω- )-f(s t a t ;ω)) 2 Calculate the target loss, and based on Update ω; update Q-Net(ω) using the parameters of Q-Net(ω). - The parameters of ).

[0073] Simulation experiment setup:

[0074] This invention uses simulation to evaluate the effectiveness and superiority of the proposed model and method. First, the parameter values ​​at a certain moment are selected as the initial state vector: [3.14000010e+00,1.46199999e+01,3.35000000e+01,3.06000000e+01,3.70000005e-01,1.00250000e+02,8.43999958e+00,3.31399994e+01,3.0709 [9991e+01,5.60000000e+01,3.65000000e+02,1.74000000e+01,7.64000000e+02,3.41000000e+01,1.76000000e+01,1.50000000e+01], where the initial value of CWOT is 33.14℃ and the initial value of EOWT is 17.60℃. Furthermore, the settings are... It was 28.60℃. It was 36.30℃. It is 0.01℃. The temperature is -0.35℃. It is 0.35℃. It was 17.1℃. The temperature is 17.7℃, λ1 is 10, and λ2 is 5.

[0075] We trained the energy consumption optimization algorithm for water-cooled units based on DQN proposed in this invention, and first selected an algorithm based on random actions as a reference. Figure 4 In the diagram, we obtain the iterative process of the rewards obtained by the two algorithms with each training epoch. It can be seen that the DQN-based optimization algorithm converges after approximately 100 training epochs, and its reward is significantly higher than that of the random action algorithm. Figure 5 In the diagram, we obtained the variation of the percentage energy reduction brought by the two algorithms with the number of training rounds. Compared with the random action algorithm, the DQN-based optimization algorithm puts the system in a lower energy consumption state than the initial state in most training rounds, and after about 100 training rounds, the algorithm converges and can stabilize the percentage energy reduction at around 77%.

[0076] The two algorithms mentioned above are both policy-based optimization algorithms. We also consider selecting three classic global optimization algorithms—simulated annealing, Bayesian optimization, and genetic algorithm—as a reference group to verify the effectiveness and superiority of the DQN-based optimization algorithm proposed in this invention from another perspective. Figure 6 and Figure 7 The graphs show a comparison of the energy consumption reduction percentage and the number of iterations required for convergence between the DQN-based optimization algorithm and three traditional global optimization algorithms. It can be seen that the DQN-based energy consumption optimization algorithm for water-cooled units proposed in this invention requires fewer convergence iterations while achieving a higher energy consumption reduction percentage. Furthermore, due to the strong generalization ability of DQN and the extremely powerful parallel computing capabilities provided by modern computers for neural network training, when the optimization algorithm proposed in this invention is applied to more complex data center energy consumption optimization scenarios, it will demonstrate significantly greater superiority over other traditional optimization algorithms.

[0077] Based on the same inventive concept, this invention discloses a data center water-cooled unit energy consumption optimization system based on DQN, comprising: a data acquisition module for acquiring measurement point data of the water-cooled unit equipment using IoT technology; a water-cooled unit energy consumption modeling module for constructing a water-cooled unit energy consumption model based on a neural network according to the acquired equipment measurement point data; an EOWT prediction modeling module for constructing an EOWT prediction model based on a neural network according to the acquired equipment measurement point data; and an energy consumption optimization module for modeling the water-cooled unit energy consumption optimization process as a Markov decision process, constructing a water-cooled unit energy consumption optimization model, and implementing water-cooled unit energy consumption optimization based on DQN to reduce water-cooled unit energy consumption, thereby achieving the effect of reducing data center PUE. Specific implementation details of the modules are described in the above method embodiments and will not be repeated here.

[0078] Based on the same inventive concept, an embodiment of the present invention discloses a computer system, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the computer program is loaded onto the processor, it implements the steps of the DQN-based data center water-cooled unit energy consumption optimization method.

[0079] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for optimizing energy consumption of data center water-cooled units based on DQN, characterized in that, Includes the following steps: Data from measurement points of water-cooled unit equipment is acquired using Internet of Things (IoT) technology. Based on the acquired equipment measurement data, a water-cooled unit energy consumption model based on neural networks is constructed. The input parameters of the water-cooled unit energy consumption model are the key equipment measurement data, and the output is the normalized power value of the water-cooled unit. The energy consumption optimization process of water-cooled units is modeled as a Markov decision process, constructing an energy consumption optimization model for water-cooled units. Based on Deep Quantity Neural Network (DQN), energy consumption optimization is implemented to reduce water-cooled unit energy consumption, thereby reducing the power efficiency (PUE) of the data center. In the energy consumption optimization model, the state includes the values ​​of the input parameters at specific time points, the action is the adjustment of the cooling water outlet temperature (CWOT), the action space is the adjustable range of CWOT each time, and boundary conditions are set for the evaporator outlet temperature (EOWT) of the water-cooled unit. Rewards include the energy efficiency reduction achieved by transitioning to the next state and the penalty for exceeding the safety boundary in the next state. The EOWT is predicted by a neural network-based EOWT prediction model. The input parameters of the EOWT prediction model are the input parameters of the water-cooled unit energy consumption model excluding EOWT, and the output is the specific value of EOWT. A data-driven approach is used, combining a multilayer perceptron (MLP) in deep neural networks to construct the water-cooled unit energy consumption model and the EOWT prediction model. Rewards for reduced energy consumption Represented as: ； in, The energy consumption values ​​for the current state and the next state are determined by calling the energy consumption model of the water-cooled unit. get, It is a pricing factor greater than 1; Penalty items Defined as: ； in, and These represent the lower and upper bounds of EOWT, respectively. To penalize pricing factors, The EOWT value, representing the next state, is obtained by calling the EOWT prediction model. This represents the model input.

2. The energy consumption optimization method for data center water-cooled units based on DQN according to claim 1, characterized in that: Considering the state space of CWOT, if the action chosen by the intelligent decision will cause CWOT to go beyond its state space, then the next state is set to the nearest CWOT boundary value.

3. The energy consumption optimization method for data center water-cooled units based on DQN according to claim 1, characterized in that: The key equipment measurement data include: chilled water supply pressure, chilled water supply temperature, condenser inlet water temperature, condenser outlet water temperature, evaporator inlet water temperature, evaporator outlet water temperature, cooling water inlet water temperature, cooling water outlet water temperature, cooling water outlet water pressure, compressor suction pressure, compressor suction temperature, compressor discharge pressure, compressor discharge temperature, water-side electric regulating valve opening feedback, refrigeration electric regulating valve opening feedback, and chiller operating percentage.

4. The energy consumption optimization method for data center water-cooled units based on DQN according to claim 1, characterized in that: The reward in the energy consumption optimization model of the water-cooled unit It consists of two parts: ； in, After performing an action, from the current state Transition to the next state The reward for the achieved energy reduction, Indicates the next state Penalty for exceeding the safety boundary in EOWT.

5. The energy consumption optimization method for data center water-cooled units based on DQN according to claim 1, characterized in that: The Q-Net in the DQN algorithm is designed based on a deep neural network (DNN). The state vector composed of measurement point data of key equipment is used as the input of the Q-Net, and the corresponding action value Q(s, a) of taking action a in each state is output. A training network and a target network are introduced to train the parameters of the Q-Net.

6. A data center water-cooled unit energy consumption optimization system based on DQN, used to implement the data center water-cooled unit energy consumption optimization method based on DQN according to any one of claims 1-5, characterized in that, include: The data acquisition module is used to acquire measurement point data of water-cooled unit equipment based on Internet of Things (IoT) technology. The water-cooled unit energy consumption modeling module is used to construct a water-cooled unit energy consumption model based on neural networks based on the acquired equipment measurement point data; the input parameters of the water-cooled unit energy consumption model are the key equipment measurement point data, and the output is the normalized value of the water-cooled unit power. The EOWT prediction modeling module is used to construct an EOWT prediction model based on neural networks based on the acquired equipment measurement point data. The input parameters of the EOWT prediction model are the input parameters of the water-cooled unit energy consumption model that does not include EOWT, and the output is the specific value of EOWT. The energy consumption optimization module is used to model the energy consumption optimization process of water-cooled units as a Markov decision process, construct an energy consumption optimization model for water-cooled units, and implement energy consumption optimization of water-cooled units based on DQN to reduce energy consumption of water-cooled units, thereby achieving the effect of reducing data center PUE. The state in the energy consumption optimization model of water-cooled units includes the value of the input parameters of the energy consumption model of water-cooled units at a specific time point, the action is the adjustment amount of cooling water outlet temperature CWOT, the action space is the adjustable range of CWOT each time, and boundary conditions are set for the evaporator outlet temperature EOWT of water-cooled units. The rewards include the energy efficiency reduction achieved by transitioning to the next state and the penalty for exceeding the safety boundary in the next state.

7. A computer system comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the computer program is loaded into the processor, it implements the steps of the DQN-based data center water-cooled unit energy consumption optimization method according to any one of claims 1-5.

Images