A method for intelligent energy-saving control of water-cooled air conditioners in computer rooms based on deep reinforcement learning

Abstract

The present invention relates to an energy-saving intelligent control method for water-cooled air conditioners in computer rooms based on deep reinforcement learning, and belongs to the technical field of energy-saving control of air conditioners. The method comprises: obtaining air conditioner operation data, environmental information data, and spatial information data, and establishing a geometric model according to the spatial information data; configuring the attributes of the geometric model and generating a digital grid diagram; defining a global control quantity through environmental information data, judging the validity of the geometric model according to the global control quantity, and visualizing the model that meets the validity to obtain a model display result diagram; optimizing and adjusting the air conditioner terminal demand parameters according to the model display result diagram and constructing a deep reinforcement learning controller; the deep reinforcement learning controller uses the lowest energy consumption as the optimization goal to mine the lowest energy consumption control strategy to obtain control parameters, and the air conditioning system automatically sets the operating parameters according to the control parameters. The intelligent start and stop of multiple host air conditioners in the computer room is realized, and the local hot spot problems existing in the computer room or data center are optimized.

Description

Technical Field

[0001] The present invention belongs to the field of air conditioning energy-saving control technology, and specifically relates to a water-cooled air conditioning energy-saving intelligent control method for a computer room based on deep reinforcement learning. Background Art

[0002] With the rapid development of modern information technology, computer rooms and data centers, as critical infrastructure supporting cloud computing and big data processing, are becoming increasingly large in scale and operational requirements. However, with increasing equipment density and workloads, heat generation within computer rooms has significantly increased, becoming a pressing issue. High temperatures in computer rooms not only threaten equipment stability and performance but can also lead to hotspots—localized temperature increases that affect equipment performance. Thermal management plays a crucial role in computer room operations and maintenance. One of its core challenges is how to efficiently dissipate heat to maintain equipment operating within a suitable temperature range. While widely used in data centers, traditional air conditioning systems are relatively inefficient, require extensive management methods, and waste energy. Furthermore, because the cooling load characteristics and temperature requirements of computer rooms differ significantly from those of traditional air conditioning systems used in residential buildings, management methods used for residential air conditioning are not applicable within computer rooms. Therefore, optimizing thermal management systems within computer rooms to improve energy efficiency and reduce energy consumption has become a key research focus.

[0003] The existing energy consumption management problems of computer rooms are mainly reflected in the following aspects: (1) The air conditioning setting temperature is too low and lasts too long, which exceeds the heat dissipation and heating requirements, resulting in energy waste; the air conditioning supply temperature and wind speed cannot be well matched, and the operating parameters are often set according to the design value, but each project site is different. There is a most energy-saving matching combination of supply temperature and wind speed to achieve the same heat dissipation, and traditional algorithms cannot give the optimal operating point; (2) Computer rooms or data centers may have local hot spots due to uneven loads. The traditional method based on indoor temperature feedback not only has a slow feedback speed, but also does not specifically improve the air flow conditions of local hot spots, resulting in excessive and non-targeted air conditioning heat dissipation energy consumption; air conditioning units have multiple operating conditions under the same cooling load, and the settings of different operating conditions and operating parameters will also have different energy consumption due to the different high-efficiency operating points of each unit. The traditional air conditioning management method does not perform dynamic intelligent regulation of the chiller. The chiller needs to be optimized accordingly for the dynamically changing cooling load, which cannot be achieved by traditional control methods. Summary of the Invention

[0004] In order to solve the above problems existing in the prior art, the present invention provides an energy-saving intelligent control method for water-cooled air conditioners in computer rooms based on deep reinforcement learning;

[0005] The purpose of the present invention can be achieved through the following technical solutions:

[0006] S1: Acquire air conditioning operation data, environmental information data, and spatial information data; establish a geometric model based on the spatial information data and equipment parameters of the computer room;

[0007] S2: adding material and size information to the blocks in the geometric model through a property editor, generating a mesh for the geometric model according to the material and size information, and obtaining a digital mesh map based on the geometric model;

[0008] S3: defining a global control quantity through the environmental information data, calculating the convergence degree of the corresponding geometric model according to the global control quantity in the digital grid diagram, presetting a model validity determination threshold, and if the convergence degree is greater than the model validity determination threshold, executing the step of S2 to regenerate the digital grid diagram; if the convergence degree is less than the model validity determination threshold, visualizing the collective model to obtain a model display result diagram;

[0009] S4: extracting local hot spots and airflow dead spots in the model display result graph, optimizing and adjusting the air conditioning terminal demand parameters in the air conditioning operation data according to the local hot spots and airflow dead spots, using the optimized air conditioning terminal demand parameters as air conditioning terminal boundary constraint requirements and constructing a deep reinforcement learning controller;

[0010] S5: The deep reinforcement learning controller mines the lowest energy consumption control strategy with the lowest energy consumption as the optimization goal to obtain control parameters, and the air-conditioning system automatically sets the operating parameters according to the control parameters.

[0011] Specifically, the blocks in the geometric model include a model calculation area, a heat source, a heat exchanger, a wall, a filter, and a fan; the model calculation area is a virtual three-dimensional grid space surrounded by the simulation calculation boundary of the computer room; the heat source includes the server, network equipment, storage equipment, generator set, and computer room lighting equipment in the computer room; the heat exchanger is the evaporator and condenser in the water-cooled air-conditioning system of the computer room; the wall is an isolation block of the simulation calculation boundary line of the computer room; the filter and the fan are components in the water-cooled air-conditioning system that drives the cooling cycle.

[0012] Specifically, the global control quantity includes heat dissipation mode, flow state, radiation, and number of iterations. The flow state form is obtained according to the Reynolds number and the Peclet number, and specifically includes laminar and turbulent forms. The radiation mode specifically includes electromagnetic radiation and infrared radiation. The heat dissipation mode specifically includes natural convection heat dissipation and forced convection heat dissipation. The natural convection heat dissipation is the heat transfer caused by the spontaneous flow of air or fluid caused by temperature difference. The forced convection heat dissipation is the heat transfer by forcing the flow of fluid through air-conditioning equipment. The number of iterations is set to 300 times.

[0013] Specifically, the deep reinforcement learning controller uses the unit data collected by energy management as the environmental state, the air-conditioning terminal demand parameters as the air-conditioning terminal boundary constraint requirements, and the adjustable parameter set as the control action. By setting the control action of reducing energy consumption as the reward and the ambient temperature increase caused by the control action as the penalty, an automatic control strategy is generated. The approximate action value function of the control action is:

[0014]

[0015] Among them, s is the current state, a is the current control action, s1 is the next state, a1 is the optimized control action, Q(s,a) is the value of executing action a in state s, r is the immediate reward obtained after executing action a, p(s1,r|s,a) is the state transition probability, which is the probability of transferring to the next state s1 and obtaining reward r after executing action a in the current state s, γ is the discount factor, which indicates the importance of future rewards, max a1 Q(s1,a1): The maximum value of the action-value function of selecting the optimal action a1 in the next state s1.

[0016] Specifically, the method of the deep reinforcement learning controller to explore the lowest energy consumption control strategy with the lowest energy consumption as the optimization goal is: based on the greedy algorithm, by setting the exploration rate, the selected optimal action and historical information are balanced for training. The selected optimal action is: at = argmaxaQ(st,a);, where a t is the optimal action, argmax a To find the parameter function, Q(st,a;w) is the action-value function;

[0017] The updating method of the exploration rate in the balanced training is:

[0018]

[0019] Among them, ε is the exploration rate, ε min is the minimum value of the exploration rate during the iteration, ε max is the maximum value of the exploration rate during the iteration process, εd is the decay rate, e is a natural number, and n is the number of iterations.

[0020] Specifically, the air conditioning terminal demand parameters include air outlet speed, air outlet temperature, air outlet angle, cooling capacity demand, and air flow demand.

[0021] The beneficial effects of the present invention are:

[0022] (1) Aiming at the local hot spot problems caused by unreasonable temperature setting, high energy consumption and uneven load of the air conditioner in the computer room, the combination of simulation and deep reinforcement learning is used to optimize the air conditioner source measurement unit combination and air flow organization. The optimization results realize the intelligent start and stop of multiple host air conditioners in the computer room, such as cyclic start and stop, which prolongs the service life of the air conditioner; at the same time, high energy efficiency control is achieved in the matching of supply air temperature and wind speed and the setting of cooling machine parameters, which significantly improves the energy efficiency of equipment and system and achieves energy saving; improves air flow organization and optimizes the local hot spot problems in the computer room or data center, which has a high reference significance for the exploration of energy-saving optimization mode of environmental control system of computer room and data center, and realizes intelligent energy saving in the operation and maintenance stage of data center.

[0023] (2) By writing the control instructions corresponding to the minimum energy consumption control strategy mined by the deep reinforcement learning controller into the on-site BA system or DDC box, the air-conditioning system automatically sets the parameters according to the instructions. With the feedback of a large amount of monitoring data, the controller automatically refines the operating experience, realizes the refined operation of the air-conditioning system, and improves the energy efficiency of the computer room. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] To facilitate understanding by those skilled in the art, the present invention is further described below with reference to the accompanying drawings.

[0025] Figure 1 This is a flow chart of a method for intelligent energy-saving control of water-cooled air conditioners in computer rooms based on deep reinforcement learning according to the present invention. DETAILED DESCRIPTION

[0026] In order to further illustrate the technical means and effects adopted by the present invention to achieve the predetermined purpose of the invention, the specific implementation methods, structures, features and effects of the present invention are described in detail below in conjunction with the accompanying drawings and preferred embodiments.

[0027] See also Figure 1 , a method for intelligent energy-saving control of water-cooled air conditioners in computer rooms based on deep reinforcement learning:

[0028] S1: Acquire air conditioning operation data, environmental information data, and spatial information data; establish a geometric model based on the spatial information data and equipment parameters of the computer room;

[0029] S2: adding material and size information to the blocks in the geometric model through a property editor, generating a mesh for the geometric model according to the material and size information, and obtaining a digital mesh map based on the geometric model;

[0030] S3: defining a global control quantity through the environmental information data, calculating the convergence degree of the corresponding geometric model according to the global control quantity in the digital grid diagram, presetting a model validity determination threshold, and if the convergence degree is greater than the model validity determination threshold, executing the step of S2 to regenerate the digital grid diagram; if the convergence degree is less than the model validity determination threshold, visualizing the collective model to obtain a model display result diagram;

[0031] S4: extracting local hot spots and airflow dead spots in the model display result graph, optimizing and adjusting the air conditioning terminal demand parameters in the air conditioning operation data according to the local hot spots and airflow dead spots, using the optimized air conditioning terminal demand parameters as air conditioning terminal boundary constraint requirements and constructing a deep reinforcement learning controller;

[0032] S5: The deep reinforcement learning controller mines the lowest energy consumption control strategy with the lowest energy consumption as the optimization goal to obtain control parameters, and the air-conditioning system automatically sets the operating parameters according to the control parameters.

[0033] In this embodiment, a geometric model is established in the CFD software, and two forms of CAD import and manual model building are used to construct the geometric model of the computer room, including walls, floors, ceilings, cabinets, air-conditioning equipment, etc. The geometric modeling tool in ICEPAK is used to draw the model of the computer room to ensure that the geometric information is complete and accurate. Object properties: Set corresponding properties for each object in the computer room (such as air-conditioning equipment, cabinets, walls, etc.), including material properties, heat dissipation coefficient, surface roughness, etc., in order to analyze heat conduction and convection heat transfer. The operating status parameters of the air-conditioning equipment, cabinets and other equipment in the computer room are input into the model, such as the wind speed and temperature setting value of the air-conditioning equipment, so as to perform simulation analysis of thermal management and air flow. According to the actual situation, the boundary conditions of the computer room are set, including air inlets and outlets, heat dissipation vents, etc., as well as the heat transfer boundary conditions between the computer room and the external environment. The model display result diagram includes the temperature cloud map, wind speed map, isothermal surface, and constant wind speed surface in the computer room.

[0034] Simulation analysis: After the model is built, ICEPAK is used to perform simulation analysis, including analysis of air flow, temperature distribution, heat conduction, etc., to evaluate the thermal management effect and air flow conditions of the computer room.

[0035] Specifically, the blocks in the geometric model include a model calculation area, a heat source, a heat exchanger, a wall, a filter, and a fan; the model calculation area is a virtual three-dimensional grid space surrounded by the simulation calculation boundary of the computer room; the heat source includes the server, network equipment, storage equipment, generator set, and computer room lighting equipment in the computer room; the heat exchanger is the evaporator and condenser in the water-cooled air-conditioning system of the computer room; the wall is an isolation block of the simulation calculation boundary line of the computer room; the filter and the fan are components in the water-cooled air-conditioning system that drives the cooling cycle.

[0036] Specifically, the global control quantity includes heat dissipation mode, flow state, radiation, and number of iterations. The flow state form is obtained according to the Reynolds number and the Peclet number, and specifically includes laminar and turbulent forms. The radiation mode specifically includes electromagnetic radiation and infrared radiation. The heat dissipation mode specifically includes natural convection heat dissipation and forced convection heat dissipation. The natural convection heat dissipation is the heat transfer caused by the spontaneous flow of air or fluid caused by temperature difference. The forced convection heat dissipation is the heat transfer by forcing the flow of fluid through air-conditioning equipment. The number of iterations is set to 300 times.

[0037] Specifically, the deep reinforcement learning controller uses the unit data collected by energy management as the environmental state, the air-conditioning terminal demand parameters as the air-conditioning terminal boundary constraint requirements, and the adjustable parameter set as the control action. By setting the control action of reducing energy consumption as the reward and the ambient temperature increase caused by the control action as the penalty, an automatic control strategy is generated. The approximate action value function of the control action is:

[0038]

[0039] Where s is the current state, a is the current control action, s1 is the next state, a1 is the optimized control action, Q(s,a) is the value of executing action a in state s, r is the immediate reward obtained after executing action a, p(s1,r|s,a) is the state transition probability, which is the probability of transferring to the next state s1 and obtaining reward r after executing action a in the current state s, γ is the discount factor, which indicates the importance attached to future rewards, and maxa1Q(s1,a1): the maximum value of the action value function of selecting the optimal action a1 in the next state s1.

[0040] In this embodiment, the neural network input of the deep reinforcement learning controller is the state, and the output is the value of each action. A fully connected network is used to express the relationship between action and state. Random and arbitrary behavioral strategies are used to collect experience and explore more unprecedented states. The target strategy is updated using multiple experiences to ensure that the control logic reaches the global optimum. The ε-greedy algorithm is used for exploration. The intelligent agent gradually stabilizes in the later stage, explores and adopts the lowest energy consumption control, and automatically realizes round-robin start and stop and high-energy consumption operation of each equipment.

[0041] Specifically, the method of the deep reinforcement learning controller to explore the lowest energy consumption control strategy with the lowest energy consumption as the optimization goal is: based on the greedy algorithm, by setting the exploration rate, the selected optimal action and historical information are balanced for training. The selected optimal action is: att = argmaxaQ(st,a:U), where a t is the optimal action, argmax a To find the parameter function, Q(st,a;w) is the action-value function;

[0042] The updating method of the exploration rate in the balanced training is:

[0043]

[0044] Among them, ε is the exploration rate, ε min is the minimum value of the exploration rate during the iteration, ε max is the maximum value of the exploration rate during the iteration, ε d is the decay rate, e is a natural number, and n is the number of iterations.

[0045] Specifically, the air conditioning terminal demand parameters include air outlet speed, air outlet temperature, air outlet angle, cooling capacity demand, and air flow demand.

[0046] The computer storage medium of the embodiment of the present invention may adopt any combination of one or more computer-readable media. The computer-readable medium may be a computer-readable signal medium or a computer-readable storage medium. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or component, or any combination thereof. More specific examples (a non-exhaustive list) of computer-readable storage media include: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In this document, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, device or device.

[0047] A computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, which carries computer-readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium that can transmit, propagate, or transport a program for use by or in conjunction with an instruction execution system, apparatus, or device.

[0048] The program code included in the computer-readable medium can be transmitted with any appropriate medium, including but not limited to wireless, electric wire, optical cable, RF or the like, or any suitable combination thereof. The computer program code for performing the operation of the present invention can be written in one or more programming languages ​​or a combination thereof, and the programming language includes an object-oriented programming language such as Java, Smalltalk, C++, and also includes a conventional procedural programming language such as "C" language or similar programming language. The program code can be executed completely on the user's computer, partially on the user's computer, as an independent software package, partially on the user's computer and partially on a remote computer, or completely on a remote computer or server. In the case of a remote computer, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computer (for example, utilizing an Internet service provider to connect through the Internet).

[0049] The above description is merely a preferred embodiment of the present invention and does not constitute any form of limitation to the present invention. Although the present invention has been disclosed as above in terms of a preferred embodiment, it is not intended to limit the present invention. Any person skilled in the art can, without departing from the scope of the technical solution of the present invention, make some changes or modifications to equivalent embodiments using the technical contents disclosed above. However, any brief modifications, equivalent changes and modifications made to the above embodiments based on the technical essence of the present invention without departing from the content of the technical solution of the present invention are still within the scope of the technical solution of the present invention.

Claims

1. A method for intelligent energy-saving control of water-cooled air conditioners in computer rooms based on deep reinforcement learning, characterized in that: include: S1: Acquire air conditioning operation data, environmental information data, and space information data; Establishing a geometric model based on the spatial information data and the parameters of the equipment room; S2: adding material and size information to the blocks in the geometric model through a property editor, generating a mesh for the geometric model according to the material and size information, and obtaining a digital mesh map based on the geometric model; S3: defining a global control quantity using the environmental information data, calculating the convergence of a corresponding geometric model in the digital grid map based on the global control quantity, and presetting a model validity determination threshold. If the convergence is greater than the model validity determination threshold, executing the step S2 to regenerate the digital grid map. If the convergence is less than the model validity determination threshold, visualizing the geometric model to obtain a model display result map. S4: extracting local hot spots and airflow dead spots in the model display result graph, optimizing and adjusting the air conditioning terminal demand parameters in the air conditioning operation data according to the local hot spots and airflow dead spots, using the optimized air conditioning terminal demand parameters as air conditioning terminal boundary constraint requirements and constructing a deep reinforcement learning controller; The global control variables include heat dissipation mode, flow state, radiation, and number of iterations. The flow state is obtained according to the Reynolds number and the Peclet number, and specifically includes laminar and turbulent forms. The radiation specifically includes electromagnetic radiation and infrared radiation. The heat dissipation mode specifically includes natural convection heat dissipation and forced convection heat dissipation. Natural convection heat dissipation is heat transfer caused by the spontaneous flow of air or fluid caused by temperature difference. Forced convection heat dissipation is heat transfer caused by forced flow of fluid through air conditioning equipment. The number of iterations is set to 300 times. The deep reinforcement learning controller uses the unit data collected by energy management as the environmental state, the air-conditioning terminal demand parameters as the air-conditioning terminal boundary constraint requirements, and the adjustable parameter set as the control action. By setting the control action of reducing energy consumption as the reward and the increase in ambient temperature caused by the control action as the penalty, an automatic control strategy is generated. The approximate action value function of the control action is: ， in, s is the current state, a is the current control action, s 1 is the next state, a 1 is the optimized control action, Q ( s , a ) for execution s Status a The value of the action, r To perform an action a After receiving the instant reward, p ( s 1, r ǀ s , a ) is the state transition probability, in the current state s Execute an action a Then transfer to the next state s 1 and get rewards r The probability of γ is the discount factor, which indicates the importance of future rewards. max a1 Q ( s 1, a 1): Next state s 1. Choose the best action a The maximum value of the action-value function of 1; S5: The deep reinforcement learning controller mines the lowest energy consumption control strategy with the lowest energy consumption as the optimization goal to obtain control parameters, and the air-conditioning system automatically sets the operating parameters according to the control parameters.

2. The method according to claim 1, characterized in that The blocks in the geometric model include a model calculation area, a heat source, a heat exchanger, a wall, a filter, and a fan; the model calculation area is a virtual three-dimensional grid space surrounded by the simulation calculation boundary of the computer room; the heat source includes the server, network equipment, storage equipment, generator set, and computer room lighting equipment in the computer room; the heat exchanger is the evaporator and condenser in the water-cooled air-conditioning system of the computer room; the wall is an isolation block of the simulation calculation boundary line of the computer room; the filter and the fan are components in the water-cooled air-conditioning system that drives the cooling cycle.

3. The method according to claim 1, characterized in that The method of the deep reinforcement learning controller to explore the lowest energy consumption control strategy with the lowest energy consumption as the optimization goal is: based on the greedy algorithm, by setting the exploration rate, the selected optimal action and historical information are balanced for training. The selected optimal action is: ,in, a t For the optimal action, argmax a To find the parameter function, Q ( s t , a ; w ) is the action-value function; The updating method of the exploration rate in the balanced training is: ， in, ε is the exploration rate, ε min is the minimum value of the exploration rate during the iteration process, ε max is the maximum value of the exploration rate during the iteration process, ε d is the attenuation rate, e is a natural number, n is the number of iterations.

4. The method according to claim 1, wherein The air conditioning terminal demand parameters include air outlet speed, air outlet temperature, air outlet angle, cooling capacity demand, and air flow demand.

Images