Cooling system control methods, devices, equipment, media and procedures products

By generating system feature vectors and using a deep reinforcement learning model to determine target control actions, the coupling problem between subsystems in the air-liquid co-source cooling system was solved, thereby improving the energy utilization rate.

CN123124277APending Publication Date: 2026-11-17CHINA UNITED NETWORK COMM GRP CO LTD +1
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202611074531.4
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2026-07-17
Publication Date
2026-11-17

AI Technical Summary

Technical Problem

Traditional PID algorithm-based distributed independent control schemes cannot decouple the hydraulic and thermal coupling between subsystems in a co-source cooling system, resulting in low energy utilization.

Method used

Based on outdoor dry-bulb temperature data, equipment power information, and cooling system operation data, a system feature vector is generated. The target control action is determined through a deep reinforcement learning model, and a unified control command is generated to coordinate the adjustment of various functional components of the cooling system.

Benefits of technology

It achieves global coordinated control of the air-liquid co-source cooling system, breaks the mutual constraints between subsystems, and improves the energy utilization rate.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN123124277A_ABST
    Figure CN123124277A_ABST
Patent Text Reader

Abstract

This application provides a control method, apparatus, equipment, medium, and program product for a cooling system, relating to the field of data processing technology, for improving the collaborative control efficiency of a co-current cooling system. The method includes: generating a system feature vector based on outdoor dry-bulb temperature data, equipment power information of the data center where the co-current cooling system is located, and operating data of the co-current hybrid cooling system; based on the system feature vector, determining at least one target control action matching the system feature vector from a set of candidate control actions to generate a control action sequence; the control action sequence includes at least one target control action; and based on the control action sequence, sequentially generating control instructions corresponding to each target control action, the control instructions being used to instruct the corresponding functional components in the co-current cooling system to execute the corresponding control action. This application is applied to cooling system control scenarios.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This application relates to the field of data processing technology, and in particular to a control method, apparatus, equipment, medium, and program product for a cooling system. Background Technology

[0002] Driven by the rapid iteration of artificial intelligence and high-performance computing technologies, the computing power density of data center computing equipment continues to rise. The power density of a single rack has broken through the limits of traditional air cooling, fully entering a new era of ultra-high-density cooling of 100kW~200kW. Traditional cooling methods can no longer meet the stable heat dissipation requirements of high-density computing equipment, and the industry is gradually adopting a variety of new cooling systems to adapt to this development need.

[0003] To address the shortcomings of traditional single-air cooling modes, such as insufficient heat dissipation capacity, low heat exchange efficiency, and uneven temperature and humidity control in data centers, the industry is promoting the application of a hybrid air-liquid cooling system that combines cold-plate liquid cooling with precision air conditioning. This system shares an outdoor cold source and primary-side water supply network between the liquid cooling distribution unit (CDU) and the precision air conditioner (PAC), thus integrating the high-efficiency heat dissipation of liquid cooling with the flexible supplementary cooling of air cooling, effectively overcoming the heat dissipation bottleneck of high-density data centers. Currently, the industry generally uses a distributed independent control scheme based on traditional PID algorithms for the control and regulation of hybrid air-liquid cooling systems. This scheme decomposes the hybrid air-liquid cooling system into two independent units: a CDU liquid cooling subsystem and a PAC air cooling subsystem. Each subsystem is equipped with a dedicated PID controller, employing a single-input single-output (SISO) local feedback regulation mode.

[0004] However, when using the aforementioned distributed independent control scheme based on the traditional PID algorithm, the CDU and PAC subsystems share the same outdoor cold source and primary water supply network, resulting in natural hydraulic and thermal coupling between the two subsystems. Furthermore, this coupling exhibits significant nonlinear characteristics as operating conditions such as heat load and cold source temperature change. Traditional PID controllers cannot decouple these nonlinear characteristics, leading to mutual constraints on the independent regulation of each subsystem and consequently, low overall power utilization of the data center. Summary of the Invention

[0005] This application provides a control method, device, equipment, medium, and program product for a cooling system, which enables global collaborative decision-making among multiple subsystems in a co-source air-liquid cooling system, eliminates mutual obstruction between subsystems, achieves optimal global allocation of cooling capacity, and improves the power utilization rate of data centers.

[0006] In a first aspect, embodiments of this application provide a method for controlling a cooling system, the method comprising:

[0007] Based on outdoor dry-bulb temperature data, equipment power information of the data center where the air-liquid co-source cooling system is located, and operating data of the air-liquid co-source hybrid cooling system, a system feature vector is generated; the above operating data includes relevant data of each functional component in the above air-liquid co-source hybrid cooling system.

[0008] Based on the aforementioned system feature vector, at least one target control action matching the aforementioned system feature vector is determined from the candidate control action set to generate a control action sequence; the aforementioned candidate control action set includes at least one candidate control action, and the aforementioned candidate control action set is obtained based on the parameter range for each functional component in the aforementioned air-liquid homogeneous cooling system; the aforementioned control action sequence includes at least one target control action.

[0009] Based on the above control action sequence, control instructions corresponding to each target control action are generated sequentially. These control instructions are used to instruct the corresponding functional components in the above-mentioned air-liquid co-source cooling system to perform the corresponding control actions.

[0010] The technical solution provided in this application brings at least the following beneficial effects: Based on outdoor dry-bulb temperature data, data center equipment power information, and relevant operating data of each functional component in the air-liquid co-source hybrid cooling system, a system feature vector is generated. The nonlinear characteristics that change with heat load, cold source temperature, and other operating conditions, as well as the operating status of each functional component, are uniformly incorporated into the feature representation. Subsequently, based on this system feature vector, a matching target control action is determined from the candidate control action set obtained based on the parameter range of each functional component to generate a control action sequence. Control commands are then generated sequentially to instruct the corresponding functional components in the air-liquid co-source cooling system to execute the corresponding control actions. Since the determination of the target control action directly depends on the unified system feature vector reflecting the overall operating conditions, and the final generated control commands uniformly act on the corresponding functional components of the entire air-liquid co-source cooling system, the independent regulation between subsystems is broken, overcoming the mutual constraint problems caused by hydraulic and thermal coupling due to sharing the same outdoor cold source and primary side water supply network. This achieves unified and coordinated control of each functional component of the system, ultimately improving energy utilization.

[0011] One possible implementation involves generating a system feature vector based on outdoor dry-bulb temperature data, equipment power information from the data on the air-liquid co-source cooling system, and the operating status data of the air-liquid co-source hybrid cooling system. This vector includes:

[0012] Based on outdoor dry-bulb temperature data and equipment power information in the data of the air-liquid co-source cooling system, environmental disturbance variables are generated;

[0013] Based on the secondary side water supply temperature data of the cooling capacity distribution unit in the liquid cooling subsystem of the above-mentioned air-liquid co-source cooling system, and the return air temperature data of the precision air conditioning in the above-mentioned air-liquid co-source cooling system, a thermodynamic state variable is generated.

[0014] Based on the water pressure difference data of the primary side supply and return water network in the above-mentioned air-liquid co-source cooling system, hydraulic state variables are constructed.

[0015] Based on the control action log of the previous control cycle of the above-mentioned air-liquid co-source cooling system, historical action state variables are constructed.

[0016] The above-mentioned environmental disturbance variables, thermal state variables, hydraulic state variables, and historical action state variables are concatenated to generate the above-mentioned system feature vector.

[0017] Another possible implementation, based on the aforementioned system feature vector, involves determining at least one target control action from the candidate control action set that matches the aforementioned system feature vector, including:

[0018] The above system feature vectors are subjected to normalization preprocessing to obtain standardized feature vectors;

[0019] Input the standardized feature vectors mentioned above into the deep reinforcement learning model;

[0020] When the deep reinforcement learning model is in exploration mode, at least one candidate control action is randomly selected from the candidate control action set as the target control action.

[0021] When the deep reinforcement learning model is in exploitation mode, the standardized features are analyzed to obtain the state action value corresponding to each candidate control action. Based on the state action value, the target control action is selected from at least one candidate control action.

[0022] Another possible implementation, after generating the control instructions corresponding to each target control action sequentially based on the above control action sequence, further includes:

[0023] Log data of the corresponding control actions performed by the corresponding functional components in the above-mentioned air-liquid co-source cooling system is collected, and a multi-dimensional comprehensive reward value is calculated based on the above-mentioned log data.

[0024] The network parameters of the deep reinforcement learning model are updated based on the multi-dimensional comprehensive reward value.

[0025] Another possible implementation involves the log data, based on a preset mask probability, including: real-time operating power data of the cooling capacity distribution unit's water pump, real-time operating power data of the precision air conditioner, and real-time operating power data of the common cooling source side; the multi-dimensional comprehensive reward value is calculated based on the log data, including:

[0026] Calculate the total power consumption of the cooling system based on the log data above.

[0027] Calculate the energy consumption bonus based on the total power consumption mentioned above;

[0028] Security penalty items are selected based on the central processing unit temperature data or the ambient temperature of the data center mentioned above.

[0029] The stability penalty term is calculated based on the difference data of each control action parameter between the current control cycle and the previous control cycle. The types of the above control action parameters include the water pump frequency parameter of the cooling capacity distribution unit, the primary valve opening parameter of the cooling capacity distribution unit, the air supply set temperature parameter of the precision air conditioner, and the primary valve opening parameter of the precision air conditioner.

[0030] The energy consumption reward, safety penalty, and stability penalty mentioned above are summed to obtain the multi-dimensional comprehensive reward value.

[0031] Another possible implementation involves selecting security penalty items based on the central processing unit temperature data or the ambient temperature of the data center, including:

[0032] When the central processing unit temperature or the ambient temperature of the data center exceeds the preset safety threshold, a fixed penalty item will be selected.

[0033] When the central processing unit temperature or the ambient temperature of the data center is within the preset temperature threshold range, a dynamic penalty term is selected based on the temperature deviation between the central processing unit temperature or the ambient temperature of the data center and the lower limit of the preset temperature threshold range. The dynamic penalty term is proportional to the deviation.

[0034] Another possible implementation involves updating the network parameters of the deep reinforcement learning model based on the aforementioned multi-dimensional comprehensive reward value, including:

[0035] The system feature vector of the current control cycle, the control action sequence, the multi-dimensional comprehensive reward value, and the system feature vector of the next control cycle are combined into a Markov transition sample.

[0036] A predetermined number of Markov transition samples are randomly selected to form a training sample set;

[0037] Calculate the target state action value corresponding to each training sample in the training sample set based on the Bellman equation;

[0038] Calculate the temporal difference error between the state-action value output by the main value network and the target state-action value.

[0039] Based on the aforementioned temporal difference error, gradient descent is performed on the aforementioned main value network to obtain the updated main value network weight parameters.

[0040] The updated main value network weight parameters will be updated to the main value network according to the preset cycle.

[0041] Another possible implementation is that each control command carries at least one functional component parameter adjustment information; each functional component parameter adjustment information is used to indicate the adjustment of the parameter of a functional component in the aforementioned air-liquid co-source cooling system; after the control commands corresponding to each target control action are generated sequentially, the method further includes:

[0042] Based on the parameter adjustment information of at least one functional component carried in each control command, the parameters of the corresponding functional component are adjusted.

[0043] Another possible implementation is that the above-mentioned functional component adjustment parameter information includes at least one of the following: cooling capacity distribution unit water pump frequency parameter adjustment information; cooling capacity distribution unit primary side valve opening parameter adjustment information; air conditioning supply air temperature parameter adjustment information; air conditioning primary side inlet water valve opening parameter adjustment information.

[0044] Another possible implementation, before generating the system feature vector based on outdoor dry-bulb temperature data, equipment power information in the data of the air-liquid co-source cooling system, and operating data of the air-liquid co-source hybrid cooling system, the above method further includes:

[0045] A physical simulation environment for a co-source cooling system is established. The physical simulation environment includes a hydraulic model of the cooling capacity distribution unit, a heat transfer model of the cooling capacity distribution unit, a three-condition energy consumption model of the precision air conditioner, a primary side supply and return water network model, and a wind-liquid load coupling model.

[0046] The initial deep reinforcement learning model was pre-trained in the aforementioned physical simulation environment to obtain the aforementioned deep reinforcement learning model.

[0047] Another possible implementation method, the above methods also include:

[0048] By dividing the parameter range of each parameter of each functional component in the above-mentioned air-liquid homogeneous cooling system into discrete levels, at least one discrete level is obtained, and one discrete level corresponds to one parameter range of one parameter in one functional component.

[0049] Generate corresponding candidate control actions for each discrete level.

[0050] Secondly, embodiments of this application provide a control device for a cooling system, comprising:

[0051] The first generation module is used to generate a system feature vector based on outdoor dry-bulb temperature data, equipment power information of the data center where the air-liquid co-source cooling system is located, and operating data of the air-liquid co-source hybrid cooling system; the above operating data includes relevant data of each functional component in the above air-liquid co-source hybrid cooling system.

[0052] The determination module is used to determine at least one target control action that matches the system feature vector from the candidate control action set, based on the system feature vector, to generate a control action sequence; the candidate control action set includes at least one candidate control action, and the candidate control action set is obtained based on the parameter range of each functional component in the air-liquid homogeneous cooling system; the control action sequence includes at least one target control action.

[0053] The second generation module is used to generate control instructions corresponding to each target control action in sequence based on the above control action sequence. The control instructions are used to instruct the corresponding functional components in the above air-liquid co-source cooling system to perform the corresponding control actions.

[0054] Another possible implementation, the first generation module mentioned above is specifically used for:

[0055] Based on outdoor dry-bulb temperature data and equipment power information in the data of the air-liquid co-source cooling system, environmental disturbance variables are generated;

[0056] Based on the secondary side water supply temperature data of the cooling capacity distribution unit in the liquid cooling subsystem of the above-mentioned air-liquid co-source cooling system, and the return air temperature data of the precision air conditioning in the above-mentioned air-liquid co-source cooling system, a thermodynamic state variable is generated.

[0057] Based on the water pressure difference data of the primary side supply and return water network in the above-mentioned air-liquid co-source cooling system, hydraulic state variables are constructed.

[0058] Based on the control action log of the previous control cycle of the above-mentioned air-liquid co-source cooling system, historical action state variables are constructed.

[0059] The above-mentioned environmental disturbance variables, thermal state variables, hydraulic state variables, and historical action state variables are concatenated to generate the above-mentioned system feature vector.

[0060] Another possible implementation is that the aforementioned determining module is specifically used for:

[0061] The above system feature vectors are subjected to normalization preprocessing to obtain standardized feature vectors;

[0062] Input the standardized feature vectors mentioned above into the deep reinforcement learning model;

[0063] When the deep reinforcement learning model is in exploration mode, at least one candidate control action is randomly selected from the candidate control action set as the target control action.

[0064] When the deep reinforcement learning model is in exploitation mode, the standardized features are analyzed to obtain the state action value corresponding to each candidate control action. Based on the state action value, the target control action is selected from at least one candidate control action.

[0065] One possible implementation of the cooling system control device provided in this application further includes:

[0066] The data acquisition module is used to collect log data of the corresponding functional components in the above-mentioned air-liquid co-source cooling system executing corresponding control actions, and to calculate a multi-dimensional comprehensive reward value based on the log data.

[0067] The update module is used to update the network parameters of the deep reinforcement learning model based on the multi-dimensional comprehensive reward value.

[0068] Another possible implementation is that the log data includes: real-time operating power data of the cooling capacity distribution unit water pump, real-time operating power data of the precision air conditioner, and real-time operating power data of the common cooling source side; the above-mentioned acquisition module is specifically used for:

[0069] Calculate the total power consumption of the cooling system based on the log data above.

[0070] Calculate the energy consumption bonus based on the total power consumption mentioned above;

[0071] Security penalty items are selected based on the central processing unit temperature data or the ambient temperature of the data center mentioned above.

[0072] The stability penalty term is calculated based on the difference data of each control action parameter between the current control cycle and the previous control cycle. The types of the above control action parameters include the water pump frequency parameter of the cooling capacity distribution unit, the primary valve opening parameter of the cooling capacity distribution unit, the air supply set temperature parameter of the precision air conditioner, and the primary valve opening parameter of the precision air conditioner.

[0073] The energy consumption reward, safety penalty, and stability penalty mentioned above are summed to obtain the multi-dimensional comprehensive reward value.

[0074] Another possible implementation is that the above-mentioned data acquisition module is specifically used for:

[0075] When the central processing unit temperature or the ambient temperature of the data center exceeds the preset safety threshold, a fixed penalty item will be selected.

[0076] When the central processing unit temperature or the ambient temperature of the data center is within the preset temperature threshold range, a dynamic penalty term is selected based on the temperature deviation between the central processing unit temperature or the ambient temperature of the data center and the lower limit of the preset temperature threshold range. The dynamic penalty term is proportional to the deviation.

[0077] Another possible implementation is that the above-mentioned update module is specifically used for:

[0078] The system feature vector of the current control cycle, the control action sequence, the multi-dimensional comprehensive reward value, and the system feature vector of the next control cycle are combined into a Markov transition sample.

[0079] A predetermined number of Markov transition samples are randomly selected to form a training sample set;

[0080] Calculate the target state action value corresponding to each training sample in the training sample set based on the Bellman equation;

[0081] Calculate the temporal difference error between the state-action value output by the main value network and the target state-action value.

[0082] Based on the aforementioned temporal difference error, gradient descent is performed on the aforementioned main value network to obtain the updated main value network weight parameters.

[0083] The updated main value network weight parameters will be updated to the main value network according to the preset cycle.

[0084] Another possible implementation is that each control command carries at least one functional component parameter adjustment information; each functional component parameter adjustment information is used to indicate the adjustment of the parameter of a functional component in the above-mentioned air-liquid co-source cooling system; the control device for the cooling system provided in this application embodiment further includes:

[0085] The adjustment module is used to adjust the parameters of the corresponding functional component based on the parameter adjustment information of at least one functional component carried in each control command.

[0086] Another possible implementation is that the above-mentioned functional component adjustment parameter information includes at least one of the following: cooling capacity distribution unit water pump frequency parameter adjustment information; cooling capacity distribution unit primary side valve opening parameter adjustment information; air conditioning supply air temperature parameter adjustment information; air conditioning primary side inlet water valve opening parameter adjustment information.

[0087] Another possible implementation, the control device for the cooling system provided in this application embodiment further includes:

[0088] A module is established to create a physical simulation environment for a wind-liquid co-sourced hybrid cooling system. The physical simulation environment includes a hydraulic model of the cooling capacity distribution unit, a thermal heat transfer model of the cooling capacity distribution unit, a three-condition energy consumption model of the precision air conditioner, a primary side supply and return water network model, and a wind-liquid load coupling model.

[0089] The training module is used to pre-train the initial deep reinforcement learning model in the aforementioned physical simulation environment to obtain the aforementioned deep reinforcement learning model.

[0090] Another possible implementation, the control device for the cooling system provided in this application embodiment further includes:

[0091] The partitioning module is used to divide the parameter range of each parameter of each functional component in the above-mentioned air-liquid homogeneous cooling system into discrete levels to obtain at least one discrete level, and one discrete level corresponds to one parameter range of one parameter in one functional component.

[0092] The third generation module is used to generate corresponding candidate control actions for each discrete level.

[0093] Thirdly, this application provides a cooling system based on the same source of air and liquid, which can use the control method of the cooling system described in the first aspect, including a liquid cooling subsystem, an air cooling subsystem, and a common cold source side, wherein a coordinated control device for the cooling system based on the same source of air and liquid and the control device for the cooling system based on the same source of air and liquid are respectively communicatively connected to the liquid cooling subsystem, the air cooling subsystem and the common cold source side.

[0094] Fourthly, this application provides an electronic device comprising: a processor and a memory; the memory storing a program or instructions executable on the processor, wherein the program or instructions, when executed by the processor, implement the method of the first aspect described above.

[0095] Fifthly, this application provides a readable storage medium on which a program or instructions are stored, which, when executed by a computer, implement the method of the first aspect described above.

[0096] Sixthly, this application provides a computer program product stored in a storage medium, which, when executed by a computer, implements the method described in the first aspect.

[0097] In a seventh aspect, embodiments of this application provide a chip including a processor and a communication interface, wherein the communication interface is coupled to the processor, and the processor is used to run programs or instructions to implement the method described in the first aspect.

[0098] The beneficial effects of the second to seventh aspects mentioned above are described in the corresponding description of the first aspect and will not be repeated here. Attached Figure Description

[0099] Figure 1 A schematic diagram of the network architecture for a cooling system control method provided in this application embodiment;

[0100] Figure 2 This is a schematic diagram of the overall structure of a wind-liquid co-source cooling system provided in an embodiment of this application;

[0101] Figure 3 A schematic diagram of the structure of a liquid cooling subsystem in a wind-liquid co-source cooling system provided in this application embodiment;

[0102] Figure 4 A schematic diagram of the structure of an air-cooled subsystem in a wind-liquid co-source cooling system provided in this application embodiment.

[0103] Figure 5 A schematic flowchart illustrating a control method for a cooling system provided in an embodiment of this application;

[0104] Figure 6 A schematic flowchart illustrating another cooling system control method provided in an embodiment of this application;

[0105] Figure 7 A schematic flowchart illustrating another cooling system control method provided in this application embodiment;

[0106] Figure 8 A schematic flowchart illustrating another cooling system control method provided in this application embodiment;

[0107] Figure 9 A schematic flowchart illustrating another cooling system control method provided in this application embodiment;

[0108] Figure 10 A schematic flowchart illustrating another cooling system control method provided in this application embodiment;

[0109] Figure 11 A schematic flowchart illustrating another cooling system control method provided in this application embodiment;

[0110] Figure 12 A schematic flowchart illustrating another cooling system control method provided in this application embodiment;

[0111] Figure 13 A schematic flowchart illustrating another cooling system control method provided in this application embodiment;

[0112] Figure 14 A schematic flowchart illustrating another cooling system control method provided in this application embodiment;

[0113] Figure 15This is a schematic diagram of a reinforcement learning controller structure provided in an embodiment of this application;

[0114] Figure 16 This is a schematic diagram of the structure of a control device for a cooling system provided in an embodiment of this application;

[0115] Figure 17 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0116] The control method, apparatus, equipment, medium, and program products of the cooling system provided in this application will now be described in detail with reference to the accompanying drawings.

[0117] The technical solutions of the embodiments of this application will be clearly described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application are within the scope of protection of this application.

[0118] The terms "first," "second," etc., used in the specification and claims of this application are used to distinguish similar objects and not to describe a specific order or sequence. It should be understood that such use of data can be interchanged where appropriate so that embodiments of this application can be implemented in orders other than those illustrated or described herein, and the objects distinguished by "first," "second," etc., are generally of the same class and the number of objects is not limited; for example, a first object can be one or more. Furthermore, in the specification and claims, "and / or" indicates at least one of the connected objects, and the character " / " generally indicates that the preceding and following objects are in an "or" relationship.

[0119] The terms "at least one," "at least one of," etc., used in the specification and claims of this application refer to any one, any two, or a combination of two or more of the included items. For example, at least one of a, b, and c can mean: "a," "b," "c," "a and b," "a and c," "b and c," and "a, b, and c," where a, b, and c can be single or multiple. Similarly, "at least two" refers to two or more items, and its meaning is similar to that of "at least one."

[0120] In the description of this application, unless otherwise stated, "a plurality of" means two or more.

[0121] The present application provides a control method, apparatus, equipment, medium, and program product for a cooling system, which can be applied to the control scenarios of air-liquid co-source cooling systems.

[0122] Currently, the industry generally uses a distributed independent control scheme based on traditional PID algorithms for the control and regulation of air-liquid hybrid cooling systems. This scheme decomposes the air-liquid hybrid cooling system into two independent units: a CDU liquid cooling subsystem and a PAC air cooling subsystem. Each subsystem is equipped with a dedicated PID controller, employing a single-input single-output (SISO) local feedback regulation mode.

[0123] However, when using the aforementioned distributed independent control scheme based on the traditional PID algorithm, the CDU and PAC subsystems share the same outdoor cold source and primary water supply network, resulting in natural hydraulic and thermal coupling between the two subsystems. Furthermore, this coupling exhibits significant nonlinear characteristics as operating conditions such as heat load and cold source temperature change. Traditional PID controllers cannot decouple these nonlinear characteristics, leading to mutual constraints on the independent regulation of each subsystem and consequently, low overall power utilization of the data center.

[0124] To address the aforementioned technical problems, this application provides a control method, device, equipment, medium, and program product for a cooling system. Based on outdoor dry-bulb temperature data, data center equipment power information, and relevant operating data of each functional component in a hybrid air-liquid cooling system, a system feature vector is generated. The nonlinear characteristics that change with heat load, cold source temperature, and other operating conditions, as well as the operating status of each functional component, are uniformly incorporated into the feature representation. Subsequently, based on this system feature vector, a matching target control action is determined from a set of candidate control actions obtained based on the parameter ranges of each functional component to generate a control action sequence. Control commands are then generated sequentially to instruct the corresponding functional components in the hybrid air-liquid cooling system to execute the corresponding control actions. Since the determination of the target control action directly depends on the unified system feature vector reflecting the overall operating conditions, and the final generated control commands uniformly act on the corresponding functional components of the entire hybrid air-liquid cooling system, the independent adjustment between subsystems is broken, overcoming the mutual constraints caused by hydraulic and thermal coupling due to sharing the same outdoor cold source and primary water supply network. This achieves unified and coordinated control of all functional components of the system, ultimately improving energy utilization.

[0125] The control method, apparatus, equipment, medium, and program products of the cooling system provided in the embodiments of this application will be described in detail below with reference to the accompanying drawings.

[0126] Figure 1 The network architecture of a cooling system control method provided in an embodiment of this application is illustrated. For example... Figure 1As shown, the network architecture includes: a control device 101 for the cooling system, a terminal device 102, and a co-source air-liquid cooling system 103. The control device 101, the terminal device 102, and the co-source air-liquid cooling system 103 are interconnected.

[0127] In some embodiments, the control device 101 of the cooling system may be a server, a computer, or a processor or processing unit within a server or computer. The server may be a single server or a server cluster consisting of multiple servers. It should be noted that the embodiments of this application do not limit the specific device form of the control device 101 of the cooling system. Figure 1 The control device 101 of the cooling system is shown as an example of a single server.

[0128] In some embodiments, the terminal device may be a mobile phone, tablet computer, laptop computer, handheld computer, in-vehicle electronic device, mobile internet device (MID), augmented reality (AR) / virtual reality (VR) device, robot, wearable device, personal computer (PC), ultra-mobile personal computer (UMPC), netbook, or personal digital assistant (PDA), etc., and the embodiments of this application do not specifically limit it. Figure 1 The example shown is a mobile phone, with terminal device 102 as an example.

[0129] In some embodiments, the control device 101 of the cooling system responds to a target instruction sent by the terminal device 102 to automatically control the air-liquid co-source cooling system 103. It generates a system feature vector based on outdoor dry-bulb temperature data, equipment power information of the data center where the air-liquid co-source cooling system is located, and operating data of the air-liquid co-source hybrid cooling system. The operating data includes relevant data of each functional component in the air-liquid co-source hybrid cooling system. Based on the system feature vector, at least one target control action matching the system feature vector is determined from a set of candidate control actions to generate a control action sequence. The set of candidate control actions includes at least one candidate control action, which is obtained based on parameter ranges for each functional component in the air-liquid co-source cooling system. The control action sequence includes at least one target control action. Based on the control action sequence, a control instruction corresponding to each target control action is generated sequentially, and the control instruction is sent to the air-liquid co-source cooling system 103. The control instruction instructs the corresponding functional component in the air-liquid co-source cooling system to perform the corresponding control action.

[0130] It should be noted that the network architecture described in the embodiments of this application is for the purpose of more clearly illustrating the technical solutions of the embodiments of this application, and does not constitute a limitation on the technical solutions provided in the embodiments of this application. As network architectures evolve, the technical solutions provided in the embodiments of this application are also applicable to similar technical problems.

[0131] This application provides a wind-liquid co-source cooling system that utilizes the control method of the cooling system described in this application. Figure 2 This is a schematic diagram of the overall structure of a wind-liquid co-source cooling system provided in an embodiment of this application. Figure 3 This is a schematic diagram of the structure of a liquid cooling subsystem in a wind-liquid co-source cooling system provided in an embodiment of this application. Figure 4 This is a schematic diagram of the structure of an air-cooled subsystem in a wind-liquid co-source cooling system provided in an embodiment of this application.

[0132] See Figures 2 to 4The system mainly consists of a cooling water circulation system, a secondary-side liquid-cooled cabinet system (i.e., a liquid-cooling subsystem), a supplementary precision air-cooled system (i.e., an air-cooling subsystem), and a monitoring and control system. The cooling water circulation system comprises two main components: an outdoor cold source and a cooling water pump and distribution network, responsible for transferring the system's overall heat to the outside. The secondary-side liquid-cooled cabinet system consists of a cooling capacity distribution unit (heat exchanger and water pump), cold-plate liquid-cooled cabinets, and a coolant distribution network. The heat exchanger connects to the primary-side outdoor cold source on the primary side and to the terminal liquid-cooled cabinets on the secondary side, transferring heat from the secondary side to the primary side and distributing the cooled coolant appropriately to the terminal cold plates. The supplementary precision air-cooled system consists of evaporator coils, expansion valves, compressors, and other components, used to control the temperature and humidity within the data center. The monitoring and control system consists of temperature sensors, pressure sensors, flow meters, electric regulating valves, and other components, monitoring the system's operating status in real time and using variable frequency drive technology to adjust flow rate and water temperature, ensuring the system's normal operation. The system as a whole forms two ring-shaped pipe networks: a primary low-temperature water ring network and a secondary liquid-cooled medium- and high-temperature ring network. Figure 2 Numbered 1, 2, 6, 9, 11, and 14 are pressure sensors; 3 is a flow sensor; 4 and 10 are two-way valves; 5 is a plate heat exchanger; 7 is a Y-type shut-off valve; 8 and 13 are temperature sensors; 12 is an IT load; and 15 is a water supply device. The liquid cooling subsystem mainly handles the heat dissipation of high heat flux density components such as CPUs and GPUs. Its heat dissipation path is: cold plate microchannels to coolant to secondary side circulation pump to plate heat exchanger to primary side cooling water pump to cooling tower, accounting for more than 70% of the load. Figure 3 Numbered 1, 5, and 6 are two-way valves; 2 is a throttling device; 3 is a compressor and circulating pump, etc.; 4 is a plate heat exchanger; and 7 is a double-coil evaporator. The water-cooled subsystem theoretically has three operating modes, controlled by two valves and the chiller: Mechanical refrigeration mode, where the cooling tower return water temperature is higher than the indoor return air temperature, only mechanical refrigeration is performed. The coil two-way valve is closed and the plate heat exchanger two-way valve is opened, allowing the refrigerant to evaporate and carry away heat from the indoor return air in the evaporator. The cooling tower return water then passes through the plate heat exchanger, carrying away heat from the refrigerant. Hybrid mode, where the cooling tower return water temperature is between the indoor return air temperature and the natural cooling switching temperature, involves opening and closing the water coil two-way valve. The cooling tower return water first exchanges heat directly with the indoor return air, then enters the plate heat exchanger to exchange heat with the refrigerant. The remaining cooling capacity is supplemented by the chiller unit. Natural cooling mode, where the cooling tower return water temperature is lower than the natural cooling switching temperature, also involves opening and closing the water coil two-way valve, but shutting down the chiller unit. Only the cooling tower return water exchanges heat with the indoor return air through the surface cooler to save energy and improve system efficiency.

[0133] In some embodiments, the liquid cooling subsystem includes a cold plate, a CDU, a secondary circulating pump, a plate heat exchanger, and primary valves.

[0134] In some embodiments, the air-cooled subsystem includes a precision air conditioning PAC, a fan, a coil, a compressor, and primary side valves.

[0135] In some embodiments, the common cold source side of the air-liquid co-source cooling system includes a cooling tower and a primary side supply and return water network.

[0136] In some embodiments, the sensor acquisition side of the air-liquid co-source cooling system includes an outdoor temperature sensor, a CDU supply and return water temperature sensor, a PAC return air temperature sensor, a differential pressure sensor, a flow sensor, and a power acquisition unit.

[0137] In some embodiments, the control execution side of the air-liquid co-source cooling system includes a variable frequency pump actuator, a valve actuator, and a precision air conditioning set temperature execution interface.

[0138] In some embodiments, the intelligent controller of the air-liquid co-source cooling system includes a state construction module, a DQN decision module, a reward calculation module, an experience playback module, a target network update module, and a security constraint module.

[0139] This method takes a system with both liquid and air cooling as the controlled object and constructs the system control process as a Markov Decision Process (MDP). The agent periodically collects the system state, selects joint control actions from the discrete action space based on the trained DQN model, and distributes the actions to the CDU and PAC to achieve coordinated allocation of liquid cooling and air cooling capabilities. This allows the system to reduce total energy consumption and improve PUE performance while meeting the temperature safety boundaries of IT equipment and the data center.

[0140] In some embodiments, the air-liquid co-source cooling system provided in this application can be used as the controlled object of the control method for the cooling system of this application, specifically including: constructing the system control process as a Markov Decision Process (MDP). The agent periodically collects the system state, selects joint control actions from the discrete action space based on the trained DQN model, and sends the actions to the CDU and PAC to achieve coordinated allocation of liquid cooling and air cooling capabilities, so that the system can reduce total energy consumption and improve PUE performance while meeting the temperature safety boundaries of IT equipment and computer room.

[0141] See Figure 5 This is a flowchart illustrating a control method for a cooling system provided in an embodiment of this application. Figure 5 As shown, the cooling system control method provided in this application embodiment can be implemented by the control device of the cooling system described above, specifically including the following steps 201 to 203.

[0142] Step 201: The control device of the cooling system generates a system feature vector based on outdoor dry-bulb temperature data, equipment power information of the data center where the air-liquid co-source cooling system is located, and operating data of the air-liquid co-source hybrid cooling system; the above operating data includes relevant data of each functional component in the above air-liquid co-source hybrid cooling system.

[0143] In some embodiments, the control device of the cooling system collects the operating status of the air-liquid co-source system to generate a system feature vector, including: generating environmental disturbance variables based on the outdoor dry-bulb temperature and the total power of the IT equipment; generating thermal state variables based on the CDU secondary side supply water temperature and PAC return air temperature; constructing hydraulic state variables based on the primary side supply and return water pressure difference; constructing historical action state variables based on the control action set of the previous control cycle; and concatenating the environmental disturbance variables, thermal state variables, hydraulic state variables, and historical action states to generate the system feature vector.

[0144] In some embodiments, environmental disturbance variables represent outdoor dry-bulb temperature and total power of IT equipment; thermal state variables represent CDU secondary side supply water temperature and PAC return air temperature; hydraulic state variables represent primary side supply and return water pressure difference; historical action state variables represent the set of control actions in the previous control cycle; and operating data represent CDU secondary side supply water temperature, PAC return air temperature, primary side supply and return water pressure difference, and the set of control actions in the previous control cycle.

[0145] Step 202: Based on the system feature vector, the control device of the cooling system determines at least one target control action that matches the system feature vector from the candidate control action set to generate a control action sequence; the candidate control action set includes at least one candidate control action, and the candidate control action set is obtained based on the parameter range of each functional component in the air-liquid co-source cooling system; the control action sequence includes at least one target control action.

[0146] In some embodiments, the control device of the cooling system discretizes the continuous control variables to construct a joint action space to obtain a set of candidate control actions. The discretization level is set according to the optimal operating range of the control component to form a four-dimensional joint action space. The system feature vector is input into the DQN main Q network to output the Q value of each discrete action. Based on the ε-greedy strategy, a balance is struck between exploration and utilization to select the target control action of the current control cycle to generate a control action sequence.

[0147] In some embodiments, the candidate control action set represents a four-dimensional joint action space formed by discretizing multiple continuous control variables; the system feature vector represents the state input of the DQN agent, which includes environmental disturbance variables, thermodynamic state variables, hydraulic state variables, and historical action states.

[0148] Step 203: Based on the above-mentioned control action sequence, the control device of the cooling system generates control instructions corresponding to each target control action in sequence. The control instructions are used to instruct the corresponding functional components in the above-mentioned air-liquid co-source cooling system to perform the corresponding control actions.

[0149] In some embodiments, the control device of the cooling system generates control instructions corresponding to each target control action in sequence based on the control action sequence and sends them to the actuator, converting the selected target action into equipment control instructions, including adjusting the frequency of the CDU secondary side water pump, adjusting the opening of the CDU primary side valve, adjusting the opening of the PAC primary side valve, and adjusting the PAC air supply set temperature.

[0150] In some embodiments, the control command represents an instruction to instruct the corresponding functional component in the air-liquid co-source cooling system to perform a corresponding control action, such as adjusting the frequency of the CDU secondary side water pump, adjusting the opening of the CDU primary side valve, adjusting the opening of the PAC primary side valve, or adjusting the PAC air supply set temperature.

[0151] In some embodiments, combined with Figure 5 ,like Figure 6 As shown, step 201 above can be implemented through steps 201a to 201e.

[0152] Step 201a: The control device of the cooling system generates environmental disturbance variables based on the outdoor environment dry-bulb temperature data and the equipment power information in the data of the air-liquid co-source cooling system.

[0153] In some embodiments, the control device of the cooling system collects the operating status of the air-liquid co-source system in each control cycle, including generating environmental disturbance variables based on outdoor dry-bulb temperature data and the total power of IT equipment. If necessary, extended status variables such as flow rate, valve position feedback, local hotspot temperature, and mode status indicators can also be added.

[0154] In some embodiments, environmental disturbance variables characterize outdoor dry-bulb temperature data and total power of IT equipment.

[0155] Step 201b: The control device of the cooling system generates a thermodynamic state variable based on the secondary side water supply temperature data of the cooling capacity distribution unit in the liquid cooling subsystem of the above-mentioned air-liquid co-source cooling system, and the return air temperature data of the precision air conditioning system in the above-mentioned air-liquid co-source cooling system.

[0156] In some embodiments, the control device of the cooling system generates thermodynamic state variables based on the secondary side supply water temperature data of the cooling capacity distribution unit and the return air temperature data of the precision air conditioning system.

[0157] In some embodiments, the thermodynamic state variables characterize the secondary side supply water temperature data and the precision air conditioning return air temperature data of the cooling capacity distribution unit.

[0158] Step 201c: The control device of the cooling system constructs hydraulic state variables based on the water pressure difference data of the primary side supply and return water network in the above-mentioned air-liquid co-source cooling system.

[0159] In some embodiments, the control device of the cooling system constructs hydraulic state variables based on the water pressure difference data of the primary side supply and return water network.

[0160] In some embodiments, hydraulic state variables characterize the water pressure difference data of the primary supply and return water network.

[0161] Step 201d: The control device of the cooling system constructs historical action state variables based on the control action log of the previous control cycle of the current control cycle of the above-mentioned air-liquid co-source cooling system.

[0162] In some embodiments, the control device of the cooling system constructs historical action state variables based on the set of control actions from the previous control cycle.

[0163] In some embodiments, the historical action state variable represents the set of control actions in the previous control cycle.

[0164] Step 201e: The control device of the cooling system concatenates the above-mentioned environmental disturbance variables, the above-mentioned thermodynamic state variables, the above-mentioned hydraulic state variables, and the above-mentioned historical action state variables to generate the above-mentioned system feature vector.

[0165] In some embodiments, the control device of the cooling system concatenates environmental disturbance variables, thermal state variables, hydraulic state variables, and historical action state variables to form the state input of the DQN agent and generate a system feature vector.

[0166] In some embodiments, the system feature vector represents the state input of the DQN agent, which includes environmental disturbance variables, thermal state variables, hydraulic state variables, and historical action state variables.

[0167] In this way, the current thermal load, environmental boundary, coupled hydraulic state, and control inertia of the system can be fully characterized, providing a basis for subsequent action decisions.

[0168] The cooling system control method provided in this application generates a system feature vector based on outdoor dry-bulb temperature data, data center equipment power information, and relevant operating data of each functional component in the air-liquid co-source hybrid cooling system. The nonlinear characteristics that change with heat load, cold source temperature, and other operating conditions, as well as the operating status of each functional component, are uniformly incorporated into the feature representation. Subsequently, based on this system feature vector, a matching target control action is determined from a set of candidate control actions obtained based on the parameter ranges of each functional component to generate a control action sequence. Control commands are then generated sequentially to instruct the corresponding functional components in the air-liquid co-source cooling system to execute the corresponding control actions. Since the determination of the target control action directly depends on the unified system feature vector reflecting the overall operating conditions, and the final generated control commands uniformly act on the corresponding functional components of the entire air-liquid co-source cooling system, the independent regulation between subsystems is broken, overcoming the mutual constraints caused by hydraulic and thermal coupling due to sharing the same outdoor cold source and primary water supply network. This achieves unified and coordinated control of each functional component of the system, ultimately improving energy utilization.

[0169] In some embodiments, combined with Figure 5 ,like Figure 7 As shown, step 202 above can be implemented through steps 202a to 202d.

[0170] Step 202a: The control device of the cooling system performs normalization preprocessing on the above system feature vectors to obtain standardized feature vectors.

[0171] In some embodiments, the control device of the cooling system receives a multidimensional state vector of the system environment (i.e., a system feature vector) and performs normalization preprocessing to obtain a standardized feature vector.

[0172] In some embodiments, the standardized feature vector represents the data after normalization preprocessing.

[0173] Step 202b: The control device of the cooling system inputs the above-mentioned standardized feature vectors into the deep reinforcement learning model.

[0174] In some embodiments, the control device of the cooling system inputs a standardized feature vector into the DQN main Q network to estimate the state-action value function in the discrete action space.

[0175] In some embodiments, the deep reinforcement learning model represents the DQN master Q network; the state-action value represents the state-action value function, i.e., the Q value, within the discrete action space.

[0176] Step 202c: When the deep reinforcement learning model is in exploration mode, the control device of the cooling system randomly selects at least one candidate control action from the candidate control action set as the target control action.

[0177] In some embodiments, when the deep reinforcement learning model is in exploration mode based on the ε-greedy policy, the control device of the cooling system explores the unknown state space and randomly selects at least one candidate control action from the set of candidate control actions as the target control action.

[0178] Step 202d: When the deep reinforcement learning model is in utilization mode, the control device of the cooling system analyzes the standardized features to obtain the state action value corresponding to each candidate control action, and selects the target control action from at least one candidate control action based on the state action value.

[0179] In some embodiments, when the deep reinforcement learning model is in utilization mode, the control device of the cooling system utilizes known high-value actions, analyzes standardized features to obtain the Q value corresponding to each candidate control action, and selects the target control action from at least one candidate control action based on the Q value to output the optimal control command for the current control cycle.

[0180] In this way, by relying on the ε-greedy strategy, the exploration of the unknown state space and the utilization of known high-value actions are effectively balanced, thereby outputting the optimal control command for the current cycle and forming a unified joint control of the wind and liquid subsystems.

[0181] In some embodiments, combined with Figure 5 ,like Figure 8 As shown, after step 203 above, the cooling system control method provided in this application further includes steps 301 to 302.

[0182] Step 301: The control device of the cooling system collects log data of the corresponding functional components in the above-mentioned air-liquid co-source cooling system executing corresponding control actions, and calculates a multi-dimensional comprehensive reward value based on the log data.

[0183] In some embodiments, after the cooling system's control device executes an action, it collects log data of the corresponding functional components in the air-liquid co-source cooling system executing corresponding control actions based on the system response, and calculates a multi-dimensional comprehensive reward value based on the system response. The multi-dimensional comprehensive reward value includes at least an energy consumption reward item, a safety penalty item, and a stability penalty item. The energy consumption reward item is calculated based on the total system power consumption, the safety penalty item is selected based on the central processing unit temperature or the computer room temperature, and the stability penalty item is calculated based on the action changes between the current control cycle and the previous control cycle.

[0184] In some embodiments, log data represents system response data.

[0185] In some embodiments, the multi-dimensional comprehensive reward value represents a reward value that includes energy consumption rewards, security penalties, and stability penalties.

[0186] In some embodiments, the energy consumption bonus represents a negative value of the total power consumption of the system, which includes at least the power consumption of the CDU pump, the power consumption of the PAC, and the power consumption of the cold source.

[0187] In some embodiments, the safety penalty term represents a significant penalty imposed when the CPU temperature or data center temperature exceeds a safety threshold, or a dynamic penalty imposed when the temperature is between a target temperature and a limit temperature.

[0188] In some embodiments, the stability penalty term characterizes the action jump penalty applied when the action change in adjacent control cycles is too large.

[0189] Step 302: The control device of the cooling system updates the network parameters of the deep reinforcement learning model based on the multi-dimensional comprehensive reward value.

[0190] In some embodiments, the control device of the cooling system stores the transition sample, which is composed of the state, action, multi-dimensional comprehensive reward value and the next state obtained in the current control cycle, into the experience replay pool; randomly samples a small batch of samples from the experience replay pool, calculates the target Q value using the Bellman equation, and updates the network parameters of the deep reinforcement learning model by gradient descent on the main Q network; at the same time, it synchronizes the main Q network parameters to the target Q network according to a preset period.

[0191] In some embodiments, a transition sample represents a Markov transition tuple consisting of the state, action, multi-dimensional comprehensive reward value, and next state obtained in the current control cycle.

[0192] In some embodiments, the master Q network represents the neural network in the deep reinforcement learning model used to estimate the state-action value function in the discrete action space.

[0193] In some embodiments, the target Q-network represents a neural network used for periodically synchronizing parameters and calculating the target value.

[0194] In this way, by unifying the three objectives of energy saving, safety, and stability into a single optimization objective, we can prevent the agent from sacrificing thermal safety for energy saving, while stabilizing the training process, weakening the temporal correlation of samples, and improving learning convergence.

[0195] In some embodiments, combined with Figure 8 ,like Figure 9 As shown, the log data includes: real-time operating power data of the cooling capacity distribution unit water pump, real-time operating power data of the precision air conditioner, and real-time operating power data of the common cooling source side; the above step 301 can be implemented through the following steps 301a and 301e.

[0196] Step 301a: The control device of the cooling system calculates the total power consumption of the cooling system based on the log data.

[0197] In some embodiments, the control device of the cooling system calculates the total power consumption of the cooling system based on log data, wherein the total power consumption includes at least the power consumption of the CDU pump, the power consumption of the PAC, and the power consumption of the cold source.

[0198] In some embodiments, total power consumption represents the sum of CDU pump power consumption, PAC power consumption, and cold source equivalent power consumption.

[0199] Step 301b: The control device of the cooling system calculates the energy consumption bonus based on the total power consumption mentioned above.

[0200] In some embodiments, the control device of the cooling system calculates an energy consumption reward based on the total power consumption, with the negative value of the total power consumption of the system serving as the reward.

[0201] Step 301c: The control device of the cooling system selects a safety penalty item based on the central processing unit temperature data of the data center or the ambient temperature of the computer room.

[0202] In some embodiments, the cooling system control device selects a safety penalty based on the central processing unit temperature data of the data center or the ambient temperature of the computer room, and applies a significant penalty when the central processing unit temperature or the ambient temperature of the computer room exceeds a safety threshold.

[0203] Step 301d: The control device of the cooling system calculates the stability penalty term based on the difference data of each control action parameter between the current control cycle and the previous control cycle. The types of the above control action parameters include the water pump frequency parameter of the cooling capacity distribution unit, the primary valve opening parameter of the cooling capacity distribution unit, the air supply set temperature parameter of the precision air conditioner, and the primary valve opening parameter of the precision air conditioner.

[0204] In some embodiments, the control device of the cooling system calculates a stability penalty term based on the difference data of each control action parameter between the current control cycle and the previous control cycle. When the action change in adjacent control cycles is too large, an action jump penalty is applied. The types of each control action parameter include the water pump frequency parameter of the cooling capacity distribution unit, the primary valve opening parameter of the cooling capacity distribution unit, the air supply set temperature parameter of the precision air conditioner, and the primary valve opening parameter of the precision air conditioner.

[0205] Step 301e: The control device of the cooling system sums the above-mentioned energy consumption reward, safety penalty, and stability penalty to obtain the above-mentioned multi-dimensional comprehensive reward value.

[0206] In some embodiments, the control device of the cooling system sums up the energy consumption reward, safety penalty, and stability penalty to obtain a multi-dimensional comprehensive reward value.

[0207] In this way, by unifying the three goals of energy saving, safety, and stability into a single optimization objective, we can prevent intelligent agents from sacrificing thermal safety for energy saving or causing frequent adjustments for temperature control.

[0208] In some embodiments, combined with Figure 9 ,like Figure 10 As shown, step 301c above may specifically include steps 301c1 to 301c2.

[0209] Step 301c1: When the temperature of the central processing unit or the ambient temperature of the computer room in the data center exceeds the preset safety threshold, the control device of the cooling system selects a fixed penalty item.

[0210] In some embodiments, when the temperature of the central processing unit of the data center or the ambient temperature of the computer room exceeds a preset safety threshold, the control device of the cooling system determines it as an over-limit condition and selects a fixed penalty item.

[0211] In some embodiments, a preset safety threshold represents a hard safety boundary for the temperature of the central processing unit or the ambient temperature of the computer room.

[0212] In some embodiments, a fixed penalty term represents a significant penalty imposed on temperature exceedance conditions.

[0213] Step 301c2: When the temperature of the central processing unit or the ambient temperature of the computer room in the data center is within the preset temperature threshold range, the control device of the cooling system selects a dynamic penalty term based on the temperature deviation between the central processing unit temperature or the ambient temperature of the computer room and the lower limit of the preset temperature threshold range. The dynamic penalty term is proportional to the deviation.

[0214] In some embodiments, when the temperature of the central processing unit or the ambient temperature of the computer room in the data center is within a preset temperature threshold range, the control device of the cooling system determines it as an early warning condition. Based on the temperature deviation between the central processing unit temperature or the ambient temperature of the computer room and the lower limit of the preset temperature threshold range, a dynamic penalty term is selected, and the dynamic penalty term is proportional to the deviation.

[0215] In some embodiments, the preset temperature threshold range characterizes the transition temperature range between the target temperature and the limit temperature.

[0216] In some embodiments, the dynamic penalty term represents the transitional penalty that increases with temperature deviation under warning conditions.

[0217] In this way, by setting fixed and dynamic penalty items, hard constraints on temperature exceeding limits and soft constraints within the warning range are achieved, effectively ensuring the thermal safety of data center operation.

[0218] In some embodiments, combined with Figure 8 ,like Figure 11 As shown, step 302 above may also include steps 302a to 302f.

[0219] Step 302a: The control device of the cooling system combines the system feature vector of the current control cycle, the control action sequence, the multi-dimensional comprehensive reward value, and the system feature vector of the next control cycle into a Markov transition sample.

[0220] In some embodiments, the control device of the cooling system combines the system feature vector of the current control cycle, the sequence of control actions, the multi-dimensional comprehensive reward value, and the system feature vector of the next control cycle into a Markov transition sample and stores it in the experience replay pool.

[0221] In some embodiments, a Markov transition sample represents a transition tuple consisting of the state, action, multi-dimensional comprehensive reward value, and next state obtained in the current control cycle.

[0222] Step 302b: The control device of the cooling system randomly selects a predetermined number of Markov transfer samples to form a training sample set.

[0223] In some embodiments, the control device of the cooling system randomly selects a predetermined number of Markov transition samples from the experience replay pool to form a training sample set, so as to break the temporal correlation between samples.

[0224] In some embodiments, the training sample set represents a small batch of transfer samples randomly drawn from the experience replay pool.

[0225] Step 302c: The control device of the cooling system calculates the target state action value corresponding to each training sample in the training sample set based on the Bellman equation.

[0226] In some embodiments, the control device of the cooling system is based on the Bellman equation and uses a target value network to calculate the target state action value corresponding to each training sample in the training sample set.

[0227] In some embodiments, the target state action value representation is based on the target Q value calculated using the Bellman equation and the target value network.

[0228] Step 302d: The control device of the cooling system calculates the timing difference error between the state action value output by the main value network and the target state action value.

[0229] In some embodiments, the control device of the cooling system calculates the temporal difference error between the state action value output by the main value network and the target state action value.

[0230] In some embodiments, the temporal difference error characterizes the difference between the state-action value output by the master value network and the target state-action value.

[0231] Step 302e: The control device of the cooling system performs gradient descent on the main value network based on the aforementioned time-series difference error to obtain the updated main value network weight parameters.

[0232] In some embodiments, the control device of the cooling system performs gradient descent optimization on the main value network based on the time-series difference error to obtain the updated main value network weight parameters.

[0233] In some embodiments, the master value network represents the neural network in a deep reinforcement learning model used to estimate the state-action value function in a discrete action space.

[0234] Step 302f: The control device of the cooling system updates the weight parameters of the main value network to the main value network according to a preset cycle.

[0235] In some embodiments, the control device of the cooling system updates the updated main value network weight parameters to the target value network according to a preset period to maintain the stability of the target value.

[0236] In some embodiments, the target value network represents a neural network used for periodically synchronizing parameters and calculating the value of a target state action.

[0237] Thus, through experience replay and target network parameter synchronization mechanisms, the temporal correlation between samples is effectively weakened, improving the convergence and stability of the deep reinforcement learning model training process.

[0238] In some embodiments, combined with Figure 8 ,like Figure 12 As shown, each control command carries at least one functional component parameter adjustment information; each functional component parameter adjustment information is used to indicate the adjustment of the parameter of a functional component in the above-mentioned air-liquid co-source cooling system; after the above-mentioned step 302, the control method of the cooling system provided in this application embodiment may further include the following step 401.

[0239] Step 401: The control device of the cooling system adjusts the parameters of the corresponding functional components based on the parameter adjustment information of at least one functional component carried in each control command.

[0240] In some embodiments, the control device of the cooling system sends control information for at least one functional component parameter adjustment carried in each control command to the actuator to adjust the parameters of the corresponding functional component. The types of parameters include the water pump frequency parameter of the cooling capacity distribution unit, the primary valve opening parameter of the cooling capacity distribution unit, the air supply temperature parameter of the air conditioner, and the primary water inlet valve opening parameter of the air conditioner.

[0241] In some embodiments, the functional component represents an actuator such as a cooling capacity distribution unit and a precision air conditioner in a wind-liquid co-source cooling system.

[0242] In this way, by directly driving the corresponding functional components to adjust their parameters through control commands, precise and coordinated control of the key equipment in the air-liquid co-source cooling system is achieved.

[0243] In some embodiments, the above-mentioned functional component adjustment parameter information includes at least one of the following: cooling capacity distribution unit water pump frequency parameter adjustment information; cooling capacity distribution unit primary side valve opening parameter adjustment information; air conditioning supply air temperature parameter adjustment information; air conditioning primary side inlet valve opening parameter adjustment information.

[0244] In some embodiments, combined with Figure 5 ,like Figure 13 As shown, prior to step 201 above, the control method of the cooling system provided in this application embodiment may further include the following steps 501 to 502.

[0245] Step 501: The control device of the cooling system establishes a physical simulation environment for the air-liquid co-source hybrid cooling system. The physical simulation environment includes a hydraulic model of the cooling capacity distribution unit, a thermal heat transfer model of the cooling capacity distribution unit, a three-condition energy consumption model of the precision air conditioner, a primary side supply and return water network model, and an air-liquid load coupling model.

[0246] In some embodiments, the control device of the cooling system establishes a physical simulation environment for the air-liquid co-source hybrid cooling system. The physical simulation environment includes a hydraulic model of the cooling capacity distribution unit, a thermal heat exchange model of the cooling capacity distribution unit, a three-condition energy consumption model of the precision air conditioner, a primary side supply and return water network model, and an air-liquid load coupling model.

[0247] In some embodiments, the physical simulation environment characterization includes the simulation test environment of the hydraulic model of the cooling capacity distribution unit, the thermal heat exchange model of the cooling capacity distribution unit, the energy consumption model of the precision air conditioner under three operating conditions, the primary side supply and return water network model, and the air-liquid load coupling model.

[0248] Step 502: The control device of the cooling system pre-trains the initial deep reinforcement learning model in the above physical simulation environment to obtain the above deep reinforcement learning model.

[0249] In some embodiments, the control device of the cooling system pre-trains the initial deep reinforcement learning model in the physical simulation environment described above to obtain the deep reinforcement learning model.

[0250] In some embodiments, the initial deep reinforcement learning model represents a deep reinforcement learning model that has not been pre-trained in a physical simulation environment.

[0251] In this way, by pre-training the initial deep reinforcement learning model in a physical simulation environment, a deep reinforcement learning model with initial decision-making capabilities is provided for subsequent control, avoiding the risk of trial and error directly on the real system.

[0252] In some embodiments, combined with Figure 5 ,like Figure 14 As shown, the control method of the cooling system provided in this application embodiment may further include the following steps 601 to 602.

[0253] Step 601: The control device of the cooling system divides the parameter range of each parameter of each functional component in the above-mentioned air-liquid co-source cooling system into discrete levels to obtain at least one discrete level. Each discrete level corresponds to a parameter range of one parameter in a functional component.

[0254] In some embodiments, the control device of the cooling system obtains at least one discrete level by dividing the parameter range of each parameter of each functional component in the above-mentioned air-liquid co-source cooling system into discrete levels, and each discrete level corresponds to a parameter range of a parameter in a functional component.

[0255] In some embodiments, the parameter range represents the optimal operating range in which the parameters of each functional component can be adjusted.

[0256] Step 602: The control device of the cooling system generates corresponding candidate control actions for each discrete level.

[0257] In some embodiments, the control device of the cooling system generates corresponding candidate control actions for each discrete level to construct a set of candidate control actions.

[0258] In some embodiments, the candidate control action set represents a set of candidate control actions generated based on discrete levels of parameters of each functional component.

[0259] In this way, the multivariable continuous control problem is transformed into a discrete decision problem that can be solved by a deep reinforcement learning model, ensuring the feasibility and rationality of the control decision.

[0260] The following describes the control method of the cooling system according to an embodiment of this application through specific examples. This includes:

[0261] The overall architecture of the cooling system control method provided in the application embodiment includes steps 1 to 6.

[0262] Step 1: Establish the physical simulation environment for the air-liquid co-source cooling system:

[0263] Establish a simulation environment capable of characterizing the coupling properties of a wind-liquid co-source system. This simulation environment should include at least the following: a CDU hydraulic model.

[0264] , (1);

[0265] in, This represents the secondary mass flow rate. Indicates the rated mass flow rate. Indicates the actual operating frequency of the water pump. Indicates the rated frequency of the water pump. Indicates the power consumption of the water pump. Indicates the rated power of the water pump. This indicates the motor efficiency.

[0266] CDU heat transfer model:

[0267] , (2);

[0268] in, This represents the CDU heat exchange power, and ε represents the heat exchanger efficiency. Indicates the minimum heat capacity. Indicates the secondary return water temperature. Indicates the primary inlet water temperature. This represents the primary mass flow rate. This indicates the specific heat capacity of the primary fluid. This represents the secondary mass flow rate. This indicates the specific heat capacity of the secondary fluid.

[0269] PAC three-condition energy consumption model:

[0270] Natural cooling: Mechanical refrigeration: Blending mode: (3);

[0271] in, This indicates the PAC power consumption under natural cooling mode. Indicates fan power consumption. This indicates the allocated power consumption of the water pump. This indicates the PAC power consumption in mechanical cooling mode. Indicates the wind-side heat load. This represents the coefficient of performance for direct expansion refrigeration. Indicates the condensation temperature. Indicates the evaporation temperature. This represents the PAC power consumption in hybrid mode, where α represents the compressor load rate. This indicates the compressor's rated power.

[0272] Primary supply and return water network model:

[0273] , (4);

[0274] in, Indicates the valve flow coefficient. R represents the valve's maximum flow coefficient, and R represents the valve's adjustability ratio. Indicates the valve opening degree. This represents the primary mass flow rate. Indicates the flow coefficient. Indicates the system pressure difference. This indicates the pressure drop of the heat exchanger.

[0275] Wind-liquid load coupling model:

[0276] , (5).

[0277] in, Indicates the air-side cooling load. Indicates liquid-cooled capture rate, This indicates the total heat load of IT equipment. Indicates other heat loads, This indicates the liquid-side cooling load.

[0278] Among them: the CDU hydraulic model is used to describe the relationship between secondary flow rate and pump frequency, as well as the cubic relationship between pump power and frequency; the CDU thermal model is used to describe the heat exchange process between the primary and secondary sides of the plate heat exchanger; the PAC three-condition model is used to describe the energy consumption changes under mechanical refrigeration mode, natural cooling mode and hybrid refrigeration mode; the air-liquid coupling model is used to describe the distribution relationship of total IT heat load between the liquid-cooled side and the air-cooled side, wherein the liquid-cooled heat capture rate is affected by the CDU pump frequency.

[0279] The purpose of step 1 is to provide an interactive, trainable, and verifiable environment for reinforcement learning, enabling the controller to learn the wind-liquid synergy during the training phase, rather than relying solely on static rules.

[0280] Step 2: Collect and construct the system state vector:

[0281] In each control cycle, the operating status of the air-liquid co-source system is collected, forming the state input of the DQN agent. The aforementioned state vector includes at least the following four types of information: environmental disturbance variables: outdoor dry-bulb temperature, total power of IT equipment; thermal state variables: CDU secondary side supply water temperature, PAC return air temperature; hydraulic state variables: primary side supply and return water pressure difference; historical action status: the set of control actions from the previous control cycle. If necessary, extended state variables such as flow rate, valve position feedback, local hotspot temperature, and mode status identifiers can also be added.

[0282] The purpose of step 2 is to fully characterize the current thermal load, environmental boundary, coupled hydraulic state, and control inertia of the system, providing a basis for subsequent action decisions. Its relationship with step 3 is that the state vector output from step 2 is directly used as the input to step 3.

[0283] Step 3: Discretize the continuous control variables and construct the joint action space:

[0284] Since DQN processes discrete actions, this invention discretizes multiple continuous control variables. These action variables include at least: CDU pump frequency; CDU valve opening; PAC air supply set temperature; and PAC valve opening.

[0285] Discrete levels are set according to the optimal operating range of the control components: CDU pump frequency: 30Hz, 35Hz, 40Hz, 45Hz, 50Hz; CDU valve opening: 20%, 40%, 60%, 80%, 100%; PAC air supply set temperature: 18℃, 20℃, 22℃, 24℃, 26℃; PAC valve opening: 20%, 40%, 60%, 80%, 100%. This forms a four-dimensional joint action space with a total of 625 action combinations. The agent selects one set of action combinations to execute in each control cycle.

[0286] The purpose of step 3 is to transform the multivariable continuous control problem into a DQN-solvable discrete decision problem, and to incorporate the control variables of air cooling and liquid cooling into the same joint action, thereby avoiding local optima caused by traditional distributed control.

[0287] Step 4: Generate joint control actions based on the DQN policy network:

[0288] The state vector obtained in step S2 is input into the DQN master Q-network, which outputs the Q-values ​​of each discrete action. Based on the ε-greedy strategy, a balance is struck between exploration and exploitation to select the target action for the current control cycle. The selected action is then converted into equipment control commands and sent to the actuators, including: adjusting the frequency converter of the CDU secondary side water pump; adjusting the opening of the CDU primary side valve; adjusting the opening of the PAC primary side valve; and adjusting the PAC air supply set temperature.

[0289] The purpose of step 4 is to establish unified joint control over the wind and liquid subsystems, rather than independent local control. After step 4 is executed, the environmental state changes, which is then evaluated in step 5.

[0290] Step 5: Calculate the multi-objective reward value:

[0291] After the action is executed, the reward value is calculated based on the system response. The reward function includes at least the following three parts: energy consumption reward item:

[0292] (6);

[0293] in, This indicates an energy consumption bonus item. This represents the weighting coefficient of the energy consumption reward item. Indicates the power consumption of the CDU pump. Indicates PAC power consumption. This represents the power consumption of the cold source.

[0294] The reward is a negative value of the total system power consumption, which includes at least the power consumption of the CDU pump, the power consumption of the PAC, and the power consumption of the cold source.

[0295] Safety penalties:

[0296] (7);

[0297] When the CPU temperature or server room temperature exceeds a safe threshold, a severe penalty will be imposed; among other things... This indicates a safety penalty item, where M represents a fixed penalty item. Indicates the temperature of the central processing unit. This indicates a preset safety threshold. This represents the weighting coefficient of the safety penalty item. This indicates the lower limit of the preset temperature threshold range.

[0298] Stability penalty:

[0299] (8);

[0300] When the action change in adjacent control cycles is too large, an action jump penalty is applied. This indicates a stability penalty term. The weighting coefficients of the stability penalty term are represented. This indicates the difference in the frequency parameters of the water pumps in the cooling capacity distribution unit. This indicates the difference in valve opening parameters on the primary side of the cooling capacity distribution unit. This indicates the difference in the set temperature parameter of the precision air conditioner's air supply.

[0301] The purpose of step 5 is to unify the three objectives of "energy saving", "safety" and "stability" into a single optimization goal, so as to prevent the intelligent agent from sacrificing thermal safety for energy saving or causing frequent adjustments for temperature control.

[0302] Step 6: Perform experience replay and network parameter update:

[0303] The state, action, reward, and next state obtained in the current control cycle are combined into a transition sample and stored in the experience replay pool. A small batch of samples are randomly sampled from the experience replay pool, the target Q value is calculated using the Bellman equation, and the main Q network is updated by gradient descent. At the same time, the parameters of the main Q network are synchronized to the target Q network according to a preset period.

[0304] Step 6 aims to stabilize the training process, reduce temporal correlations among samples, and improve learning convergence. This step, together with steps 4 and 5, forms a closed loop for reinforcement learning. The controller structure comprised of steps 3, 4, 5, and 6 is as follows: Figure 15 As shown, this architecture effectively decouples the online decision feedforward mechanism from the offline parameter optimization mechanism. During the decision-making phase, the controller first receives the multi-dimensional state vector of the system environment and performs normalization preprocessing. Subsequently, the preprocessed data is input into the main Q-network to estimate the state-action value function in the discrete action space. The algorithm relies on... The strategy effectively balances the exploration of the unknown state space with the utilization of known high-value actions, thereby outputting the optimal control command for the current cycle. During the parameter update phase, the system introduces an experience replay mechanism to store historical Markov transition tuples. Furthermore, the algorithm breaks down the temporal correlation of training data through random mini-batch sampling. Simultaneously, it combines a periodically synchronized target Q-network to calculate the target value and temporal difference (TD) error, and finally updates the weight gradients of the backbone neural network through backpropagation. The synergistic effect of these mechanisms effectively ensures the stability and convergence of the model training process.

[0305] It should be noted that the above-described method embodiments, or the various possible implementations of the method embodiments, can be executed individually, or, provided there is no conflict, they can be combined with each other. The specific implementation can be determined according to actual usage requirements, and this application embodiment does not impose any restrictions on this.

[0306] As can be seen, the above mainly describes the solutions provided by the embodiments of this application from a methodological perspective. To achieve the above functions, the embodiments of this application provide corresponding hardware structures and / or software modules for executing each function. Those skilled in the art should readily recognize that, in conjunction with the modules and algorithm steps of the various examples described in the embodiments disclosed herein, the embodiments of this application can be implemented in hardware or a combination of hardware and computer software. Whether a function is executed in hardware or by computer software driving hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0307] This application embodiment can divide the control device of the cooling system into functional modules according to the above method example. For example, each function can be divided into a separate functional module, or two or more functions can be integrated into one processing module. The integrated module can be implemented in hardware or as a software functional module. Optionally, the module division in this application embodiment is illustrative and only represents one logical functional division; other division methods may be used in actual implementation.

[0308] In some embodiments, this application also provides a control device for a cooling system. The control device may include one or more functional modules for implementing the control method of the cooling system described in the above embodiments.

[0309] For example, Figure 16 This is a schematic diagram of the structure of a control device for a cooling system provided in an embodiment of this application. Figure 16 As shown, the control device 900 of the cooling system includes:

[0310] The first generation module 901 is used to generate a system feature vector based on outdoor dry-bulb temperature data, equipment power information of the data center where the air-liquid co-source cooling system is located, and operating data of the air-liquid co-source hybrid cooling system; the above operating data includes relevant data of each functional component in the above air-liquid co-source hybrid cooling system.

[0311] The determining module 902 is used to determine at least one target control action that matches the system feature vector from the candidate control action set based on the system feature vector, so as to generate a control action sequence; the candidate control action set includes at least one candidate control action, and the candidate control action set is obtained based on the parameter range of each functional component in the air-liquid homogeneous cooling system; the control action sequence includes at least one target control action.

[0312] The second generation module 903 is used to generate control instructions corresponding to each target control action in sequence based on the above control action sequence. The control instructions are used to instruct the corresponding functional components in the above air-liquid co-source cooling system to perform the corresponding control actions.

[0313] The control device for the cooling system provided in this application generates a system feature vector based on outdoor dry-bulb temperature data, data center equipment power information, and relevant operating data of each functional component in the air-liquid co-source hybrid cooling system. The nonlinear characteristics that change with heat load, cold source temperature, and other operating conditions, as well as the operating status of each functional component, are uniformly incorporated into the feature representation. Subsequently, based on this system feature vector, a matching target control action is determined from the candidate control action set obtained based on the parameter range of each functional component to generate a control action sequence. Control commands are then generated sequentially to instruct the corresponding functional components in the air-liquid co-source cooling system to execute the corresponding control actions. Since the determination of the target control action directly depends on the unified system feature vector reflecting the overall operating conditions, and the final generated control commands uniformly act on the corresponding functional components of the entire air-liquid co-source cooling system, the independent regulation between subsystems is broken, overcoming the mutual constraint problems caused by hydraulic and thermal coupling due to sharing the same outdoor cold source and primary water supply network. This achieves unified and coordinated control of each functional component of the system, ultimately improving energy utilization.

[0314] In some embodiments, the first generation module 901 is specifically used for:

[0315] Based on outdoor dry-bulb temperature data and equipment power information in the data of the air-liquid co-source cooling system, environmental disturbance variables are generated;

[0316] Based on the secondary side water supply temperature data of the cooling capacity distribution unit in the liquid cooling subsystem of the above-mentioned air-liquid co-source cooling system, and the return air temperature data of the precision air conditioning in the above-mentioned air-liquid co-source cooling system, a thermodynamic state variable is generated.

[0317] Based on the water pressure difference data of the primary side supply and return water network in the above-mentioned air-liquid co-source cooling system, hydraulic state variables are constructed.

[0318] Based on the control action log of the previous control cycle of the above-mentioned air-liquid co-source cooling system, historical action state variables are constructed.

[0319] The above-mentioned environmental disturbance variables, thermal state variables, hydraulic state variables, and historical action state variables are concatenated to generate the above-mentioned system feature vector.

[0320] In other embodiments, the determining module 902 is specifically used for:

[0321] The above system feature vectors are subjected to normalization preprocessing to obtain standardized feature vectors;

[0322] Input the standardized feature vectors mentioned above into the deep reinforcement learning model;

[0323] When the deep reinforcement learning model is in exploration mode, at least one candidate control action is randomly selected from the candidate control action set as the target control action.

[0324] When the deep reinforcement learning model is in exploitation mode, the standardized features are analyzed to obtain the state action value corresponding to each candidate control action. Based on the state action value, the target control action is selected from at least one candidate control action.

[0325] In other embodiments, the acquisition module is used to acquire log data of the corresponding functional components in the above-mentioned air-liquid homogeneous cooling system performing corresponding control actions, and to calculate a multi-dimensional comprehensive reward value based on the log data.

[0326] The update module is used to update the network parameters of the deep reinforcement learning model based on the multi-dimensional comprehensive reward value.

[0327] In other embodiments, the log data includes: real-time operating power data of the cooling capacity distribution unit water pump, real-time operating power data of the precision air conditioner, and real-time operating power data of the common cooling source side; the acquisition module is specifically used for:

[0328] Calculate the total power consumption of the cooling system based on the log data above.

[0329] Calculate the energy consumption bonus based on the total power consumption mentioned above;

[0330] Security penalty items are selected based on the central processing unit temperature data or the ambient temperature of the data center mentioned above.

[0331] The stability penalty term is calculated based on the difference data of each control action parameter between the current control cycle and the previous control cycle. The types of the above control action parameters include the water pump frequency parameter of the cooling capacity distribution unit, the primary valve opening parameter of the cooling capacity distribution unit, the air supply set temperature parameter of the precision air conditioner, and the primary valve opening parameter of the precision air conditioner.

[0332] The energy consumption reward, safety penalty, and stability penalty mentioned above are summed to obtain the multi-dimensional comprehensive reward value.

[0333] In other embodiments, the above-mentioned acquisition module is specifically used for:

[0334] When the central processing unit temperature or the ambient temperature of the data center exceeds the preset safety threshold, a fixed penalty item will be selected.

[0335] When the central processing unit temperature or the ambient temperature of the data center is within the preset temperature threshold range, a dynamic penalty term is selected based on the temperature deviation between the central processing unit temperature or the ambient temperature of the data center and the lower limit of the preset temperature threshold range. The dynamic penalty term is proportional to the deviation.

[0336] In other embodiments, the above-described update module is specifically used for:

[0337] The system feature vector of the current control cycle, the control action sequence, the multi-dimensional comprehensive reward value, and the system feature vector of the next control cycle are combined into a Markov transition sample.

[0338] A predetermined number of Markov transition samples are randomly selected to form a training sample set;

[0339] Calculate the target state action value corresponding to each training sample in the training sample set based on the Bellman equation;

[0340] Calculate the temporal difference error between the state-action value output by the main value network and the target state-action value.

[0341] Based on the aforementioned temporal difference error, gradient descent is performed on the aforementioned main value network to obtain the updated main value network weight parameters.

[0342] The updated main value network weight parameters will be updated to the main value network according to the preset cycle.

[0343] In other embodiments, each control command carries at least one functional component parameter adjustment information; each functional component parameter adjustment information is used to instruct the adjustment of the parameter of a functional component in the aforementioned air-liquid co-source cooling system; the control device for the cooling system provided in this application embodiment further includes:

[0344] The adjustment module is used to adjust the parameters of the corresponding functional component based on the parameter adjustment information of at least one functional component carried in each control command.

[0345] In other embodiments, the above-mentioned functional component adjustment parameter information includes at least one of the following: cooling capacity distribution unit water pump frequency parameter adjustment information; cooling capacity distribution unit primary side valve opening parameter adjustment information; air conditioning supply air temperature parameter adjustment information; air conditioning primary side inlet valve opening parameter adjustment information.

[0346] In other embodiments, the control device for the cooling system provided in this application further includes:

[0347] A module is established to create a physical simulation environment for a wind-liquid co-sourced hybrid cooling system. The physical simulation environment includes a hydraulic model of the cooling capacity distribution unit, a thermal heat transfer model of the cooling capacity distribution unit, a three-condition energy consumption model of the precision air conditioner, a primary side supply and return water network model, and a wind-liquid load coupling model.

[0348] The training module is used to pre-train the initial deep reinforcement learning model in the aforementioned physical simulation environment to obtain the aforementioned deep reinforcement learning model.

[0349] In other embodiments, the control device for the cooling system provided in this application further includes:

[0350] The partitioning module is used to divide the parameter range of each parameter of each functional component in the above-mentioned air-liquid homogeneous cooling system into discrete levels to obtain at least one discrete level, and one discrete level corresponds to one parameter range of one parameter in one functional component.

[0351] The third generation module is used to generate corresponding candidate control actions for each discrete level. It should be noted that the control device of the cooling system can implement all the processes described in the above method embodiments and achieve the same beneficial effects; therefore, to avoid repetition, it will not be elaborated further here.

[0352] In the case where the functions of the integrated modules described above are implemented in hardware, this application provides a possible structural schematic diagram of the electronic device involved in the above embodiments. For example... Figure 17 As shown, the electronic device 90 includes: a processor 92, a communication interface 93, and a bus 94. Optionally, the electronic device 90 may also include a memory 91.

[0353] Processor 92 may implement or execute various exemplary logic blocks, modules, and circuits described in conjunction with the disclosure of this application. Processor 92 may be a central processing unit, a general-purpose processor, a digital signal processor, an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. It may implement or execute various exemplary logic blocks, modules, and circuits described in conjunction with the disclosure of this application. Processor 92 may also be a combination that implements computational functions, such as including one or more microprocessor combinations, a combination of a DSP and a microprocessor, etc.

[0354] Communication interface 93 is used to connect with other devices via a communication network. This communication network can be Ethernet, wireless access network, wireless local area network (WLAN), etc.

[0355] The memory 91 may be a read-only memory (ROM) or other type of static storage device capable of storing static information and instructions, random access memory (RAM) or other type of dynamic storage device capable of storing information and instructions, or electrically erasable programmable read-only memory (EEPROM), disk storage media or other magnetic storage devices, or any other medium capable of carrying or storing desired program code in the form of instructions or data structures and accessible by a computer, but is not limited thereto.

[0356] As one possible implementation, the memory 91 can exist independently of the processor 92. The memory 91 can be connected to the processor 92 via a bus 94 and is used to store instructions or program code. When the processor 92 calls and executes the instructions or program code stored in the memory 91, it can implement the control method of the cooling system provided in the embodiments of this application.

[0357] In another possible implementation, the memory 91 can also be integrated with the processor 92.

[0358] Bus 94 can be an Extended Industry Standard Architecture (EISA) bus, etc. Bus 94 can be divided into address bus, data bus, control bus, etc. For ease of representation, Figure 17 The bus is represented by a single thick line, but this does not mean that there is only one bus or one type of bus.

[0359] Through the above description of the implementation methods, those skilled in the art can clearly understand that, for the sake of convenience and brevity, only the division of the above functional modules is used as an example. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the service calling device can be divided into different functional modules to complete all or part of the functions described above.

[0360] This application embodiment also provides a chip, which includes a processor and a communication interface. The communication interface and the processor are coupled. The processor is used to run programs or instructions to implement the various processes of the control method embodiment of the cooling system and can achieve the same technical effect. To avoid repetition, it will not be described again here.

[0361] It should be understood that the chip mentioned in the embodiments of this application may also be referred to as a system-on-a-chip, system chip, chip system, or system-on-a-chip, etc.

[0362] This application also provides a readable storage medium storing a program or instructions. When executed by a computer, the program or instructions implement the cooling system control method provided in the above embodiments. It is understood that all or part of the processes in the above method embodiments can be executed by computer instructions instructing related hardware. The readable storage medium can be any of the foregoing embodiments or memory. The readable storage medium can also be an external storage device of the service invocation device, such as a pluggable hard drive, SmartMedia Card (SMC), Secure Digital (SD) card, flash card, etc., equipped on the service invocation device. Further, the readable storage medium can include both internal storage units of the service invocation device and external storage devices. The readable storage medium is used to store the computer program and other programs and data required by the service invocation device. The readable storage medium can also be used to temporarily store data that has been output or will be output.

[0363] This application also provides a computer program product, which is stored in a storage medium and, when executed by a computer, implements the control method of the cooling system provided in the above embodiments.

[0364] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element. Furthermore, it should be noted that the scope of the methods and apparatuses in the embodiments of this application is not limited to performing functions in the order shown or discussed, but may also include performing functions substantially simultaneously or in the reverse order, depending on the functions involved. For example, the described methods may be performed in a different order than described, and various steps may be added, omitted, or combined. Additionally, features described with reference to certain examples may be combined in other examples.

[0365] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a computer software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0366] The embodiments of this application have been described above with reference to the accompanying drawings. However, this application is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of this application without departing from the spirit and scope of the claims, and all of these forms are within the protection scope of this application.

Claims

1. A control method for a cooling system, characterized in that, include: Based on outdoor dry-bulb temperature data, equipment power information of the data center where the air-liquid co-source cooling system is located, and operating data of the air-liquid co-source hybrid cooling system, a system feature vector is generated; the above operating data includes relevant data of each functional component in the above air-liquid co-source hybrid cooling system. Based on the aforementioned system feature vector, at least one target control action matching the aforementioned system feature vector is determined from the candidate control action set to generate a control action sequence; the aforementioned candidate control action set includes at least one candidate control action, and the aforementioned candidate control action set is obtained based on the parameter range for each functional component in the aforementioned air-liquid homogeneous cooling system; the aforementioned control action sequence includes at least one target control action. Based on the above control action sequence, control instructions corresponding to each target control action are generated sequentially. These control instructions are used to instruct the corresponding functional components in the above-mentioned air-liquid co-source cooling system to perform the corresponding control actions.

2. The control method for the cooling system according to claim 1, characterized in that, Based on the outdoor dry-bulb temperature data, the equipment power information in the data of the air-liquid co-source cooling system, and the operating status data of the air-liquid co-source hybrid cooling system, a system feature vector is generated, including: Based on outdoor dry-bulb temperature data and equipment power information in the data of the air-liquid co-source cooling system, environmental disturbance variables are generated; Based on the secondary side water supply temperature data of the cooling capacity distribution unit in the liquid cooling subsystem of the above-mentioned air-liquid co-source cooling system, and the return air temperature data of the precision air conditioning in the above-mentioned air-liquid co-source cooling system, a thermodynamic state variable is generated. Based on the water pressure difference data of the primary side supply and return water network in the above-mentioned air-liquid co-source cooling system, hydraulic state variables are constructed. Based on the control action log of the previous control cycle of the above-mentioned air-liquid co-source cooling system, historical action state variables are constructed. The above-mentioned environmental disturbance variables, thermal state variables, hydraulic state variables, and historical action state variables are concatenated to generate the above-mentioned system feature vector.

3. The control method for the cooling system according to claim 1, characterized in that, Based on the aforementioned system feature vector, determining at least one target control action matching the aforementioned system feature vector from the candidate control action set includes: The above system feature vectors are subjected to normalization preprocessing to obtain standardized feature vectors; Input the standardized feature vectors mentioned above into the deep reinforcement learning model; When the deep reinforcement learning model is in exploration mode, at least one candidate control action is randomly selected from the candidate control action set as the target control action. When the deep reinforcement learning model is in exploitation mode, the standardized features are analyzed to obtain the state action value corresponding to each candidate control action. Based on the state action value, the target control action is selected from at least one candidate control action.

4. The control method for the cooling system according to claim 1, characterized in that, After generating the control instructions corresponding to each target control action sequentially based on the above control action sequence, the method further includes: Log data of the corresponding control actions performed by the corresponding functional components in the above-mentioned air-liquid co-source cooling system is collected, and a multi-dimensional comprehensive reward value is calculated based on the above-mentioned log data. The network parameters of the deep reinforcement learning model are updated based on the multi-dimensional comprehensive reward value.

5. The control method for the cooling system according to claim 4, characterized in that, The aforementioned log data includes: real-time operating power data of the cooling capacity distribution unit water pump, real-time operating power data of the precision air conditioner, and real-time operating power data of the common cooling source side; the multi-dimensional comprehensive reward value calculated based on the aforementioned log data includes: Calculate the total power consumption of the cooling system based on the log data above. Calculate the energy consumption bonus based on the total power consumption mentioned above; Security penalty items are selected based on the central processing unit temperature data or the ambient temperature of the data center mentioned above. The stability penalty term is calculated based on the difference data of each control action parameter between the current control cycle and the previous control cycle. The types of the above control action parameters include the water pump frequency parameter of the cooling capacity distribution unit, the primary valve opening parameter of the cooling capacity distribution unit, the air supply set temperature parameter of the precision air conditioner, and the primary valve opening parameter of the precision air conditioner. The energy consumption reward, safety penalty, and stability penalty mentioned above are summed to obtain the multi-dimensional comprehensive reward value.

6. The control method for the cooling system according to claim 5, characterized in that, The aforementioned security penalty items, selected based on the central processing unit temperature data or the ambient temperature of the data center, include: When the central processing unit temperature or the ambient temperature of the data center exceeds the preset safety threshold, a fixed penalty item will be selected. When the central processing unit temperature or the ambient temperature of the data center is within the preset temperature threshold range, a dynamic penalty term is selected based on the temperature deviation between the central processing unit temperature or the ambient temperature of the data center and the lower limit of the preset temperature threshold range. The dynamic penalty term is proportional to the deviation.

7. The control method for the cooling system according to claim 4, characterized in that, The above-mentioned updates to the network parameters of the deep reinforcement learning model based on the above-mentioned multi-dimensional comprehensive reward value include: The system feature vector of the current control cycle, the control action sequence, the multi-dimensional comprehensive reward value, and the system feature vector of the next control cycle are combined into a Markov transition sample. A predetermined number of Markov transition samples are randomly selected to form a training sample set; Calculate the target state action value corresponding to each training sample in the training sample set based on the Bellman equation; Calculate the temporal difference error between the state-action value output by the main value network and the target state-action value. Based on the aforementioned temporal difference error, gradient descent is performed on the aforementioned main value network to obtain the updated main value network weight parameters. The updated main value network weight parameters will be updated to the main value network according to the preset cycle.

8. The control method for the cooling system according to claim 4, characterized in that, Each control command carries at least one functional component parameter adjustment information; each functional component parameter adjustment information is used to indicate the adjustment of the parameter of one functional component in the above-mentioned air-liquid co-source cooling system; After generating the control instructions corresponding to each target control action in sequence, the method further includes: Based on the parameter adjustment information of at least one functional component carried in each control command, the parameters of the corresponding functional component are adjusted.

9. The control method for the cooling system according to claim 8, characterized in that, The aforementioned functional component adjustment parameter information includes at least one of the following: cooling capacity distribution unit water pump frequency parameter adjustment information; cooling capacity distribution unit primary side valve opening parameter adjustment information; air conditioning supply air temperature parameter adjustment information; air conditioning primary side inlet water valve opening parameter adjustment information.

10. The control method for the cooling system according to claim 1, characterized in that, Before generating the system feature vector based on outdoor dry-bulb temperature data, equipment power information in the data of the air-liquid co-source cooling system, and operating data of the air-liquid co-source hybrid cooling system, the above method also includes: A physical simulation environment for a co-source cooling system is established. The physical simulation environment includes a hydraulic model of the cooling capacity distribution unit, a heat transfer model of the cooling capacity distribution unit, a three-condition energy consumption model of the precision air conditioner, a primary side supply and return water network model, and a wind-liquid load coupling model. The initial deep reinforcement learning model was pre-trained in the aforementioned physical simulation environment to obtain the aforementioned deep reinforcement learning model.

11. The control method for the cooling system according to claim 1, characterized in that, The above methods also include: By dividing the parameter range of each parameter of each functional component in the above-mentioned air-liquid homogeneous cooling system into discrete levels, at least one discrete level is obtained, and one discrete level corresponds to one parameter range of one parameter in one functional component. Generate corresponding candidate control actions for each discrete level.

12. A control device for a cooling system, characterized in that, include: The first generation module is used to generate a system feature vector based on outdoor dry-bulb temperature data, equipment power information of the data center where the air-liquid co-source cooling system is located, and operating data of the air-liquid co-source hybrid cooling system; the above operating data includes relevant data of each functional component in the above air-liquid co-source hybrid cooling system. The determination module is used to determine at least one target control action that matches the system feature vector from the candidate control action set, based on the system feature vector, to generate a control action sequence; the candidate control action set includes at least one candidate control action, and the candidate control action set is obtained based on the parameter range of each functional component in the air-liquid homogeneous cooling system; the control action sequence includes at least one target control action. The second generation module is used to generate control instructions corresponding to each target control action in sequence based on the above control action sequence. The control instructions are used to instruct the corresponding functional components in the above air-liquid co-source cooling system to perform the corresponding control actions.

13. A co-source cooling system for air and liquid, which can use any one of the methods described in 1-10, includes a liquid cooling subsystem, an air cooling subsystem, and a common cold source side, wherein a co-source cooling system control device and a control device for the co-source cooling system are respectively communicatively connected to the liquid cooling subsystem, the air cooling subsystem, and the common cold source side.

14. An electronic device, characterized in that, It includes a processor and a memory, wherein the memory stores a program or instructions that can run on the processor, and the program or instructions, when executed by the processor, implement the control method of the cooling system as described in any one of claims 1-11.

15. A readable storage medium, characterized in that, The aforementioned readable storage medium stores a program or instructions that, when executed by a computer, implement the control method of the cooling system as described in any one of claims 1-11.

16. A computer program product, characterized in that, The aforementioned computer program product is stored in a storage medium, and when the aforementioned computer program product is executed by a computer, it implements the control method of the cooling system as described in any one of claims 1-11.