A liquid cooling system intelligent regulation and control method and system based on real-time feedback of a server baseboard management controller

Abstract

The application discloses a kind of based on server substrate management controller real-time feedback liquid cooling system intelligent regulation and control method and system, belong to data center liquid cooling regulation and control technical field.This method is from each server BMC chip real-time acquisition chip-level temperature, power consumption and utilization rate and other hardware health data by IPMI or Redfish protocol;Temperature sequence is constructed and temperature change rate is calculated, and thermal shock precursor characteristics are identified;PID feedback+feedforward compound control algorithm is used to calculate the opening of electric regulating valve corresponding to cabinet or server, while globally optimizing the total flow of liquid cooling system pump and the setting of liquid supply temperature, finally the control instruction is issued to the corresponding execution equipment.The application can respond to thermal shock in advance, reduce temperature overshoot, realize on-demand precise cooling and global energy efficiency optimization, and adapt to the heat dissipation demand of high-computing-power load scene.

Description

Technical Field

[0001] This invention belongs to the field of data center liquid cooling control technology, specifically relating to an intelligent control method and system for a liquid cooling system based on real-time feedback from a server baseboard management controller. Background Technology

[0002] With the rapid development of artificial intelligence and high-performance computing, the computing power demand and energy consumption of data centers are growing exponentially. Cooling systems account for 30%-40% of the total energy consumption of data centers, constituting a huge operating cost.

[0003] Current mainstream data center cooling technologies and control strategies suffer from the following significant technical deficiencies: Heat dissipation technology bottleneck: With the rapid increase in the heat density of CPU / GPU chips, traditional air cooling technology has approached its physical heat dissipation limit and cannot meet the heat dissipation requirements of high-power chips, making liquid cooling technology an inevitable choice.

[0004] Limited information sources for control strategies: Existing liquid cooling systems primarily rely on indirect parameters such as rack inlet / outlet temperatures and coolant flow rates for adjustment. However, the actual temperature change rate of the chip far exceeds that of the coolant. This lag-based control strategy, based on indirect parameters, cannot effectively cope with the instantaneous, severe, and nonlinear thermal shocks brought about by AI / HPC workloads, easily leading to localized overheating or system oscillations, such as... Figure 10 As shown.

[0005] The control granularity is too coarse: Existing solutions usually only control global parameters (such as total pump flow rate and chiller outlet water temperature), which cannot achieve precise on-demand cooling for different racks or even different servers.

[0006] Discrepancies exist between simulation modeling and real-world scenarios: In recent years, some thermal management solutions based on digital twin modeling and deep reinforcement learning have emerged. These solutions involve building high-fidelity physical models, training them on a large scale, and then deploying them to real-world systems. However, because simulation models cannot fully reproduce the complexity of the real physical world (Sim-to-Realgap), the policies learned in the training environment often deviate from expectations during actual deployment, requiring additional tuning and adaptation costs.

[0007] To improve the control precision of liquid cooling systems, the industry has explored coupled control techniques. For example, Chinese patent CN113157069A discloses a coupled heat dissipation control method and system for a server liquid cooling system. This method simultaneously collects coolant temperature and chip temperature, adjusts the opening of the electric regulating valve based on the difference between the coolant temperature and the set temperature, and adjusts the speed of the coolant inlet pump based on the difference between the chip temperature and the set temperature, thus expanding the control dimensions through dual-parameter coupled adjustment. However, this type of coupled control scheme still has the following unresolved technical limitations: First, the control logic only uses a simple threshold comparison switching adjustment, without introducing closed-loop feedback differential control and feedforward prediction mechanism. It cannot cope with the instantaneous nonlinear thermal shock brought by AI / HPC load, and the temperature overshoot is large. In essence, it is still a post-event response mode and does not fundamentally eliminate the control lag. Secondly, the decision input only focuses on the single parameter of temperature and does not combine multi-dimensional operating data such as real-time power consumption of the chip and CPU / GPU utilization to predict the thermal state trend. It is unable to intervene and regulate in advance in the early stage of sudden load changes and accelerated temperature rise, and its ability to cope with sudden thermal shocks is insufficient. Third, the control granularity is only for local coupling control of the electric regulating valve and liquid inlet pump of a single server. It does not establish a global collaborative optimization mechanism for multiple racks and multiple servers at the data center level. It cannot simultaneously optimize the total pump flow and liquid supply temperature when the overall load fluctuates, and it is difficult to take into account thermal safety and cooling efficiency at the whole system level. Fourth, the control parameters are fixed thresholds, which cannot adaptively match the control parameters according to the chip type and business load mode. This makes it difficult to adapt to the heat dissipation requirements of different working scenarios such as idle, inference, and training, resulting in insufficient scenario generalization.

[0008] Therefore, the market urgently needs an intelligent liquid cooling control method that can directly perceive multi-dimensional operating data at the chip level, has the ability to predict thermal shock, achieve global collaborative optimization, and adapt to different load scenarios. Summary of the Invention

[0009] The present invention aims to overcome the shortcomings of the prior art and provide an intelligent control method and system for liquid cooling system based on real-time feedback from server board management controller.

[0010] The above-mentioned objectives of the present invention are achieved by the following technical means: In a first aspect, the present invention provides an intelligent control method for a liquid cooling system based on real-time feedback from a server baseboard management controller, comprising the following steps: Data acquisition steps: Collect chip-level hardware health data in real time from the BMC chip of each server via IPMI or Redfish standard protocol, either in-band or out-of-band; the hardware health data includes at least chip temperature data, chip real-time power consumption, and CPU / GPU utilization; the chip temperature data includes at least any one of the following: CPU core temperature, CPU package temperature, GPU core temperature, GPU hotspot temperature, and GPU memory temperature; Thermal state sensing steps: The temperature data of each chip collected continuously are arranged in chronological order to form the temperature sequence of the corresponding chip. Trend analysis is performed on the temperature sequence of each chip to calculate the rate of temperature change. Identify early signs of thermal shock; On-demand cooling decision-making steps: Based on the current chip temperature, temperature deviation integral, and temperature change rate, a PID feedback + feedforward composite control algorithm is used to calculate the opening of the electric regulating valve for the corresponding rack or server; the control parameters of the PID feedback + feedforward composite control algorithm are dynamically adjusted according to the chip type and load mode. Global collaborative optimization steps: Analyze the chip temperature distribution and cooling requirements of all servers, and optimize the total pump flow and liquid supply temperature settings of the liquid cooling system. Command execution steps: Send the valve opening command to the electric regulating valve of the corresponding cabinet or server, and send the global control command, including the pump total flow setting and the liquid supply temperature setting, to the coolant distribution unit.

[0011] Furthermore, the PID feedback + feedforward composite control algorithm includes: Feedback control section: based on temperature deviation Deviation integral and the rate of temperature change The feedback control quantity is calculated using the PID control formula: In the formula, It is a feedback control variable. It is the target temperature. The temperature deviation at time t is the difference between the current temperature and the target temperature. The time integral of the temperature deviation This is the proportional gain coefficient, used for rapid response to temperature deviations. This is the integral gain coefficient, used to eliminate steady-state temperature deviation. The differential gain coefficient is used to suppress temperature change trends and reduce overshoot. The rate of temperature change of the chip; Feedforward control section: When the temperature change rate exceeds a preset threshold for two consecutive cycles, or the temperature change rate and CPU / GPU utilization increase significantly at the same time, or the second derivative of the difference between the chip temperature series and the temperature series is positive, feedforward control is triggered, and the feedforward control quantity is calculated. If a "thermal shock warning" is detected, a feedforward activation increment is superimposed. Final valve opening In the formula, This is the feedforward control variable. This is the feedforward gain coefficient. For the rate of temperature change of the chip, For valve opening, This is a limiting function that constrains the output value to a normalized range of 0 to 1, corresponding to the valve being fully closed to fully open.

[0012] Preferably, it also includes an anti-integral saturation control step: when the final valve opening reaches the normalized upper limit and the chip temperature continues to rise, the accumulation calculation of the integral term in the PID feedback control is frozen to avoid control saturation and temperature overshoot caused by continuous accumulation of integral; when the chip temperature falls back to the target temperature range, the integral term is gradually released and the integral regulation effect is restored to eliminate control lag and ensure smooth and stable valve regulation process.

[0013] Preferably, it also includes a parameter adaptation step: dynamically adjusting the parameters of the PID feedback + feedforward composite control according to the chip type and load mode, wherein the load mode includes idle mode, inference mode and training mode.

[0014] Furthermore, in the parameter adaptation step, the adjustment parameters for each load mode of the CPU and GPU adopt the optimal values ​​that have been experimentally calibrated. The calibration of the preferred parameter values ​​is based on the following: CPUs have higher thermal inertia and a relatively mild heating rate, so the integration effect is emphasized to ensure steady-state accuracy; GPUs have lower thermal inertia, a rapid and nonlinear temperature rise, so the feedforward and differential effects are emphasized to respond quickly to thermal shocks. Optionally, in the parameter adaptation step, the adjustment parameters for each load mode of the CPU and GPU are as follows: CPU idle mode: =0.3, =0.05, =0, =0.1; CPU inference mode: =0.5, =0.08, =0.15, =0.3; CPU training mode: =0.7, =0.12, =0.4, =0.5; GPU idle mode: =0.25, =0, =0, =0.1; GPU inference mode: =0.55, =0.09, =0.25, =0.4; GPU training mode: =0.8, =0.1, =0.5, =0.65.

[0015] Preferably, it also includes a digital twin-assisted calibration step: Collect no less than 30 days of historical BMC data, including chip temperature data, chip real-time power consumption, CPU / GPU utilization, valve opening records, pump flow rate, and liquid supply temperature, and build a data-driven chip temperature response model; optimize the control strategy parameters in a digital twin environment, and then deploy the optimized strategy to the actual system; The control strategy parameter optimization settings include: optimizing the opening of the electric regulating valve corresponding to the rack or server, optimizing the total pump flow and liquid supply temperature of the liquid cooling system, and optimizing the adjustment parameters of the CPU and GPU for each load mode.

[0016] Preferably, the thermal shock precursor feature identification includes: when the GPU temperature rises by more than 8°C within 10 seconds, or the rate of temperature change... >0.5℃ / s and CPU / GPU utilization When the value is greater than 20% / s, a feedforward early warning signal is triggered.

[0017] Preferably, the data acquisition step has an acquisition cycle of 5-30 seconds and supports both synchronous polling acquisition and asynchronous event subscription.

[0018] On the other hand, the present invention also provides an intelligent control system for a liquid cooling system based on real-time feedback from a server board management controller, characterized in that the intelligent control method for a liquid cooling system based on real-time feedback from a server board management controller includes the following modules disposed in the intelligent control controller: Data acquisition module: Used to collect chip-level hardware health data from the BMC chip of each server in real time; Thermal state sensing module: used to identify precursory features of thermal shock; On-demand cooling decision module: used to calculate the opening degree of the electric regulating valve for the corresponding rack or server; Global Collaborative Optimization Module: This module optimizes the parameters of the liquid cooling system by adjusting the total pump flow rate and liquid supply temperature settings based on the chip temperature distribution and cooling requirements of all servers. Command execution module: Used to send valve opening commands to the electric regulating valves of the corresponding cabinets or servers, and to send global control commands, including pump total flow setting and liquid supply temperature setting, to the coolant distribution unit.

[0019] Preferably, each rack's liquid inlet pipe or each server's cold plate inlet is equipped with an independent electric regulating valve.

[0020] The beneficial effects of adopting the above technical solution are as follows: It is important to emphasize that the various technical features of this invention are not simply an aggregation of isolated technical means, but rather constitute a complete technical closed loop of "chip-level real-time sensing → thermal shock precursor identification → valve-level precise control → system-level global optimization". BMC multi-dimensional data acquisition provides the data foundation for thermal state sensing, which in turn provides the triggering basis for feedforward control. Feedforward control and PID feedback work together to achieve a balance between rapid response and steady-state accuracy. Global collaborative optimization, based on valve-level control, achieves optimal system-level energy efficiency. Positive feedback and synergistic enhancement are formed between each link, collectively achieving an overall technical effect of "1+1>2". This is a systemic innovation of existing liquid cooling control technology, not a localized improvement. Specifically, compared with existing technologies, this invention has the following significant beneficial effects: 1. Compared to the threshold comparison-based switching regulation used in existing coupled control schemes, this invention employs a PID feedback + feedforward composite control algorithm. This algorithm ensures steady-state temperature accuracy through PID closed-loop control and intervenes early in the thermal shock phase through feedforward control. Experimental data shows that the thermal shock response delay of this scheme is only about 5 seconds, more than 80% shorter than the threshold-based coupled control scheme; the temperature overshoot is only 4℃~5℃, about 50% less than existing coupled control schemes, fundamentally solving the problems of lag and overshoot caused by reactive response.

[0021] 2. Existing coupling control schemes rely solely on coolant temperature and chip temperature as the basis for decision-making. This invention collects comprehensive hardware health data from the BMC chip, including chip temperature, real-time power consumption, and CPU / GPU utilization, through IPMI or Redfish standard protocols. By analyzing temperature sequence trends, it identifies early signs of thermal shock and can trigger regulation in the early stages of load surges and accelerated temperature rise, truly achieving proactive thermal management that "prevents problems before they occur."

[0022] 3. Existing coupled control schemes only perform local adjustment of valves and pumps on a single server. This invention constructs a two-level control architecture of "independent control of rack-level electric regulating valves + global optimization of system-level pump total flow and liquid supply temperature." Each rack / server can independently adjust its cooling capacity as needed, while the system parameters are dynamically optimized globally based on the overall heat load. Compared to single-server coupled control schemes, this invention can achieve an additional cooling energy saving of over 10% in data center deployment scenarios, while eliminating the risk of local hotspots.

[0023] 4. Existing coupled control schemes use fixed threshold parameters, which cannot adapt to different load scenarios. This invention supports dynamic adjustment of PID and feedforward parameters according to chip type and three load modes: idle, inference, and training. It strengthens the feedforward and differential functions to address the characteristics of GPUs with low thermal inertia and rapid temperature rise, and strengthens the integral steady-state accuracy to address the characteristics of CPUs with high thermal inertia, thus achieving refined thermal management with "one policy per chip and one policy per load".

[0024] 5. The control data source of this invention comes entirely from real-time feedback from a real server BMC, and the decision-making is based on the actual chip temperature state rather than approximate results under simulation conditions. An optional digital twin-assisted calibration module driven by real BMC historical data can be added, eliminating the need for complex processes of simulation, training, migration, and calibration. The accuracy of control is guaranteed upon deployment, and there is no deviation between the high-fidelity physical simulation model and the actual scenario.

[0025] 6. Each server in the data center has a built-in BMC chip and supports IPMI or Redfish protocols, which can obtain comprehensive chip-level health data without the need for additional hardware sensors; the standardized interface is widely compatible with various server brands and models, and supports unified monitoring and management across vendors and platforms; compared with complex solutions that require large-scale reinforcement learning training, the control algorithm of this invention has low computing power requirements and low operation and maintenance costs.

[0026] 7. This invention upgrades the data center cooling system from a "passive, indirect-parameter-responding consumable infrastructure" to an "intelligent asset management platform that actively senses the thermal state of chips." Its precise and efficient cooling capabilities directly reduce the data center's PUE, actively responding to the direction of green and sustainable development, and have significant commercial and social value. Attached Figure Description

[0027] Figure 1 This is a diagram illustrating the overall logical architecture of the technical solution of this invention.

[0028] Figure 2 A schematic diagram of chip-level sensor data acquired via IPMI or Redfish interface; Figure 3 This is a schematic diagram of the intelligent control execution process; Figure 4 Here is the flowchart of the PID feedback + feedforward composite control logic; Figure 5 A schematic diagram illustrating the principle of chip-level valve control and on-demand coolant distribution; Figure 6 This is a schematic diagram of the architecture of the data-driven digital twin-assisted calibration module of the present invention; Figure 7 This is a schematic diagram of the real-time closed-loop control process based on BMC direct sensing of the present invention. Figure 8 This is a comparison chart of the temperature control response speed of the present invention and the prior art; Figure 9 This is a comparison chart of the cooling energy consumption of the present invention and the prior art; Figure 10 This diagram illustrates the indirect parameter dependence and response lag of existing data center cooling solutions. Detailed Implementation

[0029] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present invention.

[0030] Many specific details are set forth in the following description in order to provide a full understanding of the invention. However, the invention may also be practiced in other ways different from those described herein, and those skilled in the art can make similar extensions without departing from the spirit of the invention. Therefore, the invention is not limited to the specific embodiments disclosed below.

[0031] English terms explained in Chinese: AI / HPC, Artificial Intelligence / High Performance Computing; Sim-to-Real gap, Simulation to Reality gap, the deviation between simulation and reality; IPMI, Intelligent Platform Management Interface; Redfish, Redfish Specification, Standard Hardware Management Interface Specification; BMC, Baseboard Management Controller; PID stands for Proportional-Integral-Derivative. SSH, Secure Shell, is a secure shell protocol. I2C, Inter-Integrated Circuit, is an internal integrated circuit bus. ADC, Analog-to-Digital Converter; TDP, Thermal Design Power; Modbus TCP, Modbus Transmission Control Protocol, is an industrial fieldbus standard based on the TCP protocol; BACnet IP, Building Automation and Control Network over Internet Protocol, is a building automation and control network protocol standard based on IP networks. CFD, Computational Fluid Dynamics; LSTM, Long Short-Term Memory; GRU, Gated Recurrent Unit; Dropout, Dropout Regularization, and random dropout regularization; Adam, Adaptive Moment Estimation, is an adaptive moment estimation optimizer. R 2 Coefficient of Determination.

[0032] This embodiment discloses a liquid cooling intelligent control method based on real-time feedback from the server baseboard management controller (BMC). It directly acquires real-time server chip-level data via the IPMI or Redfish standard protocol, and performs precise rack-level valve control on the liquid cooling system accordingly. This constructs a three-level intelligent control logic: "chip-level thermal state perception - rack-level valve precise control - global collaborative optimization." The overall system architecture is as follows: Figure 1 As shown.

[0033] The overall system consists of: one or more server racks (each server is equipped with a BMC chip), a liquid cooling circulation loop (including pumps, heat exchangers, pipes, and coolant distribution units), independent electric regulating valves installed on the liquid inlet pipes of each rack or at the cold plate inlet of each server, and an intelligent control controller based on real-time feedback from the BMC.

[0034] From the perspective of the overall system hierarchy, it can be divided into a three-level architecture: Hardware perception and execution layer: including BMC chip, CPU / GPU chip, electric regulating valve, pump / coolant distribution unit; Protocol and interface layer: including IPMI protocol, Redfish, SSH / command line; Application and intelligent decision-making layer: namely, intelligent control and controller, with built-in core functional modules.

[0035] The single-cycle execution chain of the above core control methods is as follows: Figure 3 As shown, the specific steps include: Data acquisition steps: Collect chip-level hardware health data from the BMC chip of each server in real time via IPMI or Redfish standard protocol, either in-band or out-of-band; the hardware health data includes chip temperature data, chip real-time power consumption, and CPU / GPU utilization, and the chip temperature data includes CPU core temperature, CPU package temperature, GPU core temperature, GPU hotspot temperature, and GPU memory temperature.

[0036] Thermal state sensing steps: The temperature data of each chip collected continuously are arranged in chronological order to form the temperature sequence of the corresponding chip. Trend analysis is performed on the temperature sequence of each chip to calculate the rate of temperature change. Identify the precursor characteristics of thermal shock.

[0037] On-demand cooling decision-making steps: Based on the current chip temperature, temperature deviation integral, and temperature change rate, a PID feedback + feedforward composite control algorithm is used to calculate the opening of the electric regulating valve for the corresponding rack or server.

[0038] Global collaborative optimization steps: Analyze the chip temperature distribution and cooling requirements of all servers, and optimize the total pump flow rate and liquid supply temperature settings of the liquid cooling system.

[0039] Command execution steps: Send the valve opening command to the electric regulating valve of the corresponding cabinet or server, and send the global control command, including the pump total flow setting and the liquid supply temperature setting, to the coolant distribution unit.

[0040] Corresponding to the above method, this embodiment also provides an intelligent control system for a liquid cooling system, which is located within an intelligent control controller and includes the following modules: Data acquisition module: Used to collect chip-level hardware health data from the BMC chip of each server in real time; Thermal state sensing module: used to identify precursory features of thermal shock; On-demand cooling decision module: used to calculate the opening degree of the electric regulating valve for the corresponding rack or server; Global Collaborative Optimization Module: Used to optimize the total pump flow and liquid supply temperature settings of the liquid cooling system based on the chip temperature distribution and cooling requirements of all servers; Command execution module: Used to send valve opening commands to the electric regulating valves of the corresponding cabinets or servers, and to send global control commands, including pump total flow setting and liquid supply temperature setting, to the coolant distribution unit.

[0041] 1. Implementation method of data acquisition module The intelligent control controller, through its built-in data acquisition module, relies on the IPMI or Redfish standard protocol to read hardware health data and operating status from the BMC chip of each server in in-band or out-of-band mode. The acquired chip-level sensor data types include... Figure 2 As shown.

[0042] Data Acquisition Example: The `ipmitoolsensorlist` command lists the monitored values ​​and thresholds of all sensors, covering information such as CPU temperature, voltage, fan speed, and power supply voltage. For specific sensors, detailed information can be obtained using the `ipmitoolsensorget` command. Each GPU contains up to six temperature sensors, including those for the core, memory, and hotspots. The `ipmitoolsensorlist` command can be used to view sensor data containing key fields such as "GPU_Temp" and "GPU_Power". The underlying driver acquires raw data from the hardware via interfaces such as I2C and ADC. Threshold sensors are responsible for acquiring continuously changing numerical information and are widely used in scenarios such as temperature, fan speed, voltage, and power consumption. The relevant acquisition execution process is as follows: Figure 2 As shown.

[0043] The data collected includes the following dimensions: Chip temperature data: CPU core temperature, CPU package temperature, GPU core temperature, GPU memory temperature, GPU hotspot temperature, as well as auxiliary temperature indicators such as memory temperature, SSD temperature, motherboard chipset temperature, and chassis intake / exhaust temperature. Power consumption metrics: CPU real-time power consumption, real-time power consumption of each GPU, and total system power consumption; Running status metrics: CPU utilization, GPU utilization, process queue length / number of tasks, fan speed; Other reference indicators: power module voltage / current, memory voltage, hard drive operating status, and PCIe device status.

[0044] The data collection strategy supports both synchronous polling and asynchronous event subscription. The collection frequency can be dynamically adjusted according to the system size, and the collection period is 5-30 seconds.

[0045] This invention features a built-in BMC chip in each server of the data center that supports IPMI or Redfish protocols, enabling comprehensive chip-level health data acquisition without the need for additional hardware sensors. The standardized interface is widely compatible with various server brands and models, supporting unified monitoring and management across vendors and platforms. Compared to complex solutions that require large-scale reinforcement learning training, the control algorithm of this invention has low computing power requirements and low operation and maintenance costs.

[0046] 2. Detailed implementation of each module of the intelligent control controller The intelligent control controller receives real-time data reported by the BMC, performs real-time analysis and prediction of the chip temperature of each server, and outputs control commands for the liquid cooling system accordingly. The controller has five built-in functional modules, and the overall system has a three-level architecture as follows: Figure 1 As shown. The controller has five built-in functional modules, and the specific implementation logic is as follows: (1) Data acquisition module It is responsible for establishing communication with the BMC of each server through IPMI or Redfish protocol, collecting chip-level hardware health data of all servers in parallel according to a preset cycle, completing data parsing, outlier filtering and caching, and providing basic data source for subsequent perception and decision-making modules.

[0047] (2) Thermal state sensing module The module collects temperature data from various chips over a continuous period and arranges them into temperature sequences for each chip in chronological order. It then tracks the real-time temperature change curves of the CPU / GPU chips in each server, identifying characteristics such as the rate of temperature rise, peak values, and fluctuation patterns. When a non-linear, rapid rise in chip temperature is detected, the module outputs a high-priority warning signal.

[0048] The rule for identifying precursor signs of thermal shock is: when the GPU temperature rises by more than 8°C within 10 seconds, or the rate of temperature change... >0.5℃ / s and CPU / GPU utilization When the value is greater than 20% / s, a feedforward early warning signal is triggered.

[0049] The determination of the threshold for identifying the precursor features of thermal shock mentioned above is based on the following: (2.1) Basis for determining the temperature change rate threshold of 0.5℃ / s This threshold was determined based on a joint analysis of the GPU chip's thermal time constant and the temperature rise characteristics of AI training loads. Typical GPUs (such as NVIDIA H100 / A100) typically exhibit a temperature change rate of no more than 0.05–0.1℃ / s in idle state; however, when an AI training task is loaded, the GPU power consumption jumps from idle power (approximately 35–55W) to near the TDP (approximately 700W) within 2–5 seconds, with a rapid increase in the temperature rise rate. Statistical analysis of measured temperature rise curves for various AI loads (GPT-type large model training, deep learning inference, etc.) revealed that when… If the temperature rises above 0.5℃ / s without intervention, the GPU temperature will rise from the safe range (≤75℃) to the danger range (≥85℃) within 15–20 seconds. However, there is an inherent delay of approximately 5–10 seconds between heat transfer and valve action in the liquid cooling system. Therefore, setting 0.5℃ / s as the feedforward trigger threshold allows sufficient cooling build-up time (approximately 10–15 seconds) before the temperature reaches the danger threshold, achieving "early response" rather than "post-event remediation."

[0050] (2.2) Basis for determining the utilization rate change threshold of 20% / s CPU / GPU utilization change rate This reflects the sudden increase in computational load. In AI training scenarios, operations such as loading large models and parallel data synchronization can cause GPU utilization to jump from near 0% to over 90% within seconds. Experimental statistics show that when... When the rate exceeds 20% / s, it is usually accompanied by a significant step change in power consumption (power consumption change rate). A temperature exceeding 200 W / s is a reliable precursor signal of an impending thermal shock. >20% / s and Using a combination of >0.5℃ / s as a composite triggering condition can effectively avoid false triggering of a single temperature threshold under noise interference and improve the reliability of feedforward control.

[0051] (2.3) Basis for determining the temperature rise of 8°C within 10 seconds This indicator is determined based on engineering experience regarding chip thermal safety margins. The recommended upper limit of operating temperature for mainstream GPUs (such as NVIDIA H100, A100, AMD MI300X, etc.) is 85℃, and the temperature overshoot protection threshold is 88-90℃. Within the safe temperature range (≤75℃), if the temperature rises by 8℃ within 10 seconds, it means the temperature rise rate is approximately 0.8℃ / s, far exceeding the normal fluctuation range, indicating that the chip is experiencing a severe thermal shock. Under these conditions, without prior intervention, the chip will reach the dangerous 85℃ threshold within the following 10-15 seconds. Therefore, setting "8℃ temperature rise within 10 seconds" as an independent trigger condition for thermal shock warning, along with... Thresholds form redundant checks to ensure the robustness of thermal shock precursor identification.

[0052] The calibration process for the above thresholds can be found in the measured temperature rise curve data of 8 H100 GPU servers in the idle → training mode jump experiment in the embodiment of this application, "Case 1".

[0053] This invention collects comprehensive hardware health data from the BMC chip via the IPMI or Redfish standard protocol, including chip temperature, real-time power consumption, and CPU / GPU utilization. By analyzing temperature sequence trends, it identifies early signs of thermal shock and can trigger regulation in the early stages of load surges and accelerated temperature rises, truly achieving proactive thermal management that "prevents problems before they occur."

[0054] (3) On-demand cooling decision module Based on chip-level sensing results, the required coolant flow rate is calculated for each rack (or each server), and the coolant is distributed on demand as follows: Figure 5 As shown. The decision-making logic includes: When the GPU chip temperature of a server exceeds the safety threshold or the temperature rise rate is abnormal, a high valve opening command is output to increase the coolant flow of the corresponding rack / server cold plate. When the server chip temperature is within a safe range and the trend is stable, a low valve opening command is output to reduce unnecessary cooling energy consumption. When the server is idle or under low load, it outputs the minimum maintaining valve opening and suggests to the global collaborative optimization module to reduce the total pump flow setting.

[0055] The core control algorithm adopts a PID feedback + feedforward composite control algorithm. The complete threshold determination and calculation output process is as follows: Figure 4 As shown, the detailed logic of each part is as follows: ① Control Structure A feedforward + feedback composite control structure is adopted: the feedback part is PID control, which calculates the control quantity based on temperature deviation, deviation integral, and temperature change rate; the feedforward part outputs additional control signals in advance based on the temperature change rate and load change characteristics.

[0056] ② Feedforward control logic (thermal shock precursor identification) Input signals include: the current rate of change of chip temperature. CPU / GPU utilization change rate The chip temperature sequence over several past cycles.

[0057] Feedforward triggering conditions (triggering occurs if any one of them is met): The rate of temperature change exceeds a preset threshold for two consecutive cycles; The rate of temperature change and CPU / GPU utilization both increased significantly at the same time (e.g.: >0.5℃ / s and >20% / s); The second derivative of the chip temperature sequence is positive (indicating accelerated heating).

[0058] The formula for calculating the feedforward control quantity is: In the formula, This is the feedforward control variable. This is the feedforward gain coefficient. To measure the chip temperature change rate, if a "thermal shock warning" is detected, an additional feedforward enable increment is applied. For example, by increasing the valve opening by 20% or by increasing it proportionally.

[0059] ③PID feedback control logic Input variables include: temperature deviation Deviation integral Rate of temperature change (As a differential term). The formula for calculating the feedback control quantity is: In the formula, Let be the feedback control quantity at time t; This is the current temperature. It is the target temperature. The temperature deviation at time t; The time integral of the temperature deviation; This is the proportional gain coefficient, used for rapid response to temperature deviations; This is the integral gain coefficient, used to eliminate steady-state temperature deviation; This is the differential gain coefficient, used to suppress temperature change trends and reduce overshoot; This represents the rate of temperature change of the chip.

[0060] The final valve opening is the sum of the feedback control quantity and the feedforward control quantity, with a limiting effect. In the formula, This is a limiting function that constrains the output value to a normalized range of 0 to 1, corresponding to the valve being fully closed to fully open.

[0061] This invention employs a PID feedback + feedforward composite control algorithm, which ensures steady-state temperature accuracy through PID closed-loop control and intervenes early in the initial stage of thermal shock through feedforward control.

[0062] It should be noted that the derivative term of the PID feedback control in this invention uses the chip temperature change rate. Non-temperature deviation rate of change Its technical significance lies in the fact that by directly using the absolute rate of temperature change as the differential input, it can more sensitively capture the rapid temperature rise trend in the early stage of thermal shock and apply inhibitory control in advance. Compared with the traditional method of using the rate of change of deviation as the differential term, this scheme responds to thermal shock more quickly and has a more significant overshoot suppression effect.

[0063] ④ Anti-integral saturation control When the final valve opening reaches the normalized upper limit and the chip temperature continues to rise, the accumulation calculation of the integral term in the PID feedback control is frozen to avoid control saturation and temperature overshoot caused by continuous accumulation of integral. When the chip temperature falls back to the target temperature range, the integral term is gradually released and the integral regulation effect is restored to eliminate control lag and ensure smooth and stable valve regulation process.

[0064] ⑤ Parameter Adaptation Mechanism The parameters of the PID feedback + feedforward composite control are dynamically adjusted based on the chip type and load mode. The load modes include idle mode, inference mode, and training mode, achieving refined thermal management with "one chip, one policy" and "one load, one policy." "One chip, one policy" refers to tuning a dedicated set of control parameters for the different thermophysical characteristics of the CPU and GPU. "One load, one policy" refers to dynamically matching corresponding control parameters for the three different business load modes: idle, inference, and training. These two approaches work together to achieve precise temperature control, effectively avoiding chip frequency throttling and performance throttling caused by thermal shock, ensuring the chip always remains within its optimal operating temperature range, thereby fully releasing computing power and ensuring stable output of operational performance.

[0065] The parameter adaptive decision framework is as follows: When the system detects that the chip's load state crosses the threshold boundary, the adaptive module automatically triggers parameter switching and simultaneously enables anti-jitter filtering (with a 5-10 second delay for confirmation to avoid frequent switching) to achieve a smooth transition. The parameter switching process employs a smooth interpolation mechanism to avoid valve oscillations caused by sudden parameter changes.

[0066] Recommended values ​​and tuning logic for CPU parameters under various load modes: CPUs have significant thermal inertia and a relatively mild heating rate, requiring high steady-state accuracy. Parameter tuning is based on the characteristics of "first-order inertia + pure time delay." Here, "first-order inertia + pure time delay" refers to the CPU's temperature response to cooling flow regulation exhibiting a gradual change characteristic of a first-order inertial system, accompanied by pure time delays caused by liquid flow and heat conduction. This thermal response model is the core basis for tuning the control parameters of the CPU under various load modes.

[0067] Recommended values ​​and tuning logic for GPU parameters under various load modes: GPUs have lower thermal inertia, faster response, higher power density, more dramatic and non-linear temperature rise, and lower target temperature threshold, thus enhancing feedforward and differential effects.

[0068] Synergistic mechanism for anti-integral saturation and parameter adaptation The anti-integral saturation control and the parameter adaptive mechanism form a "fast and slow dual-loop" cooperative structure on the time scale, and their interaction logic is as follows: A. Division of labor based on time scale The parameter adaptive mechanism operates on a "slow time scale" (switching confirmation delay of 5-10 seconds, parameter smooth transition time constant τ=3-5 seconds), responsible for adjusting the baseline value of the control gain according to the macroscopic changes in the chip load mode (idle → inference → training). The anti-integral saturation control operates on a "fast time scale" (real-time judgment and execution in each control cycle), responsible for temporarily freezing integral accumulation under extreme conditions such as valve opening saturation to prevent control performance degradation.

[0069] B. Priority and Mutual Exclusion Logic Under normal operating conditions: The parameter adaptive mechanism is active, dynamically updating based on the real-time detected load pattern. , , , The baseline value; anti-integral saturation control is in standby mode (under condition monitoring, not triggered).

[0070] Saturation condition: When it is detected that "the final valve opening has reached the normalized upper limit ( And the chip temperature continues to rise. When ")" is triggered, anti-integral saturation control is activated, immediately freezing the accumulation of the integral term. (Keep the current value unchanged). In this state, the parameter adaptation mechanism continues to operate—that is, it will still update when the load mode switches. , , The benchmark value, but The update is temporarily ineffective (the points system is frozen); it will take effect after the saturation period ends. The new benchmark values ​​will gradually take effect.

[0071] Integral processing during parameter switching: When parameter adaptation triggers control parameter switching (such as switching from inference mode to training mode). The baseline value has changed. To avoid valve oscillation caused by a sudden change in the integral term, the following smooth transition strategy is adopted: New integral term value = Old integral term value × (Old... / new The attenuation factor λ is calculated as follows: λ decreases linearly from 1.0 to 0.95 within 5 control cycles after parameter switching, so that the integral term is slightly released after parameter switching, avoiding control oscillations caused by the superposition of "integral saturation" and "parameter mutation".

[0072] C. Synergistic Effect The "fast and slow dual-loop" collaborative mechanism ensures that: when the load mode is stable, parameter adaptive optimization continuously optimizes the control gain, while anti-integral saturation serves as a safety baseline to prevent runaway under extreme conditions; during load mode switching, smooth parameter transition and coordinated release of the integral term avoid abrupt control changes; when the valve saturates, anti-integral saturation prioritizes system stability, while parameter adaptive operation continues in the background, quickly restoring optimal control after saturation is relieved. These two mechanisms work together rather than conflict, jointly achieving robust temperature control across all operating conditions.

[0073] This invention supports dynamic adjustment of PID and feedforward parameters based on chip type and three load modes: idle, inference, and training. It enhances the feedforward and differential functions to address the characteristics of GPUs with low thermal inertia and rapid temperature rise, and enhances the integral steady-state accuracy to address the characteristics of CPUs with high thermal inertia, thereby achieving refined thermal management with "one strategy per chip and one strategy per load".

[0074] (4) Global Cooperative Optimization Module Building upon rack-level fine-tuning, global parameters—total pump flow rate and liquid supply temperature settings—are simultaneously optimized to achieve dual-dimensional synergy at the system and chip levels. When multiple high-load servers require intensive cooling simultaneously, the total pump flow rate is appropriately increased and / or the liquid supply temperature is reduced to ensure overall cooling capacity. When the overall load is low, the pump speed is significantly reduced (pump power consumption is proportional to the cube of flow rate), while valve regulation ensures cooling of hot spots, achieving optimal energy efficiency.

[0075] (5) Instruction execution module Responsible for distributing and executing decision results via industrial fieldbus: Valve opening command: The valve opening command of each cabinet / server is sent to the corresponding electric control valve via industrial fieldbus such as ModbusTCP and BACnetIP; Global control command: Send the total pump flow rate setting and supply temperature setting to the coolant distribution unit (CDU) to complete system-level collaborative optimization.

[0076] Each rack's liquid inlet pipe or each server's cold plate inlet is equipped with an independent electric regulating valve, supporting continuous opening adjustment from 0% to 100%.

[0077] 3. Optional digital twin-assisted calibration module This invention can optionally be configured with a digital twin-assisted calibration module for strategy verification and parameter optimization, with the architecture as follows: Figure 6 As shown. This module is built entirely based on real historical BMC data, and there is no deviation between the physical simulation and the actual system.

[0078] (1) Construction process of data-driven twin model Step 1: Historical Data Collection and Cleaning Collect at least 30 days of real-time BMC data, including chip temperature (CPU / GPU), chip power consumption, CPU / GPU utilization, valve opening records, pump flow rate, and liquid supply temperature; complete data cleaning by removing outliers and filling in missing values ​​using linear interpolation.

[0079] The data preprocessing and validation methods in the construction of digital twin models are as follows: ① Data preprocessing process Outlier Detection and Handling: Outliers are identified using the 3σ criterion (Raida criterion), which involves calculating the mean for each sensor field. and standard deviation It will fall Data points outside the specified interval are marked as outliers. For isolated outliers (data is normal before and after the interval), linear interpolation is used as a replacement; for continuous outliers (lasting for more than 10 collection cycles), they are marked as invalid data segments and removed entirely to avoid introducing false information through interpolation.

[0080] Missing value imputation: For randomly distributed missing values ​​(missing rate < 5%), linear interpolation is used for imputation; for consecutive missing values ​​(missing rate ≥ 5%), a combination of forward imputation and backward imputation is used, that is, the weighted average of the mean of the first 3 valid values ​​and the mean of the last 3 valid values ​​of the missing segment is used for imputation.

[0081] Data normalization: Standardize all input features by field using Z-Score, i.e. ,in and These are the mean and standard deviation of the field on the training set, respectively. Standardized input features are all of the same magnitude, avoiding the adverse effects of differences in scale on LSTM model training.

[0082] Time window sliding window construction: Supervised learning samples are constructed using a sliding window approach. The window width W = 10 time steps (corresponding to 150 seconds of historical data), and the prediction step size H = 1 time step (corresponding to the temperature value 15 seconds later). The input feature dimension of each sample is W × (number of input features), and the output target is the temperature value at the next time step. .

[0083] ② Dataset partitioning and validation methods The cleaned dataset was divided into three parts in chronological order: a training set (the first 60% of the data, used for model parameter learning), a validation set (the middle 20% of the data, used for hyperparameter tuning and early stopping control), and a test set (the last 20% of the data, used for final model performance evaluation). The reason for using time-series partitioning instead of random partitioning is to avoid future data "leaking" into the training process and to ensure the model's generalization performance in real-world deployment scenarios (i.e., using historical data to predict future data).

[0084] ③ Model training and regularization The LSTM model employs a three-layer stacked structure with 64 hidden units per layer. The Dropout rate is set to 0.2 (applied only to non-output layers). Early stopping is used to prevent overfitting: training terminates when the validation set loss no longer decreases for 15 consecutive epochs, and the model weights are rolled back to the epoch with the minimum validation set loss. The optimizer is Adam, with an initial learning rate of 0.001. Cosine AnnealingLR is used to gradually reduce the learning rate during training. .

[0085] Step 2: Data-driven modeling A chip temperature response model to control input is constructed using system identification or deep learning methods: Input feature: Chip temperature at the previous moment Current valve opening Current pump flow rate Current liquid supply temperature Current chip power consumption ; Output target: Chip temperature at the next moment ; Recommended model architecture: linear regression (fast and interpretable) or LSTM / GRU (suitable for nonlinear and long-term dependencies).

[0086] Step 3: Twin Environment Construction The trained model is encapsulated as an interactive temperature predictor and embedded in a virtual simulation layer of the control loop, replacing traditional CFD or thermophysical models. It supports single-step prediction, multi-step rolling prediction, strategy playback and comparison.

[0087] (2) Twin-assisted calibration process (offline mode) Strategy replay: Input historical real load sequences, simulate the behavior of the controller in a twin environment, and compare the peak temperature, energy consumption, and oscillation amplitude under different control parameters; Parameter optimization: Use grid search, Bayesian optimization or reinforcement learning to find the optimal PID feedback + feedforward parameters in the twin environment. The optimization objective is the integral of temperature deviation (IAE) + valve action cost + energy consumption. Policy validation: Verify the robustness of the control policy under multiple different load scenarios (training / inference / burst load).

[0088] After optimization, the optimized strategy will be deployed to the actual system. The optimization targets include the opening parameters of the electric regulating valves of the corresponding racks or servers, the total pump flow and liquid supply temperature setting parameters of the liquid cooling system, and the adjustment parameters of the GPU for each load mode.

[0089] 4. Intelligent control closed-loop process In each control cycle, the system executes a closed-loop control process, such as... Figure 7 As shown: Step 1 (Acquisition): The data acquisition module sends sensor query commands to each server BMC via IPMI or Redfish interface to obtain hardware health data such as chip temperature, power consumption, and utilization rate for the current period.

[0090] Step 2 (Sensing): The thermal state sensing module constructs a temperature sequence from the continuously collected chip temperature data in chronological order, performs trend analysis on the temperature sequence, calculates the rate of temperature change, and identifies any precursory features of an impending thermal shock.

[0091] Step 3 (Decision): The on-demand cooling decision module takes the current chip temperature and temperature change rate as core inputs, and calculates the required flow rate for the corresponding rack / server by combining the temperature safety threshold of each chip; at the same time, the global collaborative optimization module counts the chip temperature distribution of all servers and determines the system-level pump flow rate and liquid supply temperature.

[0092] Step 4 (Execution): The instruction execution module sends valve instructions to the electric regulating valves of the corresponding racks / servers and sends global instructions to the coolant distribution unit.

[0093] Step 5 (Feedback): Return to Step 1 and enter the next control cycle to achieve closed-loop adaptive regulation. 5. Specific Implementation Examples

[0095] This embodiment implements the method of the present invention in a liquid-cooled data center containing N GPU servers. Each server is equipped with a BMC chip, and each server in the liquid cooling system is equipped with an independent electric regulating valve (installed before the cold plate inlet). The pump and cold source are uniformly controlled by the coolant distribution unit. The system architecture is as follows. Figure 1 As shown.

[0096] Step 1: Establish BMC data acquisition channel Collect the BMC management IP address, administrator username, and password of all servers in the data center; deploy intelligent control controllers on the data center servers, and establish secure connections with the BMC of each server through their built-in data acquisition modules using the IPMI or Redfish standard protocol; configure the acquisition cycle to 15 seconds, and collect indicators including the temperature of each CPU core, CPU package temperature, the temperature of each GPU core, GPU memory temperature, GPU hotspot temperature, real-time power consumption of each GPU, total power consumption, CPU / GPU utilization, system load queue length, and chassis inlet / outlet temperature.

[0097] The `ipmitoolsensorlist` command was used to test sensor data readout, confirming that data containing key fields such as "CPU_Temp", "GPU_Temp", and "GPU_Power" could be acquired. The data acquisition process is as follows: Figure 2 As shown.

[0098] Step 2: Set the chip temperature safety threshold and control strategy Three temperature thresholds are set for each chip: Safe operating range: CPU ≤ 85℃, GPU ≤ 75℃; Warning range: CPU, 85℃~95℃; GPU, 75℃~85℃; Danger zone: CPU ≥ 95℃, GPU ≥ 85℃ (triggers emergency frequency reduction or forced full-flow cooling).

[0099] Define a temperature change rate threshold: set a "thermal shock warning" signal if the temperature rises by more than 8°C within 10 seconds.

[0100] Step 3: Deploy intelligent control controllers The intelligent control and control system runs on a central control server and includes five core functional modules. The overall system has a three-tier architecture as follows: Figure 1 As shown: Thermal state sensing module: This module performs sliding window smoothing on chip temperature data collected every 15 seconds to create a temperature sequence, calculating real-time temperature values ​​and the rate of temperature change. When the rate of temperature change exceeds a preset threshold, it triggers a feedforward control signal, sending a pre-opening command for the valves. For example, if a GPU chip's temperature rises from 65℃ to 73℃ within 10 seconds (dT / dt≈0.8℃ / s), the module immediately outputs a 30%~50% pre-opening command to the corresponding valve, establishing a cooling margin before the temperature reaches the danger zone.

[0101] On-demand cooling decision module: Calculates valve opening commands based on chip-level sensing results. The valve opening calculation formula is as follows: ,in This represents the deviation between the current temperature and the target temperature. The time integral of the temperature deviation For the rate of temperature change, The task load weight is used; in multi-chip scenarios, the result with the highest cooling demand among all chips is taken as the final valve opening to ensure that the hottest chip is adequately cooled; the valve opening is normalized to the range of [0,1] and then sent via Modbus TCP protocol. The PID control algorithm for calculating the valve opening is explained below: The above function expression is not a new algorithm independent of the aforementioned composite control formula, but rather a derivative of it. and A unified summary of the physical quantities required for the calculation, and the formula for the final valve opening. These are different expressions of the same PID feedback + feedforward composite control algorithm, and their technical content is completely corresponding: Among them... For the input of the proportional term in feedback control; This is the input for the integral term; Simultaneously serving as the basis for calculating the feedback differential term, feedforward trigger determination, and feedforward control quantity; weight parameters Then it is responsible for dynamically allocating resources according to the load mode. , , , and The specific value is not an independently calculated numerical variable, but rather a multi-dimensional indicator based on real-time data collected by the BMC, including CPU / GPU utilization, chip power consumption as a percentage of TDP, and temperature change rate. After 5-10 seconds of anti-jitter filtering to determine the chip's current idle, inference, or training load mode, the control parameter group for that load mode under the corresponding chip type is mapped and matched. This group is then substituted into the composite control formula in the form of control gain weights to participate in the valve opening calculation. The parameter switching process uses a smooth interpolation mechanism to avoid valve oscillation caused by parameter abrupt changes.

[0102] Global collaborative optimization module: Statistically analyzes the temperature distribution and valve opening of all chips, calculates the total pump flow setting (positively correlated with the total valve opening and the overall thermal load of the chips) and the liquid supply temperature setting (to maximize the liquid supply temperature to save cold source energy consumption while ensuring thermal safety).

[0103] This invention constructs a two-level control architecture of "independent control of rack-level electric regulating valves + global optimization of system-level pump total flow and liquid supply temperature". Each rack / server can independently adjust the cooling capacity as needed, while the system parameters are dynamically optimized at the global level based on the overall heat load.

[0104] Step 4: Closed-loop control operation The closed-loop process of intelligent control strategy is as follows: Figure 7 As shown: At time T0 (collection): The data acquisition module initiates parallel queries to all server BMCs to collect data for the current period. The collection time is approximately 2-5 seconds. T0+5s (Perception and Decision): The controller reads the collected data and completes temperature sequence construction, temperature change rate calculation, thermal shock identification, valve opening calculation, and global parameter calculation within 1 second; T0+6s (Execution): The instruction execution module sends the valve opening instruction to the corresponding electric regulating valve and the global setting to the coolant distribution unit controller; T0+21s (next cycle): Enter the next control cycle and repeat the above process to achieve continuous closed-loop control.

[0105] Step 5: Optional – Digital Twin-Assisted Calibration (Offline Mode) A data-driven model of data center thermal behavior was constructed using 31 days of historically collected BMC data. The control strategy was backtested and parameters were fine-tuned offline in a twin environment before being deployed to the actual system. Because the twin model is built based on real historical BMC data, the optimization results do not suffer from the discrepancy between simulation and reality. The architecture is as follows: Figure 6 As shown.

[0106] 6. Experimental Verification Case Case 1: Experiment on Thermal Shock Response to Sudden Load Increase in GPU AI Training Server Cluster (1) Experimental environment and system configuration This experiment was conducted in the liquid-cooled pilot server room of the Intelligent Computing Center, deploying 8 high-density GPU AI training servers, distributed across 2 standard 42U liquid-cooled racks (4 servers per rack). The server room adopted a heat dissipation architecture of cold plate liquid cooling + outdoor cooling tower. The core components of the liquid cooling system are configured as follows: Initial control parameter settings (previously set in the intelligent control controller before the test begins): System stable operating status (idle mode baseline data): When the server is in an idle state (idle mode is determined by: GPU utilization ≤10%, power consumption ratio relative to TDP ≤20%, and temperature change rate dT / dt ≤0.1℃ / s), the system operating data is shown in the table below: Global parameters of the liquid cooling system (idle state): Hardware operation data of a single H100 GPU server (idle state, collected in real time via BMC / Redfish): (2) Thermal shock event response: Idle → Training mode abruptly At the 5th minute, a server-wide AI training task was manually triggered—loading a large GPT-like model training job. All 64 H100 GPUs on the 8 servers entered training mode (training mode determination criteria: GPU utilization ≥ 80%, power consumption ratio relative to TDP ≥ 85%, temperature change rate dT / dt ≥ 0.5℃ / s). The system recorded the following process in real time: Thermal shock precursor identification stage (T0~T0+5s): Numerical Explanation: This set of data comes from the GPU thermal shock feedforward control calculation stage of the experimental case, corresponding to the calculation formula of the feedforward control in this scheme. The source and calculation logic of each value are as follows: Where 0.65 is the feedforward gain coefficient in GPU training mode. , is a fixed control parameter pre-tuned for the GPU training scenario in the parameter adaptive mechanism, matching the high heat density and rapid temperature rise thermophysical characteristics of the GPU, used to linearly map the temperature change rate to the valve opening adjustment amount; 0.61℃ / s is the GPU temperature change rate calculated in real time at time T0+4s. The temperature sequence, obtained by the thermal state sensing module based on continuous periodic acquisition of the BMC, is calculated using differential operations. This value exceeds the feedforward trigger threshold of 0.5℃ / s; 0.20 is the preset feedforward activation increment. This is an additional fixed opening increment that is superimposed when a thermal shock warning is triggered, used to quickly establish a cooling margin. The three are controlled according to the feedforward control formula. Substituting the values ​​into the calculation, the result is the feedforward control quantity under this thermal shock scenario. After being superimposed with the PID feedback control quantity and subjected to amplitude limiting, the output is the final valve opening command.

[0107] Response control phase (T0+5s~T0+10s): Steady-state running data in training mode (after the algorithm has fully taken effect): (3) Comparison of algorithm response effects To verify the superiority of the control algorithm of this invention, three sets of comparative experiments were set up under exactly the same server configuration and AI training load: Experimental conclusion: Effective identification of thermal shock precursors: The response hysteresis principle of traditional indirect parameter control schemes is as follows Figure 10As shown, the indirect sensing based on coolant temperature suffers from significant heat conduction delay; this solution, based on direct acquisition at the BMC chip level, eliminates this lag at its source. A measured comparison of the response speeds of the two solutions is as follows: Figure 8 As shown.

[0108] Figure 8 This is a comparison chart of the temperature control response speed of the present invention and existing technologies. The horizontal axis represents time (seconds), and the vertical axis represents the GPU core temperature (°C). The comparison objects are: ① constant open-loop control (valve opening fixed at 45%); ② threshold coupling control scheme of CN113157069A; ③ the present invention's scheme (PID feedback + feedforward composite control based on BMC real-time sensing). Experimental conditions: 8 H100 GPU AI training servers, loaded with GPT-type large model training tasks, liquid supply temperature 42°C, acquisition period 15 seconds, room temperature 25°C. The data comes from the actual test record of continuous operation for 30 minutes in "Case 1" of the embodiment of this application. As can be seen from the figure, the present invention begins to respond to the temperature rise about 5 seconds after the load surge, and the peak temperature is controlled within 84°C, while the threshold coupling control scheme has a response delay of about 30 seconds, and the peak temperature is about 90°C. It has sufficient cooling margin to cope with the impact before the temperature reaches the 85°C alarm line, achieving true "early response" rather than post-event remediation. The thermal shock response delay is only 5 seconds, which is more than 80% shorter than the traditional control scheme, completely eliminating the inherent time delay drawback of traditional indirect sensing control.

[0109] PID feedback + feedforward composite control achieves precise temperature control of ±3℃: Thanks to the synergy of feedforward and feedback, the system controls the peak GPU temperature to 84℃ during the peak of thermal shock (about 8℃ lower than the traditional solution), and the steady-state temperature is stable within ±3℃. The temperature overshoot is only 4℃ / 5℃ (about 50% less than the traditional solution of 8℃ / 10℃). The system does not reach the dangerous threshold of 85℃ throughout the process, effectively ensuring the thermal safety and performance stability of the AI ​​training cluster.

[0110] The cooling system achieves approximately 22% energy savings: through precise chip-level sensing and on-demand control, the system adaptively adjusts the total pump flow and valve opening based on actual needs, ensuring the chip temperature remains safe throughout the process. Figure 9 As shown, Figure 9 This is a comparison chart of cooling energy consumption between the present invention and existing technologies. The horizontal axis represents the load mode (idle, inference, training), and the vertical axis represents the total power consumption of the liquid cooling system (including pump, CDU, and cold source). Comparison schemes: ① Constant open loop (valve fixed at 45%); ② CN113157069A threshold coupling control; ③ The present invention (BMC + PID + feedforward + two-stage regulation). Experimental conditions are the same. Figure 8Data shows that under idle load, the total power consumption of this invention is 8.5kW, which is 15% lower than that of constant open-loop; under inference load, the total power consumption of this invention is 20.5kW, which is about 25% lower than that of constant open-loop; under training load, the total power consumption of this invention is 32.0kW, which is about 22% lower than that of constant open-loop (41.0kW) and about 12% lower than that of threshold coupling (36.5kW), verifying the energy efficiency advantages of this invention at the data center level.

[0111] Experimental data shows that the thermal shock response delay of this solution is only about 5 seconds, which is more than 80% shorter than the threshold-based coupled control solution; the temperature overshoot is only 4℃~5℃, which is about 50% less than the existing coupled control solution, fundamentally solving the lag and overshoot problems caused by after-the-fact response. Compared with the single-server coupled control solution, this invention can achieve an additional cooling energy saving of more than 10% in data center-level deployment scenarios, while eliminating the risk of local hotspots.

[0112] Case Study 2: Data-Driven Digital Twin-Assisted Calibration – Offline Parameter Optimization (GPU Component) Digital Twin Construction Process Data preparation stage: Historical BMC data was collected from an existing liquid-cooled data center (configuration: 4 A100 servers from a certain company, with a cold plate liquid cooling system). The continuous collection period was 45 days, resulting in approximately 180,000 records. Data cleaning removed abnormal data points caused by temporary manual intervention, downtime maintenance, etc., achieving an effective data retention rate of approximately 94%. The specific sensor data types and coverage areas collected are as follows: Model training and calibration: Based on the aforementioned 45 days of historical data, an LSTM model was used to construct a device-temperature response model. Input features included the current temperature T(t), valve opening v(t), coolant distribution unit flow rate m_flow(t), supply temperature T_fluid(t), and current power consumption P(t). The output target was the temperature T(t+1) at the next moment. After grid search optimization, a three-layer LSTM structure (64-dimensional hidden layers) was adopted with Dropout regularization of 0.2 to prevent overfitting. The Adam optimizer had a learning rate of 0.001 and a batch size of 32. After training, the model achieved a mean squared error (MSE) of <0.12 on the remaining 15 days of test data, with a goodness-of-fit R-value of R0. 2 =0.94.

[0113] (2) Offline parameter optimization effect In a digital twin environment, the control parameters for GPU training modes ( A Bayesian optimization search was performed, with the optimization objective being J = α・IAE + β・valve actuation amplitude + γ・energy consumption estimation, and weights α = 0.5, β = 0.2, and γ = 0.3. The same real load sequence (including typical thermal shock scenarios) was used for each strategy replay to ensure the comparability of the optimization results. The optimized parameters were deployed to the GPU training mode control of a real system (GPU portion) and validated under the same load scenario, comparing the measured control effects before and after optimization: (3) Validation of the key value of twin calibration Twin calibration conclusion: This digital twin calibration scheme is built entirely based on real historical BMC data. The daily average deviation between the twin model's predicted temperature and the actual temperature is controlled within 0.8℃. In typical test scenarios, the deviation between the expected performance of Sim-to-RealGap optimization parameters and the measured performance of the real system is less than 3%. Bayesian optimization in the digital twin environment... After fine-tuning and optimized parameter deployment, the measured peak temperature in GPU training mode decreased by another 1.9℃, temperature overshoot decreased by 28%, and cooling energy consumption decreased by approximately 6%. This means: By using digital twin-assisted calibration, this invention can complete large-scale parameter space search without making any changes to the real system; the optimization results can take effect without additional calibration when deployed in reality, realizing "twin is real"; compared with traditional physical simulation schemes, it avoids complex fluid dynamics modeling and huge computing power requirements, while avoiding the deviation problem between the simulation model and the actual system.

[0114] The control data source of this invention comes entirely from real-time feedback from a real server BMC, and the decision-making is based on the actual chip temperature state rather than approximate results from a simulation environment. An optional digital twin-assisted calibration module driven by real BMC historical data can be added, eliminating the need for complex processes of simulation, training, migration, and calibration. This ensures control accuracy upon deployment and avoids any discrepancies between the high-fidelity physical simulation model and the actual scenario.

[0115] Overall conclusion of the experiment: The two cases above respectively verify two key aspects of the core technical solution of this invention: Case 1: By constructing an actual liquid-cooled cluster of GPU AI training servers, the "second-level response" capability of the PID feedback + feedforward composite control algorithm based on BMC real-time perception under thermal shock scenarios was verified (response latency was reduced by about 85%, and temperature overshoot was reduced by about 50%), and about 22% cooling energy saving was achieved while ensuring that the chip temperature did not reach the dangerous threshold throughout the process.

[0116] Case 2: By constructing a data-driven digital twin model based on real BMC historical data, the "zero bias" advantage of the twin-assisted calibration module was verified (predicted temperature difference <0.8℃, Sim-to-RealGap completely eliminated). After completing multi-parameter joint optimization in the twin environment, the peak measured temperature of the real system was reduced by another 1.9℃, and the cooling energy consumption was reduced by about 6%, providing solid technical support for long-cycle parameter adaptation and continuous optimization.

[0117] This invention upgrades data center cooling systems from "reactive, indirect-parameter-responding consumable infrastructure" to "an intelligent asset management platform that proactively senses chip thermal status." Its precise and efficient cooling capabilities directly reduce the data center's Power Usage Effectiveness (PUE), actively responding to green and sustainable development trends, and possessing significant commercial and social value.

[0118] The above description is merely a preferred embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for intelligent control of a liquid cooling system based on real-time feedback from a server board management controller, characterized in that, Includes the following steps: Data acquisition steps: Collect chip-level hardware health data from the BMC chip of each server in real time via IPMI or Redfish standard protocol, either in-band or out-of-band; the hardware health data includes at least chip temperature data, chip real-time power consumption, and CPU / GPU utilization. Thermal state sensing steps: The temperature data of each chip collected continuously are arranged in chronological order to form the temperature sequence of the corresponding chip. Trend analysis is performed on the temperature sequence of each chip to calculate the rate of temperature change. Identify early signs of thermal shock; On-demand cooling decision-making steps: Based on the current chip temperature, temperature deviation integral, and temperature change rate, a PID feedback + feedforward composite control algorithm is used to calculate the opening of the electric regulating valve for the corresponding rack or server; the control parameters of the PID feedback + feedforward composite control algorithm are dynamically adjusted according to the chip type and load mode. Global collaborative optimization steps: Analyze the chip temperature distribution and cooling requirements of all servers, and optimize the total pump flow and liquid supply temperature settings of the liquid cooling system. Command execution steps: Send the valve opening command to the electric regulating valve of the corresponding cabinet or server, and send the global control command, including the pump total flow setting and the liquid supply temperature setting, to the coolant distribution unit.

2. The intelligent control method for a liquid cooling system based on real-time feedback from a server board management controller according to claim 1, characterized in that, The PID feedback + feedforward composite control algorithm includes: Feedback control section: based on temperature deviation Deviation integral and the rate of temperature change The feedback control quantity is calculated using the PID control formula: In the formula, It is a feedback control variable. It is the target temperature. The temperature deviation at time t is the difference between the current temperature and the target temperature. The time integral of the temperature deviation This is the proportional gain coefficient, used for rapid response to temperature deviations. This is the integral gain coefficient, used to eliminate steady-state temperature deviation. The differential gain coefficient is used to suppress temperature change trends and reduce overshoot. The rate of temperature change of the chip; Feedforward control section: When the temperature change rate exceeds a preset threshold for two consecutive cycles, or the temperature change rate and CPU / GPU utilization increase significantly at the same time, or the second derivative of the difference between the chip temperature series and the temperature series is positive, feedforward control is triggered, and the feedforward control quantity is calculated. If a "thermal shock warning" is detected, a feedforward activation increment is superimposed. Final valve opening In the formula, This is the feedforward control variable. This is the feedforward gain coefficient. For the rate of temperature change of the chip, For valve opening, This is a limiting function that constrains the output value to a normalized range of 0 to 1, corresponding to the valve being fully closed to fully open.

3. The intelligent control method for a liquid cooling system based on real-time feedback from a server board management controller according to claim 2, characterized in that, It also includes an anti-integral saturation control step: when the final valve opening reaches the normalized upper limit and the chip temperature continues to rise, the accumulation calculation of the integral term in the PID feedback control is frozen to avoid control saturation and temperature overshoot caused by continuous accumulation of integral; when the chip temperature falls back to the target temperature range, the integral term is gradually released and the integral regulation effect is restored to eliminate control lag and ensure smooth and stable valve regulation process.

4. The intelligent control method for a liquid cooling system based on real-time feedback from a server board management controller according to claim 2, characterized in that, It also includes a parameter adaptation step: dynamically adjusting the parameters of the PID feedback + feedforward composite control according to the chip type and load mode, wherein the load mode includes idle mode, inference mode and training mode.

5. The intelligent control method for a liquid cooling system based on real-time feedback from a server board management controller according to claim 4, characterized in that, In the parameter adaptation step, the adjustment parameters for each load mode of the CPU and GPU adopt the optimal values ​​that have been experimentally calibrated. The calibration basis for the preferred parameter values ​​is as follows: CPU has a large thermal inertia and a relatively mild heating rate, so it focuses on integral action to ensure steady-state accuracy; GPU has a small thermal inertia, a rapid and nonlinear temperature rise, so it focuses on feedforward and differential action to respond quickly to thermal shock.

6. The intelligent control method for a liquid cooling system based on real-time feedback from a server board management controller according to claim 5, characterized in that, It also includes a digital twin-assisted calibration step: Collect no less than 30 days of historical BMC data, including chip temperature data, real-time chip power consumption, CPU / GPU utilization, valve opening records, pump flow rate, and liquid supply temperature. After data cleaning and processing, construct a data-driven chip temperature response model. Embed the chip temperature response model into a digital twin environment, and optimize the control strategy parameters through historical load sequence playback and parameter optimization algorithms. After robustness verification in multiple load scenarios, deploy the optimized control strategy to the actual liquid cooling system. The parameters to be optimized include: PID feedback and feedforward control parameters for the opening of the electric regulating valve of the rack or server, total pump flow and liquid supply temperature setting parameters of the liquid cooling system, and adjustment parameters for each load mode of the CPU and GPU.

7. The intelligent control method for a liquid cooling system based on real-time feedback from a server board management controller according to claim 2, characterized in that, The thermal shock precursor feature identification includes: when the GPU temperature rises by more than 8°C within 10 seconds, or the rate of temperature change... >0.5℃ / s and CPU / GPU utilization When the value is greater than 20% / s, a feedforward early warning signal is triggered.

8. The intelligent control method for a liquid cooling system based on real-time feedback from a server board management controller according to claim 1, characterized in that, The data acquisition process has a collection cycle of 5-30 seconds and supports both synchronous polling and asynchronous event subscription.

9. A liquid cooling system intelligent control system based on real-time feedback from a server board management controller, characterized in that, The intelligent control method for a liquid cooling system based on real-time feedback from a server board management controller, as described in any one of claims 1-8, includes the following modules disposed in the intelligent control controller: Data acquisition module: Used to collect chip-level hardware health data from the BMC chip of each server in real time; Thermal state sensing module: used to identify precursory features of thermal shock; On-demand cooling decision module: used to calculate the opening degree of the electric regulating valve for the corresponding rack or server; Global Collaborative Optimization Module: This module optimizes the parameters of the liquid cooling system by adjusting the total pump flow rate and liquid supply temperature settings based on the chip temperature distribution and cooling requirements of all servers. Command execution module: Used to send valve opening commands to the electric regulating valves of the corresponding cabinets or servers, and to send global control commands, including pump total flow setting and liquid supply temperature setting, to the coolant distribution unit.

10. The intelligent control system for a liquid cooling system according to claim 9, characterized in that, Each rack's liquid inlet pipe or each server's cold plate inlet is equipped with an independent electric regulating valve.

Images