Multi-GPU server cluster liquid cooling flow distribution method based on reinforcement learning

Abstract

The invention relates to the technical field of liquid cooling flow distribution, and discloses a multi-GPU server cluster liquid cooling flow distribution method based on reinforcement learning, and the method comprises the steps: S1, collecting the operation state parameters of each GPU in a multi-GPU server, enabling the operation state parameters to comprise core temperature, voltage, current and load utilization rate, carrying out the thermal behavior modeling through a sliding time window, generating a thermal dynamic feature vector representing the GPU short-time thermal trend; and S2, collecting the flow velocity, the temperature difference of inlet and outlet water and the thermal resistance of the cold plate of each channel of the liquid cooling system in real time, and representing the liquid cooling assembly as a graph structure. According to the method, the technical scheme of joint modeling based on the graph neural network and reinforcement learning is adopted, fusion coding is performed on the GPU thermal dynamic characteristics and the topological state of the liquid cooling system, and an Actor-Critic architecture is introduced to realize self-adaptive regulation and control of the liquid cooling flow, so that the technical effect of improving the heat dissipation efficiency and the energy consumption balance control capability of the multi-GPU server cluster is achieved.

Description

Technical Field

[0001] The present invention relates to the technical field of liquid cooling flow distribution, and in particular to a liquid cooling flow distribution method for a multi-GPU server cluster based on reinforcement learning. Background Art

[0002] With the rapid development of artificial intelligence (AI) technology in recent years, reinforcement learning and graph neural networks have demonstrated strong capabilities in dynamic system control and complex structure modeling. Introducing these technologies into liquid cooling flow distribution systems provides new ideas for implementing adaptive and intelligent cooling strategies.

[0003] For example, the Chinese invention application with publication number CN118158985A discloses a liquid-cooled server cluster and a data center liquid cooling system, which specifically relates to the field of temperature control technology. By designing a server cluster architecture, deploying a liquid-cooled server cluster, and installing power meters and temperature sensors at key locations of the liquid-cooled server cluster, the power consumption and temperature parameters are monitored, and the data is transmitted to the data center for storage via the network. The data is connected to the data center to obtain temperature data, and the temperature gradient at different locations in the server cluster is evaluated based on the obtained temperature data. Based on the evaluation results, the heat dissipation layout of the server cluster is optimized, and the feedback control method is used to continuously adjust the cooling water flow and temperature so that the operating temperature of the server cluster is maintained within the set stable range. According to the scale of the deployed liquid-cooled server cluster, flow scheduling is performed, and the data flow in and out of the server cluster is monitored. The energy consumption of the liquid cooling system is calculated by combining flow scheduling and data flow monitoring.

[0004] The shortcomings of the above patents are:

[0005] On the one hand, liquid cooling systems in existing multi-GPU server clusters typically rely on static or preset temperature control models to distribute cooling flow. However, such models cannot reflect the thermal fluctuations generated by GPUs under varying tasks and loads in real time. Especially when GPUs frequently switch computing tasks or experience short-term high load peaks, the liquid cooling system cannot adjust the flow rate in a timely manner, resulting in delayed cooling response or excessive flow regulation. This can cause localized overheating of some GPUs or waste of liquid cooling resources, seriously affecting system stability and energy efficiency.

[0006] On the other hand, some existing liquid cooling flow optimization methods incorporate reinforcement learning-based policy models. However, these models often rely on specific GPU networking structures and liquid cooling channel configurations for training, and lack the ability to adapt to dynamic changes in hardware topology and fluid paths. When the GPU topology or liquid cooling system configuration changes in a server cluster due to maintenance, expansion, or component replacement, the original reinforcement learning model cannot be migrated and quickly adapted to the new environment, requiring retraining. This increases deployment costs and limits the engineering applicability and actual deployment efficiency of the liquid cooling control strategy.

[0007] To this end, the present invention proposes a liquid cooling flow distribution method for a multi-GPU server cluster based on reinforcement learning to solve the above-mentioned problems. Summary of the Invention

[0008] In view of the shortcomings of the existing technology, the present invention provides a multi-GPU server cluster liquid cooling flow distribution method based on reinforcement learning to solve the problems raised in the above background technology.

[0009] To achieve the above objectives, the present invention is implemented through the following technical solutions: a multi-GPU server cluster liquid cooling flow distribution method based on reinforcement learning, comprising:

[0010] S1, collecting operating status parameters of each GPU in a multi-GPU server, including core temperature, voltage, current, and load utilization, performing thermal behavior modeling using a sliding time window, and generating a thermal dynamic feature vector representing the short-term thermal trend of the GPU;

[0011] S2, collecting the flow rate, inlet and outlet water temperature difference, and cold plate thermal resistance of each channel of the liquid cooling system in real time, and representing the liquid cooling components as a graph structure, in which nodes represent cooling components and edges represent liquid flow paths and flow resistance parameters, to form a liquid cooling topology state representation;

[0012] S3 combines the thermal dynamic feature vector generated by S1 and the liquid cooling topology state representation formed by S2 as state input, encodes them through a graph neural network, and feeds them into a reinforcement learning model based on an actor-critic architecture. The reinforcement learning model generates liquid cooling flow scheduling instructions based on the combined state during the inference phase.

[0013] S4, sending the liquid cooling flow scheduling instruction generated in S3 to the liquid cooling execution control device to specify the flow distribution ratio of each channel;

[0014] S5, the liquid cooling execution control device dynamically adjusts the flow distribution ratio of each channel according to the instruction, and feeds back the actual flow value, actual temperature difference change and execution response time;

[0015] S6, uses the actual flow value, actual temperature difference change and execution response time fed back by S5 as input to update the thermal dynamic feature vector of S1 and the liquid cooling topology state representation of S2, and establishes a closed-loop adaptive flow distribution process.

[0016] Preferably, the S3 adopts a policy migration mechanism. When the server topology or liquid cooling path changes, the existing policy is automatically migrated through graph structure embedding comparison without the need for retraining.

[0017] The graph neural network adopts a heterogeneous graph convolution architecture to process the GPU interconnection topology and the liquid cooling channel topology in sequence, and fuse them to form a unified graph representation vector.

[0018] Preferably, the feedback signal in S6 includes the deviation between the actual flow value and the target value, the flow execution error, the response time and the actual GPU temperature fluctuation range, which are used to update the state space at the next moment.

[0019] Preferably, the multi-GPU server cluster liquid cooling flow distribution method is deployed in the edge computing node to achieve local rapid response adjustment of GPU thermal behavior and liquid cooling status;

[0020] The multi-GPU server cluster liquid cooling flow distribution method supports the multi-GPU cluster to quickly switch the liquid cooling strategy under different task scheduling scenarios, and introduces an experience-based heuristic strategy in the initial learning stage of the system.

[0021] Preferably, in S1, generating a thermal dynamic feature vector by performing thermal behavior modeling through a sliding time window further includes:

[0022] S1.1, collect the real-time operating parameter set D of each GPU = {T c ,V,I,U}, where:

[0023] T c is the GPU core temperature vector, V is the GPU voltage, I is the GPU current, and U is the load utilization;

[0024] Define the sliding time window W t :W t =[t-kΔt,t], where k is the window length, Δt is the sampling interval, and t is the current timestamp;

[0025] Normalize the original data:

[0026] is the normalized GPU operating parameter matrix, D is the original GPU operating parameter set, and D max is the normalized maximum boundary matrix, D min is the normalized minimum boundary matrix;

[0027] If ||D-μ is detected W ||>3σ W , μ W is the mean value within the window, σ W If it is the standard deviation, it is determined to be abnormal data and interpolation compensation is enabled;

[0028] S1.2, build an ARIMA temperature prediction model based on window data:

[0029]

[0030] Among them, p is the autoregressive order, d is the difference order, d is the moving average order, B is the backshift operator, ε t is white noise, is the autoregressive coefficient of the ARIMA model, B i is the backshift operator, is the GPU core temperature vector at the current time t, θ j is the moving average coefficient of the ARIMA model, B j is the jth power of the backshift operator;

[0031] Calculate the rate of temperature change:

[0032] R T is the temperature change rate, Δt is the sampling interval;

[0033] If the model fitting error MSE>0.5, switch to exponential smoothing prediction:

[0034] is the predicted temperature at the current time t+1, α is the exponential smoothing factor, is the predicted temperature at the current time t;

[0035] S1.3, combine real-time data and prediction results to generate feature vectors:

[0036] F thermal is the thermal dynamic eigenvector;

[0037] When the residual Activate the abnormal status flag:

[0038] otherwise is a vector concatenation operator, 1 indicates abnormal thermal behavior is detected, and 0 indicates the thermal state is normal.

[0039] Preferably, in S2, collecting liquid cooling system parameters in real time and constructing a liquid cooling topology state representation further includes:

[0040] S2.1, collect real-time data set of each channel L = {v j ,ΔT j ,R j},in:

[0041] v j is the flow velocity of the jth channel, is the inlet and outlet water temperature difference, is the outlet temperature of the jth channel, is the water inlet temperature of the jth channel, is the cold plate thermal resistance, P j is the channel thermal load power;

[0042] Define the dynamic flow resistance coefficient:

[0043]

[0044] in, is the dynamic flow resistance coefficient of the jth channel, v nom is the rated flow rate, γ is the dynamic adjustment coefficient of flow resistance;

[0045] If ΔT is detected j >ΔT max or v j <v min , marking the channel as abnormal state, ΔT max is the temperature difference threshold, v min is the minimum flow rate threshold;

[0046] S2.2, construct a liquid cooling system graph structure G = (V, E), where:

[0047] Node V = {v gpu ,v pump ,v radiator} is GPU cold plate, water pump, radiator, v gpu For GPU cold plate assembly, v pump For the water pump assembly, v radiator For the radiator assembly,

[0048] Edge E is the liquid path connecting the nodes, and the edge properties include flow resistance and flow capacity Define edge weights:

[0049] w jk is the edge weight;

[0050] If channel j is marked as abnormal, it is mandatory to set w jk =0.1w jk ;

[0051] S2.3, use graph attention network to generate node embedding:

[0052]

[0053] in, is the embedding vector of node v in the l+1 layer, σ is the activation function, is the embedding vector of the adjacent node u in the lth layer, α vu is the attention coefficient, W (l) is a trainable weight matrix, N(v) is the set of neighbor nodes of node v, and u is the adjacent node of node v;

[0054] Read the entire graph to generate the topological state vector:

[0055] h G is the liquid cooling topology state vector, is the embedding vector of node v in layer L, is the vector concatenation operator, MEAN is the global average pooling operation, and MAX is the global maximum pooling operation;

[0056] If there is an abnormal channel in the graph, G Append the exception flag bit 1 at the end, otherwise append 0.

[0057] Preferably, in S3, generating traffic scheduling instructions through graph neural network and reinforcement learning further includes:

[0058] S3.1, input the thermal dynamic feature vector F generated by S1 thermal The liquid cooling topology state vector h output by S2 G ;

[0059] Construct the joint state matrix:

[0060]

[0061] Among them, S joint is the joint state matrix, F thermal is the thermal dynamic eigenvector, W th is the trainable projection matrix, is the element-wise product, h G is the liquid cooling topology state vector, W cool is the liquid cooling topology feature projection matrix;

[0062] If it is detected that the liquid cooling topology state vector contains an abnormal flag bit h G [-1]=1, activate the degradation mode, let W cool ←0.5W cool ;

[0063] S3.2, using a dual-path graph convolutional network to process the GPU interconnection topology G in turn gpu With liquid cooling channel topology G cool :

[0064]

[0065] in, The convolution output feature matrix of the GPU interconnection topology graph at layer l+1 is: is the output feature matrix of the l+1th layer liquid cooling channel graph convolution, is the normalization factor, A gpu is the GPU interconnect topology adjacency matrix, is the feature matrix of GPU nodes in layer l, is the trainable weight matrix of the l-th layer GPU graph convolution, is the characteristic matrix of the liquid cooling node at the lth layer, is the trainable weight matrix of the l-th layer liquid-cooled graph convolution, j∈N(i) is the set of neighbor nodes of node i, ReLU is the rectified linear unit activation function, α ij is the edge weight attention coefficient;

[0066] Fusion of two-way features generates a unified graph representation:

[0067] H fusion is the heterogeneous topology fusion feature vector, is the L-th layer GPU topology feature matrix, is the final layer liquid cooling topology feature matrix;

[0068] If the GPU node temperature exceeds the safety threshold T c >T safe , append the temperature emergency flag [1] to the fusion feature, otherwise append [0];

[0069] S3.3, construct the policy network π(a|s) based on the PPO algorithm:

[0070] a t =softmax(W p ReLU(W f H fusion +b f )),

[0071] Among them, a t is the action space, W p is the weight matrix of the policy network output layer, W f is the hidden layer weight matrix of the policy network, b f is the bias vector of the hidden layer of the policy network;

[0072] Define the triggering condition for policy migration: when the graph embedding similarity between the new topology G′ and the original topology G is: sim(G,G′)=cos(h G ,h G′ ), strategy migration is achieved through weight interpolation

[0073] W new =λW old +(1-λ)W init , W new is the weight of the new network after strategy migration, W old is the strategy weight obtained from the original topology training, W init is the network initialization weight, λ is the strategy migration interpolation coefficient;

[0074] If the fused feature contains the temperature emergency flag, the action space constraint is a min Sets the lower limit constraint for the traffic allocation ratio.

[0075] Preferably, in S4, sending the liquid cooling flow scheduling instruction to the execution device further includes:

[0076] S4.1, input the original traffic distribution ratio vector a generated by S3 t =[a1,a2,...,a m ];

[0077] Calculate the total system flow capacity constraint:

[0078]

[0079] Among them, Q total is the total flow capacity constraint, Q max is the maximum total flow, is the upper limit of the physical flow rate of the j-th channel pipeline, a j is the original flow distribution ratio of the jth channel;

[0080] If Q total >Q max , perform proportional truncation:

[0081] Keep a j ′=a j ;

[0082] S4.2, obtain the abnormal channel set E marked in S2 fault :

[0083] E fault ={j|ΔT j >ΔT max ∨v j <v min}, where ΔT j is the temperature difference between the inlet and outlet water of the jth channel, ΔT max is the maximum threshold of the inlet and outlet water temperature difference, v min is the minimum flow rate threshold;

[0084] For each abnormal channel j, calculate the emergency adjustment coefficient:

[0085] β j is the emergency adjustment coefficient of the abnormal channel, v nom is the rated flow rate, ΔT base is the reference temperature difference threshold;

[0086] Update the allocation ratio:

[0087] a j ″ is the final distribution ratio after adjustment, a j ′ is the flow distribution ratio after truncation, a k ′ is the cutoff ratio of other channels;

[0088] Enforced Constraints

[0089] S4.3, convert the final distribution ratio into a PWM control signal:

[0090] PWM j is the j-th PWM control signal;

[0091] If PWM j >255, forced setting PWM j =255;

[0092] Generate a command message CMD with a timestamp:

[0093] CMD={t,[PWM1,...,PWM m ],CRC16([PWM1,...,PWM m ])},

[0094] If the response time t delay =t current -t last >50ms, enable instruction compression encoding to reduce transmission delay, where t delay is the instruction interval time, t current is the current time, t last The time of the last instruction.

[0095] Preferably, in said S5, the liquid cooling execution control device dynamic adjustment and feedback further includes:

[0096] S5.1, the PWM control signal generated by S4 is PWM j Convert to valve opening:

[0097]

[0098] Calculate the actual flow rate based on the valve characteristic curve:

[0099]

[0100] If a valve response delay is detected Enable prediction compensation:

[0101]

[0102] Among them, θ j is the valve opening of the jth channel, θ max is the maximum mechanical opening of the valve, is the actual flow velocity of the jth channel, v nom is the rated flow rate, e is the base of natural logarithm, k v is the flow rate response coefficient, The response delay time of channel j is t current is the current timestamp, t cmd Timestamp for issuing instructions, is the predicted compensation flow rate for the jth channel, dt is the time differential interval, is the rate of change of flow velocity in the jth channel;

[0103] S5.2, real-time collection of feedback data from each channel:

[0104]

[0105] Calculate flow execution deviation:

[0106]

[0107] like Continuous 3 sampling cycles, triggering abnormal alarm and recording events:

[0108]

[0109] Among them, F feedback is the feedback data set, is the actual inlet and outlet water temperature difference of the jth channel, is the actual thermal resistance of the jth channel, The deviation is implemented for the j-th channel flow, is the target flow rate of the jth channel, Alert j is the abnormal alarm flag of the jth channel;

[0110] S5.3, Calculate system-level performance indicators:

[0111]

[0112] Among them, w1 and w2 are weight coefficients, J is the system-level performance index, T j GPU is the core temperature of the j-th associated GPU, T target Control temperature for GPU target;

[0113] Generate a timestamped feedback vector:

[0114]

[0115] like Forced to add emergency cooling flag.

[0116] Preferably, in S6, the closed-loop adaptive flow distribution process is updated, further comprising:

[0117] S6.1, input S5 feedback vector Feedback t :

[0118]

[0119] Update the thermal dynamic feature vector of S1:

[0120]

[0121] in, is the updated thermal dynamic feature vector, is the current thermal dynamic feature vector, ⊙ is the element-by-element product, is the partial derivative of the performance index with respect to GPU temperature, is the partial derivative of the performance index with respect to the actual flow rate, M is the sliding window mask matrix, and α is the learning rate;

[0122] If an Alert is detected j =1, forcefully reset the thermal dynamic feature vector of the corresponding channel;

[0123] S6.2, update the graph structure edge weights of S2 according to the actual flow resistance change:

[0124]

[0125] Where γ is the attenuation coefficient, is the updated graph edge weight, w jk is the edge weight of the original graph, To actually measure the flow resistance, To predict the flow resistance value;

[0126] If channel j Lasts for 5 seconds, marked as permanently failed node and removed:

[0127] G′ is the updated liquid cooling topology, and G is the original liquid cooling topology;

[0128] S6.3, calculate the timing difference error:

[0129] δ t =J+λV(s t+1 )-V(s t ),

[0130] Where λ is the discount factor, δ t is the timing differential error, J is the system-level performance index, V(s t+1 ) is the next state value estimate, V(s t ) is the current state value estimate;

[0131] Update policy network parameters:

[0132]

[0133] Among them, θ new is the updated policy network parameter, θ old is the original policy network parameter, η is the policy network learning rate, is the policy gradient;

[0134] If ||δ t ||<0.1, triggering the early stopping mechanism:

[0135] η←0.5η and α←0.8α,.

[0136] The present invention provides a method for allocating liquid cooling flow in a multi-GPU server cluster based on reinforcement learning. It has the following beneficial effects:

[0137] 1. The present invention adopts a technical solution based on joint modeling of graph neural networks and reinforcement learning. By fusing and encoding the GPU thermal dynamic characteristics with the topological state of the liquid cooling system, and introducing the Actor-Critic architecture to achieve adaptive regulation of the liquid cooling flow, the technical effect of improving the heat dissipation efficiency and energy consumption balance control capabilities of multi-GPU server clusters is achieved. Compared with the technical solutions in the prior art that rely on static threshold control or simple linear scheduling strategies, this solution solves the problems of untimely response of liquid cooling resource allocation and insufficient heat dissipation control accuracy under dynamic load changes.

[0138] 2. The present invention adopts a strategy update mechanism that supports graph structure embedding migration and feedback closed-loop optimization. By introducing channel anomaly detection, strategy migration trigger conditions and feedback-driven state space update process, it achieves the technical effect of improving the robustness of the liquid cooling system to adapt to structural changes and abnormal operating conditions. Compared with the existing technical solutions in which the reinforcement learning strategy has no migration capability for topology changes, high retraining overhead and lagging feedback mechanism, it solves the shortcomings of poor generalization ability and low convergence efficiency of liquid cooling scheduling strategies in scenarios where the topology of multi-server clusters frequently changes. BRIEF DESCRIPTION OF THE DRAWINGS

[0139] Figure 1 Flowchart of the present invention. DETAILED DESCRIPTION

[0140] To help those skilled in the art understand the present invention, the following will provide a clear and complete description of the technical solutions in the embodiments of the present invention, in conjunction with the accompanying drawings. Obviously, the described embodiments are only partial embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0141] The present invention is described in detail below with reference to the accompanying drawings:

[0142] Example:

[0143] Please see the attached Figure 1 An embodiment of the present invention provides a method for allocating liquid cooling flow in a multi-GPU server cluster based on reinforcement learning, comprising:

[0144] S1 collects the operating status parameters of each GPU in a multi-GPU server, including core temperature, voltage, current, and load utilization. It then performs thermal behavior modeling using a sliding time window to generate a thermal dynamic feature vector representing the short-term thermal trend of the GPU.

[0145] S1.1, collect the real-time operating parameter set D of each GPU = {T c ,V,I,U}, where:

[0146] T c is the GPU core temperature vector, V is the GPU voltage, I is the GPU current, and U is the load utilization;

[0147] Define the sliding time window W t :W t =[t-kΔt,t], where k is the window length, Δt is the sampling interval, and t is the current timestamp;

[0148] Normalize the original data:

[0149] is the normalized GPU operating parameter matrix, D is the original GPU operating parameter set, and D max is the normalized maximum boundary matrix, D min is the normalized minimum boundary matrix;

[0150] If ||D-μ is detected W ||>3σ W , μ W is the mean value within the window, σ W If it is the standard deviation, it is determined to be abnormal data and interpolation compensation is enabled;

[0151] S1.2, build an ARIMA temperature prediction model based on window data:

[0152]

[0153] Among them, p is the autoregressive order, d is the difference order, d is the moving average order, B is the backshift operator, ε t is white noise, is the autoregressive coefficient of the ARIMA model, B i is the backshift operator, is the GPU core temperature vector at the current time t, θ j is the moving average coefficient of the ARIMA model, B j is the jth power of the backshift operator;

[0154] Calculate the rate of temperature change:

[0155] R T is the temperature change rate, Δt is the sampling interval;

[0156] If the model fitting error MSE>0.5, switch to exponential smoothing prediction:

[0157] is the predicted temperature at the current time t+1, α is the exponential smoothing factor, is the predicted temperature at the current time t;

[0158] S1.3, combine real-time data and prediction results to generate feature vectors:

[0159] F thermal is the thermal dynamic eigenvector;

[0160] When the residual Activate the abnormal status flag:

[0161] otherwise is a vector concatenation operator, 1 indicates abnormal thermal behavior is detected, and 0 indicates normal thermal status;

[0162] S2 collects the flow rate, inlet and outlet water temperature difference, and cold plate thermal resistance of each channel of the liquid cooling system in real time, and represents the liquid cooling components as a graph structure. The nodes in the graph represent the cooling components, and the edges represent the liquid flow paths and flow resistance parameters, forming a liquid cooling topology state representation;

[0163] S2.1, collect real-time data set of each channel L = {v j ,ΔT j ,R j},in:

[0164] v j is the flow velocity of the jth channel, is the inlet and outlet water temperature difference, is the outlet temperature of the jth channel, is the water inlet temperature of the jth channel, is the cold plate thermal resistance, P j is the channel thermal load power;

[0165] Define the dynamic flow resistance coefficient:

[0166]

[0167] in, is the dynamic flow resistance coefficient of the jth channel, v nom is the rated flow rate, γ is the dynamic adjustment coefficient of flow resistance;

[0168] If ΔT is detected j >ΔT max or v j <v min , marking the channel as abnormal state, ΔT max is the temperature difference threshold, v min is the minimum flow rate threshold;

[0169] S2.2, construct a liquid cooling system graph structure G = (V, E), where:

[0170] Node V = {v gpu ,v pump ,v radiator} is GPU cold plate, water pump, radiator, v gpu For GPU cold plate assembly, v pump For the water pump assembly, v radiator For the radiator assembly,

[0171] Edge E is the liquid path connecting the nodes, and the edge properties include flow resistance and flow capacity Define edge weights:

[0172] w jk is the edge weight;

[0173] If channel j is marked as abnormal, it is mandatory to set w jk =0.1w jk ;

[0174] S2.3, use graph attention network to generate node embedding:

[0175]

[0176] in, is the embedding vector of node v in the l+1 layer, σ is the activation function, is the embedding vector of the adjacent node u in the lth layer, α vu is the attention coefficient, W (l) is a trainable weight matrix, N(v) is the set of neighbor nodes of node v, and u is the adjacent node of node v;

[0177] Read the entire graph to generate the topological state vector:

[0178] h G is the liquid cooling topology state vector, is the embedding vector of node v in layer L, is the vector concatenation operator, MEAN is the global average pooling operation, and MAX is the global maximum pooling operation;

[0179] If there is an abnormal channel in the graph, G Append the exception flag bit 1 at the end, otherwise append 0;

[0180] S3 combines the thermal dynamic feature vector generated by S1 and the liquid cooling topology state representation formed by S2 as state input. After being encoded through a graph neural network, it is fed into a reinforcement learning model based on an actor-critic architecture. During the inference phase, the reinforcement learning model generates liquid cooling flow scheduling instructions based on the combined state.

[0181] S3 uses a policy migration mechanism. When the server topology or liquid cooling path changes, the existing policy is automatically migrated through graph structure embedding comparison, eliminating the need for retraining.

[0182] The graph neural network uses a heterogeneous graph convolutional architecture to sequentially process the GPU interconnection topology and liquid cooling channel topology, fusing them into a unified graph representation vector.

[0183] S3.1, input the thermal dynamic feature vector F generated by S1 thermal The liquid cooling topology state vector h output by S2G ;

[0184] Construct the joint state matrix:

[0185]

[0186] Among them, S joint is the joint state matrix, F thermal is the thermal dynamic eigenvector, W th is the trainable projection matrix, is the element-wise product, h G is the liquid cooling topology state vector, W cool is the liquid cooling topology feature projection matrix;

[0187] If it is detected that the liquid cooling topology state vector contains an abnormal flag bit h G [-1]=1, activate the degradation mode, let W cool ←0.5W cool ;

[0188] S3.2, using a dual-path graph convolutional network to process the GPU interconnection topology G in turn gpu With liquid cooling channel topology G cool :

[0189]

[0190]

[0191] in, The convolution output feature matrix of the GPU interconnection topology graph at layer l+1 is: is the output feature matrix of the l+1th layer liquid cooling channel graph convolution, is the normalization factor, A gpu is the GPU interconnect topology adjacency matrix, is the feature matrix of GPU nodes in layer l, is the trainable weight matrix of the l-th layer GPU graph convolution, is the characteristic matrix of the liquid cooling node at the lth layer, is the trainable weight matrix of the l-th layer liquid-cooled graph convolution, j∈N(i) is the set of neighbor nodes of node i, ReLU is the rectified linear unit activation function, α ij is the edge weight attention coefficient;

[0192] Fusion of two-way features generates a unified graph representation:

[0193] H fusion is the heterogeneous topology fusion feature vector, is the L-th layer GPU topology feature matrix, is the final layer liquid cooling topology feature matrix;

[0194] If the GPU node temperature exceeds the safety threshold T c >T safe , append the temperature emergency flag [1] to the fusion feature, otherwise append [0];

[0195] S3.3, construct the policy network π(a|s) based on the PPO algorithm:

[0196] a t =softmax(W p ReLU(W f H fusion +b f )),

[0197] Among them, a t is the action space, W p is the weight matrix of the policy network output layer, W f is the hidden layer weight matrix of the policy network, b f is the bias vector of the hidden layer of the policy network;

[0198] Define the triggering condition for policy migration: when the graph embedding similarity between the new topology G′ and the original topology G is: sim(G,G′)=cos(h G ,h G′ ), strategy migration is achieved through weight interpolation

[0199] W new =λW old +(1-λ)W init , W new is the weight of the new network after strategy migration, W old is the strategy weight obtained from the original topology training, W init is the network initialization weight, λ is the strategy migration interpolation coefficient;

[0200] If the fused feature contains the temperature emergency flag, the action space constraint is a min A lower limit constraint for the flow distribution ratio;

[0201] S4, sending the liquid cooling flow scheduling instruction generated in S3 to the liquid cooling execution control device to specify the flow distribution ratio of each channel;

[0202] S4.1, input the original traffic distribution ratio vector a generated by S3 t =[a1,a2,...,a m ];

[0203] Calculate the total system flow capacity constraint:

[0204]

[0205] Among them, Q total is the total flow capacity constraint, Q max is the maximum total flow, is the upper limit of the physical flow rate of the j-th channel pipeline, a j is the original flow distribution ratio of the jth channel;

[0206] If Q total >Q max , perform proportional truncation:

[0207] Keep a j ′=a j ;

[0208] S4.2, obtain the abnormal channel set E marked in S2 fault :

[0209] E fault ={j|ΔT j >ΔT max ∨v j <v min}, where ΔT j is the temperature difference between the inlet and outlet water of the jth channel, ΔT max is the maximum threshold of the inlet and outlet water temperature difference, v min is the minimum flow rate threshold;

[0210] For each abnormal channel j, calculate the emergency adjustment coefficient:

[0211] β j is the emergency adjustment coefficient of the abnormal channel, v nom is the rated flow rate, ΔT base is the reference temperature difference threshold;

[0212] Update the allocation ratio:

[0213] a j ″ is the final distribution ratio after adjustment, a j ′ is the flow distribution ratio after truncation, a k ′ is the cutoff ratio of other channels;

[0214] Enforced Constraints

[0215] S4.3, convert the final distribution ratio into a PWM control signal:

[0216] PWM j is the j-th PWM control signal;

[0217] If PWMj >255, forced setting PWM j =255;

[0218] Generate a command message CMD with a timestamp:

[0219] CMD={t,[PWM1,...,PWM m ],CRC16([PWM1,...,PWM m ])},

[0220] If the response time t delay =t current -t last >50ms, enable instruction compression encoding to reduce transmission delay, where t delay is the instruction interval time, t current is the current time, t last The time of the last instruction;

[0221] S5, the liquid cooling execution control device dynamically adjusts the flow distribution ratio of each channel according to the instruction, and feeds back the actual flow value, actual temperature difference change and execution response time;

[0222] S5.1, the PWM control signal generated by S4 is PWM j Convert to valve opening:

[0223]

[0224] Calculate the actual flow rate based on the valve characteristic curve:

[0225]

[0226] If a valve response delay is detected Enable prediction compensation:

[0227]

[0228] Among them, θ j is the valve opening of the jth channel, θ max is the maximum mechanical opening of the valve, is the actual flow velocity of the jth channel, v nom is the rated flow rate, e is the base of natural logarithm, k v is the flow rate response coefficient, The response delay time of channel j is t current is the current timestamp, t cmd Timestamp for issuing instructions, is the predicted compensation flow rate for the jth channel, dt is the time differential interval, is the rate of change of flow velocity in the jth channel;

[0229] S5.2, real-time collection of feedback data from each channel:

[0230]

[0231] Calculate flow execution deviation:

[0232]

[0233] like Continuous 3 sampling cycles, triggering abnormal alarm and recording events:

[0234]

[0235] Among them, F feedback is the feedback data set, is the actual inlet and outlet water temperature difference of the jth channel, is the actual thermal resistance of the jth channel, The deviation is implemented for the j-th channel flow, is the target flow rate of the jth channel, Alert j is the abnormal alarm flag of the jth channel;

[0236] S5.3, Calculate system-level performance indicators:

[0237]

[0238] Among them, w1 and w2 are weight coefficients, J is the system-level performance index, T j GPU is the core temperature of the j-th associated GPU, T target Control temperature for GPU target;

[0239] Generate a timestamped feedback vector:

[0240]

[0241] like Forced to add emergency cooling flag;

[0242] S6 uses the actual flow value, actual temperature difference change, and execution response time fed back by S5 as input to update the thermal dynamic feature vector of S1 and the liquid cooling topology state representation of S2, and establish a closed-loop adaptive flow distribution process;

[0243] The feedback signal in S6 includes the deviation between the actual flow value and the target value, the flow execution error, the response time, and the actual GPU temperature fluctuation range, which are used to update the state space at the next moment;

[0244] The liquid cooling flow distribution method for multi-GPU server clusters is deployed in edge computing nodes to achieve local rapid response adjustment of GPU thermal behavior and liquid cooling status;

[0245] The liquid cooling flow distribution method for multi-GPU server clusters supports rapid switching of liquid cooling strategies in different task scheduling scenarios, and introduces an experience-based heuristic strategy in the initial learning phase of the system.

[0246] S6.1, input S5 feedback vector Feedback t :

[0247]

[0248] Update the thermal dynamic feature vector of S1:

[0249]

[0250] in, is the updated thermal dynamic feature vector, is the current thermal dynamic feature vector, ⊙ is the element-by-element product, is the partial derivative of the performance index with respect to GPU temperature, is the partial derivative of the performance index with respect to the actual flow rate, M is the sliding window mask matrix, and α is the learning rate;

[0251] If an Alert is detected j =1, forcefully reset the thermal dynamic feature vector of the corresponding channel;

[0252] S6.2, update the graph structure edge weights of S2 according to the actual flow resistance change:

[0253]

[0254] Where γ is the attenuation coefficient, is the updated graph edge weight, w jk is the edge weight of the original graph, To actually measure the flow resistance, To predict the flow resistance value;

[0255] If channel j Lasts for 5 seconds, marked as permanently failed node and removed:

[0256] G′ is the updated liquid cooling topology, and G is the original liquid cooling topology;

[0257] S6.3, calculate the timing difference error:

[0258] δ t =J+λV(s t+1 )-V(st ),

[0259] Where λ is the discount factor, δ t is the timing differential error, J is the system-level performance index, V(s t+1 ) is the next state value estimate, V(s t ) is the current state value estimate;

[0260] Update policy network parameters:

[0261]

[0262] Among them, θ new is the updated policy network parameter, θ old is the original policy network parameter, η is the policy network learning rate, is the policy gradient;

[0263] If ||δ t ||<0.1, triggering the early stopping mechanism:

[0264] η←0.5η and α←0.8α.

[0265] By collecting the server's core temperature, voltage, current, and load utilization in real time, we can understand the server's current working status and thermal behavior, providing data support for subsequent temperature prediction and traffic allocation.

[0266] By modeling data using a sliding time window, we can track and capture dynamic changes in server core temperature and load, generating a thermal dynamic feature vector. This method can adjust the response to sudden thermal behavior.

[0267] Normalization helps eliminate potential differences in physical quantities between devices, ensuring that all data is compared using the same standards. This unified data standardization improves model stability and avoids calculation errors caused by differences in data magnitude.

[0268] Based on a temperature prediction model and an interpolation compensation mechanism for abnormal data, this system can promptly detect abnormal behavior and automatically adjust flow strategies to avoid system failures caused by sudden temperature changes. The model's adaptability significantly improves the system's fault tolerance and enhances the accuracy of liquid cooling flow scheduling.

[0269] The topological state representation of the liquid cooling system and the use of graph neural networks enable the system to comprehensively model and analyze the liquid cooling flow paths and component states. By optimizing flow distribution through the graph structure, adjustments can be made to each cooling channel to avoid uneven heat distribution or inefficient cooling.

[0270] By jointly encoding thermal dynamic feature vectors and liquid cooling topology state vectors through a graph neural network, the reinforcement learning model can make real-time optimization decisions, improving the efficiency and responsiveness of traffic distribution. A policy migration mechanism enables seamless transitions when topology or liquid cooling paths change, saving time and resources in retraining and making the system more resilient and adaptable.

[0271] During liquid cooling flow scheduling, the system dynamically adjusts flow ratios based on feedback from each channel and anomaly detection mechanisms to avoid overall performance degradation caused by localized flow anomalies. This mechanism ensures stable system operation in the face of sudden anomalies, minimizing equipment overheating and wasted cooling resources.

[0272] Through a feedback mechanism, the system can adjust the liquid cooling flow distribution in real time based on actual flow and temperature differential changes, creating an adaptive control system. This feedback signal can be used to optimize next-step decisions, improving the accuracy and response speed of the liquid cooling system, and thus maintaining optimal system operation.

[0273] The system can generate alarms and record events when abnormal situations occur. Combined with real-time feedback, the system can solve current problems and extract experience from historical data to continuously optimize its decision-making model.

[0274] During the final flow distribution phase, the liquid cooling execution control device dynamically adjusts the flow ratios among channels based on the dispatch instructions, precisely controlling flow rates through valve openings. By compensating for valve response delays and correcting for execution deviations, the system ensures that flow distribution remains within the expected range, preventing poor cooling performance due to untimely responses or execution errors.

[0275] System-level performance indicators provide a comprehensive view of the cooling system's overall operating efficiency. By monitoring the temperature and flow rate of each channel, the system can take timely action when performance degrades, ensuring the equipment operates within a safe temperature range.

[0276] While embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to these embodiments without departing from the principles and spirit of the invention, and that the scope of the invention is defined by the appended claims and their equivalents.

Claims

1. A method for allocating liquid cooling flow in a multi-GPU server cluster based on reinforcement learning, characterized in that: include: S1, collecting operating status parameters of each GPU in a multi-GPU server, including core temperature, voltage, current, and load utilization, performing thermal behavior modeling using a sliding time window, and generating a thermal dynamic feature vector representing the short-term thermal trend of the GPU; S2, collecting the flow rate, inlet and outlet water temperature difference, and cold plate thermal resistance of each channel of the liquid cooling system in real time, and representing the liquid cooling components as a graph structure, in which nodes represent cooling components and edges represent liquid flow paths and flow resistance parameters, to form a liquid cooling topology state representation; S3 combines the thermal dynamic feature vector generated by S1 and the liquid cooling topology state representation formed by S2 as state input, encodes them through a graph neural network, and feeds them into a reinforcement learning model based on an actor-critic architecture. The reinforcement learning model generates liquid cooling flow scheduling instructions based on the combined state during the inference phase. S4, sending the liquid cooling flow scheduling instruction generated in S3 to the liquid cooling execution control device to specify the flow distribution ratio of each channel; S5, the liquid cooling execution control device dynamically adjusts the flow distribution ratio of each channel according to the instruction, and feeds back the actual flow value, actual temperature difference change and execution response time; S6, uses the actual flow value, actual temperature difference change and execution response time fed back by S5 as input to update the thermal dynamic feature vector of S1 and the liquid cooling topology state representation of S2, and establishes a closed-loop adaptive flow distribution process.

2. The method for allocating liquid cooling flow in a multi-GPU server cluster based on reinforcement learning according to claim 1, characterized in that: S3 uses a policy migration mechanism. When the server topology or liquid cooling path changes, the existing policy is automatically migrated through graph structure embedding comparison without the need for retraining. The graph neural network adopts a heterogeneous graph convolution architecture to process the GPU interconnection topology and the liquid cooling channel topology in sequence, and fuse them to form a unified graph representation vector.

3. The method for allocating liquid cooling flow in a multi-GPU server cluster based on reinforcement learning according to claim 1, characterized in that: The feedback signal in S6 includes the deviation between the actual flow value and the target value, the flow execution error, the response time and the actual GPU temperature fluctuation range, which is used to update the state space at the next moment.

4. The method for allocating liquid cooling flow in a multi-GPU server cluster based on reinforcement learning according to claim 1, characterized in that: The multi-GPU server cluster liquid cooling flow distribution method is deployed in edge computing nodes to achieve local rapid response adjustment of GPU thermal behavior and liquid cooling status; The multi-GPU server cluster liquid cooling flow distribution method supports the multi-GPU cluster to quickly switch the liquid cooling strategy under different task scheduling scenarios, and introduces an experience-based heuristic strategy in the initial learning stage of the system.

5. The method for allocating liquid cooling flow in a multi-GPU server cluster based on reinforcement learning according to claim 1, characterized in that: In S1, generating a thermal dynamic feature vector by performing thermal behavior modeling through a sliding time window further includes: S1.1, collect the real-time operating parameter set D of each GPU = {T c ,V,I,U}, where: T c is the GPU core temperature vector, V is the GPU voltage, I is the GPU current, and U is the load utilization; Define the sliding time window W t :W t =[t-kΔt,t], where k is the window length, Δt is the sampling interval, and t is the current timestamp; Normalize the original data: is the normalized GPU operating parameter matrix, D is the original GPU operating parameter set, and D max is the normalized maximum boundary matrix, D min is the normalized minimum boundary matrix; If ||D-μ is detected W ||>3σ W , μ W is the mean value within the window, σ W If it is the standard deviation, it is determined to be abnormal data and interpolation compensation is enabled; S1.2, build an ARIMA temperature prediction model based on window data: Among them, p is the autoregressive order, d is the difference order, d is the moving average order, B is the backshift operator, ε t is white noise, is the autoregressive coefficient of the ARIMA model, B i is the backshift operator, is the GPU core temperature vector at the current time t, θ j is the moving average coefficient of the ARIMA model, B j is the jth power of the backshift operator; Calculate the rate of temperature change: R T is the temperature change rate, Δt is the sampling interval; If the model fitting error MSE>0.5, switch to exponential smoothing prediction: is the predicted temperature at the current time t+1, α is the exponential smoothing factor, is the predicted temperature at the current time t; S1.3, combine real-time data and prediction results to generate feature vectors: F thermal is the thermal dynamic eigenvector; When the residual Activate the abnormal status flag: F thermal ←F thermal ⊕[1], otherwise ⊕[0], ⊕ is a vector concatenation operator, 1 means abnormal thermal behavior is detected, and 0 means the thermal state is normal.

6. The method for allocating liquid cooling flow in a multi-GPU server cluster based on reinforcement learning according to claim 1, characterized in that: In S2, real-time collection of liquid cooling system parameters and construction of a liquid cooling topology state representation further include: S2.1, collect real-time data set of each channel L = {v j ,ΔT j ,R j },in: v j is the flow velocity of the jth channel, is the inlet and outlet water temperature difference, is the outlet temperature of the jth channel, is the water inlet temperature of the jth channel, is the cold plate thermal resistance, P j is the channel thermal load power; Define the dynamic flow resistance coefficient: in, is the dynamic flow resistance coefficient of the jth channel, v nom is the rated flow rate, γ is the dynamic adjustment coefficient of flow resistance; If ΔT is detected j >ΔT max or v j <v min , marking the channel as abnormal state, ΔT max is the temperature difference threshold, v min is the minimum flow rate threshold; S2.2, construct a liquid cooling system graph structure G = (V, E), where: Node V = {v gpu ,v pump ,v radiator } is GPU cold plate, water pump, radiator, v gpu For GPU cold plate assembly, v pump For the water pump assembly, v radiator For the radiator assembly, Edge E is the liquid path connecting the nodes, and the edge properties include flow resistance and flow capacity Define edge weights: w jk is the edge weight; If channel j is marked as abnormal, it is mandatory to set w jk =0.1w jk ; S2.3, use graph attention network to generate node embedding: in, is the embedding vector of node v in the l+1 layer, σ is the activation function, is the embedding vector of the adjacent node u in the lth layer, α vu is the attention coefficient, W (l) is a trainable weight matrix, N(v) is the set of neighbor nodes of node v, and u is the adjacent node of node v; Read the entire graph to generate the topological state vector: h G is the liquid cooling topology state vector, is the embedding vector of node v at layer L, ⊕ is the vector concatenation operator, MEAN is the global average pooling operation, and MAX is the global maximum pooling operation; If there is an abnormal channel in the graph, G Append the exception flag bit 1 at the end, otherwise append 0.

7. The method for allocating liquid cooling flow in a multi-GPU server cluster based on reinforcement learning according to claim 1, characterized in that: In S3, generating traffic scheduling instructions through graph neural network and reinforcement learning further includes: S3.1, input the thermal dynamic feature vector F generated by S1 thermal The liquid cooling topology state vector h output by S2 G ; Construct the joint state matrix: Among them, S joint is the joint state matrix, F thermal is the thermal dynamic eigenvector, W th is the trainable projection matrix, is the element-wise product, h G is the liquid cooling topology state vector, W cool is the liquid cooling topology feature projection matrix; If it is detected that the liquid cooling topology state vector contains an abnormal flag bit h G [-1]=1, activate the degradation mode, let W cool ←0.5W cool ; S3.2, using a dual-path graph convolutional network to process the GPU interconnection topology G in turn gpu With liquid cooling channel topology G cool : in, The convolution output feature matrix of the GPU interconnection topology graph at layer l+1 is: is the output feature matrix of the l+1th layer liquid cooling channel graph convolution, is the normalization factor, A gpu is the GPU interconnect topology adjacency matrix, is the feature matrix of GPU nodes in layer l, is the trainable weight matrix of the l-th layer GPU graph convolution, is the characteristic matrix of the liquid cooling node at the lth layer, is the trainable weight matrix of the l-th layer liquid-cooled graph convolution, j∈N(i) is the set of neighbor nodes of node i, ReLU is the rectified linear unit activation function, α ij is the edge weight attention coefficient; Fusion of two-way features generates a unified graph representation: H fusion is the heterogeneous topology fusion feature vector, is the L-th layer GPU topology feature matrix, is the final layer liquid cooling topology feature matrix; If the GPU node temperature exceeds the safety threshold T c >T safe , append the temperature emergency flag [1] to the fusion feature, otherwise append [0]; S3.3, construct the policy network π(a|s) based on the PPO algorithm: and t =softmax(W p ·ReLU(W f H fusion +b f )), Among them, a t is the action space, W p is the weight matrix of the policy network output layer, W f is the hidden layer weight matrix of the policy network, b f is the bias vector of the hidden layer of the policy network; Define the triggering condition for policy migration: when the graph embedding similarity between the new topology G′ and the original topology G is: sim(G,G′)=cos(h G ,h G′ ), strategy migration is achieved through weight interpolation W new =λW old +(1-λ)W init , W new is the weight of the new network after strategy migration, W old is the strategy weight obtained from the original topology training, W init is the network initialization weight, λ is the strategy migration interpolation coefficient; If the fused feature contains the temperature emergency flag, the action space constraint is a min Sets the lower limit constraint for the traffic allocation ratio.

8. The method for allocating liquid cooling flow in a multi-GPU server cluster based on reinforcement learning according to claim 1, characterized in that: In S4, the liquid cooling flow scheduling instruction is sent to the execution device, further comprising: S4.1, input the original traffic distribution ratio vector a generated by S3 t =[a1,a2,...,a m ]; Calculate the total system flow capacity constraint: Among them, Q total is the total flow capacity constraint, Q max is the maximum total flow, is the upper limit of the physical flow rate of the j-th channel pipeline, a j is the original flow distribution ratio of the jth channel; If Q total >Q max , perform proportional truncation: Keep a j ′=a j ; S4.2, obtain the abnormal channel set E marked in S2 fault : E fault ={j|ΔT j >ΔT max ∨v j <v min }, where ΔT j is the temperature difference between the inlet and outlet water of the jth channel, ΔT max is the maximum threshold of the inlet and outlet water temperature difference, v min is the minimum flow rate threshold; For each abnormal channel j, calculate the emergency adjustment coefficient: β j is the emergency adjustment coefficient of the abnormal channel, v nom is the rated flow rate, ΔT base is the reference temperature difference threshold; Update the allocation ratio: a j ″ is the final distribution ratio after adjustment, a j ′ is the flow distribution ratio after truncation, a k ′ is the cutoff ratio of other channels; Enforced Constraints S4.3, convert the final distribution ratio into a PWM control signal: PWM j is the j-th PWM control signal; If PWM j >255, forced setting PWM j =255; Generate a command message CMD with a timestamp: CMD={t,[PWM1,...,PWM m ],CRC16([PWM1,...,PWM m ])}, If the response time t delay =t current -t last >50ms, enable instruction compression encoding to reduce transmission delay, where t delay is the instruction interval time, t current is the current time, t last The time of the last instruction.

9. The method for allocating liquid cooling flow in a multi-GPU server cluster based on reinforcement learning according to claim 1, characterized in that: In S5, the liquid cooling execution control device dynamic adjustment and feedback further includes: S5.1, the PWM control signal generated by S4 is PWM j Convert to valve opening: Calculate the actual flow rate based on the valve characteristic curve: If a valve response delay is detected Enable prediction compensation: Among them, θ j is the valve opening of the jth channel, θ max is the maximum mechanical opening of the valve, is the actual flow velocity of the jth channel, v nom is the rated flow rate, e is the base of natural logarithm, k v is the flow rate response coefficient, The response delay time of channel j is t current is the current timestamp, t cmd Timestamp for issuing instructions, is the predicted compensation flow rate for the jth channel, dt is the time differential interval, is the rate of change of flow velocity in the jth channel; S5.2, real-time collection of feedback data from each channel: Calculate flow execution deviation: like Continuous 3 sampling cycles, triggering abnormal alarm and recording events: Among them, F feedback is the feedback data set, is the actual inlet and outlet water temperature difference of the jth channel, is the actual thermal resistance of the jth channel, The deviation is implemented for the j-th channel flow, is the target flow rate of the jth channel, Alert j is the abnormal alarm flag of the jth channel; S5.3, Calculate system-level performance indicators: Among them, w1 and w2 are weight coefficients, J is the system-level performance index, T j GPU is the core temperature of the j-th associated GPU, T target Control temperature for GPU target; Generate a timestamped feedback vector: like Forced to add emergency cooling flag.

10. The method for allocating liquid cooling flow in a multi-GPU server cluster based on reinforcement learning according to claim 1, characterized in that: In S6, the closed-loop adaptive traffic allocation process is updated, further comprising: S6.1, input S5 feedback vector Feedback t : Update the thermal dynamic feature vector of S1: in, is the updated thermal dynamic feature vector, is the current thermal dynamic feature vector, ⊙ is the element-by-element product, is the partial derivative of the performance index with respect to GPU temperature, is the partial derivative of the performance index with respect to the actual flow rate, M is the sliding window mask matrix, and α is the learning rate; If an Alert is detected j =1, forcefully reset the thermal dynamic feature vector of the corresponding channel; S6.2, update the graph structure edge weights of S2 according to the actual flow resistance change: Where γ is the attenuation coefficient, is the updated graph edge weight, w jk is the edge weight of the original graph, To actually measure the flow resistance, To predict the flow resistance value; If channel j Lasts for 5 seconds, marked as permanently failed node and removed: G ′ is the updated liquid cooling topology diagram, and G is the original liquid cooling topology diagram; S6.3, calculate the timing difference error: δ t =J+λV(s t+1 )-V(s t ), Where λ is the discount factor, δ t is the timing differential error, J is the system-level performance index, V(s t+1 ) is the next state value estimate, V(s t ) is the current state value estimate; Update policy network parameters: Among them, θ new is the updated policy network parameter, θ old is the original policy network parameter, η is the policy network learning rate, is the policy gradient; If ||δ t ||<0.1, triggering the early stopping mechanism: η←0.5η and α←0.8α.

Images