A method for optimizing thermal management of energy storage cabinets based on deep reinforcement learning

Abstract

The application provides a kind of energy storage electric cabinet thermal management optimization method based on deep reinforcement learning, belongs to the battery thermal management field based on reinforcement learning technical field;First, the electric cabinet thermal state data acquisition and physical characteristics are constructed, and the standardized electric cabinet thermal state distribution prediction is output;Then, the physical information neural network is constructed based on heat transfer mechanism to realize the battery temperature at future time, finally, the action under the current state is output based on the physical perception map attention strategy network trained by reinforcement learning, and the action is mapped into energy storage electric cabinet thermal management control instruction after being clipped in safety range.The application proposes an electric cabinet level thermal management strategy optimization method based on the physical perception map attention strategy network and double-flow physical constraint value network based on reinforcement learning, converts the continuous action output by deep reinforcement learning into executable instructions that meet the safety of battery and the constraints of actuator, thereby improving the landing reliability of thermal management strategy in actual energy storage electric cabinet.

Description

Technical Field

[0001] This invention belongs to the technical field of battery thermal management based on reinforcement learning, and particularly relates to an optimization method for thermal management of energy storage cabinets based on deep reinforcement learning. Background Technology

[0002] With the large-scale deployment of electrochemical energy storage systems in grid peak shaving, renewable energy consumption, and industrial and commercial backup power, the safe operation and energy efficiency management of energy storage cabinets are receiving increasing attention. Energy storage cabinets typically integrate dozens to hundreds of battery cells. During charging and discharging, these cells generate Joule heat and reaction heat. If this heat cannot be dissipated evenly and promptly, it will lead to excessively high local temperatures, widening temperature differences between cells, accelerating aging, and inducing the risk of thermal runaway. Therefore, how to reduce the energy consumption of the thermal management system while ensuring the safety of the battery cell temperature has become a key issue in the operation and control of energy storage cabinets.

[0003] Currently, the main methods for thermal management of energy storage cabinets include the following: (1) Rule-based and threshold-based thermal management method: This method uses signals such as cell temperature and current collected by the Battery Management System (BMS) to start the fan or liquid cooling circuit when the temperature exceeds a fixed threshold. This method is simple to implement and has a low cost, but the fixed threshold is difficult to balance the temperature control accuracy and energy consumption requirements under different operating conditions, does not make full use of the thermal coupling information inside the cabinet, and has insufficient adaptability across operating conditions; (2) Thermal management method based on mechanism model or reduced-order model: This method models and optimizes the thermal behavior of the electrical cabinet through computational fluid dynamics (CFD) simulation, lumped parameter thermal model or model predictive control (MPC). This method has clear physical interpretability, but the mechanism model has a large computational load, is difficult to deploy online, and has insufficient robustness to parameter drift and unknown operating conditions; (3) Thermal management method based on pure data-driven neural networks: This method uses data-driven models such as Long Short-Term Memory (LSTM) networks and Artificial Neural Networks (ANN) to predict or control the temperature of the battery cell. This type of method can capture the statistical regularity in the data, but it lacks physical consistency constraints, requires a large amount of labeled data, and has limited generalization ability and interpretability.

[0004] In summary, existing thermal management methods for energy storage cabinets have certain limitations in terms of temperature control accuracy and energy consumption balance, online computational efficiency, physical consistency constraints, and cross-condition generalization ability. Rule-based and threshold-based methods are difficult to optimize dynamically, mechanistic model-based methods have limited online deployment, and purely data-driven methods lack physical constraints. Summary of the Invention

[0005] To address the above problems, this invention provides a method for optimizing the thermal management of energy storage cabinets based on deep reinforcement learning, comprising the following steps: S1: Collect surface temperature of each cell, total current of the cabinet, total voltage of the cabinet, coolant flow rate, and fan speed to construct a cabinet-level multi-source sensing tensor; collect voltage, internal resistance, and current of each cell to construct a cell-level state monitoring tensor; construct an adjacency matrix and a spatial distance matrix based on the cell arrangement structure and heat conduction path to form a battery module topology tensor; calculate volumetric heat generation rate, thermal diffusion potential, and cell thermal property parameters and splice them into a cell physical characteristic tensor. S2: Four types of tensors are spliced ​​together to form a standardized multi-scale fusion feature tensor of the thermal state of the electrical cabinet and input into the physical information neural network. The temporal feature tensor, spatial feature tensor, physical feature tensor and cell state feature tensor are extracted and spliced ​​together to form a fusion feature tensor. Through the physical-data collaborative prediction head, the predicted distribution of cell temperature at future moments is output. S3 uses a physical perception graph attention strategy network constructed by reinforcement learning to perform online reasoning on the predicted distribution of cell temperature, coolant flow rate, fan speed and total current of the electrical cabinet at future moments. It outputs coolant flow rate, fan speed and current limit values, limits the output within a safe range and restricts its rate of change, and then maps it to the liquid cooling pump PWM duty cycle, fan PWM duty cycle and the maximum allowable current of the converter.

[0006] Preferably, the cell-level state monitoring tensor is transmitted through the cell management unit. The voltage and internal resistance of each cell are collected in seconds. The total current of the cabinet is distributed to each cell according to the number of parallel branches to obtain the current of each cell. The voltage, internal resistance, and current of each cell are arranged according to the cell number to form the cell-level status monitoring tensor. The battery module topology tensor is constructed based on the cell arrangement structure inside the battery cabinet. It uses cells as nodes, thermal conductivity relationships between adjacent cells as edges to construct an adjacency matrix, and spatial distance distances between adjacent cells as edge weights to construct a spatial distance matrix. Together, they constitute the battery module topology tensor.

[0007] Preferably, when constructing the physical characteristic tensor of the battery cell, firstly, the volumetric heat generation rate of each battery cell is calculated based on the Bernardi heat generation model. Secondly, the thermal diffusion potential between each cell and adjacent cells is calculated based on the discretized heat conduction equation. Finally, the volumetric heat generation rate, thermal diffusion potential, and thermal property parameters of each cell are concatenated into a cell physical characteristic tensor. The cell thermal property parameters include density. Specific heat capacity With thermal conductivity .

[0008] Preferably, the physical information neural network includes: The temporal coding branch uses a two-layer long short-term memory network to encode the temporal dimension N of the cabinet-level multi-source sensing tensor. Each layer has 64 hidden units, and the output is a temporal feature tensor with a dimension of M×64. A spatial encoding branch is constructed, and a two-layer graph convolutional network is used to encode the cell node dimension M using the adjacency matrix of the battery module topology tensor as the graph structure. Each layer has 32 output channels and outputs a spatial feature tensor with dimension M×32. The physical coding branch uses two fully connected layers to encode the volumetric heat generation rate, thermal diffusion potential, and thermal property parameters of the battery cell in the physical feature tensor. Each layer has 32 neurons and outputs a physical feature tensor with a dimension of M×32. The cell state projection branch uses a single fully connected layer to dimensionally align the cell-level state monitoring tensor, mapping the voltage, internal resistance, and current in the cell-level state monitoring tensor to an M×32 cell state feature tensor, so that the cell state features are in the same semantic space as the temporal, spatial, and physical features after network encoding.

[0009] Preferably, the training process of the physical perception graph attention policy network constructed using reinforcement learning is as follows: First, the thermal management problem of the energy storage cabinet is modeled as a Markov decision process, where the state consists of the cell temperature distribution, coolant flow rate, fan speed, and total cabinet current at historical moments; the action consists of the coolant flow rate, fan speed, and current limit for the next control cycle; the reward function is a weighted sum of temperature safety terms, temperature difference uniformity terms, and cooling system energy consumption terms; second, a physical perception graph attention policy network is constructed, including a temporal Transformer encoder, a graph attention encoder, and a conditionally normalized flow action sampling head, outputting continuous actions and conditional log probabilities; third, a dual-flow physical constraint value network is constructed, including state value branches. The system includes an action value branch and a physical constraint gating module. The state value branch reuses the output of the temporal Transformer encoder and the graph attention encoder to produce scalar state values, while the action value branch separately encodes continuous actions to produce scalar action values. The gating module dynamically fuses the two value estimates based on the degree to which the historical cell temperature approaches the safety threshold to obtain a scalar value estimate. The soft actor-commentator algorithm is used to jointly train the physical perception graph attention policy network and the two-stream physical constraint value network. The physical perception graph attention policy network updates the conditional log probability with the goal of maximizing the expected cumulative reward and policy entropy, while the two-stream physical constraint value network updates the value estimate based on the scalar value estimate with the goal of minimizing the temporal difference error.

[0010] Preferably, the physical perception graph attention strategy network specifically comprises: First, a temporal Transformer encoder is constructed, using two Transformer encoder layers to encode the historical state sequence. Each layer has a hidden dimension of 128 and four attention heads. Second, a graph attention encoder is constructed, using two graph attention layers to encode the cell temperature information contained in each state vector of the encoded historical state sequence to obtain hidden features. Each layer contains four attention heads, with 64 output channels and an ELU activation function. Finally, a conditionally normalized flow action sampling head is constructed, using four affine coupled flow layers to transform the hidden features into continuous action vectors. Each flow layer consists of two fully connected layers with 256 neurons and a Swish activation function. The policy network simultaneously outputs the action and its conditional log probability. .

[0011] Preferably, the dual-stream physical constraint value network specifically comprises: A state value estimation branch is constructed, sharing the graph attention encoder and temporal Transformer encoder of the physical perception graph attention policy network, outputting 128-dimensional state value features, which are then mapped to scalar state values ​​through a fully connected projection layer. Secondly, an action value estimation branch is constructed to evaluate the expected cumulative reward of performing a specific action in a given state. Two fully connected layers are used to encode the action vector 'a', with 128 neurons in each layer and ReLU activation function, outputting 128-dimensional action value features. These features are then mapped to a scalar action value through a fully connected projection layer. Secondly, a physical constraint gating module is constructed. Based on the proximity of the average cell temperature in the historical cell temperature distribution to the safety threshold, gating weights are generated for the first cell. Temperature proximity of individual battery cells ,in This is the average cell temperature. For safe temperature threshold, The temperature margin scaling factor is used to control the width of the gating transition region. A fully connected layer then compresses the concatenated proximity vector, which is then normalized to the (0,1) interval using a Sigmoid activation function to obtain the scalar gating weights. Finally, the scalar state value, scalar action value, and gating weights are fused to output a scalar value estimate. .

[0012] Preferably, limiting the output within a safe range and restricting its rate of change specifically involves: Current limit Limited to the maximum continuous discharge current allowed by the cell. With minimum operating current Between, and the absolute value of the rate of change of the current limit does not exceed When the average highest temperature in the predicted cell temperature distribution exceeds the emergency threshold at a future time. At that time, the current limit will be immediately reduced to And force the cooling system to operate at maximum power. Based on the requirement for early warning of thermal runaway in battery cells, a temperature of 5°C is set to ensure sufficient safety margin; Take 0.5 To obtain safe current limits Coolant flow rate Restricted to 0 and Between, and the absolute value of the coolant flow rate change rate does not exceed To obtain a safe coolant flow rate Fan speed Restricted to 0 and Between, and the absolute value of the rate of change of the fan speed does not exceed To obtain a safe fan speed ; , , and The change rate limit was obtained based on the cell specifications and the capability calibration of the cooling system actuators. , and The calibration was obtained based on the step response test of the liquid-cooled pump and the fan under standard operating conditions.

[0013] Preferably, in step S3, the safe coolant flow rate is... The mapping is to the PWM duty cycle of the liquid cooling pump, and the mapping relationship is as follows: ,in The maximum flow rate of the liquid cooling pump; next, the safe fan speed. The mapping is to the PWM duty cycle of the wind turbine, and the mapping relationship is as follows: ,in The maximum speed of the fan; finally, the safe current limit. As the maximum allowable current value issued by the converter, the PWM duty cycle of the liquid-cooled pump, the PWM duty cycle of the fan, and the maximum allowable current value issued by the converter constitute the thermal management control command of the energy storage cabinet.

[0014] Compared with the prior art, the present invention has the following innovative features: (1) A physical information neural network thermal state prediction method based on heat transfer mechanism is proposed. The Bernardi heat generation model and the residual of the discretized energy conservation equation are embedded into the neural network training to realize the integration of data-driven and physical consistency, thereby improving the accuracy and interpretability of future cell temperature prediction. (2) A cabinet-level thermal management strategy optimization method based on physical perception graph attention strategy network and dual-flow physical constraint value network is proposed. The graph attention mechanism is used to encode the spatial thermal coupling relationship between cells, the conditional normalized flow action sampling head is used to model the continuous action distribution, and the physical constraint gating dynamic adjustment value estimation is used to output a thermal management control strategy that adapts to the thermal state of the cabinet. (3) A method for generating thermal management control instructions based on safety constraints and execution protection is proposed. Through action space pruning, rate of change limitation and fault degradation mechanism, the continuous actions output by deep reinforcement learning are converted into executable instructions that meet the constraints of cell safety and actuator, thereby improving the reliability of thermal management strategy in actual energy storage cabinet. Attached Figure Description

[0015] Figure 1 This is a complete flowchart of the overall technical route of the present invention.

[0016] Figure 2 This is a flowchart of the thermal status data acquisition and physical feature construction of the electrical cabinet of the present invention.

[0017] Figure 3 This is a schematic diagram of the physical information neural network structure of the present invention.

[0018] Figure 4 This is a flowchart illustrating the generation and closed-loop execution of thermal management control commands in this invention.

[0019] Figure 5 This is a diagram illustrating the optimized architecture of the deep reinforcement learning thermal management strategy of this invention.

[0020] Figure 6 This is a comparison chart of the predicted cell temperature at future times in an embodiment of the present invention.

[0021] Figure 7 This is a diagram illustrating the action trajectory of the thermal management control strategy in an embodiment of the present invention.

[0022] Figure 8 This is a comparison chart of energy consumption and temperature safety of different thermal management methods in the embodiments of the present invention.

[0023] Figure 9 This is a comparison chart of ablation experiment results in the embodiments of the present invention. Detailed Implementation

[0024] This invention provides a method for optimizing the thermal management of energy storage cabinets based on deep reinforcement learning. The overall technical process is as follows: Figure 1 As shown: S1, Thermal Status Data Acquisition and Physical Feature Construction of the Energy Storage Cabinet: This step aims to acquire the operating status data of the energy storage cabinet and construct physical features that integrate the heat transfer mechanism. First, temperature, current, and voltage sensors are placed inside the energy storage cabinet to sample the surface temperature of each cell, the total current of the cabinet, and the total voltage of the cabinet. Coolant flow rate and fan speed are read from the liquid cooling pump controller and fan controller to construct a cabinet-level multi-source sensing tensor. Second, the voltage, internal resistance, and current of each cell are collected through the cell management unit to construct a cell-level status monitoring tensor. Based on the cell arrangement structure and heat conduction path, an adjacency matrix and a spatial distance matrix are constructed to form the battery module topology tensor. Third, based on Bernardi products... The thermal model calculates the volumetric heat generation rate of each cell and the thermal diffusion potential between each cell and adjacent cells based on the discretized heat conduction equation. The volumetric heat generation rate, thermal diffusion potential, and cell thermal property parameters are concatenated into a cell physical feature tensor. Finally, the cabinet-level multi-source sensing tensor, cell-level state monitoring tensor, battery module topology tensor, and cell physical feature tensor are concatenated into a cabinet thermal state multi-scale fusion feature tensor by a sliding time window. After Z-score normalization, the normalized cabinet thermal state multi-scale fusion feature tensor is output. S2, Thermal State Prediction Based on Heat Transfer Mechanism-Assisted Physical Information Neural Network: First, the standardized multi-scale fusion feature tensor of the thermal state of the electrical cabinet is input into the physical information neural network. The temporal feature tensor, spatial feature tensor, physical feature tensor, and cell state feature tensor are extracted through the temporal coding branch, spatial coding branch, physical coding branch, and cell state projection branch, and then concatenated into a fusion feature tensor. Second, the fusion feature tensor is input into the physical-data collaborative prediction head, which outputs the predicted distribution of cell temperature at future times. Abnormal prediction values ​​that exceed the cell operating temperature range or have a negative temperature variance are pruned, marked, and removed. S3, Thermal Management Strategy Optimization and Instruction Execution Based on Deep Reinforcement Learning: This step aims to output a thermal management control strategy adapted to the thermal state of the battery cabinet based on the trained physical perception graph attention strategy network, and convert the control strategy into executable instructions to complete closed-loop control. First, the predicted distribution of cell temperature, coolant flow rate, fan speed, and total current of the battery cabinet at future moments are input into the physical perception graph attention strategy network constructed based on reinforcement learning for online inference, outputting coolant flow rate, fan speed, and current limits. Second, the current limits, coolant flow rate, and fan speed are limited within a safe range and their rate of change is restricted, which is then mapped to the liquid cooling pump PWM duty cycle, fan PWM duty cycle, and converter maximum allowable current. After being encapsulated into CAN communication frames by the battery management system, these are sent for execution, realizing online optimization of the thermal management strategy. The training process of the physical perception graph attention policy network is as follows: First, the thermal management problem of the energy storage cabinet is modeled as a Markov decision process. The state consists of the cell temperature distribution, coolant flow rate, fan speed, and total current of the cabinet at historical moments. The action consists of the coolant flow rate, fan speed, and current limit for the next control cycle. The reward function is composed of a weighted sum of temperature safety terms, temperature difference uniformity terms, and cooling system energy consumption terms. Second, a physical perception graph attention policy network is constructed, which includes a temporal Transformer encoder, a graph attention encoder, and a conditionally normalized flow action sampling head, outputting continuous actions and conditional log probabilities. Third, a dual-flow physical constraint value network is constructed, which includes state value branches, action value branches, and a physical constraint gating module. The state value branch reuses the outputs of the temporal Transformer encoder and the graph attention encoder to produce scalar state values, while the action value branch separately encodes continuous actions to produce scalar action values. The gating module dynamically fuses the two value estimates based on the degree to which the historical cell temperature approaches the safety threshold to obtain a scalar value estimate. The soft actor-commentator algorithm is used to jointly train the physical perception graph attention policy network and the two-stream physical constraint value network. The physical perception graph attention policy network updates the conditional log probability with the goal of maximizing the expected cumulative reward and policy entropy, while the two-stream physical constraint value network updates the value estimate based on the scalar value estimate with the goal of minimizing the temporal difference error.

[0025] The present invention will be further described below with reference to embodiments. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without inventive effort are within the scope of protection of the present invention.

[0026] S1. Acquisition of thermal status data and construction of physical characteristics of electrical cabinet To comprehensively perceive the thermal state of the energy storage cabinet and construct interpretable physical characteristics, the process involves multi-source sensor data fusion and heat transfer mechanism calculations to form a multi-scale fusion feature tensor of the cabinet's thermal state, specifically as follows: Figure 2 As shown.

[0027] S1-1 Cabinet-level Multi-source Sensor Tensor Acquisition: First, temperature sensors are installed on the surface of each battery cell in the energy storage cabinet and at the air inlet and outlet of the cabinet. Current and voltage sensors are installed at the main positive and negative terminals of the cabinet. The sampling period is one second, during which the surface temperature of each battery cell, the total current of the cabinet, and the total voltage of the cabinet are collected. At the same time, the coolant flow rate and fan speed are read from the liquid cooling pump controller and the fan controller to form a cabinet-level multi-source sensing tensor with a dimension of N×5, where N is the length of the time window. Next, the cabinet-level multi-source sensing tensor is timestamped to remove outliers caused by communication packet loss. Finally, the aligned cabinet-level multi-source sensing tensor is output to S1-2.

[0028] S1-2 Construction of Cell-Level State Monitoring Tensor and Battery Module Topology Tensor: First, through the cell management unit... The system collects the voltage and internal resistance of each battery cell in seconds. The total current of the battery cabinet is distributed to each cell according to the number of parallel branches to obtain the current of each cell. The voltage, internal resistance, and current of each cell are arranged by cell number to form a cell-level state monitoring tensor with a dimension of M×3, where M is the total number of cells. Next, based on the cell arrangement structure inside the battery cabinet, an adjacency matrix is ​​constructed with cells as nodes and the thermal conductivity between adjacent cells as edges. A spatial distance matrix is ​​constructed with the spatial distance between adjacent cells as edge weights. These two matrices together constitute the battery module topology tensor. Third, the cell-level state monitoring tensor undergoes consistency verification. Outliers exceeding the physically reasonable range are removed using the 3σ criterion, and missing time points are filled using linear interpolation. Finally, the cell-level state monitoring tensor and the battery module topology tensor are output to S1-3.

[0029] S1-3 Cell Physical Characteristic Tensor Construction: First, the volumetric heat generation rate of each cell was calculated based on the Bernardi heat generation model. The specific calculation formula for the Bernardi heat production model is as follows: ; Where I is the current of each cell allocated in S1-2, R is the internal resistance of each cell collected in S1-2, and T is the surface temperature of each cell collected in S1-1. The open-circuit voltage temperature coefficient is obtained from the cell datasheet or offline calibration experiments; secondly, the thermal diffusion potential between each cell and adjacent cells is calculated based on the discretized heat conduction equation, as detailed below: For each cell thermal diffusion potential Calculate using the formula: ; in For battery cells The adjacent cell set, and Adjacent cells With battery cells Surface temperature, The spatial distance between two battery cells is given. The thermal diffusivity is proportional to the temperature difference between adjacent cells and inversely proportional to the spatial distance. Furthermore, the volumetric heat generation rate, thermal diffusivity, and thermal property parameters of each cell are concatenated to form a cell physical characteristic tensor. The cell thermal property parameters include density. Specific heat capacity With thermal conductivity These parameters are obtained from the cell datasheet or material handbook, and the cell physical characteristic tensor has a dimension of M×5; finally, the cell physical characteristic tensor is output to S1-4.

[0030] S1-4 Generation and Normalization of Multi-Scale Fusion Feature Tensor of Electrical Cabinet Thermal State: First, the cabinet-level multi-source sensing tensor, the cell-level state monitoring tensor, the battery module topology tensor, and the cell physical feature tensor are concatenated according to a time window to obtain the cabinet thermal state multi-scale fusion feature tensor, whose dimension is... ,in The time window length, First, the number of feature channels is determined. Second, the multi-scale fusion feature tensor of the electrical cabinet thermal state is time-aligned to ensure that the timestamps of the data from each sensor are consistent, and training samples and inference samples are constructed with a 60-second sliding window and a step size of 1 second. Third, the aligned multi-scale fusion feature tensor of the electrical cabinet thermal state is Z-score standardized, and the mean and standard deviation of each feature channel on the training samples are calculated. This statistic is then used to normalize the inference samples to obtain the standardized multi-scale fusion feature tensor of the electrical cabinet thermal state. Finally, the standardized multi-scale fusion feature tensor of the electrical cabinet thermal state is output to S2.

[0031] S2. Thermal state prediction based on physical information neural network assisted by heat transfer mechanism. To achieve high-precision prediction of the thermal state of the electrical cabinet and ensure the physical consistency of the prediction results, this invention designs a physical information neural network, specifically as follows: Figure 3 As shown, the structure of the S2 physical information neural network is illustrated: the input is temporally encoded by a long short-term memory network and spatially thermally coupled encoded by a graph convolutional network to obtain a fused feature tensor. The prediction head outputs the predicted distribution of cell temperature at future moments, and the residual of the discretized energy conservation equation is used as a soft-constraint embedding loss function.

[0032] S2-1 Construction of Input and Encoding Layers for Physical Information Neural Networks The system employs a standardized electrical cabinet thermal state multi-scale fusion feature tensor as input to the physical information neural network, with dimensions N×M×K. Next, a temporal encoding branch is constructed, using a two-layer long short-term memory network to encode the temporal dimension N of the electrical cabinet-level multi-source sensing tensor. Each layer has 64 hidden units, outputting a temporal feature tensor with dimensions M×64. Then, a spatial encoding branch is constructed, using a two-layer graph convolutional network (GCN) to encode the cell node dimension M using the adjacency matrix of the battery module topology tensor as the graph structure. Each layer has 32 output channels, outputting a spatial feature tensor with dimensions M×32. Finally, a physical encoding branch is constructed, using two fully connected layers... The FC (Full-Functional) method encodes the volumetric heat generation rate, thermal diffusion potential, and thermal property parameters of the battery cell's physical feature tensor. Each layer has 32 neurons, and the output is a physical feature tensor with a dimension of M×32. Further, a battery cell state projection branch is constructed, using a single fully connected layer to dimensionally align the battery cell-level state monitoring tensor. This maps the voltage, internal resistance, and current in the battery cell-level state monitoring tensor to an M×32 battery cell state feature tensor, placing the battery cell state features in the same semantic space as the temporal, spatial, and physical features encoded by the network. Finally, the temporal feature tensor, spatial feature tensor, physical feature tensor, and battery cell state feature tensor are concatenated into an M×128 fusion feature tensor. This concatenation operation allows the four types of features to complement and fuse at the same battery cell node dimension, preserving the specialized representation capabilities of each branch while also... Explicit information interaction is achieved through dimensional splicing, and the output is sent to S2-2. The above four branches do not operate independently, but rather form a complementary and synergistic relationship: the temporal coding branch captures the temporal evolution of cell temperature and electrical parameters, the spatial coding branch uses the battery module topology to characterize the spatial thermal coupling relationship between cells, the physical coding branch embeds the Bernardi heat generation model and the heat transfer mechanism of thermal diffusion potential into the feature learning process, and the cell state projection branch retains the original information of the cell's current operating state. The coding results of the temporal, spatial, and physical branches together constitute the dynamic-structural-mechanistic representation of the cell's thermal state, which is then fused with the steady-state operating information provided by the cell state projection branch. This injects physical consistency and spatial correlation constraints into the data-driven features, improving the accuracy and interpretability of subsequent temperature predictions.

[0033] S2-2 Probabilistic Cell Temperature Prediction Distribution Output under Physics-Guided Attention Constraints This sub-step aims to map the fused feature tensor to the predicted distribution of cell temperature at future moments based on a physics-guided attention mechanism and heteroscedastic uncertainty modeling, and to ensure the reliability of the prediction results through physical rationality verification. First, the fused feature tensor output from S2-1 is input into the physics-data collaborative prediction head; this prediction head consists of parallel temperature mean estimation branches and temperature variance estimation branches, each branch employing two fully connected layers. The network consists of several layers (FC), each with 128 neurons and a ReLU activation function. The output of the final layer in the temperature variance estimation branch is processed using the Softplus activation function to ensure the temperature variance remains positive. Secondly, a physical guidance attention mechanism is introduced into the temperature mean estimation branch. The volumetric heat generation rate and thermal diffusivity potential in the cell physical feature tensor output by S1-3 are used as physical guidance signals, which are concatenated to obtain a physical guidance tensor of dimension M×2. Thirdly, the physical guidance tensor is input into an attention weight generation network consisting of a single fully connected layer with M neurons. The output is normalized using the Softmax function to obtain physical attention weights of dimension M×1. These weights assign higher response values ​​to cells with larger volumetric heat generation rates or more significant thermal diffusivity potentials. Finally, the physical attention... The weights and fusion feature tensors are multiplied element-wise along the cell dimension to obtain a fusion feature tensor weighted by a physical guidance attention mechanism. Then, the weighted fusion feature tensor is input into the temperature mean estimation branch to output the temperature mean of each cell at future time. Simultaneously, the unweighted fusion feature tensor is input into the temperature variance estimation branch to output the temperature variance of each cell. The temperature mean and temperature variance together constitute the predicted cell temperature distribution at future time, with a dimension of M×2. Next, the physical rationality of the predicted cell temperature distribution at future time is checked. If the temperature mean of a cell exceeds the cell's operating temperature range or the temperature variance is non-positive, the predicted value of that cell is clipped to the corresponding boundary and marked as an anomaly, eliminating anomaly predictions that fail the physical rationality check. Finally, the predicted cell temperature distribution at future time is output to S3.

[0034] S2-3 Physical Equation Residual Constraint Embedding: First, based on the discretized energy conservation equation Construct the physical residual term, where For cell density, For specific heat capacity, Thermal conductivity, The volumetric heat production rate calculated for S1-3, with the time derivative term. Using a first-order difference approximation, the spatial Laplace term ²T adopts a second-order central difference approximation between adjacent cells; Secondly, the predicted distribution of cell temperature at future moments output by the physical information neural network is substituted into the discretized energy conservation equation, and the L2 norm of the equation residual is calculated as the physical loss term. And adopt the volumetric heat production rate reference value. Dimensionlessization of the physical loss term, i.e. , Take the 95th percentile of the absolute value of volumetric heat production rate in the training sample; Secondly, regarding data loss items Use temperature reference value The square of the product is dimensionless, i.e. , Take the 95th percentile of the absolute temperature values ​​in the training samples, and then weight the dimensionless physical loss term and the data loss term accordingly. Perform weighted summation to construct the total loss function. Finally, the Adam optimizer was used to train the physical information neural network with a learning rate of 1e-3. An early stopping strategy was adopted during training, and training was stopped when the total loss on the validation set no longer decreased for 10 consecutive epochs.

[0035] S3. Optimization of Thermal Management Strategies Based on Deep Reinforcement Learning To achieve online optimization of cabinet-level thermal management strategies and generation of executable instructions, this invention designs a deep reinforcement learning framework based on a physical perception graph attention policy network and a dual-stream physical constraint value network, as well as a control instruction generation and execution module. This framework first performs online inference to output a thermal management control strategy, mapping the control strategy to current limits and cooling intensity instructions, which are then executed after safety constraint pruning and execution protection. Subsequently, the process of Markov decision process environment modeling, policy network construction, value network construction, and joint training is described, as follows: Figure 4 and Figure 5 As shown.

[0036] S3-1 Online Inference of Thermal Management Control Strategy: During the online inference phase, the parameters of the physical perception graph attention policy network and the dual-flow physical constraint value network learned during the training phase are solidified. The predicted distribution of cell temperature at future moments is combined with the coolant flow rate, fan speed, and total cabinet current collected and output by S1-1 to form a state vector s, which is then input into the trained physical perception graph attention policy network, and the output is an action vector. ,in For coolant flow rate, For fan speed and This is the current limit.

[0037] S3-2 Safety Constraint Cutting: The motion vector output by S3-1 is subject to safety constraint trimming to ensure that the coolant flow rate, fan speed, and current limits all meet the cell safety and actuator physical constraints; current limit Limited to the maximum continuous discharge current allowed by the cell. With minimum operating current Between, and the absolute value of the rate of change of the current limit does not exceed Furthermore, when the average highest temperature in the predicted cell temperature distribution exceeds the emergency threshold at a future time... At that time, the current limit will be immediately reduced to And force the cooling system to operate at maximum power. Based on the requirement for early warning of thermal runaway in battery cells, a temperature of 5°C is set to ensure sufficient safety margin; Take 0.5 To obtain safe current limits Coolant flow rate Restricted to 0 and Between, and the absolute value of the coolant flow rate change rate does not exceed To obtain a safe coolant flow rate Fan speed Restricted to 0 and Between, and the absolute value of the rate of change of the fan speed does not exceed To obtain a safe fan speed ; , , and The change rate limit was obtained based on the cell specifications and the capability calibration of the cooling system actuators. , and The safe current limit was obtained based on the step response test of the liquid-cooled pump and fan under standard operating conditions. Safe coolant flow rate and safe fan speed Output to S3-3.

[0038] The standard operating condition step response test calibration process is as follows: The liquid-cooled pump and fan are installed in the standard test circuit. Under rated voltage, rated load, and ambient temperature of 25℃, step excitations of 10%, 50%, 90%, and 100% of the rated duty cycle are applied from zero flow and zero speed states, respectively. The flow rate or speed response curves are recorded at a sampling frequency of 1kHz. The rise time of the response curve from the 10% steady-state value to the 90% steady-state value under each step condition is extracted. The average rise time for each operating condition is taken as the dynamic response characteristic time of the actuator; the maximum allowable change within a single control cycle T is set to... ; The coefficient divided by 3 is used to retain a safety margin and suppress overshoot; current limit change rate limit. Based on the current response test of the converter under the same standard operating conditions, and according to the maximum allowable current... With current response rise time Obtained by calibration in the same manner.

[0039] S3-3 Energy Storage Cabinet Thermal Management Control Command Mapping: Safe coolant flow rate The mapping is to the PWM duty cycle of the liquid cooling pump, and the mapping relationship is as follows: ,in The maximum flow rate of the liquid cooling pump; next, the safe fan speed. The mapping is to the PWM duty cycle of the wind turbine, and the mapping relationship is as follows: ,in The maximum speed of the fan; finally, the safe current limit. As the maximum allowable current value issued by the converter, the PWM duty cycle of the liquid-cooled pump, the PWM duty cycle of the fan, and the maximum allowable current value issued by the converter constitute the thermal management control command of the energy storage cabinet.

[0040] S3-4 Instruction Issuance and Execution: The battery management system encapsulates the thermal management control commands of the energy storage cabinet output by S3-3 into CAN communication frames and sends them to the inverter, liquid-cooled pump controller and fan controller. Then, the inverter adjusts the charging and discharging current according to the maximum allowable current value issued by the inverter, the liquid-cooled pump controller adjusts the flow rate according to the PWM duty cycle of the liquid-cooled pump, and the fan controller adjusts the speed according to the PWM duty cycle of the fan.

[0041] S3-5 Post-Execution Temperature Feedback Acquisition and Scrolling Optimization: First, in the next sampling cycle after the control command is executed, the temperature feedback after execution is collected by the temperature sensor, with a dimension of M×1. Second, the temperature feedback after execution is combined with the current current and voltage data to update the cabinet-level multi-source sensing tensor and the cell-level state monitoring tensor. Third, the updated data is input into S1 to reconstruct the standardized cabinet thermal state multi-scale fusion feature tensor. Finally, based on the updated cabinet thermal state multi-scale fusion feature tensor, S2 and S3 recalculate the future cell temperature prediction distribution and thermal management control strategy to achieve online rolling optimization of the thermal management strategy.

[0042] Figure 4 The complete process of S3 control command generation and closed-loop execution is demonstrated: the thermal management control strategy output by S3-1 is mapped into current limit and cooling intensity commands, which are then executed through the battery management system after safety constraint trimming and execution protection, and the temperature feedback after execution is collected and sent back to S1 to achieve online rolling optimization.

[0043] S3-6 Markov Decision Process Environmental Modeling: During model training, the state vector is defined as the combination of cell temperature distribution, coolant flow rate, fan speed and total current of the electrical cabinet at a historical moment; the state vectors of N historical moments are stacked according to time windows to form a historical state sequence; the action vector is defined as the combination of coolant flow rate, fan speed and current limit that needs to be decided in the next control cycle. Then, construct the reward function. ,in For temperature safety items, For the first Average temperature of each cell The upper limit of the safe operating temperature for battery cells, when the temperature of all battery cells is below When this item is 0, the temperature of any cell exceeds [a certain value]. This can result in negative penalties. For the term of uniform temperature difference, This represents the standard deviation of the temperature of all battery cells, used to suppress the widening of temperature differences between battery cells; This is an energy consumption item for the cooling system. and The power consumption coefficients of the liquid cooling pump and the fan are respectively determined according to the actuator specifications. Since the power consumption of the pump and fan is approximately proportional to the cube of the flow rate and the rotational speed, cubic terms are used for modeling. Finally, during the training phase, the three terms are dimensionless based on the 95th quantiles of the temperature safety term, temperature difference uniformity term, and energy consumption term in the training set, making the three terms of the same order of magnitude before setting weights. , , We perform a weighted summation and combine the state space, action space, and reward function into a Markov decision process.

[0044] S3-7 Construction of a Physical Perception Map Attention Strategy Network: The physical perception graph attention policy network comprises a graph attention encoder, a temporal Transformer encoder, and an action sampling head. First, a temporal Transformer encoder is constructed, using two layers to encode the historical state sequence. Each layer has a hidden dimension of 128 and four attention heads. Second, a graph attention encoder is constructed, using two graph attention network (GAT) layers to encode the cell temperature information contained in each state vector of the encoded historical state sequence, obtaining hidden features. Each layer contains four attention heads, has 64 output channels, and uses the ELU activation function. Finally, a conditional normalizing flow (CNF) action sampling head is constructed, using four affine coupled flow layers to transform the hidden features into continuous action vectors. Each flow layer consists of two fully connected layers with 256 neurons and uses the Swish activation function. The policy network simultaneously outputs the action and its conditional log probability. .

[0045] S3-8 Construction of a Two-Stream Physically Constrained Value Network: A two-stream physical constraint value network is used to evaluate the expected cumulative reward of following the current policy in a given state. It includes a state value estimation branch, an action value estimation branch, and a physical constraint gating module. First, the state value estimation branch is constructed, sharing the graph attention encoder and temporal Transformer encoder of the physical perception graph attention policy network, outputting 128-dimensional state value features, which are then mapped to scalar state values ​​through a fully connected projection layer. Secondly, an action value estimation branch is constructed to evaluate the expected cumulative reward of performing a specific action in a given state. Two fully connected layers are used to encode the action vector 'a', with 128 neurons in each layer and ReLU activation function, outputting 128-dimensional action value features. These features are then mapped to a scalar action value through a fully connected projection layer. Secondly, a physical constraint gating module is constructed, which generates gating weights based on the proximity of the average cell temperature in the historical cell temperature distribution to the safety threshold. Specifically, for the first... Temperature proximity of individual battery cells ,in This is the average cell temperature. For safe temperature threshold, The temperature margin scaling factor is used to control the width of the gating transition region. A fully connected layer then compresses the concatenated proximity vector, which is then normalized to the (0,1) interval using a Sigmoid activation function to obtain the scalar gating weights. Finally, the scalar state value, scalar action value, and gating weights are fused to output a scalar value estimate. The fusion formula is .

[0046] S3-9 Joint Training of Policy Network and Value Network: First, a soft actor-critic algorithm is used to jointly train the Physics Perception Graph Attention Policy Network and the Two-Stream Physics Constraint Value Network. The Physics Perception Graph Attention Policy Network updates the conditional log probability with the objective of maximizing the expected cumulative reward and policy entropy, while the Two-Stream Physics Constraint Value Network uses scalar value estimation. Based on this, the value estimate is updated with the goal of minimizing the temporal difference error, with a discount factor γ=0.99, a soft update coefficient τ=0.005, and the temperature parameter α automatically adjusted. Secondly, during the training phase, a replay buffer is used to store state transition samples, with a batch size of 256 and a total training step count of 100,000. Thirdly, during the training phase, historical observation data is used to calculate rewards and update network parameters. Finally, after training converges, the network parameters are solidified for use in the online inference phase.

[0047] Figure 5 The S3 deep reinforcement learning architecture is demonstrated: the physical perception graph attention policy network outputs continuous actions via a graph attention encoder, a temporal Transformer encoder, and a conditionally normalized stream action sampling head; the dual-stream physical constraint value network outputs value estimates via a state value estimation branch, an action value estimation branch, and a physical constraint gating, and the two are jointly trained in the soft actor-critic algorithm framework.

[0048] Simulation experiment: To verify the effectiveness of the proposed deep reinforcement learning-based thermal management optimization method for energy storage cabinets, this experiment uses an energy storage cabinet containing 48 battery cells as the research object. A physical information neural network and a deep reinforcement learning model are constructed using the PyTorch framework in a Python 3.10 environment. Experimental data are generated using a thermal simulation model of the energy storage cabinet based on the Bernardi heat generation model and heat conduction equations, simulating the thermal response process of the cabinet under 1C charge-discharge cycles, ambient temperature variations from 25℃ to 40℃, and different combinations of coolant flow rates and fan speeds. The simulation model has a time step of 1 second and a total simulation duration of 3600 seconds. During the training phase, 10,000 control trajectories are generated using the simulation model and divided into training, validation, and test sets in an 8:1:1 ratio. During the online inference phase, the control performance is evaluated on the test set.

[0049] analyze Figure 6It can be seen that the physical information neural network proposed in this invention, when predicting the highest temperature of the battery cell in the next 60 seconds, shows a prediction curve that is consistent with the trend of the simulated true value, with a root mean square error of 0.38℃, which is better than the 0.71℃ of the pure data-driven long short-term memory network. By introducing residual constraints from the energy conservation equation, the physical information neural network exhibits smaller prediction deviations in the high-temperature rise and cooling response stages, indicating that the heat transfer mechanism assistance can effectively improve the accuracy of thermal state prediction.

[0050] analyze Figure 7 It can be seen that during the 1C charge / discharge cycle of the battery cabinet, the coolant flow rate and fan speed output by the method of this invention dynamically change with the cell temperature. When the cell temperature approaches the safety threshold, the cooling intensity increases to a higher level, and after the temperature drops, the cooling intensity decreases accordingly. The current limit is appropriately reduced when the temperature rises abnormally. The control action shows a continuous and smooth trend, and there is no frequent start-stop phenomenon caused by simple threshold control.

[0051] analyze Figure 8 It can be seen that, compared with the fixed threshold control strategy, the method of the present invention reduces the cooling system energy consumption by approximately 16.2% while keeping the maximum cell temperature below 45℃; compared with the rule-based proportional-integral-derivative control strategy, the maximum temperature difference between cells is reduced by approximately 2.8℃. These results demonstrate that the method of the present invention achieves a good balance between temperature safety and energy consumption.

[0052] analyze Figure 9 It can be seen that after removing the physical residual constraints of the physical information neural network, the root mean square error of temperature prediction increased from 0.38℃ to 0.61℃; after replacing the physical perception graph attention policy network with a general multilayer perceptron policy network, the cooling system energy consumption increased by about 8.5% and the maximum temperature difference between cells increased by about 1.6℃; after removing the safety constraint pruning of S3-2, the control action exceeded the limit multiple times. The above ablation experiments show that physical constraints, graph attention policy structure, and safety pruning mechanism all contribute positively to the overall performance of the method of this invention.

[0053] Table 1 Result Evaluation Table

[0054] Analysis of Table 1 shows that the method of this invention achieves the expected goals in four dimensions: prediction accuracy, control effect, energy efficiency, and ablation experiment. The root mean square error of temperature prediction is less than 0.5℃, the maximum cell temperature is below the safety threshold of 45℃, and the energy consumption of the cooling system is reduced by 16.2% compared to the fixed threshold strategy. The ablation experiment further demonstrates that physical residual constraints, graph attention strategy networks, and safety constraint pruning contribute positively to prediction accuracy, control energy consumption, and operational safety, respectively. These three factors synergistically support the feasibility of the method of this invention in the thermal management of energy storage cabinets.

[0055] The above description is merely a preferred embodiment of this application and is not intended to limit this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

[0056] While the specific embodiments of the present invention have been described above, they are not intended to limit the scope of protection of the present invention. Those skilled in the art should understand that various modifications or variations that can be made by those skilled in the art without creative effort based on the technical solutions of the present invention are still within the scope of protection of the present invention.

Claims

1. A method for optimizing thermal management of energy storage cabinets based on deep reinforcement learning, characterized in that, Includes the following processes: S1: Collect surface temperature of each cell, total current of the cabinet, total voltage of the cabinet, coolant flow rate, and fan speed to construct a cabinet-level multi-source sensing tensor; collect voltage, internal resistance, and current of each cell to construct a cell-level state monitoring tensor; construct an adjacency matrix and a spatial distance matrix based on the cell arrangement structure and heat conduction path to form a battery module topology tensor; calculate volumetric heat generation rate, thermal diffusion potential, and cell thermal property parameters and splice them into a cell physical characteristic tensor. S2: Four types of tensors are spliced ​​together to form a standardized multi-scale fusion feature tensor of the thermal state of the electrical cabinet and input into the physical information neural network. The temporal feature tensor, spatial feature tensor, physical feature tensor and cell state feature tensor are extracted and spliced ​​together to form a fusion feature tensor. Through the physical-data collaborative prediction head, the predicted distribution of cell temperature at future moments is output. S3 uses a physical perception graph attention strategy network constructed by reinforcement learning to perform online reasoning on the predicted distribution of cell temperature, coolant flow rate, fan speed and total current of the electrical cabinet at future moments. It outputs coolant flow rate, fan speed and current limit values, limits the output within a safe range and restricts its rate of change, and then maps it to the liquid cooling pump PWM duty cycle, fan PWM duty cycle and the maximum allowable current of the converter.

2. The method for optimizing thermal management of energy storage cabinets based on deep reinforcement learning as described in claim 1, characterized in that: The cell-level state monitoring tensor is transmitted through the cell management unit. The voltage and internal resistance of each cell are collected in seconds. The total current of the cabinet is distributed to each cell according to the number of parallel branches to obtain the current of each cell. The voltage, internal resistance, and current of each cell are arranged according to the cell number to form the cell-level status monitoring tensor. The battery module topology tensor is constructed based on the cell arrangement structure inside the battery cabinet. It uses cells as nodes, thermal conductivity relationships between adjacent cells as edges to construct an adjacency matrix, and spatial distance distances between adjacent cells as edge weights to construct a spatial distance matrix. Together, they constitute the battery module topology tensor.

3. The method for optimizing thermal management of energy storage cabinets based on deep reinforcement learning as described in claim 1, characterized in that: When constructing the physical characteristic tensor of the battery cell, firstly, the volumetric heat generation rate of each battery cell is calculated based on the Bernardi heat generation model. Secondly, the thermal diffusion potential between each cell and adjacent cells is calculated based on the discretized heat conduction equation. Finally, the volumetric heat generation rate, thermal diffusion potential, and thermal property parameters of each cell are concatenated into a cell physical characteristic tensor. The cell thermal property parameters include density. Specific heat capacity With thermal conductivity .

4. The method for optimizing thermal management of energy storage cabinets based on deep reinforcement learning as described in claim 1, characterized in that: The physical information neural network includes: The temporal coding branch uses a two-layer long short-term memory network to encode the temporal dimension N of the cabinet-level multi-source sensing tensor. Each layer has 64 hidden units, and the output is a temporal feature tensor with a dimension of M×64. A spatial encoding branch is constructed, and a two-layer graph convolutional network is used to encode the cell node dimension M using the adjacency matrix of the battery module topology tensor as the graph structure. Each layer outputs 32 channels and outputs a spatial feature tensor with dimension M×32. The physical coding branch uses two fully connected layers to encode the volumetric heat generation rate, thermal diffusion potential, and thermal property parameters of the battery cell in the physical feature tensor. Each layer has 32 neurons and outputs a physical feature tensor with a dimension of M×32. The cell state projection branch uses a single fully connected layer to dimensionally align the cell-level state monitoring tensor, mapping the voltage, internal resistance, and current in the cell-level state monitoring tensor to an M×32 cell state feature tensor, so that the cell state features are in the same semantic space as the temporal, spatial, and physical features after network encoding.

5. The method for optimizing thermal management of energy storage cabinets based on deep reinforcement learning as described in claim 1, characterized in that: The training process of the physical perception graph attention policy network constructed using reinforcement learning is as follows: First, the thermal management problem of the energy storage cabinet is modeled as a Markov decision process. The state consists of the cell temperature distribution, coolant flow rate, fan speed, and total cabinet current at historical moments; the action consists of the coolant flow rate, fan speed, and current limit for the next control cycle; the reward function is a weighted sum of temperature safety, temperature difference uniformity, and cooling system energy consumption terms. Second, a physical perception graph attention policy network is constructed, including a temporal Transformer encoder, a graph attention encoder, and a conditionally normalized flow action sampling head, outputting continuous actions and conditional log probabilities. Third, a dual-flow physical constraint value network is constructed, including state value branches, dynamic... The system includes value branching and physical constraint gating modules. The state value branch reuses the output of the temporal Transformer encoder and the graph attention encoder to produce scalar state values, while the action value branch separately encodes continuous actions to produce scalar action values. The gating module dynamically fuses the two value estimates based on the degree to which the historical cell temperature approaches the safety threshold to obtain a scalar value estimate. The soft actor-commentator algorithm is used to jointly train the physical perception graph attention policy network and the two-stream physical constraint value network. The physical perception graph attention policy network updates the conditional log probability with the goal of maximizing the expected cumulative reward and policy entropy, while the two-stream physical constraint value network updates the value estimate based on the scalar value estimate with the goal of minimizing the temporal difference error.

6. The method for optimizing thermal management of energy storage cabinets based on deep reinforcement learning as described in claim 5, characterized in that: The physical perception graph attention strategy network is specifically as follows: First, a temporal Transformer encoder is constructed, using two Transformer encoder layers to encode the historical state sequence. Each layer has a hidden dimension of 128 and four attention heads. Second, a graph attention encoder is constructed, using two graph attention layers to encode the cell temperature information contained in each state vector of the encoded historical state sequence to obtain hidden features. Each layer contains four attention heads, with 64 output channels and an ELU activation function. Finally, a conditionally normalized flow action sampling head is constructed, using four affine coupled flow layers to transform the hidden features into continuous action vectors. Each flow layer consists of two fully connected layers with 256 neurons and a Swish activation function. The policy network simultaneously outputs the action and its conditional log probability. .

7. The method for optimizing thermal management of energy storage cabinets based on deep reinforcement learning as described in claim 5, characterized in that: The dual-stream physical constraint value network is specifically as follows: A state value estimation branch is constructed, sharing the graph attention encoder and temporal Transformer encoder of the physical perception graph attention policy network, outputting 128-dimensional state value features, which are then mapped to scalar state values ​​through a fully connected projection layer. ; Secondly, an action value estimation branch is constructed to evaluate the expected cumulative reward of performing a specific action in a given state. Two fully connected layers are used to encode the action vector 'a', with 128 neurons in each layer and ReLU activation function, outputting 128-dimensional action value features. These features are then mapped to a scalar action value through a single fully connected projection layer. ; Next, a physical constraint gating module is constructed. Gating weights are generated based on the proximity of the average cell temperature in the historical cell temperature distribution to the safety threshold. For the first... Temperature proximity of individual battery cells ,in This is the average cell temperature. For safe temperature threshold, The temperature margin scaling factor is used to control the width of the gating transition region. A fully connected layer then compresses the concatenated proximity vector, which is then normalized to the (0,1) interval using a Sigmoid activation function to obtain the scalar gating weights. ; Finally, the scalar state value, scalar action value, and gating weights are fused to output a scalar value estimate. .

8. The method for optimizing thermal management of energy storage cabinets based on deep reinforcement learning as described in claim 1, characterized in that: The specific steps for limiting the output within a safe range and restricting its rate of change are as follows: Current limit Limited to the maximum continuous discharge current allowed by the cell. With minimum operating current Between, and the absolute value of the rate of change of the current limit does not exceed When the average highest temperature in the predicted cell temperature distribution exceeds the emergency threshold at a future time. At that time, the current limit will be immediately reduced to And force the cooling system to operate at maximum power. Based on the requirement for early warning of thermal runaway in battery cells, a temperature of 5°C is set to ensure sufficient safety margin; Take 0.5 To obtain safe current limits Coolant flow rate Restricted to 0 and Between, and the absolute value of the coolant flow rate change rate does not exceed To obtain a safe coolant flow rate Fan speed Restricted to 0 and Between, and the absolute value of the rate of change of the fan speed does not exceed To obtain a safe fan speed ; , , and The change rate limit was obtained based on the cell specifications and the capability calibration of the cooling system actuators. , and The calibration was obtained based on the step response test of the liquid-cooled pump and the fan under standard operating conditions.

9. The method for optimizing thermal management of energy storage cabinets based on deep reinforcement learning as described in claim 1, characterized in that: In step S3, the safe coolant flow rate is... The mapping is to the PWM duty cycle of the liquid cooling pump, and the mapping relationship is as follows: ,in This is the maximum flow rate of the liquid cooling pump; Next, adjust the safe fan speed. The mapping is to the PWM duty cycle of the wind turbine, and the mapping relationship is as follows: ,in The maximum speed of the fan; finally, the safe current limit. As the maximum allowable current value issued by the converter, the PWM duty cycle of the liquid-cooled pump, the PWM duty cycle of the fan, and the maximum allowable current value issued by the converter constitute the thermal management control command of the energy storage cabinet.

Images