Variable cooling coupled battery fast charging control method based on security reinforcement learning

Abstract

This invention discloses a fast-charging control method for a variable cooling coupled battery based on safety reinforcement learning, comprising: S1. Establishing an electrothermal coupling model of a lithium-ion battery as the environment for reinforcement learning, and constructing a state space and action space; S2. Constructing a safety reinforcement learning agent; S3. Training the safety reinforcement learning agent using a dual-timescale update mechanism: at each simulation time step, performing gradient descent updates on the dual-Critic value network and the Cost Critic safety evaluation network; S4. Inputting the multi-dimensional vector of the battery's current operating state into the trained Actor policy network, and outputting the optimal charging current and heat dissipation command. This invention introduces an reachability constraint mechanism to improve the safety and control stability during the battery's fast charging process, constructs a nonlinear delay relationship between battery surface heat dissipation and core temperature rise, and accurately reserves thermal safety margins for cooling and early heat dissipation.

Description

Technical Field

[0001] This invention relates to the field of electric vehicle and energy storage system technology, specifically to a fast charging control method for variable cooling coupled batteries based on safety reinforcement learning. Background Technology

[0002] Lithium-ion batteries, due to their high energy density and long cycle life, have become the core energy storage component for electric vehicles and large-scale energy storage systems. The Battery Management System (BMS) is crucial for ensuring the safe, efficient, and reliable operation of battery packs. Many functions of the BMS, such as state-of-charge estimation, state-of-health estimation, and equalization control, highly rely on accurate measurement data from voltage, temperature, and current sensors.

[0003] Fast charging technology, as a core component of lithium-ion batteries, has garnered significant attention due to its potential to shorten charging time and alleviate range anxiety. However, simply increasing the charging current can accelerate battery aging, reduce capacity, and pose safety risks due to overheating. Therefore, LIB charging faces challenges, as fast charging and safety are often conflicting goals.

[0004] Current control methods can generally be divided into two categories: the first is model-based heuristic strategies, including constant current constant voltage (CCCV) charging protocols and their various variants, such as multi-stage constant current (MCC), multi-stage CCCV, and boost charging. While less complex, these non-model-based charging strategies are empirically designed, do not consider battery dynamics, and suffer performance degradation due to inappropriate charging parameter selection. The second category is model-based strategies, primarily including equivalent circuit modeling (ECM) and electrochemical modeling. Equivalent circuit models (ECMs) simulate the external characteristics of the battery through circuit elements. However, these models rely on empirical simplifications and struggle to accurately capture the complex electrochemical processes within the battery. Electrochemical models, on the other hand, are based on coupled multiphysics equations, describing the diffusion, reaction, and potential energy distribution of lithium ions in electrode particles from a microscopic mechanism perspective. However, this approach faces challenges such as high computational complexity and difficulty in obtaining parameters.

[0005] To address the shortcomings of existing technologies, there is an urgent need to design a fast-charging control method for variable cooling coupling batteries based on safety reinforcement learning to solve the aforementioned technical problems. Summary of the Invention

[0006] This application discloses a fast charging control method for a variable cooling coupled battery based on security reinforcement learning, including the following steps: S1. Establish an electrothermal coupling model of a lithium-ion battery as the environment for safety reinforcement learning, and construct a state space and an action space; wherein, the state space is a multi-dimensional vector of the battery's operating status; and the action space is a joint control quantity that relates the battery charging efficiency and thermal safety calculations. S2. Construct a safety reinforcement learning agent, which includes an Actor policy network, a dual-Critic value network, a Cost Critic safety evaluation network, and a Lambda network; wherein the dual-Critic value network is used to evaluate the cumulative reward of the charging efficiency, the Cost Critic safety evaluation network is used to evaluate the cumulative reward of thermal safety, and the Lambda network is used to balance the weights between the dual-Critic value network and the Cost Critic safety evaluation network. S3. The security reinforcement learning agent is trained using a dual time-scale update mechanism: at each simulation time step, gradient descent updates are performed on the dual Critic value network and the Cost Critic security evaluation network; after every d simulation time steps, the parameters of the Actor policy network and the Lambda network are updated once, where d is a preset delay update interval. S4. Input the multi-dimensional vector of the current battery operating state into the trained Actor policy network, and output the optimal charging current and heat dissipation command.

[0007] As a further optimization of the above scheme, the electrothermal coupling model in step S1 includes a second-order RC equivalent circuit model of the electrochemical part and a core-surface dual-state thermal model of the thermodynamic part. The core-surface dual-state thermal model establishes the dynamic relationship between the core temperature and the surface temperature of the battery, and the equation is: in, For the core temperature, For surface temperature, For ambient temperature, For current, and These are the core heat capacity and surface heat capacity of the battery, respectively. The internal thermal resistance between the core and the surface; This refers to the battery's internal resistance. The entropy change coefficient, It is Faraday's constant. It represents the variable convective thermal resistance from the surface to the environment.

[0008] As a further optimization of the above scheme, the Cost Critic safety evaluation network in step S2 is used to assess the expected cumulative cost of the current state-action for violating safety constraints, including the maximum permissible voltage. and maximum allowable temperature .

[0009] As a further optimization of the above scheme, after constructing the action space as a joint control variable relating battery charging efficiency and thermal safety calculation, a composite reward function is constructed, which includes charging speed reward, energy consumption penalty, and safety constraint penalty based on reachability analysis. The specific construction includes the following: Construct arbitrary An expression that includes both charging speed reward and energy consumption penalty at all times: in, for The reward value at any moment, The preset charging current value, For preset charging power value, For indicator functions, In a charged state, For the preset state of charge, where , and These are respectively represented as the charging speed weighting coefficient, the energy consumption penalty weighting coefficient, and the energy target reward weighting coefficient; Constructing the instant cost function Training the Cost Critic security evaluation network: In the above formula, and These are the voltage-based and temperature-based values, respectively. for Value of the general agent at any time , , , and This is represented as the current voltage value, the upper voltage limit, the current temperature value, the upper temperature limit, and the lower temperature limit; This is a truncation function.

[0010] As a further optimization of the above scheme, the training of the security reinforcement learning agent based on the dual-timescale update mechanism described in step S3 specifically includes the following steps: S31, Collection and storage of empirical data, including the following steps: In each At each step, the security reinforcement learning agent is substituted into the electrothermal coupling model constructed in step S1 for calculation to obtain the tuple. Store the tuple in the buffer. ; In tuple expressions, for Real-time environmental values, To enhance the security of learning agents in Actions at any moment for The environmental value at the next time step. This indicates the end of this round of interaction; S32, Calculation of Bellman's objective value, the steps include: S321, Action entropy regularization, calculates the entropy term for the next time step: ,in It is a temperature parameter; and These represent the state and action of the next moment, respectively. For Actor policy networks; S322, based on the traditional Bellman equation, introduces a dual-value network minimization and entropy regularization to construct an improved Bellman objective value equation: in, The target value for the Critic value network. As a reward value, Indicates the round termination mask. This is a discount factor used to measure the weight of the reward; This indicates that the dual-value network takes the minimum value. The output of the Critic value network represents the state. ,action The expected long-term cumulative reward for a security reinforcement learning agent; S323, Based on the improved Bellman objective value equation, the improved Bellman operator is used to calculate the Bellman objective value: in, The target value of the Critic value network is calculated based on the improved Bellman operator. For the value of the general agent, The output of the Critic value network computed based on the improved Bellman operator. These are the parameters of the cost network, and the output is in the state. ,action The expected long-term cumulative cost of violations for security reinforcement learning agents.

[0011] As a further optimization of the above solution, step S3 also includes the following steps: S33, Fast Timescale Update: Critic Value Network and Cost Critic Security Evaluation Network, steps include: S331, the dual Critic value network is optimized by minimizing the mean squared error, and the corresponding loss function is as follows: In the above formula, where For batch size, For the goal value; and This is the output of the dual-critic value network. and These are the parameters of two independent Critic value networks, and the output state. ,action The expected cumulative reward; Gradient descent is performed based on the Adam optimizer, and the update formula is as follows: ;in, For the parameters of the Critic value network, ; The learning rate for the Critic value network. This represents the gradient of the loss function with respect to the Critic value network parameters. As the assignment sign, the flag parameter is updated along the gradient direction; S332, optimize the Cost Critic security evaluation network by minimizing the mean square error, and the corresponding loss function is as follows: In the above formula, This is expressed as the loss function of the Cost Critic security evaluation network. The predicted output of the CostCritic security evaluation network. The target value for the Cost Critic security evaluation network; The optimization operations for the Cost Critic security evaluation network are as follows. ,in The parameters for the Cost Critic security evaluation network, The learning rate of the Cost Critic security evaluation network. This represents the gradient of the loss function with respect to the parameters of the Cost Critic security evaluation network.

[0012] S34, Slow Time-Scale Update: Actor Policy Network and Lambda Network, steps include: S341, Construct the Lagrange function In the above formula, For Actor policy networks, For buffer Expectations Let be the logarithmic probability of the strategy. This indicates that the dual-value network takes the minimum value. The output of the Critic value network represents the state. ,action The expected long-term cumulative reward for a security reinforcement learning agent; S342, Update Actor Policy Network Parameters of the Actor policy network Find the gradient of the Lagrange function and perform gradient descent to update the formula: In the above formula, The learning rate of the Actor policy network. This represents the gradient of the loss function with respect to the Actor policy network parameters; S343, Update Lambda Network Based on the Lambda network, the tightest constraint bound is obtained, and the loss function is defined as follows: in It is a preset safety tolerance threshold.

[0013] S35, for all target networks that do not participate in gradient backpropagation, updates are only performed through moving average: The target network includes Target Actor, Target Critic, and Target Cost Critic. For the target network parameters, For local network parameters, This is the smoothing coefficient.

[0014] As a further optimization of the above scheme, the second-order RC equivalent circuit model includes a voltage source to simulate the battery open-circuit voltage, at least one series ohmic internal resistance and a parallel polarized RC network to capture the terminal voltage response characteristics of the battery under different states of charge and dynamic currents. The core-surface dual-state thermal model is used to calculate the heat generated inside the battery due to Joule heating and electrochemical reactions, as well as to calculate and describe the nonlinear physical process of heat conduction from the battery core to the surface and then dissipation from the surface to the environment through convection.

[0015] This invention also provides a variable cooling coupling battery fast charging control system based on safety reinforcement learning, the system comprising: An environment generation module is used to establish an electrothermal coupling model of a lithium-ion battery as an environment for safety reinforcement learning, and to construct a state space and an action space; wherein, the state space is a multi-dimensional vector of the battery's operating status; and the action space is a joint control quantity that relates the battery's charging efficiency and thermal safety calculations. An agent construction module is used to construct a safety reinforcement learning agent, which includes an Actor policy network, a dual-Critic value network, a Cost Critic safety evaluation network, and a Lambda network; wherein the dual-Critic value network is used to evaluate the cumulative reward of charging efficiency, the Cost Critic safety evaluation network is used to evaluate the cumulative reward of thermal safety, and the Lambda network is used to balance the weights between the dual-Critic value network and the Cost Critic safety evaluation network. The agent training module is used to train the security reinforcement learning agent using a dual time-scale update mechanism: at each simulation time step, gradient descent updates are performed on the dual Critic value network and the Cost Critic security evaluation network; every d simulation time steps, the parameters of the Actor policy network and the Lambda network are updated once, where d is a preset delay update interval. The instruction output module is used to input the multi-dimensional vector of the current battery operating state into the trained Actor policy network and output the optimal charging current and heat dissipation instructions.

[0016] The present invention also provides a computer-readable storage medium storing computer program instructions, which, when executed by a processor, implement the steps of the variable cooling coupled battery fast charging control method based on security reinforcement learning as described above.

[0017] The present invention also provides a computer program product in which the instructions, when executed by the processor of an electronic device, cause the electronic device to perform the steps of the variable cooling coupled battery fast charging control method based on security reinforcement learning as described above.

[0018] The present invention adopts the above-described technical solution, and compared with the prior art, it has the following beneficial effects: 1. This invention innovatively introduces an reachability constraint mechanism, significantly improving safety and control stability during battery fast charging. By constructing an adaptive safety architecture based on a Cost Critic safety evaluation network and a Lambda network, it achieves early warning and proactive avoidance of risks: the Cost Critic safety evaluation network predicts the cumulative probability risk of violating voltage or temperature boundaries over a future period based on the current state-action pairs; the Lambda network dynamically adjusts the penalty weights according to the risk prediction value; this not only fundamentally eliminates hard limit violations at the algorithmic level, but also avoids the severe oscillations at the boundaries of traditional PID control, significantly extending the battery's cycle life.

[0019] 2. This invention establishes a precise electrothermal coupling model to establish a nonlinear delay relationship between battery surface heat dissipation and core temperature rise. This not only achieves instant cooling but also allows for preemptive increases in fan power before the arrival of high-current pulses, reserving a thermal safety margin. The active thermal management capability of this invention enables the battery to maintain a high-rate current for a longer period during the initial and middle stages of charging, fully utilizing the battery's heat absorption capacity at low temperatures. Simultaneously, the introduced energy consumption penalty term prompts the intelligent agent to find the optimal fan power while ensuring safety, avoiding energy waste caused by blindly running the fan at full speed. This achieves the dual optimization goals of shortening charging time and improving overall system energy efficiency. Attached Figure Description

[0020] Other features, objects, and advantages of this application will become more apparent from the following detailed description of non-limiting embodiments with reference to the accompanying drawings: Figure 1 This is a schematic diagram of the process of the present invention; Figure 2 This is the output curve of charging current and fan power when the penalty weight coefficient is set to 0.8 according to the present invention; Figure 3 This is the output graph of voltage, temperature and SOC results when the penalty weight coefficient is set to 0.8 according to the present invention. Detailed Implementation

[0021] It should be understood that the specific embodiments described herein are for illustrative purposes only and are not intended to limit the scope of the invention.

[0022] like Figure 1-3 As shown in the figure, this application discloses a fast charging control method for a variable cooling coupled battery based on security reinforcement learning, which specifically includes the following steps: S1. Establish an electrothermal coupling model of a lithium-ion battery as the environment for safety reinforcement learning, and construct a state space and an action space. The state space is a multi-dimensional vector representing the battery's operating status; the action space is a joint control variable relating battery charging efficiency and thermal safety calculations. Specifically, the electrothermal coupling model includes a second-order RC equivalent circuit model for the electrochemical part and a core-surface two-state thermal model for the thermodynamic part. The electrochemical part uses a second-order RC equivalent circuit model, which includes a voltage source to simulate the battery's open-circuit voltage, as well as a series ohmic internal resistance and a parallel polarized RC network, used to accurately capture the battery's terminal voltage response characteristics under different states of charge and dynamic currents. The thermodynamic part uses a core-surface two-state thermal model, abandoning the single-node temperature assumption. This thermal model not only calculates the heat generated inside the battery due to Joule heating and electrochemical reactions, but also describes in detail the nonlinear physical process of heat conduction from the battery core to the surface, and then dissipation from the surface to the environment through convection heat transfer. The process is particularly noteworthy. A variable convective heat transfer coefficient is introduced into the thermal model, and a functional mapping is established between it and the cooling fan power. This allows for the simulation of the real-time impact of active cooling on battery temperature evolution. The state space and dynamic space are constructed to formalize the battery fast-charging optimization problem as a constrained Markov decision process. To comprehensively perceive the battery's operating status, the state space is defined as a multi-dimensional vector, containing real-time monitored battery state of charge, terminal voltage, and estimated core and surface temperatures. To achieve synergistic optimization of charging efficiency and thermal safety, the action space is defined as a joint control variable, including the charging current rate and the cooling system's fan power. This joint control variable serves as the system input to the electrothermal coupling model, while the actual observable terminal voltage and surface temperature serve as the system output. The joint action design allows the control strategy to actively adjust the fan power for pre-cooling or energy saving while increasing charging speed through high current, thereby maximizing system energy efficiency while meeting safety constraints. S2. Construct a safety reinforcement learning agent, which includes an Actor policy network, a dual-Critic value network, a Cost Critic safety evaluation network, and a Lambda network. The dual-Critic value network evaluates the cumulative reward of charging efficiency, and the Cost Critic safety evaluation network evaluates the cumulative reward of thermal safety. The Lambda network balances the weights between the dual-Critic value network and the Cost Critic safety evaluation network. Specifically, the Actor policy network receives the current battery state and outputs a deterministic optimal joint control command. The dual-Critic value network evaluates the expected cumulative reward of the current action in parallel, eliminating overestimation bias by taking the minimum of the two values. The Cost Critic safety evaluation network is independent of task rewards and is specifically used to predict the cumulative risk probability of the current strategy leading to battery overheating or overcharging in the future. The Lambda network acts as a dynamic weight adjuster, adaptively outputting Lagrange multipliers based on the risk level fed back by the Cost Critic network to adjust the weights of safety constraints in the optimization objective. S3. The safety reinforcement learning agent is trained using a dual-timescale update mechanism: at each simulation time step, gradient descent updates are performed on the dual Critic value network and the Cost Critic safety evaluation network; every d simulation time steps, the parameters of the Actor policy network and the Lambda network are updated once, where d is a preset delayed update interval; specifically, to solve the timescale mismatch problem between the large inertia of battery thermal response and the fast dynamics of electrical response, a frequency domain separation parameter update strategy is implemented; the Critic value network and the Cost Critic safety evaluation network are updated with gradient descent at each simulation time step to keenly capture small changes in environmental state and quickly correct value assessment errors; the Actor policy network and the Lambda network adopt a delayed update mechanism, that is, the parameters are adjusted only once every few value network update steps, and the combination of fast and slow rhythms acts as a low-pass filter, effectively smoothing the output trajectory of the control strategy and avoiding violent oscillations in charging current caused by valuation noise, thereby protecting battery life; S4. Input the multi-dimensional vector of the current battery operating state into the trained Actor policy network, and output the optimal charging current and heat dissipation command. Specifically, after training, the current battery state is input into the Actor policy network, which outputs the optimal charging current and fan power command to perform closed-loop control of the battery. After the offline training has fully converged, the Actor policy network is extracted and deployed in the actual controller. During the online operation phase, the current battery state data is collected or observed in real time, normalized, and then input into the Actor policy network. The network outputs the optimal charging current setpoint and fan power command in milliseconds, controlling the power system and thermal management system respectively. This process forms a closed-loop control, ensuring that the battery always operates along the trajectory of highest efficiency and strictly meets the temperature and voltage safety boundaries throughout the entire charging cycle.

[0023] Specifically, the electrothermal coupling model constructed in S1 of this embodiment aims to build a virtual environment that can replace a real physical battery. This electrothermal coupling model is used to reflect the nonlinear polarization effect and thermal hysteresis effect under high-rate current. S11, Constructing a second-order RC equivalent circuit model: In order to balance computational efficiency and simulation accuracy, this embodiment selects a second-order RC equivalent circuit model.

[0024] S111, Defining State Variables and Differential Equations: The model contains two core state variables: state of charge and polarization voltage. The evolution of SOC follows the ampere-hour integral method, and its differential form is: In the above formula This is the load current (charge is negative and discharge is positive, unit: ampere A). This refers to the battery's nominal capacity. For Coulomb efficiency, For time variables, 3600 is the time unit conversion factor; polarization voltage The dynamic processes of concentration polarization and electrochemical polarization within the battery are reflected by parallel polarization resistors. and polarization capacitor The description is as follows: Its differential equation is: This equation reflects the inertial characteristics of voltage response, meaning that the voltage does not reach a steady state instantaneously with a sudden change in current, but rather changes exponentially.

[0025] S112, Output equation and parameter nonlinear mapping: Battery terminal voltage It consists of open-circuit voltage, ohmic voltage drop, and polarization voltage: In this step, the key lies in the nonlinearization of the model parameters, specifically the open-circuit voltage parameter. These are not constants, but rather related to SOC and temperature. The binary nonlinear function; in this embodiment, offline data is obtained through hybrid pulse power characteristic test (HPPC), and real-time calculation is performed in the simulation using a two-dimensional lookup table. For example, at low SOC or low temperature, the ohmic resistance The voltage increases significantly, causing the charging voltage to rise rapidly to the cutoff voltage, thus limiting the charging speed.

[0026] S12, Constructing a core-surface dual-state thermal model: To address the heat accumulation and conduction delay during fast charging, a single-node model cannot describe the dangerous "internal heat, external cold" condition. Therefore, this embodiment implements a core-surface dual-state thermal model. S121, Precise calculation of heat production power: Total heat production power It consists of two parts: irreversible Joule heating. and reversible entropy heat ; Specifically, this embodiment introduces the entropy-heat coefficient. Within a specific SOC range, this coefficient may be positive or negative, indicating that there is an endothermic or exothermic reaction during battery charging and discharging. In high-current fast charging, although Joule heating is dominant, the introduction of entropy heating improves the model accuracy by about 0.5℃ to 1.5℃, which is crucial for safety boundary control.

[0027] S122, Differential Equations for Radial Conduction and Convection Heat Dissipation: Defining Core Temperature and surface temperature The coupled differential equations: in, These are the core heat capacity and surface heat capacity of the battery, respectively (J / K). The internal thermal resistance (K / W) between the core and the surface. The variable convection thermal resistance (K / W) from the surface to the environment. The ambient temperature.

[0028] Physical meaning analysis: Equation (15) shows that the change in core temperature depends on the amount of heat generated and the amount of heat dissipated outward; Equation (2) shows that the surface temperature depends on the heat received from the core and the heat dissipated to the environment, which explains why the surface temperature rises slowly in the early stage of high current charging (thermal inertia) and why the surface temperature may rebound after charging stops.

[0029] S13, Nonlinear Modeling and Coupling of the Cooling System: To achieve variable cooling coupling, establish the relationship between fan power and... The functional relationship; S131, Action Space Mapping: Cooling Action Output by the Agent First, it is mapped to the dimensionless power ratio. : Actual power consumption ; S132, Forced Convection Heat Transfer Coefficient Modeling: Based on the empirical correlation between the Nusselt number and the Reynolds number, the convection heat transfer coefficient... With wind speed The relationship is exponential, while wind speed is related to fan power. This embodiment simplifies it to the following nonlinear equation: in It is the natural convection coefficient. It is the gain coefficient. It is a non-linear exponent, indicating that as the fan power increases, the heat dissipation effect improves, but the marginal benefit of the improvement decreases.

[0030] S133, Variable thermal resistance coupling: Ultimate variable convection thermal resistance for: in Given the battery surface area, this formula was actually substituted into the differential equation of step S122, realizing the direct intervention of the control action on the physical environment.

[0031] Specifically, in embodiment S1 of this invention, the optimization problem of fast battery charging is formalized into a constrained Markov decision process by constructing a state space and a dynamic space, transforming the physical problem into a mathematical form solvable by reinforcement learning. The process is as follows: S14, State Space Definition and Normalization: To eliminate the influence of dimensional differences on the gradient of a neural network, the state vector must be strictly normalized. State vector construction: ,choose Instead As a state, it is because It is the direct object of security constraints, and in actual deployment, Estimated by the observer; Normalization process: The final input to the network .

[0032] S15, Combined Action Space Definition: Define joint control variables to achieve electro-thermal synergistic optimization: in Control the charging current rate. Control fan power; S16, Design of the Composite Reward Function and Cost Function Specifically, to achieve fast charging, the reward function provides positive incentives for high charging current; to reduce system operating costs, an energy consumption penalty term for fan power is introduced to encourage the agent to avoid ineffective overcooling; and a constraint penalty based on safety reachability analysis is introduced: when the Cost Critic safety evaluation network predicts that the current state is in a high-risk area, i.e., there is a high probability that irreversible temperature or voltage exceedances will occur in the future, a large penalty term is introduced to force the policy update direction to quickly retreat back to the safety boundary, thereby transforming hard physical constraints into soft optimization objectives. S161, Task Reward: Build An expression that includes both charging speed reward and energy consumption penalty at all times: in, for The reward value at any moment, The preset charging current value, For preset charging power value, For indicator functions, In a charged state, For the preset state of charge, where , and These are respectively represented as the charging speed weighting coefficient, the energy consumption penalty weighting coefficient, and the energy target reward weighting coefficient; Note: Due to Since the value is negative (in the direction of charging), we take a negative sign to make it a positive reward. The highest weight is given to fast charging; With a smaller weight, unnecessary full-power cooling is suppressed; S162. Cost of Violation: This is a core component of security control, involving the construction of an immediate cost function. Training the CostCritic security evaluation network: In the above formula, and These are the voltage-based and temperature-based values, respectively. for Value of the general agent at any time , , , and This is represented as the current voltage value, the upper voltage limit, the current temperature value, the upper temperature limit, and the lower temperature limit; This is a truncation function; The costs are in vector form, corresponding to temperature and voltage constraints, respectively.

[0033] Specifically, in embodiment S2 of the present invention, a secure reinforcement learning agent is constructed, comprising five deep neural networks. The specific construction and execution process of the secure reinforcement learning agent includes: Network initialization operations: Construct and initialize the Actor policy network, dual Critic value network, Cost Critic security evaluation network, and Lambda network; among them, the dual Critic value network adopts a parallel architecture to mitigate value overestimation, the Cost Critic network is independent of task rewards and is specifically used to establish the mapping relationship from the state-action space to security violation risk; the Lambda network is designed to output non-negative Lagrange multipliers to dynamically adjust the weight of security constraints in the optimization objective; Experience replay and sampling: Establish an experience replay buffer of preset capacity, and store experience tuples containing the current state, executed action, environmental reward, safety cost, next state and termination flag in each control cycle; when the amount of data in the buffer reaches the warm-up threshold, a random batch sampling strategy is used to extract experience batches to break the temporal correlation of data and improve training stability. Construction of a multi-objective composite loss function: A composite loss function is constructed for the Actor network that maximizes the cumulative reward, maximizes the policy entropy, and minimizes the penalty for safety violations. The penalty for safety violations is obtained by multiplying the risk estimate output by the Cost Critic network with the dynamic multiplier output by the Lambda network, thus transforming the hard physical constraint into a differentiable soft optimization objective. A dual increasing loss function is constructed for the Lambda network, which adaptively adjusts the multiplier size according to the current risk level. Iterative Updates and Parameter Optimization: An adaptive moment estimation optimizer is used to update the parameters of each network using gradient descent; among them, the dual Critic value network and the Cost Critic security evaluation network minimize the temporal difference error, the Actor network maximizes the composite objective function, and the Lambda network maximizes the Lagrange dual function. More specifically, the construction process of a security reinforcement learning agent includes the following: S21, Actor Policy Network Structure: Input layer (3) -> Hidden layer 1 (128, ReLU) -> Hidden layer 2 (128, ReLU) -> Output layer (2 branches); Function: Outputs parameters of the action distribution; branch 1 outputs the mean. Branch 2 outputs the logarithm and standard deviation. ; Specifically, the input is the battery's state vector, with three dimensions including SOC, terminal voltage, and so on. The temperature is T; the hidden layer consists of two fully connected layers, each with 128 neurons, using ReLU activation to extract nonlinear features of the current state; the output layer has two branches, outputting action distribution parameters for random sampling. Branch 1 outputs the optimal action baseline in the current state, and branch 2 outputs the action exploration intensity, which is obtained by taking the exponent. ; Sampling strategy: To achieve gradient differentiability, a reparameterization technique is employed. in, The sampling noise follows a standard normal distribution, where It is the identity matrix. Activation functions ensure that actions are strictly limited to Within the scope; specifically, the reparameterization technique moves random sampling from inside the network to the outside, allowing gradients to propagate backward and avoiding gradient breakage caused by direct sampling. S22, Dual Critic Value Network Structure: Input layer (3+2) -> Hidden layer 1 (128, ReLU) -> Hidden layer 2 (128, ReLU) -> Output layer (1, Linear); Function: Evaluate state-action pairs Long-term cumulative rewards; Specifically, the input is a state-action pair. The system consists of a 3D state vector and a 2D action vector; the hidden layers are two fully connected layers, each with 128 neurons, using the ReLU activation function to extract joint state-action features; the output layer outputs a scalar. , representing the long-term cumulative reward of a state-action pair, that is, the value of the action in the current state; Dual network mechanism: introduction and Two networks, taking the following when calculating the target value. It effectively alleviates the value overestimation problem in DDPG-type algorithms, effectively suppresses overestimation, significantly improves the reliability of the strategy, and perfectly adapts to the high requirements of battery control for safety and stability. S23, Cost Critic Security Evaluation Network Structure: Similar to the Critic value network, but with independent parameters; Function: Explicitly evaluate the current state Take action Subsequently, the potential cumulative risk of future violations; its output The physical meaning of is a measure of "safe accessibility"; if The high reading indicates that no matter how much remedial measures are taken in the future, the battery is highly likely to overheat or overvoltage. Specifically, the structure is as follows: input layer (state + action, 3+2 dimensions) → 2 layers of 128-neuron ReLU hidden layers → 1-dimensional linear output layer; parameters are trained independently and do not share weights with the dual-critic value network to avoid coupling interference between safety assessment and reward assessment; S24, Lambda Network Structure: Input layer (3) -> Hidden layer (128, ReLU) -> Output layer (2, Softplus); Function: For each state Output a set of Lagrange multipliers ; Activation function: Use Softplus to ensure This network implements state relevance for penalty weights, meaning it outputs a large value in dangerous regions. Output small in safe area .

[0034] Specifically, the input is the battery state vector. The hidden layer consists of one layer with 128 neurons and the activation function is ReLU, used to extract the safety features of the state; the output layer outputs two Lagrange multipliers to realize the state correlation of the penalty weights; the core parameters that transform the safety-constrained optimization problem into an unconstrained optimization problem are used to make dynamic trade-offs between reward maximization and safety constraints; Specifically, embodiment S3 of this invention describes the training of a secure reinforcement learning agent based on a dual-timescale update mechanism, detailing the data-driven updating of the network parameters. The training process is based on experience replay, with a batch size of 128, and includes the following steps: S31, Collection and storage of empirical data In each At each step, the environment constructed by the security reinforcement learning agent based on the electrothermal coupling model built in step S1 is substituted into the calculation to obtain the tuple. Store the tuple in the buffer. ; In tuple expressions, for Real-time environmental values, To enhance the security of learning agents in Actions at any moment for The environmental value at the next time step. This indicates the end of this round of interaction; S32, Calculation of Bellman's Objective Value After sampling batch data from the buffer, the target value for each network is first calculated; S321, Action entropy regularization, calculates the entropy term for the next time step: ,in It is a temperature parameter; and These represent the state and action of the next moment, respectively. For Actor policy networks; S322, based on the traditional Bellman equation, introduces a dual-value network minimization and entropy regularization to construct an improved Bellman objective value equation: in, The target value for the Critic value network. As a reward value, Indicates the round termination mask. This is a discount factor used to measure the weight of the reward; This indicates that the dual-value network takes the minimum value. The output of the Critic value network represents the state. ,action The expected long-term cumulative reward for a security reinforcement learning agent; S323, Based on the improved Bellman objective value equation, the improved Bellman operator is used to calculate the Bellman objective value: in, The target value of the Critic value network is calculated based on the improved Bellman operator. For the value of the general agent, The output of the Critic value network computed based on the improved Bellman operator. These are the parameters of the cost network, and the output is in the state. ,action Below, the expected long-term cumulative cost of violations for security reinforcement learning agents; S33, Fast Timescale Updates: Critic Value Network and Cost Critic Security Evaluation Network The Critic value network updates its parameters most frequently, ensuring that every step is updated to guarantee low bias in value assessment. S331, the dual Critic value network is optimized by minimizing the mean squared error, and the corresponding loss function is as follows: In the above formula, where For batch size, For the goal value; and This is the output of the dual-critic value network. and These are the parameters of two independent Critic value networks, and the output state. ,action The expected cumulative reward; Gradient descent is performed based on the Adam optimizer, and the update formula is as follows: ;in, For the parameters of the Critic value network, ; The learning rate for the Critic value network. This represents the gradient of the loss function with respect to the Critic value network parameters. As the assignment sign, the flag parameter is updated along the gradient direction; S332, optimize the Cost Critic security evaluation network by minimizing the mean square error, and the corresponding loss function is as follows: In the above formula, This is expressed as the loss function of the Cost Critic security evaluation network. The predicted output of the CostCritic security evaluation network. The target value for the Cost Critic security evaluation network; The optimization operations for the Cost Critic security evaluation network are as follows. ,in The parameters for the Cost Critic security evaluation network, The learning rate of the Cost Critic security evaluation network. This represents the gradient of the loss function with respect to the parameters of the Cost Critic security evaluation network.

[0035] S34, Slow Timescale Updates: Actor Policy Networks and Lambda Networks To address the timescale mismatch between the slow thermal response and fast electrical response of the battery system, and to stabilize policy learning, this implementation adopts a delayed update strategy, setting the update frequency d=2, meaning that the Actor policy network is updated only once for every two updates to the Critic value network. S341, Construct the Lagrange function In the above formula, For Actor policy networks, For buffer Expectations Let be the logarithmic probability of the strategy. This indicates that the dual-value network takes the minimum value. The output of the Critic value network represents the state. ,action The expected long-term cumulative reward for a security reinforcement learning agent; S342, Update Actor Policy Network Parameters of the Actor policy network Find the gradient of the Lagrange function and perform gradient descent to update the formula: In the above formula, The learning rate of the Actor policy network. This is the gradient of the loss function with respect to the Actor policy network parameters; this step increases the probability density where the reward is high and decreases the probability density where the weighted risk is high. S343, Update Lambda Network Based on the Lambda network, the tightest constraint bound is obtained, and the loss function is defined as follows: in This is the preset safety tolerance threshold; it is particularly important to note that the optimizer defaults to minimization, so minimizing a negative value is equivalent to maximizing the item within the parentheses; when When (risk exceeds the limit), the gradient direction makes Increase the penalty, thus imposing a larger penalty in the next Actor policy network update.

[0036] when (When in the safe zone) the gradient direction makes Reduce to 0, avoiding excessive conservatism; S35, for all target networks that do not participate in gradient backpropagation, updates are only performed through moving average: The target network includes Target Actor, Target Critic, and Target Cost Critic. For the target network parameters, For local network parameters, For smoothing coefficients; specifically, smoothing coefficients The value is extremely small to ensure that the target value changes slowly.

[0037] Specifically, the training of the security reinforcement learning agent based on the dual-timescale update mechanism described in step S4 also includes the following steps: S4. After completing the training of the safety reinforcement learning agent, the multi-dimensional vector describing the battery's operating status is input into the Actor policy network, which outputs the optimal charging current and fan power command. The steps for outputting the optimal battery charging trajectory based on temperature and voltage, according to the battery's multi-dimensional vector state and the charging policy generated by the safety reinforcement learning agent, include the following: The trained Actor policy network is the final controller. S41, State Observation and Estimation The BMS battery system samples at a frequency of 10Hz or higher: Direct measurement: Battery terminal voltage Load current Battery surface temperature ; Indirect estimation: Utilizing a pre-integrated Extended Kalman Filter (EKF) observer, based on and Real-time estimation ;based on Real-time estimation of core temperature using thermal models ; S42, Forward Reasoning and Action Execution Preprocessing: Substitute the observed values ​​into step S21 for normalization; Inference: Input Actor policy network; for online applications, directly take the distribution mean. As output, no random noise is added, ensuring the determinism and stability of the control: ; Anti-normalization: Convert to actual current command and fan power commands ; S43, Safety Redundancy Mechanism In the implementation of the project, physical hard constraints independent of AI are set. and These are the device's own voltage and temperature hard constraint parameters, specifically including the following: If detected The hardware overvoltage protection is immediately triggered, cutting off the main circuit; If detected The fan is immediately forced to operate at maximum power and the current is reduced to 0.

[0038] More specifically, through the above-described implementation methods, the embodiments of the present invention achieve the following control effects: Low-temperature acceleration: During the initial charging phase (low SOC, low temperature), the Cost Critic safety assessment network predicts extremely low risk. The Actor strategy network outputs the maximum allowable current, enabling extremely rapid power replenishment; Pre-cooling strategy: When the temperature is close to the threshold but has not exceeded the limit, the Cost Critic safety evaluation network predicts the future temperature rise trend, and the Actor strategy network turns on the fan in advance to use thermal inertia to suppress the temperature rise rate, thereby extending the duration of high current. Flexible boundary control: When the state approaches 42°C or 3.6V, As the current increases exponentially, the Actor strategy network automatically calculates the current value that is maintained just below the boundary, achieving oscillation-free boundary following control.

[0039] In summary, this invention takes a typical lithium-ion power battery as the research object and simulates real charging conditions in an electro-thermal coupling simulation environment, executing the entire process of the variable cooling coupling battery fast charging control method based on safety reinforcement learning described in this invention. From constructing a high-fidelity electro-thermal coupling model integrating nonlinear convection heat dissipation characteristics and defining a joint action space including charging rate and fan power, to constructing a safety reinforcement learning agent including a Cost Critic safety evaluation network and a Lambda network, and then using a dual time scale update mechanism to achieve asynchronous iterative optimization of the policy network and the value network, this invention not only verifies the superiority of the method in multi-objective collaborative optimization, but also confirms the effectiveness of its safety constraint processing mechanism based on reachability analysis. Specifically, by dynamically adjusting the Lagrange multipliers, it ensures that the battery core temperature and terminal voltage strictly converge within the preset safety boundary under different initial states. At the same time, the scheduling capability of active thermal management also demonstrates the excellent robustness and adaptability of this invention in balancing rapid energy replenishment and system energy efficiency, effectively solving the technical pain points of existing diagnostic methods such as insufficient model accuracy and insensitivity to early small faults.

[0040] This invention also discloses a variable cooling coupling battery fast charging control system based on security reinforcement learning, comprising: An environment generation module is used to establish an electrothermal coupling model of a lithium-ion battery as an environment for safety reinforcement learning, and to construct a state space and an action space; wherein, the state space is a multi-dimensional vector of the battery's operating status; and the action space is a joint control quantity that relates the battery's charging efficiency and thermal safety calculations. An agent construction module is used to construct a safety reinforcement learning agent, which includes an Actor policy network, a dual-Critic value network, a Cost Critic safety evaluation network, and a Lambda network; wherein the dual-Critic value network is used to evaluate the cumulative reward of charging efficiency, the Cost Critic safety evaluation network is used to evaluate the cumulative reward of thermal safety, and the Lambda network is used to balance the weights between the dual-Critic value network and the Cost Critic safety evaluation network. The agent training module is used to train the security reinforcement learning agent using a dual time-scale update mechanism: at each simulation time step, gradient descent updates are performed on the dual Critic value network and the Cost Critic security evaluation network; every d simulation time steps, the parameters of the Actor policy network and the Lambda network are updated once, where d is a preset delay update interval. The instruction output module is used to input the multi-dimensional vector of the current battery operating state into the trained Actor policy network and output the optimal charging current and heat dissipation instructions.

[0041] It should be noted that the above-described embodiment of the variable cooling coupled battery fast charging control system based on safety reinforcement learning is implemented using the same technical means as the variable cooling coupled battery fast charging control method based on safety reinforcement learning, and therefore will not be described in detail here.

[0042] Furthermore, the method according to this application can also be implemented as a computer program or computer program product, which includes computer program code instructions for performing some or all of the steps in the method described above.

[0043] Alternatively, this application may be implemented as a computer-readable storage medium (or a non-transitory machine-readable storage medium or a machine-readable storage medium) storing executable code (or computer program or computer instruction code) thereon, which, when executed by a processor of an electronic device (or a cloud server, etc.), causes the processor to perform part or all of the steps of the above-described method according to this application.

[0044] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element. Furthermore, it should be noted that the scope of the methods and apparatuses in the embodiments of this application is not limited to performing functions in the order shown or discussed, but may also include performing functions substantially simultaneously or in the reverse order, depending on the functions involved. For example, the described methods may be performed in a different order than described, and various steps may be added, omitted, or combined. Additionally, features described with reference to certain examples may be combined in other examples.

[0045] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a computer software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods of the various embodiments of this application.

[0046] The embodiments of this application have been described above with reference to the accompanying drawings. However, this application is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of this application without departing from the spirit and scope of the claims, and all of these forms are within the protection scope of this application.

[0047] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "illustrative embodiment," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of this application. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0048] Although embodiments of this application have been shown and described, those skilled in the art will understand that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of this application, the scope of which is defined by the claims and their equivalents.

Claims

1. A variable cooling coupled battery fast charging control method based on security reinforcement learning, characterized in that, Includes the following steps: S1. Establish an electrothermal coupling model of a lithium-ion battery as the environment for safety reinforcement learning, and construct a state space and an action space; wherein, the state space is a multi-dimensional vector of the battery's operating status; and the action space is a joint control quantity that relates the battery charging efficiency and thermal safety calculations. S2. Construct a safety reinforcement learning agent, which includes an Actor policy network, a dual-Critic value network, a Cost Critic safety evaluation network, and a Lambda network; wherein the dual-Critic value network is used to evaluate the cumulative reward of the charging efficiency, the Cost Critic safety evaluation network is used to evaluate the cumulative reward of thermal safety, and the Lambda network is used to balance the weights between the dual-Critic value network and the Cost Critic safety evaluation network. S3. The security reinforcement learning agent is trained using a dual time-scale update mechanism: at each simulation time step, gradient descent updates are performed on the dual Critic value network and the Cost Critic security evaluation network; after every d simulation time steps, the parameters of the Actor policy network and the Lambda network are updated once, where d is a preset delay update interval. S4. Input the multi-dimensional vector of the current battery operating state into the trained Actor policy network, and output the optimal charging current and heat dissipation command.

2. The variable-cooling-coupling battery fast charging control method based on security-reinforced learning according to claim 1, characterized in that, The electrothermal coupling model in step S1 includes a second-order RC equivalent circuit model for the electrochemical part and a core-surface dual-state thermal model for the thermodynamic part. The core-surface dual-state thermal model establishes the dynamic relationship between the core temperature and the surface temperature of the battery, and the equation is as follows: ； ； in, For the core temperature, For surface temperature, For ambient temperature, For current, and These are the core heat capacity and surface heat capacity of the battery, respectively. The internal thermal resistance between the core and the surface; This refers to the battery's internal resistance. The entropy change coefficient, It is Faraday's constant. It represents the variable convective thermal resistance from the surface to the environment.

3. The variable-cooling-coupling battery fast charging control method based on security-reinforced learning according to claim 1, characterized in that, The Cost Critic safety evaluation network in step S2 is used to evaluate the expected cumulative cost of violating safety constraints, including the maximum allowed voltage and the maximum allowed temperature for the current state-action pair.

4. The variable-cooling-coupling battery fast charging control method based on security-reinforced learning according to claim 1, characterized in that, After constructing the action space as a joint control variable relating battery charging efficiency and thermal safety calculations, a composite reward function is constructed, which includes charging speed reward, energy consumption penalty, and safety constraint penalty based on reachability analysis. The specific construction includes the following: Constructing arbitrary The expression containing the charging speed reward and the energy consumption penalty at the moment: ； in, for Reward value at any moment The preset charging current value, For preset charging power value, For indicator functions, In a charged state, For the preset state of charge, where , and These are respectively represented as the charging speed weighting coefficient, the energy consumption penalty weighting coefficient, and the energy target reward weighting coefficient; Constructing the cost function Performing Cost Critic safety evaluation network training: ； ； ； In the above formula, and These are the voltage-based and temperature-based values, respectively. for Value of the general agent at any time , , , and This is represented as the current voltage value, the upper voltage limit, the current temperature value, the upper temperature limit, and the lower temperature limit; This is a truncation function.

5. The fast charging control method for a variable cooling coupled battery based on safety reinforcement learning according to claim 1, characterized in that, Step S3, which describes training the security reinforcement learning agent based on a dual-timescale update mechanism, specifically includes the following steps: S31, Collection and storage of empirical data, including the following steps: At each moment, the safety reinforcement learning agent is substituted into the electric-thermal coupling model constructed based on step S1 for calculation to obtain a tuple The tuple is stored in a buffer ; In tuple expressions, for Real-time environmental values, To enhance the security of learning agents in Momentary actions for The environmental value at the next moment. This indicates the end of this round of interaction; S32, Calculate the Bellman objective value, the steps include: S321, Action entropy regularization, calculates the entropy term for the next time step: ,in It is a temperature parameter; and These represent the state and action of the next moment, respectively. For Actor policy networks; S322, based on the traditional Bellman equation, introduces a dual-value network minimization and entropy regularization to construct an improved Bellman objective value equation: ； in, The target value for the Critic value network. As a reward value, Indicates the round termination mask. This is a discount factor used to measure the weight of the reward; This indicates that the dual-value network takes the minimum value. The output of the Critic value network represents the state. ,action The expected long-term cumulative reward for a security reinforcement learning agent; S323, Based on the improved Bellman objective value equation, the improved Bellman operator is used to calculate the Bellman objective value: ； in, The target value of the Critic value network is calculated based on the improved Bellman operator. For the value of the general agent, The output of the Critic value network computed based on the improved Bellman operator. These are the parameters of the cost network, and the output is in the state. ,action The expected long-term cumulative cost of violations for security reinforcement learning agents.

6. The variable-cooling-coupling battery fast charging control method based on security-reinforced learning according to claim 5, wherein Step S3 further includes the following steps: S33, Fast Timescale Update: Critic Value Network and Cost Critic Security Evaluation Network, steps include: S331, the dual Critic value network is optimized by minimizing the mean squared error, and the corresponding loss function is as follows: ； In the above formula, where For batch size, For the goal value; and This is the output of the dual-critic value network. and These are the parameters of two independent Critic value networks, and the output state. ,action The expected cumulative reward; Gradient descent is performed based on the Adam optimizer, and the update formula is: ;in, For the parameters of the Critic value network, ; The learning rate for the Critic value network. This represents the gradient of the loss function with respect to the Critic value network parameters. As the assignment sign, the flag parameter is updated along the gradient direction; S332, optimize the Cost Critic security evaluation network by minimizing the mean square error, and the corresponding loss function is as follows: ； In the above formula, This is expressed as the loss function of the Cost Critic security evaluation network. The predicted output of the Cost Critic security evaluation network. The target value for the Cost Critic security evaluation network; The optimization operations for the Cost Critic security evaluation network are as follows. ,in The parameters for the CostCritic security evaluation network. The learning rate of the Cost Critic security evaluation network. This represents the gradient of the loss function with respect to the parameters of the Cost Critic security evaluation network. S34, Slow Time-Scale Update: Actor Policy Network and Lambda Network, steps include: S341, Construct the Lagrange function ； In the above formula, For Actor policy networks, For buffer Expectations Let be the logarithmic probability of the strategy. This indicates that the dual-value network takes the minimum value. The output of the Critic value network represents the state. ,action The expected long-term cumulative reward for a security reinforcement learning agent; S342, Update Actor Policy Network parameters of the actor policy network the gradient of the Lagrangian function is computed and gradient descent is performed with the update formula: ； In the above formula, is the learning rate of the Actor policy network, is the gradient of the loss function with respect to the Actor policy network parameters; S343, Update Lambda Network Based on the Lambda network, the tightest constraint bound is obtained, and the loss function is defined as follows: ； wherein is a pre-set safety tolerance threshold; S35, for all target networks that do not participate in gradient backpropagation, updates are only performed through moving average: ； The target network includes Target Actor, Target Critic, and Target Cost Critic. For the target network parameters, For local network parameters, This is the smoothing coefficient.

7. A fast-charging control method for a variable cooling coupled battery based on safety reinforcement learning according to claim 2, characterized in that, The second-order RC equivalent circuit model includes a voltage source to simulate the battery open-circuit voltage, at least one series ohmic internal resistance and a parallel polarized RC network to capture the terminal voltage response characteristics of the battery under different states of charge and dynamic currents. The core-surface dual-state thermal model is used to calculate the heat generated inside the battery due to Joule heating and electrochemical reactions, as well as to calculate and describe the nonlinear physical process of heat conduction from the battery core to the surface and then dissipation from the surface to the environment through convection.

8. A variable cooling coupled battery fast charging control system based on security reinforcement learning, characterized in that, The system includes: An environment generation module is used to establish an electrothermal coupling model of a lithium-ion battery as an environment for safety reinforcement learning, and to construct a state space and an action space; wherein, the state space is a multi-dimensional vector of the battery's operating status; and the action space is a joint control quantity that relates the battery's charging efficiency and thermal safety calculations. An agent construction module is used to construct a safety reinforcement learning agent, which includes an Actor policy network, a dual-Critic value network, a Cost Critic safety evaluation network, and a Lambda network; wherein the dual-Critic value network is used to evaluate the cumulative reward of charging efficiency, the Cost Critic safety evaluation network is used to evaluate the cumulative reward of thermal safety, and the Lambda network is used to balance the weights between the dual-Critic value network and the Cost Critic safety evaluation network. The agent training module is used to train the security reinforcement learning agent using a dual time-scale update mechanism: at each simulation time step, gradient descent updates are performed on the dual Critic value network and the Cost Critic security evaluation network; every d simulation time steps, the parameters of the Actor policy network and the Lambda network are updated once, where d is a preset delay update interval. The instruction output module is used to input the multi-dimensional vector of the current battery operating state into the trained Actor policy network and output the optimal charging current and heat dissipation instructions.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer program instructions that, when executed by a processor, implement the steps of the method as described in any one of claims 1-7.

10. A computer program product, characterised in that, When the instructions in the computer program product are executed by the processor of the electronic device, the electronic device causes the electronic device to perform the steps of the method as described in any one of claims 1-7.

Images