Vanadium flow battery charge-discharge control method based on physical reinforcement learning

Abstract

The application discloses a multi-stack vanadium flow battery charging and discharging control method based on physical reinforcement learning, relates to the field of energy storage system optimization control, and aims to solve the problem of multi-dimensional operation imbalance of vanadium flow batteries. The method comprises the following steps: collecting the terminal voltage, current, state of charge, electrolyte flow and environmental temperature of each stack, and constructing a state vector based on the collected data. An initial action is generated based on the state vector by using a double deep Q network, and the initial action is constrained and corrected by a physical action space projection operator. A multi-objective reward mechanism is set to drive policy optimization. A regularization term of a physical dynamic equation is integrated into the loss function of the double deep Q network, the policy parameters are optimized synchronously through a soft update mechanism of an online network and a target network, and optimized action parameters are obtained. According to the optimized action parameters, the circulating pump speed is adjusted by a PI controller to match the electrolyte flow and the stack current, and the state of charge and the polarization voltage state of each stack are updated.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of energy storage system optimization control, and particularly relates to a multi-stack vanadium flow battery charging and discharging control method based on physical reinforcement learning. BACKGROUND

[0002] As a key technology in the field of large-scale energy storage, vanadium flow battery is widely used in renewable energy grid connection and power grid peak regulation due to its independent adjustable power and capacity, long cycle life and environmental friendliness.

[0003] However, the energy storage system composed of multiple stacks often faces the problem of uneven operation. At the electrochemical level, the difference in state of charge of each stack leads to a decrease in effective capacity, and at the fluid dynamics level, uneven distribution of electrolyte exacerbates the difference in internal resistance, reducing efficiency and causing safety hazards. Existing control methods often use equivalent circuit models for predictive control, but are affected by model errors and are difficult to deal with nonlinear problems. Data-driven reinforcement learning easily violates safety boundaries, and expert rule strategies lack dynamic adaptability. SUMMARY

[0004] The present application aims to provide a multi-stack vanadium flow battery charging and discharging control method based on physical reinforcement learning, which aims to solve the problem of multi-dimensional uneven operation of vanadium flow battery and the limitations of existing control methods.

[0005] To achieve the above-mentioned purpose, the present application adopts the following technical solutions:

[0006] In a first aspect, the present application provides a multi-stack vanadium flow battery charging and discharging control method based on physical reinforcement learning, comprising: S1: collecting the terminal voltage, current, state of charge, electrolyte flow rate and environmental temperature of each stack, and constructing a state vector representing the real-time operating state of the multi-stack system based on the collected data. Wherein, the stack refers to a battery pack formed by a plurality of vanadium flow battery monomers in series. S2: generating an initial action using a double deep Q network and according to the state vector, and constraining and correcting the initial action through a physical action space projection operator. S3: setting a multi-objective reward mechanism to drive strategy optimization to achieve minimum loss rate and balance of state of charge among stacks. S4: integrating a regularization term of a physical dynamic equation into the loss function of the double deep Q network, and synchronously optimizing the strategy parameters through the soft update mechanism of the online network and the target network to obtain the optimized action parameters. Wherein, the physical dynamic equation refers to a mathematical equation describing the electrochemical reaction law of the stack. S5: adjusting the circulating pump speed through a PI controller to match the electrolyte flow rate and the stack current according to the optimized action parameters, and updating the state of charge and polarization voltage of each stack.

[0007] The state vector includes the state of charge, terminal voltage, current, electrolyte flow rate, ambient temperature and equivalent circuit model parameters of each stack. Among them, the equivalent circuit model parameters refer to the resistance parameters and capacitance parameters in the equivalent circuit model of the stack based on the electrochemical mechanism.

[0008] In step S2, the initial action includes a current distribution coefficient and a flow adjustment coefficient. The current distribution coefficient refers to the proportion of the current allocated to each stack in the total current of the system, and the sum of the current distribution coefficients of all stacks is 1. The flow adjustment coefficient is a correction coefficient used to adjust the electrolyte flow rate of each stack. The value of the flow adjustment coefficient is related to the state of charge and current of the stack. In step S2, the physical action space projection operator includes a current distribution ratio projection operator and a flow coefficient projection operator. The current distribution ratio projection operator is used to ensure that the sum of the current distribution coefficients is 1, and the flow coefficient projection operator is used to constrain the flow adjustment coefficient according to the electrode reaction law of the stack. Among them, the constraint correction of the current distribution ratio projection operator is:

[0009] .

[0010] The constraint correction of the flow coefficient projection operator is:

[0011] .

[0012] wherein tanh is a nonlinear activation function, represents the current distribution ratio of the i-th stack, and satisfies , , represents the flow adjustment coefficient of the i-th stack, and satisfies , is the safety flow threshold of the electrolyte, ; , respectively represent the state of charge, terminal voltage, current, electrolyte flow rate and ambient temperature of the i-th stack.

[0013] In step S3, a weighted reward function is constructed.

[0014] wherein .

[0015] .

[0016] .

[0017] , , is a weight coefficient for balancing the contributions of each index, is the total loss of the system, This represents the standard deviation of the state of charge of each fuel cell stack. Physical constraint compliance penalties are constructed using electrodynamic constraint equations; Identify model parameters using historical data; This represents the system's maximum power loss. This represents the average state of charge of each fuel cell stack. These are the electrodynamic constraint equations; This is the loss coefficient.

[0018] The electrodynamic constraint equations include:

[0019] .

[0020] .

[0021] .

[0022] .

[0023] in, For the first The charging and discharging current of each fuel cell stack For the first The terminal voltage of each fuel cell stack For the first Electrolyte flow rate of each fuel cell stack For the first The state of charge of each fuel cell stack For the first The loss of each fuel cell stack; I total P represents the total current required by the system. demand is the power requirement; c is the electrolyte concentration.

[0024] In step S4, the regularization term of the physical dynamic equation is constructed based on the power conservation equation and the state-of-charge consistency criterion, and is used to embed electrochemical laws into the policy update process of reinforcement learning. The online network is used to update parameters in real time, and the target network is used to periodically synchronize the parameters of the online network. Stable policy updates are achieved through the collaborative optimization of the two.

[0025] In step S5, the PI controller controls the electrolyte flow rate by adjusting the speed of the variable frequency pump, so that the actual flow rate tracks the set flow rate, wherein the set flow rate is determined based on the stack current and the flow rate adjustment coefficient.

[0026] In the process of optimizing strategy parameters in step S4, dynamic simulation verification of the corrected action is also performed based on the equivalent circuit model. If the simulation results violate the power balance or state of charge consistency constraints, the action parameters are further adjusted.

[0027] The state of charge of each stack is updated based on the ampere-hour integration method, and the change in the state of charge is calculated according to the charging and discharging current of the stack and the rated capacity.

[0028] Compared with the prior art, the application has the following beneficial effects:

[0029] 1. The multi-stack vanadium flow battery charging and discharging control method based on physical reinforcement learning provided by the application constrains and corrects the current distribution coefficient and the flow adjustment coefficient through a physical action space projection operator, dynamically optimizes the current and flow distribution of each stack by means of the constraint of the state of charge standard deviation in the reward function and the state of charge consistency criterion in the physical regularization term, effectively reduces the difference in the state of charge between the stacks, avoids local overcharging and overdischarging, and prolongs the overall service life of the system. While ensuring that the current and flow match the physical limits of the stack, the multi-objective reward function is combined to minimize the system loss, significantly reducing energy waste and thus improving the charging and discharging efficiency.

[0030] 2. The power conservation equation and other physical dynamic equations are embedded in reinforcement learning as regularization terms to verify and correct the action parameters through dynamic simulation, strictly constrain the current, flow and power balance, prevent safety hazards caused by parameter overlimiting, and improve the stability of the system. The online-target network soft update mechanism of the double deep Q network is combined with the parameter identification of the equivalent circuit model to realize the bidirectional coupling of electrochemical rules and reinforcement learning, accelerate the policy convergence speed, improve the control precision and adaptability, and cope with the nonlinear characteristics under complex working conditions. BRIEF DESCRIPTION OF DRAWINGS

[0031] In order to more clearly illustrate the technical solutions of the embodiments of the application, the following will briefly introduce the drawings needed to be used in the embodiment description. Obviously, the drawings in the following description are only some embodiments of the application, and other drawings can also be obtained by those skilled in the art without creating any creative labor.

[0032] Figure 1 A flow chart of a multi-stack vanadium flow battery charging and discharging control method based on physical reinforcement learning provided by an embodiment of the application is provided.

[0033] Figure 2 A vanadium flow battery single stack equivalent circuit model provided by an embodiment of the application is provided.

[0034] Figure 3 A non-equalization control comparison schematic diagram provided by an embodiment of the application is provided.

[0035] Figure 4 A schematic diagram of a multi-stack vanadium flow battery energy storage system structure provided by an embodiment of the application is provided. DETAILED DESCRIPTION

[0036] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present invention.

[0037] The terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of this invention, unless otherwise stated, "a plurality of" means two or more.

[0038] In embodiments of the invention, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, article, or apparatus that includes that element.

[0039] In embodiments of the present invention, the terms "exemplary" or "for example" are used to indicate that something is an example, illustration, or description. Any embodiment or design described as "exemplary" or "for example" in embodiments of the present invention should not be construed as being more preferred or advantageous than other embodiments or designs. Rather, the use of the terms "exemplary" or "for example" is intended to present the relevant concepts in a specific manner.

[0040] This application provides a charging and discharging control method for a multi-staple vanadium redox flow battery based on physical reinforcement learning. For example,... Figure 1 As shown. The method includes:

[0041] S1: Collect the terminal voltage of each fuel cell stack. Current State of charge Electrolyte flow rate and ambient temperature ,in Representing the Each fuel cell stack was analyzed. A state vector characterizing the real-time operating state of the multi-stack system was constructed based on the collected data. .

[0042] In this context, a battery stack refers to a battery pack formed by connecting multiple vanadium redox flow battery cells in series. The state vector includes the state of charge, terminal voltage, current, electrolyte flow rate, ambient temperature, and equivalent circuit model parameters for each stack. The equivalent circuit model parameters refer to the resistance and capacitance parameters in the equivalent circuit model of the battery stack, constructed based on electrochemical mechanisms. The state vector characterizes the real-time operating state of the multi-stack system.

[0043] As one possible implementation, high-precision voltage sensors and coulomb counters are used to monitor the terminal voltage of each fuel cell stack in real time. and The value is obtained simultaneously through an electromagnetic flowmeter, which acquires the electrolyte flow rate of each fuel cell stack. It is used in conjunction with a PT100 temperature sensor to measure the temperature of the fuel cell stack. This provides real-time parameters for the physical model. In addition, a Hall effect current sensor combined with a differential amplifier circuit is used to detect the charging and discharging current. The state of charge (SPC) update for each fuel cell stack is based on the ampere-hour integral method, which calculates the change in SPC based on the charging and discharging current and rated capacity of the fuel cell stack.

[0044] Reference Figure 2 In some embodiments, for the equivalent circuit model of the fuel cell stack constructed based on the electrochemical mechanism, the parameters of the equivalent model of the fuel cell stack are identified by historical data. and The equivalent circuit model of the fuel cell stack includes state equations and output equations, used to characterize the stack polarization voltage. With terminal voltage The dynamic relationship.

[0045] Specifically, in practical applications, battery resistance is relatively low. With mass production of batteries, coupled with improvements in intelligent manufacturing and production processes, the parameters of individual batteries are not significantly different. Furthermore, each vanadium redox flow battery undergoes screening before being connected in series. A vanadium redox flow battery is connected in series to form a stack, and the parameters of each vanadium redox flow battery can be considered to be basically the same. Therefore, when establishing the equivalent circuit model of the stack, the parameter differences of individual vanadium redox flow batteries can be ignored, that is, a single stack can be represented as:

[0046]

[0047] The parameters in the equivalent circuit model of the fuel cell stack are based on the parameters of a single vanadium redox flow cell. and As explained above, the structure of the equivalent circuit model of the fuel cell stack is the same as that of a single vanadium redox flow battery, where historical data is used to identify model parameters. and The relationship is as follows:

[0048]

[0049] The following is the equivalent loss model of the battery:

[0050]

[0051]

[0052]

[0053]

[0054]

[0055]

[0056] Core voltage Using the Nernst equation, it can be expressed as:

[0057] in, This is the standard electrode potential of a vanadium redox flow cell, according to the Nernst equation. The ideal gas constant is 8.314 J / (K mol); For temperature, take 298K at room temperature (25°C). The Faraday constant is set to 96500 C / mol.

[0058] The continuous state equation in the equivalent circuit model of the fuel cell stack is:

[0059]

[0060]

[0061] The output equation in the equivalent circuit model of the fuel cell stack is:

[0062] For example, suppose To ensure the time series data is suitable for constant values, the continuous state equation is discretized using Euler methods with a time step of [value missing]. The specific formula is as follows:

[0063] in, , , .

[0064] As one possible approach, the process of identifying the equivalent model parameters of the fuel cell stack using historical data includes the following steps: First, setting initial guess values. Calculate using initial parameters and discrete state equations The output equation yields Next, the error is calculated, which is the predicted value of the output equation. Compared with measured values Mean square error Finally, the Levenberg-Marquardt algorithm is used to update the parameters until the objective function is achieved. Convergence is achieved, and the final parameters of each fuel cell stack are obtained. .

[0065] S2: Utilize a dual-depth Q-network and based on the state vector Generate initial actions The initial action includes the current distribution coefficient. and flow regulation coefficient And through the physical action space projection operator The initial action is constrained and corrected to ensure that the action meets the physical limits of the stack current and flow rate.

[0066] Specifically, a state vector is defined based on the current physical quantity state data. Furthermore, based on the electrochemical kinetic equations, a real-time physical model for each fuel cell stack was established, and the following constraints were defined:

[0067]

[0068]

[0069]

[0070] in, Representing the The charging and discharging current of each fuel cell stack The rated capacity of the fuel cell stack, Electrolyte concentration, This represents the electrolyte flow rate.

[0071] In step S2, the current distribution coefficient This refers to the proportion of the current allocated to each fuel cell stack to the total system current, and the sum of the current allocation coefficients of all fuel cell stacks is 1, which satisfies... The flow rate adjustment coefficient refers to the correction factor used to adjust the electrolyte flow rate of each fuel cell stack. The value of is related to the charge state and current of the fuel cell, that is, it satisfies . In step S2, the physical action space projection operator includes a current distribution ratio projection operator and a flow coefficient projection operator. The current distribution ratio projection operator is used to ensure that the sum of the current distribution coefficients is 1, and the flow coefficient projection operator is used to constrain the flow regulation coefficient according to the electrode reaction law of the fuel cell stack. The constraint correction of the current distribution ratio projection operator is as follows:

[0072] This projection ensures The total current distribution is conserved, and numerical robustness is achieved through scaling factors.

[0073] The constraint correction for the flow coefficient projection operator is as follows:

[0074] .

[0075] in, Representing the The current distribution ratio of each fuel cell stack satisfies , Representing the The flow regulation coefficient of each fuel cell stack satisfies This represents the safe flow rate threshold for the electrolyte. To prevent division by zero anomalies; tanh is a non-linear activation function.

[0076] For example, the complete projection operation expression is: .

[0077] S3: Set up a multi-objective reward mechanism to drive strategy optimization in order to minimize the loss rate and balance the state of charge among the stacks.

[0078] In step S3, a weighted reward function is constructed. .

[0079] in, .

[0080] .

[0081] .

[0082] , , The weighting coefficients used to balance the contributions of various indicators. For the total system loss, This represents the standard deviation of the state of charge of each fuel cell stack. Physical constraint compliance penalties are constructed using electrodynamic constraint equations; Identify model parameters using historical data; This represents the system's maximum power loss. This represents the average state of charge of each fuel cell stack. These are the electrodynamic constraint equations; This is the loss coefficient.

[0083] The electrodynamic constraint equations include:

[0084] .

[0085] .

[0086] .

[0087] .

[0088] in, For the first The charging and discharging current of each fuel cell stack For the first The terminal voltage of each fuel cell stack For the first Electrolyte flow rate of each fuel cell stack For the first The state of charge of each fuel cell stack For the first The loss of each fuel cell stack, I total P represents the total current required by the system. demand This represents the power requirement. The aim is to avoid overcharging or over-discharging due to uneven current distribution by ensuring that the sum of the current distribution of all fuel cells matches the total system demand. This is used to ensure that the total output power of the fuel cell stack can still meet the system requirements after taking into account internal losses, thus ensuring a balance between power supply and demand. By adjusting the synchronization between flow rate and current, energy loss caused by excessive or insufficient flow rate can be prevented. By controlling the consistency between the rate of change of state of charge and the actual charging and discharging current, errors caused by deviations in the calculation of state of charge are eliminated.

[0089] For example, such as Figure 3 As shown, Figure 3 The shaded area represents the State of Charge (SOC) of the battery stack during operation. Because batteries have slight differences in SOC at the factory, this difference in SOC between different stacks widens over time during operation. Inconsistent SOC leads to a gradual increase in performance differences between stacks. Some stacks may remain at a high SOC for extended periods, while others remain at a low SOC. This imbalance reduces the overall efficiency of the battery pack, affecting battery output power and range. Figure 3 It can be seen that without considering the balance constraint, the SOC of different fuel cell stacks varies greatly during operation, while with the balance constraint considered, the SOC of different fuel cell stacks tends to be consistent during operation. The method provided in this application embodiment ensures the consistency of SOC of different fuel cell stacks by applying multiple constraints.

[0090] S4: Integrating the regularization term of the physical dynamics equation into the loss function of a dual-depth Q-network Through the soft update mechanism of the online network and the target network The strategy parameters are simultaneously optimized to obtain the optimized action parameters. The physical dynamic equations refer to the mathematical equations describing the electrochemical reaction laws of the fuel cell stack.

[0091] In step S4, the regularization term of the physical dynamic equation is constructed based on the power conservation equation and the state-of-charge consistency criterion, and is used to embed electrochemical laws into the policy update process of reinforcement learning. The online network is used to update parameters in real time, and the target network is used to periodically synchronize the parameters of the online network. Stable policy updates are achieved through the collaborative optimization of the two.

[0092] In the process of optimizing strategy parameters in step S4, dynamic simulation verification of the corrected action is also performed based on the equivalent circuit model. If the simulation results violate the power balance or state of charge consistency constraints, the action parameters are further adjusted.

[0093] For example, the Bellman equation under physical constraints is defined as follows:

[0094] in, The state transition probabilities are represented by a physical model, and a dual-deep Q-network structure is used to enhance learning stability, including an online network. Real-time parameter updates Target network Periodic synchronization ; To reinforce the discount factor in learning, which is used to weigh the importance of current rewards and future rewards, the value is usually in the range of 0-1. The closer the value is to 1, the more important future rewards are.

[0095] The updated target value is calculated according to the following formula:

[0096]

[0097] in, Indicates the next state The optimal action is selected. The loss function is:

[0098] in, The physical regularization coefficients are used to update the parameters using gradient descent.

[0099]

[0100] in, For learning rate, The gradient of the loss function L(θ) with respect to the online network parameters θ is used for parameter updates in gradient descent, indicating the direction and magnitude of parameter adjustments; the target network parameters are... Regularly update online network parameters using a soft update method. copy: ; It is the soft update coefficient (usually taken as...). Finally, the optimal action for the current state is output, which is the optimized action parameters. .

[0101] S5: Based on the optimized action parameters, the PI controller adjusts the circulation pump speed to match the electrolyte flow rate. With the current of the fuel cell stack And update the state of charge of each stack. With polarization voltage state.

[0102] In step S5, the PI controller controls the electrolyte flow rate by adjusting the speed of the variable frequency pump, so that the actual flow rate tracks the set flow rate, where the set flow rate is based on the stack current and the flow rate adjustment coefficient. Sure.

[0103] Specifically, based on the current-flow matching relationship of the fuel cell stack branches, the flow setpoint for each branch is calculated:

[0104]

[0105]

[0106] in, The polarization voltage decay coefficient of the i-th stack determines the rate of natural decay of the polarization voltage. This represents the total current demand of the system. The flow rate regulation coefficient is generated by the PIRL algorithm and adjusted for constraints. Speed ​​adjustment is achieved by using a variable frequency drive (VFD) system to regulate the speed of the circulating pump. to make the actual flow Tracking settings The characteristic equations for pump flow rate and speed are shown below:

[0107]

[0108] in, Pump speed, Pump flow coefficient, Pipeline resistance coefficient Represents pipeline pressure drop. Electrolyte density, control signal Due to flow deviation Decide:

[0109]

[0110] in, and The parameters for the flow loop PI controller are then set. Finally, based on the adjusted flow... and current Update each fuel cell stack Value, stack temperature Wait, and complete the closed-loop control.

[0111] In step S1, core operating parameters of the fuel cell stack (terminal voltage, current, state of charge, etc.) are collected to construct a vector that comprehensively reflects the real-time state of the multi-stack system, providing basic input for subsequent control decisions. For step S2, a dual-deep Q-network is used to generate initial control actions (current distribution, flow regulation), achieving the initial output of intelligent decision-making. Then, a physical constraint projection operator is used to correct the actions, ensuring they conform to the physical limits of stack current and flow (e.g., conservation of total current distribution and flow not exceeding safety thresholds). This step addresses the potential violation of physical laws by purely data-driven methods, ensuring decision-making flexibility while maintaining the boundaries of safe system operation. Step S3 designs a reward function that integrates loss rate and state of charge balance, minimizing energy loss and balancing the differences in state of charge (SOC) among the stacks, thus extending the overall system lifetime. Step S4 embeds the physical dynamic equations describing electrochemical reaction laws as regularization terms into the loss function, forcing reinforcement learning policy updates to follow electrochemical mechanisms. Through a soft update mechanism between the online network and the target network, stable optimization of policy parameters is achieved. This step achieves deep coupling between the physical model and reinforcement learning, preventing the policy from converging to local optima that do not conform to electrochemical laws, while improving the stability and convergence speed of the learning process. Step S5, based on the optimized action parameters, precisely adjusts the circulating pump speed using a PI controller to achieve dynamic matching between electrolyte flow rate and current; simultaneously, it updates the stack SOC and polarization voltage, forming a closed loop of "decision-execution-feedback." This ensures that the optimization objective is translated into actual operational results, and provides the latest system state for the next round of decision-making through state updates, maintaining the continuity and adaptability of control.

[0112] This invention deeply integrates physical models with reinforcement learning, innovatively constructing a multi-dimensional dynamic optimization system. It ensures action feasibility through a physical constraint projection operator, and achieves a dynamic balance between energy efficiency, state-of-charge equilibrium, and physical constraints by combining a multi-objective reward function. An innovative physically regularized dual-deep Q-network architecture is designed to incorporate electrochemical laws into the policy update process, significantly improving convergence speed and stability. Finally, relying on a high-precision sensor network and a closed-loop actuator, it achieves precise matching control of flow rate and current.

[0113] For example, such as Figure 4As shown in the embodiments of this application, a structure for a multi-stack vanadium redox flow battery energy storage system is also provided. In this battery energy storage system, multiple stacks are connected in parallel, and the positive and negative electrolytes are respectively supplied to the stacks through their respective circulation pumps.

[0114] In the description of this specification, specific features, structures, materials, or characteristics may be combined in any suitable manner in one or more embodiments or examples.

[0115] The above are merely specific embodiments of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A charge / discharge control method for a multi-stacking vanadium redox flow battery based on physical reinforcement learning, characterized in that, include: S1: Collect terminal voltage, current, state of charge, electrolyte flow rate, and ambient temperature of each fuel cell stack. Based on the collected data, construct a state vector characterizing the real-time operating state of the multi-fuel cell stack system. The state vector includes the state of charge, terminal voltage, current, electrolyte flow rate, ambient temperature, and equivalent circuit model parameters for each stack. The equivalent circuit model parameters refer to the resistance and capacitance parameters in the stack equivalent circuit model constructed based on the electrochemical mechanism. S2: Utilize a dual-deep Q-network and generate initial actions based on the state vector. The initial motion is constrained and corrected by the physical motion space projection operator; S3: Set up a multi-objective reward mechanism to drive strategy optimization in order to minimize the loss rate and balance the state of charge among the stacks; S4: The regularization term of the physical dynamic equation is integrated into the loss function of the dual deep Q network. The policy parameters are optimized synchronously through the soft update mechanism of the online network and the target network to obtain the optimized action parameters. The physical dynamic equation refers to the mathematical equation describing the electrochemical reaction law of the fuel cell. The regularization term of the physical dynamic equation is constructed based on the power conservation equation and the state of charge consistency criterion, and is used to embed the electrochemical law into the policy update process of reinforcement learning. S5: Based on the optimized action parameters, the PI controller adjusts the speed of the circulating pump to match the electrolyte flow rate and the stack current, and updates the state of charge and polarization voltage state of each stack. In step S2, the initial action includes a current distribution coefficient and a flow rate adjustment coefficient. The current distribution coefficient refers to the proportion of the current allocated to each stack to the total current of the system, and the sum of the current distribution coefficients of all stacks is 1. The flow rate adjustment coefficient refers to the correction coefficient used to adjust the electrolyte flow rate of each stack, and the value of the flow rate adjustment coefficient is related to the state of charge and current of the stack. In step S2, the physical action space projection operator includes a current distribution ratio projection operator and a flow coefficient projection operator; the current distribution ratio projection operator is used to ensure that the sum of the current distribution coefficients is 1, and the flow coefficient projection operator is used to constrain the flow regulation coefficient according to the electrode reaction law of the fuel cell stack; wherein, the constraint correction of the current distribution ratio projection operator is as follows: ； The constraint correction for the flow coefficient projection operator is as follows: ； Where tanh is a nonlinear activation function. ; Representing the The current distribution ratio of each fuel cell stack satisfies , Representing the The flow regulation coefficient of each fuel cell stack satisfies ; This represents the safe flow rate threshold for the electrolyte. ; , ... In step S3, a weighted reward function is constructed. ; in, ; ； ； , , The weighting coefficients used to balance the contributions of various indicators. For the total system loss, The standard deviation of the state of charge of each fuel cell stack; Physical constraint compliance penalties are constructed using electrodynamic constraint equations; Identify model parameters using historical data; This represents the system's maximum power loss. This represents the average state of charge of each fuel cell stack. These are the electrodynamic constraint equations; This is the loss coefficient.

2. The charging and discharging control method for a multi-staple vanadium redox flow battery based on physical reinforcement learning according to claim 1, characterized in that, The electrodynamic constraint equations include: ； ； ； ； in, For the first The charging and discharging current of each fuel cell stack For the first The terminal voltage of each fuel cell stack For the first Electrolyte flow rate of each fuel cell stack For the first The state of charge of each fuel cell stack For the first The loss of each fuel cell stack, I total P represents the total current required by the system. demand c is the power requirement; c is the electrolyte concentration. This is the rated capacity of the fuel cell stack.

3. The charging and discharging control method for a multi-staple vanadium redox flow battery based on physical reinforcement learning according to claim 1, characterized in that, In step S4, the online network is used to update parameters in real time, and the target network is used to periodically synchronize the parameters of the online network. Stable updates of the strategy are achieved through the collaborative optimization of the two.

4. The charging and discharging control method for a multi-stacking vanadium redox flow battery based on physical reinforcement learning according to claim 1, characterized in that, In step S5, the PI controller controls the electrolyte flow rate by adjusting the speed of the variable frequency pump, so that the actual flow rate tracks the set flow rate, wherein the set flow rate is determined based on the stack current and the flow rate adjustment coefficient.

5. The charging and discharging control method for a multi-staple vanadium redox flow battery based on physical reinforcement learning according to claim 1, characterized in that, In the process of optimizing strategy parameters in step S4, dynamic simulation verification of the corrected action is also performed based on the equivalent circuit model. If the simulation results violate the power balance or state of charge consistency constraints, the action parameters are further adjusted.

6. The charge / discharge control method for a multi-stacking vanadium redox flow battery based on physical reinforcement learning according to claim 5, characterized in that, The state of charge (SOC) update of each fuel cell stack is based on the ampere-hour integral method, which calculates the change in SOC based on the charging and discharging current and rated capacity of the fuel cell stack.

Images