Methods and systems for predicting pump frequency in fluid energy storage systems

Abstract

This application provides a method and system for predicting the pump frequency of a flow battery energy storage system. The prediction method includes: obtaining historical data, including the electrolyte flow rate of the flow battery and the pump frequency of the electrolyte delivery pump; constructing a dataset based on the historical data, and dividing the dataset into an initial dataset and a training dataset; constructing an initial neural network model based on the initial dataset; designing a reward function based on the training dataset, and training the initial neural network model based on the reward function to obtain a trained neural network model, wherein the lower the pump frequency, the higher the weight coefficient corresponding to the pump frequency in the reward function; and using the trained neural network model to predict the probability distribution parameters of changes in the pump frequency.

Description

Technical Field

[0001] This application relates primarily to the field of energy storage technology, and in particular to a method and system for predicting the pump frequency of a fluid flow energy storage system. Background Technology

[0002] Current flow storage systems use pumps to deliver electrolyte, and the flow rate of the electrolyte is adjusted by regulating the pump frequency. However, existing pump frequency control suffers from the following technical problems: Flow storage systems employ a fixed target value pump frequency control strategy. This strategy, to ensure operational safety under extreme conditions, conservatively sets the pump frequency, meaning that a high pump frequency is used to guarantee a sufficiently large flow rate of electrolyte. This control method results in excessive pump energy consumption and low efficiency under most normal operating conditions, leading to a generally low overall energy conversion efficiency of the flow storage power station and causing significant energy waste. Summary of the Invention

[0003] This application addresses the aforementioned technical problems by providing a method and system for predicting the pump frequency of a flow energy storage system, which can reduce pump energy consumption and improve the overall energy conversion efficiency of the flow energy storage power station.

[0004] To address the aforementioned technical problems, this application provides a method for predicting the pump frequency of a flow energy storage system. The flow energy storage system includes a flow battery and an electrolyte delivery pump. The prediction method includes: obtaining historical data, which includes the electrolyte flow rate of the flow battery and the pump frequency of the electrolyte delivery pump; constructing a dataset based on the historical data, and dividing the dataset into an initial dataset and a training dataset; constructing an initial neural network model based on the initial dataset; designing a reward function based on the training dataset, and training the initial neural network model based on the reward function to obtain a trained neural network model, wherein the lower the pump frequency, the higher the weight coefficient corresponding to the pump frequency in the reward function; and using the trained neural network model to predict the probability distribution parameters of changes in pump frequency.

[0005] In one embodiment of this application, the historical data further includes any one of the following: the charge / discharge state of the flow battery, open-circuit voltage, current, voltage, and electrolyte temperature.

[0006] In one embodiment of this application, designing the reward function based on the training dataset includes: representing the reward function using the following formula:

[0007] Where r is the reward value of the reward function, f is the pump frequency, Δf is the change in pump frequency, V is the voltage of the flow battery at the current moment, Vhis is the voltage of the flow battery at the previous moment, V-Vhis represents the voltage stability, P is the charge / discharge power, Pset is the setpoint for charge / discharge power, P-Pset represents the deviation between the charge / discharge power and the setpoint, ΔOCV is the difference between the open-circuit voltage of the flow battery at the current moment and the open-circuit voltage of the flow battery at the previous moment, Δt is the time difference, and ΔOCV / Δt represents the rate of change of state of charge or the rate of change of open-circuit voltage. It is the weighting coefficient of the pump frequency. It is a weighting coefficient for the deviation between the charging / discharging power and the set charging / discharging power value. It is a weighting factor for voltage stability. It is the weighting coefficient for pump frequency fluctuation. It is the weighting coefficient for the rate of change of state of charge.

[0008] In one embodiment of this application, training the initial neural network model according to the reward function includes: obtaining a reward value according to the reward function; constructing an offline training dataset based on the reward value and the training dataset, wherein the offline training dataset includes a set of multiple state-action-reward tuples, wherein the state-action-reward tuple includes the current state, the action taken at the current time, the reward value obtained after taking the action at the current state, and the state at the next time after taking all possible actions; calculating a loss function based on the offline training dataset; updating the parameters of the online evaluation network and the target evaluation network according to the gradient descent of the loss function; and training a policy objective function based on the offline training dataset, and updating the parameters of the initial neural network model according to the gradient ascent of the policy objective function.

[0009] In one embodiment of this application, the loss function includes a conservative Q-learning loss, which is calculated using the following formula: ,

[0010] in, To conserve the learning loss of Q, This refers to the offline training dataset. For the state Calculate the expectation of the distribution on the offline training dataset. To calculate the expectation of the joint distribution of state and action, The state at the current moment, The action taken at the current moment. For all actions The complete collection, This represents the expected cumulative reward that can be obtained after performing an action in the current state.

[0011] In one embodiment of this application, the loss function includes KL divergence loss, and training the initial neural network model according to the reward function further includes: after each update of the parameters of the initial neural network model, calculating the KL divergence loss between the initial neural network model after the parameter update and the initial neural network model before the parameter update, and stopping the training of the initial neural network model in response to the KL divergence loss being greater than or equal to a first preset threshold.

[0012] In one embodiment of this application, the loss function includes Bellman loss, which is calculated using the following formula:

[0013] ,

[0014] Where Total Loss is the loss function, Bellman Loss is the Bellman loss, CQL Loss is the conservative Q-learning loss, and KL Loss is the KL divergence loss. and These are the preset weighting coefficients.

[0015] In one embodiment of this application, the prediction method further includes: in response to the KL divergence loss between the trained neural network model and the initial neural network model being greater than a second preset threshold, and the second preset threshold being greater than a first preset threshold, using the initial neural network model to predict the probability distribution parameters of the pump frequency change value.

[0016] In one embodiment of this application, the prediction method further includes: applying the probability distribution parameters of the pump frequency change value predicted by the trained neural network model to the flow energy storage system; in response to the deviation between the voltage of the flow battery and a preset voltage being greater than or equal to a first percentage of the preset voltage, and / or the deviation between the current of the flow battery and a preset current being greater than or equal to a second percentage of the preset current, using the initial neural network model to predict the probability distribution parameters of the pump frequency change value.

[0017] In one embodiment of this application, after using the trained neural network model to predict the probability distribution parameters of the pump frequency change value, the method further includes: smoothing the mean value of the pump frequency change value.

[0018] This application also provides a pump frequency prediction system for a fluid energy storage system, comprising: a memory for storing instructions executable by a processor; and a processor for executing the instructions to implement the method described above.

[0019] This application acquires historical data (including electrolyte flow rate and pump frequency) of the flow battery and electrolyte delivery pump, divides it into an initial dataset and a training dataset, constructs an initial neural network model based on the initial dataset, and trains the initial neural model using a designed reward function. During training, lower pump frequencies in the historical data are given higher weights, so that the trained neural network model can predict pump frequencies that meet the electrolyte flow rate requirements but are lower than the pump frequencies obtained by existing pump frequency control strategies, thereby effectively reducing pump consumption and improving system energy efficiency. Attached Figure Description

[0020] The accompanying drawings are included to provide a further understanding of this application; they are incorporated into and constitute a part of this application. The drawings illustrate embodiments of this application and, together with this specification, serve to explain the principles of this application. In the drawings:

[0021] Figure 1 This is a flowchart of a prediction method according to an embodiment of this application;

[0022] Figure 2 This is a schematic diagram of the pump frequency curves corresponding to multiple times obtained based on a PID control strategy.

[0023] Figure 3 Is adopted Figure 2 The diagram shows the pump consumption curve corresponding to the pump frequency.

[0024] Figure 4 This is a schematic diagram of the pump frequency curves corresponding to multiple times obtained by a prediction method based on an embodiment of this application;

[0025] Figure 5 Is adopted Figure 4 A schematic diagram of the pump consumption curve corresponding to the pump frequency in the illustrated embodiment.

[0026] Figure 6 This is a flowchart of a prediction method according to another embodiment of this application;

[0027] Figure 7 This is a flowchart of a prediction method according to another embodiment of this application;

[0028] Figure 8 This is a block diagram of a prediction system according to an embodiment of this application. Detailed Implementation

[0029] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are merely some examples or embodiments of this application. For those skilled in the art, these drawings can be applied to other similar scenarios without creative effort. Unless obvious from the context or otherwise specified, the same reference numerals in the drawings represent the same structures or operations.

[0030] As indicated in this application and claims, unless the context clearly indicates otherwise, the words "a," "an," "an," and / or "the" are not specifically singular and may include plural forms. Generally speaking, the terms "comprising" and "including" only indicate the inclusion of explicitly identified steps and elements, which do not constitute an exclusive list, and the method or apparatus may also include other steps or elements.

[0031] Flowcharts are used in this application to illustrate the operations performed by the system according to embodiments of this application. It should be understood that the preceding or following operations are not necessarily performed in exact order. Instead, various steps can be processed in reverse order or simultaneously. Furthermore, other operations may be added to these processes, or one or more steps may be removed from these processes.

[0032] Current fluid energy storage systems require pumps to deliver electrolyte, and their flow control commonly employs the PID (Proportional-Integral-Derivative) control algorithm. This method works by setting a target value (electrolyte flow rate), comparing the feedback value (real-time electrolyte flow rate) with the target value, and then using proportional (P), integral (I), and derivative (D) operations to output a result (pump frequency) to the frequency converter, thus controlling the pump frequency. In this process, the target value of the PID control algorithm is fixed. To ensure safe operation under extreme conditions, this algorithm sets a conservative target value, meaning it sets a high pump frequency to ensure a sufficiently large electrolyte flow rate. This results in the pump delivering a flow rate far exceeding actual demand, leading to excessive energy consumption and low efficiency in the energy storage system. Furthermore, according to regulations, the power station's energy conversion efficiency must reach 65% during high-voltage side metering. However, since pump consumption accounts for 20-25% of the net loss of the entire energy storage system, the high pump frequency in the above algorithm will lead to a generally low overall energy conversion efficiency of the flow storage power station, which is difficult to reach 65%. The overall energy storage system has a large loss, which is not conducive to energy saving.

[0033] To address the aforementioned issues, this application proposes a method for predicting the pump frequency of a flow energy storage system, which includes a flow battery and an electrolyte delivery pump. This application acquires historical data (including electrolyte flow rate and pump frequency) from the flow battery and electrolyte delivery pump, dividing it into an initial dataset and a training dataset. An initial neural network model is constructed based on the initial dataset and trained using a designed reward function. During training, lower pump frequencies from the historical data are given higher weights, enabling the trained neural network model to predict pump frequencies that meet electrolyte flow requirements while being lower than those obtained by existing pump frequency control strategies. This effectively reduces pump consumption and improves system energy efficiency.

[0034] Figure 1 A flowchart of a prediction method according to an embodiment of this application is shown. Figure 1 As shown, the method for predicting the pump frequency of the fluid energy storage system in this application includes:

[0035] Step S11: Obtain historical data, including the electrolyte flow rate of the flow battery and the pump frequency of the electrolyte delivery pump. The pumps used in this application include variable frequency pumps commonly used in flow batteries, such as magnetic pumps.

[0036] In some embodiments, historical data further includes any one of the following: state of charge / discharge of the flow battery, open-circuit voltage, current, voltage, and electrolyte temperature. By collecting historical data of the above parameters during the operation of the flow battery system, the neural network model can perform offline reinforcement learning to determine the pump frequency that meets the electrolyte flow requirements under various operating conditions. Therefore, the pump frequency predicted by the well-trained neural network model can also meet the needs of various operating conditions. It should be understood that the more types of parameters in the historical data, the more pump frequencies suitable for more operating conditions can be predicted. Historical data for any of the above parameters can be selected according to actual operating requirements.

[0037] In the above embodiments, the current of the flow battery includes the input current when the flow battery is charging and / or the output current when the flow battery is discharging, and the voltage of the flow battery includes the input voltage when the flow battery is charging and / or the output voltage when the flow battery is discharging.

[0038] In some embodiments, the historical data in step S11 includes data obtained using a PID control strategy.

[0039] The open-circuit voltage in step S11 can be obtained directly through measurement. In some embodiments, depending on actual engineering needs, the battery state of charge (SOC) can also be calculated from the obtained open-circuit voltage, and the battery state of charge can be used as a parameter in the historical data of this application to predict the pump frequency.

[0040] In some embodiments, the charging and discharging states in step S11 include three values: 1, -1, and 0, which represent the charging state, discharging state, and resting state, respectively.

[0041] In some embodiments, before obtaining the dataset in step S11, the historical data is further normalized.

[0042] Step S12: Based on historical data, construct a dataset and divide it into an initial dataset and a training dataset.

[0043] In some embodiments, in step S12, the normalized dataset is divided into an initial dataset and a training dataset according to the time dimension. For example, the dataset containing historical data that is relatively far from the current time is used as the initial dataset, and the dataset containing historical data that is relatively close to the current time is used as the training dataset. In some embodiments, in step S12, the normalized historical data can also be divided into three parts according to the time dimension: an initial dataset, a training dataset, and a validation dataset. The validation dataset is used to test and validate the trained neural network model.

[0044] Step S13: Construct an initial neural network model based on the initial dataset.

[0045] In some embodiments, the initial neural network model in step S13 includes an input layer, a hidden layer, and an output layer. The input layer is used to input a vector consisting of historical data of multiple parameters in the initial dataset (charge / discharge state, open-circuit voltage, current, voltage, electrolyte flow rate, electrolyte temperature, and the pump frequency of the electrolyte delivery pump); the hidden layer includes three fully connected hidden layers, and the activation function of the hidden layer can be the ReLU activation function; the output layer is the probability distribution parameter of the predicted change value (Δf) of the pump frequency.

[0046] In some embodiments, the initial neural network model is represented by formula (1):

[0047] (1)

[0048] Here, Input is a vector composed of historical data for multiple parameters, and Output is the output of the initial neural network model, namely the probability distribution parameters of Δf (including the mean μ and standard deviation σ). , and These are the biases, activation functions, and weights of the first hidden layer of a neural network model. , and These are the biases, activation functions, and weights of the second layer of a neural network model, which are hidden layers. , and It refers to the biases, activation functions, and weights of the third hidden layer of a neural network model.

[0049] In some embodiments, the parameters of the initial neural network model in formula (1) are obtained by fitting historical data in the initial dataset, namely, the specific values ​​of the bias, activation function and weight of the three hidden layers mentioned above.

[0050] In some embodiments, the probability distribution parameters of Δf include the mean μ and the standard deviation σ.

[0051] Step S14: Design a reward function based on the training dataset, and train an initial neural network model based on the reward function to obtain a trained neural network model. The lower the pump frequency, the higher the weight coefficient corresponding to the pump frequency in the reward function. In some embodiments, designing the reward function based on the training dataset in step S14 includes using the following formula to represent the reward function:

[0052] (2)

[0053] Where r is the reward value of the reward function, f is the pump frequency, Δf is the change in pump frequency, V is the voltage at the current moment, Vhis is the voltage at the previous moment, V-Vhis represents the voltage stability, P is the charging / discharging power, Pset is the charging / discharging power setpoint, P-Pset represents the deviation between the charging / discharging power and the setpoint, ΔOCV is the difference between the open-circuit voltage at the current moment and the open-circuit voltage at the previous moment, Δt is the time difference, and ΔOCV / Δt represents the rate of change of state of charge or the rate of change of open-circuit voltage. It is the weighting coefficient of pump frequency. It is a weighting coefficient for the deviation between the charging / discharging power and the set charging / discharging power value. It is a weighting factor for voltage stability. It is the weighting coefficient for pump frequency fluctuation. It is the weighting coefficient for the rate of change of state of charge.

[0054] In some embodiments, the charge / discharge power can be obtained based on the voltage and current of the flow battery.

[0055] In some embodiments, the aforementioned weighting coefficients are set according to actual engineering requirements and can be adjusted accordingly during training.

[0056] In some embodiments, The value range is 0.003-0.01. The value range is 0.4-1.0. The value range is 0.1-0.3. The value range is 0.02-0.1. The value range is 0.01 - 0.1.

[0057] In some embodiments, step S14 includes: obtaining a reward value based on a reward function; constructing an offline training dataset based on the reward value and a training dataset, wherein the offline training dataset includes a set of multiple state-action-reward tuples, wherein the state-action-reward tuple includes the current state, the action taken at the current time, the reward value obtained after taking the action at the current state, and the state at the next time after taking all possible actions; calculating a loss function based on the offline training dataset; updating the parameters of the online evaluation network and the target evaluation network based on the gradient descent of the loss function; and training a policy objective function based on the offline training dataset, and updating the parameters of the initial neural network model based on the gradient ascent of the policy objective function.

[0058] By obtaining reward values ​​according to a reward function, constructing an offline training dataset based on these reward values, and calculating a loss function using the offline training dataset, the parameters of both the online evaluation network and the target evaluation network are updated. This allows historical data to be fully learned, enabling the well-trained model to predict pump frequencies under various operating conditions. Gradient descent of the loss function is used to update the parameters of the online evaluation network, making its predictions increasingly accurate.

[0059] In some embodiments, the state-action-reward tuple is adopted. Let be the state at the current time, a be the action taken by the policy function at the current time, and r be the reward value obtained after taking the action in the current state. The state at the next moment is obtained after the policy function has taken all possible actions.

[0060] This application employs an online evaluation network and a target evaluation network for reinforcement learning. Both evaluation networks are deep neural networks, fully connected networks containing several hidden layers. The online evaluation network takes `s` and `a` as inputs and outputs the evaluation Q-value, used to assess the value of performing action `a` in state `s`. The target evaluation network takes `s` as input... Given all possible actions, the output is the highest possible evaluation value among those actions. Among them, the possible action is the action that may be taken in the next moment, which is a part of the actions in the training dataset.

[0061] In some embodiments, the loss function includes a conservative Q-learning loss, which is a conservative constraint (CQL) and is calculated using the following formula (3):

[0062] (3)

[0063] in, To conserve the learning loss of Q, This represents the offline training dataset. For the state Calculate the expectation of the distribution on the offline training dataset. To calculate the expectation of the joint distribution of state and action, For all actions The complete collection, This represents the expected cumulative reward that can be obtained after performing an action in the current state.

[0064] By incorporating conservative Q-learning loss into the loss function, the Q-values ​​of existing actions in the dataset are enhanced, and the parameters of the online evaluation network and the target evaluation network are updated. This avoids overly aggressive optimization of the pump frequency, which could result in a pump frequency that is too low to meet the electrolyte flow rate requirements.

[0065] In some embodiments, the loss function includes KL divergence loss (policy stability constraint (KL)). Training the initial neural network model based on the reward function further includes: after each update of the parameters of the initial neural network model, calculating the KL divergence loss between the initial neural network model with updated parameters and the initial neural network model before the parameter update; and stopping the training of the initial neural network model in response to the KL divergence loss being greater than or equal to a first preset threshold. By incorporating KL divergence loss into the loss function and using it to update the parameters of the online evaluation network and the target evaluation network, excessive deviation between the trained neural network model and the initial neural network model due to over-learning during training is avoided, which could lead to a pump frequency value that is too low and cannot meet the electrolyte flow rate delivery requirements.

[0066] In some embodiments, the KL divergence loss is calculated using the following formula (4):

[0067] (4)

[0068] in, For KL divergence loss, For the updated neural network model, For the initial neural network model, For the updated neural network model in state Select action The probability, For the updated neural network model in state Select action The probability is given by E, which represents the expected value.

[0069] In some embodiments, the first preset threshold is 0.05. When the KL divergence loss is ≥0.05, the loop is exited and the parameters of the neural network model are no longer updated.

[0070] In some embodiments, the first preset threshold can be set according to the actual learning depth required.

[0071] In some embodiments, the maximum number of training epochs for updating the parameters of the neural network model is num_epochs=1000.

[0072] In some embodiments, the loss function further includes: prediction error (Bellman loss), which is calculated using the following formula (5):

[0073] (5)

[0074] in, For Bellman's loss, Here, MSE represents the mean squared error, which is the discount factor. This represents the expected cumulative reward that can be obtained after performing possible actions in the next state. For the state at the next moment, For possible actions.

[0075] In some embodiments, the discount factor is 0.99.

[0076] In some embodiments, the loss function includes: Bellman loss, conservative Q-learning loss, and KL divergence loss, and the loss function is calculated using the following formula (6):

[0077] (6)

[0078] Where Total Loss is the loss function. and These are the preset weighting coefficients.

[0079] In some embodiments, , In some embodiments, and It can be set according to actual training needs.

[0080] In some embodiments, θ represents the parameters of the online evaluation network, and the gradient function of the loss function with respect to θ is: ,in, This indicates that the gradient is calculated, and the parameters of the online evaluation network are updated using gradient descent. Where η² is the learning rate, which can take the value 0.0003. In the above embodiment, the loss function is a combination of Bellman loss, conservative Q-learning loss, and KL divergence loss. The direction of gradient descent is the direction in which each loss function reduces the difference. For Bellman loss, reducing the difference means reducing the difference between the evaluation Q value of the online evaluation network and the evaluation Q value of the target evaluation network (reward r + target evaluation network). For conservative Q-learning loss, reducing the difference means reducing the difference between the expected evaluation Q value corresponding to the unfamiliar action a and the expected evaluation Q value corresponding to the historical action a. For KL divergence loss, reducing the difference means reducing the divergence of the action distribution between the updated neural network model and the initial neural network model. In some embodiments, θ′ represents the parameters of the target evaluation network, and the parameters θ′ of the target evaluation network are updated using gradient descent, i.e., Where τ is the soft update rate of the target evaluation network, which can take the value 0.005.

[0081] In some embodiments, the policy objective function is calculated using the following formula (7):

[0082] (7)

[0083] Where, objective is the calculated result of the policy objective function. Represents the "entropy regularization coefficient", when When the size is large, the exploration of unknown spaces is intensified. When the yield is small, the use of known high-yield strategies is strengthened.

[0084] In the above embodiments, the policy objective function is a quantification of the quality of the policy function (the initial neural network model or the neural network model under training). By calculating the policy objective function and performing gradient calculation on the policy objective function, the parameters of the neural network model are updated through gradient ascent, thereby obtaining a neural network model that better meets expectations.

[0085] In some embodiments, with The parameters in the neural network model (i.e., Equation (1)) are weights and biases. Since the activation function is not directly updated, the policy objective function is... The gradient function is: Update the parameters of the policy network using gradient ascent. : .

[0086] In some embodiments, this application employs the Adam optimizer, updates the learning rate η1 of the neural network model parameters to 0.0003, updates the policy network parameters by maximizing the policy objective function, and trains the parameters in batches of 256.

[0087] To avoid overly aggressive optimization, this application also includes a protection mechanism.

[0088] In some embodiments, the prediction method of this application further includes: in response to the KL divergence loss between the trained neural network model and the initial neural network model being greater than a second preset threshold, and the second preset threshold being greater than a first preset threshold, using the initial neural network model as the trained neural network model. By switching to the initial neural network model when the KL divergence loss exceeds the second preset threshold, and using the probability distribution parameters of the pump frequency change value predicted by the initial neural network model, it avoids the situation where the optimized trained neural network model becomes overly biased, resulting in a pump frequency value that is too low to meet the electrolyte flow rate delivery requirements.

[0089] In some embodiments, the first percentage is 3% and the second percentage is 5%.

[0090] In some embodiments, the preset voltage is the voltage that should be under the PID control strategy under the same operating conditions, and the preset current is the current that should be under the PID control strategy under the same operating conditions.

[0091] Step S15: Predict the probability distribution parameters of the pump frequency change values ​​using the trained neural network model. In some embodiments, step S15 is followed by smoothing the mean value of Δf. Smoothing the mean value of Δf avoids step fluctuations in the pump frequency during operation, thus preventing them from affecting the pump's lifespan.

[0092] In some embodiments, after step S15, the method further includes: switching the trained neural network model to a deterministic output mode, that is, directly outputting the mean and standard deviation of the probability distribution parameters of Δf, smoothing the mean of Δf on the time axis to obtain Δf corresponding to multiple time points, and obtaining the specific pump frequency corresponding to each time point that does not jump with time, and applying this pump frequency to the flow energy storage system. Specifically, the pump frequency obtained using a PID control strategy can be used as the initial pump frequency. The initial pump frequency and the Δf corresponding to the current time are added to obtain the pump frequency at the current time. By sequentially adding the pump frequency at the current time and the Δf corresponding to the next time point, the pump frequency at the next time point is obtained, thus obtaining the pump frequencies corresponding to multiple time points. In some embodiments, the prediction method of this application further includes: applying the probability distribution parameters of the pump frequency change value predicted by the trained neural network model to the flow energy storage system. In response to the deviation between the voltage of the flow battery and a preset voltage being greater than or equal to a first percentage of the preset voltage, and / or the deviation between the current of the flow battery and a preset current being greater than or equal to a second percentage of the preset current, the initial neural network model is used to predict the probability distribution parameters of the pump frequency change value. In the above embodiments, when using a well-trained neural network model to predict the pump frequency causes the voltage and / or current of the flow battery to deviate too much, the system switches to the initial neural network model to predict the pump frequency so that the flow battery can operate normally.

[0093] In some embodiments, after step S15, the method further includes: predicting the pump frequency using a validation dataset, and calculating the pump power difference error and pump consumption index based on the simulation environment.

[0094] Figure 2 The diagram shows the pump frequency curves corresponding to multiple times obtained based on the PID control strategy. Figure 3 It shows the use of Figure 2 The diagram shows the pump consumption curve corresponding to the pump frequency. Figure 4 A schematic diagram of pump frequency curves corresponding to multiple times obtained based on the prediction method of this application is shown. Figure 5 It shows the use of Figure 4 The illustrated embodiment shows a schematic diagram of the pump consumption curve corresponding to the pump frequency. Wherein, Figures 2-5 The horizontal axis represents time, in seconds. Figure 2 and Figure 4 The vertical axis represents frequency, with the unit being Hz. Figure 3 and Figure 5 The vertical axis represents pump consumption, with units of kWh.

[0095] contrast Figure 2 and Figure 4It can be seen that, under the same time conditions, the pump frequency obtained using the prediction method of this application is lower than the pump frequency obtained using the PID control strategy. Therefore, in Figure 3 and Figure 5 As can be seen, the pump consumption corresponding to the pump frequency obtained by using the prediction method of this application is lower than the pump consumption corresponding to the pump frequency obtained by using the PID control strategy. The prediction method of this application can reduce pump consumption by 7-10%.

[0096] In addition, such as Figure 2 As shown, the pump frequency-time curve obtained using the PID control strategy has multiple steps 21, indicating a step-like switching of the pump frequency. And as... Figure 4 As shown, the pump frequency-time curve obtained using the prediction method of this application has no steps, the switching of pump frequency has no abrupt changes, and the curve is smooth.

[0097] The following embodiments provide other exemplary processes for the prediction method of this application.

[0098] Figure 6 A flowchart of a prediction method according to another embodiment of this application is shown. Figure 6 As shown, in some embodiments of this application, the prediction method of this application includes:

[0099] Step S601: Historical Dataset That is, as mentioned above, we obtain the dataset and then divide it into the initial dataset and the training dataset.

[0100] Step S602: Reward Function That is, as mentioned above, designing a reward function based on the training dataset.

[0101] Step S603: Offline Dataset That is, as mentioned above, an offline training dataset is constructed based on the reward value and the training dataset.

[0102] Step S604: Sample Batch data, that is, obtain packaged data from the offline training dataset.

[0103] Step S605: Calculate the target Q value That is, using: r+y Q_target(s', π(s')) calculates the output of the target evaluation network.

[0104] Step S606: Calculate the current Q value That is, to calculate the output of the online evaluation network.

[0105] Step S607: Calculate the total loss function, which includes Bellman Loss, COL Loss, and KL Loss.

[0106] Step S608: Update the evaluation network using gradient descent, that is, update the parameters of the online evaluation network and the target evaluation network according to the gradient descent of the loss function.

[0107] Step S609: Calculate the policy objective function.

[0108] Step S610: Update the policy network using gradient ascent. That is, update the parameters of the neural network model according to the gradient of the policy objective function.

[0109] Step S611: Output optimization strategy That is, outputting a well-trained neural network model.

[0110] In this context, step S601 is equivalent to steps S11 to S12 as described above, and steps S602 to S611 are equivalent to step S14 as described above.

[0111] Figure 7 A flowchart of a prediction method according to another embodiment of this application is shown. Figure 7 As shown, in some embodiments of this application, the prediction method of this application includes:

[0112] Step S71: Data Acquisition and Preprocessing. This involves obtaining historical data and performing normalization processing.

[0113] Step S72: Initial strategy π-old fitting. That is, based on the initial dataset, fit the parameters of the initial neural network model.

[0114] Step S73: New strategy π-new initialization. That is, the parameters of the fitted initial neural network model are used as the parameters for subsequent training of the neural network model.

[0115] Step S74: Design the reward function.

[0116] Step S75: Offline dataset construction. That is, constructing an offline training dataset based on the reward values ​​and the training dataset.

[0117] Step S76: CQL algorithm training (including KL constraints), which involves calculating the loss function including KL loss and updating the parameters of the online evaluation network and the target evaluation network based on the gradient descent of the loss function.

[0118] Step S77: KL divergence monitoring and early stopping, that is, monitoring the KL divergence loss between the initial neural network model before updating parameters and the initial neural network model after updating parameters. In response to the KL divergence loss being greater than or equal to a first preset threshold, stopping the training of the neural network model and proceeding to step S78; otherwise, returning to step S76.

[0119] Step S78: Simulation Verification. That is, using the verification dataset to predict the pump frequency, and based on the simulation environment, calculate the pump power difference error and pump consumption index.

[0120] Step S79: Security Deployment. That is, setting up protection mechanisms includes: Step S791: KL divergence monitoring, i.e., in response to the KL divergence loss between the trained neural network model and the initial neural network model being greater than a second preset threshold, employing the initial neural network model or a PID control strategy; and Step S792: Voltage and Current Monitoring PID Backup, i.e., in response to the deviation between the flow battery voltage and a preset voltage being greater than or equal to a first percentage of the preset voltage, and / or, the deviation between the flow battery current and a preset current being greater than or equal to a second percentage of the preset current, employing the initial neural network model or a PID control strategy.

[0121] In this context, step S71 is equivalent to step S11 as described above, steps S72 to S73 are equivalent to step S13 as described above, and steps S74 to S77 are equivalent to step S14 as described above.

[0122] This application's prediction method introduces offline reinforcement learning into the pump control field of flow batteries. By using historical data and setting a reward function, a well-trained neural network model is obtained, and a protection mechanism is implemented to achieve a unification of strategy optimization and safety control. Using this prediction method can reduce pump consumption by 7-10%, improve the overall system efficiency, and solve the long-standing energy efficiency bottleneck problem in the field of flow energy storage.

[0123] An embodiment of this application also proposes a method such as Figure 8 A pump frequency prediction system 80 for a fluid flow energy storage system is shown. According to... Figure 8 The prediction system 80 may include an internal communication bus 81, a processor 82, a read-only memory (ROM) 83, a random access memory (RAM) 84, and a communication port 85. When applied to a personal computer, the prediction system 80 may also include a hard disk 86.

[0124] The internal communication bus 81 enables data communication between components of the prediction system 80. The processor 82 can perform judgments and issue prompts. In some embodiments, the processor 82 may consist of one or more processors. The communication port 85 enables data communication between the prediction system 80 and external systems. In some embodiments, the prediction system 80 can send and receive information and data from a network via the communication port 85.

[0125] The prediction system 80 may also include different forms of program storage units and data storage units, such as a hard disk 86, read-only memory (ROM) 83, and random access memory (RAM) 84, capable of storing various data files used for computer processing and / or communication, as well as possible program instructions executed by the processor 82. The processor executes these instructions to implement the main part of the method. The results of the processor processing are transmitted to the user equipment via a communication port and displayed on the user interface.

[0126] It should be noted that, in order to simplify the description of the present application and thus aid in the understanding of one or more embodiments of the invention, the foregoing description of the embodiments of the present application sometimes combines multiple features into a single embodiment, drawing, or description thereof. However, this disclosure method does not imply that the subject matter of the application requires more features than those mentioned. In fact, the embodiments contain fewer features than all the features of the single embodiments disclosed above.

[0127] In some embodiments, numbers describing the quantity of components and attributes are used. It should be understood that such numbers used in the description of embodiments are modified in some examples with the terms "approximately," "approximately," or "generally." Unless otherwise stated, "approximately," "approximately," or "generally" indicates that the numbers are allowed to vary by ±20%. Accordingly, in some embodiments, the numerical parameters used in this application are approximate values, which may be changed according to the characteristics required by individual embodiments. In some embodiments, numerical parameters should take into account specified significant digits and employ a general method of digit preservation. Although the numerical ranges and parameters used to confirm their breadth of range in some embodiments of this application are approximate values, in specific embodiments, such numerical values ​​are set as precisely as feasible.

Claims

1. A method of predicting a pump frequency of a flow-based energy storage system, the flow-based energy storage system comprising a flow battery and an electrolyte delivery pump, the method comprising: determining a state of charge of the flow battery; determining a state of health of the flow battery; determining a state of the electrolyte delivery pump; and predicting the pump frequency based on the state of charge, the state of health, and the state of the electrolyte delivery pump. The method comprises: obtaining historical data, the historical data comprising electrolyte flow of the flow battery and pump frequency of the electrolyte delivery pump; constructing a data set according to the historical data, and dividing an initial data set and a training data set from the data set; constructing an initial neural network model according to the initial data set; designing a reward function according to the training data set, and training the initial neural network model according to the reward function to obtain a trained neural network model, wherein, in response to the pump frequency being lower, a weight coefficient corresponding to the pump frequency in the reward function is higher, and the designing of the reward function according to the training data set comprises: representing the reward function by using the following formula: ， wherein r is a reward value of the reward function, f is the pump frequency, Δf is a change value of the pump frequency, V is a voltage of the flow battery at a current time, Vhis a voltage of the flow battery at a previous time, V-Vhis a voltage stability, P is a charge-discharge power, Pset is a charge-discharge power set value, P-Pset is a deviation of the charge-discharge power from the charge-discharge power set value, ΔOCV is a difference between an open circuit voltage of the flow battery at the current time and an open circuit voltage of the flow battery at the previous time, Δt is a time difference, ΔOCV / Δt is a state of charge change rate or an open circuit voltage change rate, is a weight coefficient of the pump frequency, is a weight coefficient of the deviation of the charge-discharge power from the charge-discharge power set value, is a weight coefficient of the voltage stability, is a weight coefficient of the pump frequency fluctuation, is a weight coefficient of the state of charge change rate or the open circuit voltage change rate; and using the trained neural network model to predict probability distribution parameters of a change value of the pump frequency.

2. The method of claim 1, wherein, The historical data further comprises any of the state of charge, open-circuit voltage, current, voltage and electrolyte temperature of the flow battery.

3. The method of claim 1, wherein, The training of the initial neural network model according to the reward function comprises: obtaining a reward value according to the reward function; constructing an offline training data set according to the reward value and the training data set, the offline training data set comprising a set of multiple state-action-reward tuples, the state-action-reward tuple comprising a state at a current time, an action taken at the current time, the reward value obtained after the action is taken at the state at the current time, and a state at a next time obtained after all possible actions are taken; calculating a loss function according to the offline training data set; updating parameters of an online evaluation network and a target evaluation network according to a gradient of the loss function; and training a policy target function according to the offline training data set, and updating parameters of the initial neural network model according to a gradient of the policy target function.

4. The method of claim 3, wherein, The loss function comprises a conservative Q-learning loss, and the conservative Q-learning loss is calculated by using the following formula: ， wherein, is a conservative Q-learning loss, denotes the offline training dataset, is an expectation over state is an expectation over the distribution on the offline training dataset, is an expectation over the joint distribution over state-action, is the state at the current time step, is the action taken at the current time step, is the full set of actions, is the full set of actions, denotes the expected cumulative reward that can be obtained by performing action at the current state.

5. The method of claim 3, wherein, The loss function comprises a KL divergence loss, and the training of the initial neural network model according to the reward function further comprises: after updating the parameters of the initial neural network model each time, calculating a KL divergence loss between the initial neural network model after the parameters are updated and the initial neural network model before the parameters are updated, and in response to the KL divergence loss being greater than or equal to a first preset threshold, stopping the training of the initial neural network model.

6. The method of claim 3, wherein, The loss function comprises a Bellman loss, and the loss function is calculated by using the following formula: ， where Total Loss is a loss function, Bellman Loss is a Bellman loss, CQL Loss is a conservative Q-learning loss, and KL Loss is a KL divergence loss, and are preset weight coefficients, respectively.

7. The method of claim 5, wherein, The method further comprises: in response to a KL divergence loss between the trained neural network model and the initial neural network model being greater than a second preset threshold, the second preset threshold being greater than the first preset threshold, using the initial neural network model to predict the probability distribution parameters of the change value of the pump frequency.

8. The method of claim 2, wherein, The method further comprises: The probability distribution parameter of the change value of the pump frequency predicted by the trained neural network model is applied to the liquid flow energy storage system, and in response to a deviation between the voltage of the liquid flow battery and a preset voltage being greater than or equal to a first percentage of the preset voltage, and / or a deviation between the current of the liquid flow battery and a preset current being greater than or equal to a second percentage of the preset current, the initial neural network model is used to predict the probability distribution parameter of the change value of the pump frequency.

9. The method of claim 1, wherein, The method further comprises smoothing the mean value of the change value of the pump frequency after the probability distribution parameter of the change value of the pump frequency is predicted by using the trained neural network model.

10. A system for predicting pump frequency of a flow-based energy storage system, the system comprising: The method comprises: a memory for storing instructions executable by a processor; and a processor for executing the instructions to implement the method of any one of claims 1-9.

Images