Energy storage system control method, system and device

Abstract

The invention discloses an energy storage system control method, system and device, and the method comprises the steps: building a charging and discharging model for a power grid and an energy storage system, and defining the state space of the charging and discharging model based on the current output parameter of the power grid and the remaining power of the energy storage system; the method comprises the steps of defining an action space of a charging and discharging model based on a current charging threshold value and a current discharging threshold value of an energy storage system, and defining a reward function of the charging and discharging model based on a voltage stabilization rate of a power grid; and adjusting a charging threshold value and a discharging threshold value of the energy storage system based on the optimized mapping relation between the state space and the action space. By establishing the model, the model can be dynamically planned, the adjusting strategy of the energy storage system is optimized through the PPO algorithm, stability and efficiency are higher, the charging threshold value and the discharging threshold value of the energy storage system can be adjusted in different output states of the power grid, and therefore the voltage stabilization rate of the power grid is adjusted, and the stability of the voltage of the power grid is guaranteed.

Description

Technical Field

[0001] The present invention relates to the field of power grid control, and in particular to a method, system and device for controlling an energy storage system. Background Art

[0002] With the acceleration of urbanization and the development of public transportation systems, the braking energy of urban rail transit systems is usually fed back to the power grid for braking energy recovery. However, if the feedback braking energy is large, it may cause grid voltage fluctuations. This is because when the braking energy is fed back, the grid needs to handle additional power input. If it is not handled properly, it may cause voltage fluctuations, resulting in abnormalities in electrical equipment.

[0003] In order to alleviate the fluctuation of grid voltage, in the prior art, an energy storage system is installed at the substation where the grid is located to store the energy of the grid, that is, when the grid voltage is greater than a preset charging threshold of the energy storage system, the energy storage system is charged, and when the grid voltage is less than a preset discharge threshold, the energy storage system discharges to the grid. However, since the amount of braking energy cannot be determined and the power consumption of electrical equipment in different situations is also different, fixed charging thresholds and discharging thresholds will also cause fluctuations in grid voltage. Summary of the invention

[0004] The purpose of the present invention is to provide a method, system and device for controlling an energy storage system. By establishing a model, the model can be dynamically planned, and the regulation strategy of the energy storage system can be optimized through the PPO algorithm. The stability and efficiency are higher, and the charging threshold and discharging threshold of the energy storage system can be adjusted under different output states of the power grid, thereby realizing the regulation of the voltage regulation rate of the power grid and ensuring the stability of the power grid voltage.

[0005] In order to solve the above technical problems, the present invention provides an energy storage system control method, comprising:

[0006] Establish charging and discharging models for the power grid and energy storage system;

[0007] Defining the state space of the charging and discharging model based on the current output parameters of the power grid and the remaining power of the energy storage system;

[0008] Defining an action space of the charge-discharge model based on a current charge threshold and a current discharge threshold of the energy storage system;

[0009] The reward function of the charging and discharging model is defined based on the voltage regulation rate of the power grid; the voltage regulation rate is the percentage of the change in the portion of the output voltage of the power grid exceeding the preset voltage range before and after the energy storage system is added to the portion of the output voltage of the power grid exceeding the preset voltage range when the energy storage system is not present;

[0010] Based on the state space, the action space and the reward function, the charge-discharge model is trained using a PPO algorithm to determine a mapping relationship between the optimized state space and the action space;

[0011] The charging threshold and the discharging threshold of the energy storage system are adjusted based on the optimized mapping relationship between the state space and the action space.

[0012] Preferably, the reward function of the charging and discharging model is defined based on the voltage regulation rate of the power grid, including:

[0013] The reward function of the charging and discharging model is defined based on the energy saving rate and voltage regulation rate of the power grid; the energy saving rate is the percentage of the change in the total output energy of the power grid before and after the energy storage system is added to the total output energy of the power grid without the energy storage system; the voltage regulation rate is the percentage of the change in the part of the output voltage of the power grid that exceeds the preset voltage range before and after the energy storage system is added to the part of the output voltage of the power grid that exceeds the preset voltage range without the energy storage system.

[0014] Preferably, the reward function of the charging and discharging model is defined based on the energy saving rate and voltage regulation rate of the power grid, including:

[0015] Based on the energy saving rate and voltage regulation rate of the power grid, a reward function of the charging and discharging model is defined by using a first formula;

[0016] The first formula is:

[0017] r k =0.4min(max(e k ,-10),1)+ 0.6min(max(v k ,-10),1);

[0018] Among them, r k is the time step before k moments The reward function in, k is the kth moment, is the time step, e k is the time step before k moments The energy saving rate within, v k is the time step before k moments The voltage regulation rate in the brackets is expressed as follows: min() is the minimum value in the brackets, and max() is the maximum value in the brackets.

[0019] Preferably, before defining the reward function of the charging and discharging model based on the energy saving rate and voltage regulation rate of the power grid, it also includes:

[0020] The energy saving rate is determined using a second formula based on the percentage of the change in the portion of the output voltage of the power grid that exceeds the preset voltage range before and after the energy storage system is added to the total output energy of the power grid when the energy storage system is not present. The second formula is:

[0021] ;

[0022] Among them, e k is the time step before The energy saving rate within, k is the kth moment, is the time step, and are the output voltage and output current of the grid when the energy storage system is added, and are respectively the output voltage and output current of the power grid when the energy storage system is not added, and max() is the maximum value in brackets.

[0023] Preferably, before defining the reward function of the charging and discharging model based on the energy saving rate and voltage regulation rate of the power grid, it also includes:

[0024] The voltage regulation rate is determined using a third formula based on the percentage of the portion of the output voltage of the power grid exceeding the preset voltage range after the energy storage system is added to the portion of the output voltage of the power grid exceeding the preset voltage range when the energy storage system is not added. The third formula includes:

[0025] ;

[0026] in, is the time step before k moments The voltage regulation rate within, k is the k moment, is the time step, is the absolute value of the difference between the output voltage of the power grid when the energy storage system is added and the preset voltage upper limit value; is the absolute value of the difference between the output voltage of the power grid when the energy storage system is not added and the preset voltage upper limit value; is the absolute value of the difference between the output voltage of the power grid when the energy storage system is added and the preset voltage lower limit value, is the absolute value of the difference between the output voltage of the grid when the energy storage system is not added and the preset voltage lower limit, and max() is the maximum value in the brackets.

[0027] Preferably, after adjusting the charging threshold and the discharging threshold of the energy storage system based on the optimized mapping relationship between the state space and the action space, the method further includes:

[0028] The operating parameters of the energy storage system when charging or discharging based on the adjusted charging threshold and discharging threshold are displayed through a display device.

[0029] Preferably, the state space of the charge-discharge model is defined based on the current output parameters of the power grid and the remaining power of the energy storage system, including:

[0030] The state space of the charge-discharge model is defined based on the current output voltage, current output current, current output power of the power grid and the remaining power of the energy storage system.

[0031] Preferably, before defining the state space of the charge-discharge model based on the current output voltage, the current output current, the current output power of the power grid and the remaining power of the energy storage system, the method further includes:

[0032] Obtaining the current output current of the power grid through a current sensor;

[0033] Obtaining the current output voltage of the power grid through a voltage sensor;

[0034] Determining a current output power of the power grid based on the current output current and the current output voltage;

[0035] Communicate with the energy storage system to obtain the remaining power of the energy storage system.

[0036] In order to solve the above technical problems, the present invention further provides an energy storage system control system, comprising:

[0037] A model building unit, used to build a charging and discharging model for the power grid and energy storage system;

[0038] A state space definition unit, configured to define a state space of the charge-discharge model based on a current output parameter of the power grid and a remaining amount of power of the energy storage system;

[0039] An action space definition unit, configured to define an action space of the charge-discharge model based on a current charge threshold and a current discharge threshold of the energy storage system;

[0040] A reward function definition unit, used to define a reward function of the charge-discharge model based on a voltage regulation rate of the power grid; the voltage regulation rate is a percentage of a change in a portion of the output voltage of the power grid exceeding a preset voltage range before and after the energy storage system is added to a portion of the output voltage of the power grid exceeding the preset voltage range when the energy storage system is not present;

[0041] A model training unit, configured to train the charge-discharge model using a PPO algorithm based on the state space, the action space, and the reward function, so as to determine a mapping relationship between the optimized state space and the action space;

[0042] A system adjustment unit, configured to adjust the charging threshold and the discharging threshold of the energy storage system based on the optimized mapping relationship between the state space and the action space

[0043] In order to solve the above technical problems, the present invention further provides an energy storage system control device, comprising:

[0044] Memory for storing computer programs;

[0045] The processor is used to implement the steps of the energy storage system control method as described above when executing a computer program.

[0046] The present application provides a method, system and device for controlling an energy storage system. First, a charging and discharging model is established for the power grid and the energy storage system. The state space of the charging and discharging model is defined based on the current output parameters of the power grid and the remaining power of the energy storage system. The action space of the charging and discharging model is defined based on the current charging threshold and the current discharging threshold of the energy storage system. The reward function of the charging and discharging model is defined based on the voltage regulation rate of the power grid. After the charging and discharging model is trained, the charging threshold and the discharging threshold of the energy storage system are adjusted based on the mapping relationship between the optimized state space and the action space. By establishing the model, the model can be dynamically planned, and the adjustment strategy of the energy storage system can be optimized through the PPO algorithm, which has higher stability and efficiency. The charging threshold and the discharging threshold of the energy storage system can be adjusted under different output states of the power grid, thereby adjusting the voltage regulation rate of the power grid and ensuring the stability of the power grid voltage. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] In order to more clearly illustrate the embodiments of the present invention, the following briefly introduces the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0048] Figure 1 A schematic flow chart of a method for controlling an energy storage system provided by the present invention;

[0049] Figure 2 A schematic diagram of the structure of an energy storage system control system provided by the present invention;

[0050] Figure 3 A schematic structural diagram of an energy storage system control device provided by the present invention. DETAILED DESCRIPTION

[0051] The core of the present invention is to provide a method, system and device for controlling an energy storage system. By establishing a model, the model can be dynamically planned, and the adjustment strategy of the energy storage system can be optimized through the PPO algorithm. The stability and efficiency are higher, and the charging threshold and discharging threshold of the energy storage system can be adjusted under different output states of the power grid, thereby realizing the adjustment of the voltage regulation rate of the power grid and ensuring the stability of the power grid voltage.

[0052] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0053] Please refer to Figure 1 , Figure 1 A flow chart of a method for controlling an energy storage system provided by the present invention, the method comprising:

[0054] S11: Establish charging and discharging models for the power grid and energy storage system;

[0055] When the rail vehicle is running, if the rail vehicle brakes, it will generate renewable energy to feed back to the power grid. At this time, the grid voltage will fluctuate due to the addition of renewable energy. Or when there are many electrical devices that need to be powered by the grid, or when the power is large, the grid will also fluctuate due to the large amount of electrical energy to be output, which will affect the safety of the electrical equipment. Therefore, the grid is usually equipped with an energy storage system. When the grid voltage is large, the grid charges the energy storage system to reduce the grid voltage, and when the grid voltage is small, the energy storage system charges the grid to increase the grid voltage. Based on this, the stability of the grid voltage can be guaranteed. Specifically, the charging threshold and discharge threshold of the energy storage system are set. When the grid voltage is greater than the charging threshold, the energy storage system is charged. When the grid voltage is less than the discharge threshold, the energy storage system discharges to the grid so that the grid voltage remains between the discharge threshold and the charging threshold. Among them, the energy storage system is usually set in the substation where the grid is located. The substation also includes a step-down transformer and a rectifier unit. The rectifier unit can rectify the alternating current into direct current in the grid to power the electrical equipment.

[0056] However, since the renewable energy generated by rail vehicles is uncertain, if the renewable energy is large, the voltage fed back to the power grid will be greater, and the fluctuation of the power grid voltage will be greater. The fixed charging threshold and discharging threshold cannot keep the power grid voltage stable in time. In order to solve the above technical problems, this application first establishes a charging and discharging model for the power grid and the energy storage system, and adjusts the charging threshold and discharging threshold of the charging and discharging model to ensure that the power grid voltage can be kept stable as soon as possible.

[0057] Among them, a charging and discharging model can be established for the power grid and energy storage system based on the Markov decision process. The Markov decision process has high versatility and flexibility, can handle the randomness and uncertainty in an environment with Markov properties, and is suitable for a variety of complex environments. It is also a dynamic programming method that can be solved by the dynamic programming method. This method can effectively handle multi-stage decision-making problems and find strategies to maximize returns. Therefore, the established charging and discharging model is simple and intuitive, and the decision-making problem is abstracted into a series of state and action transfer processes. This model can be easily understood and applied to practical problems, making the decision-making process more transparent and controllable, considering future returns, that is, decisions made in the current state will affect future states and returns. This global consideration can help decision makers formulate long-term strategies instead of focusing only on immediate interests.

[0058] In addition, specifically, a simulation environment is built on the Matlab / Simulink platform to establish a charging and discharging model for the processor to interact with it.

[0059] S12: defining the state space of the charging and discharging model based on the current output parameters of the power grid and the remaining power of the energy storage system;

[0060] The state space defined in this embodiment can be adjusted based on the current output parameters of the power grid, and the amount of chargeable or dischargeable energy of the current energy storage system can be determined based on the remaining amount of energy in the energy storage system. Therefore, the state space can input the current output parameters of the power grid and the remaining amount of energy in the energy storage system as the state space into the charge and discharge model. For example, when the remaining amount of energy in the energy storage system is large, the charging threshold should be adjusted to a small value and the discharging threshold should be adjusted to a large value. If the remaining amount of energy in the energy storage system is small, the charging threshold should be adjusted to a large value and the discharging threshold should be adjusted to a small value.

[0061] As a preferred embodiment, the state space of the charge-discharge model is defined based on the current output parameters of the power grid and the remaining power of the energy storage system, including:

[0062] The state space of the charging and discharging model is defined based on the current output voltage, current output current, current output power of the power grid and the remaining power of the energy storage system.

[0063] The state of the grid is determined by the current output voltage, current output current and current output power of the grid, that is, whether the grid voltage fluctuates, so that the charging and discharging model can adjust the charging threshold and the discharging threshold based on the actual state of the grid. The state space s is expressed as:

[0064] s=[P,SOC,i rec ,u rec ]; where P is the current output power, SOC is the remaining power of the energy storage system, i rec is the current output current, u rec is the current output voltage.

[0065] As a preferred embodiment, before defining the state space of the charge and discharge model based on the current output voltage, current output current, current output power of the power grid and the remaining power of the energy storage system, the method further includes:

[0066] Obtain the current output current of the power grid through a current sensor;

[0067] Obtain the current output voltage of the power grid through a voltage sensor;

[0068] determining a current output power of the power grid based on the current output current and the current output voltage;

[0069] Communicate with the energy storage system to obtain the remaining power of the energy storage system.

[0070] In addition, in this embodiment, the current output voltage of the power grid is obtained through a voltage sensor, and the current output current is obtained through a current sensor, which has high accuracy and is relatively easy to install. The current output power of the power grid is determined based on the current output current and the current output voltage, such as the current output power of the power grid is determined based on the product of the current output current and the current output voltage, and communicates with the energy storage system to obtain the remaining power of the energy storage system.

[0071] S13: defining an action space of a charge-discharge model based on a current charge threshold and a current discharge threshold of the energy storage system;

[0072] By defining the current charging threshold and the current discharging threshold as the action space, the charging and discharging model can generate the corresponding action space based on the state space to adjust the charging threshold and discharging threshold of the energy storage system in real time based on the current output parameters of the power grid and the remaining power of the energy storage system. The action space A is expressed as:

[0073] ,in, is the change in the charging threshold, is the change amount of the discharge threshold. The current charge threshold can be determined based on the change amount of the charge threshold and the initial charge threshold. The current discharge threshold can be determined based on the change amount of the discharge threshold and the initial discharge threshold.

[0074] S14: defining a reward function of the charging and discharging model based on the voltage regulation rate of the power grid; the voltage regulation rate is the percentage of the change in the portion of the power grid output voltage exceeding the preset voltage range before and after the energy storage system is added to the portion of the power grid output voltage exceeding the preset voltage range when there is no energy storage system;

[0075] When defining the reward function of the charging and discharging model, the voltage regulation rate of the power grid is mainly considered, that is, the part of the output voltage of the power grid exceeding the preset voltage range when the energy storage system is not added is used as the denominator, and the difference between the part of the output voltage of the power grid exceeding the preset voltage range when the energy storage system is not added and the part of the output voltage of the power grid exceeding the preset voltage range when the energy storage system is added is used as the numerator to calculate the voltage regulation rate. Specifically, as a preferred embodiment, before defining the reward function of the charging and discharging model based on the energy saving rate and the voltage regulation rate of the power grid, it also includes:

[0076] Based on the percentage of the change in the portion of the grid output voltage exceeding the preset voltage range before and after the energy storage system is added to the portion of the grid output voltage exceeding the preset voltage range when there is no energy storage system, the voltage regulation rate is determined using a third formula, the third formula including:

[0077] ;

[0078] in, is the time step before k moments The voltage regulation rate within, k is the k moment, is the time step, The absolute value of the difference between the output voltage of the power grid when the energy storage system is added and the preset voltage upper limit value; The absolute value of the difference between the output voltage of the power grid when the energy storage system is not added and the preset voltage upper limit value; is the absolute value of the difference between the output voltage of the grid and the preset voltage lower limit when the energy storage system is added, It is the absolute value of the difference between the output voltage of the grid and the preset voltage lower limit when the energy storage system is not added, and max() is the maximum value in the brackets.

[0079] The preset voltage upper limit value is the upper limit value of the preset voltage range, and the preset voltage lower limit value is the lower limit value of the preset voltage range. max() is the maximum value of the two values ​​in the brackets. , The n in the formula means not, fw means flywheel, i.e. flywheel energy storage system, and nfw means no energy storage system is added. h Indicates the preset voltage upper limit, u l represents the preset voltage lower limit, and t represents time.

[0080] First, the absolute value of the difference between the preset voltage upper limit and the preset voltage upper limit when the output voltage of the power grid when the energy storage system is not added is integrated, and the integral result is added to the result of integrating the absolute value of the difference between the preset voltage lower limit and the preset voltage lower limit when the output voltage of the power grid when the energy storage system is not added. The total energy of the output voltage of the power grid exceeding the preset voltage range when the energy storage system is not added is obtained, and the largest value between the total energy when the energy storage system is not added and 0.1 is selected as the denominator, that is, when the total energy when the energy storage system is not added is greater than 0.1, the total energy when the energy storage system is not added is used as the denominator, and when the total energy when the energy storage system is not added is less than 0.1, 0.1 is used as the denominator, so as to achieve special processing for the case where the denominator is extremely small.

[0081] The absolute value of the difference between the preset voltage upper limit and the preset voltage upper limit when the output voltage of the power grid when the energy storage system is added is integrated, and the result of the integration is added to the result of the integration of the absolute value of the difference between the preset voltage lower limit and the preset voltage lower limit when the output voltage of the power grid when the energy storage system is added, so as to obtain the total energy of the output voltage of the power grid exceeding the preset voltage range when the energy storage system is added, and the difference between the total energy when the energy storage system is not added and the total energy when the energy storage system is added is determined as the change in the part of the output voltage of the power grid exceeding the preset voltage range before and after the energy storage system is added, that is, determined as the numerator.

[0082] It should be noted that when calculating the voltage regulation rate, the output voltage of the power grid when the energy storage system is added is the voltage that can be output by the power grid based on the charging threshold and discharging threshold corresponding to the new state space after the charging and discharging model generates a new action space based on the state space.

[0083] S15: Based on the state space, action space and reward function, the charge and discharge model is trained using the PPO algorithm to determine the mapping relationship between the optimized state space and action space;

[0084] In this embodiment, after defining the state space, action space and reward function, the PPO (Proximal Policy Optimization) algorithm is used to train the charge and discharge model until the reward function reaches the maximum value or remains stable, so that the mapping relationship between the optimized state space and action space can be determined. The PPO algorithm has the advantages of being simple and easy to implement, high stability, and adaptability to complex environments. It uses a reinforcement learning algorithm to learn the optimal dynamic adjustment strategy for charge and discharge thresholds, and has the function of demonstrating the operation data of the energy storage system. Through continuous learning and optimization, it can effectively improve the operating efficiency and stability of the energy storage system.

[0085] The processor interacts with the simulation environment based on Matlab / Simulink to output the action status, and stores the acquired experience in the experience replay pool for updating the parameters of the Actor and Critic network models. In the online application state, it interacts with the physical energy storage system and directly outputs actions. In online applications, the collected experience information can also be used to update the model parameters in real time to continuously optimize the energy management effect.

[0086] S16: Adjust the charging threshold and discharging threshold of the energy storage system based on the mapping relationship between the optimized state space and the action space.

[0087] After determining the mapping relationship between the optimized state space and the action space, the charging threshold and discharging threshold of the energy storage system are adjusted to adjust the charging and discharging threshold of the energy storage system based on the actual output of the power grid, thereby further ensuring the stability of the power grid voltage.

[0088] In addition, the processor that executes the corresponding program in this application can be designed based on the Raspberry Pi CM4 Sensing platform with high flexibility and scalability. It is easy to deploy and adapt to energy storage systems of different sizes and types. It has the characteristics of intelligence, real-time monitoring, flexible communication, cost reduction, scalability and environmental protection. It can monitor the operating status of the energy storage system in real time and dynamically adjust the charging and discharging thresholds according to the monitoring data to optimize the real-time operating status of the energy storage system.

[0089] In summary, by establishing the model, the model can be dynamically planned, and the regulation strategy of the energy storage system can be optimized through the PPO algorithm, which has higher stability and efficiency. The charging threshold and discharging threshold of the energy storage system can be adjusted under different output states of the power grid, thereby achieving the regulation of the voltage regulation rate of the power grid and improving the stability of the power grid voltage and the reliability of the energy storage system.

[0090] Based on the above embodiments:

[0091] As a preferred embodiment, the reward function of the charging and discharging model is defined based on the target voltage regulation rate of the power grid, including:

[0092] The reward function of the charging and discharging model is defined based on the energy saving rate and voltage regulation rate of the power grid; the energy saving rate is the percentage of the change in the total output energy of the power grid before and after the addition of the energy storage system to the total output energy of the power grid without the energy storage system; the voltage regulation rate is the percentage of the change in the part of the power grid's output voltage that exceeds the preset voltage range before and after the addition of the energy storage system to the part of the power grid's output voltage that exceeds the preset voltage range when there is no energy storage system.

[0093] In this embodiment, when defining the reward function, not only the stability of the grid voltage, that is, the voltage regulation rate, is taken into account, but also the energy saving rate of the grid is taken into account, that is, the energy storage system can save electricity while ensuring the stability of the grid voltage by storing electric energy.

[0094] As a preferred embodiment, the reward function of the charging and discharging model is defined based on the energy saving rate and voltage regulation rate of the power grid, including:

[0095] Based on the energy saving rate and voltage regulation rate of the power grid, the reward function of the charging and discharging model is defined using the first formula;

[0096] The first formula is:

[0097] r k =0.4min(max(e k ,-10),1)+ 0.6min(max(v k ,-10),1);

[0098] Among them, r k is the time step before k moments The reward function within, k is the kth moment, e k is the time step before k moments Energy saving rate within, v k is the time step before k moments The voltage regulation rate in the brackets is taken as min(), and max() is taken as the maximum value in the brackets.

[0099] In this embodiment, when defining the reward function, the weights of the energy saving rate and the voltage stabilization rate are set respectively. For example, the weight of the energy saving rate is set to 0.4, and the weight of the voltage stabilization rate is set to 0.6. That is, the voltage stabilization rate is more important than the energy saving rate. Of course, this application does not limit this.

[0100] Among them, the energy saving rate is first compared with -10, and then the larger value between the energy saving rate and -10 is compared with 1, and the value less than 1 is multiplied by the weight of the energy saving rate 0.4; the voltage stabilization rate is compared with -10, and then the larger value between the voltage stabilization rate and -10 is compared with 1, and the value less than 1 is multiplied by the weight of the voltage stabilization rate 0.6; the two products are added together to obtain the reward function.

[0101] By truncating the energy saving rate and voltage regulation rate within the interval of [−10,1], the gradient explosion or gradient vanishing problem during training can be reduced, the convergence speed of the model can be accelerated, and the generalization ability of the model can be improved.

[0102] Among them, max() takes the maximum value of the two values ​​in the brackets, and min() takes the minimum value of the two values ​​in the brackets. is the time step.

[0103] As a preferred embodiment, before defining the reward function of the charging and discharging model based on the energy saving rate and voltage regulation rate of the power grid, it also includes:

[0104] Based on the percentage of the total power grid output energy change before and after the energy storage system is added to the total power grid output energy without the energy storage system, the energy saving rate is determined using the second formula, which is:

[0105] ;

[0106] Among them, e k for Time step before moment The energy saving rate within, k is the kth moment, is the time step, and They are the output voltage and output current of the grid when the energy storage system is added. and They are the output voltage and output current of the grid when no energy storage system is added, and max() is the maximum value in brackets.

[0107] Specifically, max() takes the maximum value of the two values ​​in the brackets, and min() takes the minimum value of the two values ​​in the brackets. and The n here means not, fw means flywheel, that is, flywheel energy storage system, nfw means no energy storage system is added, and t means time.

[0108] First, the output voltage and output current of the power grid when the energy storage system is added are integrated to obtain the output energy of the power grid after the energy storage system is added. The output voltage and output current of the power grid when the energy storage system is not added are integrated to obtain the output energy of the power grid before the energy storage system is added. The difference between the output energy of the power grid before the energy storage system is added and the output energy of the power grid after the energy storage system is added is determined as the change in the output energy of the power grid before and after the energy storage system is added. The output energy of the power grid after the energy storage system is not added is taken as the absolute value and the largest value between 0.1 is selected as the denominator, that is, when the absolute value of the output energy of the power grid after the energy storage system is not added is greater than 0.1, the absolute value of the output energy of the power grid after the energy storage system is not added is used as the denominator, and when the absolute value of the output energy of the power grid after the energy storage system is not added is less than 0.1, 0.1 is used as the denominator to achieve special treatment of the case where the denominator is extremely small.

[0109] As a preferred embodiment, after adjusting the charging threshold and the discharging threshold of the energy storage system based on the mapping relationship between the optimized state space and the action space, the method further includes:

[0110] The operating parameters of the energy storage system when charging or discharging based on the adjusted charging threshold and discharging threshold are displayed through the display device.

[0111] In this embodiment, not only the charging threshold and the discharging threshold of the energy storage system are adjusted based on the mapping relationship between the optimized state space and the action space, but also the operating parameters of the energy storage system when charging based on the adjusted charging threshold and the operating parameters when discharging based on the adjusted discharging threshold are displayed, so that the staff can check the charging and discharging status of the energy storage system in time.

[0112] That is, the processor can communicate with the energy storage system through the Modbus TCP protocol to achieve real-time transmission and demonstration of data, such as charging threshold, discharging threshold and the remaining power of the energy storage system; the processor can also read the pre-stored energy storage system operating status information, and then forward this information to the human-machine interface (HMI) to achieve offline demonstration of data.

[0113] In summary, the processor can connect voltage sensors and current sensors to read the output current and output voltage of the power grid and determine the output power. Through the Ethernet port, the processor can communicate with the SCU of the energy storage system using the Modbus TCP protocol to read the system status information (such as the remaining power, etc.) stored in the holding registers in the SCU, and send control instructions such as the charging threshold and the discharging threshold to the SCU by modifying the values ​​of the holding registers in the SCU. The processor can connect an external display screen via HDMI or a laptop via RealVNC to project the self-implemented demonstration interface based on Python tkinter, or forward the energy storage system operation status information to the HMI via Modbus TCP and display it on the HMI.

[0114] The process of training the charge and discharge model based on the PPO algorithm is as follows:

[0115] Training algorithm for charging and discharging model based on PPO algorithm

[0116] 1: Initialize the Actor and Critic networks and their parameters based on random weights and

[0117] 2: Repeat for each segment:

[0118] 3: Initialize the state s of the simulation environment t

[0119] 4: Initialize experience replay buffer B

[0120] 5: For each step t, repeat:

[0121] 6: Select Action ;

[0122] 7: Execute action , convert it into charge and discharge thresholds, and observe the reward and the next state s t+1 ;

[0123] 8: Storage Experience To the experience replay buffer;

[0124] 9: until the maximum fragment length is reached;

[0125] 10: Calculate each Target value ;

[0126] 11: For each s i Estimating the objective function ;

[0127] 12: For each Calculate the probability ratio of new and old strategies ;

[0128] 13: For each training cycle, repeat:

[0129] 14: Update using gradient descent , based on MSE loss ;

[0130] 15: Update using gradient descent ,based on ;

[0131] 16: Clear the experience replay buffer B;

[0132] 17: until the maximum number of fragments is reached;

[0133] 18: Save network parameters.

[0134] Where π represents the policy function and its parameter is θ. represents the probability distribution of choosing action a in environment state s given parameters θ. represents the parameters of the value function, which is used to estimate the long-term expected return of taking an action in a given state. Specifically, represents a neural network with parameters ω, which is used to predict the long-term expected return after taking an action in a given state. This network is often called a value network, and its purpose is to help the policy network make better decisions. N represents the number of samples. ε is the set limit value.

[0135] Please refer to Figure 2 , Figure 2 A schematic diagram of the structure of an energy storage system control system provided by the present invention, the system comprises:

[0136] A model building unit 21, used to build a charging and discharging model for the power grid and the energy storage system;

[0137] A state space definition unit 22, used to define a state space of a charge and discharge model based on current output parameters of the power grid and the remaining power of the energy storage system;

[0138] An action space definition unit 23, used to define an action space of a charge and discharge model based on a current charge threshold and a current discharge threshold of the energy storage system;

[0139] The reward function definition unit 24 is used to define the reward function of the charge and discharge model based on the voltage regulation rate of the power grid; the voltage regulation rate is the percentage of the output voltage of the power grid exceeding the preset voltage range after the energy storage system is added to the output voltage of the power grid exceeding the preset voltage range when there is no energy storage system;

[0140] A model training unit 25 is used to train the charge-discharge model using a PPO algorithm based on the state space, the action space and the reward function to determine a mapping relationship between the optimized state space and the action space;

[0141] The system adjustment unit 26 is used to adjust the charging threshold and the discharging threshold of the energy storage system based on the mapping relationship between the optimized state space and the action space.

[0142] For an introduction to the energy storage system control system provided by the present invention, please refer to the above method embodiment, and the present invention will not be described in detail here.

[0143] Please refer to Figure 3 , Figure 3 A schematic diagram of the structure of an energy storage system control device provided by the present invention, the device comprising:

[0144] A memory 31, used for storing computer programs;

[0145] The processor 32 is used to implement the steps of the energy storage system control method as described above when executing a computer program.

[0146] For an introduction to the energy storage system control device provided by the present invention, please refer to the above method embodiment, and the present invention will not be described in detail here.

[0147] It should also be noted that, in this specification, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "comprises", "comprising" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the statement "comprising a ..." does not exclude the presence of other identical elements in the process, method, article or device including the element.

[0148] The above description of the disclosed embodiments enables those skilled in the art to implement or use the present invention. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for controlling an energy storage system, characterized in that: include: Establish charging and discharging models for the power grid and energy storage system; Defining the state space of the charging and discharging model based on the current output parameters of the power grid and the remaining power of the energy storage system; Defining an action space of the charge-discharge model based on a current charge threshold and a current discharge threshold of the energy storage system; The reward function of the charging and discharging model is defined based on the voltage regulation rate of the power grid; the voltage regulation rate is the percentage of the change in the portion of the output voltage of the power grid exceeding the preset voltage range before and after the energy storage system is added to the portion of the output voltage of the power grid exceeding the preset voltage range when the energy storage system is not present; Based on the state space, the action space and the reward function, the charge-discharge model is trained using a PPO algorithm to determine a mapping relationship between the optimized state space and the action space; The charging threshold and the discharging threshold of the energy storage system are adjusted based on the optimized mapping relationship between the state space and the action space.

2. The energy storage system control method according to claim 1, characterized in that: The reward function of the charging and discharging model is defined based on the voltage regulation rate of the power grid, including: The reward function of the charging and discharging model is defined based on the energy saving rate and voltage regulation rate of the power grid; the energy saving rate is the percentage of the change in the total output energy of the power grid before and after the addition of the energy storage system to the total output energy of the power grid without the energy storage system; the voltage regulation rate is the percentage of the change in the portion of the output voltage of the power grid that exceeds the preset voltage range before and after the addition of the energy storage system to the portion of the output voltage of the power grid that exceeds the preset voltage range without the energy storage system.

3. The energy storage system control method according to claim 2, characterized in that: The reward function of the charging and discharging model is defined based on the energy saving rate and voltage regulation rate of the power grid, including: Based on the energy saving rate and voltage regulation rate of the power grid, a reward function of the charging and discharging model is defined by using a first formula; The first formula is: r k =0.4min(max(e k ,-10),1)+ 0.6min(max(v k ,-10),1); Among them, r k is the time step before k moments The reward function in, k is the kth moment, is the time step, e k is the time step before k moments The energy saving rate within, v k is the time step before k moments The voltage regulation rate in the brackets is expressed as follows: min() is the minimum value in the brackets, and max() is the maximum value in the brackets.

4. The energy storage system control method according to claim 2, characterized in that: Before defining the reward function of the charging and discharging model based on the energy saving rate and voltage regulation rate of the power grid, it also includes: The energy saving rate is determined using a second formula based on the percentage of the change in the portion of the output voltage of the power grid that exceeds the preset voltage range before and after the energy storage system is added to the total output energy of the power grid when the energy storage system is not present. The second formula is: ； Among them, e k is the time step before k moments The energy saving rate in, k is the kth moment, is the time step, and are the output voltage and output current of the grid when the energy storage system is added, and are respectively the output voltage and output current of the power grid when the energy storage system is not added, and max() is the maximum value in brackets.

5. The energy storage system control method according to claim 2, characterized in that: Before defining the reward function of the charging and discharging model based on the energy saving rate and voltage regulation rate of the power grid, it also includes: The voltage regulation rate is determined using a third formula based on the percentage of the portion of the output voltage of the power grid exceeding the preset voltage range after the energy storage system is added to the portion of the output voltage of the power grid exceeding the preset voltage range when the energy storage system is not added. The third formula includes: ； Among them, v k is the time step before k moments The voltage regulation rate within, k is the k moment, is the time step, is the absolute value of the difference between the output voltage of the power grid when the energy storage system is added and the preset voltage upper limit value; is the absolute value of the difference between the output voltage of the power grid when the energy storage system is not added and the preset voltage upper limit value; is the absolute value of the difference between the output voltage of the power grid when the energy storage system is added and the preset voltage lower limit value, is the absolute value of the difference between the output voltage of the grid when the energy storage system is not added and the preset voltage lower limit, and max() is the maximum value in the brackets.

6. The energy storage system control method according to claim 1, characterized in that: After adjusting the charging threshold and the discharging threshold of the energy storage system based on the optimized mapping relationship between the state space and the action space, the method further includes: The operating parameters of the energy storage system when charging or discharging based on the adjusted charging threshold and discharging threshold are displayed through a display device.

7. The energy storage system control method according to any one of claims 1 to 6, characterized in that: Defining the state space of the charging and discharging model based on the current output parameters of the power grid and the remaining power of the energy storage system includes: The state space of the charge-discharge model is defined based on the current output voltage, current output current, current output power of the power grid and the remaining power of the energy storage system.

8. The energy storage system control method according to claim 7, characterized in that: Before defining the state space of the charge-discharge model based on the current output voltage, the current output current, the current output power of the power grid and the remaining power of the energy storage system, the method further includes: Obtaining the current output current of the power grid through a current sensor; Obtaining the current output voltage of the power grid through a voltage sensor; Determining a current output power of the power grid based on the current output current and the current output voltage; Communicate with the energy storage system to obtain the remaining power of the energy storage system.

9. An energy storage system control system, characterized in that: include: A model building unit, used to build a charging and discharging model for the power grid and energy storage system; A state space definition unit, configured to define a state space of the charge-discharge model based on a current output parameter of the power grid and a remaining amount of power of the energy storage system; An action space definition unit, configured to define an action space of the charge-discharge model based on a current charge threshold and a current discharge threshold of the energy storage system; A reward function definition unit, used to define a reward function of the charge-discharge model based on a voltage regulation rate of the power grid; the voltage regulation rate is a percentage of a change in a portion of the output voltage of the power grid exceeding a preset voltage range before and after the energy storage system is added to a portion of the output voltage of the power grid exceeding the preset voltage range when the energy storage system is not present; A model training unit, configured to train the charge-discharge model using a PPO algorithm based on the state space, the action space, and the reward function, so as to determine a mapping relationship between the optimized state space and the action space; A system adjustment unit is used to adjust the charging threshold and the discharging threshold of the energy storage system based on the optimized mapping relationship between the state space and the action space.

10. An energy storage system control device, characterized in that: include: Memory for storing computer programs; A processor is used to implement the steps of the energy storage system control method as described in any one of claims 1 to 8 when executing a computer program.

Images