Reinforcement learning based soc dynamic partition energy storage system reactive power compensation method

Abstract

This invention discloses a reactive power compensation method for a dynamic SOC partitioning energy storage system based on reinforcement learning, belonging to the field of microgrid voltage regulation technology. The reactive power compensation method includes the following steps: First, the voltage, frequency, and the difference between rated voltage and rated frequency at the PCC point of the microgrid are measured in real time to obtain voltage deviation and frequency deviation; then, the SOC interval boundaries are dynamically adjusted according to bivariate threshold logic; second, reactive power commands are generated by combining the updated SOC partitioning, the current SOC state, and grid disturbance information; finally, the energy storage capacitor is driven to achieve millisecond-level instantaneous compensation. This invention's reactive power compensation method overcomes the rigid boundary compensation caused by traditional fixed SOC partitioning, achieving dynamic SOC adaptation to grid disturbances, precise reactive power regulation, and multi-timescale collaborative compensation. It solves the problems of insufficient SOC utilization, slow response speed, and weak grid support capacity in existing technologies, improving the voltage and frequency stability of the microgrid and the lifespan of energy storage.

Description

Technical Field

[0001] This invention relates to a reactive power compensation method for an energy storage system, specifically a reactive power compensation method for a SOC dynamic partitioned energy storage system based on reinforcement learning, belonging to the field of microgrid voltage regulation technology. Background Technology

[0002] With a high proportion of new energy sources being integrated into the grid, the problem of system voltage fluctuations is becoming increasingly prominent. Reactive power compensation, as a core means of maintaining voltage, is particularly important. Current new power electronic compensation devices, such as SVC and SVG, are unable to meet the demands of modern power grids due to their lack of energy throughput capacity, susceptibility to transient support saturation, and poor adaptability to continuous fluctuations.

[0003] Currently, energy storage systems have become a focal point of modern power systems due to their advantages such as fast response speed and flexible four-quadrant control of active and reactive power. However, traditional reactive power control strategies for energy storage typically employ a fixed SOC zoning method, switching charging and discharging modes based on preset thresholds. This method still has some fundamental drawbacks: 1) When the power grid experiences severe voltage fluctuations, energy storage may be limited in operation due to the SOC approaching the fixed boundary, failing to provide emergency support and triggering a chain reaction of voltage instability accidents; 2) Fixed zoning strategy leads to the suppression of available energy storage capacity. More than 30% of the theoretically available capacity is idle due to rigid boundary constraints, which significantly increases investment costs.

[0004] Therefore, existing technologies have serious limitations in terms of grid adaptability, economy, and equipment reliability. There is an urgent need for a reactive power control strategy that can dynamically partition the SOC to solve the problem of voltage instability. Summary of the Invention

[0005] The purpose of this invention is to solve the problems of voltage instability and suppression of available energy storage capacity in existing reactive power control strategies based on fixed SOC partitioning. This invention provides a reactive power compensation method for SOC dynamic partitioning energy storage systems based on reinforcement learning. This method overcomes the rigid boundary compensation caused by traditional fixed SOC partitioning, and can release energy storage potential and adaptively adjust reactive power output through dynamic partitioning, while also efficiently stabilizing grid voltage.

[0006] To achieve the above objectives, the present invention adopts the following technical solution: a reactive power compensation method for a SOC dynamic partitioned energy storage system based on reinforcement learning, comprising the following steps: S1. Real-time acquisition of voltage and frequency at the microgrid's point of common coupling (PCC) and calculation of deviation: The three-phase voltage U and frequency signal f at the microgrid's point of common coupling (PCC) are acquired in real time, and then compared with the preset reference voltage U. ref and preset reference frequency f refThe differences were used to calculate the real-time voltage deviation ΔU and frequency deviation Δf, respectively. The formula for calculating voltage deviation ΔU is: , The formula for calculating the frequency deviation Δf is: ; S2. Calculate the SOC working interval boundary based on the voltage / frequency deviation and the dynamic mapping rule obtained based on reinforcement learning: The state of charge (SOC) is divided into three intervals: a low interval (0-20%), a middle interval (20%-80%), and a high interval (80%-100%). The adjustment of the SOC working interval boundaries is implemented using a reinforcement learning (RL) framework. This framework trains an agent to adaptively learn the relationship between grid disturbances and the SOC boundaries to obtain the optimal dynamic mapping rule. The optimal dynamic mapping rule is the adjustment amount chosen by the agent for the upper limit of the low interval and the lower limit of the high interval. , Here are the expressions for voltage deviation ΔU and frequency deviation Δf; to ensure the safety of the energy storage system, an absolute lower limit SOC for the low range is set. (low_min) =5%, the absolute upper limit of SOC in the high range (high_max) =95%, and all dynamic adjustment calculations adhere to this safety boundary; Preset allowable voltage fluctuation value U for microgrid allow ±5%, preset frequency deviation f allow ±0.5Hz; when and At that time, the boundaries of the SOC interval remain unchanged; when and At that time, if Then the lower limit of the high interval Lower range upper limit ;like Then the lower limit of the high interval Lower range upper limit ; when and At that time, if Then the lower limit of the high interval Lower range upper limit ;like Then the lower limit of the high interval Lower range upper limit ; When the voltage offset percentage is less than U allow and frequency deviation less than f allow At that time, the interval boundaries are restored to the initial division of the low interval (0-20%), the middle interval (20%-80%), and the high interval (80%-100%). When the voltage offset percentage is greater than U allow and frequency deviation greater than f allow And when all deviations are greater than 0, then the lower limit of the high interval is... Lower range upper limit If both are less than 0, then lower the lower limit of the high interval and lower the upper limit of the low interval. When the voltage offset percentage is greater than U allow and frequency deviation greater than f allow When the voltage offset and frequency deviation change in opposite directions (ΔU×Δf<0), a SOC interval sequential adjustment mechanism based on normalized weights is adopted: first, the normalized deviation weight is calculated, and then the voltage offset weight coefficient W is calculated. U and frequency deviation weighting coefficient W f To ensure the primary goal of reactive power compensation for the power grid, a voltage priority coefficient is introduced. Based on the comprehensive weight ratio, the attenuation factor γ of the frequency adjustment during the superposition process is calculated; in, , , The adjusted interval boundary is: the lower limit of the higher interval. Lower range upper limit ; When the absolute value of the voltage offset percentage decreases to 5% and the absolute value of the frequency deviation decreases to within 0.5Hz, the interval boundaries are restored to the initial division of the low interval (0-20%), the middle interval (20-80%), and the high interval (80-100%). S3. Generate reactive power commands based on dynamic zoning and voltage / frequency droop coefficient: After completing the dynamic SOC partitioning in step S2 above, the corresponding droop coefficient is called according to the current SOC value range, and the reactive power command Q is generated by combining the voltage / frequency deviation and remaining capacity. ref The pre-set droop coefficients for each interval are as follows: Low range: Voltage droop coefficient Frequency droop coefficient In the formula, pu is the per-unit value; Mid-range: Voltage droop coefficient Frequency droop coefficient ; High range: Voltage droop coefficient Frequency droop coefficient ; If the current SOC is in the low range, then according to the first droop coefficient K U K f Reactive power command generated from voltage / frequency deviation Prioritize the protection of energy storage; If the current SOC is in the middle range, then according to the second droop coefficient , and the current remaining SOC rem =SOC max -SOC and voltage / frequency deviation generate reactive power instructions to dynamically allocate active / reactive power proportionally; If the current SOC is in the high range, then according to the third droop coefficient , Generate reactive power instructions with the current remaining capacity Prioritize the release of energy to support high reactive power output; S4. Reactive power command drives energy storage converter output compensation: The reactive power command Q generated in step S3 ref The signal is transmitted via CAN bus to the control module of the power storage converter (PCS). The PCS uses a dq decoupling control strategy to generate the inverter's modulation signal: first, the three-phase voltage and current signals are converted into dq-axis components through Clark and Park transformations; then, the control module generates the modulation signal based on the Q-axis... ref Calculate the reference value of the dq axis voltage , The PWM drive signal is generated through inverse Park transformation and SVPWM; the energy storage converter outputs reactive power according to the reactive power command through DC / AC conversion.

[0007] In step S2, the internal structure of reinforcement learning (RL) includes the state space S. t Action Space A t and reward function R t ; Where the state space S t : In the formula, ΔU and Δf represent the real-time voltage deviation and frequency deviation at the PCC point of the microgrid, respectively, and SOC norm The real-time state of charge is normalized to the [0-1] interval. sign() is the deviation polarity sign function. When it takes a value of +1, it represents a positive deviation of the parameter, and when it takes a value of -1, it represents a negative deviation of the parameter. Action Space A t The discrete adjustment range chosen by the agent for the SOC interval boundaries, i.e., the adjustment amount for the upper limit of the lower interval and the lower limit of the higher interval. In the formula, a and b are training parameters; Reward function R t : In the formula, R stability R stability This is a power grid stability benefit term, and its value is negatively correlated with the absolute values ​​of real-time voltage deviation ΔU and frequency deviation Δf. R SOC_utilR is the revenue item for energy storage capacity utilization. penalty This is a penalty item for exceeding limits and causing oscillations.

[0008] The training method for the agent employs a Deep Q-Network (DQN) algorithm. The algorithm's internal structure includes a neural network, experience replay, and a target network. The neural network structure utilizes a multi-layer fully connected neural network to optimize the action value function. The dataset consists of historical microgrid operation data. The training process utilizes an experience replay mechanism, storing experience tuples (St, At, Rt, St+1) in a buffer. Batch data is extracted from the buffer for training, and the target Q-value y is calculated using the Bellman equation. i The weights θ of the Q-Network are updated with the goal of minimizing the mean squared error. Simultaneously, a target network mechanism is used to periodically synchronize the network weights, improving the convergence and stability of the training process. After training, the agent's policy is extracted as an explicit mapping rule, and the optimal dynamic mapping rule is obtained by analyzing the Q-value matrix. .

[0009] In step S1, a preset reference voltage U is used. ref =380V, preset reference frequency f ref =50HZ.

[0010] The beneficial effects of this invention are: 1) This invention overcomes the limitations of traditional fixed SOC partitioning, where reactive power output jumps due to abrupt boundary changes lead to compensation failures. Unlike traditional fixed partitioning, it divides the SOC into three intervals: low, medium, and high. However, the critical values ​​of each interval are no longer fixed. Through machine learning-based preset dynamic mapping rules, combined with real-time correction based on the amplitude and direction of voltage and frequency deviations, the SOC interval boundaries return to their initial values ​​when the voltage and frequency deviations recover to within the preset deviation range. This invention dynamically adjusts the SOC interval boundaries according to changes in the microgrid, achieving smooth interval transitions. It solves the problems of poor flexibility in reactive power regulation strategies and high battery safety risks in existing technologies. It is particularly suitable for microgrid scenarios with a high proportion of distributed power sources, and has significant practical value and promotional significance.

[0011] 2) In the method of the present invention, after completing the dynamic partitioning of SOC, the device calculates the target reactive power command based on the current interval, combined with the real-time voltage deviation and frequency deviation, as well as the voltage and frequency droop coefficients corresponding to each interval and the constraints of the remaining SOC capacity. This method of multi-parameter fusion calculation improves the accuracy of adjustment and reduces the risk of battery overcharging and over-discharging.

[0012] 3) In the method of the present invention, the generated reactive power command signal is transmitted to the control module of the energy storage converter (PCS). The modulation signal of the inverter is generated through dq decoupling control. The energy storage converter outputs reactive power according to the reactive power command through DC / AC conversion. It can ensure that the energy storage system can timely and effectively suppress grid voltage / frequency fluctuations and improve the operation stability and power supply reliability of the microgrid through high-precision control and fast response.

[0013] 4) In the method of the present invention, a suitable high-precision sensor is installed at the PCC point of the microgrid to realize the real-time detection of the voltage and frequency of the microgrid. The sampling frequency of the voltage acquisition is not less than 1KHZ, and the frequency acquisition measurement module has a switching rate of 0.01HZ. The sensor accurately captures the voltage and frequency, and the voltage deviation and frequency deviation are obtained by the difference with the reference voltage and frequency to realize the rapid perception of the abnormal state of the microgrid. Attached Figure Description

[0014] Figure 1 This is a flowchart of the reactive power compensation method of the present invention; Figure 2 This is a logic diagram of the reactive power compensation method of the present invention; Figure 3 This is a schematic diagram of the dynamic state of charge (SOC) partitioned energy storage system control mechanism based on reinforcement learning in the reactive power compensation method of the present invention; Figure 4 The waveforms of the microgrid voltage, frequency, SOC, and reactive power compensation command measured in Embodiment 1 of the present invention are shown. Figure 5 The waveforms of the microgrid voltage, frequency, SOC, and reactive power compensation command measured in Embodiment 2 of the present invention are shown. Detailed Implementation

[0015] The present invention will be further explained and described below with reference to the accompanying drawings and specific embodiments.

[0016] Example: Figure 1-3 As shown, this invention provides a reactive power compensation method for a SOC dynamic partitioned energy storage system based on reinforcement learning, comprising the following steps: S1. Real-time acquisition of voltage and frequency at the microgrid's point of common coupling (PCC) and calculation of deviation: Real-time acquisition of three-phase voltage U and frequency signal f at the microgrid's point of common coupling (PCC), with a preset reference voltage U. ref =380V, preset reference frequency f ref =50Hz; then based on the preset reference voltage U ref and preset reference frequency f ref The differences were used to calculate the real-time voltage deviation ΔU and frequency deviation Δf, respectively. The formula for calculating voltage deviation ΔU is: , The formula for calculating the frequency deviation Δf is: .

[0017] S2. Calculate the SOC working interval boundary based on the voltage / frequency deviation and the dynamic mapping rule obtained based on reinforcement learning: The state of charge (SOC) is divided into three ranges: a low range of 0-20%, a middle range of 20-80%, and a high range of 80-100%. To ensure the safety of the energy storage system, an absolute lower limit for SOC is set for the low range. (low_min) =5%, the absolute upper limit of SOC in the high range (high_max) =95%, all dynamic adjustment calculations adhere to this safety boundary; the preset allowable voltage fluctuation value U of the microgrid. allow ±5%, preset frequency deviation f allow It is ±0.5Hz.

[0018] The adjustment of the SOC working interval boundary is implemented based on a reinforcement learning (RL) framework. It obtains the optimal dynamic mapping rule by training an agent to adaptively learn the relationship between grid disturbances and the SOC boundary. The optimal dynamic mapping rule is the adjustment amount selected by the agent for the upper limit of the lower interval and the lower limit of the higher interval. , Here are the expressions for voltage deviation ΔU and frequency deviation Δf.

[0019] The internal structure of reinforcement learning (RL) includes the state space S. t Action Space A t and reward function R t ; Where the state space S t : In the formula, ΔU and Δf represent the real-time voltage deviation and frequency deviation at the PCC point of the microgrid, respectively, and SOC norm The real-time state of charge is normalized to the [0-1] interval. sign() is the deviation polarity sign function. When it takes a value of +1, it represents a positive deviation of the parameter (i.e., the voltage or frequency is too high). When it takes a value of -1, it represents a negative deviation of the parameter (i.e., the voltage or frequency is too low).

[0020] Action Space A t The discrete adjustment range chosen by the agent for the SOC interval boundaries, i.e., the adjustment amount for the upper limit of the lower interval and the lower limit of the higher interval. In the formula, a and b are training parameters.

[0021] Reward function R t : In the formula, R stabilityThis is a power grid stability benefit term, whose value is negatively correlated with the absolute values ​​of real-time voltage deviation ΔU and frequency deviation Δf, used to guide the agent to prioritize mitigating power grid disturbances; R SOC_util This is the revenue term for energy storage capacity utilization, used to incentivize the system to maximize the release of the dynamic reactive power support potential of energy storage within its safe operating range; R penalty The penalty for exceeding limits and oscillations is used to suppress the setting of a safety dead zone for exceeding the SOC range boundary (i.e., exceeding limits and violating regulations) and to avoid the undesirable behavior of high-frequency chattering of control commands (i.e., frequent action jumps).

[0022] The agent is trained using a Deep Q-Network (DQN) algorithm. The algorithm's internal structure includes a neural network, an experience replay network, and a target network. The neural network structure employs a multi-layer fully connected neural network to optimize the action value function. The dataset consists of historical microgrid operation data. The training process utilizes an ExperienceReplay mechanism, storing experience tuples (St, At, Rt, St+1) in a buffer. Batch data is extracted from the buffer for training, and the target Q-value y is calculated using the Bellman equation. i The weights θ of the Q-Network are updated with the goal of minimizing the mean squared error. Simultaneously, a target network mechanism is used to periodically synchronize the network weights, improving the convergence and stability of the training process. After training, the agent's policy is extracted as an explicit mapping rule, and the optimal dynamic mapping rule is obtained by analyzing the Q-value matrix. .

[0023] when and At that time, the boundaries of the SOC interval remain unchanged.

[0024] when and At that time, if Then the lower limit of the high interval Lower range upper limit ;like Then the lower limit of the high interval Lower range upper limit .

[0025] when and At that time, if Then the lower limit of the high interval Lower range upper limit ;like Then the lower limit of the high interval Lower range upper limit .

[0026] When the voltage offset percentage is less than U allowand frequency deviation less than f allow At that time, the interval boundaries are restored to the initial division of the low interval (0-20%), the middle interval (20%-80%), and the high interval (80%-100%). When the voltage offset percentage is greater than U allow and frequency deviation greater than f allow And when all deviations are greater than 0, then the lower limit of the high interval is... Lower range upper limit If both are less than 0, then lower the lower limit of the high interval and lower the upper limit of the low interval. When the voltage offset percentage is greater than U allow and frequency deviation greater than f allow When the voltage offset and frequency deviation change in opposite directions (ΔU×Δf<0), a SOC interval sequential adjustment mechanism based on normalized weights is adopted: first, the normalized deviation weight is calculated, and then the voltage offset weight coefficient W is calculated. U and frequency deviation weighting coefficient W f To ensure the primary goal of reactive power compensation for the power grid, a voltage priority coefficient is introduced. Based on the comprehensive weight ratio, the attenuation factor γ of the frequency adjustment during the superposition process is calculated; in, , , The adjusted interval boundary is: the lower limit of the higher interval. Lower range upper limit ; When the absolute value of the voltage offset percentage decreases to 5% and the absolute value of the frequency deviation decreases to within 0.5 Hz, the interval boundaries are restored to the initially defined low interval (0-20%), middle interval (20-80%), and high interval (80-100%). , .

[0027] S3. Generate reactive power commands based on dynamic zoning and voltage / frequency droop coefficient: After completing the dynamic SOC partitioning in step S2 above, the corresponding droop coefficient is called according to the current SOC value range, and the reactive power command Q is generated by combining the voltage / frequency deviation and remaining capacity. ref The pre-set droop coefficients for each interval are as follows: Low range: Voltage droop coefficient Frequency droop coefficient In the formula, pu is the per-unit value; Mid-range: Voltage droop coefficient Frequency droop coefficient ; High range: Voltage droop coefficient Frequency droop coefficient .

[0028] If the current SOC is in the low range, then according to the first droop coefficient K U K f Reactive power command generated from voltage / frequency deviation Prioritize the protection of energy storage; If the current SOC is in the middle range, then according to the second droop coefficient , and the current remaining SOC rem =SOC max -SOC and voltage / frequency deviation generate reactive power instructions to dynamically allocate active / reactive power proportionally; If the current SOC is in the high range, then according to the third droop coefficient , Generate reactive power instructions with the current remaining capacity Prioritize the release of energy to support high reactive power output.

[0029] To illustrate more specifically the application process and effects of the method of the present invention in actual microgrid disturbance scenarios, the following description is based on two typical embodiments.

[0030] Example 1: When a microgrid experiences a sudden disconnection of a local load, causing the frequency to rise to 50.6 Hz, the voltage remains stable at 380 V; that is, the voltage deviation is 0 and the frequency deviation Δf = 0.6 Hz.

[0031] The judgment of "high frequency" triggers dynamic adjustment of the SOC range: lower limit of the high range. Lower range upper limit .

[0032] And since the current state of charge (SOC) is 50% and falls within the middle range, the reactive power command is calculated using the middle range droop factor: ,because After simplification, we get .

[0033] The energy storage system outputs inductive reactive power through a converter to suppress frequency rise. When the frequency deviation is less than 0.5 Hz, the interval boundary returns to its initial value.

[0034] like Figure 4 As shown, when t is 0, the microgrid is disturbed due to the sudden disconnection of local load. Figure (a) shows the voltage change. It can be seen that during this disturbance, since the grid voltage did not fluctuate beyond the limit, the voltage curve remained stable at around 380V, proving that this operating condition belongs to a pure frequency disturbance scenario.

[0035] As can be seen from Figure (b), the load shedding caused the frequency to rise instantaneously to 50.6 Hz. After triggering the SOC range dynamic adjustment and compensation strategy described in this invention, the frequency rise trend was quickly contained, and after about 2 seconds, it smoothly dropped back and stabilized at the rated value of 50 Hz, demonstrating excellent frequency smoothing capability.

[0036] Figure (c) shows the output curve of the reactive power command Qref of the energy storage converter. It can be seen that when the local load is suddenly cut off, the system detects that the frequency is too high, and the control module instantly calculates and sends out 0.15 pu of inductive reactive power. As the frequency gradually recovers to 50 Hz, the required reactive power also gradually decreases to 0.

[0037] Figure (d) shows the trajectory of the State of Charge (SOC) of the energy storage system. During the process of suppressing frequency rise, due to the active power surplus on the grid side, the droop control loop of the energy storage system absorbs some energy while outputting inductive reactive power, causing the SOC curve to smoothly climb from the initial 50% starting point. When the frequency returns to a safe range, active power absorption stops, and the SOC then stabilizes at a new equilibrium point.

[0038] Example 2: Due to the starting of a large-capacity motor, the voltage at the PCC point in the microgrid drops to 361V, while the frequency stabilizes at the rated 50Hz; that is, the voltage deviation is... The frequency deviation is 0.

[0039] The judgment of "low voltage" triggers dynamic adjustment of the SOC range: lower limit of the high range. Lower range upper limit It did not exceed the dead zone.

[0040] Assuming the current state of charge (SOC) is 85% and in the high range, the reactive power command is calculated using the high range droop factor: .

[0041] The energy storage converter outputs capacitive reactive power, injecting reactive power into the grid, effectively supporting the voltage at the PCC point, causing it to rise from 361V to a stable value of 380V, and then the interval boundary returns to its initial value.

[0042] like Figure 5 As shown in Figure (f), at t=0, the microgrid experiences a voltage sag disturbance due to the starting of a large-capacity motor. Figure (f) shows that during the disturbance, the system frequency remains stable near the rated value of 50Hz without any out-of-limit fluctuations, indicating that this operating condition is a typical pure voltage disturbance scenario.

[0043] Figure (e) shows that, under the current surge from the start of a large load, the voltage at the PCC point of the microgrid momentarily drops to 361V. After triggering the SOC range dynamic adjustment and compensation strategy described in this invention, the voltage drop trend is quickly contained, and the voltage smoothly recovers and stabilizes at the rated value of 380V after about 2.5 seconds, demonstrating the system's excellent voltage support and rapid recovery capability. During this transient process, The reactive power command output curve in Figure (g) shows that after the control module detects that the voltage is low and the current SOC is in the high range (85%), it instantly calculates and issues a capacitive reactive power command Qref with an amplitude of approximately -0.456 pu. This emergency compensation command smoothly decays to zero as the voltage gradually recovers.

[0044] Meanwhile, as shown in Figure (h) of the state of charge trajectory, during the adjustment phase of the output capacitive reactive power support voltage recovery, the SOC shows a decreasing trend from the initial value of 85% (down to approximately 84.97%). Once the voltage returns to the safe range, the reactive power compensation action ends, and the SOC immediately stabilizes at a new equilibrium point. This response characteristic effectively avoids unnecessary consumption of the active power of the energy storage battery under a single voltage disturbance.

[0045] The above embodiments demonstrate that the reactive power compensation method of the present invention can adaptively adjust the SOC operating range according to different grid disturbance types and generate accurate reactive power compensation commands, thereby achieving rapid and efficient support for microgrid voltage and frequency under different SOC states.

[0046] S4. Reactive power command drives energy storage converter output compensation: The reactive power command Q generated in step S3 ref The signal is transmitted via CAN bus to the control module of the power storage converter (PCS). The PCS uses a dq decoupling control strategy to generate the inverter's modulation signal: first, the three-phase voltage and current signals are converted into dq-axis components through Clark and Park transformations; then, the control module generates the modulation signal based on the Q-axis... ref Calculate the reference value of the dq axis voltage , The PWM drive signal is generated through inverse Park transformation and SVPWM; the energy storage converter outputs reactive power according to the reactive power command through DC / AC conversion, so that the voltage at the PCC point is close to 380V and the frequency is 50HZ, thereby realizing the stable compensation of the microgrid.

[0047] The reactive power compensation method of this invention breaks through the rigid boundary compensation caused by the fixed partitioning of the traditional SOC, and realizes the dynamic adaptation of SOC to grid disturbances, precise reactive power control and multi-time scale collaborative compensation. It solves the problems of insufficient SOC utilization, slow response speed and weak grid support capacity in the prior art, and improves the voltage and frequency stability of microgrid and energy storage life.

[0048] The above description is only used to illustrate the technical solution of the present invention and is not intended to limit it. Any other modifications or equivalent substitutions made by those skilled in the art to the technical solution of the present invention, as long as they do not depart from the spirit and scope of the technical solution of the present invention, should be covered within the scope of the claims of the present invention.

Claims

1. A reactive power compensation method for a SOC dynamic partitioned energy storage system based on reinforcement learning, characterized in that: Includes the following steps: S1. Real-time acquisition of voltage and frequency at the microgrid's point of common coupling (PCC) and calculation of deviation: The three-phase voltage U and frequency signal f at the microgrid's point of common coupling (PCC) are acquired in real time, and then compared with the preset reference voltage U. ref and preset reference frequency f ref The differences were used to calculate the real-time voltage deviation ΔU and frequency deviation Δf, respectively. The formula for calculating voltage deviation ΔU is: , The formula for calculating the frequency deviation Δf is: ; S2. Calculate the SOC working interval boundary based on the voltage / frequency deviation and the dynamic mapping rule obtained based on reinforcement learning: The state of charge (SOC) is divided into three intervals: a low interval (0-20%), a middle interval (20%-80%), and a high interval (80%-100%). The adjustment of the SOC working interval boundaries is implemented using a reinforcement learning (RL) framework. This framework trains an agent to adaptively learn the relationship between grid disturbances and the SOC boundaries to obtain the optimal dynamic mapping rule. The optimal dynamic mapping rule is the adjustment amount chosen by the agent for the upper limit of the low interval and the lower limit of the high interval. , Here are the expressions for voltage deviation ΔU and frequency deviation Δf; to ensure the safety of the energy storage system, an absolute lower limit SOC for the low range is set. (low_min) =5%, the absolute upper limit of SOC in the high range (high_max) =95%, and all dynamic adjustment calculations adhere to this safety boundary; Preset allowable voltage fluctuation value U for microgrid allow ±5%, preset frequency deviation f allow ±0.5Hz; when and At that time, the boundaries of the SOC interval remain unchanged; when and At that time, if Then the lower limit of the high interval Lower range upper limit ;like Then the lower limit of the high interval Lower range upper limit ; when and At that time, if Then the lower limit of the high interval Lower range upper limit ;like Then the lower limit of the high interval Lower range upper limit ; When the voltage offset percentage is less than U allow and frequency deviation less than f allow At that time, the interval boundaries are restored to the initial division of the low interval (0-20%), the middle interval (20%-80%), and the high interval (80%-100%). When the voltage offset percentage is greater than U allow and frequency deviation greater than f allow And when all deviations are greater than 0, then the lower limit of the high interval is... Lower range upper limit ; If both are less than 0, then lower the lower limit of the high interval and lower the upper limit of the low interval; When the voltage offset percentage is greater than U allow and frequency deviation greater than f allow When the voltage offset and frequency deviation change in opposite directions (ΔU×Δf<0), a SOC interval sequential adjustment mechanism based on normalized weights is adopted: first, the normalized deviation weight is calculated, and then the voltage offset weight coefficient W is calculated. U and frequency deviation weighting coefficient W f To ensure the primary goal of reactive power compensation for the power grid, a voltage priority coefficient is introduced. Based on the comprehensive weight ratio, the attenuation factor γ of the frequency adjustment during the superposition process is calculated; in, , , The adjusted interval boundary is: the lower limit of the higher interval. Lower range upper limit ; When the absolute value of the voltage offset percentage decreases to 5% and the absolute value of the frequency deviation decreases to within 0.5Hz, the interval boundaries are restored to the initial division of the low interval (0-20%), the middle interval (20-80%), and the high interval (80-100%). S3. Generate reactive power commands based on dynamic zoning and voltage / frequency droop coefficient: After completing the dynamic SOC partitioning in step S2 above, the corresponding droop coefficient is called according to the current SOC value range, and the reactive power command Q is generated by combining the voltage / frequency deviation and remaining capacity. ref The pre-set droop coefficients for each interval are as follows: Low range: Voltage droop coefficient Frequency droop coefficient In the formula, Per unit value; middle Range: Voltage droop coefficient Frequency droop coefficient ; High range: Voltage droop coefficient Frequency droop coefficient ; If the current SOC is in the low range, then according to the first droop coefficient K U K f Reactive power command generated from voltage / frequency deviation Prioritize the protection of energy storage; If the current SOC is in the middle range, then according to the second droop coefficient , and the current remaining SOC rem =SOC max -SOC and voltage / frequency deviation generate reactive power instructions to dynamically allocate active / reactive power proportionally; If the current SOC is in the high range, then according to the third droop coefficient , Generate reactive power instructions with the current remaining capacity Prioritize the release of energy to support high reactive power output; S4. Reactive power command drives energy storage converter output compensation: The reactive power command Q generated in step S3 ref The signal is transmitted via CAN bus to the control module of the power storage converter (PCS). The PCS uses a dq decoupling control strategy to generate the inverter's modulation signal: first, the three-phase voltage and current signals are converted into dq-axis components through Clark and Park transformations; then, the control module generates the modulation signal based on the Q-axis... ref Calculate the reference value of the dq axis voltage , The PWM drive signal is generated through inverse Park transformation and SVPWM; the energy storage converter outputs reactive power according to the reactive power command through DC / AC conversion.

2. The reactive power compensation method for a dynamic zoned energy storage system based on reinforcement learning according to claim 1, characterized in that: In step S2, the internal structure of reinforcement learning (RL) includes the state space S. t Action Space A t and reward function R t ; Where the state space S t : In the formula, ΔU and Δf represent the real-time voltage deviation and frequency deviation at the PCC point of the microgrid, respectively, and SOC norm The real-time state of charge is normalized to the [0-1] interval. sign() is the deviation polarity sign function. When it takes a value of +1, it represents a positive deviation of the parameter, and when it takes a value of -1, it represents a negative deviation of the parameter. Action Space A t The discrete adjustment range chosen by the agent for the SOC interval boundaries, i.e., the adjustment amount for the upper limit of the lower interval and the lower limit of the higher interval. In the formula, a and b are training parameters; Reward function R t : In the formula, R stability This is a power grid stability benefit term, and its value is negatively correlated with the absolute values ​​of real-time voltage deviation ΔU and frequency deviation Δf. R SOC_util R is the revenue item for energy storage capacity utilization. penalty This is a penalty item for exceeding limits and causing oscillations.

3. The reactive power compensation method for a dynamic zoned energy storage system based on reinforcement learning according to claim 2, characterized in that: The training method for the agent employs a Deep Q-Network (DQN) algorithm. The algorithm's internal structure includes a neural network, experience replay, and a target network. The neural network structure utilizes a multi-layer fully connected neural network to optimize the action value function. The dataset consists of historical microgrid operation data. The training process utilizes an ExperienceReplay mechanism, storing experience tuples (St, At, Rt, St+1) in a buffer. Batch data is extracted from the buffer for training, and the target Q-value y is calculated using the Bellman equation. i The weights θ of the Q-Network are updated with the goal of minimizing the mean squared error. At the same time, a target network mechanism is used to periodically synchronize the network weights to improve the convergence and stability of the training process. After training, the agent's policy is extracted as an explicit mapping rule, and the optimal dynamic mapping rule is obtained by analyzing the Q-value matrix. .

4. The reactive power compensation method for a dynamic zoned energy storage system based on reinforcement learning according to claim 1, characterized in that: In step S1, a preset reference voltage U is used. ref =380V, preset reference frequency f ref =50HZ.

Images