Control algorithm of photovoltaic energy storage inverter system

Abstract

The invention discloses a control algorithm of a photovoltaic energy storage inverter system, which belongs to the technical field of photovoltaic energy storage, and comprises the steps of combining PPO deep reinforcement learning, dynamically optimizing a grid-connected control strategy of an inverter by adaptively adjusting a virtual inertia and a damping coefficient, and simultaneously compensating disturbance in the system in real time by utilizing LADRC. According to the method, the robustness and the stability of the system under the complex power grid working condition are ensured, the technical problem of improving the robustness, the dynamic response capability and the grid-connected control precision of the photovoltaic energy storage inverter system is solved, the flexibility of the system is improved, the dynamic performance of the system under the non-linear interference condition is improved, and the reliability of the system is improved. The stability and robustness of the system are maintained, and the reliability and stability of the whole grid-connected process are also significantly improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of photovoltaic energy storage, and in particular relates to a control algorithm of a photovoltaic energy storage inverter system. Background Art

[0002] With the rapid development of renewable energy, especially the popularization of photovoltaic energy storage technology, photovoltaic energy storage systems have become a key component of modern power systems. As the core equipment for photovoltaic energy storage system grid connection, the control strategy design of the inverter directly affects the stability and efficiency of the system and the power grid. In grid-connected operation, the inverter must be able to respond quickly and accurately to fluctuations in grid frequency and voltage to ensure the dynamic stability of the system. However, existing inverter control algorithms often have lags and instabilities when dealing with the uncertainty and complex dynamic characteristics of the power grid.

[0003] Virtual Synchronous Generator (VSG) technology simulates the dynamic characteristics of traditional synchronous generators by introducing virtual inertia, thereby improving the responsiveness of photovoltaic energy storage inverters when the grid fluctuates. However, the existing VSG control algorithm relies on fixed inertia and damping parameters and cannot adapt to drastic changes in the grid environment. At the same time, the Quasi Proportional Resonant (QPR) controller performs well in eliminating steady-state errors, but its design often depends on specific operating parameters and is difficult to adapt to complex and changing operating environments. Therefore, when faced with complex grid conditions, the robustness and dynamic response capabilities of the traditional VSG-QPR control strategy may be significantly limited.

[0004] Disadvantages of existing technology:

[0005] Poor adaptability: The existing VSG algorithm relies on fixed inertia and damping parameters and cannot flexibly respond to rapid fluctuations in grid frequency and voltage.

[0006] Design limitations: The QPR controller relies on specific operating parameters and is difficult to achieve good control results under different operating conditions.

[0007] Insufficient dynamic response capability: The traditional VSG-QPR control algorithm is prone to control lag or system instability when dealing with complex dynamic power grid environments. Summary of the invention

[0008] The purpose of the present invention is to provide a control algorithm for a photovoltaic energy storage inverter system, which solves the technical problem of improving the robustness, dynamic response capability and grid-connected control accuracy of the photovoltaic energy storage inverter system.

[0009] To achieve the above object, the present invention adopts the following technical solution:

[0010] A control algorithm for a photovoltaic energy storage inverter system includes the following steps:

[0011] Step 1: Initialize the parameters of the photovoltaic energy storage inverter system and collect system environment variables, specifically by setting the control parameters of the virtual synchronous generator VSG, simulating the rotational inertia of the power grid and suppressing frequency fluctuations; initialize the extended state observer ESO gain parameters of the LADRC (linear active disturbance rejection control) controller to estimate disturbances in real time; set the proportional gain, resonance gain and resonance frequency for the QPR controller to optimize the dynamic response of the grid-connected current; initialize the control parameters of the PPO (proximal policy optimization reinforcement learning) algorithm; set the reference voltage and current values ​​of the grid-connected system as the target output parameters of the inverter, collect the real-time voltage, current and frequency of the inverter and the power grid, and convert them into the input reference coordinates of the PWM control;

[0012] Step 2: The photovoltaic energy storage inverter system estimates the external disturbance of the photovoltaic energy storage inverter in real time through the extended state observer ESO, and compensates it through the control law of the LADRC controller;

[0013] Step 3: The photovoltaic energy storage inverter system is constructed and the QPR controller is used to eliminate the steady-state error of the grid-connected current through its transfer function; the grid-connected current of the inverter is kept consistent with the set value by adjusting the proportional gain and resonance gain of the QPR controller;

[0014] Step 4: The photovoltaic energy storage inverter system collects and defines state quantities to evaluate grid-connected performance; uses an adaptive proximal strategy to optimize the PPO algorithm, performs strategy updates, and optimizes the strategy by maximizing the objective function; adjusts the virtual inertia gain and damping coefficient based on the updated strategy to optimize the output control parameters of the photovoltaic energy storage inverter; trains the PPO algorithm multiple times until the strategy converges;

[0015] Step 5: The trained PPO model and its optimized hyperparameter data are solidified into the FPGA hardware in the photovoltaic energy storage inverter system. The AD sampling values ​​are transmitted to the FPGA through EMFI communication. The VSG virtual inertia and virtual damping reference values ​​are calculated by the improved PPO algorithm of the FPGA, and then transmitted from the FPGA to the DSP for calculation. Finally, the PWM controller of the photovoltaic energy storage inverter system gradually increases the output power of the inverter by adjusting the duty cycle to achieve stable grid-connected operation.

[0016] Preferably, when executing step 1, the following steps are specifically included:

[0017] Step 1-1: Set the control parameters of the virtual synchronous generator VSG, including setting the virtual inertia and damping coefficient to simulate the system inertia and suppress frequency fluctuations;

[0018] Step 1-2: Set the LADRC controller parameters, including initializing the extended state observer ESO gain parameters for real-time disturbance estimation;

[0019] Step 1-3: Setting the QPR controller parameters, including proportional gain, resonance gain, resonance frequency and grid base frequency, to adjust the dynamic response of the grid-connected current and eliminate steady-state errors;

[0020] Step 1-4: Set the PPO algorithm control parameters, including initializing the state of the PPO algorithm, outputting actions, and evaluating rewards;

[0021] Step 1-5: Set the reference voltage and reference current of the grid-connected system to provide target values ​​for the inverter;

[0022] Step 1-6: Collect the parameters of the photovoltaic energy storage inverter system and the grid-connected power grid, including the output voltage, current, and frequency of the inverter, and use them as input references for the PWM algorithm and convert them into coordinate coefficients.

[0023] Preferably, when executing step 2, the following steps are specifically included:

[0024] Step 2-1: Use the extended state observer ESO to estimate the disturbance of the system. The specific formula is as follows:

[0025]

[0026] in, represents the system disturbance estimated by the extended state observer ESO, and z3(t) is calculated by the following formula:

[0027]

[0028] Among them, z1, z2, z3 are all state variables of the extended state observer ESO, β1, β2, β3 are all gain coefficients of the extended state observer ESO, and y is the PWM duty cycle timing signal output by the photovoltaic energy storage inverter system;

[0029] Step 2-2: Compensate according to the disturbance through the control law of the LADRC controller. The specific formula is as follows:

[0030]

[0031] Among them, u(t) is the control quantity output by the LADRC controller, and u0(t) is the control quantity when there is no disturbance.

[0032] Preferably, when executing step 3, the following steps are specifically included:

[0033] Step 3-1: Construct the transfer function of the QPR controller to eliminate the steady-state error of the grid-connected current. The transfer function of the QPR controller is as follows:

[0034]

[0035] Among them, K p is the proportional gain, K r is the resonance gain, ω0 is the resonance frequency, ω0 is the fundamental frequency of the power grid, and ζ is the damping coefficient;

[0036] Step 3-2: Adjust the proportional gain and resonance gain of the QPR controller, and control the grid-connected current of the photovoltaic energy storage inverter based on the output of the QPR controller;

[0037] The output current control signal of the QPR controller is calculated by the following formula:

[0038] u QPR (t) = G QPR (s)·e I (t);

[0039] Among them, e I (t) is the error between the reference current and the actual current output by the photovoltaic energy storage inverter, G QPR (s) represents the transfer function of the QPR controller, u QPR (t) represents the output current control signal of the QPR controller.

[0040] Preferably, when executing step 4, the following steps are specifically included:

[0041] Step 4-1: Collect system output state quantities, including voltage deviation and frequency deviation;

[0042] Step 4-2: Evaluate grid connection performance based on frequency response and voltage response;

[0043] Step 4-3: Optimize the objective function according to the strategy and perform proximal strategy update;

[0044] Step 4-4: Adjust the virtual inertia gain and damping coefficient according to the learned strategy;

[0045] Step 4-5: Repeat the method flow from step 4-1 to step 4-4, and perform multiple trainings until the strategy reaches stability; output the grid-connected control model and parameters obtained by the PPO algorithm.

[0046] Preferably, when executing step 4-1, the system output state quantity is defined as S t :

[0047] S t = {ΔV t, Δf t , I t};

[0048] Where, ΔV t is the grid voltage deviation, defined as the difference between the reference voltage and the actual voltage; Δf t is the grid frequency deviation, defined as the difference between the reference frequency and the actual frequency; I t is the instantaneous value of the grid-connected current;

[0049] When executing step 4-2, use the following frequency deviation evaluation indicators to monitor system performance:

[0050]

[0051] Where Δf rms is the RMS value of the frequency deviation; T is the evaluation period, r t It shows the evaluation results of the monitoring system performance;

[0052] When executing step 4-3, the PPO algorithm is used to optimize the strategy function π θ , specifically to maximize the following objective function:

[0053] J(θ)=E t [min(r t (θ)A t G recent ,clip(r t (θ), 1-∈, 1+∈)A t )];

[0054] Among them, θ is the parameter of the strategy; r t (θ) is the probability ratio, which represents the ratio of the new strategy to the old strategy; A t is the advantage function, which is used to evaluate the quality of an action in a certain state and is related to Δf rms and ΔV t proportional to, the gain is 1; clip(·) is clip(r t (θ), 1-∈, 1+∈) function, represents the restricted range function; represents the hyperparameter that controls the clipping range;

[0055] G recent Represents the adaptive parameter, which is calculated by the following formula:

[0056]

[0057] Where t represents the current time step, τ is a time constant representing the integration time of the past return, is the average return of the batch at time step i;

[0058] When executing step 4-4, specifically, based on the learned strategy and the current optimal objective function, the virtual inertia gain parameter α1 and the damping coefficient gain parameter α2 of the photovoltaic energy storage inverter are output, and the virtual inertia J of the photovoltaic energy storage inverter output model is adjusted. v and the damping coefficient D v , the specific formula is as follows:

[0059]

[0060] in, is the reference value of virtual inertia; is the reference value of the damping coefficient; α1 and α2 are both adaptive gain parameters obtained through PPO learning;

[0061] In step 4-5, the strategy reaches stability, which means that the strategy's benefits no longer increase significantly, or the training reaches a predetermined number of iterations.

[0062] Preferably, when executing step 5, the following steps are specifically included:

[0063] Step 5-1: Deploy the trained PPO model to the FPGA controller;

[0064] Step 5-2: By controlling the duty cycle, the output power of the PWM controller is increased, so that the inverter power output gradually reaches the grid-connected requirements.

[0065] A control algorithm for a photovoltaic energy storage inverter system described in the present invention solves the technical problems of improving the robustness, dynamic response capability and grid-connected control accuracy of the photovoltaic energy storage inverter system. The present invention adopts a PPO algorithm, and adjusts the virtual inertia and damping parameters in real time through dynamic interaction with the power grid environment, and can adaptively cope with drastic changes in frequency and voltage, thereby improving the flexibility of the system. The enhanced inertia VSG based on LADRC of the present invention improves the dynamic performance of the system under nonlinear interference conditions through real-time disturbance estimation and compensation, and avoids control lag. The control algorithm of the present invention can maintain the stability and robustness of the system under complex working conditions, and effectively cope with load changes and frequency fluctuations in different operating environments. By combining VSG, LADRC and PPO, the photovoltaic energy storage inverter can not only better cope with grid-connected frequency fluctuations, but also significantly improve the reliability and stability of the entire grid-connected process. BRIEF DESCRIPTION OF THE DRAWINGS

[0066] Figure 1 It is the main flow chart of the present invention;

[0067] Figure 2 is a flow chart of step 1 of the present invention;

[0068] Figure 3is a flow chart of step 4 of the present invention;

[0069] Figure 4 It is a schematic block diagram of the hardware structure of the photovoltaic energy storage inverter of the present invention;

[0070] Figure 5 It is a schematic diagram of the control structure of the photovoltaic energy storage inverter system of the present invention. DETAILED DESCRIPTION

[0071] Depend on Figure 1-Figure 5 A control algorithm of a photovoltaic energy storage inverter system shown includes the following steps:

[0072] Step 1: Initialize the parameters of the photovoltaic energy storage inverter system and collect system environment variables, specifically by setting the control parameters of the virtual synchronous generator VSG, simulating the rotational inertia of the power grid and suppressing frequency fluctuations; initialize the extended state observer ESO (Extended State Observer, ESO) gain parameters of the LADRC controller to estimate the disturbance in real time; set the proportional gain, resonance gain and resonance frequency for the QPR (Quasi-Proportional Resonant, QPR quasi-proportional resonant control) controller to optimize the dynamic response of the grid-connected current; initialize the control parameters of the PPO algorithm; set the reference voltage and current values ​​of the grid-connected system as the target output parameters of the inverter, collect the real-time voltage, current and frequency of the inverter and the power grid, and convert them into the input reference coordinates of the PWM control;

[0073] The specific steps include:

[0074] Step 1-1: Set the control parameters of the virtual synchronous generator VSG, including setting the virtual inertia and damping coefficient to simulate the system inertia and suppress frequency fluctuations;

[0075] The initial value of virtual inertia is set according to the design parameters of the photovoltaic energy storage system and is used to simulate the system's rotational inertia to buffer the fluctuation of the grid frequency. The initial value of the reference inertia can be set to a predetermined value.

[0076] The initial value of the damping coefficient is designed according to the photovoltaic energy storage system to provide a suppression effect on frequency fluctuations and is set to the best value for damping grid frequency fluctuations.

[0077] Step 1-2: Set the LADRC controller parameters, including initializing the extended state observer ESO gain parameters for real-time disturbance estimation; the extended state observer (ESO) in the LADRC controller is used for disturbance estimation, and the initialization ESO gain parameters are:

[0078] β1=100, β2=200, β3=300.

[0079] Step 1-3: Setting the QPR controller parameters, including proportional gain, resonance gain, resonance frequency and grid base frequency, to adjust the dynamic response of the grid-connected current and eliminate steady-state errors;

[0080] The initial gain parameter setting of the QPR controller includes the proportional gain K p =1.0 and resonance gain K r =0.5, used to adjust the dynamic response of the grid-connected current and eliminate steady-state errors.

[0081] The resonant frequency ω0 = 2π × 50 rad / s, set to the grid base frequency f ref The frequency is 50 Hz, which is used to ensure the resonance characteristics of the QPR controller.

[0082] Step 1-4: Set the PPO algorithm control parameters, including initializing the state of the PPO algorithm, outputting actions, and evaluating rewards;

[0083] Step 1-5: Set the reference voltage and reference current of the grid-connected system to provide the target value for the inverter, V ref For 220V, I ref For 10A, f ref is 50Hz.

[0084] Step 1-6: Collect the parameters of the photovoltaic energy storage inverter system and the grid-connected power grid, including the output voltage, current, and frequency of the inverter, and use them as input references for the PWM algorithm and convert them into coordinate coefficients.

[0085] Collect the inverter output three-phase voltage u iabc , three-phase current i sabc , frequency f s , Real-time three-phase voltage u of the power grid after filtering oabc , three-phase current i oabc , frequency f t , the reference DC bus voltage u used for the inverter PWM algorithm dc , and transformed into αβ coordinate coefficient value through Clark transformation, that is, u iαβ ,i sαβ ,i oαβ ,i oαβ .

[0086] In this embodiment, the photovoltaic energy storage inverter system is an existing technology, specifically including DSP, FPGA, switching control equipment, DC / DC, bidirectional DC / AC, bidirectional DC / DC, photovoltaic module PV, energy storage battery group, communication module and various sensor modules, etc., and its data transmission is linked to the local area network host computer, cloud server or client software, and data collection is collected from the user-side load and distribution network. DSP focuses on tasks in digital signal processing, performs tasks such as low-power and high-efficiency encoding and decoding processing, and can also accelerate such operations through highly optimized instruction sets, thereby further improving the performance of neural network reasoning and reducing power consumption. FPGA can implement customized hardware accelerators specifically for deep learning core operations such as convolution operations and matrix multiplication. The communication module transmits the data collected from sensors or other devices to the FPGA accelerated preprocessing and neural network model for processing, and transmits the processing results back to the execution device to realize network control and application. At the same time, the optimized EtherCAT network communication protocol is adopted to achieve high transmission speed and low communication delay control.

[0087] Step 2: The photovoltaic energy storage inverter system estimates the external disturbance of the photovoltaic energy storage inverter in real time through the extended state observer ESO, and compensates it through the control law of the LADRC controller;

[0088] When executing step 2, the specific steps include:

[0089] Step 2-1: Use the extended state observer ESO to estimate the disturbance of the system. The specific formula is as follows:

[0090]

[0091] in, represents the system disturbance estimated by the extended state observer ESO, and z3(t) is calculated by the following formula:

[0092]

[0093] Among them, z1, z2, z3 are all state variables of the extended state observer ESO, β1, β2, β3 are all gain coefficients of the extended state observer ESO, and y is the PWM duty cycle timing signal output by the photovoltaic energy storage inverter system;

[0094] Step 2-2: Compensate according to the disturbance through the control law of the LADRC controller. The specific formula is as follows:

[0095]

[0096] Among them, u(t) is the control quantity output by the LADRC controller, and u0(t) is the control quantity when there is no disturbance.

[0097] Step 3: The photovoltaic energy storage inverter system is constructed and the QPR controller is used to eliminate the steady-state error of the grid-connected current through its transfer function; the grid-connected current of the inverter is kept consistent with the set value by adjusting the proportional gain and resonance gain of the QPR controller;

[0098] When executing step 3, the specific steps include:

[0099] Step 3-1: Construct the transfer function of the QPR controller to eliminate the steady-state error of the grid-connected current. The transfer function of the QPR controller is as follows:

[0100]

[0101] Among them, K p is the proportional gain, K r is the resonance gain, ω0 is the resonance frequency, ω0 is the fundamental frequency of the power grid, and ζ is the damping coefficient;

[0102] Step 3-2: Adjust the proportional gain and resonance gain of the QPR controller, and control the grid-connected current of the photovoltaic energy storage inverter based on the output of the QPR controller;

[0103] The output current control signal of the QPR controller is calculated by the following formula:

[0104] u QPR (t) = G QPR (s)·e I (t);

[0105] Among them, e I (t) is the error between the reference current and the actual current output by the photovoltaic energy storage inverter, G QPR (s) represents the transfer function of the QPR controller, u QPR (t) represents the output current control signal of the QPR controller.

[0106] Step 4: The photovoltaic energy storage inverter system collects and defines state quantities to evaluate grid-connected performance; uses an adaptive proximal strategy to optimize the PPO algorithm, performs strategy updates, and optimizes the strategy by maximizing the objective function; adjusts the virtual inertia gain and damping coefficient based on the updated strategy to optimize the output control parameters of the photovoltaic energy storage inverter; trains the PPO algorithm multiple times until the strategy converges;

[0107] When executing step 4, the specific steps include:

[0108] Step 4-1: Collect system output state quantities, including voltage deviation and frequency deviation;

[0109] Preferably, when executing step 4-1, the system output state quantity is defined as St :

[0110] S t = {ΔV t , Δf t , I t};

[0111] Where, ΔV t is the grid voltage deviation, defined as the difference between the reference voltage and the actual voltage; Δf t is the grid frequency deviation, defined as the difference between the reference frequency and the actual frequency; I t is the instantaneous value of the grid-connected current;

[0112] Step 4-2: Evaluate grid connection performance based on frequency response and voltage response;

[0113] When executing step 4-2, use the following frequency deviation evaluation indicators to monitor system performance:

[0114]

[0115] Where Δf rms is the RMS value of the frequency deviation; T is the evaluation period, r t It shows the evaluation results of the monitoring system performance;

[0116] Step 4-3: Optimize the objective function according to the strategy and perform proximal strategy update;

[0117] When executing step 4-3, the PPO algorithm is used to optimize the strategy function π θ , specifically to maximize the following objective function:

[0118] J(θ)=E t [min(r t (θ)AtG recent ,clip(r t (θ), 1-∈, 1+∈)A t )];

[0119] Among them, θ is the parameter of the strategy; r t (θ) is the probability ratio, which represents the ratio of the new strategy to the old strategy; A t is the advantage function, which is used to evaluate the quality of an action in a certain state and is related to Δf rms and ΔV t proportional to, the gain is 1; clip(·) is clip(r t (θ), 1-∈, 1+∈) function, represents the restricted range function; represents the hyperparameter that controls the clipping range;

[0120] G recentRepresents the adaptive parameter, which is calculated by the following formula:

[0121]

[0122] Where t represents the current time step, τ is a time constant representing the integration time of the past return, is the average return of the batch at time step i;

[0123] Step 4-4: Adjust the virtual inertia gain and damping coefficient according to the learned strategy;

[0124] When executing step 4-4, specifically, based on the learned strategy and the current optimal objective function, the virtual inertia gain parameter α1 and the damping coefficient gain parameter α2 of the photovoltaic energy storage inverter are output, and the virtual inertia J of the photovoltaic energy storage inverter output model is adjusted. v and the damping coefficient D v , the specific formula is as follows:

[0125]

[0126] in, is the reference value of virtual inertia; is the reference value of the damping coefficient; α1 and α2 are both adaptive gain parameters obtained through PPO learning;

[0127] Step 4-5: Repeat the method flow from step 4-1 to step 4-4 and conduct multiple trainings until the strategy reaches stability, that is, the frequency deviation evaluation index reaches the grid-connected index; output the grid-connected control model and parameters obtained by the PPO algorithm.

[0128] In step 4-5, the strategy reaches stability, which means that the strategy's benefits no longer increase significantly, or the training reaches a predetermined number of iterations.

[0129] Step 5: The trained PPO model and its optimized hyperparameter data are solidified into the FPGA hardware in the photovoltaic energy storage inverter system. The AD sampling values ​​are transmitted to the FPGA through EMFI communication. The VSG virtual inertia and virtual damping reference values ​​are calculated by the improved PPO algorithm of the FPGA, and then transmitted from the FPGA to the DSP for calculation. Finally, the PWM controller of the photovoltaic energy storage inverter system gradually increases the output power of the inverter by adjusting the duty cycle to achieve stable grid-connected operation.

[0130] When executing step 5, the specific steps include:

[0131] Step 5-1: Deploy the trained PPO model to the FPGA controller;

[0132] Step 5-2: By controlling the duty cycle, the output power of the PWM controller is increased, so that the inverter power output gradually reaches the grid-connected requirements.

[0133] In view of the problems of the existing photovoltaic energy storage inverter grid-connected control strategy in dealing with grid frequency and voltage fluctuations, insufficient dynamic performance, and difficulty in adapting to complex grid conditions, this invention proposes an enhanced virtual synchronous grid-connected control method for photovoltaic energy storage inverters. This method combines PPO deep reinforcement learning, dynamically optimizes the inverter's grid-connected control strategy by adaptively adjusting the virtual inertia and damping coefficient, and uses LADRC to compensate for disturbances in the system in real time, ensuring the robustness and stability of the system in the face of complex grid conditions. In addition, the QPR controller can accurately eliminate steady-state errors in grid-connected current control, further improving the control accuracy of the grid-connected process.

[0134] A control algorithm for a photovoltaic energy storage inverter system described in the present invention solves the technical problems of improving the robustness, dynamic response capability and grid-connected control accuracy of the photovoltaic energy storage inverter system. The present invention combines the adaptive Proximal Policy Optimization (PPO) algorithm in Deep Reinforcement Learning (DRL) to realize adaptive grid-connected control of the photovoltaic energy storage inverter. Through dynamic interaction with the power grid environment, the virtual inertia and damping parameters are adjusted in real time, and the system can be adaptively responded to drastic changes in frequency and voltage, thereby improving the flexibility of the system. The present invention is based on an enhanced inertia VSG of Linear Active Disturbance Rejection Control (LADRC). Through real-time disturbance estimation and compensation, the dynamic performance of the system under nonlinear interference conditions is improved, and control lag is avoided. The control algorithm of the present invention can maintain the stability and robustness of the system under complex working conditions, and effectively respond to load changes and frequency fluctuations in different operating environments. By combining VSG, LADRC and PPO, the photovoltaic energy storage inverter can not only better respond to grid-connected frequency fluctuations, but also significantly improve the reliability and stability of the entire grid-connected process.

Claims

1. A control algorithm for a photovoltaic energy storage inverter system, characterized in that: The steps include: Step 1: Initialize the parameters of the photovoltaic energy storage inverter system and collect system environment variables, specifically by setting the control parameters of the virtual synchronous generator VSG, simulating the rotational inertia of the power grid and suppressing frequency fluctuations; initialize the extended state observer ESO gain parameters of the LADRC controller to estimate the disturbance in real time; set the proportional gain, resonance gain and resonance frequency for the QPR controller to optimize the dynamic response of the grid-connected current; initialize the control parameters of the PPO algorithm; set the reference voltage and current values ​​of the grid-connected system as the target output parameters of the inverter, collect the real-time voltage, current and frequency of the inverter and the power grid, and convert them into the input reference coordinates of the PWM control; Step 2: The photovoltaic energy storage inverter system estimates the external disturbance of the photovoltaic energy storage inverter in real time through the extended state observer ESO, and compensates it through the control law of the LADRC controller; Step 3: The photovoltaic energy storage inverter system is constructed and the QPR controller is used to eliminate the steady-state error of the grid-connected current through its transfer function; the grid-connected current of the inverter is kept consistent with the set value by adjusting the proportional gain and resonance gain of the QPR controller; Step 4: The photovoltaic energy storage inverter system collects and defines state quantities and evaluates the grid-connected performance; the PPO algorithm is optimized using the adaptive proximal strategy, the strategy is updated, and the strategy is optimized by maximizing the objective function; Based on the updated strategy, the virtual inertia gain and damping coefficient are adjusted to optimize the output control parameters of the photovoltaic energy storage inverter; Train the PPO algorithm multiple times until the strategy converges; Step 5: The trained PPO model and its optimized hyperparameter data are solidified into the FPGA hardware in the photovoltaic energy storage inverter system. The AD sampling values ​​are transmitted to the FPGA through EMFI communication. The VSG virtual inertia and virtual damping reference values ​​are calculated by the improved PPO algorithm of the FPGA, and then transmitted from the FPGA to the DSP for calculation. Finally, the PWM controller of the photovoltaic energy storage inverter system gradually increases the output power of the inverter by adjusting the duty cycle to achieve stable grid-connected operation.

2. A control algorithm for a photovoltaic energy storage inverter system as claimed in claim 1, characterized in that: When executing step 1, the specific steps include: Step 1-1: Set the control parameters of the virtual synchronous generator VSG, including setting the virtual inertia and damping coefficient to simulate the system inertia and suppress frequency fluctuations; Step 1-2: Set the LADRC controller parameters, including initializing the extended state observer ESO gain parameters for real-time disturbance estimation; Step 1-3: Setting the QPR controller parameters, including proportional gain, resonance gain, resonance frequency and grid base frequency, to adjust the dynamic response of the grid-connected current and eliminate steady-state errors; Step 1-4: Set the PPO algorithm control parameters, including initializing the state of the PPO algorithm, outputting actions, and evaluating rewards; Step 1-5: Set the reference voltage and reference current of the grid-connected system to provide target values ​​for the inverter; Step 1-6: Collect the parameters of the photovoltaic energy storage inverter system and the grid-connected power grid, including the output voltage, current, and frequency of the inverter, and use them as input references for the PWM algorithm and convert them into coordinate coefficients.

3. The control algorithm of a photovoltaic energy storage inverter system according to claim 1, characterized in that: When executing step 2, the specific steps include: Step 2-1: Use the extended state observer ESO to estimate the disturbance of the system. The specific formula is as follows: in, represents the system disturbance estimated by the extended state observer ESO, and z3(t) is calculated by the following formula: Among them, z1, z2, z3 are all state variables of the extended state observer ESO, β1, β2, β3 are all gain coefficients of the extended state observer ESO, and y is the PWM duty cycle timing signal output by the photovoltaic energy storage inverter system; Step 2-2: Compensate according to the disturbance through the control law of the LADRC controller. The specific formula is as follows: Among them, u(t) is the control quantity output by the LADRC controller, and u0(t) is the control quantity when there is no disturbance.

4. The control algorithm of a photovoltaic energy storage inverter system according to claim 1, characterized in that: When executing step 3, the specific steps include: Step 3-1: Construct the transfer function of the QPR controller to eliminate the steady-state error of the grid-connected current. The transfer function of the QPR controller is as follows: Among them, K p is the proportional gain, K r is the resonance gain, ω0 is the resonance frequency, ω0 is the fundamental frequency of the power grid, and ζ is the damping coefficient; Step 3-2: Adjust the proportional gain and resonance gain of the QPR controller, and control the grid-connected current of the photovoltaic energy storage inverter based on the output of the QPR controller; The output current control signal of the QPR controller is calculated by the following formula: u QPR (t)=G QPR (s)·e I (t); Among them, e I (t) is the error between the reference current and the actual current output by the photovoltaic energy storage inverter, G QPR (s) represents the transfer function of the QPR controller, u QPR (t) represents the output current control signal of the QPR controller.

5. The control algorithm of a photovoltaic energy storage inverter system according to claim 1, characterized in that: When executing step 4, the specific steps include: Step 4-1: Collect system output state quantities, including voltage deviation and frequency deviation; Step 4-2: Evaluate grid connection performance based on frequency response and voltage response; Step 4-3: Optimize the objective function according to the strategy and perform proximal strategy update; Step 4-4: Adjust the virtual inertia gain and damping coefficient according to the learned strategy; Step 4-5: Repeat the method flow from step 4-1 to step 4-4, and perform multiple trainings until the strategy reaches stability; output the grid-connected control model and parameters obtained by the PPO algorithm.

6. A control algorithm for a photovoltaic energy storage inverter system as claimed in claim 5, characterized in that: When executing step 4-1, the system output state quantity is defined as S t : S t ={ΔV t ,Δf t ,I t }; Where, ΔV t is the grid voltage deviation, defined as the difference between the reference voltage and the actual voltage; Δf t is the grid frequency deviation, defined as the difference between the reference frequency and the actual frequency; I t is the instantaneous value of the grid-connected current; When executing step 4-2, use the following frequency deviation evaluation indicators to monitor system performance: Where Δf rms is the RMS value of the frequency deviation; T is the evaluation period, r t It shows the evaluation results of the monitoring system performance; When executing step 4-3, the PPO algorithm is used to optimize the strategy function π θ , specifically to maximize the following objective function: J(θ)=E t [min(r t (i)A t G recent ,clip(r t (θ), 1-∈, 1+∈)A t )]; Among them, θ is the parameter of the strategy; r t (θ) is the probability ratio, which represents the ratio of the new strategy to the old strategy; A t is the advantage function, which is used to evaluate the quality of an action in a certain state and is related to Δf rms and ΔV t proportional to, the gain is 1; clip(·) is clip(r t (θ), 1-∈, 1+∈) function, represents the limit range function; ò represents the hyperparameter controlling the cropping range; G recent Represents the adaptive parameter, which is calculated by the following formula: Where t represents the current time step, T is a time constant, which represents the integration time of the past return. is the average return of the batch at time step j; When executing step 4-4, specifically, based on the learned strategy and the current optimal objective function, the virtual inertia gain parameter α1 and the damping coefficient gain parameter α2 of the photovoltaic energy storage inverter are output, and the virtual inertia J of the photovoltaic energy storage inverter output model is adjusted. v and the damping coefficient D v , the specific formula is as follows: in, is the reference value of virtual inertia; is the reference value of the damping coefficient; α1 and α2 are both adaptive gain parameters obtained through PPO learning; In step 4-5, the strategy reaches stability, which means that the strategy's benefits no longer increase significantly, or the training reaches a predetermined number of iterations.

7. The control algorithm of a photovoltaic energy storage inverter system according to claim 1, characterized in that: When executing step 5, the specific steps include: Step 5-1: Deploy the trained PPO model to the FPGA controller; Step 5-2: By controlling the duty cycle, the output power of the PWM controller is increased, so that the inverter power output gradually reaches the grid-connected requirements.

Images