A grid-connected inverter reconfigurable high-order discrete control method

Abstract

The application discloses a reconfigurable high-order discrete control method of a grid-connected inverter, and belongs to the technical field of new energy grid-connected power electronic control. The method comprises an offline training stage and an online control stage. In the offline training stage, an actor network is iteratively trained by using a deep reinforcement learning algorithm in the environment of multiple power grid short-circuit ratio (SCR) working conditions and random changes of dq axis step instructions of a current loop, a learnable parameter matrix of the actor network is extracted as a target coefficient matrix, and the target coefficient matrix is configured to a reconfigurable high-order discrete controller before online control. In the online control stage, inductor current on the inverter side and capacitor voltage of an LCL filter are collected online, total voltage control amounts on dq axes are obtained through Park transformation and error calculation, and switching control signals are generated through inverse Park transformation and SVPWM. The application can realize dq axis cross-coupling active compensation, improve the adaptability to power grid impedance changes and weak power grid working conditions, and improve current dynamic response and grid-connected power quality.

Description

Technical Field

[0001] This invention relates to the field of power electronic control technology for grid-connected new energy sources, specifically a reconfigurable high-order discrete control method for grid-connected inverters. Background Technology

[0002] Three-phase LCL grid-connected inverters are the core power electronic interface devices in new energy grid-connected systems, and their current inner-loop control performance directly determines the grid-connected power quality and system stability. With the high proportion of new energy generation such as photovoltaic and wind power being connected, the power grid exhibits weak grid characteristics, with a significant increase in grid impedance, a significant decrease in the short-circuit ratio (SCR), and the equivalent inductance may exhibit frequency-dependent characteristics, placing more stringent requirements on the current control of grid-connected inverters.

[0003] The mainstream current loop control scheme in current engineering practice employs a dual proportional-integral (PI) controller with a fixed cross-coupling term in a dq synchronous rotating coordinate system: the d-axis channel uses a PI controller to adjust the d-axis current error and outputs a d-axis voltage command; the q-axis channel uses a PI controller to adjust the q-axis current error and outputs a q-axis voltage command; the cross-coupling between the d and q axes is achieved through a fixed... ωL Feedforward term (where ω The angular frequency of the power grid. L Passive cancellation is performed using a filter inductor.

[0004] However, when the system model parameters are inaccurate, the fixed-structure PI controller cannot provide sufficient control degrees of freedom to cope with uncertainties, resulting in a significant degradation in dynamic and steady-state performance; at the same time, due to the cross-decoupling terms in the traditional scheme... ωL This is a static feedforward quantity calculated based on the rated inductance parameters. As the penetration rate of new energy sources continues to increase, the power grid exhibits weak grid characteristics (lower short-circuit ratio (SCR) and significantly increased grid impedance), and the equivalent inductance is no longer constant and may exhibit frequency-dependent characteristics. At this point, a fixed... ωL The decoupling term cannot accurately compensate for the actual coupling between the d and q axes, and may even introduce additional errors, causing the system to experience low-frequency oscillations or even instability under weak power grid conditions.

[0005] Currently, numerous patents and academic papers have analyzed and reported on current loop control methods, for example: 1. A Chinese invention patent, CN119010616B, entitled "Control Method and Device for LCL-type Grid-connected Inverter," was authorized and announced on March 28, 2025. It proposes a control method for LCL-type grid-connected inverters based on a deep Q-network (DQN), directly determining the switching state of the three-phase full-bridge inverter. However, the DQN algorithm is applicable to discrete operating spaces, and the control method that directly determines the switching state cannot guarantee the smoothness of the output voltage and the quality of grid-connected power. Furthermore, this method lacks adaptability and robustness to complex operating conditions such as weak grid impedance variations and dq-axis cross-coupling.

[0006] 2. The article titled “Stability-Oriented Multiobjective Control Design for Power Converters Assisted by Deep Reinforcement Learning” by Shan Jiang, Yu Zeng, Ye Zhu, Josep Pou, and Georgios Konstantinou, published in IEEE Transactions on Power Electronics, 2023, 38(10):12394-12400, proposes a method for optimizing multiobjective control parameters of power electronic converters based on a deep reinforcement learning-assisted framework. This method can guarantee system stability under both weak and strong power grid conditions. However, this method mainly optimizes the gain adjustment parameters and does not address the adaptive compensation problem of dq-axis cross-coupling at the control structure level.

[0007] 3. The article titled "Data-Driven Reactive Power and Voltage Optimization Control for New Energy Distribution Networks Considering Photovoltaic Power Supply Reliability" by Zhang Bo, Gao Yuan, Li Tiecheng, Hu Xuekai, and Jia Jiaoxin, published in the Proceedings of the CSEE (Chinese Society for Electrical Engineering), 2024, Vol. 44, No. 15, pp. 5934-5946, proposes a method for optimizing the inner-loop control parameters of photovoltaic inverters based on the Deep Deterministic Policy Gradient Algorithm (DDPG), which can improve the speed of reactive power and voltage optimization. However, this method still relies on the framework structure of the traditional PI controller and tunes its gain parameters, failing to address the mismatch problem of fixed decoupling terms under weak grid conditions. Summary of the Invention

[0008] The technical problem this invention aims to solve is the difficulty in designing control structures and parameters, and insufficient grid adaptability, inherent in current grid-connected inverters operating in complex grid environments with high penetration of new energy sources. This invention proposes a reconfigurable high-order discrete control method for grid-connected inverters. By modifying the control structure of the current loop and utilizing reinforcement learning for parameter self-tuning of the reconfigurable high-order discrete controller, it achieves good dynamic and steady-state performance even in complex environments.

[0009] The technical solution of the present invention is as follows: A reconfigurable high-order discrete control method for a grid-connected inverter, the grid-connected inverter comprising a DC power supply, a three-phase bridge inverter circuit, a three-phase LCL filter, a grid impedance, and a three-phase grid connected in sequence; the control method involves a reconfigurable high-order discrete control structure, denoted as a controller, which includes a d-axis master channel controller. G dd (z), dq cross-coupled controller G dq( z), qd cross-coupled controller G qd (z) and q-axis main channel controller G qq (z); The control method includes an offline training phase and an online control phase; During the offline training phase, in a training environment with various grid short-circuit ratio (SCR) operating conditions and random variations in the current loop dq-axis step command, iterative training is performed using a deep reinforcement learning algorithm to obtain the target coefficient matrix of the controller. K B ; Before the start of the online control phase, the target coefficient matrix is... K B Configured to the controller; during each control cycle of the online control phase, using the target coefficient matrix configured... K B The controller performs a reconfigurable high-order discrete control on the grid-connected inverter. The steps for any control cycle are as follows: Step 1: Collect the inductor current on the inverter side. i L1_abc and LCL filter capacitor voltage v C1_abc The Park transformation is then performed to obtain the inductor current on the dq axis inverter side. i L1_d , i L1_q and dq axis LCL filter capacitor voltage v C1_d ,v C1_q ; Step 2, provide the reference value for the dq-axis current of the current loop. i d_ref , i q_ref The dq-axis current error was calculated. e d , e q ; Step 3, adjust the dq axis current error e d , e q Input controller, output total voltage control quantity for d and q axes u d , u q ; Step 4, control the total voltage of the dq axis. u d , u q Given the dq-axis gain K d , K q and dq axis LCL filter capacitor voltage v C1_d , v C1_q Obtain the dq axis output voltage v inv_d , v inv_q The three-phase modulated signal is then obtained through inverse Park transform. v inv_abc The PWM wave is generated by space vector modulation (SVPWM) and used for switching control of power devices in a three-phase bridge inverter circuit. After completing the reconfigurable high-order discrete control for the current control cycle, return to step 1 and proceed to the next control cycle.

[0010] Preferably, the dq-axis current error in step 2 e d , e q The expression is: e d = i d_ref - i L1_d e q = i q_ref - i L1_q .

[0011] Preferably, step 3 is implemented as follows: S1, dq axis current error e d , e q The input controller specifically includes: d-axis current error e d Total d-axis voltage control quantity in the previous control cycle u d Input d-axis main channel controller G dd (z): d-axis current error e d Total d-axis voltage control quantity in the previous control cycle u d 'Input dq cross-coupled controller G dq (z); q-axis current error e q Total q-axis voltage control quantity in the previous control cycle u q 'Input qd cross-coupled controller G qd (z); q-axis current error e q Total q-axis voltage control quantity in the previous control cycle u q 'Input q-axis main channel controller G qq (z); S2, the outputs of the four high-order discrete controllers are as follows: d-axis main channel controller G dd (z) Output d-axis main channel voltage component u dd ; dq cross-coupled controller G dq (z) Output d-axis error to q-axis voltage cross-compensation component u dq ; qd cross-coupled controller G qd (z) Output q-axis error to d-axis voltage cross-compensation component u qd ; q-axis main channel controller G qq(z) Output q-axis main channel voltage component u qq ; S3, the total voltage control quantity of the dq axis u d ,u q The expression is: u d = u dd + u qd u q = u dq + u qq .

[0012] Preferably, the dq axis output voltage in step 4 v inv_d , v inv_q The expressions are as follows: v inv_d = u d K d + v C1_d v inv_q = u q K q + v C1_q .

[0013] Preferably, the offline training phase includes the following steps: Z1, introducing a state space S Action space A Reward function R ; Let a total of T Each discrete sample is recorded as the current sample. k , k =1,2,..., T ; The state space S Defined as all states s The set, which will include the current samples k The state at that time is recorded as the current state. s k , sk = ( e d (k), e q (k), u d (k), u q (k)); The action space A Defined as all actions a The set that contains the current state s k The action at that time is recorded as the current action. a k , a k =( u d (k), u q (k)); The reward function R Defined as including all rewards r The set that contains the current state s k The corresponding reward is recorded as the current reward. r k , r k =-( e d 2 (k)+ e q 2 (k)); u d (k) , u q (k) is related to the current state s k The corresponding total voltage control quantity of the dq axis e d (k), e q (k) represents the current state. s k The corresponding dq axis current error; Z2 constructs a neural network based on the dual-delay deep deterministic policy gradient (TD3) algorithm. The neural network includes an Actor network, a first Critic network, a second Critic network, a target Actor network, a first target Critic network, and a second target Critic network. The Actor network includes basis functions and a learnable parameter matrix. The basis functions are defined by states... sFor input, and input state s The learnable parameter matrix of the Actor network is directly output as the state feature; the learnable parameter matrix of the Actor network is denoted as the coefficient matrix. K The Actor network is based on the state s sum coefficient matrix K Output Action a , a =π( s ; K Both the first and second Critic networks have input states. s ,action a The first Critic network outputs the first action value function. Q 1( s , a The second Critic network outputs the second action value function. Q 2( s , a ); Z3 initializes all neural network parameters and initializes the experience replay pool. P ; Z4, given the training step size step and the maximum training step size step. m Where step=1,2,...,step m And will complete the process from step=1 to step=step. m The iterative process is denoted as one training round; Iterative training is performed for one training round based on deep reinforcement learning algorithms; Z5, given the training round number m and the maximum number of training rounds M, where m=1,2,...,M; repeat Z4, and randomly set the grid short-circuit ratio SCR condition and the current loop dq axis step command during the training process. When the training process of M rounds is completed, the iterative training based on the deep reinforcement learning algorithm ends, and the trained Actor network is obtained. Z6, extract the learnable parameter matrix from the trained Actor network, and determine the learnable parameter matrix as the target coefficient matrix. K B .

[0014] Preferably, the implementation process of Z4 is as follows: Z4.1, in the current state s k The Actor network then adjusts its operation based on the current state. s k The current action is output along with the current learnable parameter matrix. a k , will the current action ak Applying this to the grid-connected inverter training environment, we obtain the next state. s k+1 and current rewards r k Generate quadruple samples ( s k , a k , s k+1 , r k ), and store it in the experience replay pool. P ; Z4.2, from the experience replay pool P A small batch of quadruple samples are randomly selected from the data. The target action in the next state is calculated through the target Actor network, and the pruned target policy smoothing noise is added to the target action. Z4.3 inputs the next state and the target action in the next state into the first target Critic network and the second target Critic network to obtain the first action value function corresponding to the target action in the next state and the target action in the next state. Q' 1( s' , a' ), second action value function Q' 2( s' , a' Compare the values ​​of the two action value functions and take the smaller value as the target value. y : y = r + min{ Q' 1( s' , a' ), Q' 2( s' , a' )} in, s' For the next state, a' The target action for the next state. Discount factor; Z4.4, with target value y As a monitoring signal, loss functions for the first Critic network and the second Critic network are constructed respectively, and the parameters of the first Critic network and the second Critic network are updated by minimizing the loss function. Z4.5 updates the Actor network parameters every two Critic network updates. The optimization objective of the Actor network is to maximize the action value function output by the first Critic network. Z4.6 uses a soft update method to update the parameters of the target Actor network, the first target Critic network, and the second target Critic network. Repeat steps Z4.1-Z4.6 until the maximum training step size is reached. m This completes one round of iterative training.

[0015] Preferably, the target coefficient matrix Z6 K B The expression is:

[0016] in, , , , These are the d-axis main channel controllers. G dd (z), dq cross-coupled controller G dq (z), qd cross-coupled controller G qd (z) and q-axis main channel controller G qq The optimal coefficient matrix of (z) is expressed as follows:

[0017] In the formula, n is the order of the higher-order discrete controller, and n is an integer not less than 3. j For delay order, j= 1,2,…,n; , , , The d-axis main channel controllers are respectively G dd (z), dq cross-coupled controller G dq (z), qd cross-coupled controller G qd (z) and q-axis main channel controller G qq The optimal current error term coefficient of (z); , , , The d-axis main channel controllers are respectivelyG dd (z), dq cross-coupled controller G dq (z), qd cross-coupled controller G qd (z) and q-axis main channel controller G qq (z) j Forward coefficient of the optimal delay error; , , , The d-axis main channel controllers are respectively G dd (z), dq cross-coupled controller G dq (z), qd cross-coupled controller G qd (z) and q-axis main channel controller G qq (z) j The optimal output feedback coefficient.

[0018] Compared with the prior art, the beneficial effects of the present invention include: 1. This invention employs a reconfigurable high-order discrete control structure to control the grid-connected inverter, significantly increasing the controller parameter degrees of freedom. The higher control degrees of freedom enable the controller to achieve frequency domain shaping over a wider frequency range, thereby improving the system's adaptability to model uncertainties, grid impedance changes, and weak grid conditions.

[0019] 2. This invention trains and optimizes the coefficient matrix of a reconfigurable high-order discrete control structure using a deep reinforcement learning algorithm, enabling the controller to achieve adaptive parameter tuning for the dynamic characteristics of weak power grids. Simulation results show that when the short-circuit ratio (SCR) is 2.5, the system settling time is reduced from 0.045s to 0.0068s; when the SCR is 3, the system settling time is reduced from 0.0385s to 0.0074s, effectively improving the current tracking speed and dynamic stability under weak power grid conditions.

[0020] 3. This invention maintains good dynamic and steady-state performance under different short-circuit ratio conditions, including weak, medium-strength, and strong power grids. Simulation results show that when SCR=2.5, the overshoot of the control method of this invention is only 1.3%, while the overshoot of the traditional PI control method is 48.6%. Simultaneously, the total harmonic distortion (THD) of the current controlled by this invention is 2.66%, lower than the 3.02% of the traditional PI control method. When SCR=5, the THD of the control method of this invention is 0.45%, a reduction of approximately 61.4% compared to the 1.14% of the traditional PI control method. When SCR=20, the THD of the control method of this invention is 0.58%, still better than the 0.72% of the traditional PI control method. This demonstrates that this invention can achieve improved dynamic performance and steady-state power quality across the entire operating range from weak to strong power grids.

[0021] 4. This invention utilizes a deep reinforcement learning algorithm to train the controller in simulation environments with various grid short-circuit ratio (SCR) conditions and random variations in the current loop dq-axis step command. By guiding the continuous optimization of the controller coefficient matrix through a reward function, the reconfigurable high-order discrete control structure can obtain control parameters based on changes in grid strength and operating state. This reduces the problems of traditional controller parameters relying on manual tuning and specific operating condition design, and improves the control robustness and engineering applicability of grid-connected inverters under complex grid conditions. Attached Figure Description

[0022] Figure 1 This is a block diagram of the main circuit and control of the grid-connected inverter in an embodiment of the present invention.

[0023] Figure 2 This is a block diagram of the reconfigurable high-order discrete control structure of the present invention.

[0024] Figure 3 This is a flowchart of the present invention.

[0025] Figure 4 This is a flowchart of the training process for the coefficient matrix of a reconfigurable high-order discrete controller for a grid-connected inverter based on the TD3 algorithm in an embodiment of the present invention.

[0026] Figure 5 Comparison of the dynamic response of the inverter-side inductor current d-axis component between the traditional PI control method and the control method of this invention when the SCR is 2.5 and 3 respectively.

[0027] Figure 6 The diagram shows a comparison of the dynamic response of the inverter-side inductor current d-axis component between the traditional PI control method and the control method of this invention when the SCR is 5 and 20, respectively.

[0028] Figure 7The graph shows a comparison of the A-phase output current spectrum and total harmonic distortion rate between the traditional PI control method and the control method of this invention when the SCR is 2.5 and 3, respectively.

[0029] Figure 8 The graph shows a comparison of the A-phase output current spectrum and total harmonic distortion rate between the traditional PI control method and the control method of this invention when the SCR is 5 and 20, respectively. Detailed Implementation

[0030] The present invention will be further described in detail below with reference to the accompanying drawings and embodiments.

[0031] Figure 1 This is a block diagram of the main circuit and control of the grid-connected inverter in an embodiment of the present invention. Figure 1 As can be seen, the grid-connected inverter includes a DC power supply, a three-phase bridge inverter circuit, a three-phase LCL filter, a grid impedance, and a three-phase grid connected in sequence.

[0032] exist Figure 1 middle, v DC This is the DC voltage at the DC power supply. L 1. C 1 and L 2 represents the inverter-side inductance, filter capacitor, and grid-side filter inductance values ​​of the three-phase LCL filter, respectively. Z g This represents the impedance value of the three-phase power grid. E g This is the voltage of the three-phase power grid.

[0033] The control method involves a reconfigurable high-order discrete control structure, denoted as a controller, which includes a d-axis master channel controller. G dd (z), dq cross-coupled controller G dq( z), qd cross-coupled controller G qd (z) and q-axis main channel controller G qq (z).

[0034] Figure 2 This is a block diagram of the reconfigurable high-order discrete control structure of the present invention. Figure 2 Chinese z -1 The delay operator is a unit. Figure 3 This is a flowchart of the present invention, by Figure 2 and Figure 3 As can be seen, the control method includes an offline training phase and an online control phase; During the offline training phase, in a training environment with various grid short-circuit ratio (SCR) operating conditions and random variations in the current loop dq-axis step command, a deep reinforcement learning algorithm is used for iterative training to obtain the target coefficient matrix of the controller. K B ; Before the start of the online control phase, the target coefficient matrix is... K B Configured to the controller; during each control cycle of the online control phase, using the target coefficient matrix configured... K B The controller performs a reconfigurable high-order discrete control on the grid-connected inverter. The steps for any control cycle are as follows: Step 1: Collect the inductor current on the inverter side. i L1_abc and LCL filter capacitor voltage v C1_abc The Park transformation is then performed to obtain the inductor current on the dq axis inverter side. i L1_d , i L1_q and dq axis LCL filter capacitor voltage v C1_d ,v C1_q .

[0035] Step 2, provide the reference value for the dq-axis current of the current loop. i d_ref , i q_ref The dq-axis current error was calculated. e d , e q .

[0036] In this embodiment, the dq-axis current error e d , e q The expression is: e d = i d_ref - i L1_d e q = i q_ref - i L1_q .

[0037] Step 3, adjust the dq axis current error ed , e q Input controller, output total voltage control quantity for d and q axes u d , u q .

[0038] In this embodiment, step 3 is implemented as follows: S1, dq axis current error e d , e q The input controller specifically includes: d-axis current error e d Total d-axis voltage control quantity in the previous control cycle u d Input d-axis main channel controller G dd (z): d-axis current error e d Total d-axis voltage control quantity in the previous control cycle u d 'Input dq cross-coupled controller G dq (z); q-axis current error e q Total q-axis voltage control quantity in the previous control cycle u q 'Input qd cross-coupled controller G qd (z); q-axis current error e q Total q-axis voltage control quantity in the previous control cycle u q 'Input q-axis main channel controller G qq (z); S2, the outputs of the four high-order discrete controllers are as follows: d-axis main channel controller G dd (z) Output d-axis main channel voltage component u dd ; dq cross-coupled controller G dq (z) Output d-axis error to q-axis voltage cross-compensation component u dq ; qd cross-coupled controllerG qd (z) Output q-axis error to d-axis voltage cross-compensation component u qd ; q-axis main channel controller G qq (z) Output q-axis main channel voltage component u qq ; S3, the total voltage control quantity of the dq axis u d ,u q The expression is: u d = u dd + u qd u q = u dq + u qq .

[0039] Step 4, control the total voltage of the dq axis. u d , u q Given the dq-axis gain K d , K q and dq axis LCL filter capacitor voltage v C1_d , v C1_q Obtain the dq axis output voltage v inv_d , v inv_q The three-phase modulated signal is then obtained through inverse Park transform. v inv_abc The PWM wave generated by space vector modulation (SVPWM) is used for the switching control of power devices in a three-phase bridge inverter circuit.

[0040] In this embodiment, the dq axis output voltage v inv_d , v inv_q The expressions are as follows: v inv_d = u d K d +v C1_d v inv_q = u q K q + v C1_q .

[0041] After completing the reconfigurable high-order discrete control for the current control cycle, return to step 1 and proceed to the next control cycle.

[0042] Figure 4 This is a flowchart illustrating the training process of the reconfigurable high-order discrete controller coefficient matrix for a grid-connected inverter based on the TD3 algorithm in this embodiment of the invention. Figure 4 As can be seen, in this embodiment, the offline training phase includes the following steps: Z1, introducing a state space S Action space A Reward function R ; Let a total of T Each discrete sample is recorded as the current sample. k , k =1,2,..., T ; The state space S Defined as all states s The set, which will include the current samples k The state at that time is recorded as the current state. s k , s k = ( e d (k), e q (k), u d (k), u q (k)); The action space A Defined as all actions a The set that contains the current state s k The action at that time is recorded as the current action. a k , a k =( u d (k), u q (k)); The reward functionR Defined as including all rewards r The set that contains the current state s k The corresponding reward is recorded as the current reward. r k , r k =-( e d 2 (k)+ e q 2 (k)); u d (k) , u q (k) is related to the current state s k The corresponding total voltage control quantity of the dq axis e d (k), e q (k) represents the current state. s k The corresponding dq axis current error.

[0043] Z2 constructs a neural network based on the dual-delay deep deterministic policy gradient (TD3) algorithm. The neural network includes an Actor network, a first Critic network, a second Critic network, a target Actor network, a first target Critic network, and a second target Critic network. The Actor network includes basis functions and a learnable parameter matrix. The basis functions are defined by states... s For input, and input state s The learnable parameter matrix of the Actor network is directly output as the state feature; the learnable parameter matrix of the Actor network is denoted as the coefficient matrix. K The Actor network is based on the state s sum coefficient matrix K Output Action a , a =π( s ; K Both the first and second Critic networks have input states. s ,action a The first Critic network outputs the first action value function. Q 1( s , a The second Critic network outputs the second action value function. Q 2( s , a ).

[0044] Z3 initializes all neural network parameters and initializes the experience replay pool. P .

[0045] Z4, given the training step size step and the maximum training step size step. m Where step=1,2,...,step m And will complete the process from step=1 to step=step. m The iterative process is denoted as one training round; The specific process of iterative training for one training round based on deep reinforcement learning algorithms is as follows: Z4.1, in the current state s k The Actor network then adjusts its operation based on the current state. s k The current action is output along with the current learnable parameter matrix. a k , will the current action a k Applying this to the grid-connected inverter training environment, we obtain the next state. s k+1 and current rewards r k Generate quadruple samples ( s k , a k , s k+1 , r k ), and store it in the experience replay pool. P ; Z4.2, from the experience replay pool P A small batch of quadruple samples are randomly selected from the data. The target action in the next state is calculated through the target Actor network, and the pruned target policy smoothing noise is added to the target action. Z4.3 inputs the next state and the target action in the next state into the first target Critic network and the second target Critic network to obtain the first action value function corresponding to the target action in the next state and the target action in the next state. Q' 1( s' , a' ), second action value function Q' 2( s' , a' Compare the values ​​of the two action value functions and take the smaller value as the target value. y : y = r + min{ Q' 1( s' , a' ), Q' 2( s' , a' )} in, s' For the next state, a' The target action for the next state. Discount factor; Z4.4, with target value y As a monitoring signal, loss functions for the first Critic network and the second Critic network are constructed respectively, and the parameters of the first Critic network and the second Critic network are updated by minimizing the loss function. Z4.5 updates the Actor network parameters every two Critic network updates. The optimization objective of the Actor network is to maximize the action value function output by the first Critic network. Z4.6 uses a soft update method to update the parameters of the target Actor network, the first target Critic network, and the second target Critic network. Repeat steps Z4.1-Z4.6 until the maximum training step size is reached. m This completes one round of iterative training.

[0046] Z5, given the training round number m and the maximum number of training rounds M, where m=1,2,...,M; Z4 is executed repeatedly, and the grid short-circuit ratio SCR condition and the current loop dq axis step command are randomly set during the training process. When the training process of M rounds is completed, the iterative training based on the deep reinforcement learning algorithm ends, and the trained Actor network is obtained.

[0047] Z6, extract the learnable parameter matrix from the trained Actor network, and determine the learnable parameter matrix as the target coefficient matrix. K B .

[0048] In this embodiment, the target coefficient matrix K B The expression is:

[0049] in, , , , These are the d-axis main channel controllers. G dd (z), dq cross-coupled controller Gdq (z), qd cross-coupled controller G qd (z) and q-axis main channel controller G qq The optimal coefficient matrix of (z) is expressed as follows:

[0050] In the formula, n is the order of the higher-order discrete controller, and n is an integer not less than 3. j For delay order, j= 1,2,…,n; , , , The d-axis main channel controllers are respectively G dd (z), dq cross-coupled controller G dq (z), qd cross-coupled controller G qd (z) and q-axis main channel controller G qq The optimal current error term coefficient of (z); , , , The d-axis main channel controllers are respectively G dd (z), dq cross-coupled controller G dq (z), qd cross-coupled controller G qd (z) and q-axis main channel controller G qq (z) j Forward coefficient of the optimal delay error; , , , The d-axis main channel controllers are respectively G dd (z), dq cross-coupled controller G dq (z), qd cross-coupled controller G qd (z) and q-axis main channel controllerG qq (z) j The optimal output feedback coefficient.

[0051] In this embodiment, the controller order n=3, and the maximum training step size is step. m The maximum number of training rounds M is 800, with a maximum of 2500.

[0052] To demonstrate the beneficial effects of the present invention, the control method of the present invention was simulated.

[0053] In the simulation, , , , The values ​​are 0.8507114, 0.2918666, 0.2588140, and 0.9914538, respectively. , , The values ​​are -0.2321669, -0.1495768, and -0.0561940, respectively. , , The values ​​are 0.008397, -0.0033254, and 0.4923337, respectively. , , The values ​​are -0.0886634, -0.2129131, and -0.1249221, respectively. , , The values ​​are 0.0137657, 0.0823357, and 0.2280503, respectively. , , The values ​​are -0.1984036, -0.2379431, and -0.0424787, respectively. , , The values ​​are -0.1614333, 0.0687921, and 0.5535021, respectively. , , The values ​​are -0.3185892, -0.2522632, and 0.1171664, respectively. , , The values ​​are 0.0005901, -0.4151196, and 0.5778188, respectively.

[0054] Figure 5This is a comparison of the dynamic response of the inverter-side inductor current d-axis component between the traditional PI control method and the control method of this invention when the SCR is 2.5 and 3, respectively. The horizontal axis represents time, and the vertical axis represents the d-axis component of the inverter-side inductor current. Figure 5 As can be seen, when the d-axis command value of the inverter-side inductor current jumps from 0.2 pu to 0.8 pu in 0.1s, the control method of the present invention shortens the settling time from 0.045s under the traditional PI control method to 0.0068s when SCR=2.5, and the overshoot decreases from 48.6% to 1.3%. When SCR=3, the settling time shortens from 0.0385s under the traditional PI control method to 0.0074s, and the overshoot decreases from 56.1% to 13.8%. That is, the control method of the present invention has a faster dynamic response capability in a weak grid environment.

[0055] Figure 6 This is a comparison of the dynamic response of the inverter-side inductor current d-axis component between the traditional PI control method and the control method of this invention when the SCR is 5 and 20 respectively. The horizontal axis represents time, and the vertical axis represents the inverter-side inductor current d-axis component. Figure 6 As can be seen, when the d-axis command value of the inverter-side inductor current jumps from 0.2 pu to 0.8 pu in 0.1s, the control method of the present invention shortens the settling time from 0.0186s under the traditional PI control method to 0.0053s when SCR=5, and the overshoot decreases from 48.6% to 25.8%. When SCR=20, the settling time shortens from 0.0061s under the traditional PI control method to 0.0047s, and the overshoot decreases from 33.9% to 27.1%. That is, the control method of the present invention has a faster dynamic response capability in a strong grid environment.

[0056] Figure 7 This chart compares the A-phase output current spectrum and total harmonic distortion (THD) of the traditional PI control method and the control method of this invention when the SCR is 2.5 and 3, respectively. The horizontal axis represents frequency, and the vertical axis represents the amplitude of each harmonic relative to the fundamental frequency. Figure 7 (a) is a comparison of the A-phase output current spectrum and total harmonic distortion rate between the traditional PI control method and the control method of the present invention when the SCR is 2.5. Figure 7 (b) is a comparison of the A-phase output current spectrum and total harmonic distortion rate between the traditional PI control method and the control method of this invention when the SCR is 3. Figure 7It can be seen that, under the condition of SCR=2.5, the THD of the A-phase output current under the traditional PI control method is 3.02%, while the THD of the A-phase output current under the control method of this invention is 2.66%; under the condition of SCR=3, the THD of the A-phase output current under the traditional PI control method is 2.82%, while the THD of the A-phase output current under the control method of this invention is 2.32%. Therefore, in a weak grid environment, the method of this invention can effectively reduce the harmonic content of the output current, making the output current spectrum distribution more concentrated on the fundamental component, reducing the harmonic amplitude, thereby improving the grid-connected current quality and system operational stability.

[0057] Figure 8 This is a comparison of the A-phase output current spectrum and total harmonic distortion (THD) of the traditional PI control method and the control method of this invention when the SCR is 5 and 20, respectively. The horizontal axis represents frequency, and the vertical axis represents the amplitude of each harmonic relative to the fundamental frequency. Figure 8 (a) is a comparison of the A-phase output current spectrum and total harmonic distortion rate between the traditional PI control method and the control method of the present invention when the SCR is 5. Figure 8 (b) is a comparison chart of the A-phase output current spectrum and total harmonic distortion rate between the traditional PI control method and the control method of this invention when the SCR is 20. Figure 8 It can be seen that, under SCR=5, the THD of the A-phase output current under the traditional PI control method is 1.14%, while the THD of the A-phase output current under the control method of this invention is 0.45%; under SCR=20, the THD of the A-phase output current under the traditional PI control method is 0.72%, while the THD of the A-phase output current under the control method of this invention is 0.58%. Therefore, it is evident that, even in a strong grid environment, the method of this invention can still reduce the harmonic content of the output current, making the output current spectrum distribution more concentrated on the fundamental component and reducing non-fundamental frequency components, thereby improving the grid-connected current quality and system operational stability.

Claims

1. A reconfigurable high-order discrete control method for a grid-connected inverter, wherein the grid-connected inverter comprises, in sequence, a DC power supply, a three-phase bridge inverter circuit, a three-phase LCL filter, a grid impedance, and a three-phase grid; characterized in that, The control method involves a reconfigurable high-order discrete control structure, denoted as a controller, which includes a d-axis master channel controller. G dd (z), dq cross-coupled controller G dq( z), qd cross-coupled controller G qd (z) and q-axis main channel controller G qq (z); The control method includes an offline training phase and an online control phase; During the offline training phase, in a training environment with various grid short-circuit ratio (SCR) operating conditions and random variations in the current loop dq-axis step command, a deep reinforcement learning algorithm is used for iterative training to obtain the target coefficient matrix of the controller. K B ; Before the start of the online control phase, the target coefficient matrix is... K B Configured to the controller; during each control cycle of the online control phase, using the target coefficient matrix configured... K B The controller performs a reconfigurable high-order discrete control on the grid-connected inverter. The steps for any control cycle are as follows: Step 1: Collect the inductor current on the inverter side. i L1_abc and LCL filter capacitor voltage v C1_abc The Park transformation is then performed to obtain the inductor current on the dq axis inverter side. i L1_d , i L1_q and dq axis LCL filter capacitor voltage v C1_d ,v C1_q ; Step 2, provide the reference value for the dq-axis current of the current loop. i d_ref , i q_ref The dq-axis current error was calculated. e d , e q ; Step 3, adjust the dq axis current error e d , e q Input controller, output total voltage control quantity for d and q axes u d , u q ; Step 4, control the total voltage of the dq axis. u d , u q Given dq-axis gain K d , K q and dq axis LCL filter capacitor voltage v C1_d , v C1_q Obtain the dq axis output voltage v inv_d , v inv_q The three-phase modulated signal is then obtained through inverse Park transform. v inv_abc The PWM wave is generated by space vector modulation (SVPWM) and used for switching control of power devices in a three-phase bridge inverter circuit. After completing the reconfigurable high-order discrete control for the current control cycle, return to step 1 and proceed to the next control cycle.

2. The reconfigurable high-order discrete control method for a grid-connected inverter according to claim 1, characterized in that, Step 2 describes the dq-axis current error. e d , e q The expression is: e d = i d_ref - i L1_d e q = i q_ref - i L1_q 。 3. The reconfigurable high-order discrete control method for a grid-connected inverter according to claim 1, characterized in that, The process of step 3 is as follows: S1, dq axis current error e d , e q The input controller specifically includes: d-axis current error e d Total d-axis voltage control quantity in the previous control cycle u d 'Input d-axis main channel controller G dd (z); d-axis current error e d Total d-axis voltage control quantity in the previous control cycle u d 'Input dq cross-coupled controller G dq (z); q-axis current error e q Total q-axis voltage control quantity in the previous control cycle u q 'Input qd cross-coupled controller G qd (z); q-axis current error e q Total q-axis voltage control quantity in the previous control cycle u q 'Input q-axis main channel controller G qq (z); S2, the outputs of the four high-order discrete controllers are as follows: d-axis main channel controller G dd (z) Output d-axis main channel voltage component u dd ; dq cross-coupled controller G dq (z) Output d-axis error to q-axis voltage cross-compensation component u dq ; qd cross-coupled controller G qd (z) Output q-axis error to d-axis voltage cross-compensation component u qd ; q-axis main channel controller G qq (z) Output q-axis main channel voltage component u qq ; S3, the total voltage control quantity of the dq axis u d ,u q The expression is: u d = u dd + u qd u q = u dq + u qq 。 4. The reconfigurable high-order discrete control method for a grid-connected inverter according to claim 1, characterized in that, Step 4 describes the dq axis output voltage. v inv_d , v inv_q The expressions are as follows: v inv_d = u d K d + v C1_d v inv_q = u q K q + v C1_q 。 5. The reconfigurable high-order discrete control method for a grid-connected inverter according to claim 1, characterized in that, The offline training phase includes the following steps: Z1, introducing a state space S Action space A Reward function R ; Let a total of T Each discrete sample is recorded as the current sample. k , k =1,2,..., T ; The state space S Defined as all states s The set, which will include the current sample k The state at that time is recorded as the current state. s k , s k = ( e d (k), e q (k), u d (k), u q (k)); The action space A Defined as all actions a The set that contains the current state s k The action at that time is recorded as the current action. a k , a k =( u d (k), u q (k)); The reward function R Defined as including all rewards r The set that contains the current state s k The corresponding reward is recorded as the current reward. r k , r k =-( e d 2 (k)+ e q 2 (k)); u d (k) , u q (k) is related to the current state s k The corresponding total voltage control quantity of the dq axis. e d (k), e q (k) represents the current state. s k The corresponding dq axis current error; Z2 constructs a neural network based on the dual-delay deep deterministic policy gradient (TD3) algorithm. The neural network includes an Actor network, a first Critic network, a second Critic network, a target Actor network, a first target Critic network, and a second target Critic network. The Actor network includes basis functions and a learnable parameter matrix. The basis functions are defined by states... s For input, and input state s It is directly output as a state feature; The learnable parameter matrix of the Actor network is denoted as the coefficient matrix. K The Actor network is based on the state s sum coefficient matrix K Output Action a , a =π( s ; K Both the first and second Critic networks have input states. s ,action a ; The first Critic network outputs the first action value function. Q 1( s , a The second Critic network outputs the second action value function. Q 2( s , a ); Z3 initializes all neural network parameters and initializes the experience replay pool. P ; Z4, given the training step size step and the maximum training step size step. m Where step=1,2,...,step m And will complete the process from step=1 to step=step. m The iterative process is denoted as one training round; Iterative training is performed for one training round based on deep reinforcement learning algorithms; Z5, given the training round number m and the maximum number of training rounds M, where m=1,2,...,M; repeat Z4, and randomly set the grid short-circuit ratio SCR condition and the current loop dq axis step command during the training process. When the training process of M rounds is completed, the iterative training based on the deep reinforcement learning algorithm ends, and the trained Actor network is obtained. Z6, extract the learnable parameter matrix from the trained Actor network, and determine the learnable parameter matrix as the target coefficient matrix. K B .

6. The reconfigurable high-order discrete control method for a grid-connected inverter according to claim 5, characterized in that, The implementation process of Z4 is as follows: Z4.1, in the current state s k The Actor network then adjusts its operation based on the current state. s k The current action is output along with the current learnable parameter matrix. a k , will the current action a k Applying this to the grid-connected inverter training environment, we obtain the next state. s k+1 and current rewards r k Generate quadruple samples ( s k , a k , s k+1 , r k ), and store it in the experience replay pool. P ; Z4.2, from the experience replay pool P A small batch of quadruple samples are randomly selected from the data. The target action in the next state is calculated through the target Actor network, and the pruned target policy smoothing noise is added to the target action. Z4.3 inputs the next state and the target action in the next state into the first target Critic network and the second target Critic network to obtain the first action value function corresponding to the target action in the next state and the target action in the next state. Q' 1( s' , a' ), second action value function Q' 2( s' , a' Compare the values ​​of the two action value functions and take the smaller value as the target value. y : y = r + min{ Q' 1( s' , a' ), Q' 2( s' , a' )} in, s' For the next state, a' The target action for the next state. Discount factor; Z4.4, with target value y As a monitoring signal, loss functions for the first Critic network and the second Critic network are constructed respectively, and the parameters of the first Critic network and the second Critic network are updated by minimizing the loss function. Z4.5 updates the Actor network parameters every two Critic network updates. The optimization objective of the Actor network is to maximize the action value function output by the first Critic network. Z4.6 uses a soft update method to update the parameters of the target Actor network, the first target Critic network, and the second target Critic network. Repeat steps Z4.1-Z4.6 until the maximum training step size is reached. m This completes one round of iterative training.

7. The reconfigurable high-order discrete control method for a grid-connected inverter according to claim 5, characterized in that, Z6 Target coefficient matrix K B The expression is: in, , , , These are the d-axis main channel controllers. G dd (z), dq cross-coupled controller G dq (z), qd cross-coupled controller G qd (z) and q-axis main channel controller G qq The optimal coefficient matrix of (z) is expressed as follows: In the formula, n is the order of the higher-order discrete controller, and n is an integer not less than 3. j For delay order, j= 1,2,…,n; , , , The d-axis main channel controllers are respectively G dd (z), dq cross-coupled controller G dq (z), qd cross-coupled controller G qd (z) and q-axis main channel controller G qq The optimal current error term coefficient of (z); , , , The d-axis main channel controllers are respectively G dd (z), dq cross-coupled controller G dq (z), qd cross-coupled controller G qd (z) and q-axis main channel controller G qq (z) j The forward coefficient of the optimal delay error; , , , The d-axis main channel controllers are respectively G dd (z), dq cross-coupled controller G dq (z), qd cross-coupled controller G qd (z) and q-axis main channel controller G qq (z) j The optimal output feedback coefficient.

Images