Control method of novel factory power supply system main converter containing new energy power generation

Abstract

The invention discloses a control method of a novel factory power supply system main converter containing new energy power generation, and belongs to the technical field of new energy power generation. The main current transformer comprises a T-type three-level current transformer and a Buck-Boost current transformer; designing a T-type three-level converter controller by adopting a deep reinforcement learning TD3 algorithm: constructing an intelligent agent based on the TD3 algorithm according to the environment state of the main converter, designing a reward function, and training the intelligent agent; transplanting the trained agent model into a main control chip of the main converter, establishing an actual control system based on the main control chip by adopting a carrier pulse width modulation strategy, and controlling the main converter to work; and designing a Buck-Boost converter controller to realize automatic balance of the neutral-point potential. The method is high in adaptive capacity and robustness, can effectively cope with the influence of illumination, wind speed and power grid voltage fluctuation and uncertainty on system control, and achieves the quick and stable control of the main converter of the novel factory power supply system containing new energy power generation.

Description

Technical Field

[0001] The present invention relates to a control method, in particular to a control method for a main converter of a new type of factory power supply system containing renewable energy power generation, belonging to the technical field of renewable energy power generation. Background Art

[0002] In order to achieve the "dual carbon" goal, more and more companies have built distributed photovoltaic power stations and / or distributed wind farms, among which the main converter of the new energy power generation system connected to the original factory power supply system is crucial. On the one hand, it converts the DC power generated by the photovoltaic component system and / or wind turbine system and the power of the energy storage equipment into AC power to supply power to the enterprise load. On the other hand, it can rectify the city power into DC power at night to charge the energy storage equipment and realize peak shaving and valley filling. To this end, the main converter must have the function of bidirectional power transmission, and at the same time, it must have the functions of small harmonics on the AC side, low power consumption, and effective response to the influence of light, wind speed fluctuations and grid voltage fluctuations to meet the requirements of the new factory power supply system.

[0003] T-type three-level converters have been used in charging piles, energy storage systems, etc. due to their advantages such as small harmonics, low voltage stress of switching devices, low cost, and the ability to achieve bidirectional energy transmission.

[0004] However, the topology of the existing T-type three-level converter has the inherent defect of unbalanced midpoint potential, which increases the complexity of the existing control methods. At present, the commonly used control methods for T-type three-level converters are mainly PID control, model predictive control (MPC), and finite control set model predictive control (FCS-MPC). Due to the fixed parameters of PI or PID control, it is difficult to achieve the optimal control effect under different working conditions. MPC achieves the optimal performance of the converter by designing an appropriate cost function, but MPC selects the action of the bidirectional switch tube at the next moment by comparing the cost size of different switch states. Therefore, MPC needs to have a modulation function, which requires MPC to be implemented by hardware with a high computing speed. FCS-MPC has the advantages of fast dynamic response speed, simple and easy to implement control algorithm, no modulation link, and easy to achieve multi-objective and constraint synchronization processing. It is increasingly used in converter systems, but there are problems such as large algorithm calculation volume, cumbersome weight factor design, low control accuracy at low frequency, and non-fixed switching frequency.

[0005] In order to solve the inherent midpoint potential imbalance problem, existing methods include PI control method, zero-sequence component injection method, finite control set model predictive control method (FCS-MPC), finite time control method (FTC), etc. However, these methods have defects such as complex calculations and long time consumption, so their rapid response ability is poor, and they cannot effectively deal with interference to the stable operation of the system caused by light, wind speed fluctuations (and thus power fluctuations), grid voltage fluctuations, etc.

[0006] The deep reinforcement learning control algorithm effectively combines the perception ability of deep learning and the decision-making ability of reinforcement learning. It can control according to the input information and is an artificial intelligence control method that is closer to human thinking. The reward function of deep reinforcement learning (DRL) is similar to the cost function of MPC, so it has the same advantages as MPC. However, DRL control is equivalent to a special PID control whose parameters can change with the external environment and does not require modulation, so its computational cost is much lower than MPC. In addition, the control effect of MPC depends on accurate mathematical models and device parameters, while the optimal action of DRL control is obtained through exploration and optimization in a random and uncertain environment. Therefore, DRL control is highly robust to both internal parameter misalignment and external environmental disturbances. As one of the most advanced training algorithms for DRL, double-delayed deep deterministic policy gradient (TD3) has been used for power electronics and motor control. The results show that TD3 control also has good control effects in uncertain environments, so it is particularly suitable for unknown time-varying disturbances (such as photovoltaic power generation and wind power generation systems with fluctuations and uncertainties in light, wind speed and wind direction). However, there is currently little research on the application of deep reinforcement learning control in photovoltaic power generation and wind power generation systems, especially in new factory power supply systems containing renewable energy power generation. Summary of the invention

[0007] The main purpose of the present invention is to: in view of the deficiencies in the above-mentioned prior art, the present invention provides a control method for the main inverter of a new factory power supply system containing renewable energy power generation, so as to achieve optimal control of multiple objectives such as effectively responding to light / wind speed and direction and grid voltage fluctuations, reducing AC side current harmonics, reducing the inverter's own losses, and automatically achieving midpoint potential balance, thereby ensuring effective control and efficient operation of the main inverter of the new factory power supply system containing renewable energy power generation.

[0008] In order to achieve the above purpose, the main converter of the present invention includes: a T-type three-level converter and a Buck-Boost DC converter; one end of the T-type three-level converter is connected to the power grid, and the other end is connected to the Buck-Boost DC converter; the other end of the Buck-Boost DC converter is connected to a DC bus; the DC bus is connected to a photovoltaic component system and a wind turbine system; the DC bus is also connected to an energy storage device.

[0009] The T-type three-level converter includes three LCL filters, three bridge arms, and two output capacitors; the bridge arm has four power switch tubes: S x1 ~S x4, x=a, b, c; one end of the LCL filter is connected to the power grid; the two output capacitors are connected in series, wherein the output capacitor connected to the positive pole of the DC side of the T-type three-level converter is called an upper capacitor, and the output capacitor connected to the negative pole of the DC side of the T-type three-level converter is called a lower capacitor, and the connection point of the two output capacitors is called the midpoint O of the T-type three-level converter; the Buck-Boost DC converter includes a DC inductor and two power switching tubes, and one end of the DC inductor is connected to the midpoint O of the T-type three-level converter.

[0010] The present invention provides a control method for a main converter of a new type of factory power supply system including renewable energy power generation, comprising the following steps:

[0011] Step 1, using a deep reinforcement learning TD3 algorithm to design a controller of the T-type three-level converter;

[0012] Step 2, adopting a carrier pulse width modulation (CBPWM) strategy, establishing an actual current tracking and DC side voltage control system based on the main control chip of the T-type three-level converter, and realizing AC side current tracking and DC side voltage stabilization control of the T-type three-level converter;

[0013] Step 3, designing a controller of the Buck-Boost DC converter, establishing an actual control system of the Buck-Boost DC converter, and realizing automatic balance control of the midpoint potential.

[0014] In step 1, the specific steps of designing the controller of the T-type three-level converter are as follows:

[0015] 11) Obtain the three-phase grid voltage e through the phase-locked loop PLL a 、e b 、e c The phase angle θ is determined, and the phase information is provided to the Park transformation and iPark transformation of the system to ensure that the three-phase modulation wave of the T-type three-level converter is synchronized with the grid voltage.

[0016] 12) The points after the three-phase AC power passes through the three LCL filters of the T-type three-level converter are respectively denoted as Ta, Tb, and Tc, and the voltages at Ta, Tb, and Tc are respectively denoted as u a 、u b 、u c ; The current flowing through points Ta, Tb, and Tc are respectively denoted as i a 、i b 、i c ; will u a 、u b 、u c After abc / dq transformation, we get u d and uq ;change i a 、i b 、i c After abc / dq transformation, we get i d and i q .

[0017] 13) Determine the environmental state S of the T-type three-level converter t is: including the DC side voltage U of the T-type three-level converter dc and its reference value U dc * ; The dq-axis component i of the AC side current of the T-type three-level converter d 、i q and its reference value i d * 、i q * ; The dq-axis component u of the AC side voltage of the T-type three-level converter d 、u q ; The output current I of the photovoltaic module system PV and its rate of change The output current I of the wind turbine system W and its rate of change The environmental state is denoted as S t , then

[0018] 14) The environmental state S of the T-type three-level converter determined in step 13) is t As the input of the agent, build an agent based on the TD3 algorithm:

[0019] The agent is composed of a strategy network and a strategy evaluation network, which contains 6 deep neural networks; the strategy network has 2 deep neural networks: composed of an Actor network and an Actor Target network, which respectively pass the corresponding network strategy π(S t |θ') and π ta (S t+1 |θ' ta ) realizes the action output A of the agent t (S t |θ'); The strategy evaluation network has four deep neural networks: two groups of Critic networks and Critic Target networks, which are respectively and k = 1, 2, evaluate and improve the strategy used to achieve the action output, where θ', θ' ta , They represent the weight coefficients of the Actor network, Actor Target network, Critic network, and Critic Target network respectively.

[0020] 15) Design reward function R t ;

[0021] The reward function R t Designed for:

[0022] R t =R1+R2 (1)

[0023] Wherein, R1 is the AC side current distortion rate and DC side voltage tracking reward function of the T-type three-level converter, and R2 is the AC side current voltage and DC side voltage of the T-type three-level converter, as well as the fluctuation suppression reward function of the output current of the photovoltaic module system and the wind turbine system;

[0024] R1 is defined as:

[0025]

[0026] Wherein, w1 is the AC side current tracking weight coefficient of the T-type three-level converter, which is used to quickly reduce the current tracking error; w2 is the DC side voltage tracking weight coefficient of the T-type three-level converter, which is used to quickly reduce the DC side voltage tracking error;

[0027] R2 is defined as:

[0028]

[0029] Wherein, w3 is the AC side current stability weight coefficient of the T-type three-level converter, which is used to reduce the AC side current fluctuation; w4 is the AC side voltage stability weight coefficient of the T-type three-level converter, which is used to reduce the AC side voltage fluctuation; w5 is the output current stability weight coefficient of the photovoltaic module system, which is used to reduce the output current fluctuation of the photovoltaic module system; w6 is the output current stability weight coefficient of the wind turbine system, which is used to reduce the output current fluctuation of the wind turbine system; w7 is the DC side voltage stability weight coefficient of the T-type three-level converter, which is used to reduce the DC side voltage fluctuation; Udc is the DC side voltage tracking error of the T-type three-level converter, e Udc =U dc * -U dc ;i d (tl),i q (tl),u d (tl),u q (tl), I PV(tl), I W (tl), e Udc (t1) are respectively the i of the first moment before the T-type three-level converter d 、i q 、u d 、u q ,I PV ,I W 、e Udc , K is the upper limit of the time for suppressing fluctuations, and l and K are both positive integers.

[0030] 16) Determine the action set: The agent constructed in step 14) is based on the environment state S t and the reward function R t , directly output the optimal action A through a fast response strategy t (S t |θ'), A t (S t |θ') There are three actions, namely: the dq-axis component u of the AC side voltage of the T-type three-level converter d 、u q The reference value u d * 、u q * , and the dq-axis component i of the AC side current of the T-type three-level converter d The reference value i d * .

[0031] 17) Using deep reinforcement learning TD3 algorithm to train the agent:

[0032] A1. Determine the training environment. In order to effectively deal with the impact of light fluctuations on the output power of the photovoltaic module system and wind speed and direction fluctuations on the output power of the wind turbine system, the training environment is the output current training conditions of the new energy power generation system:

[0033] In the output current per unit value-time curve (I s * -t), random current per-unit noise δI is inserted every 5s. s * , calculate the training current per unit value I' according to formula (4) s * (s=1,2, representing the photovoltaic module system and wind turbine system respectively), and at the beginning of a new training round, δI s * Reset I' according to formula (4) s * ;

[0034]

[0035] In the formula, w d is the deviation width, w d ≥1; δI s * Subject to average distribution U; I s * is the per unit value of the system output current, defined as: I s * =I s / I sn , where I s is the actual value of the system output current, I sn is the rated value of the system output current.

[0036] A2. Create an interface for the agent to interact with the environment information, initialize the network weight parameters of the agent, and set the relevant parameters for the agent training, as follows:

[0037] Initialize the weight parameters θ' of the Actor main network and the Critic main network, And the corresponding target network Actor Target network, target network Critic Target network weight parameter θ' ta , k = 1, 2;

[0038] Set the learning rate μ of the Actor main network; set the discount factor γ of deep reinforcement learning; set the maximum number of training rounds N, the maximum number of steps in each training round T, the update frequency I of the target network Actor Target network and Critic Target network, and the weight of the Critic network Update frequency I p ,I ta ;

[0039] Load the actual output current per unit value I of the photovoltaic module system and the wind turbine system s * , current per unit noise δI s * ;

[0040] Set up and initialize the experience database.

[0041] A3. At the beginning of each training round, firstly, according to the actual output current per unit value I of the new energy power generation system s * , current per unit noise δI s *According to formula (4), the output current per unit value training condition I of this training round is determined sh * ,h=1,2,…,N;I' sh * is I' of the hth training round s * .

[0042] A4. Based on the environmental state S t , in order to prevent the network strategy π(S t |θ') falls into a local optimum, adding action exploration noise N to the output of the Actor main network t , get action A t (S t |θ'), providing an opportunity to explore "better", namely:

[0043] A t (S t |θ′)=π(S t |θ′)+N t (5)

[0044] In the formula, π(S t |θ') is the output of the Actor network; N t The expected value is 0 and the variance is σ 2 The normal distribution of t satisfy:

[0045]

[0046] A5. After each action is executed, the next state of the environment S is obtained t+1 , and the reward function R is calculated according to formula (1) t , and the environmental state S t 、Action A t (S t |θ'), reward function value R t , the environmental state S at the next moment t+1 The experience data are stored in the experience memory as an experience data set.

[0047] A6. Randomly and uniformly sample M small batches of samples from the experience database, that is, M experience data groups (S ti ,A ti ,R ti ,S ti+1 ), where i = 1, 2, 3,…,M.

[0048] A7. Update the weight parameters of the Critic main network in a way that minimizes the error Right now

[0049]

[0050] Where Y ti is the expected target value function, defined as:

[0051]

[0052] In the formula, ε represents the target smooth noise, which is the truncated normal distribution noise, that is:

[0053]

[0054] In the formula, σ clip is the standard deviation and c is the cutoff factor.

[0055] A8. Update the weight parameters θ', θ' of the Actor network, Actor Target network, and Critic Target network in a delayed update manner ta and Specifically, the weights of the Critic network Every update I p The weight parameter θ' of the Actor network is updated once. The weight of the Critic network Every update I ta Second, the weight parameters θ' of the Actor Target network and the Critic Target network ta and Update once; if the number of training steps is the update frequency I of the Critic network p If it is a multiple of , then go to step A9, otherwise go to step A11; p ,I ta is the update frequency, which is a positive integer, and I ta >I p .

[0056] A9. Update the weight parameter θ' of the Actor network by performing gradient ascent on the expected value J, that is,

[0057] θ′=θ′+μ▽ θ′ J (10)

[0058] In the formula, the expression of J is:

[0059]

[0060] ▽ θ′ J represents the derivative of J with respect to θ':

[0061]

[0062] In the formula, ▽θ′ π(S ti |θ′) represents the derivative of π(S ti |θ') with respect to θ'.

[0063] A10. If the number of training steps is a multiple of the update frequency I of the Critic network ta then the weight parameters θ' of the Actor Target network ta and the weight parameters of the Critic Target network are updated by exponential smoothing according to Equation (13), that is:

[0064]

[0065] where τ is the update factor and τ ≤ 1.

[0066] A11. If the number of training steps has not reached the maximum number of steps T per training episode, then return to step A4; otherwise, the current training episode ends, and the agent model obtained in this training episode is saved, and the average reward function value R av and its rate of change are calculated.

[0067] A12. If the absolute value of R av is less than or equal to R * av for n (n < N) consecutive training episodes, where R * av is a positive real number close to 0, and the rate of change of R av is less than 5% for n (n < N) consecutive training episodes, then end the entire training process; otherwise, go to step A13.

[0068] A13. Determine whether the maximum number of training episodes N has been reached. If so, end the entire training process; if not, return to step A3 and start the next training episode.

[0069] The specific method of step 2 is as follows:

[0070] 21) Transplant the model of the agent trained in step 1 into the main control chip of the T-type three-level converter. The main control chip outputs an action A t (S t (S t |θ') according to the actual environmental state S of the T-type three-level converter, and obtains the dq-axis components u d 、u q reference values u d * 、u q * of the AC-side voltage of the T-type three-level converter;

[0071] 22) The AC side voltage reference value u of the T-type three-level converter obtained in step 21) is d * and u q * , after dq / abc coordinate transformation, the reference value u of the AC side voltage of the T-type three-level converter is obtained a * 、u b * 、u c * ; Then, u a * 、u b * 、u c * After being modulated by the CBPWM module, a driving signal is generated to drive the power switch tube of the T-type three-level converter to control the operation of the T-type three-level converter.

[0072] The specific method of step 3 is as follows:

[0073] 31) Let the reference value U2 of the lower capacitor voltage U2 of the T-type three-level converter be * For U dc / 2,

[0074] 32) Set this reference value U2 * The difference between the actual measured value U2 and the reference value i of the current of the DC inductor L of the Buck-Boost DC converter is obtained through the first PI controller. L * ;

[0075] 33) will i L * Its actual measured value i L The difference is sent to the PWM module through the second PI controller to generate a driving signal for the Buck-Boost DC converter, thereby controlling the Buck-Boost DC converter to achieve automatic balance control of the midpoint potential.

[0076] The present invention uses a deep reinforcement learning TD3 controller to control a T-type three-level converter to achieve AC side current tracking and DC side voltage stability, and uses a PI controller to control a Buck-Boost DC converter to automatically achieve midpoint potential balance of the T-type three-level converter. Compared with the prior art, the present invention has the following beneficial effects:

[0077] 1. It overcomes the limitations of traditional controllers in completing complex tasks. The present invention designs a reward function that matches the modulation method, realizes the optimal control of multi-objective tasks, can effectively reduce the harmonic distortion rate of the current on the AC side of the converter, and has strong adaptability and robustness to light fluctuations, wind speed and direction changes, and grid voltage fluctuations. It can effectively deal with the impact of light, wind speed and grid voltage fluctuations and uncertainties on system control, and realize rapid and stable control of the main converter of the new factory power supply system containing renewable energy power generation.

[0078] 2. Overcomes the influence of midpoint potential fluctuation on the control performance of the T-type three-level converter. The present invention is simple to control and adopts a Buck-Boost DC converter, which can automatically achieve midpoint potential balance of the T-type three-level converter, simplifies the control algorithm of the T-type three-level converter, greatly improves the performance of the converter system, and ensures its fast and stable control. BRIEF DESCRIPTION OF THE DRAWINGS

[0079] Figure 1 It is a schematic diagram of the topological structure of a new type of factory power supply system containing renewable energy power generation according to the present invention.

[0080] Figure 2 This is a schematic diagram of the topological structure of the main converter according to the present invention.

[0081] Figure 3 This is a control block diagram of the main converter described in the present invention.

[0082] Figure 4 Schematic diagram of the structure and training process of the intelligent agent described in the present invention.

[0083] Figure 5 The figure is a flow chart of training the intelligent agent described in the present invention.

[0084] Figure 6 1 is the experimental waveform of the midpoint potential of the T-type three-level converter of the present invention.

[0085] Among them, 1-main converter, 11-T-type three-level converter, 12-Buck-Boost DC converter; 2-intelligent body; 3-grid; 4-DC bus; 5-photovoltaic module system; 6-wind turbine system; 7-energy storage device; 8-first PI controller; 9-second PI controller. DETAILED DESCRIPTION

[0086] The present invention will be further described in detail below in conjunction with the accompanying drawings.

[0087] like Figure 1As shown, one end of the main converter 1 of the present invention is connected to the power grid 3 through a circuit breaker and a contactor, and the other end is connected to a DC bus 4; the DC bus is connected to a photovoltaic component system 5 and a wind turbine system 6; the DC bus is also connected to an energy storage device 7.

[0088] like Figure 2 As shown, the main converter 1 of the present invention includes a T-type three-level converter 11 and a Buck-Boost DC converter 12; one end of the T-type three-level converter 11 is connected to the power grid 3, and the other end is connected to the Buck-Boost DC converter 12; the other end of the Buck-Boost DC converter 12 is connected to the DC bus 4; the T-type three-level converter 11 includes three LCL filters (L x -C x -L gx , x=a,b,c), three bridge arms, two output capacitors (C1, C2); each bridge arm has four power switches: S x1 ~S x4 , x=a, b, c; one end of the LCL filter is connected to the grid 3; two output capacitors C1 and C2 are connected in series, wherein the output capacitor C1 connected to the positive pole of the DC side of the T-type three-level converter 11 is called the upper capacitor, the output capacitor C2 connected to the negative pole of the DC side of the T-type three-level converter 11 is called the lower capacitor, and the connection point of C1 and C2 is called the midpoint O of the T-type three-level converter 11; the bidirectional switch units S of the three bridge arms x2 -S x3 The other end of (x=a,b,c) is short-circuited and connected to point O.

[0089] The DC converter 12 includes a DC inductor L and two power switches VT1 and VT2. One end of the DC inductor L is connected to the midpoint O of the T-type three-level converter. The Buck-Boost DC converter 12 is used to extract current from the midpoint O of the T-type three-level converter 11 or inject current into the midpoint O of the T-type three-level converter 11 to achieve automatic midpoint potential balance.

[0090] The present invention provides a control method for a main converter of a new type of factory power supply system including renewable energy power generation, comprising the following steps:

[0091] Step 1, such as Figure 3 As shown in FIG. 1 , a controller of a T-type three-level converter 11 is designed by using a deep reinforcement learning TD3 algorithm; the main process is: the part that can reflect the variable information of the main converter 1 of the new factory power supply system containing renewable energy generation is defined as the environment; the part that can perform actions and control the operation of the main converter 1 is defined as the agent 2. The agent 2 is based on the environmental state S t , reward function R t Output Action A t, Action A t Then it acts on the environment to achieve the optimal environment, thereby effectively controlling the operation of the main converter 1.

[0092] The details are as follows:

[0093] 11) Obtain the three-phase grid voltage e through the phase-locked loop PLL a 、e b 、e c The phase angle θ is determined, and the phase information is provided to the Park transformation and iPark transformation of the system to ensure that the three-phase modulation wave of the T-type three-level converter 11 is synchronized with the grid voltage.

[0094] 12) If Figure 2 As shown, the points after the three-phase AC power passes through the three LCL filters of the T-type three-level converter 11 are respectively recorded as Ta, Tb, and Tc points, and the voltages at Ta, Tb, and Tc points are respectively recorded as u a 、u b 、u c ; The current flowing through points Ta, Tb, and Tc are respectively denoted as i a 、i b 、i c ; will u a 、u b 、u c After abc / dq transformation, we get u d and u q ;change i a 、i b 、i c After abc / dq transformation, we get i d and i q , where i d Related to active power, i q is related to reactive power, so i q The reference value i q * Can be set to 0, that is, let i q * =0 to achieve unity power factor operation.

[0095] 13) Determine the environmental state S of the T-type three-level converter 11 t is: the DC side voltage U of the T-type three-level converter 11 dc and its reference value U dc * ; The dq axis component i of the AC side current of the T-type three-level converter 11 d 、i q and its reference value i d * 、i q *; T-type three-level converter 11 AC side voltage dq axis component u d 、u q ; Output current I of photovoltaic module system 5 PV and its rate of change The output current I of the wind turbine system 6 W and its rate of change The above environmental state is recorded as S t , then

[0096] 14) The environmental state S of the T-type three-level converter 11 determined in step 13) is t As the input of Agent 2, construct Agent 2 based on the TD3 algorithm:

[0097] like Figure 4 As shown in the figure, agent 2 consists of a policy network and a policy evaluation network, which contains 6 deep neural networks: the policy network has 2 deep neural networks: composed of an Actor network and an Actor Target network, which respectively pass the corresponding network strategy π(S t |θ') and π ta (S t+1 |θ' ta ) realizes the action output A of agent 2 t (S t |θ'); The strategy evaluation network has four deep neural networks: two groups of Critic networks and Critic Target networks, which are respectively and k = 1, 2, evaluate and improve the strategy used to achieve the action output, where θ', θ' ta , They represent the weight coefficients of the Actor network, Actor Target network, Critic network, and Critic Target network respectively.

[0098] 15) Design reward function R t ;

[0099] Reward function R t Designed for:

[0100] R t =R1+R2 (1)

[0101] Wherein, R1 is the AC side current distortion rate and DC side voltage tracking reward function of the T-type three-level converter 11, and R2 is the fluctuation suppression reward function of the AC side current voltage and DC side voltage of the T-type three-level converter 11, and the output current of the photovoltaic component system 5 and the wind turbine system 6;

[0102] R1 is defined as:

[0103]

[0104] Wherein, w1 is the current tracking weight coefficient of the T-type three-level converter 11 on the AC side, which is used to quickly reduce the current tracking error; w2 is the voltage tracking weight coefficient of the T-type three-level converter 11 on the DC side, which is used to quickly reduce the voltage tracking error on the DC side;

[0105] R2 is defined as:

[0106]

[0107] Wherein, w3 is the AC side current stability weight coefficient of the T-type three-level converter 11, which is used to reduce the AC side current fluctuation; w4 is the AC side voltage stability weight coefficient of the T-type three-level converter 11, which is used to reduce the AC side voltage fluctuation; w5 is the output current stability weight coefficient of the photovoltaic component system 5, which is used to reduce the output current fluctuation; w6 is the output current stability weight coefficient of the wind turbine system 6, which is used to reduce the output current fluctuation; w7 is the DC side voltage stability weight coefficient of the T-type three-level converter 11, which is used to reduce the DC side voltage fluctuation; Udc is the DC side voltage tracking error of the T-type three-level converter, e Udc =U dc * -U dc ;i d (tl),i q (tl),u d (tl),u q (tl), I PV (tl), I W (tl), e Udc (t1) are respectively the i of the T-type three-level converter 11 at the first moment. d 、i q 、u d 、u q ,I PV ,I W 、e Udc , K is the upper limit of the time for suppressing fluctuations, and l and K are both positive integers.

[0108] 16) Determine the action set: Step 14) The constructed agent 2 is based on the environment state St and the reward function R t , directly output the optimal action A through a fast response strategy t (S t |θ'), A t (S t |θ') There are three actions, namely: the dq axis component u of the AC side voltage of the T-type three-level converter 11 d 、u q The reference value u d * 、u q * , and the dq-axis component i of the AC side current of the T-type three-level converter 11 d The reference value i d * .

[0109] 17) Figure 4 , Figure 5 As shown, the deep reinforcement learning TD3 algorithm is used to train agent 2:

[0110] A1. Determine the training environment. In order to effectively deal with the impact of light fluctuations on the output power of the photovoltaic module system 5 and wind speed and direction fluctuations on the output power of the wind turbine system 6, the training environment is determined as the output current training conditions of the new energy power generation system (taking the photovoltaic module system 5 and the wind turbine system 6 as examples):

[0111] The output current per unit value-time curve (I s * -t), random current per-unit noise δI is inserted every 5s. s * , calculate the training current per unit value I' according to formula (4) s * (s=1,2, representing PV module system 5 and wind turbine system 6 respectively), and at the beginning of a new training round, δI s * Reset I' according to formula (4) s * ;

[0112]

[0113] In the formula, w d is the deviation width, w d ≥1; δI s * Subject to average distribution U; I s * is the per unit value of the system output current, defined as: I s* =I s / I sn , where I s is the actual value of the system output current, I sn is the rated value of the system output current.

[0114] A2. Create an interface for Agent 2 to interact with environmental information, initialize the network weight parameters of Agent 2, and set the relevant parameters for Agent 2 training, as follows:

[0115] Initialize the weight parameters θ' of the Actor main network and the Critic main network. And the corresponding target network Actor Target network, target network Critic Target network weight parameter θ' ta , k = 1, 2;

[0116] Set the learning rate μ of the Actor main network; set the discount factor γ of deep reinforcement learning; set the maximum number of training rounds N, the maximum number of steps in each training round T, the update frequency I of the target network Actor Target network and Critic Target network, and the weight of the Critic network Update frequency I p ,I ta ;

[0117] Load the actual output current per unit value I of the photovoltaic module system 5 and the wind turbine system 6 s * , current per unit noise δI s * ;

[0118] Set up and initialize the experience database;

[0119] A3. At the beginning of each training round, first according to I s * ,δI s * According to formula (4), the output current per unit value training condition I' of this training round is determined sh * ,h=1,2,…,N;I' sh * is I' of the hth training round s * .

[0120] A4. Based on the current environment state S t , in order to prevent the network strategy π(S t|θ') falls into a local optimum, adding action exploration noise N to the output of the Actor main network t , get action A t (S t |θ'), providing an opportunity to explore "better", namely:

[0121] A t (S t |θ′)=π(S t |θ′)+N t (5)

[0122] In the formula, π(S t |θ') is the output of the Actor network; N t The expected value is 0 and the variance is σ 2 The normal distribution of t satisfy:

[0123]

[0124] A5. After each action is executed, the next state of the environment S is obtained t+1 , and the reward function R is calculated according to formula (1) t , and the environmental state S t 、Action A t (S t |θ'), reward function value R t , the environmental state S at the next moment t+1 The experience data are stored in the experience memory as an experience data set.

[0125] A6. Randomly and uniformly sample M small batches of samples from the experience database, that is, M experience data groups (S ti ,A ti ,R ti ,S ti+1 ), where i = 1, 2, 3,…,M.

[0126] A7. Update the weight parameters of the Critic main network in a way that minimizes the error Right now

[0127]

[0128] Where Y ti is the expected target value function, defined as:

[0129]

[0130] In the formula, ε represents the target smooth noise, which is the truncated normal distribution noise, that is:

[0131]

[0132] In the formula, σ clip is the standard deviation and c is the cutoff factor.

[0133] A8. Update the weight parameters θ', θ' of the Actor network, Actor Target network, and Critic Target network in a delayed update manner ta and Specifically, the weights of the Critic network Every update I p The weight parameter θ' of the Actor network is updated once. The weight of the Critic network Every update I ta Second, the weight parameters θ' of the Actor Target network and the Critic Target network ta and Update once; if the number of training steps is the update frequency of the Critic network I p If it is a multiple of , then go to step A9, otherwise go to step A11; p ,I ta is the update frequency, which is a positive integer, and I ta >I p .

[0134] A9. Update the weight parameter θ' of the Actor network by performing gradient ascent on the expected value J, that is,

[0135] θ′=θ′+μ▽ θ′ J (10)

[0136] In the formula, the expression of J is:

[0137]

[0138] ▽ θ′ J represents the derivative of J with respect to θ':

[0139]

[0140] In the formula, ▽ θ′ π(S ti |θ′) represents π(S ti |θ') with respect to θ'.

[0141] A10. If the number of training steps is the update frequency of the Critic network I ta If it is a multiple of , the weight parameter θ' of the Actor Target network is calculated by exponential smoothing according to formula (13). ta and the weight parameters of the Critic Target network Perform an update, i.e.:

[0142]

[0143] In the formula, τ is the update factor, and τ ≤ 1.

[0144] A11. If the number of training steps has not reached the maximum number of steps T for each training episode, return to step A4; otherwise, the current training episode ends. At the same time, save the agent model obtained in this training episode and calculate the average reward function value R for this episode av and its rate of change.

[0145] A12. If the absolute value of R av is less than or equal to R * av for n (n < N) consecutive training episodes, where R * av is a positive real number close to 0. For example, let R * av = 0.001, and the rate of change of R av is less than 5% for n (n < N) consecutive training episodes, then end the entire training process; otherwise, enter step A13.

[0146] A13. Determine whether the maximum number of training episodes N has been reached. If it has, end the entire training process; if not, return to step A3 and start the next training episode.

[0147] Step 2: Adopt the CBPWM control strategy to establish an actual current tracking and DC-side voltage control system for the main control chip of the T-type three-level converter 11, and realize the AC-side current tracking and DC-side voltage stable control of the T-type three-level converter 11; the specific method is as follows:

[0148] 21) Transplant the model of the agent trained in step 1 into the main control chip of the T-type three-level converter 11. The main control chip outputs the action A t (S t (S t |θ') according to the actual environmental state S of the T-type three-level converter 11, that is: the dq-axis components u d 、u q of the AC-side voltage of the T-type three-level converter 11, the reference values u d * 、u q * ;

[0149] 22) The u d * and u q *, after dq / abc coordinate transformation, the reference value u of the AC side voltage of the T-type three-level converter 11 is obtained a * 、u b * 、u c * ; Then, u a * 、u b * 、u c * After modulation by the CBPWM module, the driving signal s is generated a 、s b 、s c , driving the power switch tube S of the T-type three-level converter 11 x1 ~S x4 , x=a, b, c, controls the T-type three-level converter 11 to work.

[0150] Step 3, such as Figure 3 As shown, a controller of the Buck-Boost DC converter 12 is designed, and a control system of the Buck-Boost DC converter 12 is established to realize automatic balance control of the midpoint potential. The details are as follows:

[0151] 31) Let the reference value U2 of the voltage U2 across the lower capacitor C2 of the T-type three-level converter 11 be * =U dc / 2, where U dc is the DC side voltage of the T-type three-level converter 11; U2 * The difference between the actual value U2 and the actual value U2 is passed through the first PI controller 8, and the output is the DC inductor L current I of the Buck-Boost DC converter 12. L Reference value I L * ;

[0152] 32) Will I L * Its current actual value I L The difference is passed through the second PI controller 9, and its output is compared with the sawtooth wave. After passing through the PWM module, the driving signals of the power switches VT1 and VT2 of the Buck-Boost DC converter 12 are obtained: when the midpoint potential is higher than its reference value, current is drawn from the midpoint O to reduce the midpoint potential, and it works in the Boost state at this time; when the midpoint potential is lower than its reference value, current is injected into the midpoint O to increase the midpoint potential, and it works in the Buck state at this time, thereby realizing automatic balance of the midpoint potential.

[0153] Figure 6The experimental waveform of the midpoint voltage balance is given a fixed modulation ratio for the T-type three-level converter 11, and only the Buck-Boost DC converter 12 is used to achieve the midpoint voltage balance. It can be seen from the figure that the Buck-Boost converter 12 can quickly achieve the balance control of the upper and lower capacitor voltages, and the midpoint potential oscillation is also well suppressed.

[0154] In summary, the present invention can achieve optimal control of multi-objective tasks with simple control, can suppress the harmonic distortion of the AC side current of the main converter, and reduces power consumption. At the same time, the midpoint potential of the T-type three-level converter can be quickly and automatically balanced, and has strong adaptability and robustness to light fluctuations, wind speed and direction changes, and grid voltage fluctuations. It can effectively cope with the influence of light, wind speed and grid voltage fluctuations and uncertainties on system control, and realize rapid and stable control of the main converter of the new factory power supply system containing renewable energy power generation.

Claims

1. A control method for a main converter of a new type of factory power supply system containing renewable energy power generation, wherein the main converter comprises a T-type three-level converter and a Buck-Boost DC converter; one end of the T-type three-level converter is connected to a power grid, and the other end is connected to the Buck-Boost DC converter; the other end of the Buck-Boost DC converter is connected to a DC bus; the DC bus is connected to a photovoltaic module system and a wind turbine system; the T-type three-level converter comprises three LCL filters, three bridge arms, and two output capacitors; the bridge arm has four power switch tubes, the two output capacitors are connected in series, and the connection point is called the midpoint O of the T-type three-level converter; the Buck-Boost DC converter comprises a DC inductor and two power switch tubes, and one end of the DC inductor is connected to the midpoint O of the T-type three-level converter; characterized in that, Use the following steps: Step 1, using a deep reinforcement learning TD3 algorithm to design a controller of the T-type three-level converter; Step 2, adopting a carrier pulse width modulation (CBPWM) strategy, establishing an actual current tracking and DC side voltage control system based on the main control chip of the T-type three-level converter, and realizing AC side current tracking and DC side voltage stabilization control of the T-type three-level converter; Step 3, designing a controller of the Buck-Boost DC converter, establishing an actual control system of the Buck-Boost DC converter, and realizing automatic balance control of the midpoint potential.

2. The control method of a main converter of a new type of factory power supply system containing renewable energy power generation according to claim 1 is characterized in that: The specific steps of designing the controller of the T-type three-level converter in step 1 are as follows: 11) Obtain the three-phase grid voltage e through the phase-locked loop PLL a 、e b 、e c The phase angle θ is calculated, and the phase information is provided to the Park transformation and iPark transformation of the system to ensure that the three-phase modulation wave of the T-type three-level converter is synchronized with the grid voltage; 12) The points after the three-phase AC power passes through the three LCL filters of the T-type three-level converter are respectively denoted as Ta, Tb, and Tc, and the voltages at Ta, Tb, and Tc are respectively denoted as u a 、u b 、u c ; The current flowing through Ta, Tb, and Tc is denoted as i a 、i b 、i c ; will u a 、u b 、u c After abc / dq transformation, we get u d and u q ;change i a 、i b 、i c After abc / dq transformation, we get i d and i q ; 13) Determine the environmental state S of the T-type three-level converter t ; 14) The environmental state S of the T-type three-level converter determined in step 13) is t As the input of the agent, build an agent based on the deep reinforcement learning TD3 algorithm: The agent is composed of a strategy network and a strategy evaluation network, which contains 6 deep neural networks; wherein the strategy network has 2 deep neural networks: composed of an Actor network and an Actor Target network, which respectively pass the corresponding network strategy π(S t |θ') and π ta (S t |θ' ta ) realizes the action output A of the agent t (S t |θ'); The strategy evaluation network has four deep neural networks: two groups of Critic networks and Critic Target networks, which are respectively and k = 1, 2, evaluate and improve the strategy used to achieve the action output, where θ', θ' ta , Represent the weight coefficients of the Actor network, Actor Target network, Critic network, and Critic Target network respectively; 15) Design reward function R t ; 16) Determine the action set: The agent constructed in step 14) is based on the environment state S t and the reward function R t , directly output the optimal action A through a fast response strategy t (S t |θ'), A t (S t |θ') There are three actions, namely: the dq-axis component u of the AC side voltage of the T-type three-level converter d 、u q The reference value u d * 、u q * , and the dq-axis component i of the AC side current of the T-type three-level converter d The reference value i d * ; 17) Using a deep reinforcement learning TD3 algorithm to train the agent, including determining a training environment.

3. The control method of the main converter of a new factory power supply system containing renewable energy power generation according to claim 2 is characterized in that: The environmental state S of the T-type three-level converter in step 13) t Determined as: including the DC side voltage U of the T-type three-level converter dc and its reference value U dc * ; The dq-axis component i of the AC side current of the T-type three-level converter d 、i q and its reference value i d * 、i q * ; The dq-axis component u of the AC side voltage of the T-type three-level converter d 、u q ; The output current I of the photovoltaic module system PV and its rate of change The output current I of the wind turbine system W and its rate of change The environmental state is denoted as S t , then 4. The control method of a main converter of a new type of factory power supply system containing renewable energy power generation according to claim 2 is characterized in that: The reward function R in step 15) t Designed for: R t =R1+R2 (1) Wherein, R1 is the AC side current distortion rate and DC side voltage tracking reward function of the T-type three-level converter, and R2 is the AC side current voltage and DC side voltage of the T-type three-level converter, as well as the fluctuation suppression reward function of the output current of the photovoltaic module system and the wind turbine system; R1 is defined as: Wherein, w1 is the AC side current tracking weight coefficient of the T-type three-level converter, which is used to quickly reduce the current tracking error; w2 is the DC side voltage tracking weight coefficient of the T-type three-level converter, which is used to quickly reduce the DC side voltage tracking error; R2 is defined as: Wherein, w3 is the AC side current stability weight coefficient of the T-type three-level converter, which is used to reduce the AC side current fluctuation; w4 is the AC side voltage stability weight coefficient of the T-type three-level converter, which is used to reduce the AC side voltage fluctuation; w5 is the output current stability weight coefficient of the photovoltaic module system, which is used to reduce the output current fluctuation of the photovoltaic module system; w6 is the output current stability weight coefficient of the wind turbine system, which is used to reduce the output current fluctuation of the wind turbine system; w7 is the DC side voltage stability weight coefficient of the T-type three-level converter, which is used to reduce the DC side voltage fluctuation; Udc is the DC side voltage tracking error of the T-type three-level converter, e Udc =U dc * -U dc ;i d (tl),i q (tl),u d (tl),u q (tl), I PV (tl), I W (tl), e Udc (t1) are respectively the i of the first moment before the T-type three-level converter d 、i q 、u d 、u q ,I PV ,I W 、e Udc , K is the upper limit of the time for suppressing fluctuations, and l and K are both positive integers.

5. The control method of a main converter of a new factory power supply system containing renewable energy power generation according to claim 2, characterized in that: The training environment in step 17) is the output current training condition of the new energy power generation system, specifically: In the output current per unit value-time curve (I s * -t), random current per-unit noise δI is inserted every 5s. s * , calculate the training current per unit value I' according to formula (4) s * , s = 1, 2, representing the photovoltaic module system and the wind turbine system respectively, and at the beginning of a new training round, δI s * Reset I' according to formula (4) s * ; In the formula, w d is the deviation width, w d ≥1; δI s * Subject to average distribution U; I s * is the per unit value of the system output current, defined as: I s * =I s / I sn , where I s is the actual value of the system output current, I sn is the rated value of the system output current.

6. The control method of a main converter of a new type of factory power supply system containing renewable energy power generation according to claim 2 is characterized in that: The specific method for training the agent in step 17) is: A1. Determine the training environment; A2. Create an interface for the agent to interact with the environment information, initialize the network weight parameters of the agent, and set the relevant parameters for the agent training, as follows: Initialize the weight parameters θ' of the Actor main network and the Critic main network, And the corresponding target network Actor Target network, target network Critic Target network weight parameter θ' ta , k = 1, 2; Set the learning rate μ of the Actor main network; set the discount factor γ of deep reinforcement learning; set the maximum number of training rounds N, the maximum number of steps in each training round T, the update frequency I of the target network Actor Target network and Critic Target network, and the weight of the Critic network Update frequency I p ,I ta ; Load the actual output current per unit value I of the photovoltaic module system and the wind turbine system s * , current per unit noise δI s * ; Set up and initialize the experience database; A3. At the beginning of each training round, firstly, according to the actual output current per unit value I of the new energy power generation system s * , current per unit noise δI s * According to formula (4), the output current per unit value training condition I of this training round is determined sh * ,h=1,2,…,N;I' sh * is I' of the hth training round s * ; A4. Based on the environmental state S t , in order to prevent the network strategy π(S t |θ') falls into a local optimum, adding action exploration noise N to the output of the Actor main network t , get action A t (S t |θ'), providing an opportunity to explore "better", namely: A t (S t |θ′)=π(S t |θ′)+N t (5) In the formula, π(S t |θ') is the output of the Actor network; N t The expected value is 0 and the variance is σ 2 The normal distribution of t satisfy: A5. After each action is executed, the next state of the environment S is obtained t+1 , and the reward function R is calculated according to formula (1) t , and the environmental state S t 、Action A t (S t |θ'), reward function value R t , the environmental state S at the next moment t+1 storing as an experience data set in the experience memory; A6. Randomly and uniformly sample M small batches of samples from the experience database, that is, M experience data groups (S ti ,A ti ,R ti ,S ti+1 ), where i = 1, 2, 3, …, M; A7. Update the weight parameters of the Critic main network in a way that minimizes the error Right now Where Y ti is the expected target value function, defined as: In the formula, ε represents the target smooth noise, which is the truncated normal distribution noise, that is: In the formula, σ clip is the standard deviation, c is the cutoff factor; A8. Update the weight parameters θ', θ' of the Actor network, Actor Target network, and Critic Target network in a delayed update manner ta and Specifically, the weights of the Critic network Every update I p The weight parameter θ' of the Actor network is updated once. The weight of the Critic network Every update I ta Second, the weight parameters θ' of the Actor Target network and the Critic Target network ta and Update once; if the number of training steps is the update frequency I of the Critic network p If it is a multiple of , then go to step A9, otherwise go to step A11; I p ,I ta is the update frequency, which is a positive integer, and I ta >I p ; A9. Update the weight parameter θ' of the Actor network by performing gradient ascent on the expected value J, that is, θ′=θ′+μ▽ θ′ J (10) In the formula, the expression of J is: ▽ θ′ J represents the derivative of J with respect to θ': In the formula, ▽ θ′ π(S ti |θ′) represents π(S ti |θ') to take the derivative of θ'; A10. If the number of training steps is the update frequency I of the Critic network ta If it is a multiple of , the weight parameter θ' of the Actor Target network is calculated by exponential smoothing according to formula (13). ta and the weight parameters of the Critic Target network To update, that is: Where, τ is the update factor, τ≤1; A11. If the number of training steps does not reach the maximum number of steps T for each training round, return to step A4; otherwise, the current training round ends, and the agent model obtained in this training round is saved, and the average reward function value R of this round is calculated. av and its rate of change; A12. If R av whose absolute value is less than or equal to R for n (n < N) consecutive training rounds * av , where R * av is a positive real number close to 0, and the change rate of R av is less than 5% for n (n < N) consecutive training rounds, then end the entire training process; Otherwise, go to step A13; A13. Determine whether the maximum number of training rounds N has been reached. If so, end the entire training process; if not, return to step A3 and start the next training round.

7. The control method of a main converter of a new factory power supply system containing renewable energy power generation according to claim 1, characterized in that: The specific method of step 2 is: 21) transplanting the model of the intelligent agent trained in step 1 into the main control chip of the T-type three-level converter, and the main control chip outputs an action according to the actual environmental state of the T-type three-level converter to obtain the dq-axis component u of the AC side voltage of the T-type three-level converter d 、u q The reference value u d * 、u q * ; 22) The reference value u of the dq axis component of the AC side voltage of the T-type three-level converter is d * and u q * , after dq / abc coordinate transformation, the reference value u of the AC side voltage of the T-type three-level converter is obtained a * 、u b * 、u c * ; Then, u a * 、u b * 、u c * After being modulated by the CBPWM module, a driving signal is generated to drive the power switch tube of the T-type three-level converter to control the operation of the T-type three-level converter.

8. The control method of a main converter of a new type of factory power supply system containing renewable energy power generation according to claim 1, characterized in that: The specific method of step 3 is: 31) Let the reference value U2 of the lower capacitor voltage U2 of the T-type three-level converter be * For U dc / 2; 32) Set this reference value U2 * The difference between the actual measured value U2 and the reference value i of the current of the DC inductor L of the Buck-Boost DC converter is obtained through the first PI controller. L * ; 33) will i L * Its actual measured value i L The difference is sent to the PWM module through the second PI controller to generate a driving signal for the Buck-Boost DC converter, thereby controlling the Buck-Boost DC converter to achieve automatic balance control of the midpoint potential.

Images