Dual-active full-bridge converter optimization control method fused with AI optimization algorithm

Abstract

The invention discloses a dual-active full-bridge converter optimization control method fused with an AI optimization algorithm, and the method comprises the steps: carrying out the offline training of the working states (V1, V2, Po) and phase-shift control variables (D1, D2, D phi) of a dual-active full-bridge converter, so as to obtain a plurality of groups of triple phase-shift control variables of the dual-active full-bridge converter under the lowest root-mean-square current and current stress, and finally, in actual use, sampling V1, V2 and Po of the dual-active full-bridge direct-current converter, calling the deep reinforcement learning model according to the actual size of sampling values, mapping out phase shift angles (D1, D2 and D phi) matched with the optimal current performance, and calculating the optimal current performance of the dual-active full-bridge direct-current converter according to the phase shift angles (D1, D2 and D phi). And current characteristic optimization control of the dual-active full-bridge converter is realized according to the final triple phase-shift control variables.

Description

Technical Field

[0001] The present invention belongs to the technical field of direct current converter control, and more specifically, relates to an optimization control method for a dual active full-bridge converter integrating an AI optimization algorithm. Background Art

[0002] The dual-active-bridge (DAB) DC converter was first proposed in the early 1990s. Figure 1 As shown, it contains a high frequency power electronic transformer T r , a series inductor L r , one input side full bridge and one output side full bridge. As one of the most popular bidirectional topologies, the dual active full bridge converter has the advantages of electrical isolation, high power density, wide voltage transmission range and easy soft switching. It is widely used in electric vehicles, smart grids and renewable energy systems.

[0003] In the traditional dual active full-bridge converter multiple phase shift control method, such as Figure 2 As shown, the two switching devices in each bridge arm adopt complementary switching modes, and the conduction phase of each switching device is 180° (ignoring the dead time). The transmission power is controlled by controlling the switching phase difference between the four bridge arms. This control method has multiple phase-shift control variables. Through the combination of each phase-shift control variable, it can be ensured that the root mean square current and current stress of the dual active full-bridge converter are reduced under the premise of a certain transmission power. However, this type of modulation method has multiple phase-shift control variables, which makes it very difficult to solve the optimal current performance and the complexity of control.

[0004] Taking the general three-phase shift control as an example, given the input side voltage V 1 and the output voltage V 2 , when the switching frequency remains unchanged, there are up to three control variables, such as Figure 2 As shown, the control variable D 1 is the internal phase shift of full bridge FB1, the control variable D 2 is the internal phase shift of full bridge FB2, the control variable D φ v p and v s 'Outer phase shift between the center points. In the traditional multiple phase shift control method, it is very difficult to find the best set of phase shift control variables to optimize the current performance of the dual active full-bridge converter. And in the closed-loop controller, how to design the closed-loop controller to make the control variables approach the optimal control variable combination is also difficult to achieve. Summary of the invention

[0005] The purpose of the present invention is to overcome the shortcomings of the prior art and provide an optimization control method for a dual active full-bridge converter integrating an AI optimization algorithm. The phase-shift control variables for controlling the dual active full-bridge converter are trained through an advanced deep reinforcement learning TD3 algorithm, thereby realizing the optimization control of the dual active full-bridge DC converter.

[0006] To achieve the above-mentioned object of the invention, the present invention discloses a dual active full-bridge converter optimization control method integrating an AI optimization algorithm, which is characterized by comprising the following steps:

[0007] (1) First, set the operating range of the dual active full-bridge converter, and then construct the mathematical expression of the current characteristics and transmission power of the dual active full-bridge converter under universal three-phase shift control;

[0008] (2) Build a deep reinforcement learning model based on the actor-critic framework, and combine the current characteristics of the dual active full-bridge converter and the mathematical expression of the transmission power to construct the reward function of deep reinforcement learning. Then, use the TD3 algorithm in the AI ​​optimization algorithm to train the deep reinforcement learning model until the deep reinforcement learning model converges.

[0009] (3) Real-time acquisition of the input voltage V of the dual active full-bridge DC converter 1 , output voltage V 2 and transmission power P o , then the state quantity s=(V 1 ,V 2 ,P o );Then the state s is input into the trained deep reinforcement learning model, and the action a is fitted through the actor network π=(D 1 ,D 2 ,D φ ), that is, the phase shift angle (D 1 ,D 2 ,D φ ), and finally according to the phase shift angle (D 1 ,D 2 ,D φ ) is used to optimize the control of the dual active full-bridge converter.

[0010] The object of the invention of the present invention is achieved in this way:

[0011] The present invention provides a dual active full-bridge converter optimization control method integrating AI optimization algorithm, and at the same time, the working state (V 1 ,V 2 ,P o ) and phase shift control quantity (D 1 ,D 2 ,D φ) to obtain multiple sets of triple phase-shift control variables of the dual-active full-bridge converter under the lowest RMS current and current stress, and integrate them into a deep reinforcement learning model. Finally, in actual use, the V 1 , V 2 , P o Sampling is performed, and the deep reinforcement learning model is called according to the actual size of the sampled value to map out the phase shift angle (D 1 ,D 2 ,D φ ), and realize the current characteristic optimization control of the dual active full-bridge converter according to the final triple phase-shift control variables.

[0012] At the same time, the dual-active full-bridge converter optimization control method integrating the AI ​​optimization algorithm of the present invention also has the following beneficial effects:

[0013] (1) The present invention uses a deep reinforcement learning method to analyze the working state (V 1 ,V 2 ,P o ) and phase shift control quantity (D 1 ,D 2 ,D φ ) values, it is easy to find the corresponding optimal set of phase-shift control variables, so as to reduce the root mean square current and current stress, thereby improving the efficiency of the dual active full-bridge converter;

[0014] (2) In closed-loop control, the sampled V 1 、V 2 and P o The corresponding value is searched in the trained strong deep learning model to map the phase shift angle (D 1 ,D 2 ,D φ ), and then optimizing the control of the dual active full-bridge DC converter according to the set of phase-shift control variables;

[0015] (3) When real-time acquisition of the V 1 、V 2 and P o When the corresponding value is not within the pre-defined range, the phase shift control variable (D 1 ,D 2 ,D φ ) can realize real-time control of the dual active full-bridge converter. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1It is the topological structure diagram of the dual active full-bridge converter;

[0017] Figure 2 It is a partial voltage and current waveform diagram of the dual active converter;

[0018] Figure 3 It is a current optimization control structure of a dual active full-bridge converter based on deep reinforcement learning of the present invention;

[0019] Figure 4 is the RMS current experimental result;

[0020] Figure 5 is the result of current stress experiment;

[0021] Figure 6 The transmission efficiency test results DETAILED DESCRIPTION

[0022] The specific implementation of the present invention is described below in conjunction with the accompanying drawings so that those skilled in the art can better understand the present invention. It should be noted that in the following description, when the detailed description of known functions and designs may dilute the main content of the present invention, these descriptions will be omitted here.

[0023] Example

[0024] For the convenience of description, the relevant professional terms appearing in the specific implementation methods are first explained:

[0025] In this embodiment, if Figure 1 As shown, the dual active full-bridge converter includes a high-frequency power electronic transformer T r , a series inductor L r , an input side full bridge and an output side full bridge. The input side full bridge includes two bridge arms, namely bridge arm 1 and bridge arm 2; bridge arm 1 includes two switching devices, namely switching device S 1 and the switching device S 2 ; Bridge arm 2 includes two switching devices, namely, switching device S 3 and the switching device S 4 The output side full bridge includes two bridge arms, namely bridge arm 3 and bridge arm 4; bridge arm 3 includes two switching devices, namely switching device S 5 and the switching device S 6 The bridge arm 4 includes two switching devices, namely, the switching device S 7 and the switching device S 8 The two switching devices in each bridge arm adopt a complementary switching mode, and the conduction phase of each switching device is 180° (ignoring the dead time).

[0026] like Figure 2 As shown, the control variable D1 is the internal phase shift of full bridge FB1, the control variable D 2 is the internal phase shift of full bridge FB2, the control variable D φ v p and v s The range of the three control variables of the external phase shift between the center points is limited to the phase shift angle of the half cycle, that is, (D 1 ,D 2 ,D φ ) is between (0~π),v p is the voltage difference between the middle points of the two bridge arms of the input full bridge, v s The voltage difference between the middle points of the two bridge arms of the full bridge is output. The transformer ratio is n:1, v s ' is v s The voltage equivalent to the primary side of the transformer, v s The amplitude is V 1 , v s ' is equal to nV 2 ,i Lk is the current flowing through the inductor.

[0027] Below we describe in detail a dual active full-bridge converter optimization control method integrating an AI optimization algorithm of the present invention, which specifically includes the following steps:

[0028] S1. Setting the working range of the dual active full-bridge converter and constructing the mathematical expressions of the current characteristics and transmission power of the dual active full-bridge converter under the universal three-phase shift control;

[0029] S1.1, set the dual active full-bridge DC converter input side voltage V 1 Range, output voltage V 2 The range and the desired transmission power P o In this embodiment, the input voltage V 1 The range is 100V~140V, the output voltage V 2 The range is 40V~50V, and the transmission power P o The range is 80W~200W;

[0030] S1.2. Construct mathematical expressions for the current characteristics and transmission power of the dual active full-bridge converter under universal three-phase shift control;

[0031]

[0032] Among them, V 1 is the input voltage of the dual active full-bridge converter, V 2 is the output voltage of the dual active full-bridge converter, and the control variable D 1 is the internal phase shift of full bridge FB1, the control variable D2 is the internal phase shift of full bridge FB2, the control variable D φ v p and v s 'The external phase shift between the center points, the range of the three control variables is limited to the phase shift angle of the half cycle, that is, (D 1 ,D 2 ,D φ ) is between (0 and π), P o is the transmission power of the dual active full-bridge converter, coefficient n=1,3,5,7,…, L is the inductance value, A and B are intermediate variables, I rms is the RMS current of the dual active full-bridge converter, f s is the switching frequency, w 0 is the angular frequency, i L is the inductor current.

[0033] S2. Use TD3 algorithm to train deep reinforcement learning model;

[0034] In this embodiment, we use the TD3 algorithm in the AI ​​optimization algorithm to train the deep reinforcement learning model. During the training process, the root mean square current and current stress of the dual active full-bridge converter are taken as the target, and a certain range of input voltage V 1 , output voltage V 2 The triple phase shift control variable (D 1 ,D 2 ,D φ ), the specific process is:

[0035] S2.1. Set the reward function of the deep reinforcement learning model r = -[α(P o -P o ') 2 +β|I rms |], where P o ' represents the transfer power in the reinforcement learning process, α and β are weight coefficients; after simulation experiments, the α value is selected as the benchmark value 1, and the β value is selected as 10;

[0036] S2.2. Build a deep reinforcement learning model based on the actor-critic framework, including: an actor network π and two critic networks θ 1 ,θ 2 , as well as a target actor network π' and two target critic networks θ 1 '、θ 2 ';

[0037] S2.3. Within the operating range of the dual active full-bridge converter, V 1 ,V2 ,P o Random sampling to get the current state s t =(V 1,t ,V 2,t ,P o,t ), and then the state s t Input to the actor network π, fit the action a t =(D 1,t ,D 2,t ,D φ,t ),like Figure 3 As shown;

[0038] S2.4, action a t Substitute into formula (1) to calculate the inductor current i L , and then determine the inductor current i L Are the following conditions met?

[0039]

[0040] If the inductor current i L If formula (2) is satisfied, then (s t ,a t ) is substituted into the reward function of the deep reinforcement learning model to calculate the reward value r(s) at the current moment t ,a t ); otherwise, increase the weight coefficient β. After simulation experiments, the optimal β value is 50, and then (s t ,a t ) is substituted into the reward function of the deep reinforcement learning model to calculate the reward value r(s) at the current moment t ,a t );

[0041] S2.5, repeat step S2.3 to get state s t+1 =(V 1,t+1 ,V 2,t+1 ,P o,t+1 ) and action a t+1 =(D 1,t+1 ,D 2,t+1 ,D φ,t+1 );

[0042] S2.6, will (s t+1 ,a t+1 ) are substituted into the target critic network θ i ', get the output value

[0043] S2.7, during the training process, the critic network is based on the current behavior a t and statust Output Q value The Q value must follow the corresponding value generated by the target critic network as closely as possible, so we choose As the benchmark, the two critic networks θ are then updated through the mean square Bellman error function i ;

[0044]

[0045] Where N is

[0046] S2.8, The maximum value of is the target update actor network π;

[0047]

[0048] in, represents the gradient, E represents the expectation; J(π) represents, s~R represents, Indicates (s t ,a t ) into the critic network θ i The Q value of the output;

[0049] S2.9. Update the two target critic networks θ i ';

[0050] θ i '←τθ i +(1-τ)θ i ',i=1,2

[0051] S2.10, update the target actor network π';

[0052] π'←τπ+(1-τ)π'

[0053] Where, τ represents the soft update factor;

[0054] S2.11, repeat steps S2.3) to S2.10 until the deep reinforcement learning model converges;

[0055] S3, controlling the dual active full-bridge converter;

[0056] Real-time acquisition of V of dual active full-bridge DC converter 1 、V 2 and P o , and then form the current state s=(V 1 ,V 2 ,P o );Then input the state s into the trained actor network π, and fit the action a=(D 1 ,D2 ,D φ ), that is, the phase shift angle (D 1 ,D 2 ,D φ ), and finally according to the phase shift angle (D 1 ,D 2 ,D φ ) is used to optimize the control of the dual active full-bridge converter.

[0057] Experimental simulation

[0058] In this embodiment, when the input voltage V 1 When the output voltage V 2 is 50V and the transmission power P o The corresponding RMS current experimental diagram is as follows: Figure 4 As shown, SPS is the root mean square current experimental diagram corresponding to the existing single-phase shift control strategy, DPS is the root mean square current experimental diagram corresponding to the existing dual-phase shift control strategy, EPS is the root mean square current experimental diagram corresponding to the existing extended phase shift control strategy, QATPS is the root mean square current experimental diagram corresponding to the three-phase shift control strategy based on Q-learning algorithm combined with ANN neural network optimization, DUTPS is the root mean square current experimental diagram corresponding to the current performance optimization control strategy integrating AI optimization algorithm corresponding to the present invention, and the input side voltage V 1 When the output voltage V 2 is 50V and the transmission power P o The corresponding current stress test diagram is shown in Figure 5 As shown, SPS is the current stress test diagram corresponding to the existing single-phase shift control strategy, DPS is the current stress test diagram corresponding to the existing dual-phase shift control strategy, EPS is the current stress test diagram corresponding to the existing extended phase shift control strategy, QATPS is the current stress test diagram corresponding to the three-phase shift control strategy based on Q-learning algorithm combined with ANN neural network optimization, DUTPS is the current stress test diagram corresponding to the current performance optimization control strategy integrating AI optimization algorithm corresponding to the present invention, and the input side voltage V 1 When the output voltage V 2 is 50V and the transmission power P o The corresponding efficiency test diagram is as follows: Figure 5As shown, SPS is the efficiency experimental diagram corresponding to the existing single-phase shift control strategy, DPS is the efficiency experimental diagram corresponding to the existing double-phase shift control strategy, EPS is the efficiency experimental diagram corresponding to the existing extended phase shift control strategy, QATPS is the efficiency experimental diagram corresponding to the three-phase shift control strategy based on Q-learning algorithm combined with ANN neural network optimization, and DUTPS is the efficiency experimental diagram corresponding to the current performance optimization control strategy integrating AI optimization algorithm corresponding to the present invention. Figure 4 , Figure 5 and Figure 6 It can be seen that the current characteristic optimization control method of the dual active full-bridge converter based on deep reinforcement learning provided by the present invention has relatively low root mean square current and current stress, which can improve the efficiency of the dual active full-bridge DC converter.

[0059] Although the above describes the illustrative specific embodiments of the present invention to facilitate those skilled in the art to understand the present invention, it should be clear that the present invention is not limited to the scope of the specific embodiments. For those of ordinary skill in the art, as long as various changes are within the spirit and scope of the present invention as defined and determined by the attached claims, these changes are obvious, and all inventions and creations using the concept of the present invention are protected.

Claims

1. A dual active full-bridge converter optimization control method integrating AI optimization algorithm, characterized in that: The following steps are involved: (1) Setting the operating range of the dual active full-bridge converter and constructing a mathematical expression of the current characteristics and transmission power of the dual active full-bridge converter under universal three-phase shift control; (1.1) Set the input voltage V1 range, output voltage V2 range and expected transmission power P of the dual active full-bridge converter application o scope; (1.2) Construct a mathematical table of current characteristics and transmission power of a dual active full-bridge converter under universal three-phase shift control; Among them, V1 is the input voltage of the dual active full-bridge converter, V2 is the output voltage of the dual active full-bridge converter, the control variable D1 is the internal phase shift of the full bridge FB1, the control variable D2 is the internal phase shift of the full bridge FB2, and the control variable D φ v p and v s 'The external phase shift between the center points, the range of the three control variables is limited to the phase shift angle of the half cycle, that is, (D1, D2, D φ ) is between (0 and π), P o is the transmission power of the dual active full-bridge converter, I rms is the RMS current of the dual active full-bridge converter, f s is the switching frequency, w0 is the angular frequency, i L is the expression of the inductor current; (2) Build a deep reinforcement learning model; (2.1) Set the reward function of the deep reinforcement learning model r = -[α(P o -P o ') 2 +β|I rms |], where P o ' represents the transfer power in the reinforcement learning process, α and β are weight coefficients; (2.2) Build a deep reinforcement learning model based on the actor-critic framework, including: an actor network φ and two critic networks θ1, θ2, as well as a target actor network φ' and two target critic networks θ1', θ2'; (2.3) Within the working range of the dual active full-bridge converter, V1, V2, P o Random sampling to get the current state s t =(V 1,t ,V 2,t ,P o,t ), and then the state s t Input to the actor network φ, fit the action a t =(D 1,t ,D 2,t ,D φ,t ); (2.4), action a t Substitute into formula (1) to calculate the inductor current i L , and then determine the inductor current i L Are the following conditions met? If the inductor current i L If formula (2) is satisfied, then (s t ,a t ) is substituted into the reward function of the deep reinforcement learning model to calculate the reward value r(s) at the current moment t ,a t ); otherwise, increase the weight coefficient β step by step, and then (s t ,a t ) is substituted into the reward function of the deep reinforcement learning model to calculate the reward value r(s) at the current moment t ,a t ); (2.5) Repeat step (2.3) to get state s t+1 =(V 1,t+1 ,V 2,t+1 ,P o,t+1 ) and action a t+1 =(D 1,t+1 ,D 2,t+1 ,D φ,t+1 ); (2.6), will (s t+1 ,a t+1 ) are substituted into the target critic network θ′ i , and get the output value (2.7) Select Then update the two critic networks θ through the mean square Bellman error function i ; (2.8), The maximum value of is the target update actor network φ; (2.9), update the two target critic networks θ i '; i i '←tθ i +(1-τ)θ i ',i=1,2 (2.10), Update the target actor network φ'; φ'←τφ+(1-τ)φ' (2.11) Repeat steps (2.3) to (2.10) until the deep reinforcement learning model converges; (3) Controlling the dual active full-bridge converter; Real-time acquisition of V1, V2 and P of dual active full-bridge DC converter o , and then judge the sampled V1, V2 and P o The corresponding value is searched in the deep reinforcement learning model in step (2.4), mapping out the phase shift angle (D1, D2, D φ ), and then the dual active full-bridge converter is optimally controlled according to the phase-shift control variable.

Images