A Model Predictive Control Method, System, and Readable Medium for Direct-Drive Wave Power Generation Systems Based on Deep Q-Networks

Abstract

This invention discloses a predictive control method, system, and readable medium for a direct-drive wave power generation system based on a deep Q-network, belonging to the technical field of renewable energy power generation. The method applies predictive control to a direct-drive wave power generation system, aiming to maximize power capture. It incorporates the actual physical limitations of the device into the algorithm's constraints, using known future wave force data over a certain time domain to predict and obtain the optimal control parameters for the wave power generation system. Furthermore, a deep Q-network algorithm is employed to adjust the optimal values ​​of the regularization coefficients in the objective function. This predictive control method can adaptively adjust the regularization coefficients based on a deep Q-network under different wave excitation force input conditions, improving algorithm flexibility and enhancing the power capture performance of the direct-drive wave power generation system while satisfying system constraints.

Description

Technical Field

[0001] This invention belongs to the technical field of renewable energy power generation, and particularly relates to wave energy power generation technology. Specifically, it relates to a model prediction and control method, system, and readable medium for a direct-drive wave power generation system based on deep Q-networks. Background Technology

[0002] The scientific development and utilization of wave energy is of great significance for alleviating the energy crisis and environmental pollution. Among the many types of wave power generation devices, direct-drive wave power generation systems have the advantages of simple system structure and high efficiency, and have become a research hotspot in the development and utilization of wave energy. In order to improve the power generation efficiency of direct-drive wave power generation systems, effective control strategies must be adopted to achieve maximum wave energy capture while meeting the actual constraints of the power generation system's state variables.

[0003] Traditional wave power generation system control strategies include real part control and complex conjugate control. Real part control captures relatively small amounts of power, while complex conjugate control captures much larger amounts, but the system state variables are prone to exceeding the actual allowable range, affecting the normal operation of the system. Model predictive control (MMC) can achieve rolling optimization control under multiple constraints and objectives, and has seen some research and application in the power control field of wave power generation systems. Furthermore, to avoid optimization failures in MMC algorithms, a regularization term is often introduced into the objective function to transform it into a convex quadratic programming model. Summary of the Invention

[0004] To address the shortcomings and deficiencies of existing technologies, this invention aims to provide a model predictive control method for direct-drive wave power generation systems based on deep Q-networks. This method enables the direct-drive wave power generation system to extract maximum power while satisfying physical constraints. Furthermore, by utilizing deep Q-networks to obtain the optimal values ​​of the regularization coefficients in the objective function, the control performance is further optimized, and the power capture efficiency is improved.

[0005] According to a first aspect of the present invention, a model predictive control method for a direct-drive wave power generation system based on a deep Q-network is proposed, comprising the following steps:

[0006] Step S1: Establish the mechanical motion model and linear permanent magnet motor model of the direct-drive wave power generation system;

[0007] Step S2: Based on the mechanical motion model and linear permanent magnet motor model of the direct-drive wave power generation system, establish the state-space model of the direct-drive wave power generation system based on model predictive control;

[0008] Step S3: Based on the control objective of the direct-drive wave power generation system, establish a power optimization objective function, and then augment the state space model according to the power optimization objective function and transform it to the discrete domain to obtain the discrete state space model of the direct-drive wave power generation system.

[0009] Step S4: Set system constraints and perform rolling optimization of state variables on the discrete state-space model until rolling predictive control is completed, thereby realizing online optimization solution of the system;

[0010] Step S5: Use a deep Q-network to learn the mapping relationship between the optimal value of the regularization coefficient and the capture power, and determine the value of the regularization coefficient in the objective function.

[0011] A further aspect of the present invention is that the mechanical motion model expression of the direct-drive wave power generation system established in step S1 is as follows:

[0012]

[0013] In the formula, M is the mass of the float, and m add The added mass of the water, together with the water, constitutes the equivalent total mass of the float;

[0014] z(t) is the heave displacement of the float. The velocity of the float. The acceleration of the float; the damping coefficient k(t) and velocity The convolution term corresponds to the radiation force f experienced by the float. R (t); float displacement z(t) and elastic coefficient k s The product of these values ​​corresponds to the equivalent spring resistance f experienced by the float. s (t); f wave The wave excitation force is used as input; f PTO It is an external electromagnetic thrust from the system's power take-off (PTO) device.

[0015] A further aspect of the present invention is to use a state-space model to approximate the radiation force f in the mechanical motion model expression. R (t).

[0016] The linear permanent magnet motor model expression for the direct-drive wave power generation system established in step S1 is as follows:

[0017]

[0018] In the formula, u d u q These are the stator d-axis and q-axis voltages, i d i q These are the stator d-axis and q-axis currents, respectively; Rs Ψ is the resistance of the stator winding. PM L is the flux linkage of the permanent magnet; ω is the electric angular velocity of the linear permanent magnet motor, ω=2πv / λ, where λ is the pole pitch of the motor; d L q These are the stator d-axis and q-axis inductances, respectively. This paper uses a surface-mount linear motor, so L can be set as follows: d =L q =L s =L ss -L M L ss For the self-inductance of the stator winding, L M This refers to the mutual inductance between the stator windings.

[0019] A further aspect of the present invention is that the state-space model expression established in step S2 is:

[0020]

[0021] In the formula, v(t) is the scaled wave excitation force; u q (t) represents the control input of the system.

[0022] A further aspect of the present invention is that the power optimization objective function established in step S3 includes the PTO force f. PTO The penalty term, the expression for the power optimization objective function is:

[0023]

[0024] In the formula, Let be the change in the velocity of the float, and r be the canonical coefficient.

[0025] A further aspect of the present invention is that, in step S4, the constraints include linear constraints and nonlinear constraints, wherein the linear constraints include the heave displacement z(t) of the float and its velocity. and PTO force f PTO The expression is as follows:

[0026]

[0027] The nonlinear constraints include the stator voltage, stator current, and field weakening flux τ of the linear permanent magnet motor, expressed as follows:

[0028]

[0029] In the formula, I s This represents the current in the stator winding.

[0030] A further aspect of the present invention is that, in step S5, the deep Q-network used constitutes the state space of the evaluation system environment based on the current wave peak frequency, wave height, and current regularization coefficients, as expressed below:

[0031] S(t) = [f p (t),h p (t),r(t)]

[0032] The increment of the regularization coefficient is set as the action value for adjustment, with an increment interval of 0.2. The action space expression is as follows:

[0033] A(t) = [-Δr, 0, +Δr]

[0034] Considering that the learning objective of the deep Q-network is to maximize wave capture power when a feasible solution exists after satisfying system constraints, the reward function is defined as follows:

[0035]

[0036] According to a second aspect of the present invention, a model predictive control system for a direct-drive wave power generation system based on a deep Q-network is proposed, comprising:

[0037] One or more processors, and

[0038] Memory used to store operable instructions;

[0039] The instructions, when executed by the one or more processors, implement the aforementioned model predictive control method for a direct-drive wave power generation system based on deep Q-networks.

[0040] In a third aspect of the invention, a computer-readable medium for storing software is provided, the software comprising instructions executable by one or more computers, the instructions causing the one or more computers to perform operations including the aforementioned process of predictive control of a direct-drive wave power generation system model based on a deep Q-network.

[0041] Compared with the prior art, the present invention has the following beneficial effects:

[0042] The proposed model predictive control method for direct-drive wave power generation systems based on deep Q-networks can adaptively adjust the regularization coefficients based on deep Q-networks for different wave excitation force inputs, thereby improving the algorithm's flexibility and enhancing the power capture performance of direct-drive wave power generation systems while satisfying system constraints. Attached Figure Description

[0043] Figure 1 This is a predictive control block diagram for a direct-drive wave power generation system model based on a deep Q-network.

[0044] Figure 2 This is a force diagram of a direct-drive wave power generation system.

[0045] Figure 3 This is a block diagram illustrating the principle of the deep Q-network algorithm. Detailed Implementation

[0046] The technical solution of the invention will be described in detail below with reference to the accompanying drawings:

[0047] Combined with appendix Figure 1 The diagram shown is a model predictive control block diagram of a direct-drive wave power generation system based on a deep Q-network. In the outer-loop model predictive control structure, to obtain higher regulation speed and control accuracy, the stator q-axis current reference value increment Δi of the linear motor is increased. q_ref d-axis current reference value increment Δi d_ref As the control output of the control algorithm, the predicted future wave force Δv is used as the uncontrollable input, along with the float's displacement z(t) and velocity. Parameters are used as state variables.

[0048] In embodiments of the present invention, a deep Q-network algorithm is used to adjust the optimal value of the regularization coefficient in the objective function, thereby improving the system's adaptability to different wave inputs; the optimal control quantity output by rolling optimization is used to obtain the q-axis current reference value i after zero-order hold. q_ref d-axis current reference value i d_ref After decoupling through the inner current loop, a PI controller is used to track the reference signal, and finally, PWM technology is used to modulate the signal to complete the optimized control process of the system.

[0049] As an example, the process of the model predictive control method for a direct-drive wave power generation system based on a deep Q-network includes: Step S1: Establishing a mechanical motion model and a linear permanent magnet motor model for the direct-drive wave power generation system; Step S2: Based on the mechanical motion model and the linear permanent magnet motor model, establishing a state-space model of the direct-drive wave power generation system based on model predictive control; Step S3: Based on the control objective of the direct-drive wave power generation system, establishing a power optimization objective function, and then augmenting the state-space model according to the power optimization objective function and transforming it to the discrete domain to obtain a discrete state-space model of the direct-drive wave power generation system; Step S4: Setting system constraints, performing rolling optimization of the state variables on the discrete state-space model until rolling predictive control is completed, realizing online optimization solution of the system; Step S5: Using a deep Q-network to learn the mapping relationship between the optimal value of the regularization coefficient and the captured power, determining the value of the regularization coefficient in the objective function.

[0050] The specific steps and processes of the embodiments of the present invention will now be described in more detail with reference to the accompanying drawings.

[0051] S1: Establish the mechanical motion model and linear permanent magnet motor model of the direct-drive wave power generation system.

[0052] (I) Establishing a mechanical motion model for a direct-drive wave power generation system

[0053] For the mechanical motion equations of a direct-drive wave power generation system, based on the force conditions of the float in the direct-drive wave power generation system (such as... Figure 2 Based on the equations shown (and Newton's second law), a simplified equation of motion for a direct-drive wave power generation system is established, expressed as follows:

[0054]

[0055] In the formula, M is the mass of the float, and m add The added mass of the water, together with the water, constitutes the equivalent total mass of the float;

[0056] Where z(t) is the heave displacement of the buoy. The velocity of the float. The acceleration of the float; the damping coefficient k(t) and velocity The convolution term corresponds to the radiation force f experienced by the float. R (t); float displacement z(t) and elastic coefficient k s The product of these values ​​corresponds to the equivalent spring resistance f experienced by the float. s (t).

[0057] The hydrodynamic parameters involved in the equations (including float mass M and float added mass m) were calculated using WAMIT software. add Damping coefficient k(t) and elastic coefficient k s );f wave The wave excitation force is used as input; f PTO It is an external electromagnetic thrust from the system's power take-off (PTO) device.

[0058] To improve the computational efficiency of the control process, in embodiments of the present invention, a state-space model is used to approximate the radiation force f in the mechanical motion model expression. R The transformation process for (t) convolution terms is as follows:

[0059] First, the Prony algorithm is used to fit the k(t) curve to obtain its complex exponential weighted sum form. Then, the Laplace transform is used to obtain the S-domain expression of the radiation coefficient. Finally, the equilibrium truncation method is used to reduce the order of the above equation to obtain the parameter A. r ∈R 4×4 B r ∈R 4×1 C r ∈R 1×4 and D r ∈R 1×1 , to obtain f R The fourth-order state-space model of (t) is expressed as follows:

[0060]

[0061] In the formula, x r (t)∈R 4×1 It has no actual physical meaning; it simply represents the state variable in the state-space model. z(t) represents the displacement of the float, and its differential term... That is, the speed of the float.

[0062] (II) Establishing a linear permanent magnet motor model for a direct-drive wave power generation system

[0063] The stator terminal voltage equation of the linear permanent magnet motor in the abc coordinate system is established as follows:

[0064]

[0065] In the formula, u s_abc The voltage vector at the stator terminals; R = diag(R s ,R s ,R s ), R s The resistance of the stator winding is L; the stator inductance matrix is ​​i abc Ψ is the stator winding current vector; PM_abc The magnetic flux generated in the stator by the permanent magnet rotor;

[0066] Transforming the stationary abc coordinate system into the dq rotating coordinate system, the voltage balance equations along the dq axis are obtained as follows:

[0067]

[0068] In the formula, ω is the electrical angular velocity of the linear permanent magnet motor, ω=2πv / λ, and λ is the pole pitch of the motor; d For the stator d-axis current, i q Let M be the stator q-axis current, M be the mutual inductance between the stator windings, and L be the mutual inductance between the stator windings. ss For the self-inductance of the stator winding, L s =L ss -M;

[0069] Based on the relationship between the parameters of the linear permanent magnet motor, the q-axis current i is derived. qs With PTO force f PTO The linear relationship:

[0070]

[0071] S2: Establish a state-space model for a direct-drive wave power generation system based on model predictive control.

[0072] An equivalent model of the motion equations of the wave power generation system is established, and its specific expression is as follows:

[0073]

[0074] In the formula, This represents the scaled-down wave excitation force;

[0075] As a control variable of the system;

[0076] Obtain control quantity u q (t) and stator q-axis current i qs The relationship between (t) is as follows:

[0077]

[0078] The rising and falling displacement z(t) and velocity of the float and intermediate quantity x r Using (t) as the state variable, the continuous state-space model of the wave power generation system is obtained as follows:

[0079]

[0080] The coefficient matrix is ​​as follows:

[0081]

[0082]

[0083]

[0084] S3: Establish the objective function for model predictive control and build a discrete state-space model.

[0085] Based on the control objectives of the direct-drive wave power generation system, a system including PTO force f is established. PTO The power optimization objective function for the penalty term is in the following form:

[0086]

[0087] Based on the objective function and considering the working principle of a direct-drive wave power generation system with weak magnetic control, the control variable u is... q (t), scaled wave excitation force v(t), d-axis current i d (t) is added as a new state variable to augment the model and transform it to the discrete domain, ultimately yielding the discrete state-space model of the system:

[0088]

[0089] The variable and coefficient matrix in the formula is as follows:

[0090]

[0091]

[0092]

[0093]

[0094]

[0095] In the formula, T l For the sampling interval, A c B c F c These are the coefficient matrices of the continuous state-space model obtained in step 3.

[0096] S4: Rolling optimization of state variables

[0097] The output sequence in the future N time domains starting from iteration time k The rolling prediction of state variables is expressed as follows:

[0098]

[0099] In the formula,

[0100]

[0101]

[0102]

[0103]

[0104] x(k) represents the system state variable at time k. This represents the future N-step increment sequence of the scaling wave force starting at time k.

[0105] Only the first increment is applied to the system to obtain the control quantity at the next time step:

[0106] U(k) = ΔU(k) ​​+ U(k-1)

[0107] Repeat the above process until rolling predictive control is completed, thereby achieving online optimization of the system.

[0108] S5: Constraints

[0109] The physical constraints of the direct-drive wave power generation system are incorporated into the algorithm's constraints during the control process. These linear constraints include the float's heave displacement z(t) and velocity. and PTO force f PTO The expression is as follows:

[0110]

[0111] Nonlinear constraints include stator voltage amplitude, stator current amplitude, and weak magnetic flux density τ.

[0112] Among them, the stator voltage amplitude limit value U max Depending on the DC-side voltage of the PWM rectifier, in SPWM pulse width modulation mode, it is the DC bus voltage U. dc 1 / 2; Stator current amplitude limit I max It depends on the motor temperature rise and the output capacity limitations of the PWM rectifier.

[0113] It should be understood that during the actual field weakening operation of a permanent magnet motor, if the direct-axis demagnetizing current is too large, the permanent magnet may face the risk of permanent demagnetization. Therefore, the field weakening rate τ should always be controlled to be less than or equal to 1. The definition of the field weakening rate τ is as follows:

[0114]

[0115] Therefore, the expression for the nonlinear constraint can be obtained as follows:

[0116]

[0117] When the generator is in the low-speed range, i is used. d =0 control; when the wave excitation force further increases, the generator terminal voltage rises. To increase the operating range of the power generation system, i should be increased at this time. d The current value is determined by switching i. d =0 control is for weak magnetic field control.

[0118] S6: Learn the optimal value of the regularization coefficient in the objective function using a deep Q-network:

[0119] Factors characterizing the system state include the wave peak frequency f. p , wave height h p Given the current regularization coefficient r(t), the state-space expression of the algorithm is defined as follows:

[0120] S=[s|f p (t),h p (t),r(t)]

[0121] Set the increment of the regularization coefficient as the action value, with an increment size of 0.2, and define the action space expression as follows:

[0122] A = [a|-Δr, 0, +Δr]

[0123] Considering that the learning objective of the deep Q-network is to maximize the captured wave power under feasible solutions within system constraints, the reward function is defined as follows:

[0124]

[0125] A deep Q-network is used to fit the theoretically optimal state-action value function Q*(s,a), i.e.:

[0126] Q(s,a,ω)≈Q*(s,a)

[0127] In the formula, ω represents the neural network parameters.

[0128] It should be understood that the Deep Q-Network algorithm consists of two Q-networks: an estimated Q-network and a target Q-network. They have the same hierarchical structure but different parameters. The parameters of the estimated Q-network are updated iteratively at each step, while the target Q-network copies the parameters of the estimated Q-network and updates them with a delay after a certain step interval, thus remaining fixed for a period of time and improving the stability of the learning process.

[0129] The Q-estimation network updates gradients iteratively by minimizing a loss function, which is defined as the square of the difference between the target Q-value and the optimal estimated Q-value, as shown in the following expression:

[0130]

[0131] In the formula, Let γ be the target Q value, ω be the decay factor, and ω be the ω value. - ω represents the target network parameters; Q(s(t),a(t),ω) represents the current estimated Q value, and ω represents the current network parameters.

[0132] The principle block diagram of the deep Q-network algorithm is as follows: Figure 3 As shown, the deep Q-network selects actions based on the ε-greedy algorithm and introduces an experience replay mechanism—the sample pairs (s(t), a(t), reward(t), s(t+1)) obtained after training with a certain step size are stored in the experience pool. During each learning cycle, a portion of samples are randomly selected for training, and the estimated Q-network parameters are updated based on gradient descent. This mechanism improves data utilization while eliminating temporal correlations between samples and avoiding getting trapped in local optima.

[0133] Training stops after the maximum number of training iterations is reached. The target network parameters at this point are saved and applied to the adaptive adjustment of the regularization coefficient of the control strategy under different wave excitation force inputs.

[0134] In summary, the model predictive control method for direct-drive wave power generation systems based on deep Q-networks proposed in this invention can adaptively adjust the regularization coefficients based on deep Q-networks under different wave excitation force input conditions, thereby improving the algorithm's flexibility and enhancing the power capture performance of direct-drive wave power generation systems while satisfying system constraints.

[0135] While the present invention has been disclosed above with reference to preferred embodiments, it is not intended to limit the invention. Those skilled in the art can make various modifications and refinements without departing from the spirit and scope of the invention. Therefore, the scope of protection of the present invention shall be determined by the claims.

Claims

1. A predictive control method for a direct-drive wave power generation system based on a deep Q-network, characterized in that, Includes the following steps: Step S1: Establish the mechanical motion model and linear permanent magnet motor model of the direct-drive wave power generation system; Step S2: Based on the mechanical motion model and linear permanent magnet motor model of the direct-drive wave power generation system, establish the state-space model of the direct-drive wave power generation system based on model predictive control; Step S3: Based on the control objective of the direct-drive wave power generation system, establish a power optimization objective function, and then augment the state space model according to the power optimization objective function and transform it to the discrete domain to obtain the discrete state space model of the direct-drive wave power generation system. Step S4: Set system constraints and perform rolling optimization of state variables on the discrete state-space model until rolling predictive control is completed, thereby realizing online optimization solution of the system; Step S5: Use a deep Q-network to learn the mapping relationship between the optimal value of the regularization coefficient and the wave capture power, and determine the value of the regularization coefficient in the objective function; In step S5, the state space of the deep Q-network used includes the wave peak frequency f. p , wave height h p and the current regularization coefficient r(t); Define the action space as the regularization coefficient increment Δr; The learning objective is to maximize wave capture power when the algorithm has a feasible solution after satisfying system constraints. Therefore, the reward function is defined as follows: 。 2. The predictive control method for a direct-drive wave power generation system based on a deep Q-network according to claim 1, characterized in that, In step S1, the mechanical motion model expression of the direct-drive wave power generation system is established as follows: ； In the formula, M is the mass of the float, and m add For the added mass of water, m add Together with M, they constitute the equivalent total mass of the float; Where z(t) is the heave displacement of the buoy. The velocity of the float. The acceleration of the float; the damping coefficient k(t) and velocity The convolution term corresponds to the radiation force f experienced by the float. R (t); float displacement z(t) and elastic coefficient k s The product of these values ​​corresponds to the equivalent spring resistance f experienced by the float. s (t); f wave This represents the wave excitation force as input; f PTO This indicates the external electromagnetic thrust from the system's power output device; In step S1, the linear permanent magnet motor model expression of the direct-drive wave power generation system is as follows: ； In the formula, u d u q These are the stator d-axis and q-axis voltages, i d i q These are the stator d-axis and q-axis currents, respectively; R s Ψ is the resistance of the stator winding. PM ω is the magnetic flux linkage of the permanent magnet; ω is the electric angular velocity of the linear permanent magnet motor. λ is the pole pitch of the motor; L d L q These are the stator d-axis and q-axis inductors, respectively.

3. The predictive control method for a direct-drive wave power generation system based on a deep Q-network according to claim 2, characterized in that, The linear permanent magnet motor is configured as a surface-mount linear motor, therefore: Let L d =L q =L s =L ss -L M ; Among them, L ss For the self-inductance of the stator winding, L M This refers to the mutual inductance between the stator windings.

4. The predictive control method for a direct-drive wave power generation system based on a deep Q-network according to claim 3, characterized in that, In step S2, the expression for the established state-space model is: ； In the formula, , representing the scaled wave excitation force; , which represents the control variables of the system.

5. The predictive control method for a direct-drive wave power generation system based on a deep Q-network according to claim 4, characterized in that, In step S4, the constraints include linear constraints and nonlinear constraints, wherein the linear constraints include the heave displacement z(t) and velocity of the float. and the external electromagnetic thrust f from the system's power output device PTO The expression is as follows: ； The nonlinear constraints include the stator voltage, stator current, and field weakening flux τ of the linear permanent magnet motor, expressed as follows: ； In the formula, I s This represents the current in the stator winding.

6. A model predictive control system for a direct-drive wave power generation system based on a deep Q-network, characterized in that, include: One or more processors, and Memory used to store operable instructions; The instructions, when executed by the one or more processors, implement the process of model predictive control of the direct-drive wave power generation system based on deep Q-networks as described in any one of claims 1 to 5.

7. A computer-readable medium for storing software, characterized in that, The software includes instructions executable by one or more computers, which, through execution, cause the one or more computers to perform operations, including the process of the model predictive control method for a direct-drive wave power generation system based on a deep Q-network as described in any one of claims 1 to 5.

Images