Data model hybrid driven wind farm modeling method based on PPO algorithm

Abstract

The present invention discloses a data model hybrid-driven wind farm modeling method based on the PPO algorithm, comprising the following steps: S1, collecting measurement data of the wind farm after being disturbed; S2, constructing an equivalent machine model of the wind farm, using differential equations to describe the transient process of the wind farm after being disturbed, and constructing the equivalent machine model based on the differential equation group; S3, parameter initial value setting and sensitivity analysis, and performing trajectory sensitivity analysis on the motor parameters and control parameters of the doubly fed wind turbine; S4, parameter identification based on the PPO reinforcement learning algorithm. The present invention integrates the methods of mechanism model modeling and parameter identification. The established equivalent machine model has clear physical meaning. At the same time, there is no need to know the exact parameters of each wind turbine in advance, which significantly reduces the difficulty of data acquisition. In addition, the deep reinforcement learning algorithm is used to replace the original particle swarm algorithm, which improves the efficiency of parameter identification.

Description

Technical Field

[0001] The present invention belongs to the technical field of power systems, and in particular relates to a data model hybrid driven wind farm modeling method based on a PPO algorithm. Background Art

[0002] With the transformation of traditional power systems to new power systems, the proportion of wind power generation in the total power generation of the power system has gradually increased. However, wind power has strong volatility and weak support, and its dynamic characteristics are quite different from those of traditional synchronous generators. The large-scale grid connection of wind farms will significantly change the stability and power quality of the power system. Therefore, establishing an accurate model of wind farms will help analyze the dynamic characteristics of the power system and thus ensure the safe and stable operation of the power grid.

[0003] Wind farms contain numerous wind turbines. Modeling each turbine individually results in extremely high-order wind farm models and a high computational load, making them difficult to apply to real-time power system calculations. The goal of wind farm modeling is to use one or a few equivalent wind turbines to represent the existing wind farm. This requires two key tasks: determining the model structure of the equivalent wind turbine and identifying the key parameters of the equivalent wind turbine model. Existing research approaches primarily fall into two categories: physical mechanism modeling and parameter identification. Physical mechanism modeling approaches require weighting the parameters of each wind turbine, assuming the parameters of each turbine are known, to obtain the parameters of the equivalent turbine. However, in reality, some wind turbine parameters may be difficult to obtain. Furthermore, motor parameters such as resistance and inductance can change in real time due to magnetic circuit saturation, temperature fluctuations, and skin effect. This uncertainty and dynamic nature of these parameters can reduce the accuracy and robustness of physical mechanism models. In addition, the accuracy of the physical model depends on a good weighting algorithm. A more accurate weighting algorithm often requires weighting based on the output power of each wind turbine, which requires collecting large-scale measurement data to describe the operating status of each wind turbine. This significantly increases the difficulty of data acquisition and the amount of computation required for the algorithm. Parameter identification methods require using a particle swarm algorithm to try out the parameters of the equivalent machine based on the dynamic measurement data of the wind farm after the disturbance, and then performing parameter identification by fitting the disturbance curve. When obtaining the parameters of the equivalent machine, this identification method has problems such as the particle swarm algorithm taking too long, insufficient measurement resulting in an unobservable identification system, and the low sensitivity of the parameter trajectory making it difficult to identify. Therefore, it is difficult to ensure the accuracy and efficiency of wind farm modeling. Summary of the Invention

[0004] The main purpose of the present invention is to overcome the shortcomings and deficiencies of the prior art and propose a data model hybrid driven wind farm modeling method based on the PPO algorithm.

[0005] In order to achieve the above object, the present invention adopts the following technical solutions:

[0006] The data model hybrid driven wind farm modeling method based on the PPO algorithm includes the following steps:

[0007] S1. Collect measurement data of the wind farm after it has been disturbed. After it has been disturbed, collect the voltage and power data of the wind farm before, during and after the disturbance.

[0008] S2. Construct an equivalent machine model of the wind farm, use differential equations to describe the transient process of the wind farm after being disturbed, and construct the equivalent machine model based on the differential equation group;

[0009] S3. Parameter initial value setting and sensitivity analysis: trajectory sensitivity analysis of the motor parameters and control parameters of the doubly fed wind turbine is performed;

[0010] S4. Parameter identification based on PPO reinforcement learning algorithm.

[0011] Furthermore, step S1 is specifically as follows:

[0012] Build a Simulink simulation model of a wind farm containing multiple doubly fed wind turbines, using maximum power point tracking and constant reactive power control.

[0013] Record the change curves of active power and reactive power output by the wind farm during the disturbance.

[0014] Furthermore, step S2 is specifically as follows:

[0015] A set of 13th-order differential equations in the dq0 coordinate system is used to describe the transient process of the wind farm equivalent machine, and an abstract simulation model is built. In software with numerical simulation functions, the basic modules of numerical calculation are used to build the quantitative relationship between various variables. The simulation software calls the appropriate numerical integration method, uses the numerical calculation method to solve the set of differential equations, and calculates the values ​​of each variable at each moment, achieving an effect similar to time domain simulation.

[0016] Furthermore, the 13th-order differential equation system specifically includes:

[0017] Two differential equations describing the electromagnetic transient process of the asynchronous motor rotor winding:

[0018]

[0019]

[0020] Among them, L r The self-inductance of the equivalent two-phase winding of the rotor is expressed as the sum of the excitation inductance and the rotor side leakage inductance L r =L m +L lr , R r Indicates the rotor side resistance, Xm represents the magnetizing reactance, e′ d and e′ q represents the transient potential of the d-axis and q-axis, u rd and u rq represents the d-axis and q-axis components of the rotor side winding terminal voltage, i rd and i rq represents the d-axis and q-axis components of the rotor current, ω s Indicates slip;

[0021] Four differential equations describing the double closed-loop PI control of the rotor-side converter:

[0022]

[0023]

[0024]

[0025]

[0026] Among them, Q * and Q represent the measured values ​​of reactive power reference value, and Indicates the reference value of the inner current loop, and ω m denote the reference value and measured value of the rotor speed, ψ s Indicates the stator flux, L s Represents the self-inductance of the stator equivalent two-phase winding, expressed as the sum of the excitation inductance and the stator side leakage inductance L s =L m +L ls ; These are PI control parameters, and the control parameters of the two current inner loops on the d-axis and q-axis are the same;

[0027] A differential equation describing the rotor motion of an asynchronous motor:

[0028]

[0029] Where J represents the moment of inertia, T m and T e They represent mechanical torque and electromagnetic torque respectively, and D represents the friction coefficient;

[0030] Three differential equations describing the dual closed-loop PI control of the stator-side converter:

[0031]

[0032]

[0033]

[0034] in, and Indicates the reference value of d-axis and q-axis current, i gd and i gq represents the measured values ​​of the d-axis and q-axis currents, and U dc Indicates the reference value and measured value of the DC bus voltage, u s It represents the amplitude of the grid voltage in the dq0 coordinate system, and its conversion relationship with the effective value is expressed as ω represents the grid voltage angular frequency, and L represents the filter inductance; All are PI control parameters;

[0035] Two differential equations describing the current in the stator-side converter filter inductor:

[0036]

[0037]

[0038] Among them, u gd and u gq represents the d-axis and q-axis voltages at the stator-side converter port, i gd and i gq represents the d-axis and q-axis currents of the stator-side converter, and R represents the resistance of the filter circuit;

[0039] A differential equation describing the DC bus capacitor voltage:

[0040]

[0041] Wherein, C represents the capacitance of the DC bus capacitor.

[0042] Furthermore, when there are n wind turbines in a wind farm, the proportional relationship between the parameters of the equivalent machine and the parameters of each wind turbine is expressed as:

[0043]

[0044]

[0045]

[0046]

[0047] Furthermore, the sensitivity analysis is as follows:

[0048] For motor parameters R s , R r , Lls , L lr , L m , T J and rotor-side converter control parameters Analyze the mean square error y between the simulation results of the equivalent machine model and the original wind farm power curve when the above parameters change within a certain range:

[0049]

[0050] Among them, N represents the number of sampling points, P i and Q i represents the active and reactive power of the equivalent machine simulation model at the i-th sampling point, and represents the active and reactive power of the wind farm model at the i-th sampling point, k1 and k2 are coefficients freely selected when measuring the indicators;

[0051] The parameter θ i In its estimated value Nearby intervals Take 100 values ​​θ at equal intervals i (1) ,θ i (2) ...θ i (100) , and let the remaining parameters θ j (j≠i) is equal to the estimated value Run the simulation model for these 100 values ​​and record the mean square error of the corresponding simulation results Where k = 1, 2…100, then the parameter θ i The sensitivity analysis needs to study θ i (k) The changes in The degree of influence, use θ i (k) The absolute value of the partial derivative is expressed as:

[0052]

[0053] If the absolute value calculated according to formula (19) is larger, it means that i Near the kth sampling point, θ i The fluctuation of the parameter θ has a significant impact on the results. i For most sampling points, the calculated partial derivatives are too large, indicating that the parameter θ i The fitting effect of the power curve is more obvious, and it is used as the parameter to be identified.

[0054] Furthermore, due to the difference in the dimensions of the various parameters, normalization is required when comparing the impact of different parameters on the simulation results. To express sensitivity:

[0055]

[0056] Furthermore, the parameter identification based on the PPO reinforcement learning algorithm is as follows:

[0057] First input the power data generated by the original wind farm and and the initial values ​​of the parameters to be identified;

[0058] Based on the PPO reinforcement learning algorithm, the parameters to be identified of the equivalent machine are corrected using continuous step sizes. Several elements of reinforcement learning are state, action, and reward, where the state is a 6-dimensional vector That is, the vector of parameters to be identified. In each step of the action, all variables to be identified will be superimposed with a correction amount So we get the next state

[0059] Correct the parameters of the equivalent machine and run the simulation model to fit the power curves P and Q. If the error between the fitted power curve and the original power curve is small, the result of parameter identification is output. Otherwise, the inverse of the mean square error of the power curve fitting is used as the reward r fed back to the agent by the environment. t , continue training, and calculate the time series difference residual δ t :

[0060]

[0061] Where γ represents the attenuation factor, Represents the state-value function of the strategy π, and the calculated residual is used to update the neural network parameters of the PPO algorithm; the iterative process will gradually correct the parameters to be identified of the equivalent machine and the parameters of the deep reinforcement learning neural network until the error of the power curve reaches the allowable range.

[0062] Furthermore, in the PPO algorithm, the Actor neural network is used to determine the correction value ΔT J ,ΔR s ,ΔR r ,ΔL m , Obey the probability density function;

[0063] Assuming that the correction value of each parameter to be identified obeys the normal distribution, the input vector of the Actor network is the current state vector, and the output vector is the mathematical expectation value of each correction value. Then, sampling is performed in the normal distribution to determine the action vector of this step. The goal of updating the parameters of the Actor network is to optimize the strategy function. The strategy of parameter update iteration is expressed as:

[0064]

[0065] Among them, θ k represents the parameters of the current policy network, Represents the advantage function of the current strategy, ε is a hyperparameter used to limit the truncation range;

[0066] Use Critic neural network to determine the value of a certain state;

[0067] The learning method of temporal difference residual is adopted to reduce the estimation error of the critic network. Its parameter update strategy is expressed as:

[0068]

[0069] Among them, ω represents the parameters of the Critic network, α ω represents the learning rate, δ t represents the time series difference residual, Represents the gradient of the state-value function.

[0070] Furthermore, in the process of training the neural network, it is necessary to use the Adam algorithm as the optimizer. At the same time, it is necessary to normalize the input parameters and perform batch normalization after the activation function of each layer of the neural network to improve the efficiency and convergence of the neural network training.

[0071] Compared with the prior art, the present invention has the following advantages and beneficial effects:

[0072] 1. The present invention adopts a combination of physical models and data-driven methods, taking into account the physical characteristics of wind turbines; the equivalent machine model has a clear physical meaning and is easier to integrate into the stability analysis of the power system. At the same time, it also improves the accuracy of modeling with the help of data-driven methods; combining physical models with data-driven methods can not only consider the main physical mechanisms, but also make full use of the large amount of observed data to improve the accuracy of the model.

[0073] 2. This invention utilizes an abstract 13th-order differential algebraic equation system as a reinforcement learning environment, accelerating parameter identification. Since parameter identification requires frequent simulation model runs, using a detailed wind turbine model simulation would be prohibitively time-consuming. The abstract model established in this invention can fundamentally describe the dynamic characteristics of wind turbine power and speed, as well as the regulation characteristics of the controller. Because the model does not directly simulate the three-phase waveforms, it avoids numerous Parker transform operations, resulting in faster simulation speeds and significantly reducing the time required for parameter identification.

[0074] 3. The present invention takes into account the motor parameters and control parameters of the doubly fed wind turbine, and uses trajectory sensitivity analysis to improve the pertinence of parameter identification; by selecting parameters that have a greater impact on power curve fitting for identification, the dimension of parameter identification can be reduced while ensuring analysis accuracy, thereby improving the efficiency of parameter identification.

[0075] 4. The present invention uses deep reinforcement learning to replace the original particle swarm algorithm to improve the efficiency of parameter identification. BRIEF DESCRIPTION OF THE DRAWINGS

[0076] Figure 1 is a flow chart of the method of the present invention;

[0077] Figure 2 This is a schematic diagram of a typical control method for a doubly fed wind turbine;

[0078] Figure 3 is the power curve of the wind farm during the disturbance;

[0079] Figure 4 It is the sensitivity curve of motor parameters of equivalent machine model;

[0080] Figure 5 It is the sensitivity curve of the control parameters of the equivalent machine model;

[0081] Figure 6 It is a flowchart of parameter identification using the PPO reinforcement learning algorithm. DETAILED DESCRIPTION

[0082] The present invention will be described in further detail below with reference to the embodiments and drawings, but the embodiments of the present invention are not limited thereto.

[0083] Example

[0084] like Figure 1 As shown, the present invention is a data model hybrid driven wind farm modeling method based on the PPO algorithm, comprising the following steps:

[0085] S1. Collecting measurement data of the wind farm after the disturbance. After the disturbance, collect the voltage and power data of the wind farm before, during, and after the disturbance. In this embodiment, specifically:

[0086] Build a Simulink simulation model of a wind farm containing multiple doubly fed wind turbines. The wind turbines use maximum power point tracking and constant reactive power control. The reference value of reactive power needs to ensure that the power factor of the wind farm is maintained at 0.98. The control diagram is shown below. Figure 2 As shown in the figure, the wind speed of the wind farm changes at 2.5 seconds and begins to rise gradually, and the output power gradually increases. At 10 seconds, a voltage sag occurs, and the grid voltage drops from 1 p.u. to 0.7 pu. The change curves of the wind farm's output active power P and reactive power Q during this disturbance are recorded. The power curve is shown in the figure. Figure 3 shown.

[0087] S2. Construct an equivalent machine model of the wind farm, use differential equations to describe the transient process of the wind farm after being disturbed, and construct the equivalent machine model based on the differential equation group;

[0088] S3. Parameter initial value setting and sensitivity analysis: trajectory sensitivity analysis of the motor parameters and control parameters of the doubly fed wind turbine is performed;

[0089] S4. Parameter identification based on PPO reinforcement learning algorithm.

[0090] In this embodiment, step S2 is specifically as follows:

[0091] A 13th-order differential equation system in the dq0 coordinate system is used to describe the transient behavior of a wind farm equivalent machine, and an abstract simulation model is constructed. In practical applications, in software with numerical simulation capabilities (such as Simulink), the basic numerical computation modules are used to establish the quantitative relationships between the various variables. The simulation software then uses appropriate numerical integration methods to solve the differential equation system and calculate the values ​​of each variable at each moment, achieving a similar effect to time-domain simulation.

[0092] In this embodiment, the 13th-order differential equation system specifically includes:

[0093] Two differential equations describing the electromagnetic transient process of the asynchronous motor rotor winding:

[0094]

[0095]

[0096] Among them, L r It represents the self-inductance of the equivalent two-phase winding of the rotor, which can be expressed as the sum of the excitation inductance and the rotor side leakage inductance L r =L m +L lr , R r Indicates the rotor side resistance, X m represents the magnetizing reactance, e′d and e′ q represents the transient potential of the d-axis and q-axis, u rd and u rq represents the d-axis and q-axis components of the rotor side winding terminal voltage, i rd and i rq represents the d-axis and q-axis components of the rotor current, ω s Indicates slip;

[0097] Four differential equations describing the double closed-loop PI control of the rotor-side converter:

[0098]

[0099]

[0100]

[0101]

[0102] Among them, Q * and Q represent the measured values ​​of reactive power reference value, and Indicates the reference value of the inner current loop, and ω m denote the reference value and measured value of the rotor speed, ψ s Indicates the stator flux, L s It represents the self-inductance of the stator equivalent two-phase winding, which can be expressed as the sum of the excitation inductance and the stator side leakage inductance L s =L m +L ls . These are all PI control parameters. Here, the control parameters of the two current inner loops on the d-axis and q-axis are set to be the same;

[0103] A differential equation describing the rotor motion of an asynchronous motor:

[0104]

[0105] Where J represents the moment of inertia, T m and T e They represent mechanical torque and electromagnetic torque respectively, and D represents the friction coefficient;

[0106] Three differential equations describing the dual closed-loop PI control of the stator-side converter:

[0107]

[0108]

[0109]

[0110] in, and Indicates the reference value of d-axis and q-axis current, i gd and i gq represents the measured values ​​of the d-axis and q-axis currents, and U dc Indicates the reference value and measured value of the DC bus voltage, u s It represents the amplitude of the grid voltage in the dq0 coordinate system, and the conversion relationship with the effective value can be expressed as ω represents the grid voltage angular frequency, and L represents the filter inductance; All are PI control parameters;

[0111] Two differential equations describing the current in the stator-side converter filter inductor:

[0112]

[0113]

[0114] Among them, u gd and u gq represents the d-axis and q-axis voltages at the stator-side converter port, i gd and i gq represents the d-axis and q-axis currents of the stator-side converter, and R represents the resistance of the filter circuit;

[0115] A differential equation describing the DC bus capacitor voltage:

[0116]

[0117] Wherein, C represents the capacitance of the DC bus capacitor.

[0118] In this embodiment, when there are n wind turbines in the wind farm, the proportional relationship between the parameters of the equivalent machine and the parameters of each wind turbine is expressed as:

[0119]

[0120]

[0121]

[0122]

[0123] In this embodiment, the sensitivity analysis is specifically as follows:

[0124] For motor parameters R s , R r , L ls , Llr , L m , T J and rotor-side converter control parameters Analyze the mean square error y between the simulation results of the equivalent machine model and the original wind farm power curve when the above parameters change within a certain range:

[0125]

[0126] Among them, N represents the number of sampling points, P i and Q i represents the active and reactive power of the equivalent machine simulation model at the i-th sampling point, and Represents the active and reactive power of the wind farm model at the i-th sampling point, k1 and k2 are coefficients that can be freely selected when measuring indicators. i In its estimated value Nearby intervals Take 100 values ​​θ at equal intervals i (1) ,θ i (2) ...θ i (100) , and let the remaining parameters θ j (j≠i) is equal to the estimated value Run the simulation model for these 100 values ​​and record the mean square error of the corresponding simulation results Where k = 1, 2…100, then the parameter θ can be i The sensitivity analysis needs to study θ i (k) The changes in The degree of influence, use θ i (k) The absolute value of the partial derivative is expressed as:

[0127]

[0128] If the absolute value calculated according to formula (19) is larger, it means that i Near the kth sampling point, θ i The fluctuation of the parameter θ has a significant impact on the results. i For most sampling points, the calculated partial derivatives are too large, indicating that the parameter θ i The fitting effect of the power curve is more obvious, and it is used as the parameter to be identified.

[0129] Since the dimensions of each parameter are different, normalization is required when comparing the influence of different parameters on the simulation results. To express sensitivity:

[0130]

[0131] The sensitivity of motor parameters and motor side control parameters are analyzed respectively. The results are as follows: Figure 4 and Figure 5 As shown. It can be seen that the parameter R s , R r , L m , T J , The sensitivity of is large, and they are used as the parameters to be identified to construct the state vector of reinforcement learning, while the remaining parameters are equal to the estimated values ​​obtained by capacity weighting.

[0132] like Figure 6 As shown in Figure 2, the parameter identification based on the PPO reinforcement learning algorithm is as follows:

[0133] First input the power data generated by the original wind farm and and the initial values ​​of the parameters to be identified;

[0134] Based on the PPO reinforcement learning algorithm, the parameters to be identified of the equivalent machine are corrected using continuous step sizes. Several elements of reinforcement learning are state, action, and reward, where the state is a 6-dimensional vector That is, the vector of parameters to be identified. In each step of the action, all variables to be identified will be superimposed with a correction amount So we get the next state Then, the parameters of the equivalent machine are corrected, and the simulation model is run to fit the power curves P and Q. If the error between the fitted power curve and the original power curve is small, the result of parameter identification is output. Otherwise, the inverse of the mean square error of the power curve fitting is used as the reward r fed back to the agent by the environment. t , continue training, and calculate the time series difference residual δ t :

[0135]

[0136] Where γ represents the attenuation factor, Represents the state-value function of the strategy π, and the calculated residual is used to update the neural network parameters of the PPO algorithm; the iterative process will gradually correct the parameters to be identified of the equivalent machine and the parameters of the deep reinforcement learning neural network until the error of the power curve reaches the allowable range.

[0137] In the PPO algorithm, the Actor neural network is used to determine the correction value ΔT J ,ΔR s ,ΔR r ,ΔL m , Obey the probability density function;

[0138] Assuming that the correction value of each parameter to be identified obeys the normal distribution, the input vector of the Actor network is the current state vector, and the output vector is the mathematical expectation value of each correction value. Then, sampling is performed in the normal distribution to determine the action vector of this step. The goal of updating the parameters of the Actor network is to optimize the strategy function. The strategy of parameter update iteration is expressed as:

[0139]

[0140] Among them, θ k represents the parameters of the current policy network, Represents the advantage function of the current strategy, ε is a hyperparameter used to limit the truncation range;

[0141] Use Critic neural network to determine the value of a certain state;

[0142] The learning method of temporal difference residual is adopted to reduce the estimation error of the critic network. Its parameter update strategy is expressed as:

[0143]

[0144] Among them, ω represents the parameters of the Critic network, α ω represents the learning rate, δ t represents the time series difference residual, Represents the gradient of the state-value function.

[0145] In the process of training neural networks, the Adam algorithm needs to be used as the optimizer. At the same time, the input parameters need to be normalized, and batch normalization processing needs to be performed after the activation function of each layer of the neural network to improve the efficiency and convergence of neural network training.

[0146] The PPO reinforcement learning algorithm can gradually reduce the error between the power curves of the equivalent machine model and the original wind farm model, ultimately identifying a set of parameters that fit the dynamic characteristics of the wind farm. After each identification, the parameters of the Actor and Critic neural networks are saved. If the equivalent model of the wind farm needs to be updated later, or if parameters of wind farms with similar characteristics need to be identified, further training can be performed based on this neural network, leveraging the transferability of neural networks to further improve the efficiency of parameter identification.

[0147] It should also be noted that, in this specification, terms such as "comprises", "includes" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or apparatus comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or apparatus. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of other identical elements in the process, method, article or apparatus comprising the element.

[0148] The above description of the disclosed embodiments is intended to enable one skilled in the art to implement or use the present invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention is not limited to the embodiments shown herein but is intended to conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A data model hybrid driven wind farm modeling method based on the PPO algorithm is characterized by: The following steps are involved: S1. Collect measurement data of the wind farm after it has been disturbed. After it has been disturbed, collect the voltage and power data of the wind farm before, during and after the disturbance. S2. Construct an equivalent machine model of the wind farm, use differential equations to describe the transient process of the wind farm after being disturbed, and construct the equivalent machine model based on the differential equation group; S3. Parameter initial value setting and sensitivity analysis: trajectory sensitivity analysis of the motor parameters and control parameters of the doubly fed wind turbine is performed; S4, parameter identification based on PPO reinforcement learning algorithm; The parameter identification based on the PPO reinforcement learning algorithm is as follows: First input the power data generated by the original wind farm and and the initial values ​​of the parameters to be identified; Based on the PPO reinforcement learning algorithm, the parameters to be identified of the equivalent machine are corrected using continuous step sizes. Several elements of reinforcement learning are state, action, and reward, where the state is a 6-dimensional vector That is, the vector of parameters to be identified. In each step of the action, all variables to be identified will be superimposed with a correction value, that is, the action vector So we get the next state Correct the parameters of the equivalent machine and run the simulation model to fit the power curves P and Q. If the error between the fitted power curve and the original power curve is small, the result of parameter identification is output. Otherwise, the inverse of the mean square error of the power curve fitting is used as the reward r fed back to the agent by the environment. t , continue training, and calculate the time series difference residual δ t : Where γ represents the attenuation factor, The state value function of the policy π is represented by the calculated residual, which is used to update the neural network parameters of the PPO algorithm. The iterative process will gradually correct the parameters to be identified of the equivalent machine and the parameters of the deep reinforcement learning neural network until the error of the power curve reaches the allowable range. In the PPO algorithm, the Actor neural network is used to determine the correction value ΔT J ,ΔR s ,ΔR r ,ΔL m , Obey the probability density function; Assuming that the correction value of each parameter to be identified obeys the normal distribution, the input vector of the Actor network is the current state vector, and the output vector is the mathematical expectation value of each correction value. Then, sampling is performed in the normal distribution to determine the action vector of this step. The goal of updating the parameters of the Actor network is to optimize the strategy function. The strategy of parameter update iteration is expressed as: Among them, θ k represents the parameters of the current policy network, Represents the advantage function of the current strategy, ε is a hyperparameter used to limit the truncation range; Use Critic neural network to determine the value of a certain state; The learning method of temporal difference residual is adopted to reduce the estimation error of the critic network. Its parameter update strategy is expressed as: Among them, ω represents the parameters of the Critic network, α ω represents the learning rate, δ t represents the time series difference residual, represents the gradient of the state value function; In the process of training neural networks, the Adam algorithm needs to be used as the optimizer. At the same time, the input parameters need to be normalized, and batch normalization processing needs to be performed after the activation function of each layer of the neural network to improve the efficiency and convergence of neural network training.

2. The data model hybrid driven wind farm modeling method based on the PPO algorithm according to claim 1 is characterized in that: Step S1 is specifically as follows: Build a Simulink simulation model of a wind farm containing multiple doubly fed wind turbines, using maximum power point tracking and constant reactive power control. Record the change curves of active power and reactive power output by the wind farm during the disturbance.

3. The data model hybrid driven wind farm modeling method based on the PPO algorithm according to claim 1 is characterized in that: Step S2 is specifically as follows: A set of 13th-order differential equations in the dq0 coordinate system is used to describe the transient process of the wind farm equivalent machine, and an abstract simulation model is built. In software with numerical simulation functions, the basic modules of numerical calculation are used to build the quantitative relationship between various variables. The simulation software calls the appropriate numerical integration method, uses the numerical calculation method to solve the set of differential equations, and calculates the values ​​of each variable at each moment, achieving an effect similar to time domain simulation.

4. The data model hybrid driven wind farm modeling method based on the PPO algorithm according to claim 3 is characterized in that: The 13th-order differential equation system specifically includes: Two differential equations describing the electromagnetic transient process of the asynchronous motor rotor winding: Among them, L r The self-inductance of the equivalent two-phase winding of the rotor is expressed as the sum of the excitation inductance and the rotor side leakage inductance L r =L m +L lr , R r Indicates the rotor side resistance, X m represents the magnetizing reactance, e′ d and e q ′ represents the transient potential of the d-axis and q-axis, u rd and u rq represents the d-axis and q-axis components of the rotor side winding terminal voltage, i rd and i rq represents the d-axis and q-axis components of the rotor current, ω s Indicates slip; Four differential equations describing the double closed-loop PI control of the rotor-side converter: Among them, Q * and Q represent the measured values ​​of reactive power reference value, and Indicates the reference value of the inner current loop, and ω m denote the reference value and measured value of the rotor speed, ψ s Indicates the stator flux, L s Represents the self-inductance of the stator equivalent two-phase winding, expressed as the sum of the excitation inductance and the stator side leakage inductance L s =L m +L ls ; These are PI control parameters, and the control parameters of the two current inner loops on the d-axis and q-axis are the same; A differential equation describing the rotor motion of an asynchronous motor: Where J represents the moment of inertia, T m and T e They represent mechanical torque and electromagnetic torque respectively, and D represents the friction coefficient; Three differential equations describing the dual closed-loop PI control of the stator-side converter: in, and Indicates the reference value of d-axis and q-axis current, i gd and i gq represents the measured values ​​of the d-axis and q-axis currents, and U dc Indicates the reference value and measured value of the DC bus voltage, u s It represents the amplitude of the grid voltage in the dq0 coordinate system, and its conversion relationship with the effective value is expressed as ω represents the grid voltage angular frequency, and L represents the filter inductance; All are PI control parameters; Two differential equations describing the current in the stator-side converter filter inductor: Among them, u gd and u gq represents the d-axis and q-axis voltages at the stator-side converter port, i gd and i gq represents the d-axis and q-axis currents of the stator-side converter, and R represents the resistance of the filter circuit; A differential equation describing the DC bus capacitor voltage: Where C represents the capacitance of the DC bus capacitor.

5. The data model hybrid driven wind farm modeling method based on the PPO algorithm according to claim 4 is characterized in that: When there are n wind turbines in a wind farm, the proportional relationship between the parameters of the equivalent machine and the parameters of each wind turbine is expressed as:

6. The data model hybrid driven wind farm modeling method based on the PPO algorithm according to claim 1 is characterized in that: The sensitivity analysis is as follows: For motor parameters R s , R r , L ls , L lr , L m , T J and rotor-side converter control parameters Analyze the mean square error y between the simulation results of the equivalent machine model and the original wind farm power curve when the above parameters change within a certain range: Among them, N represents the number of sampling points, P i and Q i represents the active and reactive power of the equivalent machine simulation model at the i-th sampling point, and represents the active and reactive power of the wind farm model at the i-th sampling point, k1 and k2 are coefficients freely selected when measuring the indicators; The parameter θ i In its estimated value Nearby intervals Take 100 values ​​θ at equal intervals i (1) ,θ i (2) ...θ i (100) , and let the remaining parameters θ j (j≠i) is equal to the estimated value Run the simulation model for these 100 values ​​and record the mean square error of the corresponding simulation results Where k = 1, 2…100, then the parameter θ i The sensitivity analysis needs to study θ i (k) The changes in The degree of influence, use θ i (k) The absolute value of the partial derivative is expressed as: If the absolute value calculated according to formula (19) is larger, it means that i Near the kth sampling point, θ i The fluctuation of the parameter θ has a significant impact on the results. i For most sampling points, the calculated partial derivatives are too large, indicating that the parameter θ i The fitting effect of the power curve is more obvious, and it is used as the parameter to be identified.

7. The data model hybrid driven wind farm modeling method based on the PPO algorithm according to claim 6 is characterized in that: Since the dimensions of each parameter are different, normalization is required when comparing the influence of different parameters on the simulation results. To express sensitivity:

Images