Control method and system for improving broadband oscillation stability of offshore wind power grid-connected system

Abstract

The invention discloses a control method and system for improving broadband oscillation stability of an offshore wind power grid-connected system. The method comprises the following steps: designing an action variable set based on a transfer function of a phase-locked loop of a bisecond-order generalized integrator; acquiring real-time information of each electrical quantity at the network side of the offshore wind power grid-connected system and establishing an environment state variable set; establishing a reward function based on a time scale according to the broadband oscillation characteristic of the offshore wind power grid-connected system; performing off-line training on the near-end strategy optimization algorithm agent under the condition that different oscillations occur in the power grid to obtain a phase-locked loop parameter control strategy; and when an actual system detects that the power grid generates broadband oscillation, the trained intelligent agent is put into use, and the phase-locked loop parameters are controlled in real time. According to the broadband oscillation phase-locked loop parameter real-time control method in which the near-end strategy optimization algorithm intelligent agent participates, real-time control over the phase-locked loop parameters is achieved, the stability of an offshore wind power grid-connected system is improved, and broadband oscillation is avoided.

Description

Technical Field

[0001] The present invention relates to the technical field of power electronic power system stability control, and in particular to a control method and system for improving the broadband oscillation stability of an offshore wind power grid-connected system. Background Art

[0002] As the proportion of offshore wind power installed capacity in the power system gradually increases, the access of offshore wind power has brought great challenges to the reliability and stability of the power grid.

[0003] When broadband oscillation occurs in the power grid, it may cause offshore wind farms to be disconnected from the grid, resulting in a large power shortage in the power system, and may even further lead to the decoupling of power systems in some local areas and large-scale power outages, posing a threat to the safe and stable operation of the entire power system that cannot be ignored.

[0004] The actual operating conditions of offshore wind power grid-connected systems are changeable and complex. The traditional method of setting the phase-locked loop parameters based on the transfer function of the dual second-order generalized integrator phase-locked loop does not combine the parameter setting process with the actual operating conditions of the offshore wind power grid-connected system when broadband oscillation occurs, and cannot give reasonable parameter settings based on its operating conditions, resulting in the stability of the offshore wind power grid-connected system under broadband oscillation failing to meet the expected requirements. In addition, the performance of the dual second-order generalized integrator phase-locked loop is determined by k s , ζ, ω n The traditional method does not consider k s and ζ、ω n The mutual influence between them leads to unreasonable parameter settings.

[0005] Obviously, with the increasing complexity and uncertainty of the power system, it is difficult for traditional optimization methods based on physical models to establish accurate models for real-time and rapid solutions. Summary of the invention

[0006] The present invention provides a control method and system for improving the broadband oscillation stability of an offshore wind power grid-connected system. By using a deep reinforcement learning algorithm, a control strategy for phase-locked loop parameters can be obtained effectively under different broadband oscillation conditions for the complex operating conditions of the offshore wind power grid-connected system, so that the offshore wind power grid-connected system can maintain stability under broadband oscillation, thereby ensuring the safe and stable operation of the offshore wind power grid-connected system.

[0007] A first aspect of the present invention provides a control method for improving the broadband oscillation stability of an offshore wind power grid-connected system, comprising:

[0008] Based on the transfer function of the dual second-order generalized integrator phase-locked loop, the generalized integrator control parameters and the synchronous coordinate system phase-locked loop control parameters are selected as the input variables of the action variable set of the intelligent agent;

[0009] Acquire real-time information of various electrical quantities on the grid side of the offshore wind power grid-connected system and establish a set of environmental state variables of the intelligent agent;

[0010] According to the characteristics of broadband oscillation of offshore wind power grid-connected system, the reward function R1 in the initial stage of oscillation, the reward function R2 in the continuous stage of oscillation and the reward function R3 in the recovery stage of oscillation are set based on the time scale, so as to obtain the overall reward function R=R1+R2+R3 of the agent;

[0011] Based on the action variable set, the environment state variable set and the overall reward function R, the agent is offline trained when low-frequency oscillation, sub- / super-synchronous oscillation and high-frequency oscillation occur in the power grid, wherein the agent is a proximal strategy optimization algorithm agent improved by introducing an approximate objective function optimization strategy;

[0012] When broadband oscillation occurs in the power grid, the trained intelligent agent is used to perform online real-time control on the phase-locked loop parameters.

[0013] Optionally, the biquad generalized integrator phase-locked loop transfer function includes:

[0014] The transfer function of the generalized integrator in the biquadric generalized integrator phase-locked loop is:

[0015] ;

[0016] In the formula, and is the transfer function of the generalized integrator in the biquadric generalized integrator phase-locked loop, k s is the control parameter in the generalized integrator, s is the Laplace operator, and ω is the frequency;

[0017] The transfer function of the synchronous frame phase-locked loop in the biquad generalized integrator is:

[0018] ;

[0019] In the formula, ζ, ω n It is the control parameter of the phase-locked loop of the synchronous coordinate system.

[0020] Optionally, the action variable set A is:

[0021] ;

[0022] In the formula, k s (t) represents the k at time t s The value of ζ(t) represents the value of ζ at time t, ω n (t) represents ω at time t n The value of .

[0023] Optionally, the environmental state variable set S is:

[0024] ;

[0025] In the formula, u dg (t) is the grid-side d-axis voltage at time t, u qg (t) is the grid-side q-axis voltage at time t, i dg (t) is the grid-side d-axis current at time t, i qg (t) is the q-axis current on the grid side at time t, P(t) is the active power on the grid side at time t, and Q(t) is the reactive power on the grid side at time t.

[0026] Optionally, setting the reward function R1 in the initial stage of oscillation includes:

[0027] In the first preset time after the oscillation occurs, the reward function in the initial stage of the oscillation is set to:

[0028] ;

[0029] In the formula, Δu qg (t) is the deviation of the grid-side q-axis voltage at time t, th is the set threshold, λ1 and λ2 are the weight coefficients of the reward function, The amount of punishment.

[0030] Optionally, setting the oscillation duration phase reward function R2 includes:

[0031] From the first preset time after the oscillation occurs to the end of the oscillation, the reward function of the oscillation continuation stage is set to:

[0032] ;

[0033] In the formula, Δi dg (t) is the deviation of the grid-side d-axis current at time t, and μ1 and μ2 are the weighting coefficients of the reward function.

[0034] Optionally, setting the oscillation recovery phase reward function R3 includes:

[0035] After the second preset time after the oscillation ends, the reward function of the oscillation recovery phase is set to:

[0036] ;

[0037] In the formula, Δu dg (t) is the deviation of the grid-side d-axis voltage at time t, and δ1 and δ2 are the weighting coefficients of the reward function.

[0038] Optionally, the architecture of the intelligent agent is: Actor-Critic architecture.

[0039] Optionally, during the training process, the update of the Actor network parameter r and the Critic network network parameters θ, ω are respectively achieved by gradient descent of the loss function and minimization of the loss function;

[0040] The loss function of the Actor network is expressed as:

[0041] ;

[0042] In the formula, A(s t ,a t ) is represented as in state s t and take action a t The advantage function when ε is the clipping factor, r t (θ) is the importance sampling ratio, where A(s t ,a t ) and r t The expression of (θ) is:

[0043] ;

[0044] ;

[0045] In the formula, is the discount factor, Indicates is the policy function with parameters, V ω (s t ) is the output value of the Critic network at time t, y t is the value of V at time t+1 ω (s t ), R t is the reward value at time t;

[0046] The update formula of the network parameter θ of the Actor network is:

[0047] ;

[0048] Where μ is the learning rate of the Actor network;

[0049] The optimization function of the Critic network and the update formula of its network parameter ω are as follows:

[0050] ;

[0051] Where σ is the learning rate of the Critic network, is the gradient operator, is the optimization function of the Critic network.

[0052] A second aspect of the present invention provides a control system for improving the broadband oscillation stability of an offshore wind power grid-connected system, comprising:

[0053] A first construction unit is used to select a generalized integrator control parameter and a synchronous coordinate system phase-locked loop control parameter as an input variable of an action variable set of an intelligent agent based on a dual second-order generalized integrator phase-locked loop transfer function;

[0054] The second construction unit is used to obtain real-time information of various electrical quantities on the grid side of the offshore wind power grid-connected system and establish an environmental state variable set of the intelligent agent;

[0055] The third construction unit is used to set the reward function R1 in the initial stage of oscillation, the reward function R2 in the continuous stage of oscillation and the reward function R3 in the recovery stage of oscillation based on the time scale according to the characteristics of the broadband oscillation of the offshore wind power grid-connected system, so as to obtain the overall reward function R=R1+R2+R3 of the intelligent agent;

[0056] A training unit is used to perform offline training on the agent based on the action variable set, the environment state variable set and the overall reward function R, respectively, when low-frequency oscillation, sub- / super-synchronous oscillation and high-frequency oscillation occur in the power grid, wherein the agent is a proximal strategy optimization algorithm agent improved by introducing an approximate objective function optimization strategy;

[0057] The control unit is used to perform online real-time control of the phase-locked loop parameters through the trained intelligent agent when broadband oscillation occurs in the power grid.

[0058] It can be seen from the above technical solutions that the present invention has the following advantages:

[0059] 1. The present invention adopts a control strategy based on a deep reinforcement learning algorithm, which can effectively take into account the optimization control of the hybrid objectives of stability and dynamic performance, and shows good environmental adaptability.

[0060] 2. In the process of training the intelligent agent, the present invention fully considers the broadband oscillation under different frequencies, and can improve the stability of the offshore wind power system under broadband oscillation.

[0061] 3. The reward function set in the present invention comprehensively considers the different characteristics of each stage of broadband oscillation and can achieve global optimization control. BRIEF DESCRIPTION OF THE DRAWINGS

[0062] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0063] Figure 1 A flow chart of a control method for improving broadband oscillation stability of an offshore wind power grid-connected system provided by an embodiment of the present invention;

[0064] Figure 2 A structural diagram of a direct-drive wind turbine provided by an embodiment of the present invention incorporated into a weak power grid system;

[0065] Figure 3 A structural diagram of a dual second-order generalized integrator phase-locked loop provided in an embodiment of the present invention;

[0066] Figure 4 A structural diagram of a proximal strategy optimization algorithm provided by an embodiment of the present invention;

[0067] Figure 5 The overall framework of the real-time control method of phase-locked loop parameters provided by the embodiment of the present invention;

[0068] Figure 6 A schematic structural diagram of a control system for improving the broadband oscillation stability of an offshore wind power grid-connected system provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0069] In order to make the purpose, features and advantages of the present invention more obvious and easy to understand, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the embodiments described below are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0070] Embodiment 1:

[0071] like Figure 2 As shown, this embodiment takes the incorporation of a direct-drive wind turbine into a weak power grid system as an example, the wind turbine is directly connected to a direct-drive permanent magnet synchronous generator (PMSG), and is connected to a weak AC power grid after a back-to-back converter.

[0072] The structure of the dual second-order generalized integrator phase-locked loop is shown in the figure below: Figure 3 As shown, the three-phase voltage u A 、uB 、u C The voltage u of the αβ axis is obtained by Clark transformer α 、u β ; The generated u α 、u β After the second order generalized integrator (SOGI), 90 ° The phase shift is realized, and the positive and negative sequence voltages are separated through positive and negative sequence calculation; after the positive and negative sequence voltages are separated, the phase is detected through a traditional phase-locked loop controller.

[0073] See also Figure 1 A control method for improving the broadband oscillation stability of an offshore wind power grid-connected system provided in an embodiment of the present invention includes:

[0074] Step 101: Based on the dual second-order generalized integrator phase-locked loop transfer function, select the generalized integrator control parameters and the synchronous coordinate system phase-locked loop control parameters as input variables of the action variable set of the intelligent agent.

[0075] It should be noted that, in this embodiment, the generalized integrator control parameter k is selected based on the biquad generalized integrator phase-locked loop transfer function. s And the synchronous coordinate system phase-locked loop control parameters ζ, ω n As the input variable of the agent's action variable set A.

[0076] Wherein, the phase-locked loop transfer function is as follows:

[0077] The transfer function of the generalized integrator in the biquadric generalized integrator phase-locked loop is:

[0078] (1)

[0079] In the formula, k s is the control parameter in the generalized integrator.

[0080] The transfer function of the synchronous frame phase-locked loop in the biquad generalized integrator is:

[0081] (2)

[0082] In the formula, ζ, ω n It is the control parameter of the phase-locked loop of the synchronous coordinate system.

[0083] The action variable set A is:

[0084] (3)

[0085] In the formula, k s (t) represents the k at time ts The value of ζ(t) represents the value of ζ at time t, ω n (t) represents ω at time t n The value of .

[0086] Step 102: Acquire real-time information of various electrical quantities on the grid side of the offshore wind power grid-connected system and establish a set of environmental state variables of the intelligent agent.

[0087] It should be noted that, in this embodiment, the real-time information of various electrical quantities on the grid side of the offshore wind power grid-connected system is obtained through the wide-area measurement system of the power system and a set S of environmental state variables of the intelligent agent is established.

[0088] The set of environmental state variables S is:

[0089] (4)

[0090] In the formula, u dg (t) is the grid-side d-axis voltage at time t, u qg (t) is the grid-side q-axis voltage at time t, i dg (t) is the grid-side d-axis current at time t, i qg (t) is the q-axis current on the grid side at time t, P(t) is the active power on the grid side at time t, and Q(t) is the reactive power on the grid side at time t.

[0091] Step 103, according to the characteristics of broadband oscillation of offshore wind power grid-connected system, the reward function R1 in the initial stage of oscillation, the reward function R2 in the continuous stage of oscillation and the reward function R3 in the recovery stage of oscillation are set based on the time scale, so as to obtain the overall reward function R=R1+R2+R3 of the intelligent agent.

[0092] It should be noted that the reward function R1 in the initial stage of oscillation in this embodiment is:

[0093] In the first preset time after the oscillation occurs, the reward function in the initial stage of the oscillation is set to:

[0094] (5)

[0095] In the formula, Δu qg (t) is the deviation of the grid-side q-axis voltage at time t, th is the set threshold, and λ1 and λ2 are the weighting coefficients of the reward function.

[0096] The reward function R2 during the oscillation continuation phase is:

[0097] From the first preset time after the oscillation occurs to the end of the oscillation, the reward function of the oscillation continuation stage is set to:

[0098] (6)

[0099] In the formula, Δi dg (t) is the deviation of the grid-side d-axis current at time t, and μ1 and μ2 are the weighting coefficients of the reward function.

[0100] The reward function R3 in the oscillation recovery phase is:

[0101] After the second preset time after the oscillation ends, the reward function of the oscillation recovery phase is set to:

[0102] (7)

[0103] In the formula, Δu dg (t) is the deviation of the grid-side d-axis voltage at time t, and δ1 and δ2 are the weighting coefficients of the reward function.

[0104] Preferably, the first preset time is 0.1s, and the second preset time is 0.2s.

[0105] Step 104: Based on the action variable set, the environment state variable set and the overall reward function R, the agent is offline trained when low-frequency oscillation, sub- / super-synchronous oscillation and high-frequency oscillation occur in the power grid, wherein the agent is a proximal strategy optimization algorithm agent improved by introducing an approximate objective function optimization strategy.

[0106] It should be noted that when low-frequency oscillation, sub / supersynchronous oscillation and high-frequency oscillation occur in the power grid, the proximal strategy optimization algorithm agent is trained offline to obtain the phase-locked loop parameter control strategy.

[0107] like Figure 4 As shown in the figure, the proximal policy optimization algorithm agent uses an actor-critic architecture and introduces an approximate objective function optimization strategy. This approximate objective function limits the difference between the new and old strategies to a smaller range, reduces the variance in the policy update process, and improves the stability of the learning effect.

[0108] During the training process, the update of the Actor network and Critic network parameters θ, ω is achieved by gradient descent of the loss function and minimization of the loss function respectively. The loss function of the Actor network can be expressed as:

[0109] (8)

[0110] In the formula, A(s t ,a t ) is represented as in state s t and take action a t The advantage function when ε is the clipping factor, which is a hyperparameter used to measure the degree of deviation of the new and old network parameters of the Actor. t(θ) is the importance sampling ratio, where A(s t ,a t ) and r t The expression of (θ) is:

[0111] (9)

[0112] (10)

[0113] Where V ω (s t ) is the output value of the Critic network at time t, y t is the value of V at time t+1 ω (s t ), R t is the reward value at time t.

[0114] The update formula of Actor network parameter θ is:

[0115] (11)

[0116] Where μ is the learning rate of the Actor network.

[0117] The optimization function of the Critic network and the update formula of its network parameter ω are:

[0118] (12)

[0119] Where σ is the learning rate of the Critic network.

[0120] See also Figure 4 , the offline training process of the agent is as follows:

[0121] Initialize the agent's Actor network parameters and Critic network parameters; perform actions in the simulation environment and collect state transition and reward data; use the collected data to update the agent's network parameters; repeat the above steps until the agent's strategy converges.

[0122] Step 105: When broadband oscillation occurs in the power grid, the trained intelligent agent performs online real-time control on the phase-locked loop parameters.

[0123] It should be noted that when the actual system detects that the power grid has broadband oscillation, the trained agent is put into use to perform online real-time control of the phase-locked loop parameters. The specific agent receives real-time status data from the wind power grid-connected system; the agent calculates the optimal adjustment strategy of the phase-locked loop parameters using a deep reinforcement learning algorithm based on the real-time status data and the preset optimization goal; the agent sends the calculated phase-locked loop parameter adjustment strategy to the phase-locked loop system in real time to achieve online real-time control of the phase-locked loop parameters, such as Figure 5 shown.

[0124] As an example, the specific parameters of the direct-drive wind turbine connected to the weak power grid system are given as follows:

[0125] Table 1 Parameters of direct-drive wind turbines connected to weak grid system

[0126]

[0127] For the proximal policy optimization algorithm, the number of layers of the deep neural network and the number of neurons in each layer affect the performance and training effect of the algorithm. The Critic network and Actor network used in this study both have three hidden layers, each containing 100, 50 and 100 neurons respectively. The hyperparameters of the proximal policy optimization algorithm include the learning rate of the Actor and Critic, the capacity of the experience replay pool, the experience replay training batch, the clipping factor and the depreciation factor. These hyperparameters need to be continuously adjusted during the training process so that the cumulative reward curve can rise quickly and gradually converge. The specific settings are shown in the following table:

[0128] Table 2 PPO hyperparameter settings

[0129]

[0130] The present invention provides a phase-locked loop real-time control method for improving the broadband oscillation stability of an offshore wind power grid-connected system, and applies a proximal strategy optimization algorithm to the real-time control problem of phase-locked loop parameters of an offshore wind power grid-connected system; the real-time control method for broadband oscillation phase-locked loop parameters involving an intelligent agent of the proximal strategy optimization algorithm realizes real-time control of phase-locked loop parameters, improves the stability of the offshore wind power grid-connected system, and avoids the occurrence of broadband oscillations.

[0131] The above is a control method for improving the broadband oscillation stability of an offshore wind power grid-connected system provided in an embodiment of the present invention. The following is a control system for improving the broadband oscillation stability of an offshore wind power grid-connected system provided in an embodiment of the present invention.

[0132] See also Figure 6 A control system for improving the broadband oscillation stability of an offshore wind power grid-connected system provided in an embodiment of the present invention includes:

[0133] The first construction unit 201 is used to select generalized integrator control parameters and synchronous coordinate system phase-locked loop control parameters as input variables of the action variable set A based on the dual second-order generalized integrator phase-locked loop transfer function.

[0134] The second building unit 202 is used to obtain the real-time information of various electrical quantities on the grid side of the offshore wind power grid-connected system and establish an environmental state variable set S.

[0135] The third construction unit 203 is used to set the reward function R1 in the initial stage of oscillation, the reward function R2 in the continuous stage of oscillation and the reward function R3 in the recovery stage of oscillation based on the time scale according to the characteristics of the broadband oscillation of the offshore wind power grid-connected system, so as to obtain the overall reward function R=R1+R2+R3.

[0136] The training unit 204 is used to perform offline training on the intelligent agent when low-frequency oscillation, sub- / super-synchronous oscillation and high-frequency oscillation occur in the power grid, wherein the intelligent agent is a proximal strategy optimization algorithm intelligent agent improved by introducing an approximate objective function optimization strategy.

[0137] The control unit 205 is used to perform online real-time control of the phase-locked loop parameters through the trained intelligent agent when broadband oscillation occurs in the power grid.

[0138] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0139] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0140] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0141] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0142] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the methods of each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc. Various media that can store program codes.

[0143] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A control method for improving the broadband oscillation stability of an offshore wind power grid-connected system, characterized in that: include: Based on the transfer function of the dual second-order generalized integrator phase-locked loop, the generalized integrator control parameters and the synchronous coordinate system phase-locked loop control parameters are selected as the input variables of the action variable set of the intelligent agent; Acquire real-time information of various electrical quantities on the grid side of the offshore wind power grid-connected system and establish a set of environmental state variables of the intelligent agent; According to the characteristics of broadband oscillation of offshore wind power grid-connected system, the reward function R1 in the initial stage of oscillation, the reward function R2 in the continuous stage of oscillation and the reward function R3 in the recovery stage of oscillation are set based on the time scale, so as to obtain the overall reward function R=R1+R2+R3 of the agent; Based on the action variable set, the environment state variable set and the overall reward function R, the agent is offline trained when low-frequency oscillation, sub- / super-synchronous oscillation and high-frequency oscillation occur in the power grid, wherein the agent is a proximal strategy optimization algorithm agent improved by introducing an approximate objective function optimization strategy; When broadband oscillation occurs in the power grid, the trained intelligent agent is used to perform online real-time control on the phase-locked loop parameters.

2. The control method for improving broadband oscillation stability of offshore wind power grid-connected system according to claim 1, characterized in that: The biquad generalized integrator phase-locked loop transfer function includes: The transfer function of the generalized integrator in the biquadric generalized integrator phase-locked loop is: ； In the formula, and is the transfer function of the generalized integrator in the biquadric generalized integrator phase-locked loop, k s is the control parameter in the generalized integrator, s is the Laplace operator, and ω is the frequency; The transfer function of the synchronous frame phase-locked loop in the biquad generalized integrator is: ； In the formula, ζ, ω n It is the control parameter of the phase-locked loop of the synchronous coordinate system.

3. The control method for improving the broadband oscillation stability of an offshore wind power grid-connected system according to claim 2, characterized in that: The action variable set A is: ； In the formula, k s (t) represents the k at time t s The value of ζ(t) represents the value of ζ at time t, ω n (t) represents ω at time t n The value of .

4. The control method for improving broadband oscillation stability of offshore wind power grid-connected system according to claim 1, characterized in that: The environmental state variable set S is: ； In the formula, u dg (t) is the grid-side d-axis voltage at time t, u qg (t) is the grid-side q-axis voltage at time t, i dg (t) is the grid-side d-axis current at time t, i qg (t) is the q-axis current on the grid side at time t, P(t) is the active power on the grid side at time t, and Q(t) is the reactive power on the grid side at time t.

5. The control method for improving broadband oscillation stability of offshore wind power grid-connected system according to claim 1, characterized in that: Setting the reward function R1 in the initial stage of the oscillation includes: In the first preset time after the oscillation occurs, the reward function in the initial stage of the oscillation is set to: ； In the formula, Δu qg (t) is the deviation of the grid-side q-axis voltage at time t, th is the set threshold, λ1 and λ2 are the weight coefficients of the reward function, The amount of punishment.

6. The control method for improving broadband oscillation stability of offshore wind power grid-connected system according to claim 1, characterized in that: Setting the oscillation duration phase reward function R2 includes: From the first preset time after the oscillation occurs to the end of the oscillation, the reward function of the oscillation continuation stage is set to: ； In the formula, Δu qg (t) is the deviation of the grid-side q-axis voltage at time t, Δi dg (t) is the deviation of the grid-side d-axis current at time t, μ1 and μ2 are the weight coefficients of the reward function, is the average reactive power during the oscillation period.

7. The control method for improving broadband oscillation stability of offshore wind power grid-connected system according to claim 1, characterized in that: Setting the oscillation recovery phase reward function R3 includes: After the second preset time after the oscillation ends, the reward function of the oscillation recovery phase is set to: ； In the formula, Δu dg (t) is the deviation of the grid-side d-axis voltage at time t, Δi dg (t) is the deviation of the grid-side d-axis current at time t, and δ1 and δ2 are the weighting coefficients of the reward function.

8. The control method for improving broadband oscillation stability of offshore wind power grid-connected system according to claim 1, characterized in that: The architecture of the intelligent agent is: Actor-Critic architecture.

9. The control method for improving broadband oscillation stability of offshore wind power grid-connected system according to claim 8, characterized in that: During the training process, the update of the Actor network parameter r and the Critic network network parameters θ, ω is achieved by gradient descent of the loss function and minimization of the loss function respectively; The loss function of the Actor network is expressed as: ； In the formula, A(s t ,a t ) is represented as in state s t and take action a t The advantage function when ε is the clipping factor, r t (θ) is the importance sampling ratio, where A(s t ,a t ) and r t The expression of (θ) is: ； ； In the formula, is the discount factor, Indicates is the policy function with parameters, V ω (s t ) is the output value of the Critic network at time t, y t is the value of V at time t+1 ω (s t ), R t is the reward value at time t; The update formula of the network parameter θ of the Actor network is: ； Where μ is the learning rate of the Actor network; The optimization function of the Critic network and the update formula of its network parameter ω are as follows: ； Where σ is the learning rate of the Critic network, is the gradient operator, is the optimization function of the Critic network.

10. A control system for improving the broadband oscillation stability of an offshore wind power grid-connected system, characterized in that: include: A first construction unit is used to select a generalized integrator control parameter and a synchronous coordinate system phase-locked loop control parameter as an input variable of an action variable set of an intelligent agent based on a dual second-order generalized integrator phase-locked loop transfer function; The second construction unit is used to obtain real-time information of various electrical quantities on the grid side of the offshore wind power grid-connected system and establish an environmental state variable set of the intelligent agent; The third construction unit is used to set the reward function R1 in the initial stage of oscillation, the reward function R2 in the continuous stage of oscillation and the reward function R3 in the recovery stage of oscillation based on the time scale according to the characteristics of the broadband oscillation of the offshore wind power grid-connected system, so as to obtain the overall reward function R=R1+R2+R3 of the intelligent agent; A training unit is used to perform offline training on the agent based on the action variable set, the environment state variable set and the overall reward function R, respectively, when low-frequency oscillation, sub- / super-synchronous oscillation and high-frequency oscillation occur in the power grid, wherein the agent is a proximal strategy optimization algorithm agent improved by introducing an approximate objective function optimization strategy; The control unit is used to perform online real-time control of the phase-locked loop parameters through the trained intelligent agent when broadband oscillation occurs in the power grid.

Images