Data driving-based load frequency control method for power system containing network-forming converter

Abstract

The invention relates to a data-driven load frequency control method for a power system containing a network-forming converter. The method comprises the following steps: firstly, according to a power system topology, constructing a time-delay nonlinear load frequency control model containing a network construction type converter; taking the model as an environment, simulating operation acquisition data by using a PID controller, and constructing a pre-training data set; then, an agent of an actor-commentator framework is constructed, and offline pre-training is completed through a data set; and finally, accessing the intelligent agent into a system for online interaction, outputting a frequency modulation instruction by an action network, and iteratively updating parameters by combining an award and a loss function to realize self-adaptive control. Compared with the prior art, the method has the advantages of being adaptive to the dynamic characteristics of the GFM, efficient, high in robustness and the like.

Description

Technical Field

[0001] This invention relates to the field of power system load frequency control technology, and in particular to a power system load frequency control method based on data-driven grid converters. Background Technology

[0002] Frequency stability in a power system is a core indicator for ensuring power quality and safe operation. Load Frequency Control (LFC), a key technology for maintaining frequency stability, functions by dynamically adjusting power output to offset frequency deviations caused by load fluctuations and power output uncertainties, ensuring that the system frequency remains within the rated range (e.g., my country's power frequency of 50Hz ± 0.2Hz). In traditional power systems, frequency regulation resources are primarily based on synchronous thermal power units. Their mechanical inertia and governor response characteristics are stable and can be accurately modeled. Therefore, LFC technology is mostly developed based on physical models, such as PID controllers and optimal controllers, all of which are designed based on the precise mathematical models of synchronous units. It has been successfully applied in thermal power-dominated systems, effectively ensuring frequency stability.

[0003] However, with the large-scale grid connection of high-proportion wind power, photovoltaics, and other new energy sources, grid-forming converters (GFMs), as core interface devices with active support capabilities, are widely used to connect new energy power generation systems to the power grid. Their large-scale integration has brought about a disruptive change to the frequency regulation mechanism of the power system, and the technical foundation upon which the traditional physical model-driven LFC method relies has been completely changed, specifically in the following two aspects: 1. The fundamental differences between GFM and traditional synchronous generator units GFM (Government Power Flow Meter) is not a passive converter in traditional renewable energy systems. Instead, it simulates the inertia and primary frequency regulation response of synchronous generator units using Virtual Synchronous Machine (VSM) technology, making it an actively supported power source. However, its dynamic characteristics are dominated by the control strategy of the power electronic converter, which is fundamentally different from traditional synchronous thermal power units. The mechanical inertia of synchronous thermal power units is determined by the physical characteristics of the rotor, with fixed parameters and stable response delays (usually on the order of seconds), possessing natural mechanical buffering characteristics. In contrast, GFM's inertia is virtual inertia, with coefficients that can be adjusted in real time via control commands, achieving millisecond-level response speeds. However, it lacks the buffering effect of mechanical inertia, and core parameters such as virtual inertia and droop coefficient are easily affected by fluctuations in renewable energy output and the switching status of the converter, exhibiting significant dynamic fluctuations. This unique characteristic of being adjustable yet volatile, fast yet fragile, makes GFM a core variable for changing the system's frequency regulation characteristics.

[0004] 2. System frequency regulation mechanism reconfiguration triggered by GFM grid connection After the large-scale grid connection of GFM, the frequency regulation system of the power system has been restructured from a single thermal power plant to a multi-coordinated system of thermal power + GFM + energy storage. The frequency regulation mechanism has also changed from a slow response dominated by mechanical inertia to a superposition of fast and slow responses coupled with virtual inertia and mechanical inertia. This restructuring directly leads to the complete failure of the adaptation basis of the traditional LFC method: on the one hand, the traditional method builds the system model based on the assumption of fixed inertia and stable response, which cannot incorporate the dynamic adjustment equation of GFM virtual inertia, and it is even more difficult to capture the coupling nonlinearity of mechanical inertia-virtual inertia, which is prone to model mismatch; on the other hand, the traditional method does not consider the 2-5 second response time difference between GFM and thermal power, which cannot achieve the orderly coordination of multi-frequency regulation resources. This may lead to the superposition of the fast response of GFM and the delayed output of thermal power, causing frequency fluctuations, or the suppression of the support potential of GFM due to control command conflicts, resulting in the waste of frequency regulation resources.

[0005] While some existing studies have focused on LFC optimization under new energy grid connection, they have mostly focused on the randomness of new energy output and have not fully recognized the core role of GFM as an active support device. Their control models have not incorporated the unique dynamic characteristics of GFM, and the design of time delay compensation and coordination strategies has not been carried out for the millisecond-level response requirements and parameter sensitivity of GFM. As a result, in high-proportion GFM access scenarios (access ratio exceeding 15%), the frequency deviation overshoot has increased significantly, the fluctuation frequency has increased significantly, and the system frequency stability cannot be guaranteed.

[0006] In summary, after grid-connected converters are connected to the power system, traditional load frequency control methods cannot adapt to the dynamic characteristics of GFM due to problems such as system modeling mismatch, failure of multi-frequency regulation resource coordination, insufficient time delay and sampling adaptation, low robustness of control strategies and low training efficiency. Therefore, they are unable to guarantee system frequency stability in the connected scenario. Summary of the Invention

[0007] The purpose of this invention is to overcome the shortcomings of the prior art by providing a power system load frequency control method based on a data-driven grid converter.

[0008] The objective of this invention can be achieved through the following technical solutions: According to one aspect of the present invention, a power system load frequency control method based on a data-driven grid converter is provided, the method comprising the following steps: S1. Based on the topology of the power system to be controlled, construct a time-delay nonlinear load frequency control system model with a grid-type converter. S2. Using the constructed model as the interactive environment, a PID controller is used to replace the intelligent agent to simulate operation with the environment under multiple working conditions. The pre-training dataset is collected and constructed based on the running data. S3. Construct an agent based on the actor-critic framework, wherein the evaluation network adopts a GRU-MLP hybrid structure, and the agent is subjected to offline supervised learning using a pre-trained dataset to update the parameters of the agent's action network and evaluation network, thus completing the pre-training of the agent. S4. Connect the pre-trained agent to the required power system environment for online interaction; in each control cycle, the action network outputs frequency regulation commands to the grid-type converter and generator set according to the current system state; at the same time, the agent parameters are iteratively updated using the reward function and loss function to achieve adaptive control of the power system load frequency containing the grid-type converter.

[0009] As a preferred technical solution, the time-delay nonlinear load frequency control system model with grid-type converter in S1 includes a non-reheat unit module, a reheat unit module, a grid-type converter module, a new energy disturbance module, a communication transmission module, a generator-load and LFC controller. The system model participates in frequency regulation through the coordinated operation of the non-reheat unit, the reheat unit and the grid-type converter.

[0010] A complete system model was constructed, incorporating a grid-connected converter (GFM), renewable energy disturbances, and various traditional generating units. By clarifying the collaborative architecture between non-reheat and reheat units and the GFM, a model foundation was provided to address the issues of GFM virtual inertia volatility and time lag in response with thermal power. This model can comprehensively simulate complex operating conditions under high-proportion renewable energy integration, providing a precise interactive environment for subsequent parameter coupling identification and implementation of dynamic compensation strategies, thus helping to improve the system's robustness under multi-source disturbances.

[0011] As a preferred technical solution, the system model is characterized by a set of linear state-space equations that include the characteristics of grid-type converters, and the virtual inertia and power response states of grid-type converters are introduced on the basis of traditional unit state equations. The linear state-space equations are as follows: ; In the formula, ; ; ; in, For state space; , , and These are the system matrix, input matrix, output matrix, and disturbance distribution matrix, respectively. for t Output at any moment; t The state at any given moment; for t The control input vector at each time step; This is the system state vector; This refers to the system frequency deviation. For non-reheat units, mechanical power deviation; For mechanical power deviation of reheat unit; This refers to the deviation in the output power of the reheat unit turbine. For non-reheat unit speed control valve position deviation; For the position deviation of the speed control valve of the reheat unit; The virtual inertial frequency state deviation of the grid-type converter; For the output power deviation of the grid-type converter; To control the input vector; For inputting the area control deviation signal of non-reheat units; For inputting the area control deviation signal of the reheat unit; This represents the total disturbance power. This refers to the load disturbance power. For photovoltaic power generation power disturbance; This is a disturbance to the power output of wind power generation.

[0012] As a preferred technical solution, the communication transmission module and the LFC controller in the system model are equivalent to controllers with time-delay characteristics, and the relationship between their outputs and the control strategy operators of the LFC controller satisfies: ; in, Output signals to the controller; For control strategy operators; For the Laplace operator; The observation transmission delay; To observe the output signal; For non-reheat units, the transmission time delay is considered. Due to the transmission time delay of the reheat unit; is the GFM transmission delay; E is the identity matrix; diag(·) is the diagonal matrix.

[0013] As a preferred technical solution, the system model adopts a subject-based sampling mechanism: the observer output signal is discretized at the sampling point after passing through the communication transmission module; the sampling step size for the grid-type converter is set to the millisecond level, and the sampling step size for the thermal power unit is set to the second level or the hundred-millisecond level; and the sampling step size is dynamically calibrated by the intelligent agent according to the environmental parameters.

[0014] As a preferred technical solution, the intelligent agents in S3 specifically include: Both the action network and the pre-learning action network adopt a multilayer perceptron structure to output control strategies based on the system state; The evaluation network adopts a GRU-MLP hybrid neural network structure, in which the temporal features of the state sequence are extracted by the gated recurrent unit layer, and then connected to the multilayer perceptron layer to output the value evaluation, so as to solve the problem of time-delay observation of system state.

[0015] As a preferred technical solution, the specific process of S4 includes: S41. At the current moment t The agent obtains the current system state through the evaluation network. State value Action network based on Output action commands Acting on the power system; S42. After the power system performs an action, it transitions to the state of the next instant. At the same time, it uses a reward function to provide immediate rewards. ; S43. Calculate the temporal difference error, and based on the temporal difference error, use the loss function to calculate the gradients of the evaluation network and the action network respectively, and update the network weight parameters through the backpropagation algorithm.

[0016] As a preferred technical solution, the reward function aims to minimize the system frequency deviation and tie-line power deviation, while constraining the control cost. Its expression is as follows: in, for t The reward value at any given moment; , and These are the weighting coefficients for frequency deviation, tie-line power deviation, and control cost, respectively. for t System frequency deviation at any given moment; for t The moment-to-moment power exchange deviation of the tie line; The total number of units participating in the control; For the first i Each control unit t The control unit provides real-time control output commands and includes grid-connected converters and generator sets.

[0017] As a preferred technical solution, the loss function includes an evaluation network loss function and an action network loss function, the expressions of which are as follows: ; ; in, To evaluate the network loss function; The sample batch size in the experience playback buffer; This is the discount factor, with a value range of (0,1); for; Let the action network loss function be used. The timing difference error is used to characterize the advantage value of the current action; For action networks in state Select action The probabilistic strategy.

[0018] According to another aspect of the present invention, a power system load frequency control system based on a data-driven grid converter is provided, the system comprising: The system modeling module is configured to construct a time-delay nonlinear load frequency control system model for a grid-type converter based on the power system topology. The pre-training data generation module is configured to use a PID controller to perform multi-condition simulations in the model and collect running data to build a pre-training dataset. The agent construction and pre-training module is configured to build an agent based on the Actor-Critic framework, wherein the evaluation network integrates a GRU-MLP hybrid structure and uses a pre-training dataset to complete the offline supervised learning of the agent. The online adaptive control module is configured to connect the pre-trained agent to the actual power system, output frequency regulation commands to the grid-type converter and generator set according to the real-time status, and update the agent parameters online using reward function and loss function.

[0019] Compared with the prior art, the present invention has the following beneficial effects: 1. In this invention, by introducing a pre-training stage based on PID data, a pre-learning and real-time collaborative mode for GFM characteristics is constructed, enabling the action network to grasp the response connection rules between GFM and thermal power in advance. Compared with traditional reinforcement learning starting from scratch (such as DDPG), this method significantly reduces the number of training convergence generations from 200 to 120, significantly improving training efficiency. At the same time, through online adaptive interaction, frequency fluctuations are effectively suppressed, reducing the system frequency overshoot by 21.9%. This method adapts to the dynamic characteristics of GFM and realizes efficient collaborative control driven by data.

[0020] 2. This invention establishes a linearized state-space representation method for GFM adaptation. By explicitly introducing the virtual inertial frequency state deviation and output power deviation of the GFM into the state equation, the coupling characteristics of mechanical inertia-virtual inertia are accurately characterized. This modeling method effectively fits the correlation between GFM virtual inertia and output force, reducing the fitting error by 35%, thereby enabling the controller to make decisions based on the accurate system state and avoiding the degradation of control performance caused by model mismatch.

[0021] 3. This invention designs a processing mechanism for the time-delay sensitivity of GFM. By constructing a transfer function containing a diagonal matrix, the signal transmission delay of different frequency modulation subjects is processed differently, and the observer, communication module, and controller are equivalent to a time-delay adapted controller. This mechanism embeds a time-delay-virtual inertia correlation term into the action value function, ensuring that the control system can still maintain stable convergence under adverse conditions when the system communication delay increases from 4s to 10s, effectively avoiding the system oscillation problem caused by the one-size-fits-all approach to time delay parameters in traditional methods.

[0022] This invention addresses the time difference between the millisecond-level response of GFM and the second-level response of thermal power, designing a separate agent dynamic sampling mechanism. By setting the sampling step size of GFM to milliseconds and that of thermal power to seconds / hundreds of milliseconds, and supporting dynamic agent calibration, the distortion of the fast response signal of GFM caused by traditional fixed sampling is avoided. This mechanism ensures that it can respond to sudden situations such as a sudden drop in GFM virtual inertia (e.g., 5s→2s) within 50ms, achieving precise coordination of frequency modulation resources at different time scales.

[0023] 4. This invention employs a hybrid neural network structure integrating GRU-MLP in the evaluation network. By utilizing GRU (Gated Recurrent Unit) to process the temporal characteristics of the state sequence, the evaluation network can incorporate multi-step state data for decision-making, effectively solving the problem of time-delayed observation of system states. Simultaneously, this structure can dynamically sense the GFM virtual inertia dynamic factor, thereby dynamically allocating the frequency regulation weights of GFM and thermal power, significantly improving the adaptability of the control strategy in time-varying environments.

[0024] 5. This invention implements a robust control strategy guided by GFM parameter fluctuations. Through online interaction and parameter iteration, the agent can identify parameter coupling changes and perform dynamic compensation. When facing scenarios where GFM and thermal power parameters fluctuate in tandem, this mechanism can quickly adjust the control strategy, making the standard deviation of the frequency recovery time only 3.2s, which is only 1 / 5 of that of traditional PID control, greatly enhancing the stability of high-proportion GFM systems.

[0025] This invention designs a multi-objective constraint reward function that considers frequency deviation, connection line power, and control cost. This function guides the agent to optimize control cost while minimizing deviation, directly improving control accuracy. Furthermore, this invention uses a loss function based on temporal difference error to update the network, ensuring the correct direction of gradient descent and accelerating the policy optimization process through advantage value calculation. This, combined with the pre-training mechanism, further consolidates the improved training efficiency and ensures that the action network and evaluation network can quickly converge to the optimal policy during online learning, reducing the risk of fluctuations in the initial stage of online adjustment. Attached Figure Description

[0026] Figure 1This is a schematic diagram of the steps of the power system load frequency control method based on a data-driven grid converter in this invention. Figure 2 This is a structural diagram of the LFC system in the embodiment; Figure 3 This is a frequency deviation response curve under load disturbance in the embodiment; Figure 4 This is a frequency deviation response curve under total power disturbance in the embodiment; Figure 5 This is a graph showing the frequency deviation response curves under different time delays in the embodiment; Figure 6a This is a frequency deviation response curve of the method in the embodiment; Figure 6b The above is a diagram showing the frequency deviation response curve of the PID control in an example. Detailed Implementation

[0027] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0028] This solution aims to address the core technical challenges of existing power system load frequency control (LFC) technology in new power systems with high proportions of renewable energy integration and grid-forming converters (GFM). As the background technology states, the large-scale integration of GFM has shifted the system frequency regulation mechanism from being dominated by mechanical inertia to a coupling of virtual inertia and mechanical inertia. Traditional technologies and existing research face numerous bottlenecks due to their inability to adapt to the unique properties of GFM, specifically including the following four aspects: 1. The model mismatch problem of traditional physics model-driven methods—inability to adapt to the dynamic virtual inertia characteristics of GFM. With the integration of high proportions of wind power, photovoltaic power, and other new energy sources, the strong randomness of their output (affected by fluctuations in wind speed and solar radiation) combined with the adjustable but volatile virtual inertia characteristics of the GFM (Geostationary Power Controller) transforms the power system from linear and weakly coupled to strongly nonlinear and multivariable coupled. Traditional LFC (Low-Frequency Coefficient) methods, based on the fixed mechanical inertia of synchronous generators, require complex parameter identification to construct a mathematical model covering the entire scenario. This not only results in long model construction cycles and high maintenance costs, but more importantly, it cannot incorporate the virtual inertia regulation equations of the GFM. The inertia coefficient of the GFM needs to be adjusted in real time according to fluctuations in new energy output, and the adjustment process is affected by the converter switching frequency and control strategy, exhibiting strong inertia-power response coupling nonlinearity, which is fundamentally different from the fixed inertia characteristics of traditional synchronous generators. This deficiency directly leads to model mismatch, resulting in decreased frequency control accuracy (frequency deviation overshoot increases by more than 30%) and lag in dynamic response, which is particularly prominent in scenarios where the GFM integration ratio exceeds 15%.

[0029] 2. The dynamic impact and insufficient compensation of communication transmission delays – millisecond-level delay-sensitive requirements of mismatched GFM In LFC systems, the acquisition of frequency observations (e.g., via SCADA systems) and the transmission of controller commands to the generator / GFM both rely on communication links, which are subject to random transmission delays (e.g., communication congestion or link failures can increase delays from 0.5s to over 3s). As emphasized in the background section, GFMs are five times more sensitive to delays than traditional generators; delays exceeding 50ms can trigger frequency oscillations. Existing methods have significant drawbacks: first, traditional fixed delay compensation strategies (e.g., preset delay parameters) are only suitable for the second-level response requirements of thermal power plants and cannot handle the millisecond-level control precision of GFMs; second, mainstream data-driven methods, when updating the evaluation network using Time Difference Error (TD Error), are limited by the reward decay coefficient and ignore the cross-time step impact of delays—in delay scenarios, the frequency modulation effect of the current control action needs to be delayed by multiple time steps to be reflected. Traditional value functions cannot accurately capture the correlation between action-delay-reward, which can easily lead to system frequency oscillations or even instability.

[0030] 3. Insufficient multi-power source coordinated frequency regulation capability – unresolved response time difference conflict between GFM and traditional units. The frequency regulation resources of new power systems have expanded from the traditional single entity of thermal power to a multi-entity entity consisting of non-reheat units, reheat units, and GFM (Geothermal Flow Mechanism). These three entities exhibit significant differences in response characteristics: GFM achieves millisecond-level response through converters and actively provides inertial support; non-reheat units have a response delay of approximately 1-2 seconds; reheat units have a response delay of 3-5 seconds; and the response time difference between GFM and thermal power can reach up to 5 seconds. Existing LFC (Low Frequency Coordination) methods do not address this characteristic by designing collaborative control logic: on the one hand, the lack of a command allocation mechanism for multiple frequency regulation resources can easily lead to excessive frequency exceeding limits due to the superposition of GFM's rapid response and thermal power's delayed output, or a gap in their responses resulting in slow frequency recovery; on the other hand, traditional controllers (such as PID controllers) cannot dynamically adjust the frequency regulation weights of each power source based on the intensity of disturbances from new energy sources, resulting in underutilization of GFM's active support capabilities and overall low efficiency of frequency regulation resources. When the GFM integration ratio exceeds 20%, the frequency regulation resource waste rate exceeds 40%.

[0031] 4. Training efficiency and robustness limitations of data-driven methods – unsuitable for complex parameter fluctuation scenarios involving GFM. Existing LFC methods based on deep reinforcement learning (DRL) (such as the DDPG algorithm) attempt to overcome the dependence on physical models, but they have two major shortcomings because they are not optimized for the characteristics of systems containing GFM: First, the training efficiency is low. In the early stage, they rely on random exploration of the action space to generate data, which not only results in slow convergence (requiring more than 200 generations to stabilize), but also makes it easy for unreasonable exploration actions to lead to improper setting of GFM virtual inertia parameters, causing frequency overshoot (deviation exceeding 0.5Hz). Second, the robustness is insufficient. When system parameters change (such as the turbine regulation rate constraint decreasing from 0.004pu / s to 0.002pu / s, or the GFM virtual inertia coefficient being adjusted from 5s to 2s due to fluctuations in new energy output, or the communication delay increasing from 4s to 10s), the frequency recovery time of existing methods is significantly extended (from 50s to more than 80s), and even continuous oscillations occur, making them unable to adapt to parameter fluctuation scenarios in new power systems.

[0032] Therefore, this solution provides an LFC method that does not rely on an accurate physical model, can dynamically adapt to time delay and GFM nonlinearity, can efficiently coordinate multiple frequency modulation resources, and has both training efficiency and robustness, in order to solve the frequency stability control problem under high proportion of new energy sources and GFM access.

[0033] Example 1 In this embodiment, a power system load frequency control method based on a data-driven grid converter is adopted, and the method steps are as follows: Figure 1 As shown, it specifically includes: S1. Based on the topology of the power system to be controlled, construct a time-delay nonlinear load frequency control system model with a grid-type converter. S2. Using the constructed model as the interactive environment, a PID controller is used to replace the intelligent agent to simulate operation with the environment under multiple working conditions. The pre-training dataset is collected and constructed based on the running data. S3. Construct an agent based on the actor-critic framework, wherein the evaluation network adopts a GRU-MLP hybrid structure, and the agent is subjected to offline supervised learning using a pre-trained dataset to update the parameters of the agent's action network and evaluation network, thus completing the pre-training of the agent. S4. Connect the pre-trained agent to the required power system environment for online interaction; in each control cycle, the action network outputs frequency regulation commands to the grid-type converter and generator set according to the current system state; at the same time, the agent parameters are iteratively updated using the reward function and loss function to achieve adaptive control of the power system load frequency containing the grid-type converter.

[0034] (I) Time-delay nonlinear LFC system model with grid converter 1. LFC system architecture and core modules considering GFM characteristics LFC system structure as follows Figure 2 As shown in the diagram, the green box represents the non-reheat unit structure, the red box represents the reheat unit structure, and the blue box represents the grid-type converter structure. The system consists of a non-reheat unit module, a reheat unit module, a grid-type converter module, a new energy disturbance module, a communication transmission module, a generator-load connection, and an LFC controller. The system participates in frequency regulation through the coordinated operation of the non-reheat unit, the reheat unit, and the grid-type converter.

[0035] This system addresses the issues raised earlier, such as the time difference between GFM and thermal power response, and the high sensitivity of GFM to time delays. It constructs a system architecture that combines multi-entity collaboration with precise time delay characterization. The core difference from traditional systems lies in upgrading the GFM from a passive interface to an active frequency modulation unit. It explicitly defines its core function of providing dynamic virtual inertia and millisecond-level response through Virtual Synchronous Machine (VSM) technology. Furthermore, it differentiates the communication characteristics of different entities through time delay-based design, laying the model foundation for resolving collaboration failures and time delay mismatches.

[0036] The functions of each module and their coupling relationship with GFM are as follows: GFM Active Support Module: Core parameters include virtual inertia coefficient H GFM (Dynamically adjustable between 2-8 seconds according to wind / solar power output fluctuations), virtual speed governor time constant T GFM (milliseconds, typically 0.05-0.1s) and droop coefficient 1 / R GFM The inertial response and primary frequency modulation are simulated using the VSM algorithm, and the output frequency modulation power ΔP is obtained. GFM Its response speed is 10-20 times faster than that of non-reheat units, making it the core force in suppressing high-frequency disturbances. Time-delay communication transmission module: Addressing the characteristic mentioned earlier that the GFM's time-delay sensitivity is 5 times that of traditional units, a time delay τ for the transmission of observed values ​​is designed. sc (Uniform characterization of frequency and power observation signal transmission delay, τϵ[0,0.5]s), thermal power command transmission delay τ r / nr (τϵ[0,3]s) and GFM command transmission delay τ GFM (Strictly limit τϵ[0,0.1]s to avoid oscillation caused by time delay exceeding 50ms), and accurately match the control requirements of each subject through different time delay parameters; New energy disturbance module: This module handles wind power output disturbance ΔP. WT Photovoltaic output disturbance ΔP PV With load disturbance ΔP L Integrate into total power disturbance ΔP D And ΔP D Directly related to GFM virtual inertial regulation—when ΔP D When the fluctuation exceeds 5%, H is dynamically adjusted, reflecting the coupling relationship between disturbance and virtual inertia.

[0037] 2. Sampling mechanism and time delay modeling based on GFM characteristics To address the time difference between the millisecond-level response of GFM and the second-level response of thermal power, a separate sampling mechanism is designed: the observer output signal y(t) is discretized at sampling point t (k=0,1,2...) after passing through the communication transmission module. The sampling step size T=50ms for GFM and T=500ms for thermal power. The sampling step size is dynamically calibrated by the agent's environmental parameters to avoid the distortion of the GFM response signal caused by traditional fixed sampling.

[0038] The LFC controller outputs commands periodically via a zero-order hold (ZOH). Combining this with time-delay characteristics, the observer, communication transmission module, and LFC controller are equivalent to a time-delay adapted controller. The relationship between its output u(t) and the LFC controller's core command ξ is as follows: ; In the formula, E is the identity matrix; the parameters in diag(·) correspond to the instruction transmission delay terms of the non-reheat unit, the reheat unit, and the GFM, respectively. By setting the τ term separately, the defect of the traditional model's one-size-fits-all delay parameter is solved, ensuring low-latency transmission of GFM instructions.

[0039] 3. Linear state-space equations containing GFM In light of the model mismatch problem discussed earlier, a virtual inertia and power response state of GFM is added to the traditional thermal power state equations. A linear state-space equation set incorporating GFM characteristics is constructed to accurately characterize the mechanical inertia-virtual inertia coupling properties. The equations are as follows: In the formula, ; .

[0040] In the formula, nr represents a non-reheat unit, and r represents a reheat unit. Δ f , Δ P m,nr / r Δ P t,r Δ P v,nr / ,r Δ P ,GFM / C,Δ P GFM / I represent two state variables: system frequency deviation, turbine mechanical power output deviation, steam reheat power deviation, governor displacement deviation, and GFM virtual governor power deviation, respectively. u This indicates the frequency modulation command issued by the controller.

[0041] (II) Data-driven LFC control method The data-driven LFC control method proposed in this scheme learns and optimizes through a large amount of data during system operation, and realizes the controller design without relying on an accurate model.

[0042] This method is based on an actor-critic framework, with the environment being a time-delay nonlinear LFC system. The agent consists of an MLP-based action network, a pre-learned action network with the same structure, and a GRU-MLP-based evaluation network. During the pre-training phase, a PID controller interacts with the environment to generate an LFC dataset, and offline learning is performed based on this data to update the parameters of the pre-learned action network and the evaluation network. During the training phase, the action network parameters are first initialized based on the pre-learned action network. Then, the action network interacts with the environment, and the data is stored in an experience replay pool. After each generation, data is sampled from the replay pool for online learning, and the action network parameters are updated based on the trained evaluation network.

[0043] (III) Simulation Analysis In this embodiment, a targeted simulation scenario is designed to address the core issues of GFM and thermal power synergistic failure, GFM time delay sensitivity, and parameter fluctuations in GFM-containing systems. First, the parameters of the LFC system with a high proportion of GFM and the control method are clarified. Second, under power disturbance, the synergistic regulation effect of the proposed method on GFM and thermal power and the frequency control performance are verified. Finally, by dynamically adjusting key parameters such as GFM time delay and virtual inertia coefficient, the robustness of the method is verified, providing data support for the effectiveness of the technical solution.

[0044] 1. Simulation parameter settings The simulated LFC system is built upon two non-reheat units, one reheat unit, and two GFM renewable energy plants. The GFM integration ratio in the system is 30% (covering the high integration ratio scenario mentioned earlier), and its core parameters are as follows: Virtual Inertia Coefficient H. GFM The baseline value is 5s (which can be dynamically adjusted from 2-8s according to fluctuations in wind / solar power output), and the virtual speed governor time constant T is... GFM =0.08s, droop coefficient 1 / R GFM =0.05, the reference value for instruction transmission delay τ GFM =50ms.

[0045] To meet the challenges of actual frequency regulation, the power disturbance design is a composite disturbance of new energy output deviation and load change: the deviation between the actual and predicted output of wind power and photovoltaic power (fluctuation range ±8% of rated power) is used as ΔP. WT ΔP PV The load surge ΔP, which is superimposed with ±4% of the rated power, is a significant change. L This results in a total power disturbance ΔP D And ΔP D When the fluctuation exceeds 5%, the dynamic adjustment of the GFM virtual inertia coefficient is triggered, which is consistent with the disturbance-virtual inertia coupling logic of the system model mentioned above.

[0046] Three typical control methods were selected for comparative verification, all tested on the same system model including GFM to ensure fairness: ① Traditional PID controller (parameters kp=4.156, ki=1.149, kd=2.432, not optimized for GFM characteristics); ② Conventional Deep Deterministic Policy Gradient (DDPG) algorithm (without incorporating GFM response time difference compensation and virtual inertia modeling); ③ Action Network Pre-Learning DDPG (PreActor-DDPG) algorithm (only optimized for training efficiency, not adapted to GFM characteristics); ④ Our method (core parameter optimizations: adding a GFM-thermal power plant response time difference judgment module, a time delay dynamic compensator, and a virtual inertia adaptive factor). The convergence generation was reduced to 120 generations (40% improvement over DDPG), and the GFM frequency overshoot during training was controlled within 0.2Hz.

[0047] 2. Frequency control performance analysis under power disturbance To verify the frequency control effect of the proposed method under power disturbances, and considering the strong coupling characteristics between GFM and renewable energy output mentioned earlier, a composite disturbance of load change + renewable energy output deviation was applied to an LFC system with 30% GFM access. First, a specific load disturbance test was conducted (renewable energy output stable), focusing on the synergistic control effect between GFM and thermal power. The frequency deviation response curves of different control methods are shown below. Figure 3 As shown in the figure; the black dashed line represents the frequency deviation response under total power disturbance when using PID control, the green dashed line represents the frequency deviation response under total power disturbance when using the DDPG method, the yellow dashed line represents the frequency deviation response under total power disturbance when using PreActor-DDPG, and the blue dashed line represents the frequency deviation response under total power disturbance when using the method proposed in this scheme.

[0048] Depend on Figure 3 As can be seen, at the instant of load disturbance triggering (0-2s), the proposed method, through the GFM-thermal power plant response time difference judgment module, prioritizes the activation of the millisecond-level response of the GFM (the virtual inertia coefficient dynamically increases from 5s to 7.2s), quickly smoothing out the initial frequency drop and avoiding the initial overshoot caused by the lag in the GFM response or the premature output of the thermal power plant in the PID and DDPG methods. In the middle of the disturbance (2-15s), by dynamically allocating the frequency regulation weight, the GFM gradually withdraws and the thermal power plant smoothly takes over the load, solving the secondary oscillation caused by the superposition of the outputs of the two in the PreActor-DDPG method. In the later stage of the disturbance (after 15s), relying on the virtual inertia adaptive factor, the virtual inertia coefficient of the GFM falls back to 5.5s, and with the stable output of the thermal power plant, the frequency convergence is accurately achieved, and the steady-state error is controlled within ±0.01Hz. Under the proposed method, the thermal power plant regulation rate is stable (maximum regulation rate 0.0035pu / s) and no nonlinear constraint is triggered, which is directly related to the rapid pre-support of the GFM.

[0049] To quantitatively evaluate the frequency dynamics and steady-state performance after adjustment using different control methods, four core indicators were selected for the frequency modulation requirements of systems containing GFM: ①Cumulative maximum frequency deviation e maxΣ (reflects the ability of GFM to suppress high-frequency disturbances); ② Cumulative recovery time t rΣ (reflects the synergistic efficiency between GFM and thermal power). ③ Cumulative time-weighted absolute error integral e ITAEΣ (Characterizing frequency stability over the entire time period); ④ Weighted composite index C Σ (Combining the overall performance of the first three items), the calculation formula is as follows: No. i Under the second disturbance, Δ f max i , ti , and Δ fi ( t These represent the maximum frequency deviation, the time of disturbance occurrence, and the frequency deviation during the recovery process, respectively. t |Δ f |≤0.02, i Indicates from the first i The time from the start of the disturbance until the overall frequency deviation is within ±0.02Hz; β The weighting coefficient is set to -5 here. The control performance indicators of different control methods are shown in Table 1.

[0050] Table 1 Control performance indicators under load disturbance As can be seen from Table 1, the core advantages of the proposed method are: e maxΣ Although slightly higher than PreActor-DDPG, based on the analysis of GFM response characteristics, its initial overshoot is controlled within 0.25Hz (far lower than 0.32Hz of DDPG), avoiding converter protection action caused by GFM overshoot; t rΣ Compared to PID, the time difference between GFM and thermal power response is reduced by 29 seconds, and the coordination strategy reduces the connection time between GFM and thermal power response from 2.3 seconds to 0.8 seconds. C Σ The overshoot was reduced by 31.4% compared to DDPG, fully demonstrating the optimization value of adapting to GFM characteristics. Although PreActor-DDPG reduced overshoot through pre-learning, the recovery time was still relatively long because it did not solve the coordination problem between GFM and thermal power. PID had the worst performance in all indicators because it was not adapted to the millisecond-level response of GFM.

[0051] To further verify the performance under complex disturbances, a total power disturbance test was conducted involving load abrupt changes and new energy output deviations (new energy output fluctuations of ±8%, triggering GFM virtual inertia for dynamic adjustment within 2-8 seconds). The frequency deviation response curves for different methods are shown below. Figure 4 As shown in the figure; the black dashed line represents the frequency deviation response under different time delays when using PID control, the green dashed line represents the frequency deviation response under different time delays when using the DDPG method, the yellow dashed line represents the frequency deviation response under different time delays when using Pre Actor-DDPG, and the blue dashed line represents the frequency deviation response under different time delays when using the method proposed in this scheme.

[0052] Depend on Figure 4 As can be seen, under total power disturbance, the random fluctuations in renewable energy output cause frequent adjustments to the virtual inertia of the GFM. Traditional methods, unable to adapt to this characteristic in real time, result in a response curve oscillation frequency 3-5 times higher than that of simple load disturbance. The proposed method, however, uses a virtual inertia adaptive factor to match GFM parameter changes in real time, controlling the oscillation amplitude within ±0.1Hz. The absolute error integral e (reflecting disturbance smoothing capability) is used for quantitative evaluation, as shown in the following formula: By calculating the integral of the absolute error e The IAE is used to describe the regulation performance of different control methods, and its formula is as follows: ; The calculated eIAE values ​​for the proposed method, PreActor-DDPG, DDPG, and PID control under total power disturbance are 19.523, 24.334, 27.425, and 28.669, respectively. The proposed method's eIAE is 31.9% lower than that of PID. The core reason is that its time-delay dynamic compensator solves the millisecond-level time delay problem of GFM command transmission. In contrast, DDPG and PID, due to insufficient time-delay compensation, cause the GFM frequency modulation action to become disconnected from the actual disturbance, resulting in a poorer smoothing effect.

[0053] 3 Robustness Analysis under System Parameter Variation In a nonlinear LFC system with a grid-type converter, robustness focuses on two scenarios: first, the dynamic changes in the GFM command transmission delay and the thermal power command delay; and second, the coordinated fluctuations of the GFM virtual inertia coefficient and the thermal power nonlinear constraint parameters. In this embodiment, tests are designed for these two scenarios to verify the proposed method's adaptability to parameter changes in systems with GFM.

[0054] To accurately match real-world scenarios, a separate time-delay increment scheme was set up for the test: a 0.04 pu step load disturbance was applied to the system, maintaining the GFM command transmission delay τ ≤ 0.1s (to avoid exceeding the baseline delay limit). The transmission delay τ of the observed values ​​(frequency / power observation signal) and the thermal power command transmission delay τ were each increased by 1s, forming test scenarios with total delays τ = 4s, 6s, 8s, and 10s. The frequency deviation response curves of the proposed method and PreActor-DDPG were obtained as follows: Figure 5 As shown, Figure 5 In the figure, the solid line represents the frequency deviation response curve of this method, and the dashed line represents the frequency deviation response curve of PreActor-DDPG.

[0055] Depend on Figure 5It is evident that the core difference between the two methods stems from their ability to adapt to the GFM time-delay-response correlation: PreActor-DDPG only optimizes the pre-learning of the action network and does not design a compensation mechanism for the time delay difference between GFM and thermal power—when τ increases to 8s, the delay of thermal power command causes its response time difference with GFM to increase from 2.3s to 4.5s, and the superposition of their outputs causes oscillations; while the proposed method identifies changes in τ in real time through a time-delay dynamic compensator and synchronously adjusts the virtual inertia coefficient of GFM (for every 2s increase in τ, H... GFM The proposed method increases the frequency recovery time from 5s to 6.5s to enhance the inertial support of the GFM and offset the effects of time delay, achieving a dynamic balance between time delay and virtual inertia. Specifically, at τ=10s, the frequency recovery time of the proposed method is reduced by 58% compared to PreActor-DDPG, the overshoot is controlled within 0.2Hz, and the GFM converter protection action is not triggered. In contrast, PreActor-DDPG suffers from oscillation, resulting in a maximum frequency deviation of -0.28Hz, which is close to the protection threshold.

[0056] 4. Robustness verification under nonlinear constraint parameter fluctuations (including GFM parameter coupling) The test scenario combines the composite changes of thermal power nonlinear constraints and GFM parameter fluctuations: Under τ=4s (basic time delay) and 0.04pu load disturbance, the following parameters are uniformly sampled [-20%, 20%] (step size 5%) to generate 8 sets of deviation models: ① Thermal power nonlinear constraint parameters (governor dead zone w, turbine acceleration rate σ, output upper and lower limits ΔP / ΔP); ② GFM core parameters (virtual inertia coefficient H, droop coefficient 1 / R), where H... GFM The fluctuation range is 2-8 seconds, forming an extreme scenario of coordinated fluctuation of thermal power + GFM parameters. The response curve is as follows: Figure 6a and 6b As shown.

[0057] Depend on Figure 6a and 6b The comparison reveals that the core flaw of PID control is parameter fixation—its proportional-integral coefficients are not correlated with the parameter coupling relationship between the GFM and the thermal power plant: when H GFM When the frequency recovery time decreases from 5s to 2s (deviation -60%) and the thermal power σ decreases to 0.002pu / s (deviation -50%), the PID controller cannot quickly increase the GFM frequency modulation weight, causing the frequency recovery time to increase from 45s to 92s, resulting in continuous oscillation. The proposed method, however, uses a parameter-coupled identification module to capture H in real time. GFM The change is linked to σ, when H GFMWhen the frequency is reduced, the GFM frequency regulation weight is immediately increased from 30% to 55%, while the thermal power regulation rate requirement is reduced to avoid triggering nonlinear constraints. Quantitative data show that the proposed method has a frequency recovery time standard deviation of 3.2s under 8 sets of deviation models, which is only 1 / 5 of that of PID; the maximum frequency deviation fluctuation range is ±0.08Hz, which is 64% smaller than that of PID, fully demonstrating its strong robustness to parameter fluctuations in systems containing GFM.

[0058] In summary, the robustness advantage of the proposed method stems from its deep adaptation to the core characteristics of GFM: it addresses the time-delay sensitivity of GFM through a time-delay dynamic compensator and adapts to the parameter fluctuations of GFM and thermal power through a parameter coupling identification module, thus solving the robustness defects of traditional methods and adapting to the scenario of dynamic parameter changes under high proportion of GFM access.

[0059] In this scheme, addressing the issues of volatile virtual inertia in grid-type converters, time lag between their responses and those of thermal power plants, and coordinated fluctuations in their parameters, the relevant technologies achieve stable control of high-proportion GFM systems through a three-layer core design: A hybrid learning architecture of GFM characteristic pre-learning and real-time collaborative learning is constructed within the Actor-Critic framework. The action network learns the response connection rules between GFM and thermal power offline, and the evaluation network incorporates the dynamic factor of GFM virtual inertia to dynamically allocate the frequency modulation weights of both, reducing the training convergence generations from 200 to 120, and lowering the frequency overshoot by 21.9% compared to traditional DDPG. To address the nonlinear and time-delay-sensitive characteristics of GFM inertia-power coupling, a residual network is used to fit the correlation between its virtual inertia (2-8s fluctuation) and output. The system reduces the fitting error by 35%, and embeds a time-delay-virtual inertia correlation term into the action value function and incorporates multi-step state data decision-making to ensure stable convergence of the system when the time delay increases from 4s to 10s, avoiding the oscillation problem of traditional methods. For the scenario of coordinated fluctuation of GFM and thermal power parameters, a parameter coupling identification + dynamic compensation robust control strategy is designed. It can respond to parameter mutations such as sudden drop in GFM virtual inertia (e.g., 5s→2s) within 50ms, dynamically adjust the GFM adjustment factor and thermal power constraints. When the parameters fluctuate within the range of [-20%, 20%], the standard deviation of the frequency recovery time is only 3.2s (1 / 5 of that of PID control), and the maximum deviation is controlled within ±0.08Hz, effectively improving the operational stability of high-proportion GFM power systems.

[0060] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in the present invention, and these modifications or substitutions should all be covered within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A data-driven based power system load frequency control method for grid-forming converters, characterized in that, The method steps comprise: S1, constructing a time-delay nonlinear load frequency control system model containing a grid-forming converter according to the topology of a required control power system; S2, using a PID controller to replace the intelligent agent to simulate the operation of the model under multiple working conditions, collecting and constructing a pre-training data set based on the operation data; S3, constructing an intelligent agent based on an actor-critic framework, wherein the evaluation network adopts a GRU-MLP hybrid structure, and the pre-training data set is used for offline supervised learning of the intelligent agent to update the action network and evaluation network parameters of the intelligent agent and complete the pre-training of the intelligent agent; and S4, connecting the pre-trained intelligent agent to the required control power system environment for online interaction; in each control cycle, the action network outputs a frequency regulation instruction to the grid-forming converter and the generator set according to the current system state; at the same time, the intelligent agent parameters are iteratively updated by using a reward function and a loss function to realize adaptive control of the load frequency of the power system containing the grid-forming converter.

2. The data-driven grid-forming converter-based power system load frequency control method of claim 1, wherein, The time-delay nonlinear load frequency control system model containing the grid-forming converter in S1 comprises a non-reheat unit module, a reheat unit module, a grid-forming converter module, a new energy disturbance module, a communication transmission module and a generator-load and LFC controller, and the system model participates in frequency regulation through the non-reheat unit, the reheat unit and the grid-forming converter.

3. The data-driven grid-forming converter-based power system load frequency control method of claim 2, wherein, The system model is characterized by a linear state space equation set containing the characteristics of the grid-forming converter, and the virtual inertia and power response state of the grid-forming converter are introduced into the state equation of the traditional unit. The linear state space equation set is as follows: ； In the formula, ； ； ； wherein, is the state space; , , and are system matrix, input matrix, output matrix and disturbance distribution matrix respectively; is the output at time t t is the state at time t is the control input vector at time is the system state vector; is the system frequency deviation; is the mechanical power deviation of non-reheat unit; is the mechanical power deviation of reheat unit; is the turbine output power deviation of reheat unit; is the governing valve position deviation of non-reheat unit; is the governing valve position deviation of reheat unit; is the virtual inertia frequency state deviation of grid-forming converter; is the output power deviation of grid-forming converter; is the control input vector; is the regional control deviation signal input of non-reheat unit; is the regional control deviation signal input of reheat unit; is the total disturbance power; is the load disturbance power; is the photovoltaic power disturbance; is the wind power disturbance.​​ 4. The data-driven, grid-forming converter-based power system load frequency control method of claim 2, wherein, The communication transmission module and the LFC controller in the system model are equivalent to a controller with a time-delay characteristic, and the relationship between the output and the control strategy operator of the LFC controller satisfies: ； wherein, is a controller output signal; is a control policy operator; is a Laplacian operator; is an observation value transmission time delay; is an observation output signal; is a non-reheat unit transmission time delay; is a reheat unit transmission time delay; is a GFM transmission time delay; E is an identity matrix; diag(·) is a diagonal matrix.

5. The data-driven, grid-forming converter-based power system load frequency control method of claim 2, wherein, The system model adopts a multi-agent sampling mechanism: the output signal of the observer is discretized at the sampling point after passing through the communication transmission module; wherein the sampling step length corresponding to the grid-forming converter is set to be millisecond level, and the sampling step length corresponding to the thermal power unit is set to be second level or hundred-millisecond level; and the sampling step length is dynamically calibrated by the intelligent agent according to the environment parameters.

6. The data-driven, grid-forming converter-based power system load frequency control method of claim 2, wherein, The intelligent agent in S3 specifically comprises: The action network and the pre-learned action network both adopt a multi-layer perception structure and are used for outputting a control strategy according to the system state; The evaluation network adopts a GRU-MLP hybrid neural network structure, extracts the time sequence characteristics of the state sequence through a gate recurrent unit layer, and outputs a value evaluation through a multi-layer perception layer to solve the time-delay observation problem of the system state.

7. The data-driven, grid-forming converter-based power system load frequency control method of claim 1, wherein, The specific process of S4 comprises: S41. At the current moment t The agent obtains the current system state through the evaluation network. State value Action network based on Output action commands Acting on the power system; S42, the power system executes the action and shifts to the next time state at the same time, the reward function feeds back the instant reward ; S43, calculating a time sequence difference error, and calculating the gradient of the evaluation network and the action network by using a loss function based on the time sequence difference error, and updating the network weight parameters through a back propagation algorithm.

8. The data-driven grid-forming converter-based power system load frequency control method of claim 7, wherein, The reward function aims to minimize the system frequency deviation and the tie-line power deviation while constraining the control cost, and its expression is as follows: in, for t The reward value at any given moment; , and These are the weighting coefficients for frequency deviation, tie-line power deviation, and control cost, respectively. for t System frequency deviation at any given moment; for t The moment-to-moment power exchange deviation of the tie line; The total number of units participating in the control; For the first i Each control unit t The control unit provides real-time control output commands and includes a grid-type converter and a generator set.

9. The data-driven grid-forming converter-based power system load frequency control method of claim 8, wherein, The loss function comprises an evaluation network loss function and an action network loss function, and its expression is as follows: ； ； wherein, is the network loss function; is the batch size of samples in the experience replay buffer; is the discount factor, with a range of (0, 1); is the; is the action network loss function; is the temporal difference error, which represents the advantage value of the current action; is the probability policy of the action network in state selects the action .

10. A data-driven, grid-forming power converter based power system load frequency control system, characterized in that, The system application is based on the data-driven grid-connected converter-based power system load frequency control system as claimed in any one of claims 1-9, and the system comprises: a system modeling module configured to construct a time-delay nonlinear load frequency control system model of the grid-connected converter-based power system according to a power system topology; a pre-training data generation module configured to simulate multiple working conditions in the model by using a PID controller, collect operation data, and construct a pre-training data set; an intelligent agent construction and pre-training module configured to construct an intelligent agent based on an Actor-Critic framework, wherein an evaluation network integrates a GRU-MLP hybrid structure, and off-line supervised learning of the intelligent agent is completed by using the pre-training data set; an online adaptive control module configured to connect the pre-trained intelligent agent to an actual power system, output frequency modulation instructions to the grid-connected converter and the generator set according to a real-time state, and update intelligent agent parameters online by using a reward function and a loss function.

Images