Data-driven online estimation method and system for inertia parameters of grid-forming converters

Abstract

The embodiment of the specification provides a data-driven online estimation method and system for inertia parameters of a grid-forming converter, and relates to the technical field of electric power. In the method, three-phase voltage time series data, three-phase current time series data and bus frequency time series data at the outlet side of the grid-forming converter are collected, and active power time series data is calculated according to the three-phase voltage time series data and the three-phase current time series data; the active power time series data and the bus frequency time series data are taken as state information to input into a trained estimation model; the estimation model outputs an action, and the action includes an estimated value of a virtual inertia parameter and an estimated value of a virtual damping parameter of the grid-forming converter. The embodiment of the specification solves the problems of parameter estimation deviation, introduction of quantization error by discrete action space to limit the upper limit of estimation precision, insufficient transient fitting capability and difficulty in obtaining virtual inertia control parameters which are manufacturer confidential information in the prior art.

Description

Technical Field

[0001] The embodiments in this specification relate to the field of power technology, specifically to a data-driven online estimation method and system for the inertia parameters of grid-type converters. Background Technology

[0002] With the continuous increase in the proportion of renewable energy power generation, the grid-connected converter (GFMC) has become a key piece of equipment for the stable operation of high-proportion renewable energy power systems. Its virtual inertia and virtual damping parameters directly determine the frequency support capability of the converter and are the core basis for grid operators to evaluate the system inertia level.

[0003] However, existing inertia parameter estimation methods have inherent limitations: on the one hand, traditional parameter identification techniques ignore frequency response delays and attenuation effects such as phase-locked loop delay, sampling calculation delay, and PWM modulation delay, resulting in significant errors when applied directly; on the other hand, existing methods are mostly invasive or offline, requiring external disturbances that affect normal equipment operation. Recent intelligent estimation methods based on deep reinforcement learning design the inertia parameter action space as discrete values, introducing quantization errors, and the reward function is designed only based on steady-state error, failing to reflect the dynamic characteristics of the frequency response. Furthermore, the virtual inertial control parameters of grid-connected converters are core confidential information of equipment manufacturers, making them difficult for grid operators to obtain.

[0004] Therefore, how to construct a non-intrusive online estimation method that fully considers the frequency response delay and attenuation characteristics in order to achieve high-precision online estimation of the virtual inertia and virtual damping of grid-type converters is a technical problem that urgently needs to be solved in this field. Summary of the Invention

[0005] In view of this, embodiments of this specification provide a data-driven online estimation method for the inertia parameters of a grid-type converter. One or more embodiments of this specification also relate to a data-driven online estimation system for the inertia parameters of a grid-type converter, a computing device, and a computer-readable storage medium, to solve problems existing in the prior art such as parameter estimation deviations caused by failure to consider the frequency measurement delay of the phase-locked loop and the delay of digital control, quantization errors introduced by the discrete action space limiting the upper limit of estimation accuracy, the inability of single-agent architecture to handle multi-machine coupling, insufficient transient fitting ability due to the reward function being based solely on steady-state error, and the virtual inertial control parameters being confidential information of the manufacturer and difficult for operators to obtain.

[0006] According to a first aspect of the embodiments of this specification, a data-driven online estimation method for the inertia parameters of a grid-type converter is provided, comprising:

[0007] Collect three-phase voltage time-series data, three-phase current time-series data, and bus frequency time-series data from the output side of the grid-type converter, and calculate active power time-series data based on the three-phase voltage time-series data and three-phase current time-series data.

[0008] The active power time-series data and the bus frequency time-series data are used as state information inputs to the trained estimation model. The estimation model is a deep reinforcement learning agent, which is trained in a simulation environment. The simulation environment is constructed by a series-parallel identification model, which is used to simulate the frequency response delay characteristics of the phase-locked loop of the grid-type converter.

[0009] The estimation model outputs an action, which includes the estimated values ​​of the virtual inertia parameters and virtual damping parameters of the grid-type converter.

[0010] In the embodiments of this specification, by collecting three-phase voltage, three-phase current, and bus frequency data at the output side of the grid-type converter and calculating active power, a deep reinforcement learning agent is trained in a simulation environment constructed using a series-parallel identification model. This simulates the frequency response delay and attenuation characteristics, such as the delay of phase-locked loop frequency measurement, signal sampling calculation, and PWM modulation, enabling the agent to learn the complete dynamic characteristics of the frequency response. This achieves non-intrusive online accurate estimation of the virtual inertia and virtual damping of the grid-type converter without relying on the internal parameters of the equipment. It solves the problems of parameter estimation deviation caused by ignoring delay factors in traditional methods, the introduction of quantization error into the discrete action space limiting the accuracy limit, the inability of single-agent architecture to handle multi-machine coupling, and the insufficient transient fitting ability due to the reward function being based only on steady-state error. This significantly improves the estimation accuracy and engineering practicality of the inertia parameters of the grid-type converter.

[0011] In some possible implementations, the estimation model includes an input layer, a shared feature extraction layer, an Actor network, and a Critic network; the input layer is used to receive the active power time-series data and the bus frequency time-series data; the shared feature extraction layer is used to perform nonlinear feature extraction on the input data; the Actor network is used to output the estimated values ​​of inertia parameters in the continuous action space, and the output layer of the Actor network uses a Gaussian strategy to output the action mean and variance; the Critic network is used to output the state value.

[0012] In the embodiments of this specification, by using a structural design in which the Actor network and the Critic network share a feature extraction layer, the agent can simultaneously learn policy mapping and state value evaluation from the same state representation. The Actor network uses a Gaussian policy to output the mean and variance of actions, achieving a balance between exploration and utilization in the continuous action space, overcoming the inherent quantization error in the discrete action space, and significantly improving the accuracy of inertia parameter estimation and training efficiency.

[0013] In some possible implementations, the estimation model is trained as follows: A simulation environment is constructed based on a series-parallel identification model; in the series-parallel identification model, the active power timing sequence at the grid connection point of a grid-connected converter is used as input, and synchronously distributed to the actual grid-connected converter branch and the neural network identification model branch; the agent outputs an action based on its current state, the action including an estimated value of the inertia parameter; the neural network identification model branch receives the action and the active power timing sequence, and outputs the model-fitted frequency; the actual grid-connected converter branch outputs the measured bus frequency; the timing error between the measured bus frequency and the model-fitted frequency is calculated; the agent is trained using a near-end policy optimization algorithm with the negative value of the timing error as the reward function; during training, the agent outputs an action based on its current state, the simulation environment calculates the timing error based on the action and returns the reward function value, and the near-end policy optimization algorithm updates the agent's parameters based on the reward function value until the reward function converges.

[0014] In the embodiments of this specification, a simulation environment is constructed by a series-parallel identification model. The active power time sequence is used as input to synchronously split the current to the real system branch and the neural network branch. The negative value of the time sequence error between the measured frequency and the model fitted frequency is used as the reward function. The agent is trained by a near-end strategy optimization algorithm, which enables the agent to continuously optimize the fitting accuracy during the transient process. This solves the problem that the traditional method is not able to achieve sufficient transient fitting ability because it is only based on steady-state error design. It realizes the effective learning of the dynamic characteristics of the frequency response of the grid-type converter.

[0015] In some possible implementations, the measured bus frequency output by the actual grid-type converter branch includes: in the series-parallel identification model, the actual grid-type converter branch incorporates a virtual inertial dynamic function that takes into account delay; the virtual inertial dynamic function incorporates the rotor motion equation, the first-order delay element of the phase-locked loop, and the PWM modulation attenuation characteristics; the measured bus frequency is determined by the following formula:

[0016]

[0017] in, for The measured bus frequency of the actual grid-type converter branch output at the specified time. for The active power timing data of the actual grid-type converter branch is input at all times. The virtual inertia coefficient of the grid-type converter is given. This refers to the virtual damping coefficient of the grid-type converter. The total equivalent delay constant for phase-locked loop frequency measurement delay, signal sampling calculation delay, and PWM modulation delay.

[0018] In the embodiments of this specification, by embedding a virtual inertial dynamic function that takes into account the delay in the actual grid-type converter branch, the first-order delay element of the phase-locked loop and the attenuation characteristics of PWM modulation are incorporated into the model, so that the measured bus frequency can reflect the actual frequency response delay and attenuation characteristics of the grid-type converter. This provides an accurate reference benchmark for subsequent error calculation and model training, and eliminates the parameter estimation deviation caused by the delay factor from the root.

[0019] In some possible implementations, the neural network identification model branch outputs a model fitting frequency, including: in the serial-parallel identification model, the neural network identification model branch receives the active power time series and the estimated inertia parameter output by the agent, performs nonlinear mapping through a shared feature extraction layer, and outputs the model fitting frequency.

[0020]

[0021] in, for The active power time-series data of the branches of the neural network identification model are input at any time. for The estimated value of the inertia parameter output by the agent at time t. These are the network parameters of the intelligent agent.

[0022] In the embodiments of this specification, the active power timing and the estimated inertia parameters output by the agent are received by the model branch through a neural network identification model. The model fitting frequency is then output through nonlinear mapping by the shared feature extraction layer, thereby realizing end-to-end learning and fitting of the frequency response characteristics of the grid-type converter and improving the model's ability to approximate the dynamic behavior of the real frequency response.

[0023] In some possible implementations, the step of calculating active power time-series data based on the three-phase voltage time-series data and the three-phase current time-series data includes: performing coordinate transformation on the three-phase voltage time-series data to obtain dq-axis voltage components, performing coordinate transformation on the three-phase current time-series data to obtain dq-axis current components, and calculating active power time-series data based on the dq-axis voltage components and the dq-axis current components.

[0024] In the embodiments of this specification, the dq-axis components are obtained by performing coordinate transformation on the three-phase voltage and current, and then the active power time series data is calculated. This realizes the feature extraction from the original electrical quantities to active power, provides the intelligent agent with input signals with clear physical meaning, and ensures the data-driven characteristics of the estimation process.

[0025] In some possible implementations, the method further includes multi-agent cooperative estimation: each grid-type converter is configured with an independent agent; each agent shares a global reward signal, which is the negative value of the system frequency deviation; and a multi-agent near-end policy optimization algorithm is used for joint training to achieve cooperative estimation of the inertia parameters of each converter.

[0026] In the embodiments of this specification, by configuring an independent agent for each grid-type converter and sharing a global reward signal, and by jointly training a multi-agent near-end policy optimization algorithm, the electrical coupling relationship of multiple converters operating in parallel can be effectively handled. This overcomes the limitation of a single agent architecture that can only handle a single converter, and realizes the collaborative online estimation of the inertia parameters of multiple grid-type converters, thereby improving the engineering applicability to multi-converter scenarios in new energy power plants.

[0027] According to a second aspect of the embodiments of this specification, a data-driven online estimation system for the inertia parameters of a grid-type converter is provided, comprising:

[0028] The data acquisition module is used to collect three-phase voltage time-series data, three-phase current time-series data and bus frequency time-series data on the output side of the grid-type converter, and to calculate active power time-series data based on the three-phase voltage time-series data and three-phase current time-series data.

[0029] The online inference module is used to input the active power time-series data and the bus frequency time-series data as state information into a trained estimation model. The estimation model is a deep reinforcement learning agent trained in a simulation environment constructed by a series-parallel identification model, which is used to simulate the frequency response delay characteristics of the phase-locked loop of the grid-type converter. The estimation model outputs actions, including estimated values ​​of the virtual inertia parameters and virtual damping parameters of the grid-type converter.

[0030] In the embodiments of this specification, the real-time acquisition of electrical quantities and active power calculation of the grid-type converter outlet side are realized through the collaborative work of the data acquisition module and the online inference module, respectively, as well as the online inference estimation of inertia parameters based on the trained deep reinforcement learning agent. A complete link from data acquisition to parameter estimation is formed between the two modules, and an end-to-end solution for data-driven online estimation of inertia parameters of grid-type converters is constructed. It can be widely applied in the grid-connected operation scenario of grid-type converters in new energy power plants, upgrading the traditional intrusive offline parameter identification scheme to a non-intrusive online accurate estimation scheme based on deep reinforcement learning.

[0031] According to a third aspect of the embodiments of this specification, a computing device is provided, comprising:

[0032] Memory and processor;

[0033] The memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions, which, when executed by the processor, implement the steps of the method described in the first aspect above.

[0034] In the embodiments of this specification, the data-driven online estimation method for the inertia parameters of the grid-type converter described above is executed by a computing device, which realizes non-intrusive online accurate estimation of the virtual inertia and virtual damping of the grid-type converter, facilitating engineering deployment and integration.

[0035] According to a fourth aspect of the embodiments of this specification, a computer-readable storage medium is provided that stores computer-executable instructions that, when executed by a processor, implement the steps of the method described in the first aspect above.

[0036] In the embodiments of this specification, the instructions for storing and executing the data-driven online estimation method for the inertia parameters of the above-described grid-type converter through a computer-readable storage medium provide a convenient software carrier for practical engineering applications, facilitating deployment and operation in new energy power station monitoring systems or edge computing devices.

[0037] One embodiment of this specification implements data-driven online estimation of inertia parameters for grid-connected converters. A simulation environment is constructed using a series-parallel identification model, and active power is synchronously distributed to the actual grid-connected converter branch and the neural network identification model branch. The negative time-series error between the measured bus frequency and the model-fitted frequency is used as the reward function to train a deep reinforcement learning agent, enabling the agent to learn frequency response delay and attenuation characteristics. The agent uses a continuous action space to output estimated values ​​of virtual inertia and virtual damping, achieving non-intrusive online accurate estimation of the grid-connected converter's inertia level without relying on internal equipment parameters. Simultaneously, a near-end policy optimization algorithm is used for training, and the policy update amplitude is constrained by shearing the objective function to ensure training stability and convergence efficiency, adapting to different grid strength and power disturbance conditions.

[0038] Furthermore, by extending the multi-agent collaborative framework, the electrical coupling problem in multi-machine parallel scenarios can be effectively addressed. This solves problems inherent in existing technologies, such as traditional methods neglecting phase-locked loop (PLL) and digital control delays leading to parameter estimation bias, quantization errors introducing discrete action spaces limiting accuracy, the inability of single-agent architectures to handle multi-machine coupling, insufficient transient fitting capabilities due to reward functions based solely on steady-state errors, disruptions to normal equipment operation caused by intrusive or offline methods, and the difficulty for operators to access virtual inertial control parameters as manufacturer confidentiality. It achieves collaborative optimization of online accurate estimation of inertial parameters and dynamic feature learning based on serial-parallel identification models and deep reinforcement learning, enabling the agent to adapt to different operating conditions. This significantly improves the accuracy and engineering practicality of inertial parameter estimation for grid-connected converters and enhances the scalability to the multi-agent collaborative estimation framework. Attached Figure Description

[0039] Figure 1 This is a schematic diagram of a typical grid-connected new energy converter control system provided in one embodiment of this specification;

[0040] Figure 2 This is a flowchart illustrating a data-driven online estimation method for the inertia parameters of a grid-type converter provided in one embodiment of this specification.

[0041] Figure 3 This is a schematic diagram of the overall process of a data-driven online estimation method for the inertia parameters of a grid-type converter provided in one embodiment of this specification;

[0042] Figure 4 This is a schematic diagram of an intelligent agent interaction environment based on a serial-parallel identification model provided in one embodiment of this specification;

[0043] Figure 5 This is a schematic diagram of an agent network structure provided in one embodiment of this specification;

[0044] Figure 6 This is a corresponding embodiment provided in this specification. and A schematic diagram of the stability boundary of a grid-type converter;

[0045] Figure 7 This is a schematic diagram of the simulation verification results of a grid-type converter under grid frequency disturbance conditions provided in one embodiment of this specification;

[0046] Figure 8 This is a schematic diagram of the frequency and active power of a grid converter at different times during a simulation process provided in one embodiment of this specification;

[0047] Figure 9 This is a schematic diagram of the frequency and active power of a grid converter at different times during a simulation process provided in one embodiment of this specification;

[0048] Figure 10 This is a schematic diagram of a data-driven online estimation system for the inertia parameters of a grid-type converter, provided in one embodiment of this specification.

[0049] Figure 11 This is a structural block diagram of a computing device provided in one embodiment of this specification. Detailed Implementation

[0050] Many specific details are set forth in the following description to provide a full understanding of this specification. However, this specification can be implemented in many other ways than those described herein, and those skilled in the art can make similar extensions without departing from the spirit of this specification. Therefore, this specification is not limited to the specific implementations disclosed below.

[0051] The terminology used in one or more embodiments of this specification is for the purpose of describing particular embodiments only and is not intended to be limiting of the one or more embodiments of this specification. The singular forms “a” and “the” as used in one or more embodiments of this specification and the appended claims are also intended to include the plural forms, unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used in one or more embodiments of this specification refers to and includes any or all possible combinations of one or more associated listed items.

[0052] It should be understood that although the terms first, second, etc., may be used to describe various information in one or more embodiments of this specification, such information should not be limited to these terms. These terms are only used to distinguish information of the same type from one another. For example, first may also be referred to as second without departing from the scope of one or more embodiments of this specification, and similarly, second may also be referred to as first. Depending on the context, the word "if" as used herein may be interpreted as "when," "when," or "in response to a determination."

[0053] With the continuous increase in the proportion of renewable energy power generation and the in-depth advancement of the construction of new power systems, grid-forming converters (GFMCs) have become key equipment for the stable operation of high-proportion renewable energy power systems. Unlike traditional grid-connected converters, GFMCs actively introduce virtual inertia and virtual damping into the control system by simulating the rotor motion equations of a synchronous generator. This enables the converter to autonomously construct voltage and frequency, providing inertia support and frequency regulation during grid disturbances. Its virtual inertia and virtual damping parameters directly determine the converter's frequency support characteristics and dynamic stability, serving as the core basis for grid operators to assess system inertia levels and formulate dispatch strategies. However, since the frequency response of grid-connected converters is based on digital signal control, the frequency measurement delay of the phase-locked loop (PLL), the signal sampling and calculation delay, and the PWM modulation delay collectively constitute a significant frequency response delay and attenuation effect. Different delay components exhibit varying amplitude-frequency and phase-frequency characteristics to disturbances in different frequency bands. In particular, the phase lag in the low-frequency band caused by the bandwidth limitation of the PLL under weak grid conditions, and the inherent delay introduced by PWM modulation in the high-frequency band, cause the actual frequency response behavior of the converter to deviate significantly from the ideal rotor motion equation. At the same time, the virtual inertia and virtual damping parameters of grid-connected converters are core confidential information of equipment manufacturers, which grid operators cannot obtain through internal channels. The combination of these two factors poses a severe challenge to the quantitative assessment of the inertia support capability of grid-connected converters.

[0054] To address the need for online estimation of inertia parameters in grid-connected converters, current engineering practices rely on traditional parameter identification techniques, such as the least squares method supported by synchronous phasor measurement units (PMUs) and wide-area measurement systems (WAMS), to identify inertia parameters by pre-setting a power step signal or applying external disturbances at the converter's grid connection point. However, this approach reveals fundamental flaws in practical grid-connected converter applications: traditional methods assume real-time frequency response, completely ignoring the inherent delay and attenuation effects of phase-locked loops and digital control loops. A significant mismatch exists between the identification model and the actual frequency response characteristics of the grid-connected converter, resulting in severe parameter estimation bias when directly applied. Furthermore, invasive or offline methods require pre-setting a power step signal or applying external disturbances, affecting normal equipment operation and failing to meet the requirements of online monitoring of grid-connected equipment. The second approach is an intelligent inertia estimation method based on deep reinforcement learning, which designs the inertia control parameter action space as discrete values ​​to avoid the problem of excessively large action space. While this type of method can achieve a certain degree of inertia parameter identification in a simulation environment, its analytical framework is based on a discretized parameter space. In real physical systems, inertia parameters are essentially continuous variables, and discretization inevitably introduces quantization errors. While improving the discretization accuracy can reduce errors, it leads to a sharp expansion of the action space and an exponential increase in training complexity, creating an irreconcilable contradiction between high accuracy and training feasibility. Furthermore, its reward function is designed only based on the steady-state error of frequency measurement data, failing to fully reflect the dynamic characteristics of the frequency response. This results in insufficient frequency response fitting ability in transient processes, fundamentally limiting the dynamic accuracy of inertia parameter estimation.

[0055] It is evident that neither traditional parameter identification methods nor discrete action space deep reinforcement learning schemes can fully consider the frequency response delay and attenuation characteristics of grid-type converters without knowing the internal parameters, and thus fail to achieve high-precision, non-intrusive online estimation of virtual inertia and virtual damping.

[0056] To address the aforementioned technical problems, this specification proposes a data-driven online estimation method and system for the inertia parameters of a grid-type converter. The method first collects three-phase voltage time-series data, three-phase current time-series data, and bus frequency time-series data from the output side of the grid-type converter. Active power time-series data is calculated based on the three-phase voltage and current time-series data. The active power time-series data and bus frequency time-series data are then input as state information into a trained deep reinforcement learning agent, and the estimated values ​​of the inertia parameters are output. The deep reinforcement learning agent is trained as follows: A simulation environment is constructed based on a series-parallel identification model. The active power time series of the grid-connected converter is used as input, synchronously splitting to the actual grid-connected converter branch and the neural network identification model branch. The actual grid-connected converter branch outputs the measured bus frequency, and the neural network identification model branch outputs the model-fitted frequency. The time series error between the measured bus frequency and the model-fitted frequency is used to calculate the reward function value. In the simulation environment, the negative value of the time series error is used as the reward function, and the deep reinforcement learning agent is trained using a near-end policy optimization algorithm. Based on this, the actual grid-connected converter branch incorporates a virtual inertial dynamic function that considers delay, and includes rotor motion equations, a first-order delay stage of the phase-locked loop, and PWM modulation attenuation characteristics. Determine the measured bus frequency, where The total equivalent delay constant for phase-locked loop frequency measurement delay, signal sampling calculation delay, and PWM modulation delay is used to fundamentally eliminate parameter estimation bias caused by delay factors. The neural network identification model branch receives active power timing data and inertia parameter estimates from the deep reinforcement learning agent. After nonlinear mapping by the shared feature extraction layer, it outputs the model-fitted frequency. The deep reinforcement learning agent uses the estimated inertia parameters from the continuous action space and uses the negative timing error as a reward function to enable the agent to learn the dynamic characteristics of the frequency response.

[0057] The embodiments in this specification completely break through the physical limitations of traditional methods, which are limited by parameter estimation bias and discrete action space quantization error due to neglecting delay factors. Without knowing the internal parameters, the frequency response delay and attenuation characteristics of the grid converter are restored, enabling the deep reinforcement learning agent to learn the complete dynamic characteristics of the frequency response, significantly improving the inertia parameter estimation accuracy, transient fitting ability and engineering practicality of the grid converter.

[0058] This specification provides a data-driven online estimation method for the inertia parameters of a grid-type converter. It also relates to a data-driven online estimation system for the inertia parameters of a grid-type converter, a computing device, and a computer-readable storage medium, which will be described in detail in the following embodiments.

[0059] The following is a detailed description of the data-driven online estimation method for the inertia parameters of a grid-type converter, as illustrated in the embodiments of this specification.

[0060] First, the principle of the grid converter will be explained.

[0061] Typical grid-connected new energy converter control system, such as Figure 1 As shown. Figure 1 In this grid-connected converter, a two-level converter is used. The DC side of the converter is powered by new energy power generation equipment such as photovoltaic cells, while the AC side is filtered at high frequency by an LC filter and connected to the AC grid at the point of common coupling (PCC). Its control architecture includes modules for signal sampling, VSG control, power control, dual-loop voltage and current control, and PWM signal generation.

[0062] According to electrical machinery theory, the rotor kinetic energy of a synchronous generator can be expressed as:

[0063]

[0064] In the formula, Let be the inertial constant of the synchronous generator. Let be the rotor angular velocity of the synchronous generator. The energy stored in the rotor is related to the inertial constant and the angular velocity. The oscillating motion of the rotor can be described by the following equation:

[0065]

[0066] In the formula, This represents the damping coefficient of the synchronous generator. and These are the mechanical torque and electromagnetic torque of the synchronous generator, respectively.

[0067] The outer loop of a grid-connected converter comprises two parts: damping and inertia control, and active and reactive power control with synchronous motor characteristics. The grid-connected converter incorporates the rotor motion equations of a synchronous generator through virtual inertial control technology. Its active power-frequency control equations simulate the swing equations of a synchronous generator. The virtual inertia parameter characterizes the converter's inertial response to frequency fluctuations, while the virtual damping parameter characterizes its ability to attenuate frequency oscillations. Virtual inertial control, by adjusting the active power-frequency characteristics, enables the converter to suppress grid frequency fluctuations. Its inertia support characteristics are directly affected by the virtual inertia and virtual damping parameters.

[0068] The frequency response of a grid-type converter involves the following processes, exhibiting significant delay and attenuation effects. In the electrical quantity measurement stage, the converter needs to measure electrical quantities, including frequency. Taking a synchronous reference coordinate system phase-locked loop (PLL) as an example, its non-ideal tracking characteristics introduce a non-negligible delay and attenuation effect into the frequency response. In the signal transmission and digital processing stages, the sampling, calculation, and PWM modulation stages of digital signal control all have inherent delays. The PLL frequency measurement delay, signal sampling and calculation delay, and PWM modulation delay collectively constitute a significant frequency response delay and attenuation effect.

[0069] The data-driven online estimation method for the inertia parameters of a grid-type converter according to embodiments of this specification will be described below with reference to the accompanying drawings. Figure 2 This is a flowchart illustrating the data-driven online estimation method for the inertia parameters of a grid-type converter provided in the embodiments of this specification. Figure 2 As shown, it includes the following steps:

[0070] In step 210, the three-phase voltage timing data, three-phase current timing data, and bus frequency timing data of the grid-type converter output side are collected, and the active power timing data are calculated based on the three-phase voltage timing data and the three-phase current timing data.

[0071] In the embodiments of this specification, three-phase voltage and current time-series data are collected from the output side of the grid-connected converter. After coordinate transformation, the dq-axis voltage and current components are obtained, and then the active power time-series data is calculated. This active power time-series data serves as the sole common input signal for the actual grid-connected converter branch and the neural network identification model branch in the series-parallel identification model, and is the fundamental data source for achieving data-driven estimation.

[0072] In the embodiments of this specification, the bus frequency timing data is bus frequency timing data measured based on a phase-locked loop (PLL). The phase lag in the low-frequency band caused by the bandwidth limitation of the PLL under weak power grid conditions introduces a non-negligible delay and attenuation effect to the frequency response. This frequency timing data, together with the active power timing data, constitutes the state input of the deep reinforcement learning agent.

[0073] In step 220, the active power time-series data and the bus frequency time-series data are used as state information input to the trained deep reinforcement learning agent. The deep reinforcement learning agent is trained in a simulation environment, which is constructed by a series-parallel identification model. The series-parallel identification model is used to simulate the frequency response delay characteristics of the phase-locked loop of the grid-type converter.

[0074] In the embodiments of this specification, the deep reinforcement learning agent includes an input layer, a shared feature extraction layer, an Actor network, and a Critic network. The input layer receives active power time-series data and bus frequency time-series data; the shared feature extraction layer performs nonlinear feature extraction on the input data; the Actor network outputs estimated inertia parameters in the continuous action space, and its output layer uses a Gaussian policy to output the action mean and variance; the Critic network outputs state values. Through the structural design of the Actor network and Critic network sharing a feature extraction layer, the agent can simultaneously learn policy mapping and state value evaluation from the same state representation; the Actor network uses a Gaussian policy to output the action mean and variance, achieving a balance between exploration and utilization in the continuous action space, overcoming the inherent quantization error in the discrete action space, and significantly improving the accuracy and training efficiency of inertia parameter estimation.

[0075] The deep reinforcement learning agent is trained as follows: a simulation environment is constructed based on a series-parallel identification model; in the series-parallel identification model, the active power time series of the grid-connected point of the grid-connected converter is used as input, and synchronously distributed to the actual grid-connected converter branch and the neural network identification model branch; the agent outputs an action based on the current state, including an estimated value of the inertia parameter; the neural network identification model branch receives the action and the active power time series, and outputs the model-fitted frequency; the actual grid-connected converter branch outputs the measured bus frequency; the time series error between the measured bus frequency and the model-fitted frequency is calculated; the negative value of the time series error is used as the reward function, and the near-end policy optimization algorithm is used to train the agent. During training, the agent outputs an action based on the current state, the simulation environment calculates the time series error based on the action and returns the reward function value, and the near-end policy optimization algorithm updates the agent's parameters based on the reward function value until the reward function converges.

[0076] In the embodiments described in this specification, the branches of the real grid-type converter incorporate a virtual inertial dynamic function that takes time delay into account. This virtual inertial dynamic function incorporates the rotor motion equation, the first-order delay element of the phase-locked loop, and the PWM modulation attenuation characteristics. The measured bus frequency is determined by the following formula: ,in, for The measured bus frequency of the branch output of the real-time grid-type converter. for Input the active power timing data of the actual grid-type converter branch at all times. The virtual inertia coefficient of the grid-type converter. This represents the virtual damping coefficient of the grid-type converter. This is the total equivalent delay constant for phase-locked loop (PLL) frequency measurement delay, signal sampling calculation delay, and PWM modulation delay. By embedding a virtual inertial dynamic function that takes delay into the actual grid-type converter branch, the first-order delay element of the PLL and the attenuation characteristics of PWM modulation are incorporated into the model, enabling the measured bus frequency to reflect the true frequency response delay and attenuation characteristics of the grid-type converter.

[0077] The neural network identification model branch receives the active power time series and the estimated inertia parameters output by the agent. After nonlinear mapping by the shared feature extraction layer, it outputs the model fitting frequency. ,in, for The active power time-series data of the neural network identification model branches are input at all times. for The estimated value of the inertia parameter output by the agent at each time step. The network parameters of the intelligent agent are used. The active power timing data and the estimated inertia parameters output by the intelligent agent are received by the model branch through the neural network identification. After nonlinear mapping through the shared feature extraction layer, the model fitting frequency is output, realizing end-to-end learning and fitting of the frequency response characteristics of the grid-type converter.

[0078] In step 230, the deep reinforcement learning agent outputs actions, including estimates of the virtual inertia parameters and virtual damping parameters of the grid converter.

[0079] In the embodiments of this specification, the deep reinforcement learning agent uses the estimated value of the inertia parameter output from the continuous action space. By using the estimated value of the inertia parameter output from the continuous action space, the inherent quantization error limitation of the discrete action space is overcome, and higher accuracy inertia parameter estimation is achieved.

[0080] In the embodiments of this specification, the estimated values ​​of the virtual inertia parameters output by the intelligent agent are used to characterize the virtual inertia level of the grid-type converter under the current operating conditions, and the estimated values ​​of the virtual damping parameters are used to characterize the virtual damping level of the grid-type converter under the current operating conditions. The virtual inertia level is used to quantitatively evaluate the inertia support capability of the grid-type converter. This result can be applied to monitor the real-time changes in the inertia support capability of the converter, assess the overall inertia level of new energy power plants, and provide a data foundation for grid dispatching and stability analysis.

[0081] The data-driven online estimation method for the inertia parameters of a grid-type converter, as described in this specification, will now be explained in detail with reference to the accompanying drawings. Figure 3 This is a schematic diagram illustrating the overall flow of the data-driven online estimation method for the inertia parameters of a grid-type converter provided in the embodiments of this specification. Figure 3 As shown, the method includes an offline training phase and an online deployment phase, specifically including the following steps:

[0082] Step 1: Analyze the frequency response mechanism and delay characteristics of the grid-type converter.

[0083] Historical electrical quantity data of the grid-type converter output side are collected, including three-phase voltage time-series data, three-phase current time-series data and bus frequency time-series data. Based on the collected data, the amplitude-frequency characteristics and phase-frequency characteristics of the frequency response are analyzed, and the delay characteristic parameters introduced by factors such as limited phase-locked loop bandwidth, control delay and sampling delay are quantitatively extracted.

[0084] In the embodiments described in this specification, when the grid-connected converter operates under weak grid conditions, the low-frequency phase lag effect of the phase-locked loop (PLL) will cause the frequency response to exhibit non-negligible delay and attenuation characteristics. By obtaining the system's frequency response transfer function model through frequency domain scanning or parameter identification methods, the time constant and phase offset of the delay element are clarified, providing a theoretical basis for accurate dynamic modeling of the frequency response in the subsequent simulation environment. The delay characteristic parameters obtained in this step will be used to constrain the dynamic response model in the interactive simulation environment, ensuring that the simulation environment and the actual system maintain consistency in frequency dynamic behavior.

[0085] Step 2: Build an interactive simulation environment based on the serial-parallel identification model, and define the state space, action space and reward function.

[0086] Based on the frequency response mechanism and delay characteristics, a series-parallel identification model is constructed, which includes branches of a real network converter and branches of a neural network identification model. On this basis, a deep reinforcement learning interactive simulation environment is built; at the same time, the state space, action space and reward function are defined.

[0087] The following is in conjunction with the appendix Figure 4 This specification provides a detailed description of the intelligent interactive environment based on the serial-parallel identification model in the embodiments of this specification. Figure 4 This diagram illustrates an intelligent agent interaction environment based on a series-parallel identification model, provided as an embodiment of this specification. Using a deep reinforcement learning framework, the diagram systematically demonstrates the specific implementation and data interaction logic of the series-parallel identification model in the online estimation of inertia parameters of a grid-type converter. Figure 4 As shown, the intelligent interactive environment consists of two main parts: the actual physical space and the deep reinforcement learning identification domain. The two are connected by a common input signal, frequency response output signal and error feedback signal to form a complete closed-loop identification link, as detailed below.

[0088] The basic principle of the series-parallel identification model is that the system to be identified and the neural network model receive exactly the same input data. Based on this, the active power time sequence of the grid-connected point of the grid-connected converter is acquired in real time. It is used as the only common input signal in the series-parallel identification model. Simultaneously, the traffic is split into two branches:

[0089]

[0090] In the formula, for The unified input of the series-parallel identification model at all times is the normalized active power at the converter grid connection point; for Time sequence of measured active power output at the outlet of the grid-type converter.

[0091] After being synchronously sampled, the common input signal is simultaneously sent to the following two branches:

[0092]

[0093] In the embodiments described in this specification, the two inputs are strictly kept to be from the same source and sampled synchronously to ensure that the input variables are completely consistent, thereby forming a parallel input channel for the identification structure and providing fair and consistent excitation conditions for subsequent error-driven parameter identification.

[0094] In the actual physical space, in the real-network converter branch, the active power input... After passing through the main circuit, virtual inertial control, and phase-locked loop delay stage, the measured bus frequency is output. Its mathematical expression is:

[0095]

[0096] In the formula, To account for the time delay in the virtual inertial dynamic function of the grid-type converter, the rotor motion equation and the first-order delay element of the phase-locked loop are incorporated. And PWM modulation attenuation characteristics; , These are the actual virtual inertia and virtual damping parameters of the grid-type converter, respectively. It is the total equivalent delay constant of phase-locked loop delay, sampling delay and PWM modulation delay.

[0097] Among them, in the neural network identification model branch in the deep reinforcement learning identification domain, the active input... The estimated continuous inertia parameters output by the Actor network The data are fed into the neural network model. The action output... Defined as:

[0098]

[0099] in, and These are the estimated values ​​of the virtual inertia and virtual damping of the network converter by the neural network at the current moment.

[0100] After the neural network model passes through a shared feature extraction layer and an equivalent dynamic mapping network of a mesh converter, the output model fitting frequency is... Its mathematical expression is:

[0101]

[0102] In the formula, A nonlinear deep mapping network constructed for sharing feature layers in neural networks; These are all the trainable weight parameters of the neural network.

[0103] In the embodiments described in this specification, the actual frequency response It reflects the frequency output behavior of the grid-type converter under real physical dynamic characteristics, fully encompassing the frequency response delay and attenuation characteristics introduced by factors such as limited PLL bandwidth, digital control delay, and PWM modulation attenuation; and estimates the frequency response. This is the frequency response prediction of the neural network for the same input signal under the current parameters. The difference between the two constitutes the fundamental source of identification error.

[0104] actual frequency response With estimated frequency response The data is synchronously fed into the frequency response comparison module to complete the dynamic frequency deviation coupling calculation and calculate the frequency error at a single moment. :

[0105]

[0106] Based on this, to further evaluate the overall recognition accuracy within the timing window, a timing window cumulative error is introduced:

[0107]

[0108] The negative value of the single-step frequency error is used as the instantaneous reward for the single step. This feedback is then sent to the PPO agent network. The immediate reward function is defined as a negative error value, meaning the smaller the error, the higher the reward.

[0109]

[0110] In addition, considering the cumulative impact in the future, the cumulative discount reward is calculated in the form of a discount reward. :

[0111]

[0112] In the formula, As a discount factor, satisfying ; This represents the total number of sampling steps for a single interactive trajectory.

[0113] In the embodiments described in this specification, the reward function Essentially, it is a monotonically decreasing function of the frequency response error; that is, the smaller the frequency response error, the larger the reward signal, and vice versa. This reward signal quantifies the accuracy of the neural network model in fitting the frequency response characteristics of the grid-type converter under the current parameters.

[0114] Based on instant rewards Calculate the generalized dominance function The algorithm employs a proximal strategy optimization method to backpropagate gradients of the sheared objective function and update the neural network parameters. Correcting inertia action output Complete one round of identification iteration:

[0115]

[0116] The updated inertia estimate is as follows:

[0117]

[0118] The updated inertia estimate is then fed back into the neural network mapping. The system continuously executes a closed-loop identification process of "input-output-error-update" until the accumulated reward is reached. Convergence means that the neural network model has accurately learned the dynamic characteristics of the inertia of the network converter. At this point, the network output is the estimated value of the identified inertia parameters.

[0119] In the embodiments described in this specification, the near-end policy optimization algorithm limits the policy update step size through a shearing objective function, avoiding training instability caused by excessively large parameter update amplitudes. The entire interactive environment forms a complete closed-loop learning mechanism through a cyclical iteration of "state input—action output—error calculation—reward feedback—parameter update," enabling the frequency output of the neural network identification model to gradually approach the actual frequency response, ultimately achieving data-driven online estimation of the inertia parameters of the grid-type converter.

[0120] In conclusion, Figure 4The agent interaction environment based on the serial-parallel identification model shown here constructs a dual-branch parallel architecture of "actual physical space" and "deep reinforcement learning identification domain." Using the active power at the grid connection point with synchronously distributed current as the sole common input signal and frequency response error as the core driving signal, a closed-loop learning mechanism is formed through reward function mapping and near-end policy optimization algorithms. This enables the deep reinforcement learning agent to identify and dynamically track the virtual inertia and virtual damping parameters of the grid-type converter online. This structure fully considers non-ideal characteristics such as phase-locked loop delay, digital control delay, and PWM modulation attenuation, using the difference between the actual and estimated frequency response as the identification basis. This avoids the dependence of traditional identification methods on precisely known system mathematical models, and has significant engineering practical value.

[0121] Step 3: Train the agent using a proximal strategy optimization algorithm.

[0122] The interactive simulation environment is combined with the near-end policy optimization algorithm to iteratively train the agent. In each training round, the agent outputs an action based on the current state, the environment executes the action and returns a reward signal and the next state. The algorithm converges based on the reward function value. If it does not converge, the parameters of the Actor network and Critic network are updated and training continues. If it converges, training is terminated and the trained agent model is saved.

[0123] The following is in conjunction with the appendix Figure 5 The training process of the agent based on proximal policy optimization in the embodiments of this specification will be described. Figure 5 This diagram illustrates the agent network structure provided in the embodiments of this specification. It shows that the agent is a dual-branch collaborative architecture of the Actor and Critic networks, and the closed-loop interaction logic between the agent and the simulation environment. Figure 5 As shown, the training of the agent relies on the proximal policy optimization algorithm framework, which specifically includes three aspects: network structure design, training mechanism, and data input strategy.

[0124] The agent consists of two complementary neural networks: the Actor network and the Critic network. They share the bottom feature extraction layer and are decoupled into independent functional branches at the high level, respectively undertaking the tasks of policy decision-making and state value evaluation.

[0125] The network input layer receives normalized real-time status data, specifically including the active power timing of the grid-connected points of the grid-connected converters. Timing with bus frequency The input data first undergoes feature transformation through a fully connected layer. This layer integrates batch normalization to accelerate training convergence and mitigate internal covariate bias. Subsequently, the feature vector is processed by a Dropout layer, which randomly deactivates some neurons to prevent overfitting. The activation function can be ReLU or Swish to introduce a non-linear mapping. After these layers of processing, the output is a shared feature tensor. This serves as the common input basis for subsequent Actor and Critic branches.

[0126] Next, the Actor network receives the shared feature tensor. After undergoing a nonlinear transformation through two fully connected layers, the output layer is connected. Given that the inertia parameter is a continuous variable, the output layer employs a Gaussian strategy, outputting the mean values ​​of the motion in each dimension of the continuous motion space. With variance The Actor network is based on the current state. Gaussian policy sampling yields continuous action outputs. That is, the estimated value of the inertia parameter at the current moment. The introduction of Gaussian strategy enables agents to conduct refined exploration in the continuous action space, avoiding quantization errors caused by discretization.

[0127] Critic network receives shared feature tensors After feature extraction through two fully connected layers, the system is connected to the output layer. The output layer uses a linear activation function to directly output the current state. The value of the state under the following conditions This value represents the expected cumulative discounted reward that the agent can obtain by starting from the current state and executing the current policy. It is used to calculate the advantage function in generalized advantage estimation. This provides directional guidance for policy updates in the Actor network.

[0128] like Figure 5 As shown, the intelligent agent and the simulation environment form a complete closed-loop interaction circuit. The specific interaction process is as follows:

[0129] (1) The simulation environment will display the current state This includes normalized active power. With frequency Input to the agent network;

[0130] (2) The Actor network determines the current state. Output Action That is, the estimated value of the inertia parameter, which is applied to the neural network identification model branch in the simulation environment;

[0131] (3) After the simulation environment executes the action, calculate the frequency response error and generate an instant reward. Simultaneously, it transitions to the state of the next time step. The reward signal is fed back to the agent along with the next state;

[0132] (4) The Actor network updates the policy gradient based on the reward signal and the advantage function; the Critic network updates the state value function based on the temporal difference error.

[0133] like Figure 5 As shown in the embodiments of this specification, the proximal policy optimization algorithm is used to collaboratively train the dual-branch network. The core training process is as follows:

[0134] The agent samples data in the simulation environment according to the current policy, collecting trajectory data including state, action, reward, and next state to construct a batch experience pool. The advantage function is calculated using a generalized advantage estimation method. To balance bias and variance, an accurate benchmark is provided for strategy evaluation. A PPO shearing objective function is constructed, using a shearing factor... Limit the probability ratio of new and old strategies The range of variation is controlled to avoid training instability caused by excessively large policy updates. Its form is as follows:

[0135]

[0136] Based on this objective function, gradient ascent is used to update the Actor network parameters, while the Critic network parameters are updated by minimizing the temporal difference error. "Multi-step sampling" and "multi-round parameter updates" are performed alternately until the cumulative reward function converges to a stable threshold, indicating that the agent has accurately learned the dynamic inertia mapping relationship of the network converter.

[0137] During the training phase, the input data for the agent is the active power timing data of the grid-connected points of the grid-type converter. Timing with bus frequency Both were normalized before being input into the network.

[0138] To enhance the generalization ability and engineering adaptability of the agent, the embodiments in this specification are fully trained under various typical operating conditions, including: different power command step scenarios: covering power command mutations of different amplitudes and rates to simulate actual scheduling command changes; different grid frequency disturbance scenarios: including various disturbance forms such as frequency offset and frequency fluctuation, to test the agent's recognition robustness under non-ideal frequency quality; and different grid strength scenarios: covering grid operating conditions under high, medium, and low short-circuit ratios to verify the agent's adaptive ability in weak and strong grid environments.

[0139] Through joint training under diverse operating conditions, the agent can learn generalization features beyond a single operating condition, ensuring stable and accurate inertia parameter estimation performance during the online deployment phase under complex and ever-changing actual operating conditions.

[0140] In conclusion, Figure 5 The agent network structure shown is based on a shared feature extraction layer. Through functional decoupling of the Actor and Critic networks, the agent network completes the policy decision-making and value assessment tasks respectively. Stable collaborative training is achieved by using the shearing objective function and generalized advantage estimation of the PPO algorithm. Combined with the multi-condition training data input strategy, the model's generalization ability and engineering practicality in complex power system scenarios are guaranteed.

[0141] Step 4: Collect and normalize real-time data.

[0142] The three-phase voltage time-series data, three-phase current time-series data, and bus frequency time-series data of the grid-type converter output side under real-time operating conditions are collected, and the electrical quantity data are normalized to ensure that each input characteristic is within a uniform numerical range.

[0143] After normalization, the bus frequency time-series data and active power time-series data constitute a real-time state vector, which serves as the input to the trained agent model. Normalization eliminates the impact of dimensional differences between different electrical quantities on the model's inference accuracy, ensuring that the input data distribution during online deployment is consistent with the data distribution during offline training.

[0144] Step 5: Input the data into the trained Actor network for forward inference.

[0145] The normalized real-time state vector is input into the stored trained agent model, and the corresponding action output is obtained through the forward propagation of the Actor network.

[0146] In the embodiments described in this specification, the Actor network adopts a multilayer perceptron structure, including an input layer, several hidden layers, and an output layer. The activation function of the hidden layers is the ReLU function, and the activation function of the output layer is the Tanh function, mapping the action output to a preset value range. The forward inference process only involves matrix multiplication and nonlinear activation operations, resulting in low computational load and low latency, meeting the engineering requirements for real-time online estimation of inertia parameters in network-type converters.

[0147] Step 6: Output the estimated inertia parameters of the grid-type converter.

[0148] Based on the action output from the forward inference of the Actor network, combined with the inertia parameter estimate from the previous moment, the inertia parameter estimate of the current grid-type converter is calculated and output.

[0149] In the embodiments of this specification, the method for updating the estimated inertia parameters is as follows:

[0150]

[0151] in, The estimated value of the inertia parameter at the current moment. The inertia parameter is the estimated value from the previous moment. This represents the action increment output by the Actor network at the current moment. The final estimated inertia parameter can be used for dynamic security assessment and adaptive control strategy adjustment of grid-connected converters, providing real-time inertia situational awareness information for power systems with a high proportion of renewable energy. This completes the data-driven online estimation of the inertia parameters of the grid-connected converter.

[0152] Furthermore, it should be noted that, for multi-machine parallel scenarios, this invention extends the aforementioned single-machine identification scheme into a multi-agent cooperative estimation framework. Its core idea is to equip each grid-type converter with an independent agent, and each agent achieves system-level cooperative estimation of the inertia parameters of each converter by sharing global reward signals and joint training.

[0153] In terms of framework composition, assuming there are N converters in the system, N agents are configured accordingly. Each agent has its own Actor network and Critic network, which are responsible for outputting the inertia estimate and evaluating the current state value of the converter, respectively. The observation information of each agent is only taken from the active power and frequency data of its own converter, without needing to obtain the internal state of other converters, thus ensuring the independence of the execution phase.

[0154] The core innovation in the training phase lies in the design of the global reward signal. Instead of using their own local rewards, each agent receives a unified system-level global reward, primarily consisting of a negative value for the common bus frequency deviation, supplemented by a slight penalty for the magnitude of the action. This aligns the optimization goals of all agents with improving the frequency response performance of the entire multi-machine system, effectively avoiding the local optima problem caused by each agent acting independently.

[0155] The training algorithm employs a multi-agent proximal policy optimization algorithm. During training, each agent's Critic network can access the state and action information of all agents, enabling centralized value evaluation and fully considering the coupling effects between multiple agents. During execution, each agent relies solely on its own observations for forward inference, without the need for mutual communication or data aggregation. The latency of a single inference is comparable to that of a single-machine solution, fully meeting the requirements for online real-time performance.

[0156] For homogeneous converters, a parameter-sharing strategy can be adopted among the agents, meaning all agents share the same set of network parameters. This significantly reduces the number of training parameters, accelerates convergence, and enhances the consistency of behavior among agents under similar operating conditions.

[0157] This multi-agent framework effectively decouples the frequency coupling effect under multi-machine parallel operation and accurately identifies the independent inertia contribution of each converter. It uses system frequency deviation as a common objective to ensure the optimality of the estimation results at the system level. Furthermore, it has no communication dependency during the execution phase, resulting in high real-time performance. Finally, it supports flexible expansion with any number of converters, demonstrating good engineering scalability. In summary, this framework provides an effective solution for online collaborative estimation of inertia parameters of grid-connected converters in multi-machine scenarios.

[0158] In summary, the embodiments of this invention fully consider the inherent delay and attenuation effects of the phase-locked loop frequency measurement and digital control stages. By using a neural network to perform end-to-end fitting of the aforementioned complex nonlinear influencing factors, it overcomes the problem of high-frequency parameter estimation failure caused by neglecting delay characteristics in traditional identification methods, maintaining high identification accuracy across the entire frequency domain. This invention only requires the acquisition of active power and frequency data during the normal operation of the converter as input, without the need for preset power step disturbances or the injection of any additional test signals into the system. The entire estimation process does not affect the normal operation of the equipment, making it highly feasible for engineering implementation and possessing good prospects for field application. Adopting a purely data-driven technical approach, it does not rely on the confidential internal control parameters and detailed structural information of grid-connected converter manufacturers. Accurate estimation of inertia parameters can be completed solely based on measurable electrical quantities external to the grid connection point, effectively solving the application bottleneck of unavailable internal equipment parameters in practical engineering. It supports expansion to a multi-agent collaborative estimation architecture, adapting to typical scenarios where multiple grid-connected converters in new energy power plants operate in parallel via a common bus. This enables independent decoupling and collaborative estimation of the inertia contribution of each converter, providing feasible technical support for power plant-level system inertia level assessment. Employing an offline training and online deployment approach, the online inference phase involves only forward propagation computation of a lightweight neural network, resulting in low computational load and low latency, fully meeting the timeliness requirements of real-time inertia parameter estimation for online monitoring in engineering projects.

[0159] like Figure 1 In this grid-connected converter, a two-level converter is used. The DC side of the converter is powered by new energy power generation equipment such as photovoltaic cells, while the AC side is filtered at high frequency by an LC filter and connected to the AC grid at the point of common coupling (PCC). Its control architecture includes modules for signal sampling, VSG control, power control, dual-loop voltage and current control, and PWM signal generation.

[0160] The outer loop of a grid-type converter includes two parts: active power control with damping and inertia, and active and reactive power control with synchronous motor characteristics. The specific formulas are as follows:

[0161]

[0162] In the formula, and These are the rotor position angle and angular velocity of the virtual synchronous motor, respectively. and The voltage amplitude and its setpoint of the reactive power loop output of the grid-type converter; and For ideal active and reactive power; and This refers to the actual active and reactive power on the output side of the grid-type converter. and These are the inertia coefficient and damping coefficient of the virtual synchronous motor.

[0163] The output reference voltage of the grid-type converter is used as the port voltage of the converter. Figure 1 The mathematical model of the filter circuit on the converter side of the grid-type new energy vehicle in the dq coordinate system is as follows:

[0164]

[0165] In the formula: , The dq coordinate components of the output current of the grid-type converter; , The dq coordinate components of the line current; , The output voltage dq coordinate components of a grid-type converter; , The dq coordinate components represent the capacitor voltage of the converter filter. , The grid voltage is represented by the dq coordinate system component.

[0166] With reference to the grid-connected converter coordinate system, the grid voltage and frequency satisfy the following equations:

[0167]

[0168] in, For grid-connected converters, the grid voltage amplitude is... The angular frequency of the power grid. This represents the phase difference between the grid voltage and the d-axis of the reference coordinate system.

[0169] The output power can be determined based on the AC side filter capacitor voltage of the grid-type converter. , With line inductance current , Calculate the active power on the output side of the grid-type converter. and reactive power The expression is:

[0170]

[0171] Dual-loop voltage and current control features fast response and high stability. The inner and outer loops work together to achieve precise control, and when combined with grid-type control, it can improve the reliability of the converter. Figure 1 Current command , With voltage command , The relationship between them is:

[0172]

[0173] Virtual inertia based on networked control loop With virtual damping The root locus, Figure 6 The corresponding and The stability boundary of the grid-type converter. Figure 6 It can be seen that: when the initial virtual damping coefficient At that time, the virtual inertia of the grid-type converter It can remain stable below 20; when When the virtual damping of the grid-type converter is at its minimum operating value, the virtual inertia of the system needs to be guaranteed. It needs to be reduced to below 14 to remain stable.

[0174] Next, a simulation model of a grid-type new energy converter was built. The simulation model consists of a main circuit, a coordinate transformation module, a grid-type control module, and a voltage and current dual-loop control module, with a virtual moment of inertia set. Virtual damping coefficient The simulation duration is set to 5 seconds.

[0175] The simulation assumes a sudden load increase of 0.03MW and a decrease in grid frequency. At this time, the rated power of the grid-connected converter is 0.08MW, and the rated voltage is 380V. When the grid frequency decreases, the grid-connected converter generates additional active power to support the grid frequency. During this process, the stability and frequency support capability of the grid-connected converter are mainly determined by the virtual inertia coefficient. and virtual damping coefficient Decide.

[0176] The following is in conjunction with the appendix Figure 7The simulation verification results of the grid-type converter in the embodiments of this specification under the condition of power grid frequency disturbance are described in detail. Figure 6 The simulation waveform diagram of the grid-type converter provided in the embodiments of this specification under the step disturbance of the power grid frequency includes the dynamic response curves of four dimensions: frequency response, active power, output voltage and output current. It fully presents the dynamic behavior of the system during the frequency disturbance and after the disturbance recovery. The specific analysis is as follows.

[0177] Figure 7 The simulation conditions are set as follows: the rated power of the grid-connected converter is 0.08MW, the rated voltage is 380V, the virtual inertia parameter J=4, the virtual damping parameter D=100, and the total simulation time is 5 seconds. At 2 seconds, the grid frequency experiences a step drop of 0.2Hz; at 3 seconds, the grid frequency recovers to the rated value of 50Hz.

[0178] Figure 7 (a) The frequency response waveform of the grid-connected converter at the grid connection point. As shown in the figure, the grid frequency drops from 50Hz to 49.8Hz at 2 seconds, and recovers to 50Hz at 3 seconds after lasting for 1 second. Throughout the entire frequency disturbance and recovery process, the frequency response of the grid-connected converter is stable, without oscillation or instability, indicating that the system has good frequency tracking stability under the parameter configuration of J=4 and D=100.

[0179] Figure 7 (b) The active power response waveform of the grid-connected converter. During the grid frequency drop at time 2 seconds, the grid-connected converter automatically increases its active power output to support the grid frequency based on the virtual inertia control characteristics. The output active power increases from the initial 0.08MW to approximately 0.12MW, and the converter current increases accordingly, demonstrating the effective support capability of the grid-connected converter for the grid frequency. After the grid frequency recovers to 50Hz at time 3 seconds, the output active power of the grid-connected converter drops back from 0.12MW to the initial operating point of 0.08MW, and the converter current decreases synchronously, allowing the system to smoothly recover to its pre-disturbance operating state. Throughout the entire power regulation process, the grid-connected converter maintains stable operation without power oscillations or instability, verifying the effectiveness of the virtual inertia control strategy during the frequency support and power stabilization phases.

[0180] Figure 7 (c) The output voltage waveform of the grid-type converter. Throughout the entire frequency disturbance and power regulation process, the output voltage amplitude remains stable. Whether in the active power increase phase at 2 seconds or the active power decrease phase at 3 seconds, the output voltage does not fluctuate or drop significantly due to power changes. This indicates that the voltage control loop of the grid-type converter has good anti-disturbance capability and can achieve stable voltage support during active power regulation.

[0181] Figure 7 (d) shows the output current waveform of the grid-type converter. As can be seen from the figure, during the 2-3 second frequency support period, the output current increases accordingly with the increase of active power; after the 3-second frequency recovery, the output current decreases synchronously with the decrease of active power. The current change is smooth throughout the process, without any impact or oscillation, further verifying the stability of the system during transient processes.

[0182] comprehensive Figure 7 The four waveforms shown demonstrate that, under the typical disturbance condition of a 0.2Hz step drop and subsequent recovery of the grid frequency, the grid-type converter, with virtual inertia J=4 and virtual damping D=100, can operate stably, accurately respond to frequency changes, autonomously generate more active power to support the grid during the frequency drop, and smoothly return to the initial power level after the frequency recovers. Simultaneously, it maintains the stability of voltage amplitude and current waveform throughout the entire process, fully verifying the correctness and effectiveness of the grid-type converter simulation model and control parameter configuration used in this invention.

[0183] Figure 8 These are the specific values ​​of frequency and active power of the grid-type converter at different times during the simulation, corresponding to the operating conditions of virtual inertia parameter J=20 and virtual damping parameter D=100. The sampling interval covers the period before the frequency disturbance, during the frequency drop, and the dynamic power adjustment stage. From the frequency data, the system frequency remains at 50Hz from 0 seconds to 1.75 seconds, indicating that the system is in rated steady-state operation before the disturbance. At 2.25 seconds, the frequency drops to 49.95Hz, at 2.5 seconds it further drops to 49.90Hz, at 3.75 seconds it continues to drop to 49.85Hz, reaching the lowest value of 49.80Hz during the disturbance period at 4.25 seconds, and slightly recovering to 49.75Hz at 4.5 seconds. Figure 6 Compared to the rapid frequency response after a 2-second disturbance and the return to 50Hz after a 3-second recovery under the J=4 condition, the frequency response at J=20 exhibits a significant lag, with a longer frequency drop duration and failure to recover to the rated value for a considerable period after the disturbance.

[0184] From the active power data, the output active power remained stable at the initial operating point of 0.08MW during the period from 0 seconds to 1.75 seconds. At 2.25 seconds, the power increased to 0.13MW; at 2.5 seconds, it slightly decreased to 0.12MW; at 3.75 seconds, it decreased to 0.11MW; at 4.25 seconds, it decreased to 0.10MW; and at 4.5 seconds, it decreased to 0.09MW. Figure 7Compared to the rapid and smooth power response under J=4 operating condition, which recovers to 0.08MW after 3 seconds, the power regulation process under J=20 exhibits obvious oscillating decay characteristics. After reaching the peak, the power does not return to a stable level, but instead experiences a continuous fluctuating decline, and even at 4.5 seconds, it still deviates from the initial operating point by 0.01MW.

[0185] In summary, after increasing the virtual inertia to J=20, although the grid-type converter still has frequency support capability and can generate more active power to support the grid during frequency drops, the dynamic response performance of the system deteriorates significantly, further verifying that the value of the virtual inertia parameter has a decisive influence on the dynamic stability of the grid-type converter.

[0186] Figure 9 The simulation results show the specific values ​​of frequency and active power of the grid-type converter at different times during the simulation process, corresponding to the operating conditions of virtual inertia parameter J=4 and virtual damping parameter D=50. The sampling interval covers the period before frequency disturbance, during frequency drop, and during dynamic power adjustment. From the frequency data, the frequency is 50Hz at 1.75 seconds, indicating that the system is initially in rated steady-state operation. The frequency drops to 49.95Hz at 2.25 seconds and further to 49.85Hz at 2.75 seconds, indicating a continuous decline in grid frequency. The frequency rises to 49.90Hz at 3.25 seconds, recovers to 50Hz at 3.75 seconds, drops to 49.95Hz again at 4.25 seconds, and drops to 49.80Hz at 4.75 seconds. Figure 6 Compared to the performance of the system under the conditions of J=4 and D=100, where the frequency quickly recovers to 50Hz and remains stable, the frequency drops again after recovering to the rated value when D=50, showing obvious oscillation attenuation characteristics. This indicates that the system's ability to suppress frequency disturbances decreases after the damping is reduced, and the dynamic response process is prolonged.

[0187] From the active power data, the output active power remained at 0.08MW throughout the entire time period from 1.75 seconds to 4.75 seconds, without significant adjustment with frequency changes. Figure 6 Compared to the dynamic adjustment process where the active power rises from 0.08MW to approximately 0.12MW during the mid-frequency drop and then falls back to 0.08MW after the frequency recovers, the active power response of the grid-type converter at D=50 is significantly insufficient. It fails to effectively increase power generation to support the grid frequency, resulting in the system frequency remaining in a fluctuating state for a long time after the disturbance. This further confirms the negative impact of insufficient damping on the system's frequency support capability.

[0188] When the virtual damping coefficient decreases from D=100 to D=50, although the grid-type converter can still maintain basic operation, the dynamic performance of the system deteriorates significantly. This further verifies the key impact of the virtual damping parameter on the dynamic stability and frequency support capability of the grid-type converter. Accurately identifying the virtual damping parameter is of great significance for ensuring the transient performance and damping characteristics of the system.

[0189] Figure 10 This is a schematic diagram of a data-driven online estimation system for the inertia parameters of a grid-type converter, as provided in the embodiments of this specification, including:

[0190] The data acquisition module is used to collect three-phase voltage time-series data, three-phase current time-series data and bus frequency time-series data on the output side of the grid-type converter, and to calculate active power time-series data based on the three-phase voltage time-series data and three-phase current time-series data.

[0191] The online inference module is used to input the active power time-series data and the bus frequency time-series data as state information into a trained estimation model. The estimation model is a deep reinforcement learning agent trained in a simulation environment constructed by a series-parallel identification model, which is used to simulate the frequency response delay characteristics of the phase-locked loop of the grid-type converter. The estimation model outputs actions, including estimated values ​​of the virtual inertia parameters and virtual damping parameters of the grid-type converter.

[0192] In the embodiments of this specification, the real-time acquisition of electrical quantities and active power calculation of the grid-type converter outlet side are realized through the collaborative work of the data acquisition module and the online inference module, respectively, as well as the online inference estimation of inertia parameters based on the trained deep reinforcement learning agent. A complete link from data acquisition to parameter estimation is formed between the two modules, and an end-to-end solution for data-driven online estimation of inertia parameters of grid-type converters is constructed. It can be widely applied in the grid-connected operation scenario of grid-type converters in new energy power plants, upgrading the traditional intrusive offline parameter identification scheme to a non-intrusive online accurate estimation scheme based on deep reinforcement learning.

[0193] Figure 11 A structural block diagram of a computing device 1100 according to one embodiment of this specification is shown. The components of the computing device 1100 include, but are not limited to, a memory 1110 and a processor 1120. The processor 1120 is connected to the memory 1110 via a bus 1130, and a database 1150 is used to store data.

[0194] The computing device 1100 also includes an access device 1140, which enables the computing device 1100 to communicate via one or more networks 1160. Examples of these networks include Public Switched Telephone Network (PSTN), Local Area Network (LAN), Wide Area Network (WAN), Personal Area Network (PAN), or combinations of communication networks such as the Internet. The access device 1140 may include one or more of any type of wired or wireless network interface (e.g., a network interface card (NIC)), such as an IEEE 802.11 Wireless Local Area Network (WLAN) wireless interface, a Wi-MAX (Worldwide Interoperability for Microwave Access) interface, an Ethernet interface, a Universal Serial Bus (USB) interface, a cellular network interface, a Bluetooth interface, or a Near Field Communication (NFC) interface.

[0195] In one embodiment of this specification, the aforementioned components of the computing device 1100 and Figure 11 Other components, not shown, can also be connected to each other, for example, via a bus. It should be understood that... Figure 11 The block diagram of the computing device shown is for illustrative purposes only and is not intended to limit the scope of this specification. Those skilled in the art can add or replace other components as needed.

[0196] Computing device 1100 can be any type of stationary or mobile computing device, including mobile computers or mobile computing devices (e.g., tablet computers, personal digital assistants, laptop computers, notebook computers, netbooks, etc.), mobile phones (e.g., smartphones), wearable computing devices (e.g., smartwatches, smart glasses, etc.) or other types of mobile devices, or stationary computing devices such as desktop computers or personal computers (PCs). Computing device 400 can also be a mobile or stationary server.

[0197] The processor 1120 executes computer-executable instructions, which, when executed by the processor, implement the steps of the data-driven online estimation method for the inertia parameters of the network converter described above. The above is an illustrative scheme of a computing device according to an embodiment of this specification. It should be noted that the technical solution of this computing device and the technical solution of the data-driven online estimation method for the inertia parameters of the network converter described above belong to the same concept. Details not described in detail in the technical solution of the computing device can be found in the description of the technical solution of the data-driven online estimation method for the inertia parameters of the network converter described above.

[0198] An embodiment of this specification also provides a computer-readable storage medium storing computer-executable instructions that, when executed by a processor, implement the steps of the data-driven online estimation method for the inertia parameters of the grid-type converter described above.

[0199] The above is an illustrative scheme of a computer-readable storage medium according to an embodiment of this specification. It should be noted that the technical solution of this storage medium belongs to the same concept as the technical solution of the data-driven online estimation method for the inertia parameters of the grid-type converter described above. Details not described in detail in the technical solution of the storage medium can be found in the description of the technical solution of the control method for reducing reactive power exchange in AC / DC systems in the above-described DC engineering.

[0200] An embodiment of this specification also provides a computer program, wherein when the computer program is executed in a computer, it causes the computer to perform the steps of the data-driven online estimation method for the inertia parameters of the grid-type converter described above.

[0201] The above is an illustrative example of a computer program according to an embodiment of this specification. It should be noted that the technical solution of this computer program belongs to the same concept as the technical solution of the data-driven online estimation method for the inertia parameters of the grid-type converter described above. Details not described in detail in the computer program's technical solution can be found in the description of the technical solution of the data-driven online estimation method for the inertia parameters of the grid-type converter described above.

[0202] The foregoing has described specific embodiments of this specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims may be performed in a different order than that shown in the embodiments and may still achieve the desired result. Furthermore, the processes depicted in the drawings do not necessarily require the specific or sequential order shown to achieve the desired result. In some embodiments, multitasking and parallel processing are possible or may be advantageous.

[0203] The computer instructions include computer program code, which may be in the form of source code, object code, executable file, or certain intermediate forms. The computer-readable medium may include any entity or device capable of carrying the computer program code, recording media, USB flash drives, portable hard drives, magnetic disks, optical disks, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc. It should be noted that the content included in the computer-readable medium may be appropriately added to or subtracted according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable media may not include electrical carrier signals and telecommunication signals.

[0204] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that the embodiments in this specification are not limited to the described order of actions, because according to the embodiments in this specification, some steps can be performed in other orders or simultaneously. Furthermore, those skilled in the art should also understand that the embodiments described in this specification are all preferred embodiments, and the actions and modules involved are not necessarily essential to the embodiments in this specification.

[0205] In the above embodiments, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0206] The preferred embodiments disclosed above are merely illustrative of this specification. Optional embodiments do not exhaustively describe all details, nor do they limit the invention to the specific implementations described. Clearly, many modifications and variations can be made based on the embodiments described herein. These embodiments are selected and specifically described in this specification to better explain the principles and practical applications of the embodiments, thereby enabling those skilled in the art to better understand and utilize this specification. This specification is limited only by the claims and their full scope and equivalents.

Claims

1. A data-driven online estimation method for the inertia parameters of a grid-type converter, characterized in that, include: Collect three-phase voltage time-series data, three-phase current time-series data, and bus frequency time-series data from the output side of the grid-type converter, and calculate active power time-series data based on the three-phase voltage time-series data and three-phase current time-series data. The active power time-series data and the bus frequency time-series data are used as state information inputs to the trained estimation model. The estimation model is a deep reinforcement learning agent, which is trained in a simulation environment. The simulation environment is constructed by a series-parallel identification model, which is used to simulate the frequency response delay characteristics of the phase-locked loop of the grid-type converter. The estimation model outputs an action, which includes the estimated values ​​of the virtual inertia parameters and virtual damping parameters of the grid-type converter.

2. The method according to claim 1, characterized in that, The estimation model includes an input layer, a shared feature extraction layer, an Actor network, and a Critic network; The input layer is used to receive the active power timing data and the bus frequency timing data; The shared feature extraction layer includes multiple fully connected layers for nonlinear feature extraction from the input data; The Actor network is used to receive the feature tensor output by the shared feature extraction layer. The output layer adopts a Gaussian strategy to output the mean and variance of the action. The action is sampled from the Gaussian distribution defined by the mean and variance. The action includes the estimated values ​​of the virtual inertia parameter and the virtual damping parameter of the grid-type converter. The Critic network receives the feature tensor output by the shared feature extraction layer. After passing through a fully connected layer, the output layer uses a linear activation function to output the current state value.

3. The method according to claim 1, characterized in that, The estimation model is trained in the following manner: The simulation environment is constructed based on a serial-parallel identification model; In the series-parallel identification model, the active power timing of the grid connection point of the grid-connected converter is used as input, and the power is synchronously distributed to the actual grid-connected converter branch and the neural network identification model branch. The agent outputs an action based on the current state, and the action includes an estimated value of the inertia parameter. The neural network identification model branch receives the action and the active power timing sequence, and outputs the model fitting frequency; The measured bus frequency of the branch output of the real grid-type converter; Calculate the timing error between the measured bus frequency and the model-fitted frequency; The agent is trained using the negative value of the time-series error as the reward function and a near-end policy optimization algorithm. During training, the agent outputs an action based on its current state, the simulation environment calculates the timing error based on the action and returns a reward function value, and the proximal policy optimization algorithm updates the agent's parameters based on the reward function value until the reward function converges.

4. The method according to claim 3, characterized in that, The real grid-type converter branch incorporates a virtual inertial dynamic function that takes time into account. The virtual inertial dynamic function incorporates the rotor motion equation, the first-order delay element of the phase-locked loop, and the PWM modulation attenuation characteristics. The measured bus frequency is determined by the following formula: ； in, for The measured bus frequency of the actual grid-type converter branch output at the specified time. for The active power timing data of the actual grid-type converter branch is input at all times. The virtual inertia coefficient of the grid-type converter is given. This refers to the virtual damping coefficient of the grid-type converter. The total equivalent delay constant for phase-locked loop frequency measurement delay, signal sampling calculation delay, and PWM modulation delay.

5. The method according to claim 3, characterized in that, The neural network identification model branch receives the active power time series and the estimated inertia parameters output by the agent, and outputs the model fitting frequency after nonlinear mapping through the shared feature extraction layer: ； in, for The active power time-series data of the branches of the neural network identification model are input at any time. for The estimated value of the inertia parameter output by the agent at time t. These are the network parameters of the intelligent agent.

6. The method according to claim 1, characterized in that, The calculation of active power time-series data based on the three-phase voltage time-series data and the three-phase current time-series data includes: The three-phase voltage time series data are transformed by coordinates to obtain the dq-axis voltage component, and the three-phase current time series data are transformed by coordinates to obtain the dq-axis current component. Active power time-series data are obtained based on the dq-axis voltage component and the dq-axis current component.

7. The method according to claim 1, characterized in that, Also includes: Each grid-type converter is configured with an agent, and each agent shares a global reward signal, wherein the global reward signal is the negative value of the system frequency deviation; A multi-agent proximal policy optimization algorithm is used to jointly train the agents, thereby achieving collaborative estimation of the inertia parameters of each of the network converters.

8. A data-driven online estimation system for the inertia parameters of a grid-type converter, characterized in that, include: The data acquisition module is used to collect three-phase voltage time-series data, three-phase current time-series data and bus frequency time-series data on the output side of the grid-type converter, and to calculate active power time-series data based on the three-phase voltage time-series data and three-phase current time-series data. The online inference module is used to input the active power time-series data and the bus frequency time-series data as state information into the trained estimation model. The estimation model is a deep reinforcement learning agent, which is trained in a simulation environment. The simulation environment is constructed by a series-parallel identification model, which is used to simulate the frequency response delay characteristics of the phase-locked loop of the grid-type converter. The estimation model outputs an action, which includes the estimated values ​​of the virtual inertia parameters and virtual damping parameters of the grid-type converter.

9. A computing device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the data-driven online estimation method for the inertia parameters of the grid-type converter as described in any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by a processor, the program implements the data-driven online estimation method for the inertia parameters of the grid-type converter as described in any one of claims 1 to 7.

Images