Virtual synchronous generator parameter optimization control method and system based on DDPG algorithm

Abstract

The invention discloses a virtual synchronous generator parameter optimization control method and system based on a DDPG algorithm, and the method comprises the following steps: constructing a high-fidelity power electronic simulation environment, setting a core variable, constructing a DDPG intelligent agent of an Actor-Critic dual-network architecture, collecting a current state variable in real time, and outputting an action variable based on an Actor network. After superposing exploration noise, acting on a virtual synchronous generator (VSG), updating parameters of the Actor network and the Critic network until training convergence, and obtaining solidified Actor network parameters; acquiring real-time operation data of a virtual synchronous generator (VSG), preprocessing the real-time operation data, inputting the preprocessed real-time operation data into the solidified Actor network to obtain optimal network parameters, and updating the optimal network parameters to the VSG in real time to update and reset the network parameters; according to the invention, real-time and adaptive adjustment of the core control parameters of the virtual synchronous generator is realized.

Description

Technical Field

[0001] This invention relates to the field of virtual synchronous generator parameter control technology, specifically to a virtual synchronous generator parameter optimization control method and system based on the DDPG algorithm. Background Technology

[0002] With the increasing penetration of intermittent and fluctuating renewable energy sources such as wind and solar power in microgrids, the generation-side structure of power systems is undergoing fundamental changes. Traditional synchronous generators are gradually being replaced by renewable energy generation units based on power electronic converters. However, most power electronic converters adopt grid-following control strategies based on phase-locked loops (PLLs), and their output power is determined by upstream energy sources (such as solar power and wind speed), lacking autonomous response capabilities to system frequency changes. This means that the converter inherently lacks the rotational inertia and damping characteristics provided by the rotating mass of a synchronous generator, resulting in a sharp decline in the equivalent inertia and damping level of microgrids, especially those operating in islanded mode. When the system encounters disturbances such as sudden load changes or severe fluctuations in wind and solar power output, the system's frequency stability domain decreases, and the rate of frequency change increases, easily triggering severe frequency oscillations or even instability, seriously threatening the safe and reliable operation of the microgrid.

[0003] To address these challenges, Virtual Synchronous Generator (VSG) technology has emerged and is considered by academia and industry as a key technological path to improve the grid-friendly integration capability of renewable energy sources. The core idea of ​​VSG technology is to simulate the intrinsic mathematical model and external operating characteristics of a synchronous generator in a converter through control algorithms. Its control typically embeds the rotor motion equations of the synchronous generator, where… For virtual rotational inertia, This represents the virtual damping coefficient. Through this equation, the VSG can be adjusted based on the grid frequency deviation. The active power output is dynamically adjusted to provide the system with inertial response and damping support similar to that of a synchronous generator. The core principle of virtual synchronous generator (VSG) technology is to reproduce the mathematical model of a synchronous generator in a power electronic inverter through control algorithms. This mainly includes two parts: rotor motion equations and electromagnetic transient equations. The rotor motion equations describe the dynamic relationship between the unit speed (frequency) and mechanical and electromagnetic torques, and are key to determining the VSG's inertia and damping characteristics. The electromagnetic transient equations, by simulating the armature reaction and excitation regulation characteristics of a synchronous generator, achieve control over the amplitude and phase of the output voltage.

[0004] In existing technologies, VSG control parameters are mostly set to fixed values. For example, in engineering practice, a set of constant parameters is often pre-tuned based on the system's rated capacity and typical disturbance scenarios. and The value remains constant throughout the entire operation. This type of method is simple in structure, easy to implement, and can achieve certain results under specific operating conditions. However, its essential drawback lies in the lack of perception and response capability to changes in operating state: when the system encounters a large power surge (such as a sudden increase in load or a sudden drop in photovoltaic output), the fixed inertia is often insufficient to suppress the rapid frequency drop; while in the small disturbance or steady-state operation phase, excessive inertia will delay the adjustment process, reduce the dynamic performance of the system, and even cause power oscillations between multiple machines due to response lag. Summary of the Invention

[0005] The purpose of this invention is to propose a virtual synchronous generator parameter optimization control method based on the DDPG algorithm to solve the problems mentioned in the background art.

[0006] To achieve the above-mentioned objectives, the first technical solution adopted by this invention is: a virtual synchronous generator parameter optimization control method based on the DDPG algorithm, comprising the following steps: S1. Construct a high-fidelity power electronics simulation environment, which includes a virtual synchronous generator main circuit model, a local load module, a renewable energy interface unit, and a power grid equivalent model. S2, set a 6-dimensional state vector and a 2-dimensional action variable as core variables, and base the core variables; S3 constructs the DDPG agent with an Actor-Critic dual-network architecture, initializes the Actor network, Critic network and corresponding target network parameters, and configures the experience replay pool, batch sampling size, network learning rate and exploration noise intensity hyperparameter. S4 sets the training rounds. Each round simulates the dynamic running process of a preset duration, collects the current state variables in real time, outputs action variables based on the Actor network, and applies exploration noise to the virtual synchronous generator VSG to update the parameters of the Actor network and Critic network until the training converges and the solidified Actor network parameters are obtained. S5 collects real-time operating data of the virtual synchronous generator (VSG), preprocesses it, inputs the preprocessed real-time operating data into the solidified Actor network to obtain the optimal network parameters, and updates the VSG in real time to update and reset the network parameters.

[0007] Furthermore, in step S3, an Actor-Critic dual-network architecture is constructed. The Actor network is responsible for transforming state variables into action variables, while the Critic network evaluates the value of the action and initializes the core hyperparameters, which are used to store the experience replay pool for training experience, the batch sampling size for each update, the network learning rate, and the exploration noise intensity.

[0008] Furthermore, in step S3, the Actor network adopts a fully connected layer structure with 64-32 neurons, maps the state vector to the optimal control action, and outputs the matching action constraint after activation function and scaling; the Critic network fuses state and action features to quantify the state-action value of the output action.

[0009] Furthermore, the experience collection and strategy iteration are used to set the training rounds, and each round of simulated dynamic operation includes: Real-time acquisition of current state variables The Actor network outputs action variables. ; The action variables are superimposed with exploration noise and applied to the VSG, with immediate rewards for environmental feedback. With the next state Generate experience groups and combine experience groups Store in the playback pool; After accumulating set experience values, update the network in batches and optimize variables by combining data analysis. If the number of times the action variable exceeds the limit is greater than or equal to the set number, then reduce the exploration noise; If the reward is lower than the set experience value, the weight of the core indicators in the reward function will be adjusted to guide the agent to converge toward the optimal strategy.

[0010] Furthermore, the training method for the virtual synchronous generator is as follows: Virtual synchronous generator outputs actions to VSG system The environment will provide feedback on the status after execution. ,award With the next state Received experience group; Store experience groups in an experience pool, randomly sample experience groups from the experience pool, and output the target action for the next state based on the Target-actor. Calculate the corresponding target Q-value based on the target-critic, and then determine the target Q-value and reward. The target value is obtained; The Critic network is optimized using the error between the current Q value and the target value as the loss, and the Actor network is optimized with the goal of maximizing the Q value. The parameters of the target network are adjusted based on the soft update formula until the actions output by the Actor network can stably optimize the performance of VSG, at which point the training is complete.

[0011] Furthermore, it also includes offline data backtracking, operational status assessment, and parameter reset; The offline data backtracking configuration is as follows: Every day, during the off-peak period of the power grid load, the previous day's operating data is analyzed and high-value experiences are selected to be added to the experience playback pool. The network parameters are updated using a small-batch incremental learning mode. The runtime status assessment and parameter reset configuration are as follows: A comprehensive evaluation of VSG's operating metrics is conducted monthly. If a metric falls below the threshold for two consecutive weeks, the model validation process is initiated. Based on the validation results, model replacement or lightweight offline training is performed to generate a control strategy adapted to the new operating conditions.

[0012] To achieve the above-mentioned objectives, the second technical solution adopted by the present invention is: a virtual synchronous generator parameter optimization control system based on the DDPG algorithm, used in the virtual synchronous generator parameter optimization control method based on the DDPG algorithm, including a simulation training module, a DDPG agent module, a data acquisition module, a parameter adjustment module, and a state evaluation module; The simulation training module is used to construct a high-fidelity power electronics simulation environment, define the core training variables, and perform offline training and strategy solidification for the DDPG agent. The DDPG agent module adopts an Actor-Critic dual-network architecture for iterative and soft updates of network parameters. The data acquisition module includes sensors and a communication module for real-time acquisition of operational data; The parameter adjustment module is used for adaptive adjustment of the parameters of the DDPG agent; The status assessment module is used to backtrack and analyze the running data daily and complete model fine-tuning, comprehensively evaluate the VSG running indicators monthly, and perform model replacement or lightweight offline training based on the assessment results.

[0013] Furthermore, it also includes a droop control module and a power calculation module; the power calculation module collects the voltage and current signals of the AC bus, calculates the actual active and reactive power, feeds it back to the droop control module and transmits it to the DDPG intelligent agent module as a status input; The droop control module includes a P droop control unit and a Q droop control unit, which respectively realize the coordinated control of active power and frequency, and reactive power and voltage.

[0014] Furthermore, the high-fidelity power electronics simulation environment also includes an LC filter circuit and a converter model. The converter model is used to simulate the conversion of DC power to AC power, and the LC filter circuit is used to reduce high-frequency harmonic interference at the converter output.

[0015] Furthermore, it also includes a phase-locked loop (PLL) module and a voltage and current dual closed-loop control module; The phase-locked loop (PLL) module collects the AC bus voltage and obtains the angular velocity of the power grid, providing a synchronization reference for the DDPG intelligent agent module and the droop control module. The voltage and current dual closed-loop control module receives the voltage reference signal output by the VSG, and generates a PWM signal by combining it with the virtual impedance setting to control the switching state of the converter.

[0016] Due to the application of the above technical solution, the present invention has the following advantages compared with the prior art: 1. Breaking through the limitations of traditional VSG fixed parameter control, it realizes real-time dynamic adjustment of key parameters such as virtual inertia and damping coefficient, enabling VSG to autonomously decide the optimal control strength based on the current grid conditions (such as the magnitude of frequency deviation, the rate of change, and the degree of power imbalance), thereby providing just the right frequency support under various disturbance conditions.

[0017] 2. Enhance the robustness and generalization ability of VSG in uncertain environments. Through a data-driven reinforcement learning mechanism, the controller can make effective decisions based solely on locally observable signals without needing to know the grid impedance, line parameters, or load characteristics. This adapts to various operating modes such as islanding, grid connection, and multi-machine collaboration, reducing the dependence on system modeling accuracy and communication reliability.

[0018] 3. Balancing frequency stability, dynamic response performance, and control smoothness. By rationally designing learning objectives, the agent is guided to reduce frequency deviation and accelerate recovery speed while avoiding frequent and drastic changes in control commands, thus extending the lifespan of power electronic devices and meeting practical engineering needs.

[0019] 4. By introducing a data-driven intelligent decision-making mechanism, this invention enables real-time and adaptive adjustment of the core control parameters of the virtual synchronous generator, significantly improving the proactive support capability and operational resilience in scenarios with a high proportion of renewable energy access. Attached Figure Description

[0020] Figure 1 The diagram shows the virtual synchronous generator control structure of the DDPG algorithm in the virtual synchronous generator parameter optimization control method based on the DDPG algorithm provided in this embodiment of the invention. Figure 2 The diagram shows the training architecture of the virtual synchronous generator agent based on the DDPG algorithm for the parameter optimization control system of the virtual synchronous generator based on the DDPG algorithm provided in the embodiment of the present invention. Figure 3 This diagram illustrates the grid connection of a virtual synchronous generator (VSG) to an AC bus using a virtual synchronous generator parameter optimization control system based on the DDPG algorithm provided in an embodiment of the present invention. Figure 4 The diagram shows the basic structure of the deep reinforcement learning algorithm for the parameter optimization control system of a virtual synchronous generator based on the DDPG algorithm provided in this embodiment of the invention. Detailed Implementation

[0021] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort should fall within the scope of protection of the present application.

[0022] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate for the embodiments of this application described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or system that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or systems.

[0023] It should be noted that, unless otherwise specified, the embodiments and features described in this application can be combined with each other. This application will now be described in detail with reference to the accompanying drawings and embodiments.

[0024] like Figure 1 As shown, this embodiment of the invention provides a virtual synchronous generator parameter optimization control method based on the DDPG algorithm, including the following steps: S1. Construct a high-fidelity power electronics simulation environment, which includes a virtual synchronous generator main circuit model, a local load module, a renewable energy interface unit, and a power grid equivalent model. S2, set a 6-dimensional state vector and a 2-dimensional action variable as core variables, and base the core variables; S3 constructs the DDPG agent with an Actor-Critic dual-network architecture, initializes the Actor network, Critic network and corresponding target network parameters, and configures the experience replay pool, batch sampling size, network learning rate and exploration noise intensity hyperparameter. S4 sets the training rounds. Each round simulates the dynamic running process of a preset duration, collects the current state variables in real time, outputs action variables based on the Actor network, and applies exploration noise to the virtual synchronous generator VSG to update the parameters of the Actor network and Critic network until the training converges and the solidified Actor network parameters are obtained. S5 collects real-time operating data of the virtual synchronous generator (VSG), preprocesses it, inputs the preprocessed real-time operating data into the solidified Actor network to obtain the optimal network parameters, and updates the VSG in real time to update and reset the network parameters.

[0025] It should be noted that as the penetration rate of renewable energy sources such as wind power and photovoltaics in the power system continues to increase, the proportion of traditional synchronous generator sets is declining, resulting in a significant weakening of the overall rotational inertia and damping capacity of the system, posing a severe challenge to frequency stability. Virtual synchronous generator (VSG) technology, by introducing an outer-loop dynamic that simulates the rotor motion characteristics of a synchronous machine into the power electronic converter control, endows the inverter with a certain inertial response and primary frequency regulation capability, and has become an important means to improve the grid connection support performance of new energy sources.

[0026] However, existing VSG control strategies generally employ fixed or segmented virtual inertia and damping coefficients, with parameter tuning relying on linearized models and empirical trial-and-error at specific operating points. Such methods struggle to balance dynamic response speed, steady-state accuracy, and system robustness when facing complex operating conditions such as random load fluctuations, sudden changes in renewable energy output, multi-machine interactive coupling, or weak grid connections. On the one hand, a fixed inertia may be insufficient to suppress rapid frequency drops under large disturbances; on the other hand, excessively large inertia during small disturbances or steady-state operation can delay the adjustment process or even trigger power oscillations. Furthermore, traditional control methods are highly dependent on the accuracy of grid parameters, and in practical engineering, problems such as model mismatch, topology changes, or communication delays often lead to decreased control performance or even instability.

[0027] To address the aforementioned issues, the Deep Deterministic Policy Gradient (DDPG) method is a deep reinforcement learning algorithm for continuous action spaces, combining an Actor-Critic architecture with a deterministic policy gradient method. It outputs deterministic control actions through an Actor network, evaluates state-action value through a Critic network, and improves training stability using experience replay and target network mechanisms. In VSG control, DDPG has significant advantages. First, VSG requires adjusting continuous variables such as inertia, damping, and power reference, making DDPG naturally suited for such continuous control tasks. Second, its model-independent nature eliminates the need for precise system modeling, allowing for online adaptive responses to renewable energy fluctuations and load disturbances. Furthermore, by designing a multi-objective reward function, DDPG can synergistically optimize frequency dynamic response, steady-state accuracy, and voltage support capability, effectively improving the operational stability and robustness of microgrids. The basic structure diagram of the deep reinforcement learning algorithm is shown below. Figure 4 As shown.

[0028] This invention constructs a new intelligent, adaptive, and model-independent VSG control paradigm, which effectively solves the core problems of insufficient inertia and frequency support in high-proportion renewable energy power systems, and provides key technical support for the safe, stable, and efficient operation of new power systems.

[0029] Specifically, this invention comprises two main phases: an offline training phase and an online control phase. In the offline training phase, a high-fidelity power electronics simulation environment is first constructed. This environment includes a complete virtual synchronous generator main circuit model (such as a three-phase inverter and LC filter), a local load module, renewable energy interfaces (such as photovoltaic or wind power simulation units), and an equivalent grid model. Within this environment, a set of engineering-implementable state observation variables are defined as inputs to the agent, including but not limited to: the deviation of the grid frequency from its rated value, the rate of change of frequency (RoCoF), the error between the active power command and the actual output, DC bus voltage fluctuations, and state-of-charge information when energy storage systems are included. All of these state variables are derived from sensors or observables already deployed in existing power electronic controllers, requiring no additional hardware support.

[0030] In the offline training phase, this invention constructs a model of an inverter, filter circuit, local load, and virtual synchronous generator system in a high-fidelity simulation environment. The intelligent controller learns the optimal control strategy by repeatedly interacting with this environment using a deep deterministic policy gradient algorithm. An experience replay mechanism and a target network structure are employed during training to effectively improve learning stability and convergence. The designed reward function comprehensively considers frequency deviation, dynamic smoothness, power point tracking accuracy, and the amplitude of control action changes, guiding the strategy towards optimization that balances steady-state accuracy and dynamic performance. The offline training phase mainly includes the following steps: The first step is defining the environment and core variables. This involves building a simulation environment that includes the VSG itself, new energy units, loads, and grid interfaces to simulate actual operating conditions such as power fluctuations, load switching, and grid disturbances. The core training variables are clearly defined: state variables include grid frequency deviation, frequency change rate, and VSG active power deviation, reflecting the system's operating state; action variables include the VSG's inertia, which needs to be adaptively adjusted. Damping At the same time, evaluation indicators such as frequency stability and response speed are set as references for strategy optimization.

[0031] The second step is network and basic parameter initialization, constructing an Actor-Critic dual-network architecture: the Actor network is responsible for transforming state variables into action variables ( The Critic network evaluates the value of actions (Q-value). Initialize core hyperparameters: the experience replay pool for storing training experience, the batch size for each update, the network learning rate, and the exploration noise intensity, providing the basic conditions for training initiation.

[0032] The third step is experience collection and strategy iteration. The training rounds are set, with each round simulating a dynamic process: real-time collection of current state variables. The Actor network outputs action variables. The noise is then superimposed on the VSG, and environmental feedback provides an immediate reward. With the next state experience group Store the data in the replay pool. After accumulating sufficient experience, update the network in batches and optimize variables based on data analysis: if action variables frequently exceed limits, reduce exploration noise; if rewards are too low, adjust the weights of core indicators in the reward function to guide the agent to converge toward the optimal policy.

[0033] The fourth step is convergence verification and policy solidification. When the average reward stabilizes over multiple rounds and the state variables (such as frequency deviation) meet the preset evaluation criteria under typical perturbations, training is considered converged. At this point, exploration noise is reduced, and extreme condition training is added to fine-tune the output accuracy of action variables. Finally, the Actor network parameters are solidified, that is, the optimal mapping relationship from state variables to action variables is determined, and offline training is completed.

[0034] In the online control phase, the offline-trained and solidified Actor network is deployed to the "parameter adaptation module" of the VSG controller. The core of this is based on real-time runtime data. , Parameters are adjusted at the millisecond level, and the model is dynamically fine-tuned through incremental data analysis to ensure adaptability under complex real-world operating conditions. The main steps of online control are as follows: The first stage involves real-time data acquisition and control decision-making. Through the sensors and communication modules of the VSG controller, 200 sets of operational data are collected per second. After preprocessing, 6-dimensional state features (grid frequency deviation) are extracted. Rate of change of frequency VSG output active power deviation VSG rotor angle deviation Supporting energy storage remaining capacity SOC and renewable energy power volatility After normalization, the input to the Actor network outputs the optimal action with no noise. , The inertia and damping parameters of the VSG are updated in real time through the control interface.

[0035] The second stage involves offline data backtesting and model fine-tuning. Every day in the early morning, during the off-peak hours of the power grid, offline analysis is performed on the previous day's operational data: abnormal operating conditions such as frequency deviations exceeding thresholds and frequent parameter adjustments are screened out, marked as "high-value experience," and added to the experience replay pool. A "small-batch incremental learning" mode is used to update the Actor / Critic network parameters, avoiding drastic adjustments that could lead to abrupt changes in the control strategy. For example, if frequency fluctuations occur for three consecutive days when the penetration rate of new energy exceeds 80%, data analysis can pinpoint the cause. The value adjustment is insufficient. To address this, the proportion of empirical samples under this working condition is increased, and the slope of the ReLU activation function in the hidden layer of the Actor network is fine-tuned to improve adaptability to high-penetration scenarios.

[0036] The third step is the operational status assessment and parameter reset mechanism. Monthly, the VSG's operational indicators are comprehensively evaluated through the data analysis module, including frequency support qualification rate, parameter adjustment smoothness, and renewable energy absorption capacity. If a certain indicator falls below the threshold for two consecutive weeks, the "model verification" process is initiated: the online-collected operational data is compared with the offline-trained operating condition data to determine whether the problem is insufficient model generalization or operating condition drift. If it is insufficient model generalization, the offline-trained backup model is used as a replacement, and incremental training is started; if it is operating condition drift (e.g., the renewable energy access ratio increases to 90%), the training environment is reconstructed based on the new operating condition data, "lightweight offline training" is performed, and control strategies adapted to the new operating conditions are generated to ensure the continued effectiveness of the technical solution.

[0037] According to an embodiment of the present invention, the experience collection and strategy iteration are used to set the number of training rounds, and each round of simulated dynamic operation includes: Real-time acquisition of current state variables The Actor network outputs action variables. ; The action variables are superimposed with exploration noise and applied to the VSG, with immediate rewards for environmental feedback. With the next state Generate experience groups and combine experience groups Store in the playback pool; After accumulating set experience values, update the network in batches and optimize variables by combining data analysis. If the number of times the action variable exceeds the limit is greater than or equal to the set number, then reduce the exploration noise; If the reward is lower than the set experience value, the weight of the core indicators in the reward function will be adjusted to guide the agent to converge toward the optimal strategy.

[0038] According to an embodiment of the present invention, the virtual synchronous generator training method is as follows: Virtual synchronous generator outputs actions to VSG system The environment will provide feedback on the status after execution. ,award With the next state Received experience group; Store experience groups in an experience pool, randomly sample experience groups from the experience pool, and output the target action for the next state based on the Target-actor. Where a′ is the target-actor network in the next state. The target action is generated below, where μ′ represents the parameterization function of the target policy network (i.e., the target actor), and its parameters change slowly through a soft update mechanism.

[0039] μ′ is the target actor network, whose parameters are updated very slowly (through soft updates). Therefore, the output action a′ is more stable. If the current policy μ is used to generate a′, the target value will change drastically with the main network, causing training oscillations or even divergence. The actual action a is generated by the current policy μ, and the corresponding target Q-value is calculated based on the target-critic. The target Q-value and reward are then used to determine the target Q-value. The target value is obtained; The Critic network is optimized using the error between the current Q value and the target value as the loss, and the Actor network is optimized with the goal of maximizing the Q value. Specifically, in the DDPG algorithm, the core process of constructing the target Q-value is as follows: when a transition sample is sampled from the empirical replay buffer... Then, the system first sets the next state. The input is fed into the target policy network (target-actor), which consists of parameters. Define and output a deterministic target action. This action is not for actual control, but rather generated specifically for value assessment. Subsequently, and They are fed together into the target-critic network to calculate the corresponding target Q value. Instant rewards combined with environmental feedback And with the discount factor γ, the time difference (TD) target value is finally constructed. This objective value serves as a monitoring signal, used to minimize the mean squared error loss of the current Critic network, thereby driving its parameter updates. The reason for using a target network is... This is because the latter is frequently updated during training, and if it is directly used to generate target actions, it will cause the TD target to oscillate violently; while parameters By using a soft update mechanism to change slowly, it can provide highly stable motion estimates, significantly improving the algorithm's convergence and robustness, enabling DDPG to work effectively in continuous control tasks (such as VSG system regulation).

[0040] The parameters of the target network are adjusted based on the soft update formula until the actions output by the Actor network can stably optimize the performance of VSG, at which point the training is complete.

[0041] It should be noted that, as Figure 2 As shown in the diagram, the core of this process is a cyclical flow of environmental feedback, experience storage, and network iterative optimization. The "environment" refers to the microgrid operating scenario containing the VSG, which is the interaction object of the intelligent agent. The action output by the agent corresponds to the inertia that VSG needs to adjust. Damping Control parameters; It is the current state of environmental feedback, which usually includes a six-dimensional feature vector reflecting the system's operating conditions, such as frequency deviation and power deviation. It is a reward given by the environment, used to quantify the optimization effect of the current action on the system. For example, the more stable the frequency, the higher the reward value. This is the execution of actions. The next state after the environment.

[0042] In addition, the "experience pool" is responsible for storing groups of experience generated through interactions. To avoid the influence of sample correlation during training; the Actor network is a policy network that outputs deterministic actions, with the input state... The optimal action will then be output. The Critic network is used to evaluate the value of actions, based on the input state. and actions The output is the state-action value (i.e., Q-value). To ensure training stability, a target policy network (Target-actor) and a target value network (Target-critic) are also set up, and their parameters are updated using soft coefficients. Slowly track main network parameters, discount factor This is used to weigh the immediate rewards against the long-term rewards, and the loss function is based on the error between the target Q value and the current Q value, optimizing the network parameters through gradient descent.

[0043] The specific interaction process includes: first, the agent outputs actions to the VSG system. The environment will provide feedback on its status after execution. ,award Next state This set of experiences is then stored in the experience pool; subsequently, experience groups are randomly sampled from the experience pool, and the target action for the next state is output through the Target-actor. Then, the target-critic calculates the corresponding target Q value, combined with the reward. The target value is obtained, and the Critic network is optimized with the error between the current Q value and the target value as the loss. At the same time, the Actor network is optimized with the goal of maximizing the Q value. Then, the parameters of the target network are adjusted by a soft update formula to make it slowly track the main network and ensure the stability of training. Finally, the above process is repeated until the action output by the Actor network can stably optimize the running effect of VSG, and the training is completed.

[0044] Specifically, after each gradient update by the Critic and Actor networks, DDPG does not directly copy the main network parameters to the target network. Instead, it uses a soft update strategy to synchronize the target network parameters. The update formula is as follows: in, This represents the parameters of the current Actor or Critic network (i.e. or ), The parameters of its corresponding target network (i.e. or ),and It is a very small positive number (usually set to 0.001). This operation is performed simultaneously on the target Actor and the target Critic, ensuring that both slowly track the changes in the main network with tiny steps. Soft updates achieve gradual synchronization through weighted averaging, greatly improving the robustness and convergence speed of the training process. The entire training loop continues, including interaction with the VSG (Virtual Synchronous Generator) environment, experience storage, network updates, and soft synchronization with the target network, until the control actions output by the Actor network (such as reference voltage, power commands, etc.) can keep the key operating indicators of the VSG system (such as frequency deviation, voltage fluctuation, power oscillation, etc.) within the allowable range, and the performance tends to stabilize. At this point, it is determined that the agent has learned an effective control strategy, and training terminates.

[0045] The soft update formula includes: or ,in Used to update the target policy network (target actor). Used to update the target value network (target critic); τ is the soft update coefficient. In DDPG, the soft update coefficient τ is used to control the degree to which the target network parameters are slowly updated to the current network parameters. By using a weighted average, the changes in the target network are made smoother and more stable, thereby improving the convergence and robustness of training.

[0046] A set of real-time measurable system state variables, such as grid frequency deviation, frequency change rate, and active power tracking error, serve as inputs to the agent. These state variables, after normalization, are fed into a pre-trained policy network. The policy network outputs continuous control commands to adjust the virtual inertia, virtual damping coefficient, and optional additional damping power signal of the virtual synchronous generator in real time. These parameters directly affect the dynamic equations or power reference values ​​of the virtual synchronous generator, thereby altering its response characteristics to external disturbances.

[0047] The control architecture of this invention is compatible with existing virtual synchronous generator inner-loop structures, adding only an intelligent parameter adjustment module at the upper layer, thus possessing good engineering feasibility and promotional value. It is applicable to various typical application scenarios such as isolated microgrids, multi-generator parallel operation, and weak grid access, effectively addressing the problems of system inertia reduction and insufficient frequency support capability caused by high-proportion renewable energy access. Furthermore, since this invention is entirely data-driven and does not rely on prior knowledge such as grid impedance, line parameters, or load models, it maintains good performance even under uncertain environments such as system topology changes, equipment parameter drift, or communication delays, demonstrating excellent robustness and generalization ability. This invention effectively compensates for the lack of rotational inertia in new energy generation, enhances the frequency and voltage support of power electronic interface power supplies for the system, and provides key technical support for building a safe, stable, and intelligent new power system.

[0048] The agent training method is as follows: DDPG-based VSG agent design to adapt to inertia Damping Based on the adjustment requirements of continuous control parameters, a closed-loop architecture of "state perception - strategy decision-making - reward feedback - experience learning" is constructed. The core input parameter for the design of the Virtual Synchronous Generator (DDPG) agent is a 6-dimensional feature vector reflecting the system's operating state, specifically including grid frequency deviation. Rate of change of frequency VSG output active power deviation VSG rotor angle deviation Supporting energy storage remaining capacity SOC and renewable energy power volatility These parameters comprehensively cover the core indicators of frequency stability; the core output parameter is the 2D continuous control parameter that VSG needs to adaptively adjust, namely inertia. With damping Both directly influence the VSG control module to achieve real-time parameter optimization, ensuring system frequency stability and dynamic response performance. The network structure employs an Actor-Critic dual-network design. The Actor network maps the state vector to the optimal control action through fully connected layers of 64-32 neurons, outputting a matched action constraint after activation and scaling. The Critic network, on the other hand, fuses state and action features, quantifying the state-action value (Q-value) of the output action to provide a basis for policy optimization. The reward function uses a multi-objective weighted design to enhance frequency stability and power stability, combining an indicator function and a penalty coefficient. To avoid exceeding the frequency limit, the learning direction of the agent is guided by the dynamic balancing of the weight coefficients of different input features.

[0049] The training of the intelligent agent uses a high-proportion renewable energy microgrid simulation model as the interactive environment, following the core process of "initialization-acquisition-playback-update-convergence". First, a simulation environment is built, including VSG ontology, renewable energy fluctuation, and load switching modules. The parameters of the Actor / Critic main network and target network are initialized, and a capacity of 10 is configured. 5 -10 6 The training process includes an experience replay pool and hyperparameters such as learning rate and discount factor. Each training round simulates a 10-second dynamic process. After the agent collects its real-time state, it outputs actions and adds noise through the Actor network to drive the VSG to update parameters and obtain immediate rewards and the next state from environmental feedback. The samples are stored in the replay pool. Once the sample size reaches a threshold, 64 random samples are selected to update the network in batches: the Critic network updates its value assessment capability with the goal of minimizing the mean squared error, while the Actor network optimizes its output by maximizing the Q-value gradient. The target network uses a soft update mechanism to ensure training stability, while linearly decaying with each round to explore noise variance. In the online phase, only the Actor network is deployed to output the noise-free optimal action in real time. Incremental learning can be initiated to iterate the strategy when encountering new conditions, forming a complete connection between "offline training - online deployment - dynamic optimization".

[0050] According to embodiments of the present invention, it also includes offline data backtracking, operational status evaluation, and parameter reset; The offline data backtracking configuration is as follows: Every day, during the off-peak period of the power grid load, the previous day's operating data is analyzed and high-value experiences are selected to be added to the experience playback pool. The network parameters are updated using a small-batch incremental learning mode. The runtime status assessment and parameter reset configuration are as follows: A comprehensive evaluation of VSG's operating metrics is conducted monthly. If a metric falls below the threshold for two consecutive weeks, the model validation process is initiated. Based on the validation results, model replacement or lightweight offline training is performed to generate a control strategy adapted to the new operating conditions.

[0051] To achieve the above-mentioned objectives, the second technical solution adopted by the present invention is: a virtual synchronous generator parameter optimization control system based on the DDPG algorithm, used in the virtual synchronous generator parameter optimization control method based on the DDPG algorithm, including a simulation training module, a DDPG agent module, a data acquisition module, a parameter adjustment module, and a state evaluation module;

[0052] The simulation training module is used to build a high-fidelity power electronics simulation environment, define the core training variables, and perform offline training and policy solidification for the DDPG agent. The DDPG agent module adopts an Actor-Critic dual-network architecture for iterative and soft updates of network parameters. The data acquisition module includes sensors and a communication module, used to collect operational data in real time; The parameter adjustment module is used for adaptive adjustment of the parameters of the DDPG agent; The status assessment module is used to backtrack and analyze running data daily and complete model fine-tuning, comprehensively evaluate VSG running indicators monthly, and perform model replacement or lightweight offline training based on the assessment results.

[0053] According to an embodiment of the present invention, it further includes a droop control module and a power calculation module; the power calculation module collects the voltage and current signals of the AC bus, calculates the actual active and reactive power, feeds it back to the droop control module and transmits it to the DDPG intelligent agent module as a status input; The droop control module includes a P droop control unit and a Q droop control unit, which respectively realize the coordinated control of active power and frequency, and reactive power and voltage.

[0054] According to an embodiment of the present invention, the high-fidelity power electronics simulation environment also includes an LC filter circuit and a converter model. The converter model is used to simulate the conversion of DC power to AC power, and the LC filter circuit is used to reduce high-frequency harmonic interference at the converter output.

[0055] According to an embodiment of the present invention, it further includes a phase-locked loop (PLL) module and a voltage and current dual closed-loop control module; The phase-locked loop (PLL) module collects the AC bus voltage and obtains the angular velocity of the power grid, providing a synchronization reference for the DDPG intelligent agent module and the droop control module. The voltage and current dual closed-loop control module receives the voltage reference signal output by the VSG, and generates a PWM signal by combining it with the virtual impedance setting to control the switching state of the converter.

[0056] It should be noted that, as Figure 3As shown in the figure, this is a virtual synchronous generator (VSG) grid-connected system for DC new energy access to AC bus. The core is to achieve stable power generation and adaptive control through the coordinated operation of multiple modules (droop control module, DDPG parameter optimization module, VSG module, etc.).

[0057] First, the front-end DC power generation equipment provides DC power, which is converted to AC by a converter. After filtering by inductors and resistors to reduce high-frequency harmonic interference, it is then connected to the AC bus. The switching state of the converter is determined by the PWM signal output by the subsequent control module, thereby adjusting the characteristics of the output AC power.

[0058] The core control part of the system revolves around the VSG model, which simulates the operating characteristics of a traditional synchronous generator: the VSG model receives reference angular velocity, reference active power, and the inertia output by the DDPG agent. and damping These two parameters are the core of the system's adaptive mechanism, adjusted in real time by the DDPG agent based on the grid conditions. The actual angular velocity output by the VSG model is integrated to obtain the virtual rotor angle, which is then combined with the reference voltage amplitude to generate a voltage reference signal, providing a basis for subsequent power regulation.

[0059] The role of the DDPG agent is to dynamically optimize the key parameters of the VSG. It collects the angular velocity, rate of change of angular velocity, and deviation signals between the actual active power and the reference value of the power grid. Based on these power grid operating conditions, it outputs an inertia adapted to the current operating conditions. and damping This will enhance the VSG's ability to support grid frequency fluctuations.

[0060] Meanwhile, the system also includes droop control and power calculation modules: the power calculation module collects voltage and current signals from the AC bus, calculates the actual active and reactive power, and feeds these signals back to the droop control module on the one hand, and transmits them to the DDPG agent as status input on the other hand; the P droop control outputs the mechanical power reference value of the VSG based on the reference angular velocity, reference active power and the actual angular velocity of the grid, to achieve coordination between active power and frequency; the Q droop control adjusts the voltage amplitude reference based on the reference reactive power, reference voltage and actual reactive power to complete the matching of reactive power and voltage.

[0061] Finally, the voltage and current dual closed-loop control module receives the voltage reference signal output by the VSG and, in conjunction with the virtual impedance setting, generates a PWM signal to control the switching of the converter, ensuring that the voltage and current output by the converter can track the reference value. At the same time, the phase-locked loop (PLL) collects the AC bus voltage to obtain the grid angular velocity, providing a synchronization reference for the DDPG intelligent agent and droop control. Combined with the smoothing effect of the filter capacitor, the stable and adaptive grid-connected operation of the DC renewable energy is finally achieved.

[0062] In summary, this application breaks through the limitations of traditional fixed parameter control of VSG, and realizes real-time dynamic adjustment of key parameters such as virtual inertia and damping coefficient. This enables the VSG to autonomously decide the optimal control strength based on the current grid conditions (such as the magnitude of frequency deviation, the rate of change, and the degree of power imbalance), thereby providing just the right frequency support under various disturbance conditions.

[0063] This application enhances the robustness and generalization ability of VSG in uncertain environments. Through a data-driven reinforcement learning mechanism, the controller can make effective decisions based solely on locally observable signals without needing to know the grid impedance, line parameters, or load characteristics. This adapts to various operating modes such as islanded, grid-connected, and multi-machine collaborative operation, reducing the dependence on system modeling accuracy and communication reliability.

[0064] This application balances frequency stability, dynamic response performance, and control smoothness. By rationally designing learning objectives, it guides the agent to reduce frequency deviation and accelerate recovery speed while avoiding frequent and drastic changes in control commands, thus extending the lifespan of power electronic devices and meeting practical engineering needs.

[0065] By introducing a data-driven intelligent decision-making mechanism, this invention enables real-time and adaptive adjustment of the core control parameters of a virtual synchronous generator, significantly improving its proactive support capabilities and operational resilience in scenarios with a high proportion of renewable energy access.

[0066] Those skilled in the art will understand that, for ease of explanation, the example is provided with one memory and one processor. In actual terminals or servers, multiple processors and memories may exist. Memory can also be referred to as storage medium or storage device, etc., and the embodiments of this application do not limit this.

[0067] It should be understood that in the embodiments of this application, the processor may be a Central Processing Unit (CPU), or it may be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The processor may also be a general-purpose microprocessor, graphics processing unit (GPU), or one or more integrated circuits to execute relevant programs to achieve the functions required by the embodiments of this application.

[0068] The processor can also be an integrated circuit chip with signal processing capabilities. In implementation, each step of this application can be completed through integrated logic circuits in the processor hardware or instructions in software form. The aforementioned processor can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this application. The steps of the methods disclosed in the embodiments of this application can be directly manifested as execution by a hardware decoding processor, or execution by a combination of hardware and software modules in the decoding processor. The software modules can reside in random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, or other mature storage media in the art. This storage medium is located in memory; the processor reads information from the memory and, in conjunction with its hardware, completes the functions required by the units included in the methods, systems, and storage media of the embodiments of this application.

[0069] It should also be understood that the memory mentioned in the embodiments of this application can be volatile memory or non-volatile memory, or may include both volatile and non-volatile memory. Non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. Volatile memory can be random access memory (RAM), which is used as an external cache.

[0070] By way of example, but not limitation, many forms of RAM are available, such as Static Random Access Memory (SRAM), Dynamic Random Access Memory (DRAM), Synchronous DRAM (SDRAM), Double Data Rate SDRAM (DDR SDRAM), Enhanced Synchronous DRAM (ESDRAM), Synchlink DRAM (SLDRAM), and Direct Rambus RAM (DR RAM).

[0071] The memory can also be a Compact Disc Read-Only Memory (CD-ROM) or other optical disc storage, optical disk storage (including compressed optical discs, laser discs, optical discs, digital universal optical discs, Blu-ray discs, etc.), magnetic disk storage media, or other magnetic storage devices, or any other medium capable of carrying or storing desired program code in the form of instructions or data structures that can be accessed by a computer, but is not limited thereto. The memory can exist independently and be connected to the processor via a bus. The memory can also be integrated with the processor. The memory can store programs, and when the program stored in the memory is executed by the processor, the processor performs the various steps of the method determined in the above embodiments of this application.

[0072] It should be noted that when the processor is a general-purpose processor, DSP, ASIC, FPGA, or other programmable logic device, discrete gate or transistor logic device, or discrete hardware component, the memory (storage module) is integrated into the processor. It should be noted that the memory described herein is intended to include, but is not limited to, these and any other suitable types of memory.

[0073] It should be understood that the term "and / or" in this article is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. Additionally, the character " / " in this article generally indicates that the preceding and following related objects have an "or" relationship.

[0074] In implementation, each step of the above method can be completed by integrated logic circuits in the processor's hardware or by instructions in software. The steps of the method disclosed in the embodiments of this application can be directly implemented by a hardware processor, or by a combination of hardware and software modules within the processor. The software modules can reside in mature storage media in the art, such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, or registers. Since this storage medium is located in memory, the processor reads information from the memory and, in conjunction with its hardware, completes the steps of the above method; to avoid repetition, these will not be described in detail here.

[0075] Those skilled in the art will recognize that the various illustrative logical blocks (ILBs) and steps described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this application.

[0076] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented, in whole or in part, as a computer-programmed program product. A computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a processor, all or part of the flow or function according to the embodiments of this application is generated. The computer can be a general-purpose computer, a computer network, or other programmable device.

[0077] This embodiment also provides a computer-readable storage medium storing a computer program that enables a computer to execute in order to implement the above-described virtual synchronous generator parameter optimization control method based on the DDPG algorithm.

[0078] It should be noted that computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, computer instructions can be transmitted from one website, computer, server, or data center to another via wired (e.g., coaxial cable, fiber optic) or wireless (e.g., infrared, wireless, microwave, etc.) means, or from one website, computer, server, or data center to a mobile phone processor via a wired means. A computer-readable storage medium can be any usable medium that a computer can access, or a data storage system such as a server or data center that integrates one or more usable media. Usable media can be magnetic media (e.g., floppy disks, hard disks), optical media (e.g., DVDs), or semiconductor media (e.g., solid-state drives), etc.

[0079] Finally, it should be noted that the above are merely preferred embodiments of the present invention and are not intended to limit the present invention. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing embodiments or make equivalent substitutions for some of the technical features. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for optimizing and controlling the parameters of a virtual synchronous generator based on the DDPG algorithm, characterized in that, Includes the following steps: S1. Construct a high-fidelity power electronics simulation environment, which includes a virtual synchronous generator main circuit model, a local load module, a renewable energy interface unit, and a power grid equivalent model. S2, set a 6-dimensional state vector and a 2-dimensional action variable as core variables, and base the core variables; S3 constructs the DDPG agent with an Actor-Critic dual-network architecture, initializes the Actor network, Critic network and corresponding target network parameters, and configures the experience replay pool, batch sampling size, network learning rate and exploration noise intensity hyperparameter. S4 sets the training rounds. Each round simulates the dynamic running process of a preset duration, collects the current state variables in real time, outputs action variables based on the Actor network, and applies exploration noise to the virtual synchronous generator VSG to update the parameters of the Actor network and Critic network until the training converges and the solidified Actor network parameters are obtained. S5 collects real-time operating data of the virtual synchronous generator (VSG), preprocesses it, inputs the preprocessed real-time operating data into the solidified Actor network to obtain the optimal network parameters, and updates the VSG in real time to update and reset the network parameters.

2. The virtual synchronous generator parameter optimization control method based on the DDPG algorithm as described in claim 1, characterized in that, In step S3, an Actor-Critic dual-network architecture is constructed. The Actor network is responsible for transforming state variables into action variables, while the Critic network evaluates the value of actions and initializes core hyperparameters, such as the experience replay pool for storing training experience, the batch sampling size for each update, the network learning rate, and the exploration noise intensity.

3. The virtual synchronous generator parameter optimization control method based on the DDPG algorithm as described in claim 2, characterized in that, In step S3, the Actor network adopts a fully connected layer structure with 64-32 neurons, maps the state vector to the optimal control action, and outputs the matching action constraint after activation function and scaling; the Critic network fuses state and action features and quantifies the state-action value of the output action.

4. The virtual synchronous generator parameter optimization control method based on the DDPG algorithm as described in claim 1, characterized in that, Experience collection and strategy iteration define the training rounds, and each round of simulated dynamic operation includes: Real-time acquisition of current state variables The Actor network outputs action variables. ; The action variables are superimposed with exploration noise and applied to the VSG, with immediate rewards for environmental feedback. With the next state Generate experience groups and combine experience groups Store in the playback pool; After accumulating set experience values, update the network in batches and optimize variables by combining data analysis. If the number of times the action variable exceeds the limit is greater than or equal to the set number, then reduce the exploration noise; If the reward is lower than the set experience value, the weight of the core indicators in the reward function will be adjusted to guide the agent to converge toward the optimal strategy.

5. The virtual synchronous generator parameter optimization control method based on the DDPG algorithm as described in claim 4, characterized in that, The training method for virtual synchronous generators is as follows: Virtual synchronous generator outputs actions to VSG system The environment will provide feedback on the status after execution. ,award With the next state Received experience group; Store experience groups in an experience pool, randomly sample experience groups from the experience pool, and output the target action for the next state based on the Target-actor. Calculate the corresponding target Q-value based on the target-critic, and then determine the target Q-value and reward. The target value is obtained; The Critic network is optimized using the error between the current Q value and the target value as the loss, and the Actor network is optimized with the goal of maximizing the Q value. The parameters of the target network are adjusted based on the soft update formula until the actions output by the Actor network can stably optimize the performance of VSG, at which point the training is complete.

6. The virtual synchronous generator parameter optimization control method based on the DDPG algorithm as described in claim 1, characterized in that, It also includes offline data backtracking, operational status assessment, and parameter reset; The offline data backtracking configuration is as follows: Every day, during the off-peak period of the power grid load, the previous day's operating data is analyzed and high-value experiences are selected to be added to the experience playback pool. The network parameters are updated using a small-batch incremental learning mode. The runtime status assessment and parameter reset configuration are as follows: A comprehensive evaluation of VSG's operating metrics is conducted monthly. If a metric falls below the threshold for two consecutive weeks, the model validation process is initiated. Based on the validation results, model replacement or lightweight offline training is performed to generate a control strategy adapted to the new operating conditions.

7. A virtual synchronous generator parameter optimization control system based on the DDPG algorithm, used in the virtual synchronous generator parameter optimization control method based on the DDPG algorithm according to any one of claims 1-6, comprising a simulation training module, a DDPG agent module, a data acquisition module, a parameter adjustment module, and a state evaluation module; characterized in that, The simulation training module is used to construct a high-fidelity power electronics simulation environment, define the core training variables, and perform offline training and strategy solidification for the DDPG agent. The DDPG agent module adopts an Actor-Critic dual-network architecture for iterative and soft updates of network parameters. The data acquisition module includes sensors and a communication module for real-time acquisition of operational data; The parameter adjustment module is used for adaptive adjustment of the parameters of the DDPG agent; The status assessment module is used to backtrack and analyze the running data daily and complete model fine-tuning, comprehensively evaluate the VSG running indicators monthly, and perform model replacement or lightweight offline training based on the assessment results.

8. The virtual synchronous generator parameter optimization control system based on the DDPG algorithm as described in claim 7, characterized in that, It also includes a droop control module and a power calculation module; the power calculation module collects the voltage and current signals of the AC bus, calculates the actual active and reactive power, feeds it back to the droop control module and transmits it to the DDPG intelligent agent module as a status input; The droop control module includes a P droop control unit and a Q droop control unit, which respectively realize the coordinated control of active power and frequency, and reactive power and voltage.

9. The virtual synchronous generator parameter optimization control system based on the DDPG algorithm as described in claim 8, characterized in that, The high-fidelity power electronics simulation environment also includes an LC filter circuit and a converter model. The converter model is used to simulate the conversion of DC power to AC power, and the LC filter circuit is used to reduce high-frequency harmonic interference at the converter output.

10. The virtual synchronous generator parameter optimization control system based on the DDPG algorithm as described in claim 9, characterized in that, It also includes a phase-locked loop (PLL) module and a voltage and current dual closed-loop control module; The phase-locked loop (PLL) module collects the AC bus voltage and obtains the angular velocity of the power grid, providing a synchronization reference for the DDPG intelligent agent module and the droop control module. The voltage and current dual closed-loop control module receives the voltage reference signal output by the VSG, and generates a PWM signal by combining it with the virtual impedance setting to control the switching state of the converter.

Images