Excitation control method based on dynamic event triggering and deep reinforcement learning

Abstract

The invention discloses an excitation control method based on dynamic event triggering and deep reinforcement learning, and belongs to the technical field of power system automation and control. Comprising the following steps: establishing a synchronous generator and excitation system simulation mathematical model for controller design and training; constructing a deep reinforcement learning training environment, and adopting a deep deterministic strategy gradient algorithm to train a deep reinforcement learning controller offline; designing a dynamic event triggering mechanism; solidifying actor network parameters of the trained deep reinforcement learning controller, deploying the actor network parameters and a dynamic event triggering mechanism to a control unit, and carrying out online simulation testing and performance evaluation. The method has the advantages that the method is safe and reliable, excellent in performance, efficient in resource and high in self-adaption, calculation and communication resource consumption of the controller is remarkably reduced on the premise that system stability and control quality are guaranteed, and the intelligent level and the operation economical efficiency of an excitation system when facing complex disturbance of a modern power grid are improved.

Description

Technical Field

[0001] This invention belongs to the field of power system automation and control technology, specifically relating to an excitation control method based on dynamic event triggering and deep reinforcement learning. More specifically, this invention relates to an innovative excitation controller design method that integrates deep reinforcement learning and advanced event-triggered control. This method aims to improve the adaptive capability, control quality, and operational efficiency of the excitation system in complex power grid environments through offline learning and online intelligent decision-making. Background Technology

[0002] The excitation control system of a synchronous generator is one of the core devices for ensuring the safe and stable operation of a power system. Its main tasks include maintaining the generator terminal voltage at a given level, rationally distributing reactive power among parallel-operating units, and improving the static and dynamic stability of the power system. Traditional excitation control generally adopts a PID (proportional-integral-derivative) regulator based on classical control theory, with the addition of a power system stabilizer (PSS) to provide positive damping and suppress low-frequency power oscillations.

[0003] However, with the continuous development of the power system, especially the grid connection of large-scale renewable energy (such as wind power and photovoltaics), the operating characteristics of the power grid are becoming increasingly complex. The widespread introduction of power electronic devices has made the system exhibit stronger nonlinearity and time-varying characteristics. Traditional linear fixed parameter controllers (such as PID+PSS) are gradually revealing their limitations when facing these new and highly random disturbances: First, their control parameters are usually tuned for specific typical operating conditions. When the system operating point deviates significantly or disturbance patterns not covered by the design conditions occur, the control performance will significantly decrease, and the adaptive capability will be insufficient. Second, PSS is mainly designed for low-frequency oscillations in the 0.1-2.0Hz frequency range. For higher frequency oscillation modes or complex multimodal oscillations, its suppression effect is limited.

[0004] Deep Reinforcement Learning (DRL), an important branch of artificial intelligence, has shown great potential in solving nonlinear system control problems due to its ability to autonomously learn complex decision-making strategies through interaction with the environment. Theoretically, a well-trained DRL controller can approximate complex nonlinear optimal control laws, potentially surpassing the performance of traditional linear controllers. However, directly applying DRL to industrial processes like excitation control, which have extremely high safety and reliability requirements, faces severe challenges: First, during the online learning or training phase, the agent needs to explore the environment through trial and error, and unstable control actions during this process may pose unacceptable risks to the safety of actual power equipment and the power grid; second, DRL controllers typically require high-frequency state sampling and control quantity calculations, which will place a heavy burden on the hardware computing resources of the excitation regulator and the system communication bandwidth.

[0005] Event-triggered control is a resource-aware control method designed to reduce unnecessary control updates and communication. Its basic idea is that control tasks and control signal updates are only triggered when the system state deviates from the equilibrium point by more than a preset threshold; otherwise, the previous control output is maintained. Existing research on applying event-triggered control to excitation systems mostly uses fixed event trigger thresholds. While fixed thresholds are simple to design, they lack adaptability: if the threshold is set too high, it may affect control performance or even stability when the system experiences large disturbances due to untimely updates; if the threshold is set too low, it approaches periodic control and fails to effectively conserve resources. Therefore, a mechanism that can intelligently adjust the trigger threshold based on the system's dynamic behavior is crucial for achieving the optimal balance between ensuring control performance and optimizing resource utilization.

[0006] In summary, the existing technology faces the following pressing problem: how to design a novel excitation control method that possesses both powerful nonlinear processing capabilities and excellent dynamic performance, ensures safety in engineering applications, and intelligently conserves computing and communication resources. To address this, the applicant has developed a beneficial design, and the technical solution described below arose from this context. Summary of the Invention

[0007] The purpose of this invention is to provide an excitation control method based on dynamic event triggering and deep reinforcement learning that is highly secure, highly adaptive, high-performance, and resource-efficient.

[0008] The objective of this invention is achieved by providing an excitation control method based on dynamic event triggering and deep reinforcement learning, comprising the following steps:

[0009] Step 1: Establish a simulation mathematical model of the synchronous generator and excitation system for controller design and training, and make it able to accurately reflect the dynamic characteristics of the system under various disturbances. The state-space equation of the synchronous generator and excitation system is used to describe the dynamic behavior of the system.

[0010] Step 2: Based on the simulation mathematical model established in Step 1, a deep reinforcement learning training environment is constructed, and a deep reinforcement learning controller is trained offline using the deep deterministic policy gradient algorithm. After training, the actor network of the controller can map an approximately optimal excitation control auxiliary signal according to the input system state vector.

[0011] Step 3: Design a dynamic event triggering mechanism. The triggering condition of this mechanism is determined by the error norm between the current state of the system and the state at the last triggering moment, the rate of change norm of the error, and a dynamic positive threshold. This dynamic threshold is related to a physical quantity that reflects the severity of the disturbance to the system and can change in the opposite direction.

[0012] Step four: Solidify the actor network parameters of the deep reinforcement learning controller trained in step two, and deploy them together with the dynamic event triggering mechanism designed in step three to the excitation regulator control unit. The dynamic event triggering mechanism continuously monitors the status of the generator and the power grid system. Only when its triggering condition is met will the deep reinforcement learning controller be called to calculate and update the current control signal output. Otherwise, the control unit will maintain the control signal unchanged at the last triggering moment.

[0013] In a specific embodiment of the present invention, the simulation mathematical model described in step one includes at least the power angle deviation, rotor angular velocity deviation, and q-axis transient potential deviation as its state variables, and its differential equation is described as follows:

[0014]

[0015] in, The derivative of the angle deviation with respect to time is given by units of 1 / 2π. This is the power angle deviation caused by the speed deviating from the synchronous speed, indicating the power angle deviation. The rate of change; Let be the system's reference angular frequency, which is a constant. This is the system's rated frequency; the unit is... ; Rotor angular velocity deviation, in units of , is the actual angular velocity of the rotor. With synchronous angular velocity difference; This is the derivative of the angular velocity deviation with respect to time, in units of . It is the rotor angular velocity deviation. The rate of change; The inertial time constant is expressed in units of 1. , is the magnitude of the rotor's moment of inertia; Electromagnetic power deviation, in units of It is the electromagnetic power output by the generator. Its initial steady-state value The difference; This is the damping coefficient, in units of... This is the natural damping effect experienced by the generator rotor; Let be the derivative of the q-axis transient potential deviation with respect to time, in units of . , is the rate of change of the transient potential deviation; The d-axis open-circuit transient time constant is expressed in units of 1000 m / s. , is the core time constant of the excitation system; This represents the excitation electromotive force deviation, in units of... , is the output of the excitation system; The q-axis synchronous potential deviation is expressed in units of... is an algebraic variable.

[0016] In another specific embodiment of the present invention, the dynamic event triggering mechanism described in step three has the following mathematical triggering condition:

[0017]

[0018] in, It is a norm; Indicates the current time system state vector Compared to the time of the last event trigger state vector The error vector between them is dimensionless; This is the time derivative of the error vector, which is dimensionless. This is a weighting coefficient greater than zero, used to adjust the influence of the rate of change term in triggering the decision, with units of... This determines the importance of the rate of error change in triggering decisions; The threshold for triggering dynamic events is dimensionless and is a positive number that changes over time, determining the strictness of the triggering conditions.

[0019] In yet another specific embodiment of the present invention, the dynamic event triggering threshold is described. The specific design formula is as follows:

[0020]

[0021] in, The baseline threshold parameter is the threshold used by the system during stable operation and is dimensionless. This is the threshold adjustment sensitivity coefficient, in units of... , is an adjustable parameter greater than zero; It is the dimensionless derivative of the generator's active power with respect to time, and is the rate of change of active power, used to characterize the magnitude and urgency of disturbances to the system in real time.

[0022] In another specific embodiment of the present invention, the reward function designed during offline training using the deep deterministic policy gradient algorithm in step two is described. The specific form is as follows:

[0023]

[0024] in, The instantaneous reward describes the environmental feedback during offline training at time t, and is dimensionless. Terminal voltage deviation, unit: ; Rotor angular velocity deviation, in units of , Active power deviation, unit: ; The auxiliary excitation control signal output by the deep reinforcement learning controller at time t, in units of All are weighting coefficients greater than zero, used to balance the optimization objectives of voltage stability, damping enhancement, and control cost, and are dimensionless.

[0025] The present invention, by employing the above-described structure, has the following advantages compared to the prior art:

[0026] Firstly, it is safe and reliable. The DRL controller is trained offline in a simulation environment, which completely avoids the safety risks that online exploration may bring to real power systems.

[0027] Secondly, it boasts excellent performance, fully utilizing the powerful capabilities of DRL in handling system nonlinearity and uncertainty, and can provide near-optimal damping and voltage regulation effects under various complex and time-varying operating conditions.

[0028] Third, it is resource efficient, with a dynamic event triggering mechanism that intelligently manages control updates, ensuring control performance during large disturbances and greatly saving computing and communication resources during steady-state periods, thus extending equipment lifespan. It is particularly suitable for resource-constrained edge controllers.

[0029] Fourth, it has strong adaptability. The trigger threshold can be dynamically and automatically adjusted according to the system, so that the controller can always maintain the best control update strategy under different operating conditions, and achieve an adaptive balance between performance and efficiency. Attached Figure Description

[0030] Figure 1 This is a block diagram illustrating the overall system architecture of the present invention;

[0031] Figure 2 This is a software flowchart of the online operation phase of the present invention;

[0032] Figure 3a This is a comparison chart of the dynamic response of the present invention with that of traditional PID control and event-triggered control using a fixed threshold;

[0033] Figure 3b This diagram compares the control update modes of the present invention with those of traditional PID control and event-triggered control using a fixed threshold. Detailed Implementation

[0034] The specific embodiments of the present invention are described in detail below with reference to the accompanying drawings. However, the description of the embodiments is not a limitation on the technical solution. Any formal but not substantive changes made based on the concept of the present invention should be considered within the scope of protection of the present invention.

[0035] In the following description, all directional (or orientational) concepts involving up, down, left, right, front, and back refer to the position of the figure being described, and are intended to facilitate public understanding. Therefore, they should not be construed as a special limitation on the technical solution provided by this invention.

[0036] This invention aims to: circumvent the risks of online learning in DRL (Deep Reinforcement Learning) by obtaining a high-performance nonlinear excitation controller through completely offline training; address the poor flexibility of fixed-threshold event-triggered control by introducing a dynamic mechanism that adaptively adjusts the trigger threshold according to the intensity of system disturbances; and organically combine these two aspects to construct a collaborative framework, ensuring that the high-performance DRL controller is activated only "when necessary" (determined by the dynamic event triggering mechanism), thereby minimizing the controller's computational and communication overhead while strictly guaranteeing system stability and dynamic quality.

[0037] The present invention provides an excitation control method based on dynamic event triggering and deep reinforcement learning, which includes the following steps.

[0038] Step 1: Establish a high-precision simulation mathematical model. Specifically, establish a simulation mathematical model of the synchronous generator and excitation system for controller design and training, ensuring that it accurately reflects the dynamic characteristics of the system under various disturbances. The state-space equations of this synchronous generator and excitation system are used to describe the dynamic behavior of the system.

[0039] Establishing a high-precision simulation mathematical model is fundamental to all subsequent design and training. A sufficiently accurate model is needed to simulate the dynamic response of a real generator-excitation system under various disturbances. This embodiment suggests using a single-machine infinite bus system model that includes the dynamics of the excitation system. The state-space description of the synchronous generator and excitation system is as follows:

[0040] Let the system state vector be ,in: The deviation is the angle of attack (rad). This refers to the rotor angular velocity deviation (pu). pu, Let q-axis transient potential deviation (pu) be the mechanical power. If constant, then the system of differential-algebraic equations for this nonlinear system can be expressed as:

[0041]

[0042] Among them and The formula is as follows:

[0043]

[0044]

[0045] To conform to professional practices in power system analysis and control, and to ensure the universality of the simulation model, the mathematical models and parameters involved in this invention are calculated using a per-unit (pu) system, i.e., the ratio of the actual value to a selected reference value. In the above formula, Electromagnetic power is the active power delivered by the generator to the power grid, measured in megohmmeters (µm). ; and q-axis potential It is a function of state variables and network parameters; The derivative of the angle deviation with respect to time is given by units of 1 / 2π. This is the power angle deviation caused by the speed deviating from the synchronous speed, indicating the power angle deviation. The rate of change; Let be the system's reference angular frequency, which is a constant. This is the system's rated frequency, in units of... It is the standard benchmark value for power system analysis, and in China it is 50. ,Right now It serves as a conversion factor here; Rotor angular velocity deviation, in units of , is the actual angular velocity of the rotor. With synchronous angular velocity difference; This is the derivative of the angular velocity deviation with respect to time, in units of . It is the rotor angular velocity deviation. The rate of change; The inertial time constant is expressed in units of 1. , is the magnitude of the rotor's moment of inertia; The generator's inertial constant is derived from the rotor's motion equations; Electromagnetic power deviation, in units of It is the electromagnetic power output by the generator. Its initial steady-state value The difference; This is the damping coefficient, in units of... This represents the natural damping caused by friction of rotating parts, wind resistance, etc. Let be the derivative of the q-axis transient potential deviation with respect to time, in units of . , is the rate of change of the transient potential deviation; The d-axis open-circuit transient time constant is expressed in units of 1000 m / s. , is the core time constant of the excitation system, which characterizes the dynamic characteristics of the magnetic field decay of the excitation winding; This represents the excitation electromotive force deviation, in units of... , is the output of the excitation system; The q-axis synchronous potential deviation is expressed in units of... is an algebraic variable; These are the d-axis synchronous reactance, d-axis transient reactance, and q-axis synchronous reactance, respectively, with units of... ; These are the total d-axis transient reactance and total q-axis synchronous reactance, including transformer and line reactance, respectively, in units of... ; This is the generator terminal voltage, in units of... ; It is the equivalent voltage output by the excitation system. It is an algebraic state variable. This is the d-axis stator current of the generator. In a single-machine infinite bus system, it can be approximated as constant or slowly changing. This model is a classic third-order model in power system transient stability analysis, and its correctness and effectiveness have been widely verified, providing a reliable virtual environment for the DRL training of this invention.

[0046] Step 2: Offline Training of the Deep Reinforcement Learning Controller. Specifically, based on the simulation mathematical model established in Step 1, a deep reinforcement learning training environment is constructed, and a deep reinforcement learning controller is trained offline using the deep deterministic policy gradient algorithm. After training, the actor network of this controller can map an approximately optimal excitation control auxiliary signal based on the input system state vector.

[0047] In this step, the model established in Step 1 is used as the simulation environment, and a Deep Deterministic Policy Gradient (DDPG) algorithm is employed to train a DRL controller. DDPG is an Actor-Critic algorithm applicable to continuous action spaces, making it well-suited for problems such as excitation control where the output is a continuous control signal.

[0048] Furthermore, the design of the state space, action space, and reward function is as follows. State Space The state vector observed by the agent, selected in this embodiment. ,in To account for the terminal voltage deviation, an integral term is introduced. It helps eliminate steady-state errors. Action space The control actions output by the intelligent agent. Defined as... This refers to a continuous auxiliary excitation control signal, which is superimposed on the output of the PID controller to jointly form the total excitation voltage reference value. Reward function Used to guide the learning direction of the intelligent agent; designed here as... ,in, The excitation system's basic task is to penalize voltage deviations and encourage the system to maintain voltage stability. The penalty for speed deviation encourages the system to suppress frequency fluctuations and power oscillations, providing positive damping; This term further strengthens the suppression of power oscillations and works synergistically with the speed deviation term; The term is a penalty for the control amplitude, preventing the controller from outputting excessively large control signals, ensuring control smoothness, and avoiding actuator saturation; weighting coefficients It is necessary to balance the priorities of various tasks through debugging. Usually, voltage stability is key. ) and damping enhancement ( The weight of ) is relatively high.

[0049] A brief overview of the DDPG algorithm: DDPG maintains four neural networks simultaneously, namely the Actor network, ), Target Actor Network ( ), Critic Network (Critic, ), Target Commentator Network ( Its core iterative process is as follows: First, experience replay generates transfer samples from the interaction between the agent and the environment. Stored in the experience replay buffer; then during network updates, the critic update will minimize the loss due to temporal difference error. Update ,in Actor updates will be implemented through a strategy-based gradient update. : The final target is a network soft update. , ,in After thorough training, the actors' network This refers to the required DRL control, which can generate near-optimal control signals.

[0050] Step 3: Design a dynamic event triggering mechanism. Specifically, the triggering condition of this mechanism is determined by the error norm between the current state of the system and the state at the last triggering moment, the rate of change norm of this error, and a dynamic positive threshold. This dynamic threshold is related to a physical quantity that reflects the severity of the disturbance to the system and can change inversely accordingly.

[0051] Step three is the key innovation of this invention. The dynamic event triggering mechanism described herein has the following mathematical triggering condition:

[0052]

[0053] in, Indicates the current state and the time of the last trigger. The error vector of the state. In this embodiment, the 2-norm (Euclidean norm) of the vector is used to comprehensively measure the magnitude of the error; The derivative of the error vector reflects the trend and speed of state changes. Introducing this term can help detect rapid deterioration of the state in advance, thus playing an "early warning" role. , is a weighting coefficient greater than zero, used to adjust the weight of the rate of change term in the triggering decision. The larger the value, the more sensitive the system is to the rate of state change; The dynamic threshold is the core of this mechanism.

[0054]

[0055] in, The rate of change of active power is a highly sensitive and readily available indicator in power systems, characterizing the magnitude and urgency of disturbances. Large disturbances (such as short circuits and generator tripping) can lead to... The absolute value of it increases dramatically.

[0056] When the system is subjected to a large disturbance Enlargement, leading to This reduction makes the triggering conditions easier to meet, thereby increasing the frequency of control updates and ensuring that the DRL controller can intervene in a timely manner to provide strong control to quell oscillations and maintain stability.

[0057] When the system is in steady state or under small disturbances Approaching zero This is a relatively large value. This makes the triggering conditions more difficult to meet, thus significantly reducing the frequency of control updates and saving computing resources and communication bandwidth. It is the baseline threshold, which determines the degree of resource saving of the system under quasi-steady state. It is the sensitivity coefficient, which determines the strength of the threshold's response to the rate of power change.

[0058] Step 4: Online Integration and Operation. Specifically, the actor network parameters of the deep reinforcement learning controller trained in Step 2 are solidified and deployed to the excitation regulator control unit along with the dynamic event triggering mechanism designed in Step 3. This dynamic event triggering mechanism continuously monitors the status of the generator and the power grid system. Only when its triggering condition is met will the deep reinforcement learning controller be invoked to calculate and update the current control signal output; otherwise, the control unit will maintain the control signal unchanged from the last triggering moment.

[0059] The trained actor network parameters The logic of the dynamic event triggering mechanism is embedded into the online operation process of the control software, such as... Figure 2 As shown, the details are as follows:

[0060] Initialization, setting the initial state Control quantity ,make , During online operation, the control unit works according to the following process;

[0061] State sampling involves sampling and obtaining the current system state in every extremely short system clock cycle. Trigger judgment and calculate error. and its derivative (This can be approximated by numerical difference), and the current rate of change of active power is obtained to calculate the dynamic threshold. Determine the inequality Whether the condition is met; control decision and output. If the condition is met (triggered), then the current state is changed. The input is fed into the existing DRL actor network to calculate the new control signal. Then update the record. , If the condition is not met (not triggered), the control output remains the value of the last trigger. Finally, the signals are superimposed, As an auxiliary signal, it is used in conjunction with the output of the basic PID controller. The values ​​are added together to obtain the total excitation voltage reference value, which is then applied to the excitation system.

[0062] The technical solution of the present invention will be described in detail below with reference to specific embodiments.

[0063] Implementation Environment: This invention is implemented in a power system containing a large synchronous generator. The basic parameters of the generator are as follows: rated power... Rated voltage Inertial constant , , , , The system operates near its rated operating conditions. The specific implementation steps are as follows.

[0064] The first step is to establish and verify the simulation model, which corresponds to step one.

[0065] First, based on the generator parameters and network structure described above, a third-order generator model is built in MATLAB, and all parameters are precisely set, including... , wait;

[0066] The model is then validated. For example, a small step disturbance (such as a step change in the excitation voltage reference value) is applied, and its dynamic response is observed to ensure that the response trend conforms to physical laws (such as the power angle and speed exhibiting oscillating decay characteristics). The response curves in classic literature can be compared to verify the correctness of the model.

[0067] The second step is offline training of the DRL controller, corresponding to step two.

[0068] First, the training environment is configured. The model that has passed the first step of verification is encapsulated into a training environment that conforms to the interface standard. The step function of this environment calculates the next state and reward by solving differential equations based on the current state and action (auxiliary excitation signal).

[0069] Next is the setting of algorithm parameters and network structure, among which,

[0070] Actor Network: It is recommended to use a fully connected neural network with two hidden layers, 400 and 300 neurons respectively, using ReLU activation function, and tanh activation function in the output layer to restrict the actions to... Within the range (i.e.) ),

[0071] The Critic network also uses two hidden layers with 400 and 300 neurons respectively. Its input is a concatenation of state and action.

[0072] Deep reinforcement learning algorithm parameters: learning rate (Actor: 0.0001, Critic: 0.001), discount factor (Note the triggering mechanism) Distinguishing features), soft update parameters Experience replay buffer size Batch size .

[0073] Reward function weights: initially set to , , , In actual training, adjustments can be made based on the results.

[0074] Training Process: During training, various types of perturbations need to be applied to the system, such as short-circuit faults at different locations and for different durations, and random load fluctuations, to allow the agent to learn to cope with various operating conditions. Training continues until the evaluation metric (such as average round reward) converges to a high and stable level. At this point, the final parameters of the actor network are saved. .

[0075] Step 3: Parameter tuning for the dynamic event triggering mechanism, corresponding to step three above.

[0076] Parameter selection: (Trigger mechanism weight) It is recommended to set the initial value to 0. It affects the sensitivity to the rate of state change. The (baseline threshold) needs to be determined through simulation experiments. The principle is that, under conditions of no disturbance or minimal disturbance, it should effectively reduce unnecessary control updates by more than 80%. This can be achieved by... Start trying. The (sensitivity coefficient) determines the threshold's response speed to disturbances and can be set to... The larger the value, the faster the threshold decreases and the more frequently it is updated.

[0077] Tuning method: Using a trained DRL controller, apply a typical disturbance (such as a three-phase short circuit that is cut off after 0.1s) in the simulation, observe the dynamic response of the system (such as the maximum swing angle of the power angle and the settling time) and the total number of control updates under different parameters, and select a set of parameters that can minimize the number of updates while ensuring that the performance degradation is acceptable (compared to continuous control).

[0078] Step 4: Online simulation testing and performance evaluation. See details for step four. Figure 2 .

[0079] System integration: Building systems such as simulation software... Figure 1 The complete control system shown deploys the trained actor network as a static function module to implement the judgment logic of the dynamic event triggering mechanism.

[0080] Test scenario:

[0081] Scenario A (large disturbance): A three-phase short circuit occurs at the generator outlet, lasting 0.15 seconds before the fault is cleared. The control effect is observed and compared with traditional PID control and event-triggered control using a fixed threshold. Expected results are as follows: Figure 3a As shown, this embodiment can quell oscillations most quickly;

[0082] Scenario B (Small Disturbance): Simulating small random fluctuations in load, by recording the number of control updates, verifies that this embodiment can significantly reduce control actions under steady state. Figure 3b The steady-state phase is shown;

[0083] Performance indicators: Quantitatively compare the indicators of each method, including the maximum deviation of the control angle, overshoot, settling time (to ±2% error band), and total number of control updates.

[0084] In summary, this invention provides a system that possesses powerful nonlinear processing capabilities and excellent dynamic performance, ensures safety in engineering applications, and intelligently conserves computing and communication resources, enabling the controller to maintain the optimal control update strategy under different operating conditions, thus achieving the invention's objective.

Claims

1. An excitation control method based on dynamic event triggering and deep reinforcement learning, characterized in that... Includes the following steps: Step 1: Establish a simulation mathematical model of the synchronous generator and excitation system for controller design and training, and make it able to accurately reflect the dynamic characteristics of the system under various disturbances. The state-space equation of the synchronous generator and excitation system is used to describe the dynamic behavior of the system. Step 2: Based on the simulation mathematical model established in Step 1, a deep reinforcement learning training environment is constructed, and a deep reinforcement learning controller is trained offline using the deep deterministic policy gradient algorithm. After training, the actor network of the controller can map an approximately optimal excitation control auxiliary signal according to the input system state vector. Step 3: Design a dynamic event triggering mechanism. The triggering condition of this mechanism is determined by the error norm between the current state of the system and the state at the last triggering moment, the rate of change norm of the error, and a dynamic positive threshold. This dynamic threshold is related to a physical quantity that reflects the severity of the disturbance to the system and can change in the opposite direction. Step four: Solidify the actor network parameters of the deep reinforcement learning controller trained in step two, and deploy them together with the dynamic event triggering mechanism designed in step three to the excitation regulator control unit. The dynamic event triggering mechanism continuously monitors the status of the generator and the power grid system. Only when its triggering condition is met will the deep reinforcement learning controller be called to calculate and update the current control signal output. Otherwise, the control unit will maintain the control signal unchanged at the last triggering moment.

2. The excitation control method based on dynamic event triggering and deep reinforcement learning according to claim 1, characterized in that: The simulation mathematical model described in step one has state variables including at least the power angle deviation, rotor angular velocity deviation, and q-axis transient potential deviation, and its differential equation is described as follows: in, The derivative of the angle deviation with respect to time is given by units of 1 / 2π. This is the power angle deviation caused by the speed deviating from the synchronous speed, indicating the power angle deviation. The rate of change; Let be the system's reference angular frequency, which is a constant. This is the system's rated frequency; the unit is... ; Rotor angular velocity deviation, in units of , is the actual angular velocity of the rotor. With synchronous angular velocity difference; This is the derivative of the angular velocity deviation with respect to time, in units of . It is the rotor angular velocity deviation. The rate of change; The inertial time constant is expressed in units of 1. , is the magnitude of the rotor's moment of inertia; Electromagnetic power deviation, in units of It is the electromagnetic power output by the generator. Its initial steady-state value The difference; This is the damping coefficient, in units of... This is the natural damping effect experienced by the generator rotor; Let be the derivative of the q-axis transient potential deviation with respect to time, in units of . , is the rate of change of the transient potential deviation; The d-axis open-circuit transient time constant is expressed in units of 1000 m / s. , is the core time constant of the excitation system; This represents the excitation electromotive force deviation, in units of... , is the output of the excitation system; The q-axis synchronous potential deviation is expressed in units of... is an algebraic variable.

3. The excitation control method based on dynamic event triggering and deep reinforcement learning according to claim 1, characterized in that: The dynamic event triggering mechanism described in step three has the following mathematical triggering condition: in, It is a norm; Indicates the current time system state vector Compared to the time of the last event trigger state vector The error vector between them is dimensionless; This is the time derivative of the error vector, which is dimensionless. This is a weighting coefficient greater than zero, used to adjust the influence of the rate of change term in triggering the decision, with units of... This determines the importance of the rate of error change in triggering decisions; The threshold for triggering dynamic events is dimensionless and is a positive number that changes over time, determining the strictness of the triggering conditions.

4. The excitation control method based on dynamic event triggering and deep reinforcement learning according to claim 3, characterized in that: The dynamic event trigger threshold The specific design formula is as follows: in, The baseline threshold parameter is the threshold used by the system during stable operation and is dimensionless. This is the threshold adjustment sensitivity coefficient, in units of... , is an adjustable parameter greater than zero; It is the dimensionless derivative of the generator's active power with respect to time, and is the rate of change of active power, used to characterize the magnitude and urgency of disturbances to the system in real time.

5. The excitation control method based on dynamic event triggering and deep reinforcement learning according to claim 1, characterized in that: The reward function designed during offline training using the deep deterministic policy gradient algorithm in step two is described above. The specific form is as follows: in, The instantaneous reward describes the environmental feedback during offline training at time t, and is dimensionless. Terminal voltage deviation, unit: ; Rotor angular velocity deviation, in units of , Active power deviation, unit: ; The auxiliary excitation control signal output by the deep reinforcement learning controller at time t, in units of ; , , , All are weighting coefficients greater than zero, used to balance the optimization objectives of voltage stability, damping enhancement, and control cost, and are dimensionless.

Images