Method and device for generating power system control action, medium and terminal

Abstract

The application discloses a power system control action generation method and device, medium and terminal, relates to the technical field of artificial intelligence and the power field, and mainly aims at the problems that the existing robustness evaluation method is difficult to comprehensively reflect the stability of the control action generated by the intelligent agent under complex disturbance conditions, cannot effectively evaluate the vulnerability of the intelligent agent under extreme operation scenarios, and cannot guarantee whether the robustness of the intelligent agent truly meets the demand, thereby causing the generated control action to be difficult to cope with the fast change operation state of the power grid. The method comprises the following steps: in the target power grid operation process, a disturbance event is monitored in real time, and current time power grid state time series data of the target power grid is acquired when the disturbance event is monitored; a control action generation intelligent agent is trained based on the completed model, a current time control action sequence is generated according to the current time power grid state time series data; and the target power grid is controlled according to the current time control action sequence to cope with the disturbance event.

Description

Technical Field

[0001] This application relates to the fields of artificial intelligence technology and power, and in particular to a method, device, medium, and terminal for generating control actions of a power system. Background Technology

[0002] With the widespread application of artificial intelligence technology in power system dispatching, operation control, and security defense, agent-based automated decision-making methods are gradually becoming an important technological direction for improving power grid operation efficiency and security. During power system operation, agents need to generate control actions in real time based on system state information. Therefore, robustness evaluation is a crucial step in the construction of agents.

[0003] In existing technologies, the vast majority of robustness assessment methods rely on a single type of operational metric for analysis.

[0004] However, relying solely on a single type of operational metric to assess the robustness of an agent makes it difficult to comprehensively reflect the overall stability of the control actions generated by the agent under complex disturbance conditions, and also fails to effectively assess the vulnerability of the agent in extreme operating scenarios. This results in the inability to guarantee whether the robustness of the agent truly meets the requirements, and consequently, the generated control actions are unable to cope with the rapidly changing operating state of the power grid. Summary of the Invention

[0005] In view of this, this application provides a method, device, medium, and terminal for generating control actions in a power system. The main purpose is that existing robustness assessment methods are unable to fully reflect the overall stability of control actions generated by an agent under complex disturbance conditions, nor can they effectively assess the vulnerability of the agent under extreme operating scenarios. This results in the inability to guarantee whether the robustness of the agent truly meets the requirements, and consequently, the generated control actions are unable to cope with the rapidly changing operating state of the power grid.

[0006] According to one aspect of this application, a method for generating power system control actions is provided, comprising: During the operation of the target power grid, disturbance events are monitored in real time, and when a disturbance event is detected, the current time-series data of the target power grid state are obtained. Based on the control action generation agent that has completed model training, the control action sequence at the current moment is generated according to the current power grid state time series data. The control action generation agent is built on an adversarial network and is obtained by training the model with the comprehensive robustness score of the control action generation agent as the learning reward. The comprehensive robustness score is used to evaluate the stability and safety of the control action generation agent under different operating stages after disturbance. The target power grid is controlled according to the current time control action sequence to respond to the disturbance event.

[0007] Preferably, before generating the control action sequence for the current moment based on the current power grid state time-series data according to the control action generation agent that has completed model training, the method further includes: Construct a power system adversarial simulation environment that matches the target power grid, and construct a disturbance generation agent and a control action generation agent in the power system adversarial simulation environment; In the power system adversarial simulation environment, the disturbance generation agent is controlled to generate simulated disturbance events, and the current simulation time-series data of the power grid state after the occurrence of the simulated disturbance events is obtained. Based on the control action generation agent, the current simulation time control action sequence is generated according to the current simulation time-series data of the power grid state. Based on the preset operation phase division rules, the current simulation time-series data of the power grid state is divided into the current simulation time-transient operation phase time-series data and the current simulation time-steady-state operation phase time-series data. For the transient operation phase, sampled values ​​corresponding to preset transient stability key features are obtained from the time series data of the transient operation phase at the current simulation time. Transient system deviation index value is calculated based on the sampled values ​​corresponding to the preset transient stability key features. Sampled values ​​corresponding to preset transient response features are obtained from the time series data of the transient operation phase at the current simulation time. Transient vulnerability manifold feature index value is calculated based on the sampled values ​​corresponding to the preset transient response features. Transient robustness fusion weight is calculated according to the disturbance intensity of the simulated disturbance event. The transient system deviation index value and the transient vulnerability manifold feature index value are weighted and fused based on the transient robustness fusion weight to obtain a transient robustness score. For the steady-state operation phase, sampled values ​​corresponding to preset steady-state stability key features are obtained from the time series data of the steady-state operation phase at the current simulation time. Based on the sampled values ​​corresponding to the preset steady-state stability key features, a steady-state system deviation index value is calculated. Also, sampled values ​​corresponding to preset steady-state response features are obtained from the time series data of the steady-state operation phase at the current simulation time. Based on the sampled values ​​corresponding to the preset steady-state response features, a steady-state vulnerability manifold feature index value is calculated. Furthermore, a steady-state robustness fusion weight is calculated based on the disturbance intensity of the simulated disturbance event. The steady-state system deviation index value and the steady-state vulnerability manifold feature index value are then weighted and fused based on the steady-state robustness fusion weight to obtain a steady-state robustness score. Based on the preset comprehensive robustness fusion weights, the transient robustness score and the steady-state robustness score are weighted and fused to obtain the comprehensive robustness score at the current simulation time. The comprehensive robustness score at the current simulation moment is used as a learning reward to update the policy parameters of the control action generating agent; Repeatedly generate simulation disturbance events, calculate the comprehensive robustness score, and update the policy parameters until the comprehensive robustness score of the control action generation agent meets the preset convergence condition, thus obtaining a control action generation agent with completed model training, and generating a control action sequence based on the control action generation agent with completed model training.

[0008] Preferably, before updating the policy parameters of the agent generating the control action by using the comprehensive robustness score at the current simulation moment as a learning reward, the method further includes: Obtain the sampled values ​​corresponding to the preset transient response features and the sampled values ​​corresponding to the preset steady-state response features, and combine them to obtain a set of response feature sampled values; For each response feature sample value in the set of response feature sample values, a small perturbation is applied to the current response feature sample value, while other response feature sample values ​​remain unchanged, resulting in a perturbed set of response feature sample values. Based on the perturbed set of response feature sample values, a perturbed comprehensive robustness score is calculated, and the difference between the perturbed comprehensive robustness score and the comprehensive robustness score at the current simulation time is calculated as the contribution of the robustness score of the current response feature corresponding to the current response feature sample value. Each response feature is sorted in descending order of its contribution to the robustness score to obtain a response feature sequence. Starting from the first position of the response feature sequence, a preset number of response features are selected sequentially to generate a set of key physical variables. A learning reward value is then generated based on the set of key physical variables and the comprehensive robustness score at the current simulation moment.

[0009] Preferably, the preset transient stability key feature is a system frequency feature, and the calculation of the transient system deviation index value based on the sampled values ​​corresponding to the preset transient stability key feature includes: Arrange the sampled values ​​corresponding to the system frequency characteristics in ascending order to obtain the system frequency characteristic sampled value sequence; According to the preset interval division rules, the system frequency feature sample value sequence is divided into multiple consecutive frequency intervals; For each frequency interval, the number of sampled values ​​within the current frequency interval is counted, and the ratio of the number to the number of sampled values ​​corresponding to the system frequency characteristics is calculated to obtain the probability of the occurrence of the interval sampling point corresponding to the current frequency interval. The transient system deviation information entropy value is calculated based on the probability of occurrence of interval sampling points in each frequency interval; Based on the theoretical maximum transient system deviation information entropy value, the transient system deviation information entropy value is normalized to obtain the transient system deviation index value.

[0010] Preferably, the preset transient response characteristics include system frequency characteristics, power angle dynamic characteristics, and voltage transient response characteristics. The calculation of the transient vulnerability manifold characteristic index value based on the sampled values ​​corresponding to the preset transient response characteristics includes: Based on the sampled values ​​corresponding to the system frequency characteristics, the sampled values ​​corresponding to the power angle dynamic characteristics, and the sampled values ​​corresponding to the voltage transient response characteristics, a transient high-dimensional sampled value matrix is ​​constructed; Each sample value in the transient high-dimensional sample value matrix is ​​normalized for its transient feature dimension to obtain a transient high-dimensional sample value matrix with normalized transient feature dimension. For each transient high-dimensional sample value that has completed transient feature dimension normalization, the current transient high-dimensional sample value that has completed transient feature dimension normalization is reduced to a low-dimensional space based on the manifold learning algorithm, so as to obtain the low-dimensional space coordinates corresponding to the current transient high-dimensional sample value that has completed transient feature dimension normalization. Obtain the preset normal cluster center coordinates, calculate the transient deviation distance between the low-dimensional spatial coordinates of each transient high-dimensional sample value that has completed transient feature dimension normalization and the preset normal cluster center coordinates, and calculate the mean of all the transient deviation distances to obtain the transient manifold deviation index value. Based on the maximum transient deviation distance, the transient manifold deviation index value is normalized to obtain the transient vulnerability manifold characteristic index value.

[0011] Preferably, the preset steady-state stability key feature is the node voltage amplitude feature, and the calculation of the steady-state system deviation index value based on the sampled values ​​corresponding to the preset steady-state stability key feature includes: Arrange the sampled values ​​corresponding to the node voltage amplitude features in ascending order to obtain the node voltage amplitude feature sampled value sequence; According to the preset interval division rules, the node voltage amplitude feature sampling value sequence is divided into multiple consecutive node voltage amplitude intervals; For each node voltage amplitude range, the number of sampled values ​​within the current node voltage amplitude range is counted, and the ratio of this number to the number of sampled values ​​corresponding to the node voltage amplitude feature is calculated to obtain the probability of the occurrence of the interval sampling point corresponding to the current node voltage amplitude range. The steady-state system deviation information entropy value is calculated based on the probability of occurrence of interval sampling points in the voltage amplitude range of each node; Based on the theoretical maximum steady-state system deviation information entropy value, the steady-state system deviation information entropy value is normalized to obtain the steady-state system deviation index value.

[0012] Preferably, the preset steady-state response characteristics include node voltage amplitude characteristics, line active power flow characteristics, and generator active power output characteristics. The calculation of the steady-state vulnerability manifold characteristic index value based on the sampled values ​​corresponding to the preset steady-state response characteristics includes: Based on the sampled values ​​corresponding to the node voltage amplitude characteristics, the sampled values ​​corresponding to the line active power flow characteristics, and the sampled values ​​corresponding to the unit active power output characteristics, a steady-state high-dimensional sampled value matrix is ​​constructed. Each sample value in the steady-state high-dimensional sample value matrix is ​​normalized to its steady-state feature dimension to obtain a steady-state high-dimensional sample value matrix with normalized steady-state feature dimension. For each steady-state high-dimensional sample value that has completed steady-state feature dimension normalization, the current steady-state high-dimensional sample value that has completed steady-state feature dimension normalization is reduced to a low-dimensional space based on the manifold learning algorithm, so as to obtain the low-dimensional space coordinates corresponding to the current steady-state high-dimensional sample value that has completed steady-state feature dimension normalization. Obtain the preset normal cluster center coordinates, calculate the steady-state deviation distance between the low-dimensional spatial coordinates of each steady-state high-dimensional sample value that has completed steady-state feature dimension normalization and the preset normal cluster center coordinates, and calculate the mean of all the steady-state deviation distances to obtain the steady-state manifold deviation index value. Based on the maximum steady-state deviation distance, the steady-state manifold deviation index value is normalized to obtain the steady-state vulnerability manifold characteristic index value.

[0013] According to another aspect of this application, a power system control action generation device is provided, comprising: The disturbance event monitoring module is used to monitor disturbance events in real time during the operation of the target power grid, and to obtain the current time-series data of the power grid status of the target power grid when a disturbance event is detected. The control action sequence generation module is used to generate a control action generation agent based on the completed model training. According to the current power grid state time series data, the control action generation agent generates a control action sequence at the current time. The control action generation agent is built based on an adversarial network and is obtained by training the model with the comprehensive robustness score of the control action generation agent as the learning reward. The comprehensive robustness score is used to evaluate the stability and safety of the control action generation agent under different operating stages after disturbance. The control action sequence execution module is used to control the target power grid according to the control action sequence at the current time in order to respond to the disturbance event.

[0014] Preferably, before the control action sequence generation module, the device further includes a control action generation agent training module, comprising: The intelligent agent construction unit is used to construct a power system adversarial simulation environment that matches the target power grid, and to construct a disturbance generation intelligent agent and a control action generation intelligent agent in the power system adversarial simulation environment; The adversarial simulation unit is used to control the disturbance generation agent to generate simulated disturbance events in the power system adversarial simulation environment, and to obtain the current simulation time-series data of the power grid state after the occurrence of the simulated disturbance event. Based on the control action generation agent, it generates the current simulation time control action sequence according to the current simulation time-series data of the power grid state. The operation phase division unit is used to divide the current simulation time-series data of the power grid state into the current simulation time-transient operation phase time-series data and the current simulation time-steady-state operation phase time-series data based on the preset operation phase division rules. The transient robustness scoring calculation unit is used to, for the transient operation phase, obtain sampled values ​​corresponding to preset transient stability key features from the time series data of the transient operation phase at the current simulation time, calculate transient system deviation index values ​​based on the sampled values ​​corresponding to the preset transient stability key features, obtain sampled values ​​corresponding to preset transient response features from the time series data of the transient operation phase at the current simulation time, calculate transient vulnerability manifold feature index values ​​based on the sampled values ​​corresponding to the preset transient response features, calculate transient robustness fusion weights according to the disturbance intensity of the simulated disturbance event, and perform weighted fusion of the transient system deviation index values ​​and the transient vulnerability manifold feature index values ​​based on the transient robustness fusion weights to obtain a transient robustness score; The steady-state robustness scoring unit is used to, for the steady-state operation phase, obtain sampled values ​​corresponding to preset steady-state stability key features from the time series data of the steady-state operation phase at the current simulation time, calculate steady-state system deviation index values ​​based on the sampled values ​​corresponding to the preset steady-state stability key features, obtain sampled values ​​corresponding to preset steady-state response features from the time series data of the steady-state operation phase at the current simulation time, calculate steady-state vulnerability manifold feature index values ​​based on the sampled values ​​corresponding to the preset steady-state response features, calculate steady-state robustness fusion weights according to the disturbance intensity of the simulated disturbance event, and perform weighted fusion of the steady-state system deviation index values ​​and the steady-state vulnerability manifold feature index values ​​based on the steady-state robustness fusion weights to obtain a steady-state robustness score; The comprehensive robustness score calculation unit is used to perform weighted fusion of the transient robustness score and the steady-state robustness score based on a preset comprehensive robustness fusion weight to obtain the comprehensive robustness score at the current simulation time. The agent policy parameter update unit is used to update the policy parameters of the agent that generates the control action by taking the comprehensive robustness score at the current simulation moment as a learning reward. The loop training unit is used to repeatedly generate simulated disturbance events, calculate the comprehensive robustness score, and update the policy parameters until the comprehensive robustness score of the control action generation agent meets the preset convergence condition, thus obtaining a control action generation agent with completed model training, and generating a control action sequence based on the control action generation agent with completed model training.

[0015] Preferably, before the agent policy parameter update unit, the control action generation agent training module further includes an attribution analysis unit, used for: Obtain the sampled values ​​corresponding to the preset transient response features and the sampled values ​​corresponding to the preset steady-state response features, and combine them to obtain a set of response feature sampled values; For each response feature sample value in the set of response feature sample values, a small perturbation is applied to the current response feature sample value, while other response feature sample values ​​remain unchanged, resulting in a perturbed set of response feature sample values. Based on the perturbed set of response feature sample values, a perturbed comprehensive robustness score is calculated, and the difference between the perturbed comprehensive robustness score and the comprehensive robustness score at the current simulation time is calculated as the contribution of the robustness score of the current response feature corresponding to the current response feature sample value. Each response feature is sorted in descending order of its contribution to the robustness score to obtain a response feature sequence. Starting from the first position of the response feature sequence, a preset number of response features are selected sequentially to generate a set of key physical variables. A learning reward value is then generated based on the set of key physical variables and the comprehensive robustness score at the current simulation moment.

[0016] Preferably, the preset transient stability key feature is the system frequency feature, and the transient robustness scoring calculation unit is used for: Arrange the sampled values ​​corresponding to the system frequency characteristics in ascending order to obtain the system frequency characteristic sampled value sequence; According to the preset interval division rules, the system frequency feature sample value sequence is divided into multiple consecutive frequency intervals; For each frequency interval, the number of sampled values ​​within the current frequency interval is counted, and the ratio of the number to the number of sampled values ​​corresponding to the system frequency characteristics is calculated to obtain the probability of the occurrence of the interval sampling point corresponding to the current frequency interval. The transient system deviation information entropy value is calculated based on the probability of occurrence of interval sampling points in each frequency interval; Based on the theoretical maximum transient system deviation information entropy value, the transient system deviation information entropy value is normalized to obtain the transient system deviation index value.

[0017] Preferably, the preset transient response characteristics include system frequency characteristics, power angle dynamic characteristics, and voltage transient response characteristics. The transient robustness scoring calculation unit is further used for: Based on the sampled values ​​corresponding to the system frequency characteristics, the sampled values ​​corresponding to the power angle dynamic characteristics, and the sampled values ​​corresponding to the voltage transient response characteristics, a transient high-dimensional sampled value matrix is ​​constructed; Each sample value in the transient high-dimensional sample value matrix is ​​normalized for its transient feature dimension to obtain a transient high-dimensional sample value matrix with normalized transient feature dimension. For each transient high-dimensional sample value that has completed transient feature dimension normalization, the current transient high-dimensional sample value that has completed transient feature dimension normalization is reduced to a low-dimensional space based on the manifold learning algorithm, so as to obtain the low-dimensional space coordinates corresponding to the current transient high-dimensional sample value that has completed transient feature dimension normalization. Obtain the preset normal cluster center coordinates, calculate the transient deviation distance between the low-dimensional spatial coordinates of each transient high-dimensional sample value that has completed transient feature dimension normalization and the preset normal cluster center coordinates, and calculate the mean of all the transient deviation distances to obtain the transient manifold deviation index value. Based on the maximum transient deviation distance, the transient manifold deviation index value is normalized to obtain the transient vulnerability manifold characteristic index value.

[0018] Preferably, the preset steady-state stability key feature is the node voltage amplitude feature, and the steady-state robustness scoring calculation unit is used for: Arrange the sampled values ​​corresponding to the node voltage amplitude features in ascending order to obtain the node voltage amplitude feature sampled value sequence; According to the preset interval division rules, the node voltage amplitude feature sampling value sequence is divided into multiple consecutive node voltage amplitude intervals; For each node voltage amplitude range, the number of sampled values ​​within the current node voltage amplitude range is counted, and the ratio of this number to the number of sampled values ​​corresponding to the node voltage amplitude feature is calculated to obtain the probability of the occurrence of the interval sampling point corresponding to the current node voltage amplitude range. The steady-state system deviation information entropy value is calculated based on the probability of occurrence of interval sampling points in the voltage amplitude range of each node; Based on the theoretical maximum steady-state system deviation information entropy value, the steady-state system deviation information entropy value is normalized to obtain the steady-state system deviation index value.

[0019] Preferably, the preset steady-state response characteristics include node voltage amplitude characteristics, line active power flow characteristics, and generator active power output characteristics. The steady-state robustness scoring calculation unit is also used for: Based on the sampled values ​​corresponding to the node voltage amplitude characteristics, the sampled values ​​corresponding to the line active power flow characteristics, and the sampled values ​​corresponding to the unit active power output characteristics, a steady-state high-dimensional sampled value matrix is ​​constructed. Each sample value in the steady-state high-dimensional sample value matrix is ​​normalized to its steady-state feature dimension to obtain a steady-state high-dimensional sample value matrix with normalized steady-state feature dimension. For each steady-state high-dimensional sample value that has completed steady-state feature dimension normalization, the current steady-state high-dimensional sample value that has completed steady-state feature dimension normalization is reduced to a low-dimensional space based on the manifold learning algorithm, so as to obtain the low-dimensional space coordinates corresponding to the current steady-state high-dimensional sample value that has completed steady-state feature dimension normalization. Obtain the preset normal cluster center coordinates, calculate the steady-state deviation distance between the low-dimensional spatial coordinates of each steady-state high-dimensional sample value that has completed steady-state feature dimension normalization and the preset normal cluster center coordinates, and calculate the mean of all the steady-state deviation distances to obtain the steady-state manifold deviation index value. Based on the maximum steady-state deviation distance, the steady-state manifold deviation index value is normalized to obtain the steady-state vulnerability manifold characteristic index value.

[0020] According to another aspect of this application, a storage medium is provided that stores at least one executable instruction, which causes a processor to perform an operation corresponding to the above-described method for generating power system control actions.

[0021] According to another aspect of this application, a terminal is provided, comprising: a processor, a memory, a communication interface, and a communication bus, wherein the processor, the memory, and the communication interface communicate with each other through the communication bus; The memory is used to store at least one executable instruction, which causes the processor to perform the operation corresponding to the above-described method for generating power system control actions.

[0022] By employing the above technical solutions, the technical solutions provided in the embodiments of this application have at least the following advantages: This application provides a method, apparatus, medium, and terminal for generating control actions in a power system. First, during the operation of the target power grid, disturbance events are monitored in real time, and the current-moment grid state time-series data is acquired when a disturbance event is detected. Second, a control action generation agent, based on a pre-trained model, generates a control action sequence for the current moment according to the current-moment grid state time-series data. The control action generation agent is constructed based on an adversarial network, and the model is trained using the comprehensive robustness score of the control action generation agent as a learning reward. This comprehensive robustness score is used to evaluate the stability and security of the control action generation agent under different operating stages after a disturbance. Finally, the target power grid is controlled according to the current-moment control action sequence to respond to the disturbance event. Compared with existing technologies, the embodiments of this application construct a control action generation agent based on adversarial networks and use a comprehensive robustness score as a reinforcement learning reward for model training. This enables the control action generation agent to actively learn to cope with complex adversarial disturbance scenarios during training. Furthermore, the comprehensive robustness score comprehensively evaluates the stability and security of the control action generation agent in different operating stages after disturbance, avoiding the one-sidedness caused by relying on only a single type of operating indicator, and can comprehensively reflect the overall stability of control actions under complex disturbance conditions. At the same time, since the comprehensive scoring mechanism can quantify the vulnerability of the control action generation agent in extreme operating scenarios and guide the agent to optimize strategy parameters through reward signals, it effectively improves the adaptability of the control action generation agent to the rapidly changing operating state of the power grid, ensuring the robustness and reliability of the generated control actions under various disturbance events. The above description is only an overview of the technical solution of this application. In order to better understand the technical means of this application and to implement it in accordance with the contents of the specification, and to make the above and other objects, features and advantages of this application more obvious and understandable, the following are specific embodiments of this application. Attached Figure Description

[0023] Various other advantages and benefits will become apparent to those skilled in the art upon reading the following detailed description of preferred embodiments. The accompanying drawings are for illustrative purposes only and are not intended to limit the scope of this application. Furthermore, the same reference numerals denote the same parts throughout the drawings. In the drawings: Figure 1 A flowchart illustrating a method for generating power system control actions according to an embodiment of this application is shown; Figure 2 A flowchart of the control action generation agent training provided in an embodiment of this application is shown; Figure 3This illustration shows a block diagram of a power system control action generation device according to an embodiment of this application; Figure 4 A schematic diagram of the structure of a terminal provided in an embodiment of this application is shown. Detailed Implementation

[0024] Exemplary embodiments of the present disclosure will now be described in more detail with reference to the accompanying drawings. While exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the disclosure to those skilled in the art.

[0025] This application provides a method for generating power system control actions, such as... Figure 1 As shown, the method includes: 101. During the operation of the target power grid, monitor disturbance events in real time, and obtain the current time-series data of the target power grid status when a disturbance event is detected.

[0026] The current grid status time-series data includes at least the current system frequency time-series data, the current power angle dynamic time-series data, the current voltage transient response time-series data, the current node voltage amplitude time-series data, the current line active power flow time-series data, and the current unit active power output time-series data. The time-series data can be taken from 0.1 seconds after the disturbance occurs to 10 seconds after the disturbance is eliminated, and can be adjusted according to actual needs; this application embodiment does not impose specific limitations.

[0027] In this embodiment of the application, the current execution end can be the control action generation module of the power grid control system.

[0028] 102. Generate an intelligent agent based on the control actions trained on the completed model, and generate the control action sequence for the current moment according to the current power grid state time sequence data.

[0029] The control action generation agent is constructed based on an adversarial network and trained using its comprehensive robustness score as a learning reward. This comprehensive robustness score is used to evaluate the stability and safety of the control action generation agent at different operational stages after a disturbance. The current control action sequence represents a set of control commands executed sequentially from the occurrence of the disturbance to system recovery, including but not limited to adjusting generator active and reactive power output, switching capacitor banks, network topology reconfiguration, and load control strategies.

[0030] 103. Control the target power grid according to the current control action sequence to deal with disturbance events.

[0031] For example, at 0.1 seconds after a frequency drop is detected, the active power output of generator 1 is increased; at 0.5 seconds, a group of capacitors is connected; and at 1.0 seconds, load reduction is performed to respond to the detected disturbance event.

[0032] Compared with existing technologies, the embodiments of this application construct a control action generation agent based on adversarial networks and use a comprehensive robustness score as a reinforcement learning reward for model training. This enables the control action generation agent to actively learn to cope with complex adversarial disturbance scenarios during training. Furthermore, the comprehensive robustness score comprehensively evaluates the stability and security of the control action generation agent in different operating stages after disturbance, avoiding the one-sidedness caused by relying on only a single type of operating indicator, and can comprehensively reflect the overall stability of control actions under complex disturbance conditions. At the same time, since the comprehensive scoring mechanism can quantify the vulnerability of the control action generation agent in extreme operating scenarios and guide the agent to optimize strategy parameters through reward signals, it effectively improves the adaptability of the control action generation agent to the rapidly changing operating state of the power grid, ensuring the robustness and reliability of the generated control actions under various disturbance events.

[0033] In one embodiment of this application, for further definition and explanation, such as Figure 2 As shown, in the embodiment, before step 102, which generates an agent based on the control actions trained on the completed model, and according to the current power grid state time-series data, generates the current control action sequence, the method further includes: 201. Construct a power system adversarial simulation environment that matches the target power grid, and construct disturbance generation agents and control action generation agents in the power system adversarial simulation environment.

[0034] In this embodiment, a power system adversarial simulation environment matching the target power grid in terms of topology, component parameters, and operating characteristics is first constructed. Specifically, a general power system simulation platform can be used for construction, and a standard test system consistent with the target power grid, such as the IEEE 30-node standard power system, can be imported. This power system adversarial simulation environment includes a complete primary equipment model, such as generators, transformers, transmission lines, loads, and corresponding protection and control logic. Further, within the power system adversarial simulation environment, disturbance generation agents and control action generation agents are constructed based on an adversarial network. The disturbance generation agent generates various extreme disturbance events during the power system adversarial simulation process, such as sudden changes in node load, transmission line interruptions, and generator output disturbances, to simulate harsh operating scenarios that a real power grid may encounter. The control action generation agent generates corresponding control action sequences based on the simulation-time power grid state timing data fed back from the power system adversarial simulation environment, such as adjusting generator active and reactive power output, switching capacitor banks, and performing load shedding. The disturbance-generating agent and the control action-generating agent operate alternately in the same power system adversarial simulation environment, forming an adversarial mechanism and providing basic data for subsequent comprehensive robustness evaluation and agent training.

[0035] 202. In the power system adversarial simulation environment, control disturbance generation agent generates simulated disturbance events and obtains the current simulation time-series data of the power grid state after the occurrence of the simulated disturbance event. Based on the control action generation agent, the control action sequence of the current simulation time is generated according to the current simulation time-series data of the power grid state.

[0036] In this embodiment, in a power system adversarial simulation environment, a disturbance generation agent is first controlled to generate a simulated disturbance event according to a preset strategy. For example, this could involve increasing the active load of a node by 30% above its rated value or disconnecting a transmission line. Optionally, the disturbance generation agent can generate the simulated disturbance event at a fixed time in each adversarial round, such as the 1.0 second after the simulation begins, or it can generate the simulated disturbance event randomly at different time points. After the simulated disturbance event occurs, the system operating state subsequently changes. At this time, the time-series data of the power grid state after the disturbance is recorded. This data can be continuously acquired according to a preset sampling step size, thereby forming the current simulation time-series data of the power grid state after the simulated disturbance event. Furthermore, a control action generation agent, using the current simulation time-series data of the power grid state as input, generates a sequence of control actions for the current simulation time based on its internal initial strategy network. For example, this could involve first increasing the active power output of a generator, then connecting a capacitor bank, and subsequently performing load shedding.

[0037] 203. Based on the preset operation phase division rules, the power grid state time series data at the current simulation time is divided into the transient operation phase time series data at the current simulation time and the steady-state operation phase time series data at the current simulation time.

[0038] Among them, the preset operation phase division rules, for example, the period from 0.1 seconds to 2.0 seconds after the disturbance occurs is divided into the transient operation phase; and the period from 5.0 seconds to 10.0 seconds after the disturbance is eliminated is divided into the steady-state operation phase.

[0039] 204. For the transient operation phase, obtain the sampled values ​​corresponding to the preset transient stability key features from the time series data of the transient operation phase at the current simulation time, and calculate the transient system deviation index value based on the sampled values ​​corresponding to the preset transient stability key features.

[0040] Among them, the key characteristic of preset transient stability is the system frequency characteristic. It can be understood that the system frequency is the most direct indicator for measuring the transient stability of the power system, and the fluctuation amplitude of the frequency after a disturbance can fully reflect the regulation effect of the control action generating agent in an emergency.

[0041] Accordingly, step 204 of the embodiment, which calculates the transient system deviation index value based on the sampled values ​​corresponding to the preset transient stability key features, specifically includes: arranging the sampled values ​​corresponding to the system frequency features in ascending order to obtain a sequence of system frequency feature sampled values; dividing the sequence of system frequency feature sampled values ​​into multiple consecutive frequency intervals according to a preset interval division rule; for each frequency interval, counting the number of sampled values ​​in the current frequency interval and calculating the ratio of the number to the number of sampled values ​​corresponding to the system frequency features to obtain the probability of occurrence of the interval sampling point corresponding to the current frequency interval; calculating the transient system deviation information entropy value based on the probability of occurrence of the interval sampling point in each frequency interval; and performing transient system deviation information entropy normalization processing on the transient system deviation information entropy value based on the theoretical maximum transient system deviation information entropy value to obtain the transient system deviation index value.

[0042] To quantify the dispersion of the system frequency distribution, the sampled values ​​corresponding to the system frequency characteristics are first arranged in ascending order of numerical value to obtain a sequence of system frequency characteristic sampled values. Then, according to a preset interval division rule, the sequence of system frequency characteristic sampled values ​​is divided into multiple consecutive frequency intervals. For example, according to the equidistant division rule, it is divided into 10 frequency intervals; the specific number of intervals can be set according to actual accuracy requirements. Further, the number of sampled values ​​falling within each frequency interval is counted, and the ratio of this number to the total number of sampled values ​​corresponding to the system frequency characteristics is calculated as the probability of occurrence of the interval sampling point for each frequency interval. It should be noted that if the system is stable and the system frequency changes little, the sampled values ​​will be concentrated in a few frequency intervals, and the probability of occurrence of sampling points in each interval will be unevenly distributed. If the system is subjected to strong disturbances and the control effect is poor, the system frequency will fluctuate significantly, and the sampled values ​​will be dispersed across multiple frequency intervals, with the probability of occurrence of sampling points in each interval tending to be evenly distributed. To quantify this dispersion, in this embodiment, the information entropy formula is used to calculate the transient system deviation information entropy value. The information entropy formula is:

[0043] in, This represents the entropy value of the transient system deviation information. Indicates the first The probability of sampling points occurring in each frequency range The number of frequency intervals is represented by the transient system deviation entropy value. A larger transient system deviation entropy value indicates a more dispersed system frequency distribution, a more severe deviation from normal operating conditions, and poorer transient stability. Finally, since the range of transient system deviation entropy values ​​depends on the number of frequency intervals, it is difficult to directly compare transient system deviation entropy values ​​under different scenarios. Therefore, in this embodiment, the calculated transient system deviation entropy value is divided by the theoretical maximum transient system deviation entropy value to obtain a transient system deviation index value between 0 and 1, thus completing the transient system deviation entropy normalization process. The theoretical maximum transient system deviation entropy value is the transient system deviation entropy value when the probability of occurrence of all interval sampling points is equal.

[0044] 205. Obtain the sampled values ​​corresponding to the preset transient response characteristics from the time series data of the transient operation phase at the current simulation moment, and calculate the transient vulnerability manifold characteristic index value based on the sampled values ​​corresponding to the preset transient response characteristics.

[0045] The preset transient response characteristics include system frequency characteristics, power angle dynamic characteristics, and voltage transient response characteristics. Power angle dynamic characteristics reflect the relative sway stability between generator rotors. Voltage transient response characteristics describe the depth of voltage drop and recovery speed at nodes after a disturbance.

[0046] Accordingly, step 205 of the embodiment calculates the transient vulnerability manifold feature index value based on the sampled values ​​corresponding to the preset transient response features. Specifically, this includes: constructing a transient high-dimensional sampled value matrix based on the sampled values ​​corresponding to the system frequency features, the power angle dynamic features, and the voltage transient response features; performing transient feature dimension normalization processing on each sampled value in the transient high-dimensional sampled value matrix to obtain a transient high-dimensional sampled value matrix with completed transient feature dimension normalization processing; and for each transient high-dimensional sampled value with completed transient feature dimension normalization processing, using a manifold learning algorithm to... The transient high-dimensional sampled values ​​after transient feature dimension normalization are reduced to a low-dimensional space to obtain the low-dimensional space coordinates corresponding to the current transient high-dimensional sampled values ​​that have completed transient feature dimension normalization. The coordinates of the preset normal cluster center are obtained, and the transient deviation distance between the low-dimensional space coordinates of each transient high-dimensional sampled value that has completed transient feature dimension normalization and the preset normal cluster center coordinates is calculated. The mean of all transient deviation distances is calculated to obtain the transient manifold deviation index value. Based on the maximum transient deviation distance, the transient manifold deviation index value is normalized to obtain the transient vulnerability manifold feature index value.

[0047] To facilitate dimensionality reduction of transient high-dimensional sampled values, the sampled values ​​corresponding to the system frequency characteristics, power angle dynamic characteristics, and voltage transient response characteristics are first aligned according to the sampling time, and a transient high-dimensional sampled value matrix is ​​constructed. Each row of the transient high-dimensional sampled value matrix corresponds to a sampling time, each column corresponds to a transient response feature, and each element in the matrix is ​​the sampled value of that feature at that sampling time. Furthermore, since the dimensions and numerical ranges of different transient response features vary significantly, directly performing dimensionality reduction on the transient high-dimensional sampled values ​​would lead to the large-dimensional transient response features dominating distance calculations. Therefore, it is necessary to normalize the transient feature dimension of each sampled value in the transient high-dimensional sampled value matrix. That is, each feature dimension is normalized independently, which can be done by subtracting the mean and then dividing by the standard deviation, resulting in a transient high-dimensional sampled value matrix with normalized transient feature dimensions. Furthermore, for each transient high-dimensional sampled value that has undergone transient feature dimension normalization, it is mapped to a two-dimensional or three-dimensional low-dimensional space based on a manifold learning algorithm. The low-dimensional space coordinates corresponding to each transient high-dimensional sampled value are output to complete the dimensionality reduction of the transient high-dimensional sampled values. The manifold learning algorithm can employ t-distributed random neighborhood embedding algorithm (t-SNE) or unified manifold approximation and projection algorithm (UMAP), etc. It should be noted that these low-dimensional coordinates preserve the similarity relationships between sampled points in the original high-dimensional space; that is, system states with similar transient behavior will be closer to each other in the low-dimensional space, while fragile states with abnormal behavior will be farther away from the normal region. To quantify this deviation, a normal cluster center coordinate system needs to be pre-configured, i.e., the cluster center coordinate system under normal operating conditions. Data from a stable operating period before the disturbance occurs can be used, and the same normalization and dimensionality reduction parameters can be applied to reduce the dimensionality to a low-dimensional space. The mean of the low-dimensional space coordinates of all normal sampled values ​​is calculated as the preset normal cluster center coordinate system. The Euclidean distance between the low-dimensional space coordinates of each transient high-dimensional sampled value that has undergone transient feature dimension normalization and the preset normal cluster center coordinate system is calculated one by one to obtain multiple transient deviation distances. Since the entire transient operation phase contains multiple sampling points, in order to further obtain the overall manifold deviation, the mean of all the aforementioned transient deviation distances is calculated to obtain the transient manifold deviation index value. The larger the transient manifold deviation index value, the further the system deviates from the normal state during the transient operation phase, and the higher its vulnerability.Finally, since the range of transient manifold deviation index values ​​is affected by the dimensionality reduction method and data distribution, it is difficult to directly compare between different adversarial rounds. Based on this, in this embodiment, the transient manifold deviation index value is divided by the maximum transient deviation distance among all sampling points in the transient phase to obtain a transient vulnerability manifold characteristic index value between 0 and 1, so as to complete the normalization processing of the transient manifold deviation index. It should be noted that the closer the transient vulnerability manifold characteristic index value is to 1, the more the state distribution of the system deviates from the normal operating manifold in the transient operation phase, and the more significant the potential vulnerability.

[0048] 206. Calculate the transient robustness fusion weights based on the disturbance intensity of the simulated disturbance events, and then perform weighted fusion of the transient system deviation index value and the transient vulnerability manifold characteristic index value based on the transient robustness fusion weights to obtain the transient robustness score.

[0049] The formula for calculating the transient robustness fusion weight is as follows: , , α represents the transient robustness fusion weight corresponding to the transient system deviation index value, and β represents the transient robustness fusion weight corresponding to the transient vulnerability manifold characteristic index value. This represents the weighting adjustment coefficient. This indicates the intensity of the simulated disturbance event.

[0050] Transient robustness score = α × transient system bias index value + β × transient vulnerability manifold characteristic index value.

[0051] 207. For the steady-state operation phase, obtain the sampled values ​​corresponding to the preset steady-state stability key features from the time series data of the steady-state operation phase at the current simulation time, and calculate the steady-state system deviation index value based on the sampled values ​​corresponding to the preset steady-state stability key features.

[0052] Among them, the key feature of the preset steady-state stability is the node voltage amplitude feature.

[0053] Accordingly, step 207 of the embodiment, which calculates the steady-state system deviation index value based on the sampled values ​​corresponding to the preset key steady-state stability features, specifically includes: arranging the sampled values ​​corresponding to the node voltage amplitude features in ascending order to obtain a sequence of node voltage amplitude feature sampled values; dividing the sequence of node voltage amplitude feature sampled values ​​into multiple consecutive node voltage amplitude intervals according to a preset interval division rule; for each node voltage amplitude interval, counting the number of sampled values ​​within the current node voltage amplitude interval, and calculating the ratio of the number to the number of sampled values ​​corresponding to the node voltage amplitude features to obtain the probability of occurrence of the interval sampling point corresponding to the current node voltage amplitude interval; calculating the steady-state system deviation information entropy value based on the probability of occurrence of the interval sampling point in each node voltage amplitude interval; and performing steady-state system deviation information entropy normalization processing on the steady-state system deviation information entropy value based on the theoretical maximum steady-state system deviation information entropy value to obtain the steady-state system deviation index value.

[0054] 208. Obtain the sampled values ​​corresponding to the preset steady-state response characteristics from the time series data of the steady-state operation phase at the current simulation moment, and calculate the steady-state vulnerability manifold characteristic index value based on the sampled values ​​corresponding to the preset steady-state response characteristics.

[0055] The preset steady-state response characteristics include node voltage amplitude characteristics, line active power flow characteristics, and unit active power output characteristics.

[0056] Accordingly, step 208 of the embodiment calculates the steady-state vulnerability manifold characteristic index value based on the sampled values ​​corresponding to the preset steady-state response characteristics. Specifically, this includes: constructing a steady-state high-dimensional sampled value matrix based on the sampled values ​​corresponding to the node voltage amplitude characteristics, the line active power flow characteristics, and the unit active power output characteristics; performing steady-state feature dimension normalization processing on each sampled value in the steady-state high-dimensional sampled value matrix to obtain a steady-state high-dimensional sampled value matrix with completed steady-state feature dimension normalization processing; and for each steady-state high-dimensional sampled value with completed steady-state feature dimension normalization processing, using a manifold learning algorithm to... The steady-state high-dimensional sampled values ​​that have undergone steady-state feature dimension normalization are reduced to a low-dimensional space to obtain the low-dimensional space coordinates corresponding to the current steady-state high-dimensional sampled value that has undergone steady-state feature dimension normalization. The coordinates of the preset normal cluster center are obtained, and the steady-state deviation distance between the low-dimensional space coordinates of each steady-state high-dimensional sampled value that has undergone steady-state feature dimension normalization and the preset normal cluster center coordinates is calculated. The mean of all steady-state deviation distances is calculated to obtain the steady-state manifold deviation index value. Based on the maximum steady-state deviation distance, the steady-state manifold deviation index value is normalized to obtain the steady-state vulnerability manifold feature index value.

[0057] 209. Calculate the steady-state robustness fusion weight based on the disturbance intensity of the simulated disturbance event, and perform weighted fusion of the steady-state system deviation index value and the steady-state vulnerability manifold characteristic index value based on the steady-state robustness fusion weight to obtain the steady-state robustness score.

[0058] The calculation process for steady-state robustness score in steps 207-209 of the embodiment is similar to the calculation process for transient robustness score in steps 204-206 of the aforementioned embodiment. For details, please refer to the detailed description in steps 204-206 of the embodiment, which will not be repeated here.

[0059] 210. Based on the preset comprehensive robustness fusion weights, the transient robustness score and the steady-state robustness score are weighted and fused to obtain the comprehensive robustness score at the current simulation time.

[0060] The preset comprehensive robustness fusion weights can be pre-configured according to actual needs. For example, if more attention is paid to the emergency response capability of the agent generated by the control action, the fusion weights corresponding to the transient robustness score can be configured to be larger, and the fusion weights corresponding to the steady-state robustness score can be configured to be smaller. If both are equally important, the two fusion weights can be configured to 0.5 respectively.

[0061] The formula for calculating the overall robustness score is as follows:

[0062] Indicates the overall robustness score. This represents the fusion weight corresponding to the transient robustness score. Indicates transient robustness score, This represents the fusion weight corresponding to the steady-state robustness score. Indicates the steady-state robustness score. .

[0063] 211. Perform attribution analysis on the set of response feature sample values ​​to generate the set of key physical variables that have the greatest impact on the overall robustness score.

[0064] As an optional implementation, step 211 of the embodiment specifically includes: obtaining the sampled values ​​corresponding to the preset transient response features and the sampled values ​​corresponding to the preset steady-state response features, and combining them to obtain a set of response feature sampled values; for each response feature sampled value in the set of response feature sampled values, applying a small perturbation to the current response feature sampled value, while keeping other response feature sampled values ​​unchanged, to obtain a set of perturbed response feature sampled values, and calculating the comprehensive robustness score after perturbation based on the set of perturbed response feature sampled values, and calculating the difference between the comprehensive robustness score after perturbation and the comprehensive robustness score at the current simulation time, as the robustness score contribution of the current response feature corresponding to the current response feature sampled value; arranging each response feature in descending order according to the robustness score contribution to obtain a response feature sequence, and selecting a preset number of response features sequentially from the first position of the response feature sequence to generate a set of key physical variables, and generating a learning reward value based on the set of key physical variables and the comprehensive robustness score at the current simulation time.

[0065] This involves applying small perturbations, such as increasing or decreasing a fixed absolute value or a fixed proportion, or standardizing the perturbation based on the variable's operating range. The robustness score contribution quantifies the influence of the response feature on the overall robustness score; a larger contribution indicates a more significant impact of the response feature on the robustness of the control action generating agent. The set of key physical variables records the main physical factors that lead to the vulnerability of the control action generating agent.

[0066] 212. Use the comprehensive robustness score at the current simulation moment as a learning reward to update the policy parameters of the agent that generates the control actions.

[0067] In this embodiment of the application, when generating the learning reward value, the set of key physical variables and the comprehensive robustness score at the current simulation moment can be combined to generate a more refined learning reward value. For example, a penalty term for key physical variables can be added to the reward function to guide the control action generating agent to focus on these vulnerable links during the training process, thereby improving the overall robustness of the strategy under adversarial perturbations.

[0068] 213. Repeat the simulation disturbance event generation, comprehensive robustness score calculation and policy parameter update until the comprehensive robustness score of the control action generation agent meets the preset convergence condition, and obtain the control action generation agent with completed model training, and generate control action sequence based on the control action generation agent with completed model training.

[0069] Among them, the preset convergence conditions may be that the comprehensive robustness score of multiple consecutive training rounds is higher than the preset comprehensive robustness score threshold, the improvement between the comprehensive robustness scores of two adjacent training rounds is less than the preset comprehensive robustness improvement threshold, or the maximum number of training rounds is reached.

[0070] This application provides a method for generating control actions in a power system. First, during the operation of the target power grid, disturbance events are monitored in real time, and the current-moment grid state time-series data is acquired when a disturbance event is detected. Second, a control action generation agent, based on a pre-trained model, generates a control action sequence for the current moment according to the current-moment grid state time-series data. The control action generation agent is constructed based on an adversarial network, and the model is trained using the comprehensive robustness score of the control action generation agent as a learning reward. This comprehensive robustness score is used to evaluate the stability and security of the control action generation agent under different operating stages after a disturbance. Finally, the target power grid is controlled according to the current-moment control action sequence to respond to the disturbance event. Compared with existing technologies, the embodiments of this application construct a control action generation agent based on adversarial networks and use a comprehensive robustness score as a reinforcement learning reward for model training. This enables the control action generation agent to actively learn to cope with complex adversarial disturbance scenarios during training. Furthermore, the comprehensive robustness score comprehensively evaluates the stability and security of the control action generation agent in different operating stages after disturbance, avoiding the one-sidedness caused by relying on only a single type of operating indicator, and can comprehensively reflect the overall stability of control actions under complex disturbance conditions. At the same time, since the comprehensive scoring mechanism can quantify the vulnerability of the control action generation agent in extreme operating scenarios and guide the agent to optimize strategy parameters through reward signals, it effectively improves the adaptability of the control action generation agent to the rapidly changing operating state of the power grid, ensuring the robustness and reliability of the generated control actions under various disturbance events.

[0071] Furthermore, as a response to the above Figure 1 The implementation of the method shown in this application provides a power system control action generation device, such as... Figure 3 As shown, the device includes: Disturbance event monitoring module 31, control action sequence generation module 32, control action sequence execution module 33; The disturbance event monitoring module 31 is used to monitor disturbance events in real time during the operation of the target power grid, and to obtain the current time-series data of the power grid status of the target power grid when a disturbance event is detected. The control action sequence generation module 32 is used to generate a control action generation agent based on the completed model training, and generate a control action sequence at the current time according to the current power grid state time series data. The control action generation agent is constructed based on an adversarial network and is obtained by training the model with the comprehensive robustness score of the control action generation agent as the learning reward. The comprehensive robustness score is used to evaluate the stability and safety of the control action generation agent under different operating stages after disturbance. The control action sequence execution module 33 is used to control the target power grid according to the control action sequence at the current time in order to respond to the disturbance event.

[0072] In specific application scenarios, before the control action sequence generation module, the device further includes a control action generation intelligent agent training module, including: The intelligent agent construction unit is used to construct a power system adversarial simulation environment that matches the target power grid, and to construct a disturbance generation intelligent agent and a control action generation intelligent agent in the power system adversarial simulation environment; The adversarial simulation unit is used to control the disturbance generation agent to generate simulated disturbance events in the power system adversarial simulation environment, and to obtain the current simulation time-series data of the power grid state after the occurrence of the simulated disturbance event. Based on the control action generation agent, it generates the current simulation time control action sequence according to the current simulation time-series data of the power grid state. The operation phase division unit is used to divide the current simulation time-series data of the power grid state into the current simulation time-transient operation phase time-series data and the current simulation time-steady-state operation phase time-series data based on the preset operation phase division rules. The transient robustness scoring calculation unit is used to, for the transient operation phase, obtain sampled values ​​corresponding to preset transient stability key features from the time series data of the transient operation phase at the current simulation time, calculate transient system deviation index values ​​based on the sampled values ​​corresponding to the preset transient stability key features, obtain sampled values ​​corresponding to preset transient response features from the time series data of the transient operation phase at the current simulation time, calculate transient vulnerability manifold feature index values ​​based on the sampled values ​​corresponding to the preset transient response features, calculate transient robustness fusion weights according to the disturbance intensity of the simulated disturbance event, and perform weighted fusion of the transient system deviation index values ​​and the transient vulnerability manifold feature index values ​​based on the transient robustness fusion weights to obtain a transient robustness score; The steady-state robustness scoring unit is used to, for the steady-state operation phase, obtain sampled values ​​corresponding to preset steady-state stability key features from the time series data of the steady-state operation phase at the current simulation time, calculate steady-state system deviation index values ​​based on the sampled values ​​corresponding to the preset steady-state stability key features, obtain sampled values ​​corresponding to preset steady-state response features from the time series data of the steady-state operation phase at the current simulation time, calculate steady-state vulnerability manifold feature index values ​​based on the sampled values ​​corresponding to the preset steady-state response features, calculate steady-state robustness fusion weights according to the disturbance intensity of the simulated disturbance event, and perform weighted fusion of the steady-state system deviation index values ​​and the steady-state vulnerability manifold feature index values ​​based on the steady-state robustness fusion weights to obtain a steady-state robustness score; The comprehensive robustness score calculation unit is used to perform weighted fusion of the transient robustness score and the steady-state robustness score based on a preset comprehensive robustness fusion weight to obtain the comprehensive robustness score at the current simulation time. The agent policy parameter update unit is used to update the policy parameters of the agent that generates the control action by taking the comprehensive robustness score at the current simulation moment as a learning reward. The loop training unit is used to repeatedly generate simulated disturbance events, calculate the comprehensive robustness score, and update the policy parameters until the comprehensive robustness score of the control action generation agent meets the preset convergence condition, thus obtaining a control action generation agent with completed model training, and generating a control action sequence based on the control action generation agent with completed model training.

[0073] In specific application scenarios, before the agent policy parameter update unit, the control action generation agent training module further includes an attribution analysis unit, used for: Obtain the sampled values ​​corresponding to the preset transient response features and the sampled values ​​corresponding to the preset steady-state response features, and combine them to obtain a set of response feature sampled values; For each response feature sample value in the set of response feature sample values, a small perturbation is applied to the current response feature sample value, while other response feature sample values ​​remain unchanged, resulting in a perturbed set of response feature sample values. Based on the perturbed set of response feature sample values, a perturbed comprehensive robustness score is calculated, and the difference between the perturbed comprehensive robustness score and the comprehensive robustness score at the current simulation time is calculated as the contribution of the robustness score of the current response feature corresponding to the current response feature sample value. Each response feature is sorted in descending order of its contribution to the robustness score to obtain a response feature sequence. Starting from the first position of the response feature sequence, a preset number of response features are selected sequentially to generate a set of key physical variables. A learning reward value is then generated based on the set of key physical variables and the comprehensive robustness score at the current simulation moment.

[0074] In specific application scenarios, the preset key feature of transient stability is the system frequency feature, and the transient robustness scoring calculation unit is used for: Arrange the sampled values ​​corresponding to the system frequency characteristics in ascending order to obtain the system frequency characteristic sampled value sequence; According to the preset interval division rules, the system frequency feature sample value sequence is divided into multiple consecutive frequency intervals; For each frequency interval, the number of sampled values ​​within the current frequency interval is counted, and the ratio of the number to the number of sampled values ​​corresponding to the system frequency characteristics is calculated to obtain the probability of the occurrence of the interval sampling point corresponding to the current frequency interval. The transient system deviation information entropy value is calculated based on the probability of occurrence of interval sampling points in each frequency interval; Based on the theoretical maximum transient system deviation information entropy value, the transient system deviation information entropy value is normalized to obtain the transient system deviation index value.

[0075] In specific application scenarios, the preset transient response characteristics include system frequency characteristics, power angle dynamic characteristics, and voltage transient response characteristics. The transient robustness scoring calculation unit is also used for: Based on the sampled values ​​corresponding to the system frequency characteristics, the sampled values ​​corresponding to the power angle dynamic characteristics, and the sampled values ​​corresponding to the voltage transient response characteristics, a transient high-dimensional sampled value matrix is ​​constructed; Each sample value in the transient high-dimensional sample value matrix is ​​normalized for its transient feature dimension to obtain a transient high-dimensional sample value matrix with normalized transient feature dimension. For each transient high-dimensional sample value that has completed transient feature dimension normalization, the current transient high-dimensional sample value that has completed transient feature dimension normalization is reduced to a low-dimensional space based on the manifold learning algorithm, so as to obtain the low-dimensional space coordinates corresponding to the current transient high-dimensional sample value that has completed transient feature dimension normalization. Obtain the preset normal cluster center coordinates, calculate the transient deviation distance between the low-dimensional spatial coordinates of each transient high-dimensional sample value that has completed transient feature dimension normalization and the preset normal cluster center coordinates, and calculate the mean of all the transient deviation distances to obtain the transient manifold deviation index value. Based on the maximum transient deviation distance, the transient manifold deviation index value is normalized to obtain the transient vulnerability manifold characteristic index value.

[0076] In specific application scenarios, the preset key feature of steady-state stability is the node voltage amplitude characteristic, and the steady-state robustness scoring calculation unit is used for: Arrange the sampled values ​​corresponding to the node voltage amplitude features in ascending order to obtain the node voltage amplitude feature sampled value sequence; According to the preset interval division rules, the node voltage amplitude feature sampling value sequence is divided into multiple consecutive node voltage amplitude intervals; For each node voltage amplitude range, the number of sampled values ​​within the current node voltage amplitude range is counted, and the ratio of this number to the number of sampled values ​​corresponding to the node voltage amplitude feature is calculated to obtain the probability of the occurrence of the interval sampling point corresponding to the current node voltage amplitude range. The steady-state system deviation information entropy value is calculated based on the probability of occurrence of interval sampling points in the voltage amplitude range of each node; Based on the theoretical maximum steady-state system deviation information entropy value, the steady-state system deviation information entropy value is normalized to obtain the steady-state system deviation index value.

[0077] In specific application scenarios, the preset steady-state response characteristics include node voltage amplitude characteristics, line active power flow characteristics, and generator active power output characteristics. The steady-state robustness scoring calculation unit is also used for: Based on the sampled values ​​corresponding to the node voltage amplitude characteristics, the sampled values ​​corresponding to the line active power flow characteristics, and the sampled values ​​corresponding to the unit active power output characteristics, a steady-state high-dimensional sampled value matrix is ​​constructed. Each sample value in the steady-state high-dimensional sample value matrix is ​​normalized to its steady-state feature dimension to obtain a steady-state high-dimensional sample value matrix with normalized steady-state feature dimension. For each steady-state high-dimensional sample value that has completed steady-state feature dimension normalization, the current steady-state high-dimensional sample value that has completed steady-state feature dimension normalization is reduced to a low-dimensional space based on the manifold learning algorithm, so as to obtain the low-dimensional space coordinates corresponding to the current steady-state high-dimensional sample value that has completed steady-state feature dimension normalization. Obtain the preset normal cluster center coordinates, calculate the steady-state deviation distance between the low-dimensional spatial coordinates of each steady-state high-dimensional sample value that has completed steady-state feature dimension normalization and the preset normal cluster center coordinates, and calculate the mean of all the steady-state deviation distances to obtain the steady-state manifold deviation index value. Based on the maximum steady-state deviation distance, the steady-state manifold deviation index value is normalized to obtain the steady-state vulnerability manifold characteristic index value.

[0078] This application provides a device for generating control actions in a power system. First, during the operation of a target power grid, disturbance events are monitored in real time, and the current state time-series data of the target power grid is acquired upon detection of a disturbance event. Second, a control action generation agent, based on a pre-trained model, generates a sequence of control actions for the current moment according to the current state time-series data of the power grid. This control action generation agent is constructed based on an adversarial network, and the model is trained using a comprehensive robustness score as a learning reward. This comprehensive robustness score is used to evaluate the stability and security of the control action generation agent under different operating stages after a disturbance. Finally, the target power grid is controlled according to the current control action sequence to respond to the disturbance event. Compared with existing technologies, the embodiments of this application construct a control action generation agent based on adversarial networks and use a comprehensive robustness score as a reinforcement learning reward for model training. This enables the control action generation agent to actively learn to cope with complex adversarial disturbance scenarios during training. Furthermore, the comprehensive robustness score comprehensively evaluates the stability and security of the control action generation agent in different operating stages after disturbance, avoiding the one-sidedness caused by relying on only a single type of operating indicator, and can comprehensively reflect the overall stability of control actions under complex disturbance conditions. At the same time, since the comprehensive scoring mechanism can quantify the vulnerability of the control action generation agent in extreme operating scenarios and guide the agent to optimize strategy parameters through reward signals, it effectively improves the adaptability of the control action generation agent to the rapidly changing operating state of the power grid, ensuring the robustness and reliability of the generated control actions under various disturbance events.

[0079] According to one embodiment of this application, a storage medium is provided, the storage medium storing at least one executable instruction, which can execute the power system control action generation method in any of the above method embodiments.

[0080] Based on this understanding, the technical solution of this application can be embodied in the form of a software product. The software product can be stored in a non-volatile storage medium (such as a CD-ROM, USB flash drive, or portable hard drive), and includes several instructions to cause a computer device (such as a personal computer, server, or network device) to execute the methods described in the various implementation scenarios of this application.

[0081] Figure 4 The diagram shows a structural schematic of a terminal according to one embodiment of the present application. The specific embodiments of the present application do not limit the specific implementation of the terminal.

[0082] like Figure 4 As shown, the terminal may include: a processor 402, a communications interface 404, a memory 406, and a communications bus 408.

[0083] The processor 402, communication interface 404, and memory 406 communicate with each other via communication bus 408.

[0084] Communication interface 404 is used to communicate with other network elements such as clients or other servers.

[0085] The processor 402 is used to execute program 410, specifically to execute the relevant steps in the above-described embodiment of the method for generating power system control actions.

[0086] Specifically, program 410 may include program code that includes computer operation instructions.

[0087] Processor 402 may be a central processing unit (CPU), an application-specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of this application. The computer device includes one or more processors, which may be processors of the same type, such as one or more CPUs; or they may be processors of different types, such as one or more CPUs and one or more ASICs.

[0088] Memory 406 is used to store program 410. Memory 406 may include high-speed RAM memory, and may also include non-volatile memory, such as at least one disk storage device.

[0089] Specifically, program 410 can be used to cause processor 402 to perform the following operations: During the operation of the target power grid, disturbance events are monitored in real time, and when a disturbance event is detected, the current time-series data of the target power grid state are obtained. Based on the control action generation agent that has completed model training, the control action sequence at the current moment is generated according to the current power grid state time series data. The control action generation agent is built on an adversarial network and is obtained by training the model with the comprehensive robustness score of the control action generation agent as the learning reward. The comprehensive robustness score is used to evaluate the stability and safety of the control action generation agent under different operating stages after disturbance. The target power grid is controlled according to the current time control action sequence to respond to the disturbance event.

[0090] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For system embodiments, since they largely correspond to method embodiments, the description is relatively simple; relevant parts can be referred to the descriptions in the method embodiments.

[0091] The above description is merely a preferred embodiment of this application and is not intended to limit this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of protection of this application.

Claims

1. A method for generating control actions in a power system, characterized in that, include: During the operation of the target power grid, disturbance events are monitored in real time, and when a disturbance event is detected, the current time-series data of the target power grid state are obtained. Based on the control action generation agent that has completed model training, the control action sequence at the current moment is generated according to the current power grid state time series data. The control action generation agent is built on an adversarial network and is obtained by training the model with the comprehensive robustness score of the control action generation agent as the learning reward. The comprehensive robustness score is used to evaluate the stability and safety of the control action generation agent under different operating stages after disturbance. The target power grid is controlled according to the current time control action sequence to respond to the disturbance event.

2. The method according to claim 1, characterized in that, Before generating the control action sequence for the current moment based on the current power grid state time-series data, the method further includes: Construct a power system adversarial simulation environment that matches the target power grid, and construct a disturbance generation agent and a control action generation agent in the power system adversarial simulation environment; In the power system adversarial simulation environment, the disturbance generation agent is controlled to generate simulated disturbance events, and the current simulation time-series data of the power grid state after the occurrence of the simulated disturbance events is obtained. Based on the control action generation agent, the current simulation time control action sequence is generated according to the current simulation time-series data of the power grid state. Based on the preset operation phase division rules, the current simulation time-series data of the power grid state is divided into the current simulation time-transient operation phase time-series data and the current simulation time-steady-state operation phase time-series data. For the transient operation phase, sampled values ​​corresponding to preset transient stability key features are obtained from the time series data of the transient operation phase at the current simulation time. Transient system deviation index value is calculated based on the sampled values ​​corresponding to the preset transient stability key features. Sampled values ​​corresponding to preset transient response features are obtained from the time series data of the transient operation phase at the current simulation time. Transient vulnerability manifold feature index value is calculated based on the sampled values ​​corresponding to the preset transient response features. Transient robustness fusion weight is calculated according to the disturbance intensity of the simulated disturbance event. The transient system deviation index value and the transient vulnerability manifold feature index value are weighted and fused based on the transient robustness fusion weight to obtain a transient robustness score. For the steady-state operation phase, sampled values ​​corresponding to preset steady-state stability key features are obtained from the time series data of the steady-state operation phase at the current simulation time. Based on the sampled values ​​corresponding to the preset steady-state stability key features, a steady-state system deviation index value is calculated. Also, sampled values ​​corresponding to preset steady-state response features are obtained from the time series data of the steady-state operation phase at the current simulation time. Based on the sampled values ​​corresponding to the preset steady-state response features, a steady-state vulnerability manifold feature index value is calculated. Furthermore, a steady-state robustness fusion weight is calculated based on the disturbance intensity of the simulated disturbance event. The steady-state system deviation index value and the steady-state vulnerability manifold feature index value are then weighted and fused based on the steady-state robustness fusion weight to obtain a steady-state robustness score. Based on the preset comprehensive robustness fusion weights, the transient robustness score and the steady-state robustness score are weighted and fused to obtain the comprehensive robustness score at the current simulation time. The comprehensive robustness score at the current simulation moment is used as a learning reward to update the policy parameters of the control action generating agent; Repeatedly generate simulation disturbance events, calculate the comprehensive robustness score, and update the policy parameters until the comprehensive robustness score of the control action generation agent meets the preset convergence condition, thus obtaining a control action generation agent with completed model training, and generating a control action sequence based on the control action generation agent with completed model training.

3. The method according to claim 2, characterized in that, Before updating the policy parameters of the control action generating agent by using the comprehensive robustness score at the current simulation moment as a learning reward, the method further includes: Obtain the sampled values ​​corresponding to the preset transient response features and the sampled values ​​corresponding to the preset steady-state response features, and combine them to obtain a set of response feature sampled values; For each response feature sample value in the set of response feature sample values, a small perturbation is applied to the current response feature sample value, while other response feature sample values ​​remain unchanged, resulting in a perturbed set of response feature sample values. Based on the perturbed set of response feature sample values, a perturbed comprehensive robustness score is calculated, and the difference between the perturbed comprehensive robustness score and the comprehensive robustness score at the current simulation time is calculated as the contribution of the robustness score of the current response feature corresponding to the current response feature sample value. Each response feature is sorted in descending order of its contribution to the robustness score to obtain a response feature sequence. Starting from the first position of the response feature sequence, a preset number of response features are selected sequentially to generate a set of key physical variables. A learning reward value is then generated based on the set of key physical variables and the comprehensive robustness score at the current simulation moment.

4. The method according to claim 2, characterized in that, The preset transient stability key feature is the system frequency feature, and the calculation of the transient system deviation index value based on the sampled values ​​corresponding to the preset transient stability key feature includes: Arrange the sampled values ​​corresponding to the system frequency characteristics in ascending order to obtain the system frequency characteristic sampled value sequence; According to the preset interval division rules, the system frequency feature sample value sequence is divided into multiple consecutive frequency intervals; For each frequency interval, the number of sampled values ​​within the current frequency interval is counted, and the ratio of the number to the number of sampled values ​​corresponding to the system frequency characteristics is calculated to obtain the probability of the occurrence of the interval sampling point corresponding to the current frequency interval. The transient system deviation information entropy value is calculated based on the probability of occurrence of interval sampling points in each frequency interval; Based on the theoretical maximum transient system deviation information entropy value, the transient system deviation information entropy value is normalized to obtain the transient system deviation index value.

5. The method according to claim 2, characterized in that, The preset transient response characteristics include system frequency characteristics, power angle dynamic characteristics, and voltage transient response characteristics. The calculation of transient vulnerability manifold characteristic index values ​​based on the sampled values ​​corresponding to the preset transient response characteristics includes: Based on the sampled values ​​corresponding to the system frequency characteristics, the sampled values ​​corresponding to the power angle dynamic characteristics, and the sampled values ​​corresponding to the voltage transient response characteristics, a transient high-dimensional sampled value matrix is ​​constructed; Each sample value in the transient high-dimensional sample value matrix is ​​normalized for its transient feature dimension to obtain a transient high-dimensional sample value matrix with normalized transient feature dimension. For each transient high-dimensional sample value that has completed transient feature dimension normalization, the current transient high-dimensional sample value that has completed transient feature dimension normalization is reduced to a low-dimensional space based on the manifold learning algorithm, so as to obtain the low-dimensional space coordinates corresponding to the current transient high-dimensional sample value that has completed transient feature dimension normalization. Obtain the preset normal cluster center coordinates, calculate the transient deviation distance between the low-dimensional spatial coordinates of each transient high-dimensional sample value that has completed transient feature dimension normalization and the preset normal cluster center coordinates, and calculate the mean of all the transient deviation distances to obtain the transient manifold deviation index value. Based on the maximum transient deviation distance, the transient manifold deviation index value is normalized to obtain the transient vulnerability manifold characteristic index value.

6. The method according to claim 2, characterized in that, The preset steady-state stability key feature is the node voltage amplitude feature. The calculation of the steady-state system deviation index value based on the sampled values ​​corresponding to the preset steady-state stability key feature includes: Arrange the sampled values ​​corresponding to the node voltage amplitude features in ascending order to obtain the node voltage amplitude feature sampled value sequence; According to the preset interval division rules, the node voltage amplitude feature sampling value sequence is divided into multiple consecutive node voltage amplitude intervals; For each node voltage amplitude range, the number of sampled values ​​within the current node voltage amplitude range is counted, and the ratio of this number to the number of sampled values ​​corresponding to the node voltage amplitude feature is calculated to obtain the probability of the occurrence of the interval sampling point corresponding to the current node voltage amplitude range. The steady-state system deviation information entropy value is calculated based on the probability of occurrence of interval sampling points in the voltage amplitude range of each node; Based on the theoretical maximum steady-state system deviation information entropy value, the steady-state system deviation information entropy value is normalized to obtain the steady-state system deviation index value.

7. The method according to claim 2, characterized in that, The preset steady-state response characteristics include node voltage amplitude characteristics, line active power flow characteristics, and generator active power output characteristics. The calculation of steady-state vulnerability manifold characteristic index values ​​based on the sampled values ​​corresponding to the preset steady-state response characteristics includes: Based on the sampled values ​​corresponding to the node voltage amplitude characteristics, the sampled values ​​corresponding to the line active power flow characteristics, and the sampled values ​​corresponding to the unit active power output characteristics, a steady-state high-dimensional sampled value matrix is ​​constructed. Each sample value in the steady-state high-dimensional sample value matrix is ​​normalized to its steady-state feature dimension to obtain a steady-state high-dimensional sample value matrix with normalized steady-state feature dimension. For each steady-state high-dimensional sample value that has completed steady-state feature dimension normalization, the current steady-state high-dimensional sample value that has completed steady-state feature dimension normalization is reduced to a low-dimensional space based on the manifold learning algorithm, so as to obtain the low-dimensional space coordinates corresponding to the current steady-state high-dimensional sample value that has completed steady-state feature dimension normalization. Obtain the preset normal cluster center coordinates, calculate the steady-state deviation distance between the low-dimensional spatial coordinates of each steady-state high-dimensional sample value that has completed steady-state feature dimension normalization and the preset normal cluster center coordinates, and calculate the mean of all the steady-state deviation distances to obtain the steady-state manifold deviation index value. Based on the maximum steady-state deviation distance, the steady-state manifold deviation index value is normalized to obtain the steady-state vulnerability manifold characteristic index value.

8. A device for generating control actions in a power system, characterized in that, include: The disturbance event monitoring module is used to monitor disturbance events in real time during the operation of the target power grid, and to obtain the current time-series data of the power grid status of the target power grid when a disturbance event is detected. The control action sequence generation module is used to generate a control action generation agent based on the completed model training. According to the current power grid state time series data, the control action generation agent generates a control action sequence at the current time. The control action generation agent is built based on an adversarial network and is obtained by training the model with the comprehensive robustness score of the control action generation agent as the learning reward. The comprehensive robustness score is used to evaluate the stability and safety of the control action generation agent under different operating stages after disturbance. The control action sequence execution module is used to control the target power grid according to the control action sequence at the current time in order to respond to the disturbance event.

9. A storage medium storing at least one executable instruction, characterized in that, The executable instructions cause the processor to perform the operation corresponding to the power system control action generation method as described in any one of claims 1-7.

10. A terminal, comprising: The processor, memory, communication interface, and communication bus are provided, wherein the processor, memory, and communication interface communicate with each other via the communication bus. The memory is used to store at least one executable instruction, characterized in that the executable instruction causes the processor to perform the operation corresponding to the power system control action generation method as described in any one of claims 1-7.

Images