Verification test method for increasing output of unit and improving emergency guarantee capability of power grid

Abstract

The invention relates to the field of power systems, and discloses a verification test method for increasing output of a unit and improving emergency guarantee capability of a power grid, which comprises the following steps: initializing a unit simulation environment after loading a unit model; obtaining a plurality of expert strategies by training a reinforcement learning model based on a unit simulation environment; a plurality of expert tracks are sampled through interaction between an expert strategy and a unit simulation environment; constructing a generator and discriminators of which the number is the same as that of the expert strategies; in the training process, a simulator track is sampled through interaction between a generator and a unit simulation environment; respectively training a discriminator with each expert track by utilizing the simulator track, calculating a reward value according to a discriminant value output by each discriminator, and training a generator by adopting a PPO algorithm according to the simulator track and the reward value; and obtaining a verification test model for increasing the output of the unit according to the trained generator. The output of the unit and the emergency response capability of the power grid can be improved, and the operation stability and the performance improvement are ensured.

Description

Technical Field

[0001] The invention relates to the field of electric power systems, and in particular to a verification test method for increasing the output of a unit to improve the emergency protection capability of a power grid. Background Art

[0002] In the power system, the output capacity of the unit is crucial to the stable operation and emergency response of the power grid. The traditional unit output verification method relies on actual operation data and experience judgment, which is not only inefficient but also difficult to simulate the performance of the unit under extreme conditions. With the expansion of the scale and complexity of the power system, the traditional verification method can no longer meet the needs of modern power grids for fast and accurate evaluation of unit output. In addition, when simulating unit behavior, existing technologies often ignore the dynamic interaction between the unit and the power grid, resulting in deviations between the verification results and the actual operation conditions.

[0003] In the process of implementing the embodiments of the present invention, the inventors found that there are at least the following problems or defects in the prior art: the existing unit output verification method lacks a comprehensive simulation of the unit's behavior under complex power grid conditions, and it is difficult to accurately evaluate the unit's output capacity under various operating conditions and the power grid's emergency response capability; at the same time, the existing method fails to fully utilize advanced machine learning technology to improve the accuracy and efficiency of verification when simulating unit behavior. Summary of the invention

[0004] The embodiment of the present invention aims to provide a verification test method for increasing the output of a unit to improve the emergency protection capability of a power grid, so as to solve the technical problems raised in the prior art.

[0005] The embodiment of the present invention solves the technical problem by adopting the following technical solution:

[0006] In the first aspect, a verification test method for increasing the output of a unit to improve the emergency protection capability of a power grid is provided, comprising:

[0007] After loading the unit model, initialize the unit simulation environment;

[0008] Based on the unit simulation environment, multiple expert strategies are obtained by training the reinforcement learning model; multiple expert trajectories are sampled through the interaction between the expert strategy and the unit simulation environment;

[0009] Construct a generator and the same number of discriminators as the expert strategy; during the training process, the generator interacts with the unit simulation environment to sample the imitator trajectory; use the imitator trajectory to train a discriminator with each expert trajectory respectively, calculate the reward value based on the discriminant value output by each discriminator, and use the PPO algorithm to train the generator based on the imitator trajectory and the reward value;

[0010] The verification test model for increasing the unit output is obtained based on the trained generator.

[0011] Furthermore, the unit simulation environment is obtained by building a power system simulation platform and introducing a simulation library; the unit model is an XML format file, which is used to define the composition and connection relationship of the unit, wherein the unit is composed of a steam turbine, a generator and a control system, the steam turbine and the generator are connected through an axis, and the control system is connected to the steam turbine and the generator through signals.

[0012] Furthermore, the initialization of the unit simulation environment includes: defining a state space, an action space and a reward function based on the unit's operating scenario; and initializing the unit's state space when the simulation environment is reset.

[0013] Furthermore, the state space includes: the output, rotation speed, temperature, pressure, fuel flow, and steam flow of the unit; the action space includes: fuel supply, steam valve opening, and cooling water flow.

[0014] Furthermore, the interaction with the unit simulation environment through expert strategies and the sampling of multiple expert trajectories are based on the same task, and the expert strategies are used to interact with the unit simulation environment respectively. The state space input and the action space output in each interaction are taken as a state-action pair. After the same number of interactions, the state-action pair sequence obtained is taken as the expert trajectory corresponding to the expert strategy; the imitator trajectory is the state-action pair sequence generated by the generator interacting with the unit simulation environment multiple times.

[0015] Furthermore, the method of using the imitator trajectory and each expert trajectory to train a discriminator is to concatenate the state-action pairs in the imitator trajectory with the state-action pairs in each expert trajectory and pass them into a discriminator, and each discriminator outputs the discrimination values ​​of the two state-action pairs; and the gradient of the corresponding discriminator is calculated according to the discrimination values ​​of the two state-action pairs to update the discriminator parameters.

[0016] Further, the calculation of the reward value according to the discriminant value output by each discriminator includes:

[0017] According to the discriminant value output by each discriminator, the probability that the discriminator is correct is calculated, and the expert accuracy and imitator accuracy of each discriminator are calculated respectively, and the sum of the two is taken as the accuracy of each discriminator;

[0018] The ratio of the accuracy of each discriminator to the sum of the accuracy of all discriminators is taken as the expert reliability coefficient of each discriminator, and the reward value is calculated according to the expert reliability coefficient of each discriminator and the discriminant value corresponding to the imitator's trajectory in each discriminator.

[0019] Furthermore, the discriminant value output by each discriminator is judged by the discriminator to be correct, including:

[0020] For the state-action pair in the expert trajectory, when the discriminant value output by the discriminator is greater than or equal to 0.5, it is considered that the discriminator has judged it correctly; for the state-action pair in the imitator trajectory, when the discriminant value output by the discriminator is less than 0.5, it is considered that the discriminator has judged it correctly.

[0021] Furthermore, the reward value is calculated by the following formula based on the expert reliability coefficient of each discriminator and the discriminant value corresponding to the imitator's trajectory in each discriminator:

[0022]

[0023] Among them, R i represents the reward value during the i-th round of training; n represents the number of expert trajectories, w j represents the expert reliability coefficient of the discriminator corresponding to the jth expert trajectory; D ij represents the discriminant value of the state action in the imitator trajectory during the i-th round of training input to the discriminator output corresponding to the j-th expert trajectory; S j represents the state space in the trajectory, A j Represents the action space in the trajectory.

[0024] Furthermore, the reward function includes: stability reward, performance improvement reward and operational reward; wherein, the stability reward is a positive reward given according to the stability of the unit operation; the performance improvement reward is a positive reward or negative reward given by judging whether the unit has successfully increased its output and improved the emergency guarantee capability of the power grid; the operational reward is a negative reward of a specified proportion given to the rationality of operations in each dimension in the action space.

[0025] The above-mentioned embodiments of the present invention have at least the following beneficial effects: by implementing the verification test method of the present invention for increasing the unit output to improve the emergency guarantee capability of the power grid, the behavior of the unit in various operating scenarios can be effectively simulated, thereby accurately evaluating the unit's output capability and the emergency response capability of the power grid. The method uses a reinforcement learning model to generate multiple expert strategies, and samples expert trajectories through interaction with the unit simulation environment. These expert trajectories can reflect the optimal performance of the unit under different operating conditions. Furthermore, by constructing a generator and a discriminator, imitator trajectories can be generated, and these trajectories can be used to train the discriminator to distinguish between expert strategies and generator strategies. This method can not only improve the verification accuracy of the unit output, but also optimize the generator through the PPO algorithm, thereby further improving the unit output and the emergency guarantee capability of the power grid.

[0026] In addition, the method of the present invention can also comprehensively evaluate the stability, performance improvement and operability of the unit operation through the design of the reward function, ensuring that the unit will not sacrifice the stability of the system and the rationality of operation while increasing the output. The application of this method can significantly improve the emergency response speed and processing capacity of the power system, which is of great significance for ensuring the safe and stable operation of the power grid. At the same time, the implementation of this method can also reduce the dependence on the actual unit operation data, reduce the verification cost, and improve the flexibility and scalability of the verification process. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] One or more embodiments are exemplarily described by the figures in the corresponding drawings, and these exemplifications do not constitute limitations on the embodiments. Elements with the same reference numerals in the drawings represent similar elements, and unless otherwise stated, the figures in the drawings do not constitute proportional limitations.

[0028] Figure 1 It is a flow chart of a verification test method for increasing the output of a unit and improving the emergency protection capability of a power grid provided by the present invention. DETAILED DESCRIPTION

[0029] In order to facilitate the understanding of the present invention, the present invention is described in more detail below in conjunction with the accompanying drawings and specific embodiments. It should be noted that when an element is described as "connecting" another element, it can be directly on another element, or there can be one or more centered elements therebetween. The orientation or positional relationship indicated by the terms "upper", "lower", "left", "right", "upper end", "lower end", "top" and "bottom" used in this specification is based on the orientation or positional relationship shown in the accompanying drawings, only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation of the present invention. In addition, the terms "first", "second", etc. are only used for descriptive purposes and cannot be understood as indicating or implying relative importance.

[0030] Unless otherwise defined, all technical and scientific terms used in this specification have the same meaning as those commonly understood by those skilled in the art of the present invention. The terms used in the specification of the present invention are only for the purpose of describing specific embodiments and are not intended to limit the present invention.

[0031] Combine the following Figure 1 , the verification test method 100 for increasing the output of a unit and improving the emergency protection capability of a power grid provided in an embodiment of the present application is described in detail through a specific embodiment.

[0032] Figure 1The flowchart of the method for verifying the emergency protection capability of the power grid by increasing the power output of the unit provided by the present invention is as follows. The method for verifying the emergency protection capability of the power grid by increasing the power output of the unit provided by one embodiment of the present invention comprises:

[0033] Step 101, after loading the unit model, initialize the unit simulation environment;

[0034] Step 102, based on the unit simulation environment, multiple expert strategies are obtained by training the reinforcement learning model; multiple expert trajectories are sampled by interacting the expert strategies with the unit simulation environment;

[0035] Step 103, construct a generator and the same number of discriminators as the expert strategy; in the training process, the generator interacts with the unit simulation environment to sample the imitator trajectory; use the imitator trajectory and each expert trajectory to train a discriminator respectively, calculate the reward value according to the discriminant value output by each discriminator, and use the PPO algorithm to train the generator according to the imitator trajectory and the reward value;

[0036] Step 104, obtaining a verification test model for increasing the unit output based on the trained generator.

[0037] Specifically, the method first needs to load the unit model and initialize the simulation environment. The unit model is an XML file that defines the composition and connection relationship of the unit, including the turbine, generator and control system. The simulation environment is realized by building a power system simulation platform and introducing a simulation library. During the initialization process, it is necessary to define the state space, action space and reward function. The setting of these parameters will directly affect the accuracy and efficiency of the simulation.

[0038] Preferably, the state space may include key parameters such as the unit's output, speed, temperature, pressure, fuel flow, and steam flow. The action space may include operations such as fuel supply, steam valve opening, and cooling water flow. The specific settings of these parameters can be adjusted according to the actual unit performance and the needs of the power grid. For example, the parameters of the state space can be set based on the historical operating data of the unit, while the operations of the action space can be optimized based on the dispatching strategy of the power grid. In this way, it can be ensured that the simulation environment can truly reflect the performance of the unit in actual operation, thereby providing a reliable basis for verification tests.

[0039] In some embodiments, the unit simulation environment is obtained by building a power system simulation platform and introducing a simulation library; the unit model is an XML format file used to define the composition and connection relationship of the unit, wherein the unit consists of a steam turbine, a generator and a control system, the steam turbine and the generator are connected through an axis, and the control system is connected to the steam turbine and the generator through signals.

[0040] It should be noted that the construction method of the unit simulation environment includes the construction of the power system simulation platform and the introduction of the simulation library. In this way, the operating status of the unit in the actual power grid can be simulated to provide accurate simulation data for subsequent verification tests.

[0041] Specifically, the unit simulation environment is obtained by building a power system simulation platform and introducing a simulation library. The power system simulation platform is a software tool that can simulate various equipment and operations in the power system, including generators, transformers, transmission lines, etc. The simulation library contains various simulation models and algorithms for simulating the physical characteristics and control logic of the unit. In this environment, the unit model is an XML format file that defines the composition and connection relationship of the unit in detail. For example, the turbine and generator are connected through shafts, and the control system is connected to the turbine and generator through signals, which are clearly specified in the XML file.

[0042] Preferably, the construction of the unit model can be customized according to the detailed parameters of the actual unit. For example, the parameters of the steam turbine may include steam pressure, temperature and flow, the parameters of the generator may include output power and speed, and the parameters of the control system may include various control strategies and response time. These parameters can be obtained based on the data provided by the unit manufacturer or through actual measurements. In addition, the construction of the simulation environment can also be adjusted according to the specific conditions of the power grid, such as considering factors such as the load characteristics of the power grid and the distribution of transmission lines to ensure the accuracy and practicality of the simulation results. In this way, a highly realistic unit simulation environment can be constructed to provide a solid foundation for verification tests.

[0043] In some embodiments, the initialization of the unit simulation environment includes: defining a state space, an action space, and a reward function based on the unit's operating scenario; and initializing the unit's state space when the simulation environment is reset.

[0044] It should be noted that the process of initializing the unit simulation environment is one of the key steps in the verification test method. The initialization process ensures that the simulation environment can accurately reflect the state of the unit in a specific operating scenario, providing a basis for subsequent strategy training and verification.

[0045] Specifically, initializing the unit simulation environment includes defining the state space, action space, and reward function based on the unit's operating scenario. The state space refers to the set of various states that can be observed by the unit in the simulation, and the action space refers to the set of various operations that the unit can perform. The reward function is used to evaluate the performance of the unit in the simulation and guide the training of the reinforcement learning model. When the simulation environment is reset, the unit's state space needs to be initialized, which usually involves setting the unit's initial state, such as output, speed, temperature, etc.

[0046] Preferably, the state space may include key parameters such as the unit's output, speed, temperature, pressure, fuel flow, steam flow, etc. The specific settings of these parameters can be determined according to the design specifications and operating requirements of the unit. For example, the output can be set to a value between the maximum output and the minimum output of the unit, and the speed can be set to the rated speed of the unit. The action space may include operations such as fuel supply, steam valve opening, cooling water flow, etc. The specific parameters of these operations can be set according to the control logic and operating constraints of the unit. The reward function can be designed as a composite function, including stability rewards, performance improvement rewards, and operability rewards to comprehensively evaluate the performance of the unit. In this way, it can be ensured that the simulation environment accurately reflects the initial state of the unit after each reset, providing a reliable starting point for subsequent verification tests.

[0047] In some embodiments, the state space includes: the output, rotation speed, temperature, pressure, fuel flow, and steam flow of the unit; the action space includes: fuel supply, steam valve opening, and cooling water flow.

[0048] Specifically, the state space includes the unit's output, speed, temperature, pressure, fuel flow, and steam flow. These parameters are key indicators of the unit's operating status, and together they define the unit's current state in the simulation environment. For example, output refers to the unit's current power generation capacity, speed is the rotation speed of the unit's rotating parts, temperature and pressure involve the thermodynamic state inside the unit, and fuel flow and steam flow are directly related to the unit's energy conversion efficiency.

[0049] Preferably, the parameter settings of the state space can be determined according to the actual operating data and design specifications of the unit. For example, the output can be set to a reasonable range based on the maximum and minimum power generation capacity of the unit, and the speed can be set to the rated speed of the unit plus a certain fluctuation range. The temperature and pressure parameters can be set according to the operating data of the unit under different operating conditions to ensure that the simulation environment can cover various thermodynamic states that the unit may encounter. The settings of fuel flow and steam flow need to consider the fuel consumption rate and thermal efficiency of the unit, as well as the possible adjustment range. In this way, it can be ensured that the state space can fully reflect the state of the unit under various operating conditions, providing accurate simulation data for subsequent strategy training and verification.

[0050] In some embodiments, the interaction with the unit simulation environment through expert strategies and the sampling of multiple expert trajectories are based on the same task, and the expert strategies are used to interact with the unit simulation environment respectively. The state space input and the action space output in each interaction are taken as a state-action pair. After the same number of interactions, the state-action pair sequence obtained is taken as the expert trajectory corresponding to the expert strategy; the imitator trajectory is the state-action pair sequence generated by the generator interacting with the unit simulation environment multiple times.

[0051] It should be noted that the process of sampling multiple expert trajectories through the interaction of expert strategies with the unit simulation environment involves using expert strategies to operate in the simulation environment to collect data for subsequent model training and verification.

[0052] Specifically, the method of interacting with the unit simulation environment through expert strategies and sampling multiple expert trajectories is based on the same task, and the expert strategies are used to interact with the unit simulation environment respectively. The state space input and the action space output in each interaction are taken as a state-action pair. After the same number of interactions, the state-action pair sequence obtained is taken as the expert trajectory corresponding to the expert strategy. This process involves simulating the operation of the unit in the simulation environment. Each operation selects an action according to the current state space, and then records the combination of this state and action. By repeating this process, a series of state-action pairs can be generated, which constitute expert trajectories.

[0053] Preferably, the generation of expert strategies can be based on historical data, expert experience or previous simulation results. The specific parameter settings of the state space and action space should be determined according to the actual operating conditions and control logic of the unit. For example, the state space may include parameters such as the real-time output, speed, temperature, etc. of the unit, while the action space may include adjustable fuel supply, steam valve opening, etc. The specific number of interactions each time can be set according to the accuracy requirements of the simulation and the limitations of computing resources. In addition, in order to improve the sampling efficiency, parallel computing technology can be used to run multiple simulation instances at the same time to quickly collect a large amount of expert trajectory data. In this way, it can be ensured that the expert trajectory can fully reflect the performance of the expert strategy under different states, providing rich data support for subsequent model training and verification.

[0054] In some embodiments, the method of using the imitator trajectory and each expert trajectory to train a discriminator is to concatenate the state-action pairs in the imitator trajectory with the state-action pairs in each expert trajectory and pass them into a discriminator, and each discriminator outputs the discrimination values ​​of the two state-action pairs; and the gradient of the corresponding discriminator is calculated according to the discrimination values ​​of the two state-action pairs to update the discriminator parameters.

[0055] It should be noted that the process of training a discriminator using the imitator trajectory separately from each expert trajectory involves comparing the imitator trajectory produced by the generator with the expert trajectory produced by the expert strategy to train the discriminator to distinguish between the two.

[0056] Specifically, the method of using the imitator trajectory and each expert trajectory to train a discriminator is to concatenate the state-action pairs in the imitator trajectory with the state-action pairs in each expert trajectory and pass them into a discriminator, and each discriminator outputs the discrimination value of the two state-action pairs. This process involves inputting the data of the imitator trajectory and the expert trajectory into the discriminator, and the task of the discriminator is to determine whether the given state-action pair comes from the expert strategy or the generator strategy. The output of the discriminator is a probability value, which indicates the probability that the input data belongs to the expert trajectory.

[0057] Preferably, the discriminator can use a deep learning model, such as a convolutional neural network (CNN) or a recurrent neural network (RNN), which can effectively process sequence data and extract features. The concatenation of state-action pairs can be designed according to the requirements of the discriminator model. For example, the vectors of state and action can be directly connected, or the information can be fused through a specific network layer.

[0058] Furthermore, the training of the discriminator can be achieved by minimizing the prediction error. For example, the cross entropy loss function can be used to train the discriminator so that it can more accurately distinguish between the expert trajectory and the imitator trajectory. In addition, in order to improve the generalization ability of the discriminator, data enhancement, regularization and other techniques can be used to prevent overfitting. In this way, it can be ensured that the discriminator can effectively learn the difference between the expert strategy and the generator strategy during the training process, thereby providing accurate discriminant information for the subsequent reward value calculation and generator training.

[0059] In some embodiments, the step of calculating the reward value according to the discriminant value output by each discriminator includes:

[0060] According to the discriminant value output by each discriminator, the probability that the discriminator is correct is calculated, and the expert accuracy and imitator accuracy of each discriminator are calculated respectively, and the sum of the two is taken as the accuracy of each discriminator;

[0061] The ratio of the accuracy of each discriminator to the sum of the accuracy of all discriminators is taken as the expert reliability coefficient of each discriminator, and the reward value is calculated according to the expert reliability coefficient of each discriminator and the discriminant value corresponding to the imitator's trajectory in each discriminator.

[0062] It should be noted that the process of calculating the reward value based on the discriminant value output by each discriminator involves evaluating the quality of the imitator trajectory generated by the generator and adjusting the training of the generator accordingly.

[0063] Specifically, the reward value is calculated according to the discrimination value output by each discriminator, including calculating the probability that the discriminator is correct according to the discrimination value output by each discriminator, calculating the expert accuracy rate and the imitator accuracy rate of each discriminator respectively, and taking the sum of the two as the accuracy rate of each discriminator; taking the ratio of the accuracy rate of each discriminator to the sum of the accuracy rates of all discriminators as the expert reliability coefficient of each discriminator, and calculating the reward value according to the expert reliability coefficient of each discriminator and the discrimination value corresponding to the imitator trajectory in each discriminator. This process involves analyzing the output of the discriminator to determine its recognition accuracy of the expert trajectory and the imitator trajectory, and calculating the reward value accordingly.

[0064] Preferably, the output of the discriminator can be a probability value indicating the probability that a given state-action pair belongs to an expert trajectory. The expert accuracy and the imitator accuracy can be calculated by comparing the output of the discriminator with the actual trajectory label. For example, if the discriminator correctly identifies a state-action pair as an expert trajectory, then this will increase the expert accuracy; conversely, if it is correctly identified as an imitator trajectory, then the imitator accuracy will be increased.

[0065] Furthermore, the expert reliability coefficient can be obtained by calculating the ratio of the accuracy of each discriminator to the total accuracy of all discriminators. The reward value can be calculated by weighted average, where the weight is the expert reliability coefficient, which ensures that the output of the discriminator has an appropriate impact on the training of the generator. In addition, the calculation of the reward value can also consider other factors, such as the confidence of the discriminator, to further improve the accuracy and effectiveness of the reward value. In this way, it can be ensured that the reward value can accurately reflect the quality of the imitator's trajectory, thereby guiding the training of the generator.

[0066] In some embodiments, the discriminant value output by each discriminator is judged by the discriminator to be correct, including:

[0067] For the state-action pair in the expert trajectory, when the discriminant value output by the discriminator is greater than or equal to 0.5, it is considered that the discriminator has judged it correctly; for the state-action pair in the imitator trajectory, when the discriminant value output by the discriminator is less than 0.5, it is considered that the discriminator has judged it correctly.

[0068] Specifically, the discriminant values ​​output by each discriminator are considered correct by the discriminator, including for the state-action pair in the expert trajectory, when the discriminant value output by the discriminator is greater than or equal to 0.5, it is considered that the discriminator has judged it to be correct; for the state-action pair in the imitator trajectory, when the discriminant value output by the discriminator is less than 0.5, it is considered that the discriminator has judged it to be correct. Here, 0.5 is a threshold value used to distinguish the probability of the discriminator output to determine the source of the state-action pair.

[0069] Preferably, the threshold value of 0.5 of the discriminant value can be adjusted according to the actual discriminator performance and application scenario. For example, if the discriminator shows a high accuracy rate during the training process, the threshold value can be appropriately increased to reduce the possibility of misjudgment. Conversely, if the performance of the discriminator is low, the threshold value can be appropriately lowered to improve the sensitivity of the discriminator. In addition, the setting of the threshold value can also be based on the results of cross-validation, by testing different threshold values ​​on different data sets, and selecting the best threshold value to balance the accuracy and misjudgment rate of the discriminator.

[0070] More specifically, the output of the discriminator can be a continuous probability value, indicating the probability that the state-action pair belongs to the expert trajectory. In actual operation, the correctness of each state-action pair can be determined based on the output probability of the discriminator and the set threshold. For example, a soft threshold method can be used instead of a strict 0.5 to allow fine-tuning of probability values ​​within a certain range to accommodate different discriminator performance and application requirements. In addition, the concept of a confidence interval can be considered. When the output probability of the discriminator is close to the threshold, the accuracy of the judgment can be increased through additional verification steps. In this way, it can be ensured that the output of the discriminator can accurately reflect the source of the state-action pair, thereby providing a reliable basis for the calculation of the reward value.

[0071] In some embodiments, the reward value is calculated by the following formula based on the expert reliability coefficient of each discriminator and the discrimination value corresponding to the imitator trajectory in each discriminator:

[0072]

[0073] Among them, R i represents the reward value during the i-th round of training; n represents the number of expert trajectories, w j represents the expert reliability coefficient of the discriminator corresponding to the jth expert trajectory; D ij represents the discriminant value of the state action in the imitator trajectory during the i-th round of training input to the discriminator output corresponding to the j-th expert trajectory; S j represents the state space in the trajectory, A j Represents the action space in the trajectory.

[0074] It should be noted that the specific formula for calculating the reward value based on the expert reliability coefficient of each discriminator and the discriminant value corresponding to the imitator's trajectory in each discriminator is a key step in the verification test method. It involves how to quantify the performance of the discriminator and adjust the training of the generator accordingly.

[0075] Specifically, the reward value is calculated by the following formula based on the expert reliability coefficient of each discriminator and the discriminant value corresponding to the imitator's trajectory in each discriminator:

[0076]

[0077] Among them, R represents the reward value in the kth round of training; N represents the number of expert trajectories, and w i represents the expert reliability coefficient of the discriminator corresponding to the i-th expert trajectory; v i represents the discriminant value of the state action in the imitator trajectory during the k-th round of training input to the discriminator output corresponding to the i-th expert trajectory; s i represents the state space in the trajectory, a i Represents the action space in the trajectory.

[0078] Preferably, the calculation of the reward value can be further refined to ensure that it can accurately reflect the performance of the discriminator and the quality of the imitator's trajectory. For example, the expert reliability coefficient w i It can be dynamically adjusted according to the performance of the discriminator on the validation set to reflect its reliability in practical applications. i It can be a continuous probability value, indicating the probability that the imitator trajectory is identified as an expert trajectory. i and action space a i It can include various operating parameters of the unit, such as output, speed, temperature, etc., as well as operations that the unit can perform, such as fuel supply, steam valve opening, etc.

[0079] More specifically, the calculation of the reward value can also take into account other factors, such as the confidence of the discriminator, the diversity of the imitator trajectories, etc. For example, a confidence parameter can be introduced, and when the discriminator has a higher confidence in its output, its contribution to the reward value increases accordingly. In addition, in order to encourage the generator to produce more diverse imitator trajectories, a diversity reward term can be designed to give additional rewards when the imitator trajectories cover a wide area of ​​the state space and action space. In this way, it can be ensured that the reward value can fully reflect the performance of the discriminator and the quality of the imitator trajectories, thereby effectively guiding the training of the generator.

[0080] In some embodiments, the reward function includes: stability reward, performance improvement reward and operational reward; wherein, the stability reward is a positive reward given based on the stability of the unit operation; the performance improvement reward is a positive reward or negative reward given by judging whether the unit has successfully increased its output and improved the emergency guarantee capability of the power grid; the operational reward is a negative reward of a specified proportion given to the rationality of operations in each dimension in the action space.

[0081] Specifically, the reward function includes stability reward, performance improvement reward and operability reward. Stability reward is a positive reward given according to the stability of the unit operation, performance improvement reward is a positive reward or negative reward given by judging whether the unit has successfully increased its output and improved the emergency guarantee capability of the power grid, and operability reward is a negative reward of a specified proportion given to the rationality of the operation of each dimension in the action space. The setting of these rewards is intended to encourage the generator to produce a stable and efficient unit operation strategy.

[0082] Preferably, the parameters of the reward function can be adjusted according to the actual operating data of the unit and the needs of the power grid. For example, the stability reward can be associated with parameters such as the unit's output fluctuations and temperature changes, and rewards are given when these parameters remain within a predetermined safety range. Performance improvement rewards can be related to the increase in the unit's output and the emergency response time of the power grid. When the unit's output increases and the power grid response time is shortened, positive rewards are given, and vice versa. Operational rewards can be related to parameters such as the unit's operating frequency and operating amplitude, encouraging the generator to take smooth and reasonable operations.

[0083] More specifically, the reward function can be implemented by designing a mathematical model to quantify the reward values ​​of the above three aspects. For example, the stability reward can be designed as a function inversely proportional to the fluctuation amplitude of the unit state, the performance improvement reward can be designed as a function proportional to the increase in unit output and the reduction in grid response time, and the operability reward can be designed as a function inversely proportional to the operating frequency and amplitude. In addition, weight coefficients can be introduced to balance the importance of different reward items to ensure that the reward function can fully reflect the comprehensive performance of the unit operation. In this way, it can be ensured that the reward function can effectively guide the training of the generator and optimize the unit's operating strategy.

[0084] The above-mentioned embodiments of the present invention have the following beneficial effects: the verification test method of the present invention for increasing the output of the unit to improve the emergency guarantee capability of the power grid can provide a comprehensive unit performance verification framework, and through the combination of simulation environment and reinforcement learning technology, realize the in-depth exploration and verification of the potential for increasing the output of the unit. This method can ensure that the potential performance of the unit in the emergency situation of the power grid is evaluated through simulation and optimization strategies without actually increasing the output of the unit, thereby providing a scientific basis for the stable operation and emergency response of the power grid.

[0085] In addition, this verification test method can significantly improve the efficiency and accuracy of unit output verification. Through the interactive sampling of expert strategies and simulation environments, as well as the collaborative work of generators and discriminators, a large amount of high-quality data can be generated for training and verification, which can not only reduce the dependence on actual unit operations, but also reduce the risks in the verification process. At the same time, by training the generator through the PPO algorithm, the unit output strategy can be continuously optimized to make it more adaptable to the dynamic changes of the power grid, thereby achieving more efficient power grid emergency protection in practical applications.

[0086] Furthermore, the storage medium of the embodiment of the present application stores program instructions that can implement all the above methods, wherein the program instructions can be stored in the above storage medium in the form of a software product, including several instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) or a processor to execute all or part of the steps of the methods described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, or terminal devices such as a computer, a server, a mobile phone, and a tablet.

[0087] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit them. Under the concept of the present invention, the technical features in the above embodiments or different embodiments can also be combined, the steps can be implemented in any order, and there are many other changes in different aspects of the present invention as above, which are not provided in detail for the sake of simplicity. Although the present invention is described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions recorded in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for verifying the emergency protection capability of power grid by increasing the output of a unit, characterized in that: The following steps are involved: After loading the unit model, initialize the unit simulation environment; Based on the unit simulation environment, multiple expert strategies are obtained by training the reinforcement learning model; multiple expert trajectories are sampled through the interaction between the expert strategy and the unit simulation environment; Build a generator and the same number of discriminators as the expert policy; During the training process, the generator interacts with the unit simulation environment to sample the imitator trajectory; a discriminator is trained using the imitator trajectory and each expert trajectory respectively, the reward value is calculated based on the discriminant value output by each discriminator, and the generator is trained using the PPO algorithm based on the imitator trajectory and the reward value; The verification test model for increasing the unit output is obtained based on the trained generator.

2. A verification test method for increasing the output of a unit to improve the emergency protection capability of a power grid according to claim 1, characterized in that: The unit simulation environment is obtained by building a power system simulation platform and introducing a simulation library; the unit model is a file in XML format, which is used to define the composition and connection relationship of the unit, wherein the unit consists of a steam turbine, a generator and a control system, the steam turbine and the generator are connected through an axis, and the control system is connected to the steam turbine and the generator through signals.

3. A verification test method for increasing the output of a unit to improve the emergency protection capability of a power grid according to claim 2, characterized in that: The initialization of the unit simulation environment includes: defining a state space, an action space and a reward function based on the unit's operating scenario; and initializing the unit's state space when the simulation environment is reset.

4. A verification test method for increasing the output of a unit to improve the emergency protection capability of a power grid according to claim 3, characterized in that: The state space includes: the unit's output, rotation speed, temperature, pressure, fuel flow, and steam flow; the action space includes: fuel supply, steam valve opening, and cooling water flow.

5. A verification test method for increasing the output of a unit to improve the emergency protection capability of a power grid according to claim 1, characterized in that: The method interacts with the unit simulation environment through expert strategies, samples multiple expert trajectories based on the same task, and uses expert strategies to interact with the unit simulation environment respectively. The state space input and the action space output in each interaction are taken as a state-action pair. After the same number of interactions, the state-action pair sequence obtained is taken as the expert trajectory corresponding to the expert strategy; the imitator trajectory is a state-action pair sequence generated by the generator interacting with the unit simulation environment multiple times.

6. A verification test method for increasing the output of a unit to improve the emergency protection capability of a power grid according to claim 5, characterized in that: The method of using the imitator trajectory and each expert trajectory to train a discriminator is to concatenate the state-action pairs in the imitator trajectory with the state-action pairs in each expert trajectory and pass them into a discriminator, and each discriminator outputs the discriminant value of the two state-action pairs; The discriminant parameters of the corresponding discriminator are updated by calculating the discriminant values ​​of the two state-action pairs.

7. A verification test method for increasing the output of a unit to improve the emergency protection capability of a power grid according to claim 6, characterized in that: The step of calculating the reward value according to the discriminant value output by each discriminator includes: According to the discriminant value output by each discriminator, the probability that the discriminator is correct is calculated, and the expert accuracy and imitator accuracy of each discriminator are calculated respectively, and the sum of the two is taken as the accuracy of each discriminator; The ratio of the accuracy of each discriminator to the sum of the accuracy of all discriminators is taken as the expert reliability coefficient of each discriminator, and the reward value is calculated according to the expert reliability coefficient of each discriminator and the discriminant value corresponding to the imitator's trajectory in each discriminator.

8. A verification test method for increasing the output of a unit to improve the emergency protection capability of a power grid according to claim 7, characterized in that: The discriminant values ​​output by each discriminator are judged by the discriminator to be correct, including: For the state-action pair in the expert trajectory, when the discriminant value output by the discriminator is greater than or equal to 0.5, it is considered that the discriminator has judged it correctly; for the state-action pair in the imitator trajectory, when the discriminant value output by the discriminator is less than 0.5, it is considered that the discriminator has judged it correctly.

9. A verification test method for increasing the output of a unit to improve the emergency protection capability of a power grid according to claim 7, characterized in that: The reward value is calculated according to the expert reliability coefficient of each discriminator and the discriminant value corresponding to the imitator trajectory in each discriminator by the following formula: Among them, R i represents the reward value during the i-th round of training; n represents the number of expert trajectories, w j represents the expert reliability coefficient of the discriminator corresponding to the jth expert trajectory; D ij represents the discriminant value of the state action in the imitator trajectory during the i-th round of training input to the discriminator output corresponding to the j-th expert trajectory; S j represents the state space in the trajectory, A j Represents the action space in the trajectory.

10. A verification test method for increasing the output of a unit to improve the emergency protection capability of a power grid according to claim 3, characterized in that: The reward function includes: stability reward, performance improvement reward and operational reward; among them, the stability reward is a positive reward given according to the stability of the unit operation; the performance improvement reward is a positive reward or negative reward given by judging whether the unit has successfully increased its output and improved the emergency guarantee capability of the power grid; the operational reward is a negative reward of a specified proportion given to the rationality of operations in each dimension in the action space.

Images