Power system control strategy generation method and system
By constructing a meta-policy model and utilizing environmental latent codes to optimize policy network parameters, the problem of power systems being unable to adapt quickly under extreme conditions was solved, achieving stable control and rapid response of the power grid.
Patent Information
- Application Number
- CN202511828695.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-12-05
- Publication Date
- 2026-02-13
AI Technical Summary
The existing power system faces the problem that its control strategies cannot adapt quickly in the face of high proportions of renewable energy integration and extreme weather conditions. In particular, it cannot effectively regulate the power grid under extreme events, leading to grid instability.
By constructing a meta-policy model and using environmental latent codes to abstractly encode different power grid operation scenarios, the policy network parameters and environmental latent codes are jointly optimized to generate an adaptive policy model that can quickly adapt to the needs of different scenarios.
It enables rapid adaptive control of the power system under extreme conditions, ensuring stable operation of the power grid and improving the cross-scenario versatility and adaptability of the control strategy.
Smart Images
Figure CN121529665A_ABST
Abstract
Description
TECHNICAL FIELD
[0001] The present application relates to the technical field of power system regulation, in particular to a power system control strategy generation method and system. BACKGROUND
[0002] With the access of high proportion of renewable energy, the uncertainty of the power system increases, and extreme weather will cause extreme and concentrated electricity demand. Without intervention, the power system will more frequently approach or touch the operating baseline. In the prior art, the training environment is usually learned based on deep learning to realize timely regulation and intervention of the power system.
[0003] However, the power system faces extensive and unknown extreme events, which requires the control strategy to have the ability to quickly adapt within minutes or even seconds. The current control methods including the generation of power grid control strategy by deep learning all have environmental limitations. In the case of large differences in operating conditions, the learned power grid control strategy may fail to meet the emergency regulation scenario. SUMMARY
[0004] Embodiments of the present application aim to provide a power system control strategy generation method and system, which can abstract and encode different power grid operating scenarios through environment latent codes to match the needs of different scenarios, and the obtained adaptive strategy model accumulates cross-scene general knowledge and has the characteristics of fast adaptation.
[0005] To solve the above technical problems, the present application provides a power system control strategy generation method, comprising: Obtain first operating data under a plurality of power grid operating scenarios, train a pre-constructed meta-strategy model according to the first operating data, and jointly optimize the strategy network parameters of the meta-strategy model and the environment latent codes corresponding to each power grid operating scenario during the training process; Obtain second operating data under a target power grid operating scenario, optimize the environment latent codes of the trained meta-strategy model to obtain an adaptive strategy model, with the optimal control performance under the second operating data as the target; Input real-time operating data of the power system into the adaptive strategy model to generate a power system control strategy.
[0006] As an improvement of the above-mentioned scheme, the pre-construction method of the meta-strategy model comprises: Construct a state space according to the dimension of the operating data of the power system; Construct an action space according to the control means of the power system; the control means include load control and output control; Construct a reward function according to the engineering target of the power system; the engineering target is to minimize the load shedding amount and to restore the voltage to the steady state as soon as possible; According to the state space, the action space and the reward function, a meta-policy model is constructed.
[0007] As an improvement of the above scheme, the first running data under a plurality of power grid running scenarios is obtained, and the pre-constructed meta-policy model is trained according to the first running data. During the training process, the policy network parameters of the meta-policy model and the environment latent code corresponding to each power grid running scenario are jointly optimized, including: A plurality of power grid running scenarios are constructed in a power grid simulation model of a power system; the power grid simulation model corresponds to the power system; According to the action space of the pre-constructed meta-policy model, the environment interaction is performed in the power grid running scenario, and the first running data is dynamically generated; The first running data is used to train the meta-policy model, and the environment latent code corresponding to each power grid running scenario is obtained to form an environment latent code set; According to the environment latent code set, the policy network parameters of the meta-policy model are optimized, so that the trained meta-policy model supports dynamic adjustment of the output based on the environment latent code.
[0008] As an improvement of the above scheme, the plurality of power grid running scenarios are constructed in the power grid simulation model of the power system, including: Obtain the load parameters of the power grid simulation model of the power system; A plurality of initial power grid running scenarios are constructed by dynamically adjusting the load parameters of the power grid simulation model; An extreme event is set in the initial power grid running scenario to construct an extreme power grid running scenario; According to the initial power grid running scenario and the extreme power grid running scenario, the power grid running scenario is obtained.
[0009] As an improvement of the above scheme, according to the action space of the pre-constructed meta-policy model, the environment interaction is performed in the power grid running scenario, and the first running data is dynamically generated, including: According to the action space of the pre-constructed meta-policy model, a control action is selected; the selection algorithm of the control action is iterated with the training of the meta-policy model; According to the control action and the day-ahead operation characteristics of the power system, the environment simulation interaction is performed in the power grid simulation model to obtain the first running data.
[0010] As an improvement of the above scheme, the first running data is used to train the meta-policy model, and the environment latent code corresponding to each power grid running scenario is obtained to form an environment latent code set, including: inputting the first operation data into the meta-strategy model, and traversing to calculate accumulated rewards under each power grid operation scenario; updating environment latent codes corresponding to each power grid operation scenario in the meta-strategy model, aiming at the maximum accumulated reward; obtaining an environment latent code set according to the environment latent codes corresponding to all power grid operation scenarios.
[0011] As an improvement of the above scheme, the strategy network parameters of the meta-strategy model are optimized according to the environment latent code set, so that the trained meta-strategy model supports dynamic adjustment of the output based on the environment latent code, comprising: fixing the environment latent code set; calculating the average accumulated reward across power grid operation scenarios according to the environment latent code set; updating the strategy network parameters of the meta-strategy model through a deep reinforcement learning algorithm according to the average accumulated reward, so that the trained meta-strategy model supports dynamic adjustment of the output based on the environment latent code.
[0012] As an improvement of the above scheme, the second operation data under the target power grid operation scenario is obtained, and the environment latent code of the trained meta-strategy model is optimized to obtain an adaptive strategy model, comprising: obtaining second operation data under a target power grid operation scenario; matching the second operation data with the environment latent codes corresponding to each power grid operation scenario to obtain a target environment latent code; optimizing the target environment latent code to obtain an adaptive strategy model according to the second operation data, aiming at the optimal control performance; the adaptive strategy model makes decisions on control actions according to the optimized target environment latent code under the target power grid operation scenario.
[0013] As an improvement of the above scheme, the real-time operation data of the power system is input into the adaptive strategy model to generate a power system control strategy, comprising: inputting the real-time operation data of the power system into the adaptive strategy model, so that the adaptive strategy model automatically matches the environment latent code according to the real-time operation data and generates a control action; generating a power system control strategy according to the control performance of the control action.
[0014] The embodiment of the application also provides a power system control strategy generation system, comprising: The model meta-training module is configured to obtain first operation data under a plurality of power grid operation scenarios, train a pre-constructed meta-strategy model according to the first operation data, and jointly optimize strategy network parameters of the meta-strategy model and environment latent codes corresponding to the plurality of power grid operation scenarios during the training process. The model adaptation training module is configured to obtain second operation data under a target power grid operation scenario, optimize the environment latent code of the trained meta-strategy model to achieve optimal control performance under the second operation data, and obtain an adapted strategy model. The model application module is configured to input real-time operation data of the power system into the adapted strategy model to generate a power system control strategy.
[0015] Compared with the prior art, the power system control strategy generation method and system provided by the present application can abstract and encode different power grid operation scenarios through environment latent codes to match the requirements of different scenarios, and the adapted strategy model obtained has accumulated cross-scenario general knowledge and has the characteristics of rapid self-adaptation. BRIEF DESCRIPTION OF DRAWINGS
[0016] Figure 1 is a step flowchart of a power system control strategy generation method provided by an embodiment of the present application; Figure 2 is a structural diagram of a power system control strategy generation system provided by an embodiment of the present application. DETAILED DESCRIPTION
[0017] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative labor fall within the scope of protection of the present application.
[0018] In the description and claims of the specification, the terms "first", "second", and the like, if any, are used for distinguishing between similar elements and not necessarily for describing a sequential or chronological order. Terms are defined with the context of the detailed description being served and that the meaning can change based on the context within which the term is used. Terms such as "including", "containing", "consisting of", and the like, are used herein to mean that the element include but are not limited to that which is specifically named. Thus, the use of such terms is meant to only cover the specific instance. The use of the term "or" in the claims is used to mean "and / or" unless explicitly indicated to refer to a selection from alternatives (using "or" in the alternative "A or B" indicates a selection from A or B but not both).
[0019] Current power system control strategy generation is generally to learn the training environment by means of a deep learning algorithm to obtain an optimal control strategy under a specific scenario. However, the uncertainty of the current power system is gradually increasing, and the operating conditions may have great differences.
[0020] The training environment originally learned by the traditional deep learning algorithm cannot be migrated to an environment with great differences from the operating conditions, and the overall control strategy needs to be relearned, which leads to the inability to meet the emergency regulation of the power grid under extreme events.
[0021] Based on the above thinking, the embodiment of the present application provides a power system control strategy generation method. Please see Figure 1 In the embodiment, the power system control strategy generation method is specifically executed through steps S1 to S3: S1, first running data under a plurality of power grid operating scenarios is acquired, a pre-constructed meta-strategy model is trained according to the first running data, and the strategy network parameters of the meta-strategy model and the environment latent code corresponding to each power grid operating scenario are jointly optimized during the training process; S2, second running data under a target power grid operating scenario is acquired, and the environment latent code of the trained meta-strategy model is optimized to obtain an adaptive strategy model, with the optimal control performance under the second running data as the target; S3, real-time running data of the power system is input into the adaptive strategy model to generate a power system control strategy.
[0022] It should be noted that the meta-strategy model is trained based on the first running data of the conventional and known power grid operating scenario. Therefore, the meta-strategy model can be directly used in the power grid operating scenario corresponding to its training environment.
[0023] The meta-strategy model is a neural network with conditional input structure, and its decision is jointly determined by the strategy network parameters and the environment latent code. The strategy network parameters store the general control logic learned from the first running data. The environment latent code is a low-dimensional abstract scene code, and each unique power grid operating scenario corresponds to a specific environment latent code, which is used to describe the current environment of the meta-strategy model. Through the environment latent code, the strategy can be quickly adapted.
[0024] When the meta-strategy model makes a decision, the strategy network parameters provide a general decision framework and computing power, and the environment latent code is precisely modulated to enable the general decision framework to output the most appropriate output in the current scenario.
[0025] In some preferred embodiments, the meta-strategy model is represented as ; wherein, is a strategy function; is a strategy network parameter, is a state observation, is an environment latent code. That is, under the condition of a given power grid state s and an environment latent code c of the current scenario, the optimal control action is output by the meta-strategy model with parameters , forming a control strategy.
[0026] The target power grid operation scenario is different from the known power grid operation scenarios described above, and refers to a new scenario that needs to be controlled. In traditional deep learning, if the operating conditions of the target power grid operation scenario are significantly different from the operating conditions of the known power grid operation scenarios, the meta-strategy model is still directly used to control the target power grid operation scenario, which will lead to a failure of the strategy and the need to retrain the entire model, which cannot quickly and stably control the power grid.
[0027] In the embodiments of the present application, the low-dimensional environment latent code in the meta-strategy model is optimized by the second operation data, and the trained meta-strategy model already has general control logic for several power operation scenarios, so that fast model optimization can be achieved with a small amount of data to obtain an adaptive strategy model. The adaptive strategy model is not only suitable for known power grid operation scenarios, but also can be quickly adapted to target power grid operation scenarios.
[0028] The trained adaptive strategy model is specifically calibrated for the target power grid scenario, and can be directly deployed in the power system without further adjustment. When applied, the adaptive strategy model is used as a forward calculation engine to map real-time operation data to a control state, thereby meeting the real-time requirements of the power system.
[0029] In the above scheme, the meta-strategy model with general knowledge and conditional adjustment is trained by several power grid operation scenarios. When a new target power grid operation scenario appears, fast scenario adaptation can be achieved through efficient adjustment of the environment latent code, which guarantees the control performance and improves the adaptation speed of the model.
[0030] As a preferred implementation, the pre-construction method of the meta-strategy model includes: constructing a state space according to the dimension of the operation data of the power system; construct an action space according to control means of the power system; the control means include load control and output control; construct a reward function according to an engineering target of the power system; the engineering target is that the amount of load shedding is minimum and the voltage is recovered to the steady state most quickly; construct a meta-policy model according to the state space, the action space and the reward function.
[0031] The data set in the state space reflects all operating parameters of the power system at a certain time, and the construction of the state space ensures that the state space can cover the actual dimension of the operating data.
[0032] In some preferred embodiments, the operating data in the state space includes bus load, historical output of adjustable units, output of grid-connected wind power plants, predicted output of grid-connected new energy units, minimum continuous operation time length of adjustable units, maximum start-stop times of adjustable units, branch power flow transmission limit, voltage level and voltage phase angle.
[0033] The action set in the action space reflects all available control means of the power system, and each element thereof represents an executable control operation combination. The construction of the action space aims to cover all effective control means.
[0034] In some preferred embodiments, the actions in the action space include load shedding control actions of bus load, output of schedulable units and output of grid-connected wind power plants; wherein the load shedding control actions are controlled at a load shedding percentage of 0 to 20% at each time step.
[0035] Fault-induced delayed voltage recovery (FIDVR) in the power system is defined as that the system voltage remains at a significantly reduced level for a few seconds after the fault is cleared. The emergency control target of the FIDVR problem is to reduce the load shedding as much as possible to restore the voltage to the steady state level.
[0036] Meanwhile, in order to ensure power balance of the power grid in the emergency state, the resources available in the emergency scheduling process considered by the present application include output of schedulable units, output of grid-connected wind power plants and emergency load shedding measures. It has a high priority to ensure continuous power supply of the load in the extreme scenario, and the loss cost caused by load shedding is much higher than that of other scheduling means. Therefore, the engineering target in the meta-training stage is that the amount of load shedding is minimum and the voltage is recovered to the steady state most quickly.
[0037] As a preferred implementation, step S1, first operating data under a plurality of power grid operating scenarios are acquired, a pre-constructed meta-policy model is trained according to the first operating data, and the policy network parameters of the meta-policy model and the environment latent code corresponding to each power grid operating scenario are jointly optimized in the training process. Steps S11-S14 are performed. S11, construct a plurality of power grid operation scenarios in a power grid simulation model of a power system; the power grid simulation model corresponds to the power system; S12, according to an action space of a pre-constructed meta-strategy model, perform environment interaction in the power grid operation scenario, and dynamically generate first operation data; S13, use the first operation data to train the meta-strategy model, to obtain an environment latent code corresponding to each power grid operation scenario, and form an environment latent code set; S14, according to the environment latent code set, optimize strategy network parameters of the meta-strategy model, so that the trained meta-strategy model supports dynamic adjustment of output based on the environment latent code.
[0038] In the embodiment of the application, the power grid operation scenario is constructed by simulation, and corresponds to the topological characteristics and dynamic characteristics of the power system, to construct a credible virtual environment, and the power grid simulation model can simulate the physical behavior of the power system.
[0039] It should be noted that in the simulation process, the first operation data is not static pure historical data, but the pre-constructed meta-strategy model issues a control instruction according to the definition of its action space in the power grid operation scenario, and receives the new state and reward returned by the simulation environment to realize environment interaction, and the data generated in the environment interaction is used as the first operation data.
[0040] In some preferred embodiments, the first operation data includes the state, action, reward and sequence constituted by the subsequent state at each time point.
[0041] In the embodiment of the application, the training process of the meta-strategy model learns to obtain a unique corresponding environment latent code for each scenario in the plurality of power grid operation scenarios, and the environment latent code is a mathematical abstract expression of the essential characteristics of the scenario.
[0042] By optimizing the strategy network parameters based on the environment latent code, the meta-strategy network has dynamic output capability. That is, when different environment latent codes are input, the same set of strategy network parameters can make the model produce different control outputs suitable for the scenario represented by the latent code for the same power grid state input.
[0043] In the above scheme, first, the power grid operation scenario corresponding to the power system is constructed, then the dynamic experience data for driving learning is generated through environment interaction, and further, a low-dimensional environment latent code with discrimination is extracted for each power grid operation scenario, and finally, the strategy network parameters are obtained by refining the general decision logic. The complex multi-scenario adaptation problem can be decomposed into two solvable sub-problems of scene feature coding and conditional strategy learning.
[0044] Further, preferably, step S11, a plurality of power grid operation scenarios are constructed in the power grid simulation model of the power system, including: obtaining load parameters of a power grid simulation model of a power system; constructing a plurality of initial power grid operation scenarios by dynamically adjusting the load parameters of the power grid simulation model; setting an extreme event under the initial power grid operation scenario to construct an extreme power grid operation scenario; obtaining a power grid operation scenario according to the initial power grid operation scenario and the extreme power grid operation scenario.
[0045] In some preferred embodiments, the load parameters are obtained through historical prior knowledge of the day-ahead operation stage of the power system and user-side submitted load and power generation curve forecasting information.
[0046] In the embodiments of the present application, different power grid environments are simulated by dynamically adjusting and combining the load parameters. Further, the load parameters are set to limit values or abnormal values to simulate extreme power grid operation conditions.
[0047] In the above scheme, by applying an extreme event to different initial power grid operation scenarios, combined scenarios with different fault severity and system vulnerability can be constructed, and the finally generated power grid operation scenario can be used to solve the dynamic response problem under a specific steady state under the impact of an extreme event.
[0048] Preferably, step S12, according to the action space of the pre-constructed meta-strategy model, the environment interaction in the power grid operation scenario is performed to dynamically generate first operation data, including: selecting a control action according to the action space of the pre-constructed meta-strategy model; the selection algorithm of the control action is iterated with the training of the meta-strategy model; performing environment simulation interaction on the power grid operation scenario in the power grid simulation model according to the control action and the day-ahead operation characteristics of the power system to obtain first operation data.
[0049] It should be noted that the selection algorithm of the control action, i.e., the update of the strategy network parameters, the specific decision of the control group is iteratively improved synchronously with the update of the strategy network parameters.
[0050] The day-ahead operation characteristics of the power system refer to the regularity or constraint conditions shown in the day-ahead scheduling and operation of the power system, such as the generation plan, unit combination, network topology and corresponding safety boundary based on load and new energy prediction.
[0051] Preferably, step S13, using the first operation data, trains the meta-policy model to obtain the environment latent code corresponding to each power grid operation scene, forms an environment latent code set, including: Input the first operation data into the meta-policy model, and traverse and calculate the cumulative reward under each power grid operation scene; Update the environment latent code corresponding to each power grid operation scene in the meta-policy model with the maximum cumulative reward as the target; According to the environment latent code corresponding to all power grid operation scenes, obtain the environment latent code set.
[0052] It should be noted that the cumulative reward refers to the total reward obtained after the meta-policy model under the current policy network parameter according to its control strategy and the environment after complete interaction in a specific power grid operation scene, which quantifies the overall control performance of the model in the power grid operation scene.
[0053] In the embodiment of the application, the environment latent code corresponding to each power grid operation scene is independently adjusted to improve the cumulative reward of each power grid operation scene, which can finely adjust the behavior mode of the meta-policy model in the power grid operation scene without changing the general policy network parameters of the model, thereby significantly improving the independent performance of the model in each specific power grid operation scene.
[0054] In some preferred embodiments, the update of the environment latent code is represented as: ; Wherein, is the cumulative reward of the meta-policy model in the power grid operation scene ; is the set of several power grid operation scenes.
[0055] Preferably, the update of the environment latent code is a policy optimization problem, which can be solved by a Bayesian optimization algorithm, which will not be described here.
[0056] In the above scheme, the potential of the conditional structure of the meta-policy model is mined to make it fully utilize the scene coding to fine-tune the output, so as to achieve the performance close to the respective optimum on diversified fixed tasks.
[0057] Preferably, step S14, according to the environment latent code set, optimizes the policy network parameters of the meta-policy model, so that the trained meta-policy model supports dynamic adjustment of the output based on the environment latent code, including: Fix the environment latent code set; According to the environment latent code set, calculate the average cumulative reward across power grid operation scenes; According to the average cumulative reward, the policy network parameters of the meta-policy model are updated by a deep reinforcement learning algorithm, so that the trained meta-policy model supports dynamic adjustment of the output based on the environment latent code.
[0058] In the embodiment of the application, under the premise that the set of environment latent codes is fixed, the current meta-policy model is used to simulate interaction in each power grid operation scenario, the cumulative reward in each scenario is calculated, and then the average of the cumulative rewards of all scenarios is taken to reflect the average performance level of the current policy network parameters in dealing with the entire set of scenarios.
[0059] The finally obtained trained meta-policy model can generate effective optimal control strategies when facing different environment distributions.
[0060] As a preferred embodiment, step S2, the second running data under the target power grid operation scenario is obtained, and the environment latent code of the trained meta-policy model is optimized to obtain an adaptive policy model, including: Obtaining second running data under a target power grid operation scenario; Matching the second running data with the environment latent codes corresponding to each power grid operation scenario to obtain a target environment latent code; According to the second running data, the target environment latent code is optimized to obtain an adaptive policy model, and the adaptive policy model makes decisions on control actions according to the optimized target environment latent code under the target power grid operation scenario.
[0061] It should be noted that the environment characteristics of the target power grid operation scenario are different from those of the existing power grid operation scenarios, and thus can be quickly distinguished in actual operation. The second running data is a data record obtained by interaction or actual simulation under the target power grid operation scenario, and is used for targeted adaptation of the trained meta-policy model.
[0062] By matching the second running data with the environment latent code, the new target power grid operation scenario can be matched with the known scenario code library, thereby significantly accelerating the adaptation convergence process and improving the performance lower limit of the optimization result.
[0063] When performing adaptive optimization of the trained meta-policy model, the environment latent code is adjusted so that it can more accurately guide the inherent policy network parameters to produce decisions more suitable for the new scenario. The entire process fixes the huge policy network and only optimizes the low-dimensional latent code, thereby having a faster optimization speed.
[0064] In the above scheme, the meta-strategy model can quickly call historical experience for preliminary matching when facing a new target power grid operation scenario, and then quickly complete the precise customization of the adaptive strategy model through lightweight optimization, while taking into account the adaptation speed and the final performance.
[0065] As a preferred embodiment, the step S3 of inputting the real-time operation data of the power system into the adaptive strategy model to generate the power system control strategy comprises: The real-time operation data of the power system is input into the adaptive strategy model, so that the adaptive strategy model automatically matches the environment latent code according to the real-time operation data, and generates a control action; According to the control performance of the control action, a power system control strategy is generated.
[0066] The adaptive strategy model can be deployed in the power system, receive real-time operation data of the power system, and automatically determine the environment latent code currently used to identify and define the current operation scenario. After determining the environment latent code, the adaptive strategy model further combines the internal strategy network parameters to calculate a specific control action. After receiving the control action, the power system makes an expected judgment on its effect, and only when the control performance corresponding to the control action meets the preset performance threshold, a power system control strategy is generated.
[0067] In some preferred embodiments, when the control performance corresponding to the control action does not meet the preset performance threshold, it may be that the adaptive strategy model has insufficient control capability for the current power grid operation condition. The second operation data within a preset time window is collected, and the adaptive strategy model is optimized using the second operation data. If necessary, the strategy network model can also be updated.
[0068] In the above scheme, the real-time changes of the power grid can be quickly responded to, and the scenario context in which the changes are located can also be understood, and a robust control strategy can be output according to the control performance.
[0069] The power system control strategy generation method provided by the embodiment of the application can abstract and encode different power grid operation scenarios through the environment latent code to match the needs of different scenarios, and the adaptive strategy model obtained accumulates cross-scenario general knowledge and has the characteristics of fast adaptation.
[0070] The embodiment of the application provides a power system control strategy system. Please refer to Figure 2 The power system control strategy system comprises a model meta-training module 11, a model adaptive training module 12 and a model application module 13, wherein: The model meta-training module 11 is configured to obtain first operation data under a plurality of power grid operation scenarios, train a pre-constructed meta-strategy model according to the first operation data, and jointly optimize strategy network parameters of the meta-strategy model and environment latent codes corresponding to the plurality of power grid operation scenarios during the training process. The model adaptation training module 12 is configured to obtain second operation data under a target power grid operation scenario, and optimize the environment latent codes of the trained meta-strategy model to obtain an adapted strategy model, with the control performance under the second operation data being optimal as the target. The model application module 13 is configured to input real-time operation data of the power system into the adapted strategy model to generate a power system control strategy.
[0071] As a preferred implementation, the pre-construction method of the meta-strategy model includes: constructing a state space according to the dimension of the operation data of the power system; constructing an action space according to the control means of the power system; the control means includes load control and output control; constructing a reward function according to the engineering target of the power system; the engineering target is to minimize the load shedding amount and to restore the voltage to the steady state as soon as possible; constructing a meta-strategy model according to the state space, the action space, and the reward function.
[0072] As a preferred implementation, the model meta-training module 11 includes: A power grid operation scenario construction unit is configured to construct a plurality of power grid operation scenarios in a power grid simulation model of the power system; the power grid simulation model corresponds to the power system; A first operation data generation unit is configured to dynamically generate first operation data by interacting with the environment in the plurality of power grid operation scenarios according to an action space of the pre-constructed meta-strategy model; An environment latent code set construction unit is configured to train the meta-strategy model by using the first operation data to obtain environment latent codes corresponding to the plurality of power grid operation scenarios, and form an environment latent code set; A strategy network parameter optimization unit is configured to optimize strategy network parameters of the meta-strategy model according to the environment latent code set, so that the trained meta-strategy model supports dynamic adjustment of the output based on the environment latent code.
[0073] Further, preferably, the power grid operation scenario construction unit is specifically configured to: obtain load parameters of a power grid simulation model of the power system; construct a plurality of initial power grid operation scenarios by dynamically adjusting the load parameters of the power grid simulation model; An extreme event is set in the initial power grid operation scenario, and an extreme power grid operation scenario is constructed; According to the initial power grid operation scenario and the extreme power grid operation scenario, a power grid operation scenario is obtained.
[0074] Preferably, the first operation data generation unit is specifically configured to: According to the action space of the pre-constructed meta-strategy model, a control action is selected; the selection algorithm of the control action is iterated with the training of the meta-strategy model; According to the control action and the day-ahead operation characteristics of the power system, an environmental simulation interaction is performed on the power grid operation scenario in the power grid simulation model, and first operation data are obtained.
[0075] Preferably, the environment latent code set construction unit is specifically configured to: The first operation data are input into the meta-strategy model, and cumulative rewards under each power grid operation scenario are calculated by iteration; The environment latent codes corresponding to each power grid operation scenario in the meta-strategy model are updated with the maximum cumulative reward as the target; According to the environment latent codes corresponding to all power grid operation scenarios, an environment latent code set is obtained.
[0076] Preferably, the strategy network parameter optimization unit is specifically configured to: The environment latent code set is fixed; According to the environment latent code set, an average cumulative reward across power grid operation scenarios is calculated; According to the average cumulative reward, the strategy network parameters of the meta-strategy model are updated through a deep reinforcement learning algorithm, so that the trained meta-strategy model supports dynamic adjustment of the output based on the environment latent code.
[0077] As a preferred embodiment, the model adaptive training module 12 is specifically configured to: Second operation data under a target power grid operation scenario are obtained; The second operation data are matched with the environment latent codes corresponding to each power grid operation scenario, and a target environment latent code is obtained; According to the second operation data, the target environment latent code is optimized with the optimal control performance as the target, and an adaptive strategy model is obtained; the adaptive strategy model makes a decision on a control action according to the optimized target environment latent code under the target power grid operation scenario.
[0078] As a preferred embodiment, the model application module 13 is specifically configured to: The real-time operation data of the power system is input into the adaptive strategy model, so that the adaptive strategy model automatically matches the environment potential code according to the real-time operation data, and generates a control action; According to the control performance of the control action, a power system control strategy is generated.
[0079] The power system control strategy generation system provided by the embodiment of the present application can abstract and encode different power grid operation scenarios by using the environment potential code, match the requirements of different scenarios, and obtain an adaptive strategy model that accumulates cross-scene general knowledge and has a fast adaptive characteristic.
[0080] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by a computer program instructing related hardware. The program can be stored in a computer readable storage medium. When the program is executed, the program can include the processes of the above-mentioned embodiments. The storage medium can be a magnetic disc, an optical disc, a read-only memory (ROM), a random access memory (RAM), or the like.
[0081] The above-mentioned is the preferred embodiment of the present application. It should be pointed out that, for those skilled in the art, without departing from the principle of the present application, a number of improvements and refinements can be made, which are also considered to be within the protection scope of the present application.
Claims
1. A method for generating a power system control strategy, characterized in that, include: First operational data under several power grid operation scenarios are obtained. Based on the first operational data, a pre-constructed meta-policy model is trained. During the training process, the policy network parameters of the meta-policy model and the environmental latent codes corresponding to each power grid operation scenario are jointly optimized. The second operating data under the target power grid operation scenario is obtained. With the goal of achieving the best control performance under the second operating data, the environmental latent code of the trained meta-policy model is optimized to obtain the adapted policy model. The real-time operating data of the power system is input into the adaptation strategy model to generate the power system control strategy.
2. The power system control strategy generation method as described in claim 1, characterized in that, The pre-construction method for the meta-policy model includes: Construct a state space based on the dimensions of power system operation data; Based on the control methods of the power system, an action space is constructed; the control methods include load control and output control. A reward function is constructed based on the engineering objectives of the power system; the engineering objectives are to minimize the load shedding and to restore the voltage to steady state as quickly as possible. A meta-policy model is constructed based on the state space, the action space, and the reward function.
3. The power system control strategy generation method as described in claim 1, characterized in that, The process involves acquiring first operational data under several power grid operation scenarios, training a pre-constructed meta-policy model based on the first operational data, and jointly optimizing the policy network parameters of the meta-policy model and the environmental latent codes corresponding to each power grid operation scenario during the training process, including: Several power grid operation scenarios are constructed in the power grid simulation model of the power system; the power grid simulation model corresponds to the power system. Based on the action space of the pre-constructed meta-strategy model, environmental interaction is performed in the power grid operation scenario to dynamically generate the first operation data; Using the first operational data, the meta-strategy model is trained to obtain the environmental latent codes corresponding to each power grid operation scenario, forming an environmental latent code set; Based on the set of environmental latent codes, the policy network parameters of the meta-policy model are optimized so that the trained meta-policy model can support dynamic adjustment of the output based on the environmental latent codes.
4. The power system control strategy generation method as described in claim 3, characterized in that, The construction of several power grid operation scenarios in the power grid simulation model of the power system includes: Obtain the load parameters of the power grid simulation model of the power system; Several initial power grid operation scenarios are constructed by dynamically adjusting the load parameters of the power grid simulation model. Extreme power grid operation scenarios are constructed by setting extreme events under the initial power grid operation scenario. Based on the initial power grid operation scenario and the extreme power grid operation scenario, the power grid operation scenario is obtained.
5. The power system control strategy generation method as described in claim 3, characterized in that, The step of interacting with the power grid operation scenario based on the action space of the pre-constructed meta-policy model to dynamically generate first operation data includes: Based on the action space of the pre-constructed meta-policy model, a control action is selected; the algorithm for selecting the control action iterates as the meta-policy model is trained. Based on the control actions and the day-ahead operating characteristics of the power system, environmental simulation interaction is performed on the power grid operating scenario in the power grid simulation model to obtain the first operating data.
6. The power system control strategy generation method as described in claim 3, characterized in that, The first operational data is used to train the meta-strategy model to obtain environmental latent codes corresponding to each power grid operation scenario, forming an environmental latent code set, including: The first set of operational data is input into the meta-strategy model, and the cumulative reward under each power grid operation scenario is calculated iteratively. With the goal of maximizing the cumulative reward, the environmental latent codes corresponding to each power grid operation scenario in the meta-strategy model are updated; Based on the environmental latent codes corresponding to all power grid operation scenarios, an environmental latent code set is obtained.
7. The power system control strategy generation method as described in claim 3, characterized in that, The step of optimizing the policy network parameters of the meta-policy model based on the set of environmental latent codes, so that the trained meta-policy model can support dynamic adjustment of the output based on the environmental latent codes, includes: Fix the set of environmental latent codes; Calculate the average cumulative reward for cross-power grid operation scenarios based on the environmental latent code set; Based on the average cumulative reward, the policy network parameters of the meta-policy model are updated using a deep reinforcement learning algorithm, so that the trained meta-policy model can dynamically adjust the output based on the environment latent code.
8. The power system control strategy generation method as described in claim 1, characterized in that, The process of acquiring second operational data under the target power grid operation scenario, aiming to achieve optimal control performance under the second operational data, optimizes the environmental latent code of the trained meta-policy model to obtain an adapted policy model, including: Acquire second operational data under the target power grid operation scenario; The second operating data is matched with the environmental latent code corresponding to each power grid operating scenario to obtain the target environmental latent code; Based on the second operating data, with the goal of achieving optimal control performance, the target environment latent code is optimized to obtain an adaptation strategy model; the adaptation strategy model makes control action decisions based on the optimized target environment latent code under the target power grid operating scenario.
9. The power system control strategy generation method as described in claim 1, characterized in that, The step of inputting real-time operating data of the power system into the adaptation strategy model to generate a power system control strategy includes: The real-time operation data of the power system is input into the adaptation strategy model so that the adaptation strategy model can automatically match the environmental latent code according to the real-time operation data and generate control actions. Based on the control performance of the control actions, a power system control strategy is generated.
10. A power system control strategy generation system, characterized in that, include: The model meta-training module is used to acquire first operating data under several power grid operation scenarios, and to train the pre-constructed meta-policy model based on the first operating data. During the training process, the policy network parameters of the meta-policy model and the environmental latent codes corresponding to each power grid operation scenario are jointly optimized. The model adaptation training module is used to obtain the second operating data under the target power grid operation scenario. With the goal of achieving the best control performance under the second operating data, the environmental latent code of the trained meta-policy model is optimized to obtain the adapted policy model. The model application module is used to input real-time operating data of the power system into the adaptation strategy model to generate power system control strategies.