Power grid optimization scheduling method and related device

Through the multi-adversarial reinforcement learning method, combined with the generative multi-adversarial network and the deep deterministic action gradient model, the problems of insufficient future profit fitting and poor adaptability of complex scenarios in power grid scheduling are solved, and the operation prediction and stability of the power grid are improved.

CN119009988APending Publication Date: 2024-11-22CHINA ELECTRIC POWER RESEARCH INSTITUTE CO LTD +4
View PDF 0 Cites 4 Cited by

Patent Information

Application Number
CN202411108347.8
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-13
Publication Date
2024-11-22

AI Technical Summary

Technical Problem

The existing grid scheduling method based on reinforcement learning lacks fitting estimates of the total returns in the future scheduling time period, resulting in insufficient ability to predict future operating trends, and is not adaptable to complex scenarios, making it difficult to access high proportions in new energy. Maintain the safety and stability of the power grid in extreme weather conditions.

Method used

Adopting a grid optimization scheduling method based on multi-adversarial reinforcement learning, the pre-trained generative multi-adversarial network model predicts the future potential rewards of the power grid, combined with the deep deterministic action gradient model, the target Q value of the critic network is adjusted to consider the immediate reward and Potential rewards in the future, achieving comprehensive consideration of power grid scheduling at all times.

Benefits of technology

It improves the ability to predict future operating trends, enhances the adaptability and stability of power grid scheduling, can better adapt to power grid fluctuations and complex scenarios, and maximizes economic benefits.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119009988A_ABST
    Figure CN119009988A_ABST
Patent Text Reader

Abstract

The invention belongs to the field of power grid dispatching, and discloses a power grid optimization dispatching method and a related device, and the method comprises the steps: obtaining a current operation state of a power grid; inputting the current operation state of the power grid into a pre-trained power grid optimization scheduling model based on multi-adversarial reinforcement learning to obtain a current operation state adjustment action of the power grid; wherein the power grid optimization scheduling model based on multi-adversarial reinforcement learning is obtained by modifying a target Q value of a commentator network in a depth deterministic action gradient model during training into the sum of predicted values of an instant reward and a future potential reward of a power grid; wherein the future potential reward prediction value of the power grid is obtained through prediction of a pre-trained generative multi-adversarial network model. A generative multi-adversarial network model is introduced to replace a target network to pre-sense operation states in all decision-making moments in the future, the defect that in traditional reinforcement learning, an evaluation network only evaluates and fits the next target income is overcome, and the power grid fluctuation condition is better adapted.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The invention belongs to the field of power grid dispatching, and relates to a power grid optimization dispatching method and related devices. Background Art

[0002] With the continuous evolution of artificial intelligence, it has become a universal technology leading the fourth industrial revolution, fully penetrating into various industries and continuously integrating and developing. In the field of control, many scholars have applied artificial intelligence technology to carry out a lot of research and practice in dispatching and operation auxiliary decision-making, new energy prediction and safety and stability analysis. In particular, autonomous learning models represented by deep reinforcement learning have become the mainstream method for solving various optimization decision-making problems. Some scholars have proposed a decentralized local voltage control method based on deep reinforcement learning, which realizes the decentralized local reactive voltage control of multi-agents that does not rely on communication when the measurement data is insufficient, effectively suppresses voltage fluctuations, and prevents node voltage from exceeding the limit. Some scholars have proposed a method for automatically generating static safety preventive control actions of dual-agents based on deep reinforcement learning. A PQ decoupled dual-agent structure with centralized training is designed, in which two agents respectively undertake the tasks of generator active power adjustment and voltage adjustment, realizing static safety preventive control of the power system. Some scholars have proposed an electric vehicle charging scheduling method considering new energy uncertainty and electricity prices. This method establishes a node multi-objective model of the optimal scheduling action, and adopts a flexible action evaluation algorithm to solve it, realizing the rapid calculation of large-scale electric vehicle charging scheduling actions.

[0003] In the optimization and scheduling of continuous time scales, deep reinforcement learning algorithms are widely used to solve complex scenario scheduling problems because they do not require precise modeling and iterative solutions compared to model-driven algorithms. Some scholars have proposed a dynamic energy scheduling method for integrated energy systems based on constrained reinforcement learning algorithms, which effectively constrains the output of units and realizes safe scheduling of system operation. In view of the growing data scale of power systems, many studies have improved reinforcement learning algorithms in terms of training efficiency and stability. Some scholars have proposed a multi-task deep reinforcement learning optimization scheduling method that integrates grid operation scenario clustering. By identifying the scenario categories of limited operating data, the model is called to quickly solve real-time scheduling tasks, realizing multi-task optimization scheduling in random scenarios. Some scholars have adopted the idea of ​​distributed optimization scheduling based on multi-agent deep reinforcement learning, converting the centralized scheduling model into a distributed optimization problem among multiple agents for solution, thereby improving the training speed and scope of application.

[0004] However, the above existing grid dispatching methods based on reinforcement learning have the following shortcomings. First, there is a lack of fitting estimates of the total revenue in the future dispatching time period, and the ability to predict future operating trends is insufficient. Especially when a high proportion of new energy is connected, the uncertainty of grid operation increases, making it difficult for traditional methods to accurately capture the dynamic changes of the grid. In the process of intraday optimization scheduling, the formulation of dispatching actions usually needs to take into account changes in the future. However, existing methods often lack in-depth consideration of this point, resulting in dispatching actions that may not be optimal, and unable to make full use of grid resources to maximize economic benefits. In addition, the adaptability to complex scenarios is not strong. During the operation of smart grids, various complex scenarios will be faced, such as new energy fluctuations, equipment failures and extreme weather. When dealing with these complex scenarios, existing dispatching methods often find it difficult to find effective response actions, resulting in threats to the safety and stability of grid operation. Summary of the invention

[0005] The purpose of the present invention is to overcome the above-mentioned shortcomings of the prior art and provide a power grid optimization scheduling method and related devices.

[0006] In order to achieve the above object, the present invention adopts the following technical solutions:

[0007] In a first aspect, the present invention provides a method for optimizing and dispatching a power grid, comprising: obtaining a current operating state of a power grid; inputting the current operating state of the power grid into a pre-trained power grid optimization and dispatching model based on multi-adversarial reinforcement learning to obtain an adjustment action for the current operating state of the power grid; wherein the power grid optimization and dispatching model based on multi-adversarial reinforcement learning is obtained by modifying the target Q value of the critic network in a deep deterministic action gradient model during training to the sum of an immediate reward and a predicted value of a future potential reward of the power grid; wherein the predicted value of the future potential reward of the power grid is based on the operating state of the power grid during training and the intraday operating state adjustment action, and is predicted using a pre-trained generative multi-adversarial network model.

[0008] Optionally, during pre-training, the pre-trained generative multi-adversarial network model: the optimization target Y of the generator of the generative multi-adversarial network model * for:

[0009] Y * = arg min Y max D V(Y)

[0010] Among them, Y is the generator of the generative multi-adversarial network model, D is the discriminator of the generative multi-adversarial network model, and V(Y) is the divergence required with the generator as the variable.

[0011] Optionally, the V(Y):

[0012]

[0013] Among them, λ is the parameter for adjusting the degree of fusion, ω i is the fusion degree weight of the i-th discriminator, I is the total number of discriminators, V' i (D i ,Y) is the divergence obtained by the i-th discriminator and generator;

[0014]

[0015] Among them, R t is the cumulative reward score from the beginning to time t, z is a predefined simple distribution, and its distribution is z~P z ,Y(z,[s t ,a t ]) is a combination of z and [s t ,a t ] is the divergence of the variable, s t is the agent state space, a t is the agent action space, E[·] is the expectation, R~P R is the probability distribution of reward scores.

[0016] Optionally, during pre-training, the pre-trained generative multi-adversarial network model updates model parameters of the generative multi-adversarial network model by means of stochastic gradient descent.

[0017] Optionally, during pre-training, the pre-trained power grid optimization scheduling model based on multi-adversarial reinforcement learning: based on a preset number of historical experience extractions, good historical experience is extracted from a good historical experience sub-pool of a historical experience pool in combination with a good historical experience extraction ratio, and bad historical experience is extracted from a bad historical experience sub-pool of a historical experience pool in combination with a bad historical experience extraction ratio; and pre-training of the power grid optimization scheduling model based on multi-adversarial reinforcement learning is performed based on the extracted good historical experience and bad historical experience.

[0018] Optionally, it also includes: based on a preset instant reward threshold, classifying historical experience in the historical experience pool whose instant rewards are greater than the instant reward threshold as good historical experience, and classifying historical experience in the historical experience pool whose instant rewards are not greater than the instant reward threshold as bad historical experience, to obtain a good historical experience sub-pool and a bad historical experience sub-pool.

[0019] According to a second aspect of the present invention, a power grid optimization and dispatching system is provided, comprising: a data acquisition module for acquiring the current operating state of the power grid; an optimization and dispatching module for inputting the current operating state of the power grid into a pre-trained power grid optimization and dispatching model based on multi-adversarial reinforcement learning to obtain the current operating state adjustment action of the power grid; wherein the power grid optimization and dispatching model based on multi-adversarial reinforcement learning is obtained by modifying the target Q value of the critic network in the deep deterministic action gradient model during training to the sum of the immediate reward and the predicted value of the future potential reward of the power grid; wherein the predicted value of the future potential reward of the power grid is based on the operating state of the power grid during training and the intraday operating state adjustment action, and is predicted by a pre-trained generative multi-adversarial network model.

[0020] Optionally, during pre-training, the pre-trained power grid optimization scheduling model based on multi-adversarial reinforcement learning: based on a preset number of historical experience extractions, good historical experience is extracted from a good historical experience sub-pool of a historical experience pool in combination with a good historical experience extraction ratio, and bad historical experience is extracted from a bad historical experience sub-pool of a historical experience pool in combination with a bad historical experience extraction ratio; and pre-training of the power grid optimization scheduling model based on multi-adversarial reinforcement learning is performed based on the extracted good historical experience and bad historical experience.

[0021] According to a third aspect of the present invention, a computer device is provided, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the above-mentioned power grid optimization and scheduling method when executing the computer program.

[0022] According to a fourth aspect of the present invention, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the above-mentioned power grid optimization scheduling method are implemented.

[0023] Compared with the prior art, the present invention has the following beneficial effects:

[0024] The power grid optimization dispatching method of the present invention obtains the current operation state adjustment action of the power grid through the pre-trained power grid optimization dispatching model based on multi-adversarial reinforcement learning based on the current operation state of the power grid, and realizes the power grid optimization dispatching. Among them, the power grid optimization dispatching model based on multi-adversarial reinforcement learning is obtained by modifying the target Q value of the critic network in the deep deterministic action gradient model during training to the sum of the immediate reward and the predicted value of the future potential reward of the power grid, and the predicted value of the future potential reward of the power grid is based on the operation state of the power grid during training and the adjustment action of the intra-day operation state, and is predicted by the pre-trained generative multi-adversarial network model. By introducing the generative multi-adversarial network model, the operation state in all future decision moments is pre-perceived and used as part of the target Q value of the critic network during training, thereby effectively compensating for the shortcomings of the critic network in traditional reinforcement learning that only fits the next step target benefit evaluation, improving the ability of the power grid optimization dispatching model based on multi-adversarial reinforcement learning to predict future operation trends, realizing the comprehensive consideration of power grid dispatching at all times, and being able to better adapt to power grid fluctuations and complex scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] Figure 1 This is a flow chart of a method for optimizing power grid scheduling according to an embodiment of the present invention.

[0026] Figure 2 This is an architecture diagram of a power grid optimization scheduling model based on multi-adversarial reinforcement learning according to an embodiment of the present invention.

[0027] Figure 3 Schematic diagram of the hybrid empirical cross-driving mechanism principle of an embodiment of the present invention.

[0028] Figure 4 This is a schematic diagram of the new energy consumption rate at each step in scenario one of the comparative experiment of an embodiment of the present invention.

[0029] Figure 5 This is a schematic diagram of the operating cost of each unit in scenario 1 under the comparative experiment of an embodiment of the present invention.

[0030] Figure 6 This is a schematic diagram of the cumulative reward value for each step in scenario one of the comparative experiment of an embodiment of the present invention.

[0031] Figure 7 This is a structural block diagram of a power grid optimization dispatching system according to an embodiment of the present invention. DETAILED DESCRIPTION

[0032] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0033] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0034] The present invention is further described in detail below in conjunction with the accompanying drawings:

[0035] See also Figure 1 In one embodiment of the present invention, a power grid optimization scheduling method is provided, which can effectively solve the problem of insufficient estimation of the total revenue fitting in the future scheduling time period in the existing power grid scheduling method, and the technical difficulty of weak prediction ability of future operation trends caused by this.

[0036] Specifically, the power grid optimization dispatching method of the present invention comprises the following steps:

[0037] S1: Get the current operating status of the power grid.

[0038] S2: Input the current operating state of the power grid into the pre-trained power grid optimization scheduling model based on multi-adversarial reinforcement learning to obtain the current operating state adjustment action of the power grid.

[0039] Among them, the power grid optimization scheduling model based on multi-adversarial reinforcement learning is obtained by modifying the target Q value of the critic network in the deep deterministic action gradient model during training to the sum of the immediate reward and the predicted value of the future potential reward of the power grid; among them, the predicted value of the future potential reward of the power grid is based on the operating status of the power grid during training and the daily operating status adjustment action, and is predicted by a pre-trained generative multi-adversarial network model.

[0040] The power grid optimization dispatching method of the present invention obtains the current operation state adjustment action of the power grid through the pre-trained power grid optimization dispatching model based on multi-adversarial reinforcement learning based on the current operation state of the power grid, and realizes the power grid optimization dispatching. Among them, the power grid optimization dispatching model based on multi-adversarial reinforcement learning is obtained by modifying the target Q value of the critic network in the deep deterministic action gradient model during training to the sum of the immediate reward and the predicted value of the future potential reward of the power grid, and the predicted value of the future potential reward of the power grid is based on the operation state of the power grid during training and the adjustment action of the intra-day operation state, and is predicted by the pre-trained generative multi-adversarial network model. By introducing the generative multi-adversarial network model, the operation state in all future decision moments is pre-perceived and used as part of the target Q value of the critic network during training, thereby effectively compensating for the shortcomings of the critic network in traditional reinforcement learning that only fits the next step target benefit evaluation, improving the ability of the power grid optimization dispatching model based on multi-adversarial reinforcement learning to predict future operation trends, realizing the comprehensive consideration of power grid dispatching at all times, and being able to better adapt to power grid fluctuations and complex scenarios.

[0041] Specifically, the power grid optimization dispatching model based on multi-adversarial reinforcement learning builds an intelligent agent based on the deep deterministic action gradient model DDPG and the generative multi-adversarial network model GMAN to solve the intraday optimization dispatching problem. GMAN is used as the target network of the intelligent agent to learn the potential relationship between the "state-action" of the power grid and the potential future rewards of the power grid, and then predict the future operation status of the power grid.

[0042] See also Figure 2 The power grid optimization dispatching model based on multi-adversarial reinforcement learning mainly includes the following core components: simulation environment, intelligent agent, actor network, critic network, generator and multiple discriminators. Among them, the simulation environment represents the current operating state of the power grid, including operating state information such as generator sets, lines and loads; the intelligent agent makes decision actions based on the current operating state of the power grid, that is, the intraday operating state adjustment actions, which include the actor network and the critic network for generating and evaluating decision actions; the generator will learn the distribution of the future potential rewards of the power grid based on historical data, and then generate the expectation of future cumulative benefits based on the current operating state of the current power grid; multiple discriminators are established in GMAN for game, and multiple discriminators will assist the generator in learning and convergence, making the generated prediction data more and more realistic. During the training process, DDPG updates the parameters of the actor network and the critic network based on the predicted value of the future potential reward of the power grid generated by GMAN, which enhances the foresight of reinforcement learning decisions and improves the convergence speed and stability of the network.

[0043] DDPG is an efficient reinforcement learning method for continuous action spaces. It significantly improves the quality of decision making in offline learning environments by combining the advantages of action gradients and value functions. DDPG uses neural networks to approximate action functions and value functions, and uses deep learning techniques to train these neural networks to find the optimal action in complex continuous action spaces.

[0044] The training process of DDPG follows the following steps: First, the main network including the Actor network and the Critic network, as well as the corresponding target networks Target Actor and Target Critic, are initialized. The design of the target network is intended to stabilize the learning process and reduce the instability caused by the rapid changes of the main network. Next, the simulation environment is initialized to obtain the environmental state, that is, the operating state of the power grid s t , which enables DDPG to learn and test under realistic conditions. DDPG then t =μ(s t |θ μ ) performs actions and obtains feedback from the environment, including rewards r t , new state t+1 And the round end mark d. After that, these data are stored as historical experience in the historical experience pool for subsequent learning, and the environment state is updated to s t+1 .

[0045] The DDPG network update process includes the following key steps:

[0046] 1) Historical experience pool sampling: extract M historical experiences M*(s) from the historical experience pool t ,a t ,r t ,s t+1 ,d),These historical experiences contain historical information and feedback of power grid operation.

[0047] 2) Actor network prediction: t+1 Input Actor network to get a' t+1 =μ'(s t+1 |θ μ' ).

[0048] 3) Evaluation of Critic Network: t+1 and a' t+1 Input the Critic network and get the expected reward value q' t+1 =Q'(s t+1 ,a' t+1 |θ Q' ).

[0049] 4) Calculate the target Q value of the Critic network: target_q = r t +γ(1-d)q' t+1 .

[0050] 5) Update of Critic network: t ,a t Input Critic network and calculate q = Q(s t ,a t |θ Q ) and calculate the loss function of the Critic network (taking the mean square error (MSE) as an example): Utilizing loss c Perform gradient backpropagation and update the Critic network.

[0051] 6) Update of Actor network: Use Actor network to make new action decisions a' t =μ(s t |θ μ ), calculate the Actor network loss function: loss a =-Q(s t ,a' t |θQ); using loss a Perform gradient backflow and update the Actor network.

[0052] In a possible implementation, the pre-trained generative multi-adversarial network model is pre-trained:

[0053] The optimization objective Y of the generator of the generative multi-adversarial network model * for:

[0054] Y * =argmin Y max D V(Y)

[0055] Among them, Y is the generator of the generative multi-adversarial network model, D is the discriminator of the generative multi-adversarial network model, and V(Y) is the divergence required with the generator as the variable.

[0056] Specifically, GMAN is used to estimate the potential future rewards and use this to guide the training process of DDPG. At the same time, GMAN replays the training generator based on historical data to achieve the learning of the future potential rewards of the power grid within the decision cycle. Specifically, assuming that the state of the agent at time step t is s t , the action taken is a t , then the cumulative reward obtained from time t to the end of the round is It can be expressed as:

[0057]

[0058] Where γ is the discount factor, It's in time The instant reward obtained. The generator of GMAN (denoted as Y, with parameter θ Y ) aims to learn the cumulative reward The distribution of , and make it output the cumulative reward prediction value Y(z,[s t ,a t ]); where z~P z It is a predefined simple distribution, which is convenient for drawing samples from. t ,a t ]) is used to enhance the training process of DDPG and make the DDPG strategy forward-looking. At the same time, the generated cumulative reward also effectively promotes the convergence of reinforcement learning. Specifically, in this embodiment, Y(z,[s t ,a t ]) It is also used as part of the target Q value of the DDPG Critic network during training, namely:

[0059] target_q=r t +Y(z,[s t ,a t ])

[0060] Therefore, the optimization goal Y of the GMAN generator is * It can be expressed as:

[0061] Y * =argmin Y DIV(P Y ,P R )

[0062] Among them, Y * Optimize the target for the generator; DIV(P Y ,P R ) is (P Y ,P R ) divergence, Y~P Y , R~P R .

[0063] In this implementation, the authenticity of the cumulative reward is evaluated by a group of discriminators, and the evaluation results of the cumulative reward by each discriminator act together on the parameter update of the generator. In GMAN, the generator updates the parameters by playing a maximum-minimum game with I homogeneous discriminators. i (The parameter is θ Di ), whose optimization goal is to distinguish the data generated by generator Y from the original data:

[0064]

[0065] Among them, R t is the cumulative reward score from the beginning to time t, z is a predefined simple distribution, and its distribution is z~P z ,Y(z,[s t ,a t ]) is a combination of z and [s t ,a t ] is the divergence of the variable, s t is the agent state space, a t is the agent action space, E[·] is the expectation, R~P R is the probability distribution of reward scores.

[0066] Through Jensen-Shannon divergence, the optimization objective Y of the generator of GMAN is * It can be expressed as:

[0067] Y * =argmin Y max D V(Y)

[0068] In a possible implementation, the V(Y):

[0069]

[0070] Among them, V(Y) is the divergence required with the generator as the variable; λ is the parameter for adjusting the degree of fusion, λ=0 corresponds to directly taking the mean, ω i is the fusion degree weight of the i-th discriminator, I is the total number of discriminators, V' i (D i ,Y) is the divergence obtained by the i-th discriminator and generator.

[0071] Specifically, considering that the instability of a single discriminator may hinder the learning of the generator, in order to ensure the stability of convergence, this implementation adopts the classic Pythagorean method as the fusion function F soft To fuse the outputs of multiple discriminators:

[0072] In a possible implementation, during pre-training, the pre-trained generative multi-adversarial network model updates model parameters of the generative multi-adversarial network model by means of stochastic gradient descent.

[0073] Specifically, calculate the discriminator D i Gradient and the gradient J of the generator Y Y :

[0074]

[0075] And update their parameters through stochastic gradient descent. Therefore, through historical experience replay training, the generator can generate cumulative reward prediction values ​​based on the current state and action of the agent. When used for power grid optimization scheduling, the cumulative reward prediction value is the future potential reward prediction value of the power grid.

[0076] In one possible implementation, the pre-trained power grid optimization scheduling model based on multi-adversarial reinforcement learning is pre-trained: based on a preset number of historical experience extractions, good historical experience is extracted from a good historical experience sub-pool of a historical experience pool in combination with a good historical experience extraction ratio, and bad historical experience is extracted from a bad historical experience sub-pool of a historical experience pool in combination with a bad historical experience extraction ratio; and pre-training of the power grid optimization scheduling model based on multi-adversarial reinforcement learning is performed based on the extracted good historical experience and bad historical experience.

[0077] Specifically, in order to improve the learning efficiency and adaptability of the power grid optimization dispatching model based on multi-adversarial reinforcement learning, a hybrid experience cross-driving mechanism is introduced in this embodiment. The hybrid experience cross-driving mechanism classifies and manages the historical experience of the intelligent agent and the power grid environment in detail to more effectively utilize the historical experience and improve the performance of the power grid optimization dispatching model based on multi-adversarial reinforcement learning.

[0078] Optionally, the power grid optimization scheduling method also includes: based on a preset instant reward threshold, classifying historical experience in the historical experience pool whose instant rewards are greater than the instant reward threshold as good historical experience, and classifying historical experience in the historical experience pool whose instant rewards are not greater than the instant reward threshold as bad historical experience, to obtain a good historical experience sub-pool and a bad historical experience sub-pool.

[0079] Specifically, in each interaction with the power grid environment, the historical experience generated by the agent of the power grid optimization dispatching model based on multi-adversarial reinforcement learning is finely classified. By introducing an immediate reward threshold, the historical experience is divided into two categories: good historical experience and bad historical experience.

[0080] Among them, good historical experience refers to historical experience with an immediate reward greater than the immediate reward threshold, representing a decision that is beneficial to the system. Good historical experience is stored in the good historical experience sub-pool, forming a sample set of historical experience that is beneficial to the task. Bad historical experience: refers to historical experience with an immediate reward that is not greater than the immediate reward threshold. Classifying historical experience as bad historical experience may require avoiding similar decisions. Bad historical experience is stored in the bad historical experience pool, forming a sample set of historical experience that is not beneficial to the task.

[0081] See also Figure 3, the key to the hybrid experience cross-driving mechanism lies in experience extraction. In this embodiment, the good historical experience extraction ratio and the bad historical experience extraction ratio are combined to extract good historical experience from the good historical experience sub-pool and bad historical experience from the bad historical experience sub-pool, respectively. Such an extraction mechanism helps to balance the learning of good historical experience and bad historical experience, making the power grid optimization scheduling model based on multi-adversarial reinforcement learning more robust. The extracted historical experience is used for the training of the power grid optimization scheduling model based on multi-adversarial reinforcement learning. This flexible experience extraction mechanism effectively combines different categories of historical experience, making the power grid optimization scheduling model based on multi-adversarial reinforcement learning more adaptable. The hybrid experience cross-driving mechanism introduces a more sophisticated experience management strategy for the deep reinforcement learning power grid optimization scheduling algorithm. By distinguishing good historical experience from bad historical experience, the power grid optimization scheduling model based on multi-adversarial reinforcement learning can use historical experience more specifically, making the power grid optimization scheduling model based on multi-adversarial reinforcement learning more adaptable to complex power grid scenarios, providing an innovative historical experience playback strategy for deep reinforcement learning power grid optimization scheduling, and providing in-depth theoretical support for the research of reinforcement learning power grid optimization scheduling algorithm.

[0082] In a possible implementation, a training and decision-making process of a power grid optimization scheduling model based on multi-adversarial reinforcement learning is specifically described.

[0083] The training process of the power grid optimization dispatching model based on multi-adversarial reinforcement learning:

[0084] 1) Network initialization: (1) Initialize the Actor network and the Critic network. (2) Initialize the generative multi-adversarial network model. (3) Initialize the simulation environment and obtain the initial state s t .

[0085] 2) Action decision and execution: (1) Use the Actor network to determine the current state s t Decide on action a t (2) Execute action a in the simulation environment t Collect new status s t+1 、Instant Rewards t and finish mark d.

[0086] 3) Experience playback: (1) Historical experience Store in the historical experience pool. (2) If the historical experience pool has sufficient historical experience, randomly select a batch of historical experience pools for training.

[0087] 4) Network update: (1) Use the generative multi-adversarial network model to estimate the predicted value of the future potential reward of the power grid. (2) According to the immediate reward r tand the discounted future potential reward forecast of the power grid to calculate the target Q value. (3) Calculate the current Q value q = Q(s t ,a t |θ Q ) and the target Q value target_q, and update the Critic network to reduce this error. (4) Update the Actor network with the goal of maximizing the expected reward evaluated by the Critic network.

[0088] 5) Generate network training: (1) Calculate the cumulative reward R corresponding to each historical experience in the round t (2) Select the state s in the current historical experience t and action a t (3) From a distribution P z (4) Using the generator Y, random noise z and input [s t ,a t ] to generate the predicted value of the future potential reward of the power grid. (5) Using multiple discriminators D i The generated predictions of the future potential rewards of the power grid are evaluated and the generator is updated to improve the authenticity of its generation. (6) The outputs of all discriminators are integrated to optimize the generator. (7) The discriminator is optimized to improve its ability to distinguish between generated data and original data.

[0089] 6) Repeat the above steps until the preset termination condition is reached.

[0090] Decision-making process of the power grid optimization dispatching model based on multi-adversarial reinforcement learning:

[0091] 1) State observation: Observe the current operating status of the power grid, that is, obtain the current output of each generator set in the power grid and the environmental status such as load demand.

[0092] 2) Action generation: Based on the current environmental state, the Actor network of the power grid optimization scheduling model based on multi-adversarial reinforcement learning generates an action, namely the intraday operation state adjustment action, which involves adjusting the output of the generator set to meet the load demand while maintaining the stability of the power grid.

[0093] 3) Action adjustment: The generated intraday operating status adjustment actions may need to be adjusted according to the actual operating constraints of the power grid to ensure their practical feasibility.

[0094] In a possible implementation, in order to verify the effectiveness of the power grid optimization scheduling model based on multi-adversarial reinforcement learning, taking the SG-126 node network model as an example, ablation experiments, comparative experiments and new energy uncertainty analysis experiments were carried out. In this implementation, the power grid optimization scheduling model based on multi-adversarial reinforcement learning is constructed by a neural network, including 1 generator, 2 discriminators, 1 Actor network and 1 Critic network. Among them, the input dimension of the generator is 1735, which is reduced to 1 through multiple layers of fully connected layers and activation functions; the two discriminators are the same, the input dimension is 1, and the dimension is increased to 128 through multiple layers of fully connected layers and then reduced to 1; the input dimension of the Actor network is 1735, which is reduced to 54 through multiple layers of fully connected layers, data normalization and activation functions; the input dimension of the Critic network is 1789, which is reduced to 1 through multiple layers of fully connected layers, data normalization and activation functions; in order to avoid gradient vanishing and gradient explosion, the LeakyReLU activation function is used in the generator and the discriminator, and the ELU activation function is used in the Actor network and the Critic network. The hyperparameter settings of the power grid optimization scheduling model based on multi-adversarial reinforcement learning are shown in Table 1.

[0095] Table 1

[0096] Hyperparameters Value Hyperparameters Value Generator learning rate 1e-11 Batch size 64 Discriminator learning rate 1e-11 Reward value discount factor 0.2 Actor network learning rate 5e-17 Historical experience pool capacity 1000000 Critic network learning rate 5e-15 Historical experience pool cutoff value 0.7

[0097] Regarding the ablation experiment: In order to solve the problem of optimizing the scheduling of power grids with a high proportion of new energy, two scheduling scenarios are selected for online decision making to evaluate the effect of the power grid optimization scheduling model based on multi-adversarial reinforcement learning of the present invention. Each scenario requires continuous optimization and adjustment of the unit output of each generator in the power grid in the next 4 hours (5min intervals) according to the operating status of the power grid. The simulation results of scenario 1 under the ablation experiment are shown in Table 2.

[0098] Table 2

[0099]

[0100] As shown in Table 2, A represents DDPG, B represents GMAN, and C represents the hybrid experience cross-driving mechanism. It can be seen that the model without the hybrid experience cross-driving mechanism ends 19 steps earlier than the power grid optimization scheduling model based on multi-adversarial reinforcement learning of the present invention, the safety score decreases by 18.601%, and the reward value also decreases by 19.272%. The results show that the hybrid experience cross-driving mechanism plays a key role in ensuring the stable operation of the power grid system. The model without GMAN ends 34 steps earlier than the power grid optimization scheduling model based on multi-adversarial reinforcement learning of the present invention, and the safety score is significantly reduced, with a decrease of up to 39.413%. In addition, the operating cost of the unit increased by 1.914% and the reward value decreased by 36.8573%. These results show that GMAN plays an important role in assisting power grid scheduling and reducing the negative impact of renewable energy fluctuations on the power grid. When this mechanism is removed, the stability and economy of the power grid system are significantly affected.

[0101] The simulation results of scenario 2 under the ablation experiment are shown in Table 3.

[0102] Table 3

[0103]

[0104]

[0105] As shown in Table 3, the model without hybrid experience cross-driving mechanism ends 5 steps earlier than the power grid optimization dispatching model based on multi-adversarial reinforcement learning, the safety score is reduced by 14.633%, and the reward value is reduced by 8.248%. This shows that the hybrid experience cross-driving mechanism plays a key role in ensuring the stable operation of the power grid system and maintaining a high reward value. The model without GMAN ends 12 steps earlier than the power grid optimization dispatching model based on multi-adversarial reinforcement learning, the safety score is reduced by 12.860%, and the reward value is reduced by 12.003%. This shows that GMAN can ensure the safe operation of the power grid system and reduce the negative impact of renewable energy fluctuations.

[0106] The performance of the model without GMAN and the model without mixed experience cross-driving mechanism is compared with the grid optimization dispatching model based on multi-adversarial reinforcement learning proposed in this invention through ablation experiments. The experimental results show that both GMAN and mixed experience cross-driving mechanism play an indispensable role in improving the robustness of grid dispatching strategy, ensuring the safe operation of grid system and mitigating the negative impact of renewable energy fluctuations.

[0107] In addition, the grid optimization dispatching model based on multi-adversarial reinforcement learning proposed in the present invention also achieves a significant improvement in the new energy consumption rate, reaching a high level of more than 98%, and realizes the efficient consumption of new energy. The experimental results show that the grid optimization dispatching model based on multi-adversarial reinforcement learning proposed in the present invention shows significant advantages in optimizing the dispatching strategy. Compared with the model without GMAN, the grid optimization dispatching model based on multi-adversarial reinforcement learning proposed in the present invention reduces the unit operating costs while maintaining a high safety score and maintains a high reward value. This is because GMAN effectively reduces the negative impact of renewable energy fluctuations on the power grid and improves the stability and economy of the dispatching strategy. Compared with the model without the hybrid experience cross-driving mechanism, a higher safety score and reward value are achieved while maintaining a high number of steps. This proves that the hybrid experience cross-driving mechanism plays a key role in improving the efficiency and robustness of strategy scheduling. This shows that both GMAN and the hybrid experience cross-driving mechanism play an important role in the model, and jointly improve the performance of the grid optimization dispatching model based on multi-adversarial reinforcement learning.

[0108] About the comparative experiment: The power grid optimization scheduling model based on multi-adversarial reinforcement learning is compared with other power grid scheduling methods in two scenarios to further verify its scheduling effect. Other scheduling algorithms include: DDPG, DCR-TD3 (a distributed dual-delay deep deterministic policy gradient method) and PPO (a proximal policy optimization method with a new objective function proposed by OpenAI).

[0109] The simulation results of scenario 1 under the comparative experiment are shown in Table 4.

[0110] Table 4

[0111]

[0112] As shown in Table 4, in scenario 1, the power grid optimization dispatching model based on multi-adversarial reinforcement learning is better than the experimental data of a single model in many aspects. The power grid optimization dispatching model based on multi-adversarial reinforcement learning proposed in the present invention runs 96 steps with both DDPG and DCR-TD3 algorithms, and runs 40 steps more than PPO. The safety score of the power grid optimization dispatching model based on multi-adversarial reinforcement learning proposed in the present invention is 1.058% and 0.184% higher than DDPG and DCR-TD3, and 89.671% higher than PPO. In terms of new energy consumption rate, the power grid optimization dispatching model based on multi-adversarial reinforcement learning proposed in the present invention is 79.524% higher than DDPG and 22.342% higher than DCR-TD3. The power grid optimization dispatching model based on multi-adversarial reinforcement learning proposed in the present invention is 31.250% higher than the best performance DCR-TD3 in terms of total reward value, and 123.486% and 83.852% higher than DDPG and PPO, which verifies the superiority of the power grid optimization dispatching model based on multi-adversarial reinforcement learning in the dispatching method.

[0113] See also Figures 4 to 6 , which more intuitively describes the utilization rate, cost and cumulative reward of renewable energy in each step of the above scheduling method in scenario 1. Figure 4 As shown in the figure, PPO and the grid optimization scheduling model based on multi-adversarial reinforcement learning achieved a relatively high renewable energy utilization rate at the beginning, but the scheduling of PPO was very unstable and ended at step 56. In contrast, the grid optimization scheduling model based on multi-adversarial reinforcement learning showed excellent stability and efficiency in terms of renewable energy utilization, and was the best performing method in the renewable energy utilization problem. In addition, the grid optimization scheduling model based on multi-adversarial reinforcement learning proposed in the present invention was also at a medium-low level in terms of cost, as shown in the figure. Figure 5 As shown. Figure 6 The overall dispatch performance is demonstrated. The reward obtained by PPO initially increases rapidly but ends quickly, which indicates its instability. DDPG and DCR-TD3 run more steps, but the rewards during the running period are significantly lower than the grid optimization dispatch model based on multi-adversarial reinforcement learning proposed in the present invention. In general, the grid optimization dispatch model based on multi-adversarial reinforcement learning achieves the highest accumulated reward, the highest stability and efficiency in terms of renewable energy utilization, and a moderate dispatch cost, verifying its superiority in the dispatch method.

[0114] The simulation results of scenario 2 under the comparative experiment are shown in Table 5.

[0115] Table 5

[0116]

[0117]

[0118] As shown in Table 5, in scenario 2, the power grid optimization scheduling model based on multi-adversarial reinforcement learning is better than the experimental data of a single algorithm in many aspects. In terms of the number of running steps, the power grid optimization scheduling model based on multi-adversarial reinforcement learning proposed by the present invention is 24 steps and 15 steps higher than DDPG and DCR-TD3, and the poorer PPO runs 72 more steps. The safety score of the power grid optimization scheduling model based on multi-adversarial reinforcement learning proposed by the present invention is 48.814% and 22.688% higher than DDPG and DCR-TD3, and 327.184% higher than PPO. In addition, the power grid optimization scheduling model based on multi-adversarial reinforcement learning proposed by the present invention reduces the unit operating costs by 2.317%, 0.782% and 0.360% compared with DDPG, DCR-TD3 and PPO. In terms of new energy consumption rate, the power grid optimization scheduling model based on multi-adversarial reinforcement learning proposed by the present invention is 4.822% higher than the optimal PPO, and 71.491% and 36.293% higher than DDPG and DCR-TD3. The power grid optimization scheduling model based on multi-adversarial reinforcement learning has an 80.726% improvement in total reward value compared with the best performing DCR-TD3, and an improvement of 202.034% and 335.649% compared with DDPG and PPO, verifying its superiority among the most advanced scheduling methods.

[0119] Regarding the new energy uncertainty analysis experiment: the power grid optimization dispatch model based on multi-adversarial reinforcement learning is compared with DDPG, DCR-TD3 and PPO in the case of a 5% new energy prediction deviation rate in scenario 1. The simulation results of scenario 1 under the new energy uncertainty analysis experiment are shown in Table 6.

[0120] Table 6

[0121]

[0122]

[0123] As shown in Table 6, the power grid optimization scheduling model based on multi-adversarial reinforcement learning is better than the experimental data of a single algorithm in many aspects. In terms of the number of running steps, the power grid optimization scheduling model based on multi-adversarial reinforcement learning proposed by the present invention is the best 96 steps, 5 more steps than the suboptimal DDPG, 7 more steps than DCR-TD3 and 62 more steps than PPO respectively. The power grid optimization scheduling model based on multi-adversarial reinforcement learning proposed by the present invention is 6.095% and 3.814% higher than DDPG and DCR-TD3 in terms of safety score, and 173.726% higher than PPO. In terms of unit operating costs, the power grid optimization scheduling model based on multi-adversarial reinforcement learning proposed by the present invention has the lowest operating cost, saving 0.165% and 0.835% compared with DDPG and DCR-TD3, and saving 4.409% compared with PPO. In terms of new energy consumption rate, the power grid optimization scheduling model based on multi-adversarial reinforcement learning proposed by the present invention is 83.111% higher than DDPG and 21.598% higher than DCR-TD3. The power grid optimization scheduling model based on multi-adversarial reinforcement learning has improved the total reward value by 36.143% compared with the best performing DCR-TD3, and by 155.375% and 172.505% compared with DDPG and PPO, verifying its superiority among the most advanced scheduling methods.

[0124] The following are device embodiments of the present invention, which can be used to implement the method embodiments of the present invention. For details not disclosed in the device embodiments, please refer to the method embodiments of the present invention.

[0125] See also Figure 7 In another embodiment of the present invention, a power grid optimization dispatching system is provided, which can be used to implement the above-mentioned power grid optimization dispatching method. Specifically, the power grid optimization dispatching system includes a data acquisition module and an optimization dispatching module.

[0126] Among them, the data acquisition module is used to obtain the current operating state of the power grid; the optimization scheduling module is used to input the current operating state of the power grid into the pre-trained power grid optimization scheduling model based on multi-adversarial reinforcement learning to obtain the current operating state adjustment action of the power grid. Among them, the power grid optimization scheduling model based on multi-adversarial reinforcement learning is obtained by modifying the target Q value of the critic network in the deep deterministic action gradient model during training to the sum of the immediate reward and the predicted value of the future potential reward of the power grid; among them, the predicted value of the future potential reward of the power grid is based on the operating state of the power grid during training and the intraday operating state adjustment action, and is predicted by the pre-trained generative multi-adversarial network model.

[0127] In one possible implementation, the pre-trained power grid optimization scheduling model based on multi-adversarial reinforcement learning is pre-trained: based on a preset number of historical experience extractions, good historical experience is extracted from a good historical experience sub-pool of a historical experience pool in combination with a good historical experience extraction ratio, and bad historical experience is extracted from a bad historical experience sub-pool of a historical experience pool in combination with a bad historical experience extraction ratio; and pre-training of the power grid optimization scheduling model based on multi-adversarial reinforcement learning is performed based on the extracted good historical experience and bad historical experience.

[0128] All relevant contents of each step involved in the aforementioned embodiment of the power grid optimization dispatching method can be referred to the functional description of the functional modules corresponding to the power grid optimization dispatching system in the embodiment of the present invention, and will not be repeated here.

[0129] The division of modules in the embodiments of the present invention is schematic and is only a logical function division. There may be other division methods in actual implementation. In addition, each functional module in each embodiment of the present invention may be integrated into one processor, or may exist physically separately, or two or more modules may be integrated into one module. The above-mentioned integrated modules may be implemented in the form of hardware or in the form of software functional modules.

[0130] In another embodiment of the present invention, a computer device is provided, the computer device comprising a processor and a memory, the memory being used to store a computer program, the computer program comprising program instructions, and the processor being used to execute the program instructions stored in the computer storage medium. The processor may be a central processing unit (CPU), or may be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc., which are the computing core and control core of the terminal, which are suitable for implementing one or more instructions, and are specifically suitable for loading and executing one or more instructions in a computer storage medium to implement the corresponding method flow or corresponding function; the processor described in the embodiment of the present invention can be used for the operation of the power grid optimization dispatching method.

[0131] In another embodiment of the present invention, the present invention also provides a storage medium, specifically a computer-readable storage medium (Memory), which is a memory device in a computer device for storing programs and data. It can be understood that the computer-readable storage medium here can include both built-in storage media in the computer device and, of course, extended storage media supported by the computer device. The computer-readable storage medium provides a storage space, which stores the operating system of the terminal. In addition, one or more instructions suitable for being loaded and executed by the processor are also stored in the storage space, and these instructions can be one or more computer programs (including program codes). It should be noted that the computer-readable storage medium here can be a high-speed RAM memory or a non-volatile memory, such as at least one disk memory. The processor can load and execute one or more instructions stored in the computer-readable storage medium to implement the corresponding steps of the power grid optimization dispatching method in the above embodiment.

[0132] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0133] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0134] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1A function specified in one or more boxes.

[0135] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0136] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the above embodiments, ordinary technicians in the relevant field should understand that the specific implementation methods of the present invention can still be modified or replaced by equivalents. Any modification or equivalent replacement that does not depart from the spirit and scope of the present invention should be covered within the scope of protection of the claims of the present invention.

Claims

1. A method for optimizing power grid dispatching, characterized in that: include: Get the current operating status of the power grid; The current operating state of the power grid is input into the pre-trained power grid optimization dispatching model based on multi-adversarial reinforcement learning to obtain the current operating state adjustment action of the power grid; Among them, the power grid optimization scheduling model based on multi-adversarial reinforcement learning is obtained by modifying the target Q value of the critic network in the deep deterministic action gradient model during training to the sum of the immediate reward and the predicted value of the future potential reward of the power grid; among them, the predicted value of the future potential reward of the power grid is based on the operating status of the power grid during training and the daily operating status adjustment action, and is predicted by a pre-trained generative multi-adversarial network model.

2. The power grid optimization dispatching method according to claim 1, characterized in that: The pre-trained generative multi-adversarial network model is pre-trained: The optimization objective Y of the generator of the generative multi-adversarial network model * for: Y * =argmin Y max D V(Y) Among them, Y is the generator of the generative multi-adversarial network model, D is the discriminator of the generative multi-adversarial network model, and V(Y) is the divergence required with the generator as the variable.

3. The power grid optimization dispatching method according to claim 2, characterized in that: The V(Y): Among them, λ is the parameter for adjusting the degree of fusion, ω i is the fusion degree weight of the i-th discriminator, I is the total number of discriminators, V' i (D i ,Y) is the divergence obtained by the i-th discriminator and generator; Among them, R t is the cumulative reward score from the beginning to time t, z is a predefined simple distribution, and its distribution is z~P z ,Y(z,[s t ,a t ]) is a combination of z and [s t ,a t ] is the divergence of the variable, s t is the agent state space, a t is the agent action space, E[·] is the expectation, R~P R is the probability distribution of reward scores.

4. The power grid optimization dispatching method according to claim 2, characterized in that: The pre-trained generative multi-adversarial network model is pre-trained: Update the model parameters of the generative multi-adversarial network model via stochastic gradient descent.

5. The power grid optimization dispatching method according to claim 1, characterized in that: The pre-trained multi-adversarial reinforcement learning-based power grid optimization scheduling model is pre-trained: Based on the preset number of historical experience extractions, good historical experience is extracted from the good historical experience sub-pool of the historical experience pool in combination with the good historical experience extraction ratio, and bad historical experience is extracted from the bad historical experience sub-pool of the historical experience pool in combination with the bad historical experience extraction ratio; and pre-training of a power grid optimization scheduling model using multi-adversarial reinforcement learning is performed based on the extracted good historical experience and bad historical experience.

6. The power grid optimization dispatching method according to claim 5, characterized in that: Also includes: Based on the preset instant reward threshold, historical experience in the historical experience pool whose instant reward is greater than the instant reward threshold is classified as good historical experience, and historical experience in the historical experience pool whose instant reward is not greater than the instant reward threshold is classified as bad historical experience, thus obtaining a good historical experience sub-pool and a bad historical experience sub-pool.

7. A power grid optimization dispatching system, characterized in that: include: A data acquisition module is used to obtain the current operating status of the power grid; The optimization scheduling module is used to input the current operation state of the power grid into the pre-trained power grid optimization scheduling model based on multi-adversarial reinforcement learning to obtain the current operation state adjustment action of the power grid; Among them, the power grid optimization scheduling model based on multi-adversarial reinforcement learning is obtained by modifying the target Q value of the critic network in the deep deterministic action gradient model during training to the sum of the immediate reward and the predicted value of the future potential reward of the power grid; among them, the predicted value of the future potential reward of the power grid is based on the operating status of the power grid during training and the daily operating status adjustment action, and is predicted by a pre-trained generative multi-adversarial network model.

8. The power grid optimization dispatching system according to claim 7, characterized in that: The pre-trained multi-adversarial reinforcement learning-based power grid optimization scheduling model is pre-trained: Based on the preset number of historical experience extractions, good historical experience is extracted from the good historical experience sub-pool of the historical experience pool in combination with the good historical experience extraction ratio, and bad historical experience is extracted from the bad historical experience sub-pool of the historical experience pool in combination with the bad historical experience extraction ratio; and pre-training of a power grid optimization scheduling model using multi-adversarial reinforcement learning is performed based on the extracted good historical experience and bad historical experience.

9. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the power grid optimization scheduling method according to any one of claims 1 to 6 are implemented.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of the power grid optimization dispatching method according to any one of claims 1 to 6 are implemented.

Citation Information

Cited By

  • Hydrogen energy system scheduling method with self-adaptive multi-step comprehensive view

    CN119918886A

  • A hydrogen energy system scheduling method with adaptive multi-step comprehensive vision

    CN119918886B

  • Power distribution network multi-scale optimization scheduling method and system based on deep reinforcement learning

    CN120373805A

  • Event-triggered energy system optimization method based on time-event dual-scale framework

    CN120409843A