A power grid topology optimization regulation method and application

Abstract

The application discloses a kind of power grid topology optimization control model construction method and application, belong to power grid topology field, comprising: constructing power grid topology optimization control model, power grid topology optimization control model contains multiple users;All original power data of user are input as state input data into power grid topology optimization control model, and power grid topology optimization control model outputs each user as buyer or seller local energy transaction strategy, and local energy transaction market is cleared according to the market of all users local energy transaction strategy, and the reward and cost of each user are obtained, and the state input data of each user, local energy transaction strategy, reward and cost are collected to obtain training data set;Training data set is used to train power grid topology optimization control model, and the state input data of each user is input into the trained power grid topology optimization control model to make it output the optimal local energy transaction strategy of each user at present.

Description

Technical Field

[0001] This invention belongs to the field of energy trading, and more specifically, relates to a method for constructing and applying a power grid topology optimization and control model. Background Technology

[0002] With rising global temperatures and environmental pollution, countries have put forward national policies to address the issues of energy development and use.

[0003] Manufacturing, as the leading industry, consumes the largest proportion of energy and emits the most greenhouse gases. China has over 20,000 industrial parks, contributing more than 30% to the national economy, but consuming approximately 69% of the energy and emitting about 31% of carbon. Therefore, all operators in the power system aim to use energy efficiently, reliably, and safely in an environmentally friendly manner. Most power systems have many users equipped with various energy carriers, including CHP units, distributed energy resources (DERs), i.e., distributed energy generators and energy storage systems, and other energy sources. Each user in the system has a wider range of energy choices and can flexibly trade energy with the main grid or other users. The emergence of local energy trading has greatly reduced losses in transmission and distribution from the main grid and alleviated the main grid's load demand. In such a localized electricity market, how users choose their trading strategies (determining the amount of electricity to sell / buy, and offering / bidding prices) to maximize their benefits becomes particularly important.

[0004] Many studies model energy trading as a Stackelberg game, a two-stage game with two players. The players make decisions in a specific order: the first to decide is called the leader, and the second is called the follower. Followers make their own decisions only after learning from the leader's decisions. The leader can then adjust its own decisions based on the followers' decisions, and this cycle continues until the game reaches a Nash equilibrium. However, this method largely relies on complex mathematical models, making the modeling process time-consuming and laborious.

[0005] Another popular approach to finding optimal energy trading strategies is reinforcement learning. Reinforcement learning does not require complex models and can solve for optimal strategies based on data. However, most current literature uses only single-agent, fully decentralized reinforcement learning algorithms, which is obviously disadvantageous in complex multi-user environments. Unstable environments may cause the algorithm to fail to obtain the optimal strategy. Summary of the Invention

[0006] In view of the above-mentioned defects or improvement needs of existing technologies, this invention provides a method and application for constructing a topology optimization control model for power system energy trading strategies based on secure reinforcement learning, in order to solve the technical problems of complex modeling or difficulty in finding the optimal trading strategy in existing energy trading problems.

[0007] To achieve the above objectives, according to a first aspect of the present invention, a method for constructing a power grid topology optimization control model is provided, comprising:

[0008] A power grid topology optimization and control model is constructed, which includes multiple users in an industrial park. The users' original power data is input into the power grid topology optimization and control model as state input data. The local energy trading strategy of each user as a buyer or seller is obtained. Market clearing is performed according to all the local energy trading strategies to obtain the reward and cost of each user. The state input data, local energy trading strategy, reward and cost of each user are collected to obtain a training dataset.

[0009] The training dataset is used to train the power grid topology optimization and control model, with the optimization objective of maximizing the sum of additional revenue and cost savings while limiting user bids or offers, to obtain a well-trained power grid topology optimization and control model.

[0010] Each user's current state input data is input into the trained power grid topology optimization and control model so that it outputs the optimal local energy trading strategy for each user.

[0011] More preferably, in the power grid topology optimization and control model, each user uses two agents to represent the buyer and seller identities respectively.

[0012] More preferably, the agent includes a policy neural network, a value neural network, and a cost neural network;

[0013] The policy neural network takes the user's state input data as input and outputs an action or a probability distribution of the action;

[0014] The value neural network takes the state input data of all users as input and outputs an approximate estimate of the state value function to assist the policy neural network in updating and learning.

[0015] The cost neural network takes the state input data of all users as input and outputs an approximate estimate of the state cost function to assist the policy neural network in updating and learning.

[0016] More preferably, the raw power data to status input data needs to undergo preprocessing, the preprocessing including:

[0017] The raw power data obtained from users in the industrial park is standardized using Max-Min; the raw power data includes the power generation data of the user's cogeneration unit (CHP), heat load, electrical load, photovoltaic (PV) power generation data, and energy storage system (ESS) storage capacity; wherein the CHP power generation data is calculated based on the heat load using the thermoelectric coupling effect.

[0018] More preferably, the electricity price in the output seller's local energy trading strategy is transformed from the interval [0,1] to the interval [p]. b,t ,p s,t The electricity sales volume changed from the interval [0,1] to the interval [0,E]. i ′ ,t ], where p b,t p is the buyback price of electricity at time t in the power grid. s,t E is the electricity price at time t on the power grid. i ′ ,t This represents the amount of electricity available for sale by seller i at time t; the electricity price in the buyer's local energy trading strategy output should also be transformed from the interval [0,1] to the interval [p]. b,t ,p s,t ].

[0019] More preferably, the rewards and costs include:

[0020] The reward is defined as follows:

[0021]

[0022] Where M t It is the market clearing price of that local energy transaction. It represents the amount of electricity that seller i successfully sold in the local market at time t. It represents the amount of electricity that buyer j successfully purchased in the local market at time t. It is the battery level of ESS at time t for user i. It represents the amount of charge that user i gives to ESS at time t. It is the collection of sellers who successfully completed local transactions. It is the set of buyers who successfully completed local transactions, and θ is a weighting factor;

[0023] The cost is defined as:

[0024]

[0025] Among them o i,t It is the price quoted by seller i at time t. It is the set of all sellers, b j,t It is the bid made by buyer j at time t. It is the collection of all buyers.

[0026] More preferably, the training process of the model includes:

[0027] The power grid topology optimization and control model also introduces Lagrange multipliers to automatically adjust the scale of rewards and costs, and further updates each network parameter and Lagrange multiplier alternately through the primal dual method during the training process;

[0028] The loss function of the policy neural network is the policy loss function of truncated proximal policy optimization (PPO-clip), and the network parameters are updated by gradient backpropagation;

[0029] The loss function for value neural networks and cost neural networks is the temporal difference error (TD error) loss function, and the network parameters are updated by gradient backpropagation;

[0030] The Lagrange multipliers are updated using gradient descent.

[0031] According to a second aspect of the present invention, a power grid topology optimization and control device is provided, comprising:

[0032] The model building module is used to construct a power grid topology optimization and control model, which includes multiple users in the power system. The original power data of the users is input into the power grid topology optimization and control model as state input data to obtain the local energy trading strategy of each user as a buyer or seller. Market clearing is performed according to all the local energy trading strategies to obtain the reward and cost of each user. The state input data, local energy trading strategy, reward and cost of each user are collected to obtain a training dataset.

[0033] The model training module is used to train the power grid topology optimization and control model using the training dataset, with the optimization objective of maximizing the sum of additional revenue and cost savings and limiting the user's bid or offer, to obtain a trained power grid topology optimization and control model.

[0034] The model output module is used to input the current state input data of each user into the trained power grid topology optimization and control model, so that it outputs the current optimal local energy trading strategy for each user.

[0035] More preferably, in the power grid topology optimization and control model, each user uses two agents to represent the buyer and seller identities respectively.

[0036] More preferably, the agent includes a policy neural network, a value neural network, and a cost neural network;

[0037] The policy neural network takes the user's state input data as input and outputs an action or a probability distribution of the action;

[0038] The value neural network takes the state input data of all users as input and outputs an approximate estimate of the state value function to assist the policy neural network in updating and learning.

[0039] The cost neural network takes the state input data of all users as input and outputs an approximate estimate of the state cost function to assist the policy neural network in updating and learning.

[0040] More preferably, the raw power data to status input data needs to undergo preprocessing, the preprocessing including:

[0041] The raw power data obtained from users in the industrial park is standardized using Max-Min; the raw power data includes the power generation data of the user's cogeneration unit (CHP), heat load, electrical load, photovoltaic (PV) power generation data, and energy storage system (ESS) storage capacity; wherein the CHP power generation data is calculated based on the heat load using the thermoelectric coupling effect.

[0042] More preferably, the electricity price in the output seller's local energy trading strategy is transformed from the interval [0,1] to the interval [p]. b,t ,p s,t The electricity sales volume changed from the interval [0,1] to the interval [0,E]. i ′ ,t ], where p b,t p is the buyback price of electricity at time t in the power grid. s,t E is the electricity price at time t on the power grid. i ′ ,t This represents the amount of electricity available for sale by seller i at time t; the electricity price in the buyer's local energy trading strategy output should also be transformed from the interval [0,1] to the interval [p]. b,t ,p s,t ].

[0043] More preferably, the rewards and costs include:

[0044] The reward is defined as follows:

[0045]

[0046] Where M t It is the market clearing price of that local energy transaction. It represents the amount of electricity that seller i successfully sold in the local market at time t. It represents the amount of electricity that buyer j successfully purchased in the local market at time t. It is the battery level of ESS at time t for user i. It represents the amount of charge that user i gives to ESS at time t. It is the collection of sellers who successfully completed local transactions. It is the set of buyers who successfully completed local transactions, and θ is a weighting factor;

[0047] The cost is defined as:

[0048]

[0049] Among them o i,t It is the price quoted by seller i at time t. It is the set of all sellers, b j,t It is the bid made by buyer j at time t. It is the collection of all buyers.

[0050] More preferably, the training process of the model includes:

[0051] The power grid topology optimization and control model also introduces Lagrange multipliers to automatically adjust the scale of rewards and costs, and further updates each network parameter and Lagrange multiplier alternately through the primal dual method during the training process;

[0052] The loss function of the policy neural network is the policy loss function of truncated proximal policy optimization (PPO-clip), and the network parameters are updated by gradient backpropagation;

[0053] The loss function for value neural networks and cost neural networks is the temporal difference error (TD error) loss function, and the network parameters are updated by gradient backpropagation;

[0054] The Lagrange multipliers are updated using gradient descent.

[0055] According to a third aspect of the present invention, a power grid topology optimization and control system is provided, comprising: a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to perform the power grid topology optimization and control method provided in the second aspect of the present invention.

[0056] According to a fourth aspect of the present invention, a machine-readable storage medium is provided, the machine-readable storage medium storing machine-executable instructions, which, when invoked and executed by a processor, cause the processor to implement the method for constructing a power grid topology optimization and control model provided in the first aspect of the present invention and / or the method for power grid topology optimization and control provided in the second aspect of the present invention.

[0057] In summary, compared with the prior art, the above-described technical solutions conceived by this invention can achieve the following beneficial effects:

[0058] (1) This invention provides a method for constructing a power grid topology optimization and control model. The power generation, load and energy storage data of users in industrial parks are processed into state input data. The energy trading strategy of users is defined as an action function. A cost function is introduced on the basis of the reward function to constrain and limit the energy price of users. The original dual method is used for optimization, which promotes more local energy trading within the power system, increases the electricity sales revenue of sellers, and reduces the electricity purchase cost of buyers.

[0059] (2) This invention provides a method for constructing a power grid topology optimization control model. Each industrial user uses two agents to represent the seller and the buyer respectively. The output dimension of the neural network is different for different agent strategies, which overcomes the shortcomings of traditional methods that limit users to a single identity.

[0060] (3) This invention provides a method for constructing a power grid topology optimization and control model. During the training phase, each user's model needs to input the state input information of all users globally. During the test and deployment phase, only the user's own state input information needs to be input. Centralized training and decentralized execution improve the training efficiency and effectiveness of the data. Attached Figure Description

[0061] Figure 1 A flowchart illustrating a method for constructing a power grid topology optimization and control model according to an embodiment of the present invention;

[0062] Figure 2 This is a schematic diagram of a user model provided in one embodiment of the present invention;

[0063] Figure 3 This is a flowchart of a local energy trading process provided in one embodiment of the present invention;

[0064] Figure 4 This is a schematic diagram of user intelligent agent interaction provided in one embodiment of the present invention;

[0065] Figure 5 This is a structural diagram of a user agent model provided in one embodiment of the present invention;

[0066] Figure 6 This is a flowchart of the power grid topology optimization and control model training process provided in one embodiment of the present invention;

[0067] Figure 7 A schematic diagram comparing the experimental results of the power grid topology optimization and control method provided by this invention with other different algorithms on a real dataset is presented. Detailed Implementation

[0068] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention. Furthermore, the technical features involved in the various embodiments of this invention described below can be combined with each other as long as they do not conflict with each other.

[0069] Example 1

[0070] like Figure 1 As shown, this invention provides a method for constructing a power grid topology optimization and control model, comprising the following steps:

[0071] S1: Construct a power grid topology optimization and control model, which includes multiple users in the industrial park. Each user has electrical load, thermal load, CHP units, PV panels, ESS (Electric Power Supply), and other flexible loads. The heat generated by each user's CHP units meets its own thermal load demand, i.e., the heat output. and heat load satisfy CHP units due to thermoelectric coupling effect It also generates electricity, of which Let t be the power generation of user k's CHP unit. and This represents the electrical efficiency and loss coefficient of user k's CHP unit. This portion of the electrical energy, along with the power generated by the PV panels, supplies the electrical load. Due to the temporal uncertainty of PV power generation, users may experience power surpluses or shortages at certain times, thus acting as sellers or buyers in the energy market. When a user has a power surplus, they can choose to sell some of the electricity and store some in the ESS (Energy Storage System), while purchasing only electricity when there is a power shortage. The ESS will supply to flexible loads. The ESS storage capacity update formula is:

[0072]

[0073] Where Δt is the time interval between two energy transactions, sp and V are the energy consumption rate and capacity limit of the ESS, respectively, and clip(x,a,b) is the cutoff function. It is the collection of all sellers. It is the collection of all buyers. Each user's status input data is constructed as follows: in It is the power generation of the PV panel of user k at time t. It is the electrical load of user k at time t. It is a collection of all users.

[0074] Specific user models, such as Figure 2As shown. All users within the power system are geographically close and connected by transmission lines, enabling internal energy trading. All users are also connected to the main grid, allowing for energy trading with the main grid. Internal energy trading has higher priority; users first trade with each other in their local energy markets. External energy trading has lower priority; any energy not traded locally will be traded with the main grid. The overall trading process is as follows: Figure 3 As shown.

[0075] Each user uses two agents; for example Figure 4 As shown, due to the uncertainty of PV power generation time, a user may change their transaction identity from buyer to seller or from seller to buyer multiple times a day. Therefore, it is inaccurate and incomplete to simply classify a user as one party or use only one intelligent agent to represent the user. Each user needs to use two intelligent agents, but at any given time, only one intelligent agent of a user participates in energy trading.

[0076] Each agent comprises a policy neural network, a value neural network, and a cost neural network. The policy neural network takes the user's state input data as input and outputs actions or a probability distribution of those actions. The actions of the buyer and seller agents differ slightly; the seller has two-dimensional actions, including a price quote. i,t Electricity sales E i,t The buyer only has a one-dimensional action in their bid (b). j,t :

[0077]

[0078] The value neural network takes as input the state input data of all users and outputs a state value function. An approximate estimate is used to assist the policy neural network in updating its learning, where the state value function... Defined as:

[0079]

[0080] Where γ is the discount factor, and π k The policy represents the strategy of agent k. The cost neural network takes the state input data of all users as input and outputs a state cost function. An approximate estimate is used to assist the policy neural network in updating its learning, where the state cost function... Defined as:

[0081]

[0082] Furthermore, the energy trading strategy optimization problem can be transformed into a constrained security reinforcement learning objective optimization problem:

[0083]

[0084] Where α k It is a threshold set artificially. This is achieved by introducing the Lagrange multiplier λ. k The above optimization problem can be further transformed into an unconstrained optimization problem:

[0085]

[0086] S2: Input the raw power data of all users in the industrial park as state input data into the power grid topology optimization and control model; perform Max-Min standardization on the raw power data obtained from the users in the industrial park, and the processing expression is as follows:

[0087]

[0088] Where x(t) and x′(t) are the values ​​of each dimension of the original state input information at time t, respectively. min and x max These are the minimum and maximum values ​​in each dimension of the input information corresponding to the original state; the original power data includes the user's CHP power generation data, heat load, electrical load, PV power generation data, and ESS storage capacity; wherein the CHP power generation data is calculated based on the heat load using the thermoelectric coupling effect.

[0089] The power grid topology optimization and control model outputs a local energy trading strategy for each user, acting as either a buyer or a seller. The output of the trading strategy maps to the output action of the strategy neural network; specifically, the output of the seller's strategy neural network... It needs to go through the following mapping:

[0090]

[0091] Where p b,t p is the buyback price of electricity at time t in the power grid. s,t E is the electricity price at time t on the power grid. i ′ ,t It represents the amount of electricity available for sale by seller i at time t; the output of the buyer's policy neural network. It needs to go through the following mapping:

[0092]

[0093] The local energy trading market clears the market based on the local energy trading strategies of all users, obtaining rewards and costs for each user. A training dataset is obtained by collecting the state input data, local energy trading strategies, rewards, and costs for each user. The reward function r... t,k Defined as:

[0094]

[0095] Where M t It is the market clearing price of that local energy transaction. It represents the amount of electricity that seller i successfully sold in the local market at time t. It represents the amount of electricity that buyer j successfully purchased in the local market at time t. It is the battery level of ESS at time t for user i. It represents the amount of charge that user i gives to ESS at time t. It is the collection of sellers who successfully completed local transactions. It is the set of buyers who have successfully completed local transactions, and θ is a weighting factor; the cost function is defined as:

[0096]

[0097] Among them o i,t It is the price quoted by seller i at time t. It is the set of all sellers, b j,t It is the bid made by buyer j at time t. It is the collection of all buyers.

[0098] S3: The power grid topology optimization and control model is trained using the training dataset. The current state input data of each user is input into the trained power grid topology optimization and control model to output the optimal local energy trading strategy for each user. In this embodiment, data from 3 days, with 24 transactions per day (1-hour transaction interval), is used as the training dataset. Specifically, the user agent model and the structure of its three neural networks are as follows... Figure 5 As shown. The value neural network and cost neural network use the temporal difference error (TD error) loss function. The parameters of the value neural network are ω. k The cost neural network has parameters μ. k Then the parameter update formulas for the two networks are:

[0099]

[0100] in:

[0101]

[0102] η Critic It is the learning rate η of the value neural network. Penalty-Critic The cost neural network uses the learning rate; the policy neural network uses a truncated proximal policy optimization (PPO-clip) policy loss function, and the policy neural network parameters are θ. k The parameter update formula is:

[0103]

[0104] in:

[0105]

[0106] β GAE ∈[0,1], where ∈ is a hyperparameter, and η Actor It is the learning rate of the policy network.

[0107] Finally, the Lagrange multipliers also need to be updated, and the update formula is as follows:

[0108]

[0109] Where η λ It is the update step size of the Lagrange multiplier.

[0110] The overall update and training process is as follows: Figure 6 As shown, it mainly includes the following three steps:

[0111] Step 1: Initialization. First, load the electrical and thermal load data, CHP unit data, and PV power generation data into the algorithm. Set the hyperparameters related to static training, and randomly set the initial parameters for the policy neural network, value neural network, and cost neural network. The memory buffer is also initialized to store the transformation tuple {s}. t,k ,a t,k ,s t+1,k ,r t,k ,c t,k}

[0112] Step Two: Interaction. The algorithm runs a total of P sets. In each time period t of each set, the user decides whether to buy or sell energy based on their power generation and load, and then displays their status information s. t,k The input is fed into the policy neural network of the corresponding buyer or seller agent, and the policy neural network will output a. t,k This information is then submitted to the energy market. The internal energy market will conduct a market clearing process, and all users will be informed of the market clearing price and the amount of energy available for trading in the power system. Then, the reward r for all agents is calculated. t,k and cost c t,k And the user will be transferred to the next state. t+1,k Finally, the tuple {s} t,k ,a t,k ,s t+1,k ,r t,k ,c t,k The data is stored in the memory buffer.

[0113] Step 3: Update. After an interaction, the data in the buffer is used to calculate the loss function and update the policy neural network, value neural network, and cost neural network via gradient backpropagation. All network parameters are updated over Q epochs of an episode, while the Lagrange multipliers are updated only once per agent. After the update, the state is reset, the buffer is cleared, and the next episode begins.

[0114] Example 2

[0115] This invention provides a power grid topology optimization and control method, comprising:

[0116] The user's status input information is input into the power grid topology optimization and control model constructed based on the above-mentioned power grid topology optimization and control model construction method to obtain the optimal trading strategy for all users.

[0117] Once trained, the model only needs to retain the policy neural network in the agent. The policy neural network only needs to be input with the state input information of the user, and no longer needs the state input information of all users globally. Therefore, it achieves centralized training and decentralized execution.

[0118] The power grid topology optimization and control method provided in this invention was numerically simulated on a real dataset of a power system consisting of seven users. The proposed model was compared with MAPPO, PPO, and stochastic strategies, and the results are as follows: Figure 7 As shown, the total cost savings and total additional revenue are based on the scenario where there is no energy trading within the power system, but only energy trading with the main grid. Compared with the other three methods, the proposed model achieves the highest total cost savings, additional revenue, and local energy trading volume. The stochastic strategy has higher per capita utility, but the number of participants in local energy trading is significantly reduced. In contrast, this model achieves higher per capita utility with a similar number of participants as the other methods.

[0119] Example 3

[0120] The present invention provides a power grid topology optimization and control system, comprising: a memory and a processor, wherein the memory stores a computer program, and the processor executes the power grid topology optimization and control method provided in Embodiment 2 of the present invention when executing the computer program.

[0121] Example 4

[0122] The present invention provides a machine-readable storage medium storing machine-executable instructions. When the machine-executable instructions are invoked and executed by a processor, the machine-executable instructions cause the processor to implement the method for constructing the power grid topology optimization and control model provided in Embodiment 1 of the present invention and / or the method for power grid topology optimization and control provided in Embodiment 2 of the present invention.

[0123] Those skilled in the art will readily understand that the above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A power grid topology optimization and control method, characterized in that, Includes the following steps: A power grid topology optimization and control model is constructed, wherein the power grid topology optimization and control model includes multiple users in the power system, and each user uses two agents to represent the buyer and seller identities respectively. The agents include a policy neural network, a value neural network and a cost neural network. The policy neural network takes the corresponding user's state input data as input and outputs an action or the probability distribution of the action; The value neural network takes the state input data of all users as input and outputs an approximate estimate of the state value function to assist the policy neural network in updating and learning. The cost neural network takes the state input data of all users as input and outputs an approximate estimate of the state cost function to assist the policy neural network in updating and learning. The user's original power data is input as state input data into the power grid topology optimization and control model to obtain the local energy trading strategy of each user as a buyer or seller. Market clearing is performed according to all local energy trading strategies to obtain the reward and cost of each user. The state input data, local energy trading strategy, reward and cost of each user are collected to obtain the training dataset. The training dataset is used to train the power grid topology optimization and control model, with the optimization objective of maximizing the sum of additional revenue and cost savings while limiting user bids or offers, to obtain a well-trained power grid topology optimization and control model. Each user's current state input data is input into the trained power grid topology optimization and control model so that it outputs the optimal local energy trading strategy for each user.

2. The power grid topology optimization and control method as described in claim 1, characterized in that, The raw power data needs to be preprocessed to become status input data. The preprocessing includes: The raw power data obtained from users in the industrial park is standardized using Max-Min; the raw power data includes the user's combined heat and power (CHP) generation data, heat load, electrical load, photovoltaic (PV) generation data, and energy storage system (ESS) storage capacity.

3. The power grid topology optimization and control method as described in claim 1, characterized in that, The output of the seller's local energy trading strategy transforms the electricity price from the interval [0,1] to the interval [ The electricity sales volume changed from the interval [0,1] to the interval [ ],in It is the buyback price of electricity at time t in the power grid. It is the electricity price at time t on the power grid. This represents the amount of electricity available for sale by seller i at time t; the electricity price in the buyer's local energy trading strategy output must also be transformed from the interval [0,1] to the interval [...]. ].

4. The power grid topology optimization and control method as described in claim 3, characterized in that, The rewards and costs include: The reward is defined as follows: in It is the market clearing price of that local energy transaction. It represents the amount of electricity that seller i successfully sold in the local market at time t. It represents the amount of electricity that buyer j successfully purchased in the local market at time t. It is the amount of electricity in ESS at time t for seller i. It represents the amount of charge that buyer i makes to ESS at time t. It is the collection of sellers who successfully completed local transactions. It is the collection of buyers who successfully completed local transactions. It is a weighting factor; The cost is defined as follows: in It is the price quoted by seller i at time t. It is the collection of all sellers. It is the bid made by buyer j at time t. It is the collection of all buyers.

5. The power grid topology optimization and control method as described in claim 1, characterized in that, The power grid topology optimization and control model is trained using the training dataset. During the training process, The power grid topology optimization and control model also introduces a Lagrange multiplier to automatically adjust the scale of rewards and costs, and alternately updates each network parameter and the Lagrange multiplier through the primal dual method; The loss function of the policy neural network is the policy loss function with truncated proximal policy optimization, and the network parameters are updated by gradient backpropagation; The loss function of value neural networks and cost neural networks is the temporal difference error loss function, and the network parameters are updated by gradient backpropagation; The Lagrange multipliers are updated using gradient descent.

6. A power grid topology optimization and control device, characterized in that, include: The model building module is used to build a power grid topology optimization and control model, wherein the power grid topology optimization and control model includes multiple users in the power system, and each user uses two agents to represent the buyer and seller identities respectively. The agents include a policy neural network, a value neural network and a cost neural network. The policy neural network takes the corresponding user's state input data as input and outputs an action or the probability distribution of the action; The value neural network takes the state input data of all users as input and outputs an approximate estimate of the state value function to assist the policy neural network in updating and learning. The cost neural network takes the state input data of all users as input and outputs an approximate estimate of the state cost function to assist the policy neural network in updating and learning. The user's original power data is input as state input data into the power grid topology optimization and control model to obtain the local energy trading strategy of each user as a buyer or seller. Market clearing is performed according to all local energy trading strategies to obtain the reward and cost of each user. The state input data, local energy trading strategy, reward and cost of each user are collected to obtain the training dataset. The model training module is used to train the power grid topology optimization and control model using the training dataset, with the optimization objective of maximizing the sum of additional revenue and cost savings and limiting the user's bid or offer, to obtain a trained power grid topology optimization and control model. The model output module is used to input the current state input data of each user into the trained power grid topology optimization and control model, so that it outputs the current optimal local energy trading strategy for each user.

7. A power grid topology optimization and control system, characterized in that, include: A memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to perform the power grid topology optimization and control method according to any one of claims 1-5.

8. A machine-readable storage medium, characterized in that, The machine-readable storage medium stores machine-executable instructions, which, when invoked and executed by a processor, cause the processor to implement the power grid topology optimization and control method according to any one of claims 1-5.

Images