Power grid topology optimization regulation and control method and application

Abstract

The invention discloses a construction method and application of a power grid topology optimization regulation and control model, and belongs to the field of power grid topologies, and the method comprises the steps: constructing a power grid topology optimization regulation and control model which comprises a plurality of users; and inputting the original power data of all the users as state input data into the power grid topology optimization regulation and control model, outputting a local energy transaction strategy of each user as a buyer or a seller by the power grid topology optimization regulation and control model, and performing market clearing by a local energy transaction market according to the local energy transaction strategies of all the users. Obtaining the reward and the cost of each user, and collecting the state input data, the local energy transaction strategy, the reward and the cost of each user to obtain a training data set; and training the power grid topology optimization regulation and control model by using the training data set, and inputting the current respective state input data of each user into the trained power grid topology optimization regulation and control model, so that the power grid topology optimization regulation and control model outputs the current optimal local energy transaction strategy of each user.

Description

Technical Field

[0001] The present invention belongs to the field of energy trading, and more specifically, relates to a construction method and application of a power grid topology optimization and control model. Background Art

[0002] With global temperatures rising and environmental pollution, countries have put forward national policies to address the problems of energy development and use.

[0003] As the leading industry, manufacturing consumes the largest proportion of energy and emits the most greenhouse gases. China has more than 20,000 industrial parks, which contribute more than 30% to the national economy, but consume about 69% of energy and emit about 31% of total carbon emissions. Therefore, all operators in the power system want to use energy efficiently, reliably and safely in an environmentally friendly way. Most power systems have many users equipped with a variety of energy carriers, including CHP units, distributed energy resources (DER), i.e. distributed energy generators and energy storage systems, and other energy sources. Each user in the system has a wider variety of energy options and can flexibly trade energy with the main grid or other users. The emergence of local energy trading has greatly reduced the losses of power transmission and distribution from the main grid and alleviated the load demand of the main grid. In such a localized power market, it becomes particularly important for users to choose their own trading strategies (determine the amount of electricity to sell / buy, quote / bid) to maximize their benefits.

[0004] Many literatures model energy trading as the Stackelberg game model, which is a two-party, two-stage game model in which the two parties make decisions in a certain order. The party that makes the decision first is called the leader, and the party that makes the decision later is called the follower. The follower makes his own decision after understanding the leader's decision. Then the leader can adjust his decision according to the follower's decision, and so on, until the game reaches the Nash equilibrium. However, this method mostly relies on complex mathematical models, and the modeling process is time-consuming and labor-intensive.

[0005] Another popular method for solving the optimal energy trading strategy is reinforcement learning. Reinforcement learning does not require complex models and can solve the optimal strategy based on data. However, most of the current literature only uses a single-agent, fully decentralized reinforcement learning algorithm, which is obviously not advantageous in a complex environment with multiple users. An unstable environment is likely to cause the algorithm to fail to obtain the optimal strategy. Summary of the invention

[0006] In response to the above defects or improvement needs of the prior art, the present invention provides a method for constructing and applying a topological optimization and control model of an electric power system energy trading strategy based on secure reinforcement learning, so as to solve the technical problems that the existing energy trading problems are complex in modeling or difficult to solve the optimal trading strategy.

[0007] To achieve the above object, according to a first aspect of the present invention, a method for constructing a power grid topology optimization control model is provided, comprising:

[0008] Constructing a power grid topology optimization control model, which includes multiple users in the industrial park, inputting the original power data of the users as state input data into the power grid topology optimization control model, obtaining the local energy trading strategy of each user as a buyer or seller, performing market clearing according to all the local energy trading strategies, obtaining the reward and cost of each user, and collecting the state input data, local energy trading strategy, reward and cost of each user to obtain a training data set;

[0009] The training data set is used to train the power grid topology optimization control model, with the optimization goal of maximizing the sum of additional revenue and cost savings and limiting the bid or quotation of the user, to obtain a trained power grid topology optimization control model;

[0010] The current state input data of each user is input into the trained power grid topology optimization control model so that it outputs the current optimal local energy trading strategy for each user.

[0011] Further preferably, in the power grid topology optimization control model, each user uses two agents to represent the identity of the buyer and the seller respectively.

[0012] Further preferably, the agent includes a strategy neural network, a value neural network and a cost neural network;

[0013] The policy neural network inputs the user's state input data and outputs an action or a probability distribution of the action;

[0014] The value neural network inputs the state input data of all users and outputs an approximate estimate of the state value function to assist the policy neural network in updating and learning;

[0015] The cost neural network inputs the state input data of all users and outputs an approximate estimate of the state cost function to assist the policy neural network in updating learning.

[0016] Further preferably, the raw power data to state input data needs to be preprocessed, and the preprocessing includes:

[0017] The raw power data obtained from users in the industrial park are Max-Min standardized; the raw power data include the user's combined heat and power (CHP) power generation data, thermal load, electrical load, photovoltaic panel (PV) power generation data, and energy storage system (ESS) storage capacity; wherein the CHP power generation data is calculated based on the thermal load by the thermoelectric coupling effect.

[0018] Further preferably, the electricity price in the output seller's local energy trading strategy is changed from the interval [0,1] to the interval [p b,t ,p s,t ], the electricity sales volume changes from the interval [0,1] to the interval [0,E i ' ,t ], where p b,t is the purchase price of electricity at the grid at time t, p s,t is the electricity price of the power grid at time t, E i ' ,t is the amount of electricity available for sale by seller i at time t; the electricity price in the output buyer's local energy trading strategy should also be changed from the interval [0,1] to the interval [p b,t ,p s,t ].

[0019] Further preferably, the rewards and costs include:

[0020] The reward is defined as:

[0021]

[0022] Among them, M t is the market clearing price of the local energy transaction at that time, is the amount of electricity that seller i successfully sold in the local market at time t, is the amount of electricity successfully purchased by buyer j in the local market at time t, is the ESS power consumption of user i at time t, is the amount of charge that user i charges to ESS at time t, It is a collection of local sellers with successful transactions. is the set of buyers who have successfully traded locally, and θ is a weight factor;

[0023] The cost is defined as:

[0024]

[0025] Among them i,t is the price offered by seller i at time t, is the set of all sellers, b j,t is the bid of buyer j at time t, is the set of all buyers.

[0026] Further preferably, the training process of the model includes:

[0027] The power grid topology optimization control model also introduces Lagrange multipliers to automatically adjust the scale of rewards and costs, and further updates each network parameter and Lagrange multiplier alternately through the primal-dual method during the training process;

[0028] The loss function of the policy neural network is the policy loss function of truncated proximal policy optimization (PPO-clip), and the network parameters are updated by gradient back propagation;

[0029] The loss function of the value neural network and the cost neural network is the temporal difference error (TD error) loss function, and the network parameters are updated by gradient back propagation;

[0030] The Lagrange multipliers are updated using the gradient descent method.

[0031] According to a second aspect of the present invention, there is provided a power grid topology optimization control device, comprising:

[0032] A model building module is used to build a power grid topology optimization and control model, wherein the power grid topology optimization and control model includes multiple users in the power system; the original power data of the users is input into the power grid topology optimization and control model as state input data, and the local energy trading strategy of each user as a buyer or seller is obtained, and the market is cleared according to all the local energy trading strategies to obtain the reward and cost of each user, and the state input data, local energy trading strategy, reward and cost of each user are collected to obtain a training data set;

[0033] A model training module is used to train the power grid topology optimization control model using the training data set, with the optimization goal of maximizing the sum of additional revenue and cost savings and limiting the bid or quotation of the user, to obtain a trained power grid topology optimization control model;

[0034] The model output module is used to input the current state input data of each user into the trained power grid topology optimization control model so that it can output the current optimal local energy trading strategy for each user.

[0035] Further preferably, in the power grid topology optimization control model, each user uses two agents to represent the identity of the buyer and the seller respectively.

[0036] Further preferably, the agent includes a strategy neural network, a value neural network and a cost neural network;

[0037] The policy neural network inputs the user's state input data and outputs an action or a probability distribution of the action;

[0038] The value neural network inputs the state input data of all users and outputs an approximate estimate of the state value function to assist the policy neural network in updating and learning;

[0039] The cost neural network inputs the state input data of all users and outputs an approximate estimate of the state cost function to assist the policy neural network in updating learning.

[0040] Further preferably, the raw power data to state input data needs to be preprocessed, and the preprocessing includes:

[0041] The raw power data obtained from users in the industrial park are Max-Min standardized; the raw power data include the user's combined heat and power (CHP) power generation data, thermal load, electrical load, photovoltaic panel (PV) power generation data, and energy storage system (ESS) storage capacity; wherein the CHP power generation data is calculated based on the thermal load by the thermoelectric coupling effect.

[0042] Further preferably, the electricity price in the output seller's local energy trading strategy is changed from the interval [0,1] to the interval [p b,t ,p s,t ], the electricity sales volume changes from the interval [0,1] to the interval [0,E i ' ,t ], where p b,t is the purchase price of electricity at the grid at time t, p s,t is the electricity price of the power grid at time t, E i ' ,t is the amount of electricity available for sale by seller i at time t; the electricity price in the output buyer's local energy trading strategy should also be changed from the interval [0,1] to the interval [p b,t ,p s,t ].

[0043] Further preferably, the rewards and costs include:

[0044] The reward is defined as:

[0045]

[0046] Among them, M t is the market clearing price of the local energy transaction at that time, is the amount of electricity that seller i successfully sold in the local market at time t, is the amount of electricity successfully purchased by buyer j in the local market at time t, is the ESS power consumption of user i at time t, is the amount of charge that user i charges to ESS at time t, It is a collection of local sellers with successful transactions. is the set of buyers who have successfully traded locally, and θ is a weight factor;

[0047] The cost is defined as:

[0048]

[0049] Among them i,t is the price offered by seller i at time t, is the set of all sellers, b j,t is the bid of buyer j at time t, is the set of all buyers.

[0050] Further preferably, the training process of the model includes:

[0051] The power grid topology optimization control model also introduces Lagrange multipliers to automatically adjust the scale of rewards and costs, and further updates each network parameter and Lagrange multiplier alternately through the primal-dual method during the training process;

[0052] The loss function of the policy neural network is the policy loss function of truncated proximal policy optimization (PPO-clip), and the network parameters are updated by gradient back propagation;

[0053] The loss function of the value neural network and the cost neural network is the temporal difference error (TD error) loss function, and the network parameters are updated by gradient back propagation;

[0054] The Lagrange multipliers are updated using the gradient descent method.

[0055] According to a third aspect of the present invention, there is provided a power grid topology optimization and control system, comprising: a memory and a processor, wherein the memory stores a computer program, and the processor executes the power grid topology optimization and control method provided by the second aspect of the present invention when executing the computer program.

[0056] According to a fourth aspect of the present invention, a machine-readable storage medium is provided, wherein the machine-readable storage medium stores machine-executable instructions. When the machine-executable instructions are called and executed by a processor, the machine-executable instructions prompt the processor to implement the method for constructing a power grid topology optimization and control model provided by the first aspect of the present invention and / or the power grid topology optimization and control method provided by the second aspect of the present invention.

[0057] In general, the above technical solutions conceived by the present invention can achieve the following beneficial effects compared with the prior art:

[0058] (1) The present invention provides a method for constructing a power grid topology optimization and control model, which processes the power generation, load and energy storage data of industrial park users into state input data, defines the user's energy trading strategy as an action function, introduces a cost function based on the reward function, constrains and limits the user's energy price, and uses the primal dual method to optimize it, thereby promoting more local energy transactions within the power system, increasing the seller's electricity sales revenue, and reducing the buyer's electricity purchase cost.

[0059] (2) The present invention provides a method for constructing a power grid topology optimization and control model. Each industrial user uses two intelligent agents to represent the seller and the buyer respectively. The output dimensions of the neural networks of different intelligent agent strategies are different, which overcomes the shortcoming of the traditional method that limits the user to a single identity.

[0060] (3) The present invention provides a method for constructing a power grid topology optimization and control model. During the training phase, each user's model needs to input the status input information of all global users. During the test deployment phase, only the user's own status input information needs to be input. Centralized training and decentralized execution improve the training efficiency and effect of data. BRIEF DESCRIPTION OF THE DRAWINGS

[0061] Figure 1 A flowchart of a method for constructing a power grid topology optimization control model provided by an embodiment of the present invention;

[0062] Figure 2 A schematic diagram of a user model provided in one embodiment of the present invention;

[0063] Figure 3 A flow chart of local energy trading provided in one embodiment of the present invention;

[0064] Figure 4 A schematic diagram of user-agent interaction provided in one embodiment of the present invention;

[0065] Figure 5 A diagram showing a user agent model structure provided in one embodiment of the present invention;

[0066] Figure 6 A training flow chart of a power grid topology optimization control model provided in one embodiment of the present invention;

[0067] Figure 7 A schematic diagram is given for comparing the experimental results of the power grid topology optimization control method provided by the present invention with other different algorithms on actual data sets. DETAILED DESCRIPTION

[0068] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention. In addition, the technical features involved in the various embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other.

[0069] Example 1

[0070] like Figure 1 As shown, the present invention provides a method for constructing a power grid topology optimization control model, comprising the following steps:

[0071] S1: Construct a grid topology optimization control model, which includes multiple users in the industrial park; each user has electrical load, thermal load, CHP unit, PV panel, ESS and other flexible loads. The heat generated by the user's CHP unit meets its own thermal load demand, that is, the heat generated and heat load satisfy Due to the thermoelectric coupling effect of CHP units It also generates electrical energy, is the CHP unit power generation of user k at time t, and is the electrical efficiency and loss coefficient of the CHP unit of user k. This part of electricity is supplied to the electrical load together with the power generation of the PV panels. Due to the uncertainty of the time of PV power generation, the user may have excess or shortage of electricity in a certain period of time, and thus play the role of seller or buyer in the energy market. When the user has excess electricity, he can choose to sell part of the electricity and store part of the electricity in ESS, and only purchase electricity when there is a shortage of electricity. ESS will supply flexible loads, and the ESS storage capacity update formula is:

[0072]

[0073] Where Δt is the time interval between two energy transactions, sp and V are the energy consumption rate and capacity limit of ESS respectively, and clip(x,a,b) is the truncation function. is the set of all sellers, is the set of all buyers. The state input data of each user is constructed as in is the PV panel power generation of user k at time t, is the electricity load of user k at time t, Is the collection of all users.

[0074] Specific user models such as Figure 2As shown. All users within the power system are geographically close to each other and connected by transmission lines, so they can conduct internal energy transactions; all users are connected to the main grid and can trade energy with the main grid; internal energy transactions have a higher priority, and users first trade with each other in the local energy market, while external energy transactions have a lower priority, and some of the energy that users fail to trade in the local market will be traded with the main grid. The overall transaction process is as follows Figure 3 shown.

[0075] Each user uses two agents; Figure 4 As shown in the figure, due to the uncertainty of PV power generation in time, users may change their trading identities from buyer to seller or from seller to buyer multiple times in one day. Therefore, it is inaccurate and incomplete to simply classify users as one party or to use only one agent to represent users. Each user needs to use two agents, but at the same time a user only has one agent participating in energy trading.

[0076] Each agent includes a policy neural network, a value neural network and a cost neural network. The policy neural network inputs the user's state input data and outputs an action or a probability distribution of the action. The actions of the buyer and seller agents are slightly different. The seller has two-dimensional actions and the quotation is i,t and electricity sales E i,t , the buyer has only one-dimensional action bid b j,t :

[0077]

[0078] The value neural network inputs the state input data of all users and outputs the state value function An approximate estimate of is used to assist the policy neural network in updating learning, where the state value function Defined as:

[0079]

[0080] where γ is the discount factor, π k Represents the strategy of agent k. The cost neural network inputs the state input data of all users and outputs the state cost function An approximate estimate of is used to assist the policy neural network in updating learning, where the state cost function Defined as:

[0081]

[0082] Furthermore, the energy trading strategy optimization problem can be transformed into a constrained security reinforcement learning objective optimization problem:

[0083]

[0084] where α k is an artificially set threshold. By introducing the Lagrange multiplier λ k The above optimization problem can be further transformed into an unconstrained optimization problem:

[0085]

[0086] S2: Input the original power data of all users in the industrial park as state input data into the power grid topology optimization control model; perform Max-Min standardization on the original power data obtained from the users in the industrial park, and the processing expression is as follows:

[0087]

[0088] Among them, x(t) and x′(t) are the values ​​of the original state input information at each dimension at time t, x min and x max are respectively the minimum and maximum values ​​in each dimension of the corresponding original state input information; the original power data includes the user's CHP power generation data, thermal load, electric load, PV power generation data, and ESS storage capacity; the CHP power generation data is calculated based on the thermal load by the thermoelectric coupling effect.

[0089] The grid topology optimization control model outputs each user's local energy trading strategy as a buyer or seller; the output of the trading strategy must be mapped to the output action of the strategy neural network. Specifically, the output of the seller's strategy neural network To go through the following mapping:

[0090]

[0091] where p b,t is the purchase price of electricity at the grid at time t, p s,t is the electricity price of the power grid at time t, E i ' ,t is the amount of electricity available for sale by seller i at time t; the output of the buyer strategy neural network To go through the following mapping:

[0092]

[0093] The local energy trading market performs market clearing according to the local energy trading strategies of all users, obtains the rewards and costs of each user, and collects the state input data, local energy trading strategies, rewards and costs of each user to obtain a training data set. The reward function r t,k Defined as:

[0094]

[0095] Among them, M t is the market clearing price of the local energy transaction at that time, is the amount of electricity that seller i successfully sold in the local market at time t, is the amount of electricity successfully purchased by buyer j in the local market at time t, is the ESS power consumption of user i at time t, is the amount of charge that user i charges to ESS at time t, It is a collection of local sellers with successful transactions. is the set of buyers who have successfully traded locally, θ is a weight factor; the cost function is defined as:

[0096]

[0097] Among them i,t is the price offered by seller i at time t, is the set of all sellers, b j,t is the bid of buyer j at time t, is the set of all buyers.

[0098] S3: Use the training data set to train the power grid topology optimization control model, and input the current state input data of each user into the trained power grid topology optimization control model, so that it can output the optimal local energy trading strategy for each user. In this embodiment, a total of 3 days, 24 transactions per day (1 hour transaction interval) data are used as the training data set. Specifically, the model of the user agent and the structure of the three neural networks therein are as follows: Figure 5 As shown. The value neural network and the cost neural network use the temporal difference error (TD error) loss function, and the value neural network parameter is ω k , the cost neural network parameter is μ k , then the parameter update formula of the two networks is:

[0099]

[0100] in:

[0101]

[0102] η Critic is the learning rate of the value neural network, η Penalty-Critic is the learning rate of the cost neural network; the policy neural network uses the policy loss function of truncated proximal policy optimization (PPO-clip), and the policy neural network parameter is θ k , the parameter update formula is:

[0103]

[0104] in:

[0105]

[0106] β GAE ∈[0,1], ∈ is a hyperparameter, η Actor is the learning rate of the policy network.

[0107] Finally, the Lagrange multiplier also needs to be updated, and the update formula is:

[0108]

[0109] where η λ is the update step size of the Lagrange multipliers.

[0110] The overall update training process is as follows Figure 6 As shown, it mainly includes the following three steps:

[0111] Step 1: Initialization. First, load the electricity and heat load data, CHP unit data, and PV power generation data into the algorithm. Set the hyperparameters related to static training, and randomly set the initial parameters of the policy neural network, value neural network, and cost neural network. The memory buffer is also initialized to store the conversion tuple {s t,k ,a t,k ,s t+1,k ,r t,k ,c t,k}.

[0112] Step 2: Interaction. The algorithm runs a total of P sets. In each time period t of each set, the user decides whether to buy or sell energy based on his own power generation and load, and then sends his own status information s t,k Input into the corresponding buyer or seller agent's policy neural network, the policy neural network will output a t,k , and submit it to the energy market. The internal energy market will be cleared, and all users will be informed of the market clearing price and the amount of energy that can be traded in the power system. Then, the rewards r of all agents are calculated t,k and cost c t,k , and the user will be transferred to the next state s t+1,k Finally, the tuple {s t,k ,a t,k ,s t+1,k ,r t,k ,c t,k} is stored in the memory buffer.

[0113] Step 3: Update. After an interaction is completed, the data in the buffer will be used to calculate the loss function and update the policy neural network, value neural network, and cost neural network through the gradient backpropagation mechanism. All network parameters will be updated in Q epochs of an episode, while the Lagrange multiplier is updated only once for each agent. After the update is completed, the state will be reset, the buffer will be cleared, and the next episode will begin.

[0114] Example 2

[0115] The present invention provides a power grid topology optimization control method, comprising:

[0116] The user's status input information is input into the power grid topology optimization control model constructed based on the above-mentioned power grid topology optimization control model construction method to obtain the optimal trading strategy for all users.

[0117] After training, the model only needs to retain the policy neural network in the agent, and the policy neural network only needs to input the user's own state input information, and no longer needs the state input information of all global users, thus achieving centralized training and decentralized execution.

[0118] The grid topology optimization control method provided by the present invention is numerically simulated on a real data set of a power system consisting of seven users. The proposed model is compared with MAPPO, PPO and random strategies. The results are as follows Figure 7 As shown in the figure. The total cost savings and total additional revenue are based on the case where there is no energy trading within the power system and only energy trading is conducted with the main grid. Compared with the other three methods, the proposed model achieves the highest total cost savings, additional revenue, and local energy trading volume. The random strategy has a higher per capita utility, but the number of participants in local energy trading is significantly reduced. In contrast, this model achieves a higher per capita utility with basically the same number of participants as the other methods.

[0119] Example 3

[0120] The present invention provides a power grid topology optimization and control system, comprising: a memory and a processor, the memory storing a computer program, and the processor executing the power grid topology optimization and control method provided in Example 2 of the present invention when executing the computer program.

[0121] Example 4

[0122] The present invention provides a machine-readable storage medium, which stores machine-executable instructions. When the machine-executable instructions are called and executed by a processor, the machine-executable instructions prompt the processor to implement the method for constructing a power grid topology optimization and control model provided in Example 1 of the present invention and / or the power grid topology optimization and control method provided in Example 2 of the present invention.

[0123] It will be easily understood by those skilled in the art that the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A power grid topology optimization control method, characterized in that: The following steps are involved: Constructing a power grid topology optimization and control model, wherein the power grid topology optimization and control model includes multiple users in the power system; Inputting the user's original power data as state input data into the power grid topology optimization control model, obtaining the local energy trading strategy of each user as a buyer or seller, performing market clearing according to all the local energy trading strategies, obtaining the reward and cost of each user, and collecting the state input data, local energy trading strategy, reward and cost of each user to obtain a training data set; The training data set is used to train the power grid topology optimization control model, with the optimization goal of maximizing the sum of additional revenue and cost savings and limiting the bid or quotation of the user, to obtain a trained power grid topology optimization control model; The current state input data of each user is input into the trained power grid topology optimization control model so that it outputs the current optimal local energy trading strategy for each user.

2. The power grid topology optimization control method according to claim 1, characterized in that: In the power grid topology optimization and control model, each user uses two intelligent agents to represent the identity of the buyer and the seller respectively.

3. The power grid topology optimization control method according to claim 2, characterized in that: The intelligent agent includes a strategy neural network, a value neural network and a cost neural network; The policy neural network inputs the state input data corresponding to the user and outputs the action or the probability distribution of the action; The value neural network inputs the state input data of all users and outputs an approximate estimate of the state value function to assist the policy neural network in updating and learning; The cost neural network inputs the state input data of all users and outputs an approximate estimate of the state cost function to assist the policy neural network in updating and learning.

4. The power grid topology optimization control method according to claim 1, characterized in that: The raw power data to state input data needs to be preprocessed, and the preprocessing includes: The original power data obtained from the users in the industrial park is standardized by Max-Min; the original power data includes the CHP power generation data of the users' combined heat and power units, thermal load, electrical load, photovoltaic panel PV power generation data, and energy storage system ESS power storage capacity.

5. The power grid topology optimization control method according to claim 1, characterized in that: The electricity price in the output seller's local energy trading strategy is changed from the interval [0, 1] to the interval [p b，t , p s,t ], the electricity sales volume changes from the interval [0, 1] to the interval [0, E′ i，t ], where p b，t is the purchase price of electricity at the grid at time t, p s，t is the electricity price of the power grid at time t, E′ i，t is the amount of electricity available for sale by seller i at time t; the electricity price in the output buyer's local energy trading strategy should also be changed from the interval [0, 1] to the interval [p b，t , p s，t ].

6. The power grid topology optimization and control method according to claim 1, characterized in that: The rewards and costs include: The reward is defined as: Among them, M t is the market clearing price of the local energy transaction at that time, is the amount of electricity that seller i successfully sold in the local market at time t, is the amount of electricity successfully purchased by buyer j in the local market at time t, is the electricity consumption of ESS by seller i at time t, is the amount of charge that buyer i charges to ESS at time t, It is a collection of local sellers with successful transactions. is the set of buyers who have successfully traded locally, and θ is a weight factor; The cost is defined as: Among them i，t is the price offered by seller i at time t, is the set of all sellers, b j，t is the bid of buyer j at time t, is the set of all buyers.

7. The power grid topology optimization control method according to claim 3, characterized in that: The training data set is used to train the power grid topology optimization control model. During the training process, The power grid topology optimization control model also introduces Lagrange multipliers to automatically adjust the scale of rewards and costs, and updates each network parameter and Lagrange multiplier alternately through the primal-dual method; The loss function of the policy neural network is the policy loss function of truncated proximal policy optimization, and the network parameters are updated by gradient back propagation; The loss function of the value neural network and the cost neural network is the temporal difference error loss function, and the network parameters are updated by gradient back propagation; The Lagrange multipliers are updated using the gradient descent method.

8. A power grid topology optimization and control device, characterized in that: include: A model building module, used to build a power grid topology optimization and control model, wherein the power grid topology optimization and control model includes multiple users in the power system; Inputting the user's original power data as state input data into the power grid topology optimization control model, obtaining the local energy trading strategy of each user as a buyer or seller, performing market clearing according to all the local energy trading strategies, obtaining the reward and cost of each user, and collecting the state input data, local energy trading strategy, reward and cost of each user to obtain a training data set; A model training module is used to train the power grid topology optimization control model using the training data set, with the optimization goal of maximizing the sum of additional revenue and cost savings and limiting the bid or quotation of the user, to obtain a trained power grid topology optimization control model; The model output module is used to input the current state input data of each user into the trained power grid topology optimization control model so that it can output the current optimal local energy trading strategy for each user.

9. A power grid topology optimization control system, characterized in that: include: A memory and a processor, wherein the memory stores a computer program, and the processor executes the power grid topology optimization and control method according to claim 8 when executing the computer program.

10. A machine-readable storage medium, characterized in that: The machine-readable storage medium stores machine-executable instructions. When the machine-executable instructions are called and executed by the processor, the machine-executable instructions prompt the processor to implement the method for constructing a power grid topology optimization and control model as described in any one of claims 1 to 7 and / or the power grid topology optimization and control method as described in claim 8.

Images