A multi-agent power distribution network dispatching method and device based on information dynamic aggregation

Abstract

The application provides a multi-agent power distribution network scheduling method and device based on information dynamic aggregation, relates to the technical field of power system operation control, and divides the power distribution network into multiple control areas, establishes a scheduling optimization model, is further converted into a Markov game model, constructs a reinforcement learning agent for distributed optimization, constructs a distributed learning architecture, each agent independently executes decision-making and model updating based on local observation, avoids global information sharing, and effectively protects the data privacy of each area. Using the optimized reinforcement learning agent, the local observation variables at the current time are dynamically encrypted to generate each encrypted local information, realize information encryption of each agent in the information interaction process, and further reduce the risk of privacy leakage. Real-time scheduling is completed by dynamically generating aggregation weights, the collaborative efficiency and decision-making performance of the multi-agent are improved, and the problems of low collaborative efficiency caused by privacy leakage risk and static aggregation strategy are solved.

Description

Technical Field

[0001] This invention relates to the field of power system operation and control technology, and in particular to a multi-agent distribution network scheduling method and device based on dynamic information aggregation. Background Technology

[0002] Traditional centralized dispatching models face problems such as single points of failure, high communication pressure, slow response speed, and poor scalability. To improve the flexibility and scalability of dispatching, a distributed dispatching architecture is used to achieve efficient collaborative management of widely distributed resources, meeting the needs of safe, economical, and stable operation of the distribution network.

[0003] However, distributed generation resources are numerous and diverse. If physical model-based optimization methods are used, precise models for each device must be maintained and globally coordinated, resulting in high modeling and maintenance costs. Furthermore, model accuracy is significantly affected by parameter uncertainties. Therefore, reinforcement learning methods that do not require precise models become a feasible approach. By deploying local agents in various regions and utilizing multi-agent reinforcement learning algorithms to achieve autonomous decision-making and collaborative optimization, the burden of centralized modeling can be reduced, and the system's dynamic changes can be adapted to, providing a new data-driven paradigm for the efficient regulation of large-scale distributed resources.

[0004] Currently, distributed control methods for power distribution networks based on multi-agent reinforcement learning still have significant shortcomings in the design of collaborative mechanisms. Existing communication architectures mostly rely on explicit sharing of observation or action information among agents, with some methods even requiring global state visibility or neural network parameter synchronization. Such information sharing mechanisms are highly susceptible to exposing sensitive operational data of each agent's jurisdiction, such as load fluctuations, power generation output, and energy storage status, posing a risk of privacy breaches and failing to meet the data security and privacy protection requirements of multiple agents in the power system. On the other hand, mainstream multi-agent reinforcement learning methods based on information aggregation typically use fixed weights or simple averaging to fuse information from each agent, lacking the ability to dynamically perceive the system's operational status. This mechanism cannot distinguish between critical and redundant information, leading to the dilution of important control commands during the aggregation process, while invalid or repetitive information is excessively propagated. In scenarios where the operating status of the power distribution network changes rapidly, static aggregation strategies are difficult to adapt to dynamic needs, restricting the collaborative efficiency and decision-making performance of multi-agent systems. Summary of the Invention

[0005] To address the privacy risks and low collaborative efficiency caused by static aggregation strategies in existing technologies, this invention provides a multi-agent power distribution network scheduling method and device based on dynamic information aggregation. By constructing a distributed learning architecture, each agent can make independent decisions and update its model based on local observations, avoiding global information sharing. At the same time, local observation variables are dynamically encrypted during information interaction, and aggregation weights are dynamically generated based on the system's operating status for information aggregation. This achieves implicit collaboration with low overhead and high information utilization while ensuring data privacy in each region.

[0006] To achieve the above objectives, the present invention provides the following technical solution:

[0007] The present invention provides a multi-agent distribution network scheduling method based on dynamic information aggregation in its first aspect, comprising: dividing the distribution network into multiple control areas, each control area containing distributed resources; establishing a distribution network scheduling optimization model with the objective of minimizing the operating cost of distributed resources, and setting constraints; transforming the distribution network scheduling optimization model into a Markov game model containing multiple agents, each agent corresponding to a control area, each agent taking local observation variables as input and local control commands as output; constructing corresponding reinforcement learning agents for each agent in the Markov game model; performing distributed reinforcement learning based on reinforcement learning algorithms to optimize the Markov game model, obtaining optimized reinforcement learning agents corresponding to each control area; using the optimized reinforcement learning agents, dynamically encrypting the local observation variables at the current time based on the global information obtained at the previous time, generating encrypted local information; dynamically aggregating the encrypted local information by dynamically generating aggregation weights to obtain the global information at the current time for collaborative perception by each agent; and generating local control commands for the corresponding area based on the encrypted local information to complete the real-time scheduling of the distribution network.

[0008] The present invention provides a preferred embodiment in its first aspect, wherein the distributed resources include: micro gas turbines, distributed photovoltaics, and distributed energy storage; the constraints include equipment constraints, voltage constraints, and power flow constraints; the equipment constraints include micro gas turbine equipment constraints, distributed photovoltaic equipment constraints, and distributed energy storage equipment constraints. This technical solution clarifies the specific types of distributed resources and sets equipment constraints for micro gas turbines, distributed photovoltaics, and distributed energy storage respectively, while also incorporating voltage and power flow constraints. By refining the resource types and constraints, the scheduling optimization model more closely reflects the physical characteristics and operational requirements of actual distribution networks, enhancing the model's accuracy and engineering applicability.

[0009] The present invention provides a preferred embodiment in its first aspect, wherein the step of transforming the distribution network scheduling optimization model into a Markov game model containing multiple agents includes: defining each control area as an agent, the number of agents being the same as the number of distribution network control areas; defining the local observation variables of the agent at the current moment, the local observation variables including the load of the local area nodes, the output of the micro gas turbine, the output of distributed photovoltaic power, the output of distributed energy storage, and the state of charge of energy storage; defining the action variables of the agent at the current moment, the action variables including local control commands for controlling controllable devices in the distributed resources of the local area; and defining the reward function of the agent at the current moment, the reward function including an operating cost term and a voltage over-limit penalty term, with each agent sharing the same reward function to achieve cooperation. This technical solution specifically specifies the definition method of agents in the Markov game model and the composition of observation variables, action variables, and reward functions. By mapping control areas to agents, defining a unified observation and action space, and sharing a cooperative reward function, a mathematical foundation for multi-agent collaborative optimization is established, ensuring the consistency of behavior of each agent in pursuing the global goal.

[0010] The present invention provides a preferred embodiment in a first aspect, wherein the step of constructing corresponding reinforcement learning agents for each agent in the Markov game model includes: for each agent, constructing a local measurement embedding network to map the local observation variables at the current time to a first implicit feature vector; constructing a global information embedding network to map the global information at the previous time to a second implicit feature vector; constructing an information encryption network to fuse the first implicit feature vector at the current time and the second implicit feature vector at the previous time to output an encrypted local information vector at the current time; constructing a policy network to generate local action variables at the current time based on the encrypted local information vector at the current time; constructing a value network to estimate the expected cumulative discount reward based on the encrypted local information vector at the current time and the local action variables at the current time; and constructing an entropy coefficient network to adaptively output an entropy coefficient based on the encrypted local information vector. This technical solution constructs a complete network structure for each agent, including a local measurement embedding network, a global information embedding network, an information encryption network, a policy network, a value network, and an entropy coefficient network. It provides a specific network architecture for implementing information encryption, dynamic aggregation, and strategy generation and evaluation. The introduction of the entropy coefficient network provides a technical means for subsequent adaptive adjustment of the exploration rate, enhancing the flexibility and adaptability of the algorithm.

[0011] The present invention provides a preferred embodiment in a first aspect, comprising the steps of: utilizing optimized reinforcement learning agents to dynamically encrypt local observation variables at the current time based on the global information acquired at the previous time step, generating encrypted local information; dynamically aggregating the encrypted local information through dynamically generated aggregation weights to obtain global information at the current time step for collaborative perception by the agents; and generating local control commands for the corresponding region based on the encrypted local information to complete the real-time scheduling of the power distribution network. The steps include: at the current time step, each optimized agent generates an encrypted local information vector based on the real-time acquired local observation variables at the current time step and the global information from the previous time step, through a local measurement embedding network, a global information embedding network, and an information encryption network; dynamically aggregating the encrypted local information vectors of each reinforcement learning agent through dynamically generated aggregation weights to obtain global information at the current time step for collaborative perception and encryption aggregation by the agents in the next time step; each agent inputting the encrypted local information vector into a policy network to generate local control commands for the corresponding region; and all agents issuing control commands to complete the power distribution network scheduling at the current time step. This technical solution specifies the steps for real-time scheduling using optimized agents, including generating encrypted information, dynamically aggregating global information, generating control commands, and issuing them. It clarifies the workflow of the trained model in actual operation, achieving collaborative perception among agents through the generation and aggregation of encrypted information, while ensuring the real-time generation and execution of scheduling commands, thus forming a complete online scheduling closed loop.

[0012] The present invention provides a preferred embodiment in its first aspect. Before the step of dynamically aggregating the encrypted local information by dynamically generating aggregation weights to obtain global information for collaborative perception by various intelligent agents, the method further includes: setting up a global aggregator and setting up a cluster composed of multiple control areas according to geographical location or electrical connection, with each cluster having a cluster aggregator; the cluster aggregator is an independent edge computing node, or is concurrently served by an intelligent agent in a certain control area within the cluster; the cluster aggregator receives the encrypted local information sent by all control areas within its cluster, performs a first aggregation, generates a cluster feature vector, and sends it to the global aggregator; the global aggregator performs a second aggregation on all cluster feature vectors to generate global information. This technical solution introduces a cluster-based hierarchical aggregation architecture, setting up a cluster aggregator and a global aggregator for two-level information aggregation. By dividing the control areas into clusters and establishing intermediate aggregation nodes, the connection complexity and data transmission volume of the communication network are effectively reduced, while adapting to the geographical distribution characteristics and hierarchical management requirements of large-scale power distribution networks, thus improving the scalability of the system.

[0013] The present invention provides a preferred embodiment in its first aspect, wherein the step of dynamically aggregating the encrypted local information vectors of each reinforcement learning agent to obtain global information at the current moment for collaborative perception and encryption aggregation at the next moment by dynamically generating aggregation weights includes: calculating the cosine similarity between the encrypted local information vector of each agent at the current moment and the global information at the previous moment, as the dynamic weight of the agent's contribution to the global information; broadcasting the encrypted local information vector and its corresponding cosine similarity at the current moment to each agent; and weighting the encrypted local information vectors of each agent at the current moment according to the cosine similarity to obtain the global information at the current moment for collaborative perception by each agent at the next moment to obtain the encrypted local information at the next moment and calculating the dynamic weights at the next moment, thereby completing information encryption and dynamic information aggregation. This technical solution specifically specifies the method for generating the dynamic aggregation weights, namely, calculating the cosine similarity between the encrypted local information vector and the global information at the previous moment as the weights, and performing a weighted average to obtain the global information at the current moment. A dynamic generation mechanism for aggregation weights was implemented, enabling global information to be adaptively fused based on the real-time contribution of each agent's information, thus avoiding the problems of invalid information propagation and dilution of key information caused by fixed weights.

[0014] The present invention provides a preferred embodiment in its first aspect, wherein the step of performing distributed reinforcement learning based on a reinforcement learning algorithm employs an improved SAC algorithm, specifically including: each agent interacting with the power distribution network environment and storing the interaction samples in its respective experience pool; randomly selecting training samples from the experience pool and calculating the value loss function, policy loss function, and entropy loss function for each agent; updating the parameters of the local measurement embedding network, global information embedding network, and information encryption network based on the value loss function and policy loss function; updating the parameters of the policy network based on the policy loss function; updating the parameters of the value network based on the value loss function; updating the parameters of the entropy coefficient network based on the entropy loss function; and repeating the above steps until the policies of each agent converge. This technical solution specifically specifies the training process for distributed reinforcement learning using the improved SAC algorithm, including experience pool storage, loss function calculation, and the updating method of each network parameter. It provides a complete and iteratively optimizable training method, which achieves adaptive adjustment of the exploration rate through the entropy loss function and entropy coefficient network, improving learning efficiency and convergence stability while ensuring policy diversity.

[0015] In a second aspect, this invention provides a multi-agent distribution network scheduling device based on dynamic information aggregation, comprising: a model building module for dividing the distribution network into multiple control areas, each control area containing distributed resources, establishing a distribution network scheduling optimization model with the objective of minimizing the operating cost of distributed resources, and setting constraints; a model transformation module for transforming the distribution network scheduling optimization model into a Markov game model containing multiple agents, each agent corresponding to a control area, each agent taking local observation variables as input and local control commands as output; and a model strengthening module for constructing corresponding strengthening mechanisms for each agent in the Markov game model. The system includes a learning agent module; a model optimization module, used to perform distributed reinforcement learning based on reinforcement learning algorithms to optimize the Markov game model and obtain the optimized reinforcement learning agents corresponding to each control region; and a real-time scheduling module, used to utilize the optimized reinforcement learning agents to dynamically encrypt the local observation variables at the current time based on the global information obtained at the previous time, generate encrypted local information, dynamically aggregate the encrypted local information by dynamically generating aggregation weights to obtain the global information at the current time for collaborative perception by each agent, and generate local control commands for the corresponding region based on the encrypted local information to complete the real-time scheduling of the distribution network.

[0016] Compared with the prior art, the present invention has the following advantages:

[0017] This invention first divides the power distribution network into multiple control areas and establishes a scheduling optimization model, which is then transformed into a Markov game model. Reinforcement learning agents are then constructed for distributed optimization, thus creating a distributed learning architecture. Each agent independently executes decisions and updates the model based on local observations, avoiding global information sharing and effectively protecting data privacy in each area. Second, this invention utilizes the optimized reinforcement learning agents to dynamically encrypt the local observation variables at the current moment based on the global information acquired at the previous moment, generating encrypted local information. This achieves information encryption during information interaction between agents, further reducing the risk of privacy leakage. Third, this invention dynamically generates aggregation weights to dynamically aggregate the encrypted local information to obtain the global information at the current moment for collaborative perception by all agents. This enables real-time scheduling, improves the collaborative efficiency and decision-making performance of the multi-agent system, meets the dynamic needs of rapidly changing power distribution network operating states, and achieves implicit collaboration with low overhead and high information utilization. This solves the problems of privacy leakage risks and low collaborative efficiency caused by static aggregation strategies in existing technologies. Attached Figure Description

[0018] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0019] Figure 1 This is a flowchart of a multi-agent power distribution network scheduling method based on dynamic information aggregation provided in Embodiment 1 of the present invention;

[0020] Figure 2 This is a flowchart of the model transformation sub-step in a multi-agent distribution network scheduling method based on dynamic information aggregation provided in Embodiment 1 of the present invention;

[0021] Figure 3 This is a flowchart of the model enhancement sub-step in a multi-agent distribution network scheduling method based on dynamic information aggregation provided in Embodiment 1 of the present invention;

[0022] Figure 4 This is a flowchart of the model optimization sub-step in a multi-agent distribution network scheduling method based on dynamic information aggregation provided in Embodiment 1 of the present invention;

[0023] Figure 5 This is a flowchart of the real-time scheduling sub-step in a multi-agent distribution network scheduling method based on dynamic information aggregation provided in Embodiment 1 of the present invention;

[0024] Figure 6 This is a block diagram of a multi-agent power distribution network dispatching device based on dynamic information aggregation provided in Embodiment 1 of the present invention;

[0025] Figure 7 This is a flowchart of the hierarchical aggregation step in a multi-agent distribution network scheduling method based on dynamic information aggregation provided in Embodiment 1 of the present invention. Detailed Implementation

[0026] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0027] Example 1: Please refer to Figure 1 This embodiment provides a multi-agent distribution network scheduling method based on dynamic information aggregation, mainly implemented through steps S1 to S5:

[0028] S1. Divide the distribution network into multiple control areas, each containing distributed resources. Establish a distribution network scheduling optimization model with the goal of minimizing the operating cost of distributed resources, and set constraints.

[0029] Step S1 mainly involves establishing a multi-agent distribution network scheduling problem model (hereinafter referred to as the distribution network scheduling optimization model). In this embodiment, the distribution network consists of N control areas, and the controllable equipment in each control area includes distributed resources such as micro gas turbines, distributed photovoltaics, and distributed energy storage. Each control area determines the distribution network based on local measurements. Calculate and issue local control commands This is used to achieve real-time dispatching of the power distribution network. Let i be the set of nodes in the i-th region; and Let be the active load and reactive load of the i-th regional node j at time t, respectively. and These represent the active and reactive power outputs of the micro gas turbine at the i-th regional node j at time t, respectively. and These are the active power output and reactive power output of the distributed photovoltaic system at the i-th regional node j at time t, respectively. and These represent the active power output and state of charge of the distributed energy storage at the i-th regional node j at time t, respectively. Let be the voltage amplitude of node j in the i-th region at time t.

[0030] In this embodiment, the objective function of the multi-agent distribution network scheduling problem model (distribution network scheduling optimization model) is to minimize the distribution network operating cost, and its expression is:

[0031] (1);

[0032] Where T is the scheduling period; Cost of unit output for micro gas turbines; The unit charge / discharge cost of distributed energy storage is 96. In this embodiment, T is 96. Although the controllable equipment in this embodiment includes distributed photovoltaics, in the optimization scheduling problem, compared with gas turbines and energy storage, the power generation cost of photovoltaics is usually not considered, that is, it is assumed that photovoltaic output and photovoltaic reactive power regulation do not generate economic losses.

[0033] In this embodiment, the constraints of the multi-agent distribution network scheduling problem model mainly include equipment constraints, voltage constraints, and power flow constraints. Among these, equipment constraints are set according to each distributed resource in the distributed resources, mainly including: micro gas turbine equipment constraints, distributed photovoltaic equipment constraints, and distributed energy storage equipment constraints. Specifically, the micro gas turbine equipment constraints include active power output constraints, reactive power output constraints, and ramping constraints. The active power output constraint is that the active power output of the micro gas turbine at the current node in the current region at the current moment is within the upper and lower limits of the active power output of the micro gas turbine at the current node in the current region. The reactive power output constraint is that the reactive power output of the micro gas turbine at the current node in the current region at the current moment is within the upper and lower limits of the reactive power output of the micro gas turbine at the current node in the current region. The ramping constraint is that the change in reactive power output of the micro gas turbine at the current node in the current region between two adjacent moments does not exceed the upper limit of the ramping rate of the micro gas turbine at the current node in the current region.

[0034] More specifically, the expressions for the active power output constraint, reactive power output constraint, and ramp constraint of the micro gas turbine are as follows:

[0035] (2);

[0036] (3);

[0037] (4);

[0038] in, and Let be the lower and upper limits of the active power output of the micro gas turbine on the i-th regional node j; and These are the lower and upper limits of the reactive power output of the micro gas turbine at the i-th regional node j, respectively. The ramp rate limit for the micro gas turbine on the i-th regional node j.

[0039] point The constraints on distributed photovoltaic (PV) equipment include capacity constraints, specifically: at the current moment, the sum of the squares of the active and reactive power outputs of distributed PV at the current node in the current region does not exceed the square of the installed capacity of distributed PV at the current node in the current region.

[0040] More specifically, the capacity constraint expression is:

[0041] (5);

[0042] in, Let be the installed capacity of distributed photovoltaic power on the i-th regional node j.

[0043] Constraints for distributed energy storage devices include active power output constraints, upper and lower bound constraints for the state of charge (SBC) limit, and state of charge transition constraints. Specifically, the active power output constraint requires that the active power output of distributed energy storage at the current node in the current region at the current moment be within the upper and lower bounds of the active power output of distributed energy storage at the current node in the current region. The upper and lower bound constraints for the SBC limit require that the state of charge of distributed energy storage at the current node in the current region at the current moment be within the upper and lower bounds of the SBC limit. The state of charge transition constraints are as follows:

[0044] When distributed energy storage is in a charging state, the state of charge of distributed energy storage at the current node in the current region at the current moment is equal to the state of charge of distributed energy storage at the current node in the current region at the previous moment minus the product of the active power output and charging efficiency of distributed energy storage at the current node in the current region at the current moment.

[0045] When distributed energy storage is in a discharging state, the state of charge of distributed energy storage at the current node in the current region at the current moment is equal to the state of charge of distributed energy storage at the current node in the current region at the previous moment minus the ratio of the active power output to the discharge efficiency of distributed energy storage at the current node in the current region at the current moment.

[0046] More specifically, the expressions for the active power output constraints, upper and lower limits of the state of charge constraints, and the state of charge transition constraints are as follows:

[0047] (6);

[0048] (7);

[0049] (8);

[0050] in, and Let be the lower and upper limits of the active power output of distributed energy storage on the i-th regional node j; and These are the upper and lower limits of the state of charge of distributed energy storage on the i-th regional node j, respectively; Let be the operating efficiency of distributed energy storage on the i-th regional node j. At that time, the distributed energy storage is in a charging state; At that time, the distributed energy storage is in a discharging state.

[0051] In this embodiment, the voltage constraint expression for the multi-agent distribution network scheduling problem model is:

[0052] (9);

[0053] in, and These are the lower and upper limits of the node voltage, respectively.

[0054] In this embodiment, the power flow constraint expression for the multi-agent distribution network scheduling problem model is as follows:

[0055] (10);

[0056] (11);

[0057] (12);

[0058] (13);

[0059] in, and These represent the active power injection and reactive power injection of the i-th region node j at time t, respectively. Let be the voltage phase angle difference between node j and node k in the i-th region at time t; These are the real and imaginary parts of the elements corresponding to nodes j and k in the admittance matrix of the i-th region, respectively.

[0060] S2. The power distribution network scheduling optimization model is transformed into a Markov game model containing multiple agents. Each agent corresponds to a control region. Each agent takes local observation variables as input and local control commands as output.

[0061] Step S2 mainly transforms the multi-agent power distribution network scheduling problem model into a Markov game model, including the number of agents, agent observation variables, agent action variables, and agent reward functions.

[0062] Please refer to Figure 2 In one optional implementation, the step of transforming the distribution network scheduling optimization model into a Markov game model involving multiple agents is mainly achieved through the following sub-steps:

[0063] S21. Define each control area as an agent, and the number of agents is the same as the number of control areas in the distribution network.

[0064] S22. Define the local observation variables of the agent at the current moment, wherein the local observation variables include the load of the local regional node, the output of the micro gas turbine, the output of the distributed photovoltaic, the output of the distributed energy storage, and the state of charge of the energy storage;

[0065] S23. Define the action variables of the intelligent agent at the current moment, wherein the action variables include local control instructions for controlling controllable devices in the distributed resources of the local area;

[0066] S24. Define the reward function of the agent at the current moment. The reward function includes an operating cost term and a voltage limit violation penalty term. All agents share the same reward function to achieve cooperation.

[0067] More specifically, in this embodiment, the Markov game model includes N agents, each corresponding to a power distribution network control area.

[0068] In this embodiment, in the Markov game model, agent i observes variables at time t. The expression is:

[0069] (14)

[0070] In this embodiment, the action variable of agent i at time t in the Markov game model The expression is:

[0071] (15)

[0072] In this embodiment, the reward function of agent i at time t in the Markov game model The expression is:

[0073] (16);

[0074] in, The weight of the voltage limit exceedance factor. Let be the voltage limit exceedance coefficient of the i-th region node j at time t, and its expression is:

[0075] (17);

[0076] in, The x here refers to... or The reward function is the same for all agents, meaning that the agents cooperate to minimize the overall operating cost of the distribution network and prevent voltage exceedances.

[0077] S3. Construct corresponding reinforcement learning agents for each agent in the Markov game model.

[0078] Step S3 mainly involves constructing reinforcement learning agents corresponding to each control region in the Markov game model, including a local measurement embedding network, a global information embedding network, an information encryption network, a policy network, a value network, and an entropy coefficient network.

[0079] Please refer to Figure 3 In an optional real-time mode, step S3 is specifically implemented through the following sub-steps:

[0080] For each agent:

[0081] S31. Construct a local measurement embedding network to map the local observation variables at the current time to the first implicit feature vector.

[0082] In this embodiment, the local measurement embedding network of the reinforcement learning agent i is Randomly initialize its parameters At time t, the local measurement embedding network The input is the locally observed variable at time t. The output is In this embodiment, the local measurement is embedded in the network. This is a fully connected neural network with two hidden layers, where the number of neurons in the input layer is... The dimension of the array is 64, the number of hidden layer neurons is 64, and the number of output layer neurons is 64. The dimension is 64.

[0083] S32. Construct a global information embedding network to map the global information from the previous time step into a second implicit feature vector.

[0084] In this embodiment, the global information embedding network of the reinforcement learning agent i is: Randomly initialize its parameters At time t, global information is embedded in the network. The input is the global information at time t-1. The output is In this embodiment, global information is embedded in the network. This is a fully connected neural network with two hidden layers, where the number of neurons in the input layer is... The dimension is 128, the number of hidden layer neurons is 64, and the number of output layer neurons is 64. The dimension is 64. Global information. It is a high-dimensional implicit vector (128 dimensions in this embodiment) that represents a condensed representation of the overall distribution network's operational status at the previous moment. It is not a simple superposition of the original measurement data, but rather a global operational status feature code formed after neural network encoding and dynamic filtering. It contains implicit expressions of key information such as the load level of each region, distributed power generation output, and energy storage status at the previous moment. Global Information It can serve as a memory carrier of the historical global situation, providing a reference for each agent when generating new encrypted information at the current moment. It also serves as a benchmark for calculating the dynamic weights (cosine similarity) at the current moment. (Output of the global information embedding network) The 128-dimensional global information is further reduced and compressed to 64 dimensions, extracting the most valuable global features for the current agent. Although the global information input to all agents is the same, the global information embedding network parameters for each agent are trained independently. Therefore, its output represents the key information obtained from interpreting the historical global situation from the agent's perspective.

[0085] S33. Construct an information encryption network to fuse the first implicit feature vector at the current time and the second implicit feature vector at the previous time, and output the encrypted local information vector at the current time.

[0086] In this embodiment, the information encryption network of the reinforcement learning agent i is Randomly initialize its parameters At time t, the information encryption network The input is and The output is encrypted local information. ,Right now In this embodiment, the information encryption network... This is a fully connected neural network with two hidden layers, where the number of neurons in the input layer is... and The sum of the dimensions is 128, the number of hidden layer neurons is 128, and the number of output layer neurons is 128. The dimension is 128.

[0087] S34. Construct a policy network to generate local action variables for the current time based on the encrypted local information vector at the current time.

[0088] In this embodiment, the policy network of the reinforcement learning agent i is Randomly initialize its parameters At time t, the policy network The input is encrypted local information. The output is a local action variable. ,Right now In this embodiment, the policy network This is a fully connected neural network with two hidden layers, where the number of neurons in the input layer is... The dimension is 128, the number of hidden layer neurons is 256, and the number of output layer neurons is... The dimension of.

[0089] S35. Construct a value network to estimate the expected cumulative discount reward based on the encrypted local information vector at the current time and the local action variables at the current time.

[0090] In this embodiment, the value network of the reinforcement learning agent i is Randomly initialize its parameters At time t, the value network The input is encrypted local information. and local action variables The output is the estimated cumulative discounted reward of the agent. In this embodiment, the value network This is a fully connected neural network with two hidden layers, where the number of neurons in the input layer is... and The sum of the dimensions of the layers is 256, the number of neurons in the hidden layer is 1, and the number of neurons in the output layer is 1.

[0091] S36. Construct an entropy coefficient network to adaptively output entropy coefficients based on the encrypted local information vector.

[0092] In this embodiment, the entropy coefficient network of the reinforcement learning agent i is: Randomly initialize its parameters At time t, the entropy coefficient network The input is encrypted local information. The output is the entropy coefficient. In this embodiment, the entropy coefficient network This is a fully connected neural network with two hidden layers, where the number of neurons in the input layer is... The dimension is 128, the number of hidden layer neurons is 256, and the number of output layer neurons is 1. In the improved SAC algorithm, an entropy coefficient network is used instead of a fixed entropy coefficient, which helps the agent adaptively learn the most suitable exploration rate in each state, thus improving the agent's learning efficiency and adaptability.

[0093] In this embodiment, during the process of constructing corresponding reinforcement learning agents for each agent in the Markov game model, the information interaction method, encryption method, and dynamic information aggregation method between reinforcement learning agents are simultaneously constructed. Specifically: at time t, agent i's local measurement embedding network... Input the locally observed variables at time t Output ; Global information embedding network of intelligent agent i Input the global information at time t-1 Output Intelligent agent i-information encryption network enter and Output encrypted local information ; Calculate local information and global information The cosine similarity is This represents agent i's contribution to the global algorithm; subsequently, all agents will share their local information. and cosine similarity Perform broadcasting and calculate global information at time t. The weighted average of the local information:

[0094] (18);

[0095] This completes the dynamic aggregation of information. On one hand, cosine similarity... The weights used for weighted aggregation are dynamically calculated based on local and global information rather than being fixed, enabling dynamic evaluation of the agent's contribution to the global process. On the other hand, the broadcast information is encrypted by the neural network, so the agent's local observations and strategies do not need to be exposed during the interaction, effectively protecting the privacy of each agent.

[0096] S4. Distributed reinforcement learning is performed based on reinforcement learning algorithms to optimize the Markov game model, obtaining the optimized reinforcement learning agents corresponding to each control region. In an optional implementation, the step of performing distributed reinforcement learning based on reinforcement learning algorithms employs an improved SAC algorithm. Step S4 mainly uses the improved SAC algorithm to perform distributed reinforcement learning to optimize the Markov game model, including local measurement embedding network updates, global information embedding network updates, information encryption network updates, policy network updates, value network updates, and entropy coefficient network updates.

[0097] Please refer to Figure 4 Step S4 is implemented through the following sub-steps:

[0098] S41. Each intelligent agent interacts with the power distribution network environment and stores the interaction samples in its own experience pool.

[0099] In this embodiment, each agent interacts with the power distribution network. For agent i, it obtains the interaction sample at time t. Stored in the experience pool of agent i In this embodiment, the size of the experience pool of agent i is... It is 8000.

[0100] S42. Randomly select training samples from the experience pool and calculate the value loss function, policy loss function, and entropy loss function for each agent.

[0101] (1) Calculate the value loss function for each agent: For agent i, from the experience pool A set of training samples is randomly selected from the middle. Calculate the value loss function In this embodiment, the training sample size... It is 128.

[0102] (19);

[0103] in, , yes The target value is expressed as:

[0104] (20);

[0105] in, , , This is the discount rate for future rewards. In this embodiment, It is 0.99. Let be the observed variable of agent i at time t. Let be the observed variable of agent i at time t+1. This provides global information at time t-1. This represents the global information at time t. This provides global information at time t+1. Let i be the encrypted local information of agent i at time t. This refers to the encrypted local information of agent i at time t+1. Let be the local action variable output by agent i at time t. Let be the local action variable output by agent i at time t+1. Let represent the immediate reward value obtained by agent i after performing an action at time t. See step S3 for the meaning of other parameters.

[0106] (2) Calculate the policy loss function for each agent: For agent i, from the experience pool A set of training samples is randomly selected from the middle. Calculate the policy loss function .

[0107] (twenty one);

[0108] in, , For the meanings of other parameters, please refer to step S3.

[0109] (3) Calculate the entropy loss function for each agent: For agent i, from the experience pool A set of training samples is randomly selected from the middle. Calculate the entropy loss function For the meanings of other parameters, please refer to step S3.

[0110] (twenty two);

[0111] in, , It is the target entropy. In this embodiment, It is -10.0.

[0112] S43. Based on the value loss function and the policy loss function, update the parameters of the local measurement embedding network, the global information embedding network, and the information encryption network.

[0113] (1) Update the local measurement embedding network: based on the value loss function calculated by each agent. With policy loss function Update local measurement embedded network parameters For agent i, the update method is as follows:

[0114] (twenty three);

[0115] in, For parameters The learning rate. In this embodiment, It is 0.001.

[0116] (2) Update global information embedding network: based on the value loss function calculated by each agent. With policy loss function Update global information embedded network parameters For agent i, the update method is as follows:

[0117] (twenty four);

[0118] in, For parameters The learning rate. In this embodiment, It is 0.001.

[0119] (3) Update the parameters of the information encryption network: based on the value loss function calculated by each agent. With policy loss function Update information encryption network parameters For agent i, the update method is as follows:

[0120] (25);

[0121] in, For parameters The learning rate. In this embodiment, It is 0.001.

[0122] S44. Update the parameters of the policy network based on the policy loss function.

[0123] Policy loss function calculated by each agent Update strategy network parameters For agent i, the update method is as follows:

[0124] (26);

[0125] in, For parameters The learning rate. In this embodiment, It is 0.0005.

[0126] S45. Update the parameters of the value network based on the value loss function.

[0127] Value loss function calculated by each agent Update network parameters For agent i, the update method is as follows:

[0128] (27);

[0129] in, For parameters The learning rate. In this embodiment, It is 0.001.

[0130] S46. Update the parameters of the entropy coefficient network based on the entropy loss function.

[0131] Entropy loss function calculated by each agent Update entropy coefficients and network parameters For agent i, the update method is as follows:

[0132] (28);

[0133] in, For parameters The learning rate. In this embodiment, It is 0.002.

[0134] S47. Repeat steps S41 to S46 until the policies of each agent converge.

[0135] In this embodiment, the criterion for determining the convergence of agent i is: .

[0136] S5. Utilizing the optimized reinforcement learning agents, based on the global information acquired at the previous time step, the local observation variables at the current time step are dynamically encrypted to generate encrypted local information. By dynamically generating aggregation weights, the encrypted local information is dynamically aggregated to obtain the global information at the current time step for collaborative perception by the agents. Based on the encrypted local information, local control commands for the corresponding regions are generated to complete the real-time scheduling of the distribution network. Step S5 mainly utilizes the optimized reinforcement learning agents corresponding to each control region to complete the multi-agent distribution network real-time scheduling.

[0137] Please refer to Figure 5 In this embodiment, step S5 is specifically implemented through the following steps:

[0138] S51. At the current moment, each optimized agent generates an encrypted local information vector based on the local observation variables acquired in real time at the current moment and the global information from the previous moment, through the local measurement embedding network, the global information embedding network, and the information encryption network.

[0139] S52. By dynamically generating aggregation weights, the encrypted local information vectors of each reinforcement learning agent are dynamically aggregated to obtain the global information at the current moment, which is then used by each agent to collaboratively perceive and encrypt the information in the next moment.

[0140] S53. Each agent inputs the encrypted local information vector into the policy network to generate local control commands for the corresponding region;

[0141] S54. All intelligent agents issue control commands to complete the power distribution network scheduling at the current moment.

[0142] Furthermore, step S52 is achieved through the following steps:

[0143] S521. Calculate the cosine similarity between the encrypted local information vector of each agent at the current time and the global information at the previous time, and use it as the dynamic weight of the agent's contribution to the global situation.

[0144] S522. Each agent broadcasts the encrypted local information vector and its corresponding cosine similarity at the current moment;

[0145] S523. The local information vectors encrypted by each agent at the current time are weighted and averaged according to the cosine similarity to obtain the global information at the current time. This information is then used by each agent to collaboratively perceive the next time to obtain the local information encrypted at the next time and to calculate the dynamic weights at the next time, thereby completing information encryption and dynamic information aggregation.

[0146] More specifically, at time t, agent i's local measurement is embedded in the network. Input the locally observed variables at time t Output ; Global information embedding network of intelligent agent i Input the global information at time t-1 Output Intelligent agent i-information encryption network enter and Output encrypted local information ; Calculate local information and global information The cosine similarity is This represents agent i's contribution to the global algorithm; subsequently, all agents will share their local information. and cosine similarity Perform broadcasting and calculate global information at time t. The weighted average of each local information is used to dynamically aggregate the information at time t; the encrypted local information Input agent i policy network Output the local control commands for the corresponding region. Once all intelligent agents have completed issuing instructions, the power distribution network scheduling at time t is completed.

[0147] Please refer to Figure 6 Corresponding to the method in this embodiment, a multi-agent distribution network dispatching device based on dynamic information aggregation is provided, which mainly consists of the following modules:

[0148] Model building module 1 is used to divide the distribution network into multiple control areas, each of which contains distributed resources. The goal is to establish a distribution network scheduling optimization model and set constraints to minimize the operating cost of distributed resources.

[0149] Model conversion module 2 is used to convert the power distribution network scheduling optimization model into a Markov game model containing multiple agents. Each agent corresponds to a control region, and each agent takes local observation variables as input and local control commands as output.

[0150] Model reinforcement module 3 is used to construct corresponding reinforcement learning agents for each agent in the Markov game model;

[0151] Model optimization module 4 is used to perform distributed reinforcement learning based on reinforcement learning algorithms to optimize the Markov game model and obtain the optimized reinforcement learning agents corresponding to each control region.

[0152] The real-time scheduling module 5 is used to utilize the optimized reinforcement learning agents to dynamically encrypt the local observation variables at the current time based on the global information obtained from the previous time step, generate various encrypted local information, dynamically aggregate the encrypted local information by dynamically generating aggregation weights to obtain the global information at the current time for collaborative perception by the agents, and generate local control commands for the corresponding area based on the encrypted local information to complete the real-time scheduling of the power distribution network.

[0153] Example 2: Considering that Example 1 uses a fully interconnected broadcast mode, the communication links grow exponentially when the number of control areas surges, easily leading to network congestion. To adapt to ultra-large-scale distribution networks and reduce communication pressure, this example proposes a cluster-based hierarchical aggregation architecture based on Example 1. The original control areas are further divided into several clusters, each containing multiple adjacent control areas, and a cluster aggregator is established as a middle-layer aggregation node, forming a three-layer information interaction structure of control area—cluster—global. By introducing a cluster aggregator as an intermediate layer, the hierarchical architecture transforms the communication mode from point-to-point full interconnection to a tree structure of edge aggregation-core distribution, significantly reducing cross-regional communication overhead and ensuring the scalability of the system in ultra-large-scale power grids from an architectural perspective.

[0154] Meanwhile, considering that actual power distribution networks have a hierarchical and partitioned geographical and electrical structure (such as substation-feeder-transformer area), hierarchical aggregation allows electrically connected agents to first aggregate information within the cluster to form local situational awareness, and then upload the condensed features to the global scale. This not only conforms to the natural physical properties of the power grid, but also improves the efficiency of global aggregation through local filtering.

[0155] Furthermore, considering that Implementation 1 achieves implicit communication through encryption, but in the fully interconnected mode, the compromise of a single node may still expose global information. The layered architecture uses a cluster aggregator as an intermediary to achieve information isolation and secondary desensitization. Even if a cluster is attacked, attackers cannot directly access the underlying data of other clusters, thus constructing a defense-in-depth system.

[0156] Based on Example 1, for step S5: S5. Utilizing the optimized reinforcement learning agents, based on the global information acquired in the previous time step, the local observation variables in the current time step are dynamically encrypted to generate encrypted local information. The encrypted local information is then dynamically aggregated using dynamically generated aggregation weights to obtain the global information in the current time step for collaborative perception by the agents. Based on the encrypted local information, local control commands for the corresponding area are generated to complete the real-time scheduling of the power distribution network. Prior to the step of dynamically aggregating the encrypted local information using dynamically generated aggregation weights to obtain the global information in the current time step for collaborative perception by the agents, hierarchical aggregation is also performed. Please refer to [reference needed]. Figure 7 Specifically, it includes the following sub-steps:

[0157] S501. Set up a global aggregator and set up a cluster consisting of multiple control areas according to geographical location or electrical connection, with each cluster having a cluster aggregator; the cluster aggregator is an independent edge computing node, or is concurrently served by an agent of a certain control area within the cluster;

[0158] S502. The cluster aggregator receives encrypted local information sent by all control areas within the cluster, performs the first aggregation, generates a cluster feature vector, and then sends it to the global aggregator.

[0159] S503. The global aggregator performs a second aggregation on all cluster feature vectors to generate global information.

[0160] The other steps are the same as in Example 1. That is, continue with:

[0161] S51. At the current moment, each optimized agent generates an encrypted local information vector based on the local observation variables acquired in real time at the current moment and the global information from the previous moment, through the local measurement embedding network, the global information embedding network, and the information encryption network.

[0162] S52. By dynamically generating aggregation weights, the encrypted local information vectors of each reinforcement learning agent are dynamically aggregated to obtain the global information at the current moment, which is then used by each agent to collaboratively perceive and encrypt the information in the next moment.

[0163] S53. Each agent inputs the encrypted local information vector into the policy network to generate local control commands for the corresponding region;

[0164] S54. All intelligent agents issue control commands to complete the power distribution network scheduling at the current moment.

[0165] More specifically, the implementation process is as follows:

[0166] First, the hierarchy is defined:

[0167] (1) Control Region (bottom layer): that is, the basic unit in Example 1. Each control region is equipped with a reinforcement learning agent, which is responsible for local observation, decision-making and local information encryption. There are N control regions in total.

[0168] (2) Cluster (Middle Layer): A collection of multiple control regions that are geographically adjacent or electrically connected. Each cluster has a cluster aggregator (which can be an independent edge computing node or an agent from a control region within the cluster). N control regions are divided into M clusters, and the set of control regions contained in the r-th cluster is denoted as […]. .

[0169] (3) Global (top level): Global information receiving end, which can be implemented by the scheduling center or global aggregator.

[0170] Secondly, the cluster aggregator function is defined as follows: Downlink: Receives encrypted local information sent by all control areas within the cluster, performs the first aggregation, and generates a cluster feature vector. Uplink: Sends the cluster feature vector to the global aggregator, receives global information from the global aggregator, and then forwards it to each control area within the cluster.

[0171] Secondly, construct the information aggregation and interaction process:

[0172] Step 1: Control zone generates encrypted local information

[0173] Each control region i ( Following the method of Example 1, based on current local observations Global information from the previous moment By using a local measurement embedding network, a global information embedding network, and an information encryption network, encrypted local information for the current moment is generated. .

[0174] Step 2: Aggregation within the cluster

[0175] Each control area will Send to the aggregator within the cluster. The cluster aggregator collects all control regions within its cluster. Then, the first aggregation is performed to obtain cluster features. Aggregation methods are available as follows:

[0176] Simple average:

[0177] (29);

[0178] Weighted average:

[0179] (30);

[0180] Attention aggregation: Deploy a multi-head attention mechanism within the cluster to aggregate the attention of each head. The data is treated as input, and after calculating attention weights, it is weighted and aggregated to obtain more refined cluster features.

[0181] Step 3: Inter-cluster aggregation

[0182] Each cluster aggregator will display the cluster characteristics of its own cluster. The data is sent to the global aggregator. The global aggregator performs a second aggregation on all M cluster features to generate global information. Aggregation methods can also include averaging, weighting, or attention mechanisms.

[0183] Step 4: Information Distribution and Global Information Utilization

[0184] The global aggregator will The broadcast is sent to all cluster aggregators. Each cluster aggregator, upon receiving the broadcast, forwards it to all control regions within its cluster. Control region i will... As global information at the current moment, the input global information is embedded in the network for generating encrypted information at the next moment, and also serves as the input context for the policy network (optional, depending on the design of the policy network).

[0185] Based on the above specific embodiments, the advantages of the present invention compared with the prior art are as follows:

[0186] 1. This invention proposes a multi-agent power distribution network scheduling method based on dynamic information aggregation. Each agent makes independent decisions and updates its model based on local observations and aggregated global information, without sharing original measurement data or strategy parameters. This fundamentally avoids the direct exposure of sensitive information (such as load characteristics, energy storage status, and power generation capacity), and effectively protects data privacy and security among multiple agents.

[0187] 2. This invention designs an implicit information interaction mechanism based on neural networks: each agent maps measurement information into a high-dimensional implicit vector through a local network to achieve information encryption; on this basis, a dynamic weight generation mechanism is introduced to adaptively adjust the aggregation weight of each agent's information according to the system's operating state to form global information, thereby achieving implicit collaboration with high information utilization and significantly improving the collaboration efficiency of multiple agents.

[0188] 3. This invention designs a cluster-based hierarchical aggregation architecture, setting up a cluster aggregator and a global aggregator for two-level information aggregation. By dividing the control area into clusters and establishing intermediate aggregation nodes, the connection complexity and data transmission volume of the communication network are effectively reduced, while adapting to the geographical distribution characteristics and hierarchical management requirements of large-scale power distribution networks, thus improving the system's scalability.

[0189] 4. This invention employs an improved SAC algorithm for distributed reinforcement learning, combined with an entropy coefficient adaptive mechanism, and explores and utilizes balancing strategies to enhance the agent's ability to perceive the overall situation of the system, thereby achieving safe, economical, and stable multi-agent collaborative scheduling.

[0190] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. The storage medium can be a magnetic disk, optical disk, read-only memory (ROM), or random access memory (RAM), etc.

[0191] The technical features of the above embodiments can be combined arbitrarily. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as the combination of these technical features does not contradict each other, it should be considered within the scope of this specification. Furthermore, the above embodiments only illustrate several implementation methods of the present invention, and their descriptions are relatively specific and detailed, but they should not be construed as limiting the scope of the present invention. For those skilled in the art, several modifications and improvements can be made without departing from the concept of the present invention, and these all fall within the protection scope of the present invention.

Claims

1. A multi-agent distribution network scheduling method based on dynamic information aggregation, characterized in that, include: The distribution network is divided into multiple control areas, each containing distributed resources. A distribution network scheduling optimization model is established with the goal of minimizing the operating cost of distributed resources, and constraints are set. The power distribution network scheduling optimization model is transformed into a Markov game model containing multiple agents. Each agent corresponds to a control region, and each agent takes local observation variables as input and local control commands as output. Construct corresponding reinforcement learning agents for each agent in the Markov game model; Distributed reinforcement learning is performed based on reinforcement learning algorithms to optimize the Markov game model and obtain the optimized reinforcement learning agents corresponding to each control region. Utilizing optimized reinforcement learning agents, based on the acquired global information from the previous time step, the local observation variables at the current time step are dynamically encrypted to generate encrypted local information. By dynamically generating aggregation weights, the encrypted local information is dynamically aggregated to obtain the global information at the current time step, which is then used for collaborative perception by the agents. Based on the encrypted local information, local control commands for the corresponding area are generated to complete the real-time scheduling of the power distribution network; including: At the current moment, each optimized agent generates an encrypted local information vector based on the local observation variables acquired in real time and the global information from the previous moment, through the local measurement embedding network, the global information embedding network, and the information encryption network. By dynamically generating aggregation weights, the encrypted local information vectors of each reinforcement learning agent are dynamically aggregated to obtain the global information at the current moment, which is then used by each agent to collaboratively perceive and encrypt the information in the next moment. Each agent inputs the encrypted local information vector into the policy network to generate local control commands for the corresponding region; All intelligent agents issue control commands to complete the power distribution network scheduling at the current moment; The step of dynamically aggregating the encrypted local information vectors of each reinforcement learning agent by dynamically generating aggregation weights to obtain global information at the current moment for collaborative perception and encryption aggregation by each agent in the next moment includes: Calculate the cosine similarity between the encrypted local information vector of each agent at the current time and the global information at the previous time, and use it as the dynamic weight of the agent's contribution to the global situation. Each agent broadcasts the encrypted local information vector and its corresponding cosine similarity at the current moment; The local information vectors encrypted by each agent at the current moment are weighted and averaged according to the cosine similarity to obtain the global information at the current moment. This information is then used by each agent to collaboratively perceive the next moment, obtain the encrypted local information for the next moment, and calculate the dynamic weights for the next moment. This completes information encryption and dynamic information aggregation.

2. The multi-agent distribution network scheduling method based on dynamic information aggregation according to claim 1, characterized in that, The distributed resources include: micro gas turbines, distributed photovoltaics, and distributed energy storage; the constraints include equipment constraints, voltage constraints, and power flow constraints; the equipment constraints include micro gas turbine equipment constraints, distributed photovoltaic equipment constraints, and distributed energy storage equipment constraints.

3. The multi-agent distribution network scheduling method based on dynamic information aggregation according to claim 1, characterized in that, The step of transforming the distribution network scheduling optimization model into a Markov game model involving multiple agents includes: Each control area is defined as an intelligent agent, and the number of intelligent agents is the same as the number of control areas in the distribution network. Define the local observation variables of the agent at the current moment. The local observation variables include the load of the local regional node, the output of the micro gas turbine, the output of the distributed photovoltaic, the output of the distributed energy storage, and the state of charge of the energy storage. Define the action variables of the intelligent agent at the current moment. The action variables include local control instructions for controlling controllable devices in the distributed resources of the local area. Define a reward function for the agent at the current moment. The reward function includes an operating cost term and a voltage limit violation penalty term. All agents share the same reward function to achieve cooperation.

4. The multi-agent distribution network scheduling method based on dynamic information aggregation according to claim 1, characterized in that, The steps for constructing corresponding reinforcement learning agents for each agent in the Markov game model include: For each agent, Construct a local measurement embedding network to map the local observation variables at the current time to the first implicit feature vector; A global information embedding network is constructed to map the global information from the previous time step into a second implicit feature vector; Construct an information encryption network to fuse the first implicit feature vector at the current time and the second implicit feature vector at the previous time, and output the encrypted local information vector at the current time. Construct a policy network to generate local action variables for the current time based on the encrypted local information vector at the current time. Construct a value network to estimate the expected cumulative discount reward based on the encrypted local information vector at the current time and the local action variable at the current time; An entropy coefficient network is constructed to adaptively output the entropy coefficient based on the encrypted local information vector.

5. The multi-agent distribution network scheduling method based on dynamic information aggregation according to claim 4, characterized in that, Before the step of dynamically aggregating the encrypted local information by dynamically generating aggregation weights to obtain the global information at the current moment for collaborative perception by various intelligent agents, the method further includes: A global aggregator is set up, and clusters consisting of multiple control areas are set up according to geographical location or electrical connection. Each cluster is set up with a cluster aggregator. The cluster aggregator is an independent edge computing node, or it can be a smart agent of a certain control area within the cluster. The cluster aggregator receives encrypted local information sent by all control areas within the cluster, performs the first aggregation, generates a cluster feature vector, and then sends it to the global aggregator. The global aggregator performs a second aggregation on all cluster feature vectors to generate global information.

6. The multi-agent distribution network scheduling method based on dynamic information aggregation according to claim 2, characterized in that, The constraints on the micro gas turbine equipment include active power output constraints, reactive power output constraints, and ramping constraints. Specifically, the active power output constraint is that the active power output of the micro gas turbine at the current node in the current region at the current moment is within the upper and lower limits of the active power output of the micro gas turbine at the current node in the current region. The reactive power output constraint is that the reactive power output of the micro gas turbine at the current node in the current region at the current moment is within the upper and lower limits of the reactive power output of the micro gas turbine at the current node in the current region. The ramping constraint is that the change in reactive power output of the micro gas turbine at the current node in the current region between two adjacent moments does not exceed the upper limit of the ramping rate of the micro gas turbine at the current node in the current region.

7. The multi-agent distribution network scheduling method based on dynamic information aggregation according to claim 2, characterized in that, The constraints on distributed photovoltaic (PV) equipment include capacity constraints, specifically: the sum of the squares of the active and reactive power outputs of distributed PV at the current node in the current region at the current moment shall not exceed the square of the installed capacity of distributed PV at the current node in the current region.

8. The multi-agent distribution network scheduling method based on dynamic information aggregation according to claim 2, characterized in that, The constraints on the distributed energy storage device include energy storage active power output constraints, state of charge (SOC) upper and lower limit constraints, and state of charge (SOC) transition constraints; wherein, the energy storage active power output constraint is that the active power output of the distributed energy storage at the current node in the current region at the current moment is within the upper and lower limits of the active power output of the distributed energy storage at the current node in the current region; the state of charge (SOC) upper and lower limit constraints are that the state of charge of the distributed energy storage at the current node in the current region at the current moment is within the upper and lower limits of the state of charge of the distributed energy storage at the current node in the current region. The charge state transition constraint is: When distributed energy storage is in a charging state, the state of charge (SOC) of the distributed energy storage at the current node in the current region at the current moment is equal to the SOC of the distributed energy storage at the previous moment minus the product of the active power output and charging efficiency of the distributed energy storage at the current node in the current region at the current moment. When distributed energy storage is in a discharging state, the SOC of the distributed energy storage at the current node in the current region at the current moment is equal to the SOC of the distributed energy storage at the previous moment minus the ratio of the active power output and discharging efficiency of the distributed energy storage at the current node in the current region at the current moment.

9. The multi-agent distribution network scheduling method based on dynamic information aggregation according to claim 4, characterized in that, The step of performing distributed reinforcement learning based on reinforcement learning algorithms involves using an improved SAC algorithm, specifically including: S41. Each intelligent agent interacts with the power distribution network environment and stores the interaction samples in its own experience pool; S42. Randomly draw training samples from the experience pool and calculate the value loss function, policy loss function and entropy loss function for each agent respectively; S43. Based on the value loss function and the policy loss function, update the parameters of the local measurement embedding network, the global information embedding network, and the information encryption network; S44. Update the parameters of the policy network based on the policy loss function; S45. Update the parameters of the value network based on the value loss function; S46. Update the parameters of the entropy coefficient network based on the entropy loss function; S47. Repeat steps S41 to S46 until the policies of each agent converge.

10. A multi-agent power distribution network dispatching device based on dynamic information aggregation, characterized in that, include: The model building module is used to divide the distribution network into multiple control areas, each containing distributed resources. The goal is to build a distribution network scheduling optimization model and set constraints to minimize the operating cost of distributed resources. The model conversion module is used to convert the power distribution network scheduling optimization model into a Markov game model containing multiple agents. Each agent corresponds to a control region, and each agent takes local observation variables as input and local control commands as output. The model reinforcement module is used to construct corresponding reinforcement learning agents for each agent in the Markov game model. The model optimization module is used to perform distributed reinforcement learning based on reinforcement learning algorithms to optimize the Markov game model and obtain the optimized reinforcement learning agents corresponding to each control region. The real-time scheduling module utilizes optimized reinforcement learning agents to dynamically encrypt local observation variables at the current moment based on the global information acquired at the previous time step, generating encrypted local information. It then dynamically aggregates this encrypted local information using dynamically generated aggregation weights to obtain the current moment's global information for collaborative perception by the agents. Based on this encrypted local information, it generates local control commands for the corresponding area, completing the real-time scheduling of the power distribution network. This includes: At the current moment, each optimized agent generates an encrypted local information vector based on the local observation variables acquired in real time and the global information from the previous moment, through the local measurement embedding network, the global information embedding network, and the information encryption network. By dynamically generating aggregation weights, the encrypted local information vectors of each reinforcement learning agent are dynamically aggregated to obtain the global information at the current moment, which is then used by each agent to collaboratively perceive and encrypt the information in the next moment. Each agent inputs the encrypted local information vector into the policy network to generate local control commands for the corresponding region; All intelligent agents issue control commands to complete the power distribution network scheduling at the current moment; The step of dynamically aggregating the encrypted local information vectors of each reinforcement learning agent by dynamically generating aggregation weights to obtain global information at the current moment for collaborative perception and encryption aggregation by each agent in the next moment includes: Calculate the cosine similarity between the encrypted local information vector of each agent at the current time and the global information at the previous time, and use it as the dynamic weight of the agent's contribution to the global situation. Each agent broadcasts the encrypted local information vector and its corresponding cosine similarity at the current moment; The local information vectors encrypted by each agent at the current moment are weighted and averaged according to the cosine similarity to obtain the global information at the current moment. This information is then used by each agent to collaboratively perceive the next moment, obtain the encrypted local information for the next moment, and calculate the dynamic weights for the next moment. This completes information encryption and dynamic information aggregation.

Images