Power grid air-ground cooperative emergency control system and method based on unmanned aerial vehicle multi-agent reinforcement learning

Abstract

The invention discloses a power grid air-ground cooperative emergency control system and method based on unmanned aerial vehicle multi-agent reinforcement learning, and belongs to the technical field of power system emergency control and intelligent cooperation. Comprising the following steps: a central coordination unit obtains a voltage state and a load importance degree of a power grid node and a position, energy and a communication state of an unmanned aerial vehicle cluster in real time, an emergency priority node is identified based on voltage recovery deviation and communication link quality, dynamic clustering is carried out, and a comprehensive communication demand priority of each node cluster is calculated; according to the method, the strong coupling optimization problem of power grid voltage recovery and emergency communication guarantee in a disaster environment is solved, the power supply reliability, the communication connectivity and the emergency response efficiency of the system are improved, the energy utilization of the unmanned aerial vehicle is optimized at the same time, and the energy utilization rate of the unmanned aerial vehicle is improved. The method is suitable for rapid recovery and cooperative scheduling of key infrastructures in extreme scenes such as earthquakes and typhoons.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of power emergency control and mobile intelligent cooperation, in particular to a power grid air-ground cooperative emergency control system and method based on unmanned aerial vehicle multi-agent reinforcement learning. BACKGROUND

[0002] With the continuous expansion of new energy access scale and the increasing complexity of distribution network structure, when the power system is subjected to natural disasters, equipment failures or external damage, local voltage fluctuations, key load power supply shortages and control instructions that cannot be issued may occur, thereby affecting the efficiency of emergency command and the continuous power supply capacity of key infrastructure such as livelihood, hospitals and communications. Although the existing power grid control system relies on fixed deployment of communication networks and automation control devices, in extreme scenarios such as earthquakes, floods and freezing, communication links are easily interrupted or degraded, making the power grid recovery process face prominent problems such as limited communication, limited perception and delayed scheduling.

[0003] Unmanned aerial vehicles have the advantages of mobility, rapid deployment and adjustable coverage, and have been gradually applied to disaster site investigation, communication emergency access and power supply support assistance scenarios in recent years. By using unmanned aerial vehicles as air mobile communication relay nodes, local communication capabilities can be quickly restored when ground communication facilities are damaged; at the same time, unmanned aerial vehicles can undertake data backhaul tasks for key areas of the power grid, providing auxiliary information support for power grid operation recovery. However, a single unmanned aerial vehicle is limited by endurance, communication bandwidth and computing resources, and is difficult to continuously and stably support the power grid emergency recovery process.

[0004] Therefore, a multi-unmanned aerial vehicle cooperative system is a feasible means to improve the resilience of disaster scenarios. Multiple unmanned aerial vehicles can dynamically adjust their positions according to the on-site emergency needs to form an air communication network that can cover the disaster area, and participate in data forwarding, information aggregation and edge collaborative computing operations. However, existing researches mostly focus on the optimization of single task objectives, such as trajectory planning, task allocation or resource allocation, and fail to achieve deep integration of power grid recovery needs and unmanned aerial vehicle cooperation capabilities. In particular, in complex disaster scenarios, there is a close coupling between power grid node voltage recovery, key load protection needs and unmanned aerial vehicle energy consumption, communication link quality, and if there is a lack of joint optimization mechanism at the system level, it is difficult to achieve the overall optimal emergency control effect.

[0005] Therefore, it is urgent to develop a joint control method that can simultaneously consider the flight and task execution characteristics of the unmanned aerial vehicle, the voltage recovery process of the power grid node, and the dynamic change characteristics of the air-ground communication link, and realize information sharing, strategy linkage, and global collaboration between the unmanned aerial vehicle cluster and the power grid node through a multi-agent collaborative learning mechanism, so as to improve the resilience and emergency disposal efficiency of the power system in a disaster scenario. Therefore, it is necessary to design a power grid air-ground collaborative emergency control system and method based on unmanned aerial vehicle multi-agent reinforcement learning. SUMMARY

[0006] The purpose of the present application is to provide a power grid air-ground collaborative emergency control system and method based on unmanned aerial vehicle multi-agent reinforcement learning to solve the problems raised in the background art.

[0007] To solve the above technical problems, the present application provides the following technical solution: a power grid air-ground collaborative emergency control system based on unmanned aerial vehicle multi-agent reinforcement learning, which is a disaster scenario-oriented unmanned aerial vehicle and power grid integrated intelligent emergency control system. The system includes a plurality of power grid control agents deployed at key nodes of the power grid , a plurality of unmanned aerial vehicle agents with communication capabilities , and a central coordination unit.

[0008] The power grid control agent is configured at each bus or distribution node to collect real-time node voltage and load state information and perform local voltage regulation and load control. The unmanned aerial vehicle agent is used to provide air communication relay, resource allocation, and path planning services in a disaster environment, and has an integrated edge computing module and wireless communication module. The central coordination unit is in communication with all power grid control agents and unmanned aerial vehicle agents to gather global state information, generate comprehensive demand priorities for each emergency node cluster, broadcast key state information to each agent, calculate global reward signals based on system comprehensive loss in the training phase, guide multi-agent policy updates, and realize collaborative control of unmanned aerial vehicles and power grids through multi-agent reinforcement learning mechanism. The unmanned aerial vehicle cluster self-adaptively plans flight trajectories and communication scheduling according to the power grid risk state, and the power grid agent dynamically adjusts load distribution and voltage recovery strategies according to the unmanned aerial vehicle feedback, thereby realizing collaborative optimization of power emergency communication and voltage control.

[0009] An emergency control method for a power grid air-ground collaborative emergency control system based on unmanned aerial vehicle multi-agent reinforcement learning, comprising the following steps:

[0010] Step A: The central coordination unit obtains the voltage amplitude of each bus node in the power grid and the key load operating state in the current time slot, as well as the position information, remaining energy and communication link quality of multiple unmanned aerial vehicles; according to the obtained voltage recovery deviation and load importance, the operating risk of the power grid node is evaluated, when the bus voltage is lower than the transient recovery envelope or the communication link is interrupted, it means that the node has voltage instability or communication loss risk, and its data interaction demand with the central control unit increases significantly, at this time, the node is judged as an emergency priority node, and the unmanned aerial vehicle needs to fly to its airspace to provide communication relay and additional special communication capacity to ensure the high-speed backhaul of key state data and the reliable issuance of complex control instructions, thereby supporting the real-time and accurate transmission of emergency control information, on this basis, based on the principle of geographical proximity and the communication coverage radius of the unmanned aerial vehicle, the emergency priority nodes are dynamically aggregated into multiple emergency node clusters, and the comprehensive demand priority of each cluster is calculated, including the weighted sum of the node operating risk value in the cluster and the overall communication loss severity of the cluster;

[0011] Step B: A joint optimization model of unmanned aerial vehicle and power grid cooperative emergency control is constructed, taking the minimum system total loss as the objective function, the system total loss is composed of three parts: comprehensive demand guarantee loss, unmanned aerial vehicle resource consumption cost and communication resource consumption cost, the decision variables of the unmanned aerial vehicle side and the power grid side are optimized, wherein the optimization variables of the unmanned aerial vehicle side include its dynamic service matching relationship with the emergency node cluster, two-dimensional trajectory planning and communication and calculation resource allocation strategy, the optimization variables of the power grid side include the voltage recovery control and load control of each node; the optimization model considers the unmanned aerial vehicle energy constraint, communication link quality constraint and power grid transient stability constraint under the unified objective function, so as to realize the integration and cooperation of unmanned aerial vehicle emergency communication and power grid recovery control on the premise of ensuring power supply continuity and network reliability;

[0012] Step C: A multi-agent reinforcement learning algorithm is used to solve the joint optimization problem, the central coordination unit broadcasts the key global state information such as the comprehensive communication demand priority of each emergency node cluster to all agents, and calculates the global reward signal based on the system comprehensive loss, which is used to update the strategy in the training stage; each unmanned aerial vehicle agent outputs its target node cluster selection, two-dimensional flight displacement and communication resource allocation joint action through the strategy network according to its own local state and received global priority information; each power grid node agent outputs voltage adjustment and load control action through the strategy network according to its local state and available unmanned aerial vehicle communication link state; in the execution stage after the strategy converges, each agent makes distributed autonomous decision and control based on the above mechanism, so as to realize the precise communication coverage of unmanned aerial vehicles to high-risk clusters and the rapid cooperation of power grid autonomous adjustment.

[0013] In further embodiments, the power grid operation risk assessment in step A includes the following processes:

[0014] (1) Calculate the node operation risk value according to the voltage recovery deviation of the power grid bus node in the current time slot and the load importance;

[0015] (2) According to the calculated risk value and the communication state, the emergency priority of the node is determined.

[0016] In further embodiments, in step B, the loss of coordination support is the sum of the integrated demand priorities of all emergency node clusters not served by the unmanned aerial vehicle, which integrates the power grid operation risk and the communication loss severity in the cluster to quantify the system risk caused by the lack of support; the unmanned aerial vehicle resource consumption cost is the total energy consumption of all unmanned aerial vehicles performing tasks; in the joint optimization model, the communication link quality between the unmanned aerial vehicle and the node directly determines the communication resource consumption cost.

[0017] In further embodiments, in step C, a multi-agent reinforcement learning algorithm is used to solve the joint optimization model in step B, and each unmanned aerial vehicle and power grid node is modeled as an independent agent to jointly learn the cooperative control strategy in the framework of centralized training and distributed execution; in the training phase, the central coordination unit collects the integrated communication demand priority of each emergency node cluster, the unmanned aerial vehicle operating state, the power grid node voltage state and the communication link quality information, constructs the global state, and calculates the global reward signal according to the total system loss, and synchronously updates the policy network and value network parameters of all agents through the policy gradient method; in the execution phase, each unmanned aerial vehicle agent outputs a joint action including target node cluster selection, two-dimensional flight displacement and communication resource allocation according to its own state and received global priority information; each power grid node agent outputs voltage regulation and load control actions according to the local voltage deviation, load state and available communication link conditions; through the mechanism of centralized training and distributed execution, the unmanned aerial vehicle realizes precise communication coverage of high-risk node clusters and the power grid node realizes rapid coordination of autonomous adjustment, and finally forms a closed-loop emergency control of communication recovery supporting voltage stability.

[0018] In further embodiments, in the multi-agent reinforcement learning solving process, each unmanned aerial vehicle is modeled as an independent unmanned aerial vehicle agent.

[0019] In further embodiments, in the multi-agent reinforcement learning solving process, each power grid node is modeled as a power grid node agent.

[0020] In further embodiments, in the multi-agent reinforcement learning solving process of step C, the coordinated action of the unmanned aerial vehicle agent and the power grid node agent in the disaster scene is taken as the goal, and a multi-agent reinforcement learning framework of centralized training and distributed execution is used for joint updating.

[0021] Compared with the prior art, the present application has the beneficial effects that: through the joint optimization control mechanism of air-ground cooperation, combined with unmanned aerial vehicle trajectory planning, communication resource allocation and power grid voltage and load adjustment strategy, the present application effectively solves the problem of mutual disconnection of power control and communication support, slow response in traditional post-disaster recovery, through the construction of a joint optimization model with the goal of minimizing the overall risk of the system, the power grid node operation risk, communication link interruption cost and unmanned aerial vehicle cluster energy consumption are comprehensively considered, the dynamic coverage of unmanned aerial vehicles to emergency priority nodes, communication relay and coordinated decision of power grid voltage recovery strategy are realized, at the same time, the multi-agent deep reinforcement learning algorithm based on centralized training and distributed execution framework is introduced, the high-dimensional mixed integer nonlinear programming problem is converted into a learnable cooperative control strategy, the continuous flight trajectory of the unmanned aerial vehicle, the communication resource scheduling decision and the continuous voltage and load adjustment instruction of the power grid are jointly optimized, which can adaptively cope with the dynamic changes of the power grid state and the communication topology in the partially observable disaster environment, while meeting the energy constraints of the unmanned aerial vehicle, the communication connectivity constraints and the power grid transient stability constraints, the power supply stability and the communication connectivity rate of the post-disaster system are significantly improved, the overall energy consumption of the unmanned aerial vehicle cluster and the system recovery delay are effectively reduced, which has excellent real-time performance, adaptability and scalability, provides an efficient intelligent solution for the rapid recovery of key infrastructure in major disasters such as earthquakes and typhoons, and has important practical application value. BRIEF DESCRIPTION OF DRAWINGS

[0022] Figure 1 is a method flowchart of the present application;

[0023] Figure 2 is an algorithm framework diagram of the present application;

[0024] Figure 3 is an emergency node dynamic scheduling scene schematic diagram of the present application;

[0025] Figure 4 is an emergency node dynamic scheduling scene schematic diagram of the present application for showing the spatial layout and task allocation mechanism of multi-unmanned aerial vehicle cooperative coverage and emergency node priority response in the case of power grid local fault or load fluctuation;

[0026] Figure 5 is an emergency scheduling system training convergence curve diagram of the present application for showing the change of total reward and convergence process of the system round of multi-unmanned aerial vehicle after reinforcement learning in the power grid emergency node dynamic scheduling task. DETAILED DESCRIPTION

[0027] In the following description, numerous specific details are set forth to provide a more thorough understanding of the present application. However, it will be apparent to one of skill in the art upon

[0028] Referring now to the drawings Figures 1-5 , the present application provides a technical solution: a power grid air-ground collaborative emergency control system based on unmanned aerial vehicle multi-agent reinforcement learning, which is a disaster scene-oriented unmanned aerial vehicle and power grid integrated intelligent emergency control system. The system includes a plurality of power grid control agents deployed at key nodes of the power grid , a plurality of unmanned aerial vehicle agents with communication capabilities , and a central coordination unit deployed at an emergency command center or dispatch control room, which has edge computing and centralized training capabilities, is used to gather power grid operation information and unmanned aerial vehicle state data, calculate the global state and comprehensive risk of the system, and issue a global reward signal to realize centralized training and strategy distribution.

[0029] The unmanned aerial vehicle agent group is distributed in the airspace of the disaster area. Each unmanned aerial vehicle integrates a flight control unit, a communication relay module, a task calculation module, and a power management system, has adaptive flight and dynamic communication capabilities, and maintains communication with the central coordination unit through 5G, LTE, or emergency microwave links. The power grid control agent group is deployed at each bus node or distribution transformer, collects voltage, current, and load state information in real time, and has a local adjustment function (such as voltage recovery, load reduction, or switching). The above three types of units form a collaborative closed-loop structure through an air-ground heterogeneous communication network, enabling the aerial relay function of the unmanned aerial vehicle and the self-recovery control of the power grid to interact with each other.

[0030] A power grid air-ground collaborative emergency control method based on unmanned aerial vehicle multi-agent reinforcement learning, comprising the following steps:

[0031] Step A: The central coordination unit obtains the voltage amplitude of each bus node in the power grid and the key load operating state in the current time slot, as well as the position information, remaining energy and communication link quality of multiple unmanned aerial vehicles; according to the obtained voltage recovery deviation and load importance, the operating risk of the power grid node is evaluated, when the bus voltage is lower than the transient recovery envelope or the communication link is interrupted, it means that the node has voltage instability or communication loss risk, and its data interaction demand with the central control unit increases significantly, at this time, the node is judged as an emergency priority node, and the unmanned aerial vehicle needs to fly to its airspace to provide communication relay and additional special communication capacity to ensure the high-speed backhaul of key state data and the reliable issuance of complex control instructions, thereby supporting the real-time and accurate transmission of emergency control information, on this basis, based on the principle of geographical proximity and the communication coverage radius of the unmanned aerial vehicle, the emergency priority nodes are dynamically aggregated into multiple emergency node clusters, and the comprehensive demand priority of each cluster is calculated, including the weighted sum of the node operating risk value in the cluster and the overall communication loss severity of the cluster;

[0032] Step B: A joint optimization model of unmanned aerial vehicle and power grid cooperative emergency control is constructed, taking the minimum system total loss as the objective function, and the system total loss is composed of three parts: comprehensive demand guarantee loss, unmanned aerial vehicle resource consumption cost and communication resource consumption cost. The decision variables of the unmanned aerial vehicle side and the power grid side are optimized, wherein the optimization variables of the unmanned aerial vehicle side include its dynamic service matching relationship with the emergency node cluster, two-dimensional trajectory planning and communication and calculation resource allocation strategy, and the optimization variables of the power grid side include the voltage recovery control and load control of each node; the optimization model considers the unmanned aerial vehicle energy constraint, communication link quality constraint and power grid transient stability constraint under the unified objective function, so as to realize the integration and cooperation of unmanned aerial vehicle emergency communication and power grid recovery control on the premise of ensuring power supply continuity and network reliability;

[0033] Step C: A multi-agent reinforcement learning algorithm is used to solve the joint optimization problem, the central coordination unit broadcasts the key global state information such as the comprehensive communication demand priority of each emergency node cluster to all agents, and calculates the global reward signal based on the system comprehensive loss, which is used to update the strategy in the training stage; each unmanned aerial vehicle agent outputs its target node cluster selection, two-dimensional flight displacement and communication resource allocation joint action through the strategy network according to its own local state and received global priority information; each power grid node agent outputs voltage adjustment and load control action through the strategy network according to its local state and available unmanned aerial vehicle communication link state; in the execution stage after the strategy converges, each agent makes distributed autonomous decision and control based on the above mechanism, so as to realize the precise communication coverage of unmanned aerial vehicles to high-risk clusters and the rapid cooperation of power grid autonomous adjustment.

[0034] In a further embodiment, in step A, the central coordination unit collects and uploads the grid operation status information in real time at the current time slot t through grid control agents deployed at each bus or distribution node of the grid, including the actual voltage measurement value of the i-th grid node in time slot t. and the real-time power of the load carried by the node. Simultaneously, the drone intelligent agent acquires operational status information of the drone swarm, including the location information of the drones, through its built-in sensors and communication modules. Current remaining battery power Performance indicators of the communication link between UAV u and power grid node i The aforementioned data is transmitted to the central coordination unit via a dedicated air-to-ground communication link and stored in a real-time database for subsequent risk assessment and mission decision-making.

[0035] Based on the acquired grid-side status information, the central coordination unit conducts an operational risk assessment for each grid node to identify high-risk nodes requiring emergency response in the current time slot. The node's operational risk value is calculated based on the voltage recovery deviation and load importance of the grid bus node in the current time slot. The calculation formula is as follows:

[0036] ;

[0037] in, This represents the operational risk value of the i-th node in time slot t. This indicates the voltage amplitude at that node. It is the steady-state reference voltage. This indicates the real-time load importance of the node. α represents the maximum allowable load that a node can carry, and β represents risk weighting coefficients, which are used to measure the proportion of the impact of voltage deviation and load occupancy on the node's operational risk, respectively.

[0038] Furthermore, the dynamic update relationship between node voltage and load is as follows:

[0039] ;

[0040] in This indicates the amount of voltage regulation performed by the node in the current time slot, used to restore voltage by adjusting the reactive power compensation device or transformer tap. This refers to the load adjustment parameters at nodes, used to reduce or transfer power to non-critical loads to ensure stable power supply to critical loads. The mapping function of the impact of load regulation on node voltage can be obtained through the power flow model. This dynamic relationship reflects the feedback effect of load regulation on voltage stability. When the voltage drop of a critical node exceeds the set threshold, the system needs to trigger local voltage recovery and load regulation measures.

[0041] Based on the calculated risk value and communication status, nodes are prioritized for emergency response: a node is considered to have a high risk or be at risk of being out of contact when it meets any of the following conditions, requiring an emergency response to be triggered:

[0042] or or ;

[0043] in To set the operational risk threshold, This is the lower limit of the transient recovery envelope. It is the communication link quality index of node i. As a link quality threshold, when any condition is met, the central coordination unit marks node i as an emergency priority node and triggers the corresponding UAV collaborative task according to its risk level to perform communication relay, data backhaul or voltage restoration support operations.

[0044] In a further embodiment, in an emergency scenario, multiple nodes may simultaneously trigger an emergency state. If each node dispatches a drone to respond individually, it will inevitably lead to resource waste and scheduling conflicts. Therefore, this embodiment introduces a dynamic clustering mechanism based on proximity and communication reachability to optimize the collaborative scheduling efficiency of limited drone resources for widely distributed emergency nodes. The system dynamically clusters emergency priority nodes. During the emergency node clustering process, the system first determines the clustering efficiency based on the principle of geographical proximity, taking into account the effective communication coverage radius of the drone. As a spatial scale benchmark for clustering, the distance between them is... Emergency priority nodes within the range are dynamically aggregated into the same node cluster. This clustering mechanism ensures that all nodes within each cluster can be effectively covered by the same drone, thus forming a reasonable service unit. Assuming that a total of [number missing] nodes are formed after clustering... A cluster of emergency nodes constitutes a set For each cluster Calculate its geometric center position :

[0045] ;

[0046] in Let i be the coordinates of section i. For clusters The number of emergency nodes included, and the geometric center will serve as the target reference location when the drone serves the cluster;

[0047] After completing spatial clustering, the system further calculates each cluster. Overall demand priority This metric is used to quantify the overall urgency and value of different node clusters in supporting UAV communication. The larger the value, the more likely the cluster is to be affected. The greater the overall urgency and value of the need for drone communication support, the more computation is required for each cluster. Overall demand priority Its expression is:

[0048] ;

[0049] in This represents the operational risk value of the i-th node in time slot t, characterizing the severity of its voltage instability; This represents the quality of the existing communication link between node i and the central coordination unit in time slot t, used to characterize the current communication state of the node. This is a reference value for the quality of an ideal communication link. , These are the weighting coefficients for the corresponding factors, used to adjust the relative proportions of power grid operation risk and the severity of communication gaps in the overall priority; This item reflects the cumulative degree of node voltage anomalies and load risks within the cluster. The value indicates the degree of communication link quality deficiency. The larger the value, the weaker or completely interrupted the communication connection between the node cluster and the central coordination unit, and the more urgent the need for communication support for UAVs. The higher the value, the higher the overall risk and the weaker the communication of the cluster, requiring priority allocation of drones to serve it.

[0050] In a further embodiment, in step B, the collaborative guarantee loss is the sum of the comprehensive demand priorities of all emergency node clusters not served by UAVs. This priority integrates the risk of power grid operation within the cluster and the severity of communication loss to quantify the system risk caused by the lack of support. The UAV resource consumption cost is the total energy consumption of all UAVs performing the task. In the joint optimization model, the quality of the communication link between the UAV and the node directly determines the communication resource consumption cost. Based on Shannon's theorem, the maximum achievable communication capacity between UAV u and node i is related to the quality of its communication link. and allocated bandwidth resources Related; further, It is a quality indicator of the communication link between UAV u and node i, and It is the spatial distance between drone u and node i. The function is given by the path loss model:

[0051] ,in ;

[0052] in Let U be the coordinates of UAV u in time slot t. Let i be the coordinates of node i in time slot t. The reference signal power is used; this model reflects the basic law that communication quality attenuates with increasing distance, providing a physical basis for trajectory optimization.

[0053] The joint optimization model solves for the following decision variables collaboratively: the dynamic service matching relationship between UAVs and emergency node clusters. Two-dimensional flight trajectory planning for drones UAV communication resource allocation Voltage recovery regulation control quantity of power grid nodes and load regulation control quantities of power grid nodes The combined solution is expressed as follows:

[0054] ;

[0055] In a further embodiment, the overall demand of the power grid It encompasses power grid operation risk items and communication link quality requirements, and consists of optimization variables from the power grid side. and Regulation; drone energy consumption item Trajectory variable and resource allocation variables A direct function, specifically, the service cluster of UAV u within time slot t. The total energy consumption of all nodes is expressed as The flight energy consumption of the UAV u Determined by a function of two-dimensional trajectory variables, where k is a comprehensive energy consumption coefficient encompassing factors such as UAV weight, aerodynamic characteristics, and propulsion efficiency, this model clearly shows that flight energy consumption is directly proportional to the Euclidean distance the UAV travels within time slot t, and mission energy consumption... This represents the energy consumption of the drone in allocating communication resources to all nodes within the service cluster.

[0056] In a further embodiment, in step B, the UAV, as an aerial mobile platform, is limited by its onboard battery power and limited communication resources. The first constraint of the optimization problem states that each UAV can serve at most one cluster. When binary variables At that time, it indicates that the drone u and the cluster match; The second constraint represents the total energy consumption of the UAV u in the current time slot during mission execution. The first constraint indicates that the remaining energy consumption of UAV u in time slot t is sufficient to support the flight and mission execution costs, and the final remaining energy consumption is not lower than the UAV's minimum energy safety threshold. In other words, the UAV must have sufficient energy to complete the mission and return safely. The second constraint indicates the total communication resource constraint for UAV u, ensuring that a single UAV is allocated to its service cluster. The total communication resources of all nodes within the cluster do not exceed its maximum available capacity; the fourth constraint represents the minimum resource guarantee constraint for a single node, requiring the UAV to be a cluster. The communication resources allocated to any node must not be lower than the minimum threshold required for the basic communication functions of that node; , The first constraint is a weighting coefficient used to balance the importance of grid stability, communication reliability, and UAV energy consumption; the second constraint represents the adjusted voltage value of node i. and the system steady-state reference voltage value The absolute voltage difference must be within the maximum allowable voltage deviation range. Internally, the grid voltage must remain stable after emergency adjustments to prevent a wider voltage collapse.

[0057] In a further embodiment, step C employs a multi-agent reinforcement learning algorithm to solve the joint optimization model from step B. Each UAV and power grid node is modeled as an independent agent, and a collaborative control strategy is jointly learned within a framework of centralized training and distributed execution. During the training phase, the central coordination unit gathers comprehensive communication demand priorities, UAV operating status, power grid node voltage status, and communication link quality information from each emergency node cluster to construct a global state. It also calculates a global reward signal based on the total system loss and synchronously updates the policy network and value network parameters of all agents using a policy gradient method. During the execution phase, each UAV agent, based on its own state and the received global priority information, outputs a joint action through the trained policy network, including target node cluster selection, two-dimensional flight displacement, and communication resource allocation. Each power grid node agent outputs voltage regulation and load control actions based on local voltage deviation, load status, and available communication link conditions. Through this centralized training and distributed execution mechanism, precise communication coverage of high-risk node clusters by UAVs and rapid coordination of autonomous adjustment by power grid nodes are achieved, ultimately forming a closed-loop emergency control system that restores communication and supports voltage stability.

[0058] In a further embodiment, U drones are modeled as independent drone agents, responsible for learning their own trajectory planning and task execution strategies. The state vector of drone agent u includes a representation of the drone's remaining energy in time slot t. This is used to reflect the sustainable execution capability of the drone; the set of Euclidean distances between the drone u and the geometric centers of each emergency node cluster. This is used to measure the flight cost of reaching different clusters; the average communication link quality between the UAV u and nodes within each node cluster. This is used to evaluate the communication performance of different clusters and to prioritize the overall communication needs of each node cluster received from the central coordination unit. The state vector is represented as follows:

[0059] ;

[0060] The action vector of the drone agent includes the drone determining the service cluster within the current time slot. Matching variables and the flight displacement in the two-dimensional plane. This is used to implement drone trajectory planning; the drone serves the cluster within the current time slot. Communication resources provided by node i within Defined as:

[0061] ;

[0062] Based on this, the policy function of the unmanned aerial vehicle agent is expressed as:

[0063] ;

[0064] In other words, the unmanned aerial vehicle (UAV) intelligent agent autonomously generates service cluster selection, flight trajectory and resource allocation actions based on its own energy state, spatial and communication relationships with each node cluster, and priority information of each node cluster, thereby achieving precise coverage and communication guarantee for the highest priority node cluster.

[0065] In a further embodiment, N power grid nodes are modeled as power grid node agents, and their state vectors include: actual node voltage. With steady-state reference voltage Voltage recovery deviation between This is used to determine whether the current voltage of a node deviates from its stable operating range; and to assess the importance of the critical load corresponding to the node. Used to determine the priority of power supply protection in emergency scenarios; communication link status This represents the existing communication quality between the node and the central coordination unit; the communication link quality index between UAV u and node i. , means as follows:

[0066] ;

[0067] The action vector of a power grid node agent is defined as:

[0068] ;

[0069] Based on this, the policy function of the power grid agent is expressed as:

[0070] ;

[0071] In other words, the intelligent agent at the power grid node autonomously generates voltage regulation commands and load regulation commands based on voltage recovery deviation, the importance of critical loads, and the communication quality with drones, thereby achieving stable local voltage recovery at the power grid node and ensuring continuous power supply to critical users in disaster scenarios.

[0072] In a further embodiment, during the multi-agent reinforcement learning solution process in step C, the goal is to achieve collaborative action between the UAV agent and the power grid node agent in a disaster scenario. A multi-agent reinforcement learning framework with centralized training and distributed execution is used for joint updates, transforming the online optimization problem into an offline learning and online rapid decision-making problem, thereby achieving adaptive collaborative control between the UAV and the power grid. Specifically, in each time slot t, the central coordination unit aggregates the local observations of all agents, constructs a centralized global state system for training, and collects the states of the UAV node agents. and the state of the smart agent at the power grid node Constructing global state:

[0073] ;

[0074] Each agent generates a joint action based on its policy function:

[0075] ;

[0076] After the joint action is executed, the system calculates a global reward signal to evaluate the current cooperative control effect based on changes in the power grid operating status, UAV energy consumption, and communication link connectivity feedback. This global reward serves as the basis for policy optimization during the training phase, causing the UAV agent to tend to choose actions that are more energy-efficient, have a more stable link, and are more conducive to voltage recovery; the grid agent tends to choose adjustment strategies that reduce voltage deviation and reduce load disturbance. The reward function is the core that guides all agents to achieve collaborative optimization. Its design directly corresponds to the comprehensive risk function in step B, transforming risk minimization into cumulative reward maximization. Its specific definition is:

[0077] ;

[0078] After the action is executed and the global reward is calculated, in order to optimize and adjust the control strategies of each agent, this invention adopts an Actor-Critic learning architecture and introduces a joint value function. To evaluate long-term performance, it measures the expected cumulative discounted return that the current global state and joint actions can obtain in long-term cooperative control, and its estimation form is:

[0079] ;

[0080] in For value network parameters, The discount factor is used to weigh the importance of immediate rewards against future rewards; to improve training stability, a target value network with the same structure but slower parameter updates is introduced. As an evaluation network with delayed updates, this network is used to compute the temporal difference objective, serving as a supervision signal for the value network update:

[0081] ;

[0082] in Target network parameters, and Homogeneous but with a low update frequency, used for stable value estimation. It is the joint action generated by each agent in the next time slot based on its current policy;

[0083] Value network parameters Updates are performed by minimizing the mean square time difference error using gradient descent.

[0084] ;

[0085] in, The learning rate of the value network is used to control the magnitude of parameter updates, and the target network parameters are... Then periodically from It came from a soft update. ,in This refers to the soft update rate.

[0086] Based on the value network update, the policy network parameters of each agent are updated according to the policy gradient to maximize the expected reward. The update form is as follows:

[0087] ;

[0088] in For the policy network parameters of the k-th agent, Its policy function, For the local state of the agent, Perform the action for it. The advantage function is used to measure the actions of agent k. In global state The advantages and disadvantages relative to the average level. It is the state value function, usually derived from... Approximate the action by taking its expectation;

[0089] Once the policy network has been sufficiently trained and converged, the system enters the distributed execution phase. In this phase, the central coordination unit and the target network are no longer needed, and each agent relies only on its own local observation state. Through its localized, pre-trained policy network Independently generate control actions:

[0090] ; ;

[0091] After the training phase, the multi-agent reinforcement learning model is deployed to the central coordination unit and the control terminals of each UAV and power grid node, enabling online real-time decision-making and collaborative control. When the system enters the execution phase, the central coordination unit gathers multi-source information in real time, including UAV location, remaining energy, power grid node voltage, load status, and communication link quality. Each UAV agent automatically adjusts its flight trajectory, power allocation, and communication resource scheduling based on the control commands output by its own strategy network, thereby achieving dynamic optimization of air coverage and link quality. At the same time, the power grid node agents perform voltage compensation, load reduction, or transfer operations based on local voltage deviation and load status, achieving rapid voltage recovery and stable system operation. All of the above actions are autonomously decided and executed by the agents in real time. The central coordination unit continuously updates the strategy parameters based on the overall system operation effect and distributes them synchronously to each terminal, enabling the system to continuously learn and adaptively optimize in a dynamic environment. Through this mechanism, the UAV swarm and the power grid system can achieve globally optimal collaborative emergency response in disaster scenarios, significantly improving emergency communication support capabilities and voltage recovery efficiency, ultimately achieving the goal of integrated air-ground intelligent emergency control.

[0092] To verify the effectiveness and feasibility of the proposed power grid air-ground collaborative emergency control method based on UAV multi-agent reinforcement learning, a simulation environment was established by combining typical power grid operating parameters and UAV collaborative communication characteristics. The collaborative coverage, resource allocation, and real-time decision-making performance of multiple UAVs in the dynamic scheduling task of emergency nodes were systematically verified. In the simulation settings, a total of 10 power grid nodes and 3 UAVs were set up in the power grid area. The power grid nodes were randomly distributed, and each node collected voltage, current, and load status information in real time. Emergency priority nodes were dynamically determined according to the voltage deviation threshold. The coverage radius of each UAV was 4.0, the maximum movement step size was 0.8, and the unit movement energy consumption was 0.1. The system is configured with a communication capacity of 10 to allocate communication resources within time slots. Emergency priority for power grid nodes is determined based on a risk threshold of 0.7, a voltage threshold of 0.90, and a communication threshold of 0.30. The UAV adopts an independent Q-learning strategy with a learning rate of 0.1, a discount factor of 0.9, and an initial exploration rate ε of 1.0 that decays to 0.001 in each round at a rate of 0.995. Each round of training contains 1000 steps, and the total number of training rounds is 2000. These parameters are set to ensure convergence speed and system stability while effectively balancing UAV maneuvering energy consumption, coverage efficiency, and voltage recovery performance.

[0093] Figure 4 Simulation results of multi-UAV collaborative emergency power grid node coverage are presented. The system consists of 10 power grid nodes and 3 UAVs. Red dots represent emergency priority nodes, blue dots represent ordinary nodes, pentagrams represent cluster centers, triangles represent UAV locations, and colored semi-transparent circular areas represent the service coverage of each UAV. Risk assessment and clustering results show that the system forms four cluster centers with priorities of 1.87, 1.50, 0.51, and 0.43, respectively. UAV 0 is located in the right-hand area of ​​the diagram, serving emergency nodes 6, 8, and 9, with a cluster priority of 1.87, which is currently the highest priority area, indicating the presence of [a specific type of cluster center]. Significant voltage fluctuations or communication risks exist. Drone 1, located at the bottom of the diagram, covers node 5 with a cluster priority of 0.51. It is in a relatively stable area and mainly undertakes status monitoring and communication maintenance tasks. Drone 2, located at the top left of the diagram, covers nodes 2, 4, and 7 with a cluster priority of 1.60. This area has a higher risk level and requires continuous scheduling to ensure power supply and communication stability. Overall, the three drones achieve balanced spatial distribution and efficient cluster collaboration, with minimal overlap in their coverage areas. They can dynamically respond and schedule resources according to the risk levels of different clusters, demonstrating the intelligence and effectiveness of the proposed multi-drone collaborative strategy in emergency power grid scenarios.

[0094] Figure 5This study demonstrates the trend of total reward changes during the reinforcement learning training process of a multi-UAV collaborative emergency power grid dispatching system. The horizontal axis represents the number of training rounds, and the vertical axis represents the total reward of the system, used to measure the task collaboration efficiency and power grid stability recovery capability of the UAV swarm at each stage. The entire training process can be divided into three stages: In the exploration stage, the UAV agents have not yet formed a stable decision-making strategy and mainly rely on random exploration to execute tasks, resulting in unstable voltage recovery of emergency nodes, low coverage efficiency, and drastic fluctuations in the total system reward; After entering the learning stage, the agents gradually master the rules of risk assessment, node clustering, and energy allocation, and can implement targeted emergency support and communication dispatch for nodes with abnormal voltage. The system reward rises rapidly and exhibits moderate-amplitude oscillations; When the number of training rounds exceeds 800, the system enters the convergence stage, and the UAVs form a better division of labor and cooperation relationship, which can dynamically cover key power grid nodes, effectively reduce overall voltage deviation and risk indicators, and the total reward tends to stabilize and remain at a high level. These results show that the proposed reinforcement learning-driven multi-UAV collaborative mechanism can achieve an adaptive learning process from random exploration to efficient dispatching in power grid emergency scenarios, and has good convergence performance and power grid stability recovery capability.

[0095] Against the backdrop of the deep integration of new power systems and low-altitude intelligent networks, air-ground coordinated emergency control has become a key direction for improving the resilience of power grids and communication support capabilities. Traditional ground communication and control systems often face problems such as infrastructure damage, network interruption and dispatch delays in complex environments such as natural disasters, extreme weather or remote mountainous areas, making it difficult to meet the requirements of low latency, high reliability communication and voltage stability in emergency response.

[0096] To address this, this invention proposes a power grid air-ground collaborative emergency control method and system based on multi-agent reinforcement learning of unmanned aerial vehicles (UAVs). By introducing a multi-UAV platform with edge computing and communication capabilities, it achieves integrated air-ground collaborative control of power grid operation status monitoring, communication link restoration, and voltage regulation. The system uses a UAV swarm as aerial mobile control nodes, forming a distributed autonomous network with the ground-based power grid control unit. This enables rapid system reconfiguration and continuous stable operation even under partial failure of ground infrastructure. In this system, the central coordination unit comprehensively considers the remaining energy of the UAVs, power grid node risks, voltage deviations, and communication link quality, dynamically allocating aerial communication and computing resources, and coordinating multiple UAVs to perform emergency tasks, thus addressing UAV scheduling and other challenges. This invention addresses the high-dimensional coupling between power grid voltage restoration and communication resource allocation. Employing a multi-agent reinforcement learning approach, it achieves joint optimization within a centralized training and distributed execution architecture. The system models both UAVs and power grid nodes as independent agents. During training, a central coordination unit aggregates the state information of each agent, calculates global rewards, and updates the joint policy network. During execution, each agent autonomously selects its flight path, communication power, and voltage regulation behavior based on local observations, enabling real-time emergency control and adaptive system optimization. Furthermore, the system can automatically identify voltage anomaly nodes and communication interruption areas after a disaster and dispatch UAVs to quickly fly over critical nodes to establish airborne relay links, coordinating with power grid nodes to complete voltage restoration and load adjustment. Through continuous policy iteration and parameter updates, the system can achieve globally optimal emergency response strategies in complex dynamic environments, significantly reducing voltage restoration time and communication latency, improving power grid emergency communication coverage and voltage stability, and realizing intelligent and autonomous collaborative control of UAV swarms and the power grid system. This approach has broad engineering application value and promising prospects for widespread adoption.

[0097] The preferred embodiments of the present invention have been described in detail above with reference to the accompanying drawings. However, the present invention is not limited to the specific details in the above embodiments. Within the scope of the technical concept of the present invention, various equivalent transformations can be made to the technical solutions of the present invention, and all such equivalent transformations fall within the protection scope of the present invention.

Claims

1. A power grid air-ground collaborative emergency control system based on unmanned aerial vehicle multi-agent reinforcement learning, characterized in that: The system is a disaster scene-oriented unmanned aerial vehicle and power grid integrated intelligent emergency control system, the system comprises a plurality of power grid control intelligent agents deployed at key nodes of a power grid , a plurality of unmanned aerial vehicle intelligent agents with communication capabilities , and a central coordination unit The power grid control agent is configured at each bus or power distribution node, is used for collecting node voltage and load state information in real time, and executes local voltage regulation and load control; the unmanned aerial vehicle agent is used for providing air communication relay, resource allocation and path planning services in a disaster environment, and has an edge computing module and a wireless communication module integrated therein; the central coordination unit is in communication connection with all power grid control agents and unmanned aerial vehicle agents, is used for converging global state information, generating comprehensive demand priority of each emergency node cluster, broadcasting key state information to each agent, calculating a global reward signal based on system comprehensive loss in a training stage, guiding multi-agent policy updating, and realizing collaborative control of the unmanned aerial vehicle and the power grid through the multi-agent reinforcement learning mechanism, so that the unmanned aerial vehicle cluster self-adaptively plans a flight trajectory and communication scheduling according to a power grid risk state, and the power grid agent dynamically adjusts load allocation and voltage recovery strategies according to unmanned aerial vehicle feedback results, thereby realizing collaborative optimization of power emergency communication and voltage control.

2. An emergency control method using the unmanned aerial vehicle multi-agent reinforcement learning-based power grid air-ground collaborative emergency control system according to claim 1, characterized in that, The method comprises the following steps: Step A: the central coordination unit acquires voltage amplitude and key load operation state of each bus node of the power grid in a current time slot, and position information, residual energy and communication link quality of the multiple unmanned aerial vehicles; according to the acquired voltage recovery deviation and load importance, the operation risk of the power grid node is evaluated, when the bus voltage is lower than the transient recovery envelope or the communication link is interrupted, it is indicated that the node has voltage instability or communication loss risk, and the data interaction demand of the node with the central control unit significantly increases, at this time, the node is judged as an emergency priority node, and the unmanned aerial vehicle needs to fly to the airspace of the node to provide communication relay and additional special communication capacity, so as to guarantee high-speed backhaul of key state data and reliable issuance of complex regulation and control instructions, thereby supporting real-time and accurate transmission of emergency regulation and control information, on this basis, the emergency priority nodes are dynamically aggregated into multiple emergency node clusters based on the principle of geographical proximity and the communication coverage radius of the unmanned aerial vehicle, and the comprehensive demand priority of each cluster is calculated, including the weighted sum of the node operation risk value in the cluster and the overall communication loss severity of the cluster; Step B: a joint optimization model of unmanned aerial vehicle and power grid collaborative emergency control is constructed, taking the minimum system total loss as an objective function, the system total loss is composed of three parts of comprehensive demand guarantee loss, unmanned aerial vehicle resource consumption cost and communication resource consumption cost, and the decision variables of the unmanned aerial vehicle side and the power grid side are optimized, wherein the optimization variables of the unmanned aerial vehicle side include the dynamic service matching relationship between the unmanned aerial vehicle and the emergency node cluster, two-dimensional trajectory planning and communication and calculation resource allocation strategy, and the optimization variables of the power grid side include voltage recovery control and load control of each node; the optimization model simultaneously considers the unmanned aerial vehicle energy constraint, communication link quality constraint and power grid transient stability constraint under the unified objective function, so as to realize integrated collaboration of unmanned aerial vehicle emergency communication and power grid recovery control on the premise of ensuring power supply continuity and network reliability. Step C: The joint optimization problem is solved using a multi-agent reinforcement learning algorithm. The central coordination unit broadcasts key global state information such as the integrated communication demand priority of each emergency node cluster to all agents. The global reward signal is calculated based on the system's overall loss, which is used to update the policy during the training phase. Each UAV agent outputs its target node cluster selection, two-dimensional flight displacement, and communication resource allocation based on its own local state and the received global priority information through the policy network. Each grid node agent outputs voltage regulation and load control actions based on its local state and available UAV communication link state through the policy network. In the execution phase after the policy converges, each agent makes distributed autonomous decisions and controls, achieving precise communication coverage of high-risk clusters by UAVs and rapid coordination of autonomous grid regulation.

3. The emergency control method according to claim 2, characterized in that: The power grid operation risk assessment in step A includes the following processes: (1) The node operation risk value is calculated based on the voltage recovery deviation of the grid bus node in the current time slot and the load importance, and the calculation formula is: ； wherein, represents the operation risk value of the i-th node at time slot t, represents the voltage amplitude of the node, is the steady-state reference voltage, represents the real-time load importance degree corresponding to the node, is the maximum allowable load that the node can bear, and α and β are risk weight coefficients for measuring the proportion of the influence of voltage deviation and load occupation on the operation risk of the node, respectively; In addition, the dynamic update relationship between node voltage and load is: ； wherein represents the voltage adjustment amount of the node in the current time slot, which is used to achieve voltage recovery by adjusting the reactive power compensation device or transformer tap; is the load adjustment amount of the node, which is used to perform reduction or transfer of non-critical load to ensure stable power supply of critical load, is a mapping function of the influence of load adjustment on the voltage of the node; (2) According to the calculated risk value and communication state, the node emergency priority is determined. When the node meets any of the following conditions, it is considered to have high risk or risk of disconnection, and the emergency response needs to be triggered: or or ; wherein is a running risk threshold, is a transient recovery envelope lower bound, is a communication link quality indicator for node i, is a link quality threshold, when any condition is met, the central coordination unit will label node i as a priority for emergency node.

4. The emergency control method according to claim 2, characterized in that: The emergency node clustering in step A includes the following processes: (1) Based on the principle of geographical proximity and the communication coverage radius of the UAV , the nodes with spatial distance within the effective communication range of the UAV are aggregated into the same emergency node cluster, assuming that there are clusters; for each cluster , the geometric center position is calculated ； wherein is the coordinate of the section i, is the number of emergency nodes contained in the cluster emergency nodes contained in the cluster (2) Then, the overall demand priority of each cluster is calculated, which is expressed as: ​ ； wherein is the existing communication link quality between node i and the central coordination unit at time slot t, used to represent the current communication status of the node, is the ideal communication link quality reference value, , is the weight coefficient of the corresponding factor, is the communication link quality missing degree, the greater the value, the weaker the overall communication connection of the node cluster with the central coordination unit or the complete interruption, and the more urgent the demand for unmanned aerial vehicle communication support.

5. The emergency control method of claim 2, wherein: In step B, the collaborative guarantee loss is the sum of the comprehensive demand priorities of all emergency node clusters not served by the UAV, which combines the power grid operation risk and the communication loss severity to quantify the system risk caused by support loss; the UAV resource consumption cost is the total energy consumption of all UAVs performing tasks; in the joint optimization model, the communication link quality between the UAV and the node directly determines the communication resource consumption cost; based on Shannon's theorem, the maximum reachable communication capacity between the UAV u and the node i is related to the communication link quality and the allocated bandwidth resource ; further, is the communication link quality indicator between the UAV u and the node i, and is the spatial distance between the UAV u and the node i , which is given by the path loss model: wherein ; wherein is the coordinate of the drone u at time slot t, is the coordinate of the node i at time slot t, is the reference signal power; The joint optimization model cooperatively solves the following decision variables: dynamic service matching relationship between the unmanned aerial vehicle and the emergency node cluster two-dimensional flight trajectory planning of the unmanned aerial vehicle communication resource allocation of the unmanned aerial vehicle voltage recovery adjustment control amount of the power grid node and load adjustment control amount of the power grid node are jointly solved, and the expression is: ； The first constraint states that each drone can serve at most one cluster. When binary variables At that time, it indicates that the drone u and the cluster match; The second constraint represents the total energy consumption of the UAV u in the current time slot during mission execution. The first constraint indicates that the remaining energy consumption of UAV u in time slot t is sufficient to support the flight and mission execution costs, and the final remaining energy consumption is not lower than the UAV's minimum energy safety threshold. In other words, the UAV must have sufficient energy to complete the mission and ensure a safe return. The second constraint represents the total communication resource constraint for UAV u, ensuring that a single UAV is allocated to its service cluster. The total communication resources of all nodes within the cluster do not exceed its maximum available capacity; the fourth constraint represents the minimum resource guarantee constraint for a single node, requiring the UAV to be a cluster. The communication resources allocated to any node must not be lower than the minimum threshold required for the basic communication functions of that node; , The first constraint is a weighting coefficient used to balance the importance of grid stability, communication reliability, and UAV energy consumption; the second constraint represents the adjusted voltage value of node i. and the system steady-state reference voltage value The absolute voltage difference must be within the maximum allowable voltage deviation range. Internally, the grid voltage must remain stable after emergency adjustments to prevent a wider voltage collapse.

6. The emergency control method according to claim 5, characterized in that: In step C, the multi-agent reinforcement learning algorithm is used to solve the joint optimization model in step B. Each UAV and grid node is modeled as an independent agent to jointly learn the cooperative control strategy in the centralized training and distributed execution framework. In the training phase, the central coordination unit collects the integrated communication demand priority of each emergency node cluster, the UAV operation state, the grid node voltage state, and the communication link quality information, constructs the global state, and calculates the global reward signal based on the system's total loss. The policy gradient method is used to update the parameters of the policy network and value network of all agents simultaneously. In the execution phase, each UAV agent outputs the joint action including target node cluster selection, two-dimensional flight displacement, and communication resource allocation based on its own state and the received global priority information through the trained policy network. Each grid node agent outputs voltage regulation and load control actions based on the local voltage deviation, load state, and available communication link conditions. Through the centralized training and distributed execution mechanism, precise communication coverage of high-risk node clusters by UAVs and rapid coordination of autonomous grid node regulation are achieved, ultimately forming a closed-loop emergency control for communication recovery and voltage stability support.

7. The emergency control method according to claim 6, characterized in that: In the multi-agent reinforcement learning solving process, each UAV is modeled as an independent UAV agent, and its state, action, and policy function are defined as follows: The state vector of the UAV agent includes: the residual energy of the UAV at time slot t , for reflecting the sustainable execution capability of the UAV; a set of Euclidean distances between the UAV u and the geometric centers of each emergency node cluster , for measuring the flight cost of reaching different clusters; the average communication link quality between the UAV u and the nodes in each node cluster , for evaluating the communication efficiency of serving different clusters; and the integrated communication demand priority of each node cluster received from the central coordination unit , the state vector is represented as follows: ； The action vector of the drone agent includes the drone determining the service cluster within the current time slot. Matching variables and the flight displacement in the two-dimensional plane. This is used to implement drone trajectory planning; the drone serves the cluster within the current time slot. Communication resources provided by node i within Defined as: ； Based on this, the policy function of the UAV agent is represented as: ； That is, the UAV agent autonomously generates service cluster selection, flight trajectory, and resource allocation actions based on its own energy state, spatial and communication relationship with each node cluster, and priority information of each node cluster, thereby achieving precise coverage and communication guarantee of the highest priority node cluster.

8. The emergency control method of claim 6, wherein: In the multi-agent reinforcement learning solving process, each grid node is modeled as a grid node agent, and the grid node agent state, action and policy function are defined as follows: The state vector of the grid node agent includes: the actual voltage of the node The voltage recovery deviation between the steady-state reference voltage The voltage recovery deviation between the steady-state reference voltage , for determining whether the current voltage of the node deviates from the stable operation range; the importance of the key load corresponding to the node , for determining the power supply guarantee priority in the emergency scenario; the communication link state , indicating the existing communication quality between the node and the central coordination unit; the communication link quality index between the unmanned aerial vehicle u and the node i , indicating the following: ； The action vector of the grid node agent is defined as: ； Based on this, the policy function of the grid node agent is represented as: ； That is, the grid node agent autonomously generates voltage regulation instructions and load regulation instructions according to the voltage recovery deviation, the importance of critical loads and the communication quality with the unmanned aerial vehicle, thereby achieving local voltage stability recovery of the grid node and guaranteeing continuous power supply for critical users in disaster scenarios.

9. The emergency control method of claim 6, wherein: In the multi-agent reinforcement learning solving process of the step C, the cooperative action of the unmanned aerial vehicle agent and the power grid node agent in the disaster scene is taken as the goal, the multi-agent reinforcement learning framework of centralized training and distributed execution is used for joint updating, and the specific process is as follows: at each time slot t, the system first constructs the global state according to the unmanned aerial vehicle node agent state and the power grid node agent state ​ ； Each agent generates a joint action according to its policy function: ； After the joint action is executed, the system calculates a global reward signal for evaluating the current cooperative control effect according to the power grid operation state change, the unmanned aerial vehicle energy consumption and the communication link connectivity feedback The global reward is used as a basis for strategy optimization in the training stage, so that the unmanned aerial vehicle agent tends to select actions that are more energy-saving, have more stable links and are more conducive to voltage recovery; the power grid agent tends to select adjustment strategies that reduce voltage deviation and reduce load disturbance, wherein the global reward is defined as: ； After the action is executed and the global reward is calculated, in order to realize the optimization and adjustment of the control strategy of each agent, the system further constructs a joint value function for measuring the overall benefit of the current global state and the joint action in long-term collaborative control, and its estimated form is: ； wherein is a value network parameter, is a discount factor, and to improve the stability of training, a target value network is introduced as a delayed updated evaluation network, whose temporal difference training target is ； wherein target network parameters, with isomorphic but with lower update frequency, for stable value estimates, is the joint action generated by the agents in the next time slot according to their current policy; The value network parameter update follows the gradient descent rule: ； wherein, is a value network learning rate for controlling the parameter update magnitude; On the basis of the value network update, the policy network parameters of each agent are updated according to the policy gradient, and the update form is: ； wherein are policy network parameters of the kth agent, is its policy function, is the local state of the agent, is its executed action, is a advantage function that measures the relative gain of the current action over the average action; In addition, for the discrete decision variable in the action space of the unmanned aerial vehicle agent, i.e. the target node cluster selection, the Gumbel-Softmax continuous relaxation method is used for differentiable optimization to realize end-to-end gradient backpropagation: ； wherein, selecting an original probability distribution of node clusters for the drone u, is a temperature parameter that controls the smoothness of the sampling process, is a random perturbation vector sampled from a Gumbel distribution; during the training phase, the probability vector after successive relaxations is used participates in the forward propagation and gradient computation; during the execution phase, the specific node cluster index is selected from the probability distribution through the argmax operation: ； The method makes the discrete service cluster selection decision differentiable in the training process, while maintaining the certainty in the execution phase, thereby ensuring the effective convergence and stable execution of the multi-agent reinforcement learning algorithm; After the policy converges, the agents in the execution phase no longer rely on the global state, but independently select actions based on the local state: ； ； Through the above alternating update of the value network and the policy network, the collaborative control strategy of the unmanned aerial vehicle agent and the grid node agent is gradually obtained, so that the trajectory planning and task scheduling of the unmanned aerial vehicle and the voltage recovery and load regulation of the grid node can achieve global optimal collaborative control in disaster scenarios, realizing real-time space-ground collaborative control of the unmanned aerial vehicle cluster for emergency communication guarantee and voltage stability support of the grid node.

Images