Heterogeneous unmanned aerial vehicle conflict resolution method based on enhanced graph reinforcement learning

Abstract

The invention relates to a heterogeneous unmanned aerial vehicle conflict resolution method based on enhanced graph reinforcement learning, belongs to the technical field of unmanned aerial vehicles, and solves the technical problems that information extraction of a heterogeneous unmanned aerial vehicle conflict resolution model in the prior art is confused, and cooperative regulation and control cannot be well realized for different unmanned aerial vehicles. The invention provides a heterogeneous unmanned aerial vehicle conflict resolution method based on enhanced graph reinforcement learning, and the method comprises the steps: firstly calculating a heuristic action through a potential field function according to the flight state of an unmanned aerial vehicle and an obstacle in an airspace; secondly, extracting heterogeneous unmanned aerial vehicle information by using a heterogeneous graph neural network, and learning correction action and hovering maneuver; and finally, model parameters are trained and updated through a reinforcement learning algorithm, so that efficient conflict resolution of heterogeneous unmanned aerial vehicles is realized.

Description

Technical Field

[0001] This invention belongs to the field of unmanned aerial vehicle (UAV) technology and relates to a heterogeneous UAV conflict resolution method based on augmented graph reinforcement learning. Background Technology

[0002] Unmanned aerial vehicles (UAVs) have been widely used in various fields due to their high maneuverability and low cost. Because flight missions in low-altitude airspace are diverse and have varying capability requirements, they need to be performed by heterogeneous UAVs. For example, delivery missions are typically performed by robust UAVs with lower maneuverability, emergency missions require highly agile UAVs with a larger safety radius, and communication missions require UAVs capable of frequent hovering. Therefore, with the increasing density of heterogeneous UAVs, more potential flight conflicts may occur in the airspace, leading to a surge in safety risks.

[0003] Conflict resolution is a key technology for ensuring the safe operation of unmanned aerial vehicles (UAVs), enabling real-time maneuvering decisions to avoid potential collisions in low-altitude airspace. Compared to traditional research methods (mathematical optimization, reactive methods), reinforcement learning, through offline training and online execution frameworks, can provide immediate responses and efficient solutions, and is considered an advanced approach. In recent years, some research has combined graph neural network methods to represent UAV information as graph-structured features, demonstrating significant advantages in enhancing the modeling of UAV interrelationships.

[0004] However, the heterogeneity of drones poses significant challenges to airspace safety management. First, heterogeneous drones possess different sizes and capabilities, while existing conflict resolution methods primarily rely on homogeneous drones sharing decision-making models for control. The heterogeneity of drones limits the capabilities of current methods. Furthermore, due to the unique hovering capabilities of rotorcraft drones, their motion patterns also exhibit heterogeneity, resulting in more varied trajectories for drones that can hover or move at any time, leading to more complex conflicts.

[0005] Therefore, this invention addresses the limitations of existing graph reinforcement learning methods in heterogeneous UAV conflict resolution by proposing a heterogeneous UAV conflict resolution method based on reinforcement graph reinforcement learning. This method can effectively reduce potential conflicts and avoid excessive trajectory deviations through collision avoidance maneuvers and hovering maneuvers. Summary of the Invention

[0006] In view of the above problems, and considering the difficulty of traditional graph reinforcement learning methods in resolving the conflict resolution issue of heterogeneous unmanned aerial vehicles (UAVs), this invention provides a conflict resolution method for heterogeneous UAVs based on reinforcement graph reinforcement learning. In this method, firstly, based on the UAV's flight state and obstacles in the airspace, heuristic actions are calculated using a potential field function; secondly, heterogeneous graph neural networks are used to extract information about the heterogeneous UAVs, learn corrective actions and hovering maneuvers; finally, reinforcement learning algorithms are used to train and update model parameters, thereby achieving efficient conflict resolution for heterogeneous UAVs.

[0007] This invention provides a heterogeneous UAV conflict resolution method based on augmented graph reinforcement learning, comprising the following specific steps:

[0008] Step 1: Initialize the training environment, the state information of each heterogeneous UAV and obstacles, and set the state space, action space, reward function, maximum number of scene iterations and maximum running time steps per round of the UAV;

[0009] Step 2: Establish the potential field function. Based on the UAV's flight state and obstacle state, obtain the gravitational force for reaching the target, the gravitational force for maintaining direction, the UAV's collision repulsion force, and the obstacle's collision repulsion force. Based on the resultant force on the UAV, obtain the initial optimal deflection angle as a heuristic action.

[0010] Step 3: Update the state space based on the heuristic actions, input it into the heterogeneous graph neural network, and extract the implicit information of the heterogeneous UAV through the ordered gating coding mechanism; based on the implicit information of the heterogeneous UAV, calculate the expected value of each action through the action decision network, and select the action with the largest expected value as the action to be executed.

[0011] Step 4: Determine the type of action to be performed. If it is a deflection angle, input it and the heuristic action into the angle smoothing function to dynamically correct the heuristic action. If it is a hovering maneuver, reduce the speed of the heterogeneous UAV to 0 and adjust the heading of the heterogeneous UAV to point to the target point.

[0012] Step 5: Calculate the reward gained by the heterogeneous UAV in performing the action to be performed at time t, and store it as an experience sample in the experience pool; batch sample the experience samples to train the conflict resolution model, iteratively optimize the parameters of the conflict resolution model, until the conflict resolution model training is completed.

[0013] Step 6: In each round of the scenario, execute the maneuvering and training process from Step 2 to Step 5 according to the time step, and update the position and speed direction of each drone until all heterogeneous drones reach the target point or reach the maximum number of running time steps in a single round, then proceed to Step 7.

[0014] Step 7: Determine if the maximum number of scene iterations has been reached. If it has, obtain the trained conflict resolution model; otherwise, return to step 1.

[0015] Step 8: At each time step, use the actual information in the application scenario as the input to the conflict resolution model, and finally output the collision avoidance maneuvers of each heterogeneous UAV.

[0016] Optionally, the specific steps of step 2 are as follows:

[0017] Step 201: Construct the arrival potential field function and the intended gravitational potential field to obtain the gravitational force for reaching the target and the direction-maintaining gravitational force;

[0018] Step 202: Based on the target gravity and the direction-maintaining gravity, obtain the gravitational potential field force intended to maintain the direction, and obtain the initial gravitational heading through the gravitational potential field;

[0019] Step 203: Based on the potential conflicts that may exist along the initial gravitational heading, use the velocity obstacle model to calculate the future conflict times of heterogeneous UAV-heterogeneous UAV node pairs and heterogeneous UAV-obstacle node pairs.

[0020] Step 204: Quantify the future conflict time of any heterogeneous UAV-heterogeneous UAV node pair into UAV conflict intensity to form a UAV conflict matrix; quantify the future conflict time of any heterogeneous UAV-obstacle node pair into obstacle conflict intensity to form an obstacle conflict matrix.

[0021] Step 205: Determine the most urgent drone conflict and obstacle conflict based on the drone and obstacle conflict matrix, and obtain the urgent drone conflict intensity and urgent obstacle conflict intensity of heterogeneous drones.

[0022] Step 206: Based on the collision intensity between the UAV and the obstacle, and the direction of the repulsive force, obtain the collision repulsive force between the heterogeneous UAV and the obstacle;

[0023] Step 207: Based on the collision repulsion force of heterogeneous UAVs and the collision repulsion force of obstacles, obtain the current potential field function resultant force and obtain the final heuristic action.

[0024] Optionally, the specific steps of step 3 are as follows:

[0025] Step 301: Update the state space: Replace the original velocity direction with the heuristic action, encode the new state space through a fully connected network, and obtain the initial state vector of each heterogeneous UAV.

[0026] Step 302: Let k=1, and obtain the implicit information of each heterogeneous UAV extracted from the first layer of the heterogeneous graph neural network based on the initial state vector of each UAV.

[0027] Step 303: Extract the implicit information of the neighboring UAV u from the (k-1)th layer of the heterogeneous graph neural network. Perform information aggregation to obtain the neighboring drone information of heterogeneous drone i after the k-th layer network aggregation;

[0028] Step 304: Based on the ordered gating coding mechanism, combine the implicit information of the heterogeneous UAV i in the (k-1)th layer network with the neighboring UAV information after aggregation in the kth layer network to obtain the intermediate fusion representation of the heterogeneous UAV i in the kth layer network.

[0029] Step 305: Obtain the gating vector based on the intermediate fusion representation of the heterogeneous UAV in the k-th layer network;

[0030] Step 306: Using the gating vector, the implicit information of the heterogeneous UAV itself extracted by the (k-1)th layer network of the heterogeneous UAV is aggregated with the neighbor UAV information after aggregation by the kth layer network to obtain the implicit information of the heterogeneous UAV itself extracted by the kth layer network.

[0031] Step 307: Based on the implicit information of the heterogeneous UAV extracted from the k-th layer network, the expected value of each action is obtained through a fully connected layer as the action decision network, and the action with the largest expected value is selected as the action to be executed.

[0032] Step 308: Determine whether k has reached the maximum number of layers in the heterogeneous graph neural network. If it has, set the action to be executed as the final action to be executed; if it has not, let k = k + 1 and return to step 303.

[0033] Optionally, the specific steps of step 5 are as follows:

[0034] Step 501: Based on the conflict situation, trajectory offset, and number of hovering after the final action to be executed according to the conflict resolution model, calculate the reward income through the reward function, and store it together with the state at time t, the selected action, and the state at time t+1 as an experience sample and store it in the experience pool.

[0035] Step 502: After reaching the preset number of training rounds, after each time step has completed the action, the conflict resolution model selects a batch of samples from the experience pool.

[0036] Step 503: Calculate the loss of the collected experience samples using an action decision network;

[0037] Step 504: Update the parameters of the conflict resolution model using the gradient descent method to gradually reduce the loss.

[0038] Compared with the prior art, the present invention has at least the following beneficial effects:

[0039] (1) Existing research mainly uses traditional graph neural networks to extract homogeneous UAV information, which is difficult to effectively extract heterogeneous node information with different attributes and capabilities. This invention proposes a heterogeneous graph neural network, which aggregates its own and neighboring node features through an ordered gating mechanism to prevent the average extraction and aggregation of node information from causing ambiguity of heterogeneous information.

[0040] (2) Compared to homogeneous UAV scenarios in traditional research, the conflict relationships of heterogeneous UAVs change rapidly, greatly increasing the difficulty of model training. This invention proposes a hierarchical decision-making framework. First, heuristic maneuvers are calculated through local optimal field functions, and then the reinforcement learning model modifies the heuristic actions from a global perspective, reducing the complexity of training a high-performance model from scratch using traditional methods.

[0041] (3) Existing research only applies traditional maneuvers such as heading deflection, neglecting the inherent hovering capability of rotary-wing UAVs. This invention incorporates the learning of heterogeneous motion modes (hovering maneuvers), enhancing the decision-making model's capabilities by expanding the basic motion selection of rotary-wing UAVs. Attached Figure Description

[0042] Figure 1 This is a schematic diagram of the heterogeneous UAV conflict resolution method based on augmented graph reinforcement learning according to the present invention.

[0043] Figure 2 This is a schematic diagram of the ordered gating mechanism of the present invention;

[0044] Figure 3 This is a schematic diagram of the deflection action correction of the present invention. Detailed Implementation

[0045] To better understand the above-described objectives, features, and advantages of the present invention, the invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be noted that, unless otherwise specified, the embodiments of the present invention and the features thereof can be combined with each other. Furthermore, the present invention can be implemented in other ways different from those described herein; therefore, the scope of protection of the present invention is not limited to the specific embodiments disclosed below.

[0046] A specific embodiment of the present invention, such as Figure 1-3 This paper discloses a heterogeneous UAV conflict resolution method based on reinforcement graph reinforcement learning, which includes the following specific steps:

[0047] Step 1: Initialize the training environment, the state information of each heterogeneous UAV and airspace obstacles, and set the UAV's state space S, action space A, reward function R, maximum number of scene iterations T and maximum running time steps N per round.

[0048] Step 101: Initialize the status information of each heterogeneous UAV;

[0049] Specifically, the state information of the heterogeneous UAV includes a randomly generated starting position in the airspace, a target point position, and a UAV speed v randomly set within a speed threshold range. i It also randomly generates directions and a safe radius.

[0050] Preferably, the speed threshold range is [15m / s, 30m / s]; the total radius is determined by the speed, and the faster the UAV, the larger the safe radius (R). i =8*v i ).

[0051] Step 102: Initialize airspace obstacles;

[0052] Preferably, the airspace obstacles are initialized as static circular areas to simulate low-altitude building clusters, and the center and radius of the obstacle circles are randomly generated.

[0053] Step 103: Set the state space S of the UAV, which includes the starting position, target point position, safety radius, speed magnitude, speed direction, and the center and radius of each obstacle.

[0054] Step 104: Set up the drone motion space A, including the deflection capability, non-deflection capability, and hovering capability of each drone.

[0055] Specifically, the deflection capability of each UAV w i Including left and right turns, the deflection capability of each drone w i Randomly generated within the deflection capability angle range.

[0056] Furthermore, the deflection capability angle range is [5°, 15°].

[0057] Step 105: Set the drone reward function R, including near-finish reward, drone collision penalty, obstacle collision penalty, trajectory deviation penalty, and hovering time penalty.

[0058] Step 106: Set the maximum number of scene iterations T and the maximum number of running time steps N per round according to the requirements.

[0059] Step 2: Establish the potential field function. Based on the UAV's flight state and the obstacle state, obtain the gravitational force for reaching the target, the gravitational force for maintaining direction, the collision repulsion force of the UAV, and the collision repulsion force of the obstacle. Based on the resultant force on the UAV, obtain the initial optimal deflection angle as the inspiration action.

[0060] Step 201: Construct the arrival potential field function and the intended gravitational potential field to obtain the gravitational force F to reach the target. target And maintain the direction of gravity F currentThe expression is:

[0061]

[0062] F current =k c ·[cos(α i ),sin(α i )]

[0063] Where, k t With k c These are predefined parameters. It is the current position of heterogeneous UAV i at time t; e i Indicates the target position of heterogeneous UAV i; α i This represents the current heading angle of the heterogeneous UAV i.

[0064] The arrival potential field function F of the present invention target This ensures that the closer the drone is to the destination, the stronger the gravitational force, thus guiding the drone to the target position efficiently; the intention is to maintain the gravitational potential field F. current It can ensure that the drone takes into account the consistency with the current heading when deflecting, and prevent the drone from making excessive trajectory deviations in order to get closer to the destination.

[0065] Step 202, based on the gravitational force F to reach the target target And maintain the direction of gravity F current To obtain the gravitational potential field force F intended to maintain att The initial gravitational heading α is obtained through the gravitational combined potential field. att The expression is:

[0066] F att =F target +F current

[0067]

[0068] Among them, F att,y F represents att The force decomposition along the y-axis; F att,x F represents att The force is decomposed along the x-axis;

[0069] Step 203: Based on the initial gravitational heading α att Potential conflicts can be identified by using a velocity obstacle model to calculate the time t when a conflict will occur in each heterogeneous UAV-to-heterogeneous UAV node pair and each heterogeneous UAV-to-obstacle node pair along the current velocity direction and magnitude. c .

[0070] Step 204: Use a negative exponential function to quantify the future collision time of any heterogeneous UAV i-heterogeneous UAV j node pair (i,j) into a UAV collision intensity W between 0 and 1. ij This forms the UAV conflict matrix G. uav The time of future collisions between any heterogeneous UAV i and obstacle k node pair (i,k) is quantized as the obstacle collision intensity w between 0 and 1. ik This forms the obstacle conflict matrix G. obs The expression is:

[0071]

[0072] in, Indicates the time t when the conflict will occur in the future. c The edge weight of the next node represents the intensity of conflict between node pairs.

[0073] Step 205: Determine the most urgent drone conflict and obstacle conflict based on the drone and obstacle conflict matrix, and obtain the urgent drone conflict intensity of heterogeneous drone i. and the intensity of conflict with emergency obstacles The specific formula is as follows:

[0074]

[0075] Step 206: Based on the conflict intensity value, multiply it by the normalized repulsive force direction to obtain the heterogeneous UAV conflict repulsive force. Repulsive force in conflict with obstacles The specific formula is as follows:

[0076]

[0077] in, P represents the position of heterogeneous UAV j that conflicts with UAV i at time t; m Indicates the location of obstacle m that conflicts with drone i.

[0078] Step 207: Based on the conflict and repulsion force of heterogeneous UAVs Repulsive force in conflict with obstacles Obtain the resultant force F of the current potential field function APF And calculate the final heuristic action α. APF The expression is:

[0079]

[0080]

[0081] Among them, F APF,y F represents APFThe force decomposition along the y-axis; F APF,x F represents APF The force is decomposed along the x-axis.

[0082] Step 3: Update the state space based on the heuristic actions, input it into the heterogeneous graph neural network, extract the heterogeneous UAV node information through the ordered gating coding mechanism, calculate the expected value Q of each action through the action decision network, and select the action with the largest expected value Q as the action to be executed.

[0083] Step 301: Update the state space: Replace the original velocity direction with the heuristic action obtained in step 2, encode the new state space through a fully connected network, and obtain the initial state vector of each heterogeneous UAV.

[0084] Furthermore, the initial state vector of the heterogeneous UAV includes the new heuristic action (deflection direction), as well as the original settings for the heterogeneous UAV's starting position, target point position, safety radius, speed magnitude, and the center and radius of each obstacle.

[0085] Step 302: Let k=1, and obtain the implicit information of each heterogeneous UAV extracted from the first layer of the heterogeneous graph neural network based on the initial state vector of each UAV.

[0086] Step 303: Extract the implicit information of the neighboring UAV u from the (k-1)th layer of the heterogeneous graph neural network. Information aggregation is performed to obtain the neighbor drone information of heterogeneous drone i after aggregation at the k-th layer of the network. The specific formula is as follows:

[0087]

[0088] Among them, AGGREGATE (k) This represents the k-th layer network of the AGGREGATE operator; This represents the information about the neighboring drone u extracted from the (k-1)th layer of the heterogeneous graph neural network; This represents the total number of neighboring heterogeneous drones.

[0089] Furthermore, the AGGREGATE operator uses the GAT model based on conflict graph networks.

[0090] Step 304: Based on the ordered gating coding mechanism, combine the implicit information of the heterogeneous UAV i in the (k-1)th layer network with the neighboring UAV information aggregated in the kth layer network. By combining these methods, we obtain the intermediate fused representation of the heterogeneous UAV i in the k-th layer network. The expression is:

[0091]

[0092] Among them, w k This represents the weight matrix of the k-th layer of a neural network (e.g., a fully connected MLP) used to extract information. This represents the implicit information of the heterogeneous UAV extracted by the (k-1)th layer of the neural network; b k This represents the bias matrix of the k-th layer network.

[0093] Step 305: Based on the intermediate fusion representation of heterogeneous UAV i in the k-th layer network, obtain the gate vector, expressed as:

[0094]

[0095] in, This represents the gate vector of the k-th layer network; This represents the gate vector of the (k-1)th layer of the network;

[0096] See Figure 2 The learnable gating vector of the present invention Using intermediate fusion representation By using the gating vector of the previous layer The accumulation of features is used to measure the fusion ratio between the node itself and its neighboring nodes, ensuring that as the depth of the neural network increases and information is transmitted, the proportion of the feature information of the heterogeneous node itself becomes larger and larger.

[0097] Step 306: Using gated vectors The hidden information of the heterogeneous drone itself extracted from the (k-1)th layer network of the heterogeneous drone. Neighbor drone information aggregated with the k-th layer network Aggregation is performed to obtain the implicit information of the heterogeneous UAVs extracted from the k-th layer network. The expression is:

[0098]

[0099] Step 307: Based on the implicit information of the heterogeneous UAV extracted from the k-th layer network, a fully connected layer is used as the action decision network to obtain the expected value Q of each action, and the action α with the largest expected value Q is selected as the action to be executed. RL The expression is:

[0100]

[0101] Step 308: Determine if k has reached the maximum number of layers in the heterogeneous graph neural network. If it has, proceed with the action α to be executed. RL This is the final action to be executed; if it is not achieved, let k = k + 1 and return to step 303.

[0102] Step 4, see Figure 3 The system determines the type of the final action to be executed. If it is a deflection angle, it is input into the angle smoothing function along with the heuristic action to achieve dynamic correction of the heuristic action. If it is a hovering maneuver, the speed of the heterogeneous UAV is reduced to 0, and the UAV's heading is adjusted to point towards the endpoint.

[0103] Furthermore, if the final action to be performed is a hovering maneuver, the heterogeneous UAV will be urgently decelerated until its speed reaches 0, based on the magnitude of its acceleration, so that the heterogeneous UAV can reach a hovering state.

[0104] Furthermore, if the final action to be performed is a deflection maneuver, the final deflection angle of heterogeneous UAV i is calculated based on the relative positional relationship between the heterogeneous UAV and the target position. To ensure the heterogeneous drone is as close to the target location as possible, the deflection angle should be as small as possible, avoiding excessive correction of the heuristic motion that could make it difficult to reach the target location. The specific formula is as follows:

[0105]

[0106] in, Indicates the position of heterogeneous UAV i at time t+1; e represents the position of heterogeneous UAV i at time t; i This indicates the target location of the heterogeneous UAV i.

[0107] Step 5: Calculate the reward obtained by finally executing the action to be executed at time t and store it as an experience sample in the experience pool; apply the DQN algorithm, and use the experience replay method to batch sample experience samples to train the conflict resolution model, iteratively optimize the conflict resolution model parameters, until the conflict resolution model training is completed.

[0108] Step 501: Based on the conflict resolution model, calculate the reward R by analyzing the conflict situation, trajectory offset, and hover count after executing the final action. t and compare it with state S at time t. t Selected action α t State S at time t+1 t+1 Shared storage as empirical samples {S t ,α t ,R t ,S t+1 The data is stored in the experience pool for subsequent training and sampling of conflict resolution models.

[0109] Step 502: After reaching the preset number of training rounds, after each time step has completed the action, the conflict resolution model selects a batch of samples from the experience pool.

[0110] Step 503: Calculate the loss of the collected experience samples using an action decision network. The expression is:

[0111] a max =argmax a Q(s t+1 ,a;θ)

[0112]

[0113] Among them, a max Let represent the action with the largest Q-value among all selectable actions; 'a' represents all actions in the action space; θ represents the parameters of the conflict resolution model; L(θ) represents the loss calculated under the parameters θ of the conflict resolution model; N represents the total number of heterogeneous UAVs i; r i G represents the reward value of heterogeneous drone i within this empirical sample; γ represents the decay factor; G t+1 G represents the graph structure matrix at time t+1; t The graph structure representing time t; a t This indicates the action selected at time t.

[0114] Step 504: Update the parameters θ of the conflict resolution model using the gradient descent method to gradually reduce the loss.

[0115] Step 6: In each round of the scenario, execute the maneuvering and training process from Step 2 to Step 5 according to the time step, and update the position and velocity direction of each UAV until all heterogeneous UAVs reach the target position or reach the maximum number of running time steps N in a single round, then proceed to Step 7.

[0116] Step 7: Determine if the maximum number of scene iterations T has been reached. If it has, obtain the trained conflict resolution model; otherwise, return to step 1.

[0117] Step 8: Deploy the trained conflict resolution model on the onboard smart board of each UAV, and use the actual information in the application scenario as input for the conflict resolution model at each time step, and finally output the collision avoidance maneuvers of each heterogeneous UAV.

[0118] Furthermore, the actual information in the application scenario includes the drone's own operating parameters, the speed and position of other detected drones, and the size and relative distance of detected obstacles.

[0119] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in the present invention should be included within the scope of protection of the present invention.

Claims

1. A heterogeneous UAV conflict resolution method based on reinforcement graph learning, characterized in that, It includes the following specific steps: Step 1: Initialize the training environment, the state information of each heterogeneous UAV and obstacles, and set the state space, action space, reward function, maximum number of scene iterations and maximum running time steps per round of the UAV; Step 2: Establish the potential field function. Based on the UAV's flight state and obstacle state, obtain the gravitational force for reaching the target, the gravitational force for maintaining direction, the UAV's collision repulsion force, and the obstacle's collision repulsion force. Based on the resultant force on the UAV, obtain the initial optimal deflection angle as a heuristic action. Step 3: Update the state space based on the heuristic actions, input it into the heterogeneous graph neural network, and extract the implicit information of the heterogeneous UAV through the ordered gating coding mechanism; based on the implicit information of the heterogeneous UAV, calculate the expected value of each action through the action decision network, and select the action with the largest expected value as the action to be executed. Step 4: Determine the type of action to be performed. If it is a deflection angle, input it and the heuristic action into the angle smoothing function to dynamically correct the heuristic action. If it is a hovering maneuver, reduce the speed of the heterogeneous UAV to 0 and adjust the heading of the heterogeneous UAV to point to the target point. Step 5: Calculate the reward gained by the heterogeneous UAV in performing the action to be performed at time t, and store it in the experience pool as an experience sample; Batch sampling of empirical samples trains the conflict resolution model, and iterative optimization of the conflict resolution model parameters is performed until the conflict resolution model training is completed. Step 6: In each round of the scenario, execute the maneuvering and training process from Step 2 to Step 5 according to the time step, and update the position and speed direction of each drone until all heterogeneous drones reach the target point or reach the maximum number of running time steps in a single round, then proceed to Step 7. Step 7: Determine if the maximum number of scene iterations has been reached. If it has, obtain the trained conflict resolution model; otherwise, return to step 1. Step 8: At each time step, use the actual information in the application scenario as the input to the conflict resolution model, and finally output the collision avoidance maneuvers of each heterogeneous UAV.

2. The heterogeneous UAV conflict resolution method according to claim 1, characterized in that, The specific steps of step 2 are as follows: Step 201: Construct the arrival potential field function and the intended gravitational potential field to obtain the gravitational force for reaching the target and the direction-maintaining gravitational force; Step 202: Based on the target gravity and the direction-maintaining gravity, obtain the gravitational potential field force intended to maintain the direction, and obtain the initial gravitational heading through the gravitational potential field; Step 203: Based on the potential conflicts that may exist along the initial gravitational heading, use the velocity obstacle model to calculate the future conflict times of heterogeneous UAV-heterogeneous UAV node pairs and heterogeneous UAV-obstacle node pairs. Step 204: Quantify the time of future conflicts between any heterogeneous UAV-heterogeneous UAV node pairs into UAV conflict intensity to form a UAV conflict matrix; The time of future collisions between any heterogeneous UAV-obstacle node pair is quantified into obstacle collision intensity, forming an obstacle collision matrix; Step 205: Determine the most urgent drone conflict and obstacle conflict based on the drone and obstacle conflict matrix, and obtain the urgent drone conflict intensity and urgent obstacle conflict intensity of heterogeneous drones. Step 206: Based on the collision intensity between the UAV and the obstacle, and the direction of the repulsive force, obtain the collision repulsive force between the heterogeneous UAV and the obstacle; Step 207: Based on the collision repulsion force of heterogeneous UAVs and the collision repulsion force of obstacles, obtain the current potential field function resultant force and obtain the final heuristic action.

3. The heterogeneous UAV conflict resolution method according to claim 2, characterized in that, The specific steps of step 3 are as follows: Step 301: Update the state space: Replace the original velocity direction with the heuristic action, encode the new state space through a fully connected network, and obtain the initial state vector of each heterogeneous UAV. Step 302: Let k=1, and obtain the implicit information of each heterogeneous UAV extracted from the first layer of the heterogeneous graph neural network based on the initial state vector of each UAV. Step 303: Extract the implicit information of the neighboring UAV u from the (k-1)th layer of the heterogeneous graph neural network. Perform information aggregation to obtain the neighboring drone information of heterogeneous drone i after the k-th layer network aggregation; Step 304: Based on the ordered gating coding mechanism, combine the implicit information of the heterogeneous UAV i in the (k-1)th layer network with the neighboring UAV information after aggregation in the kth layer network to obtain the intermediate fusion representation of the heterogeneous UAV i in the kth layer network. Step 305: Obtain the gating vector based on the intermediate fusion representation of the heterogeneous UAV in the k-th layer network; Step 306: Using the gating vector, the implicit information of the heterogeneous UAV itself extracted by the (k-1)th layer network of the heterogeneous UAV is aggregated with the neighbor UAV information after aggregation by the kth layer network to obtain the implicit information of the heterogeneous UAV itself extracted by the kth layer network. Step 307: Based on the implicit information of the heterogeneous UAV extracted from the k-th layer network, the expected value of each action is obtained through a fully connected layer as the action decision network, and the action with the largest expected value is selected as the action to be executed. Step 308: Determine whether k has reached the maximum number of layers in the heterogeneous graph neural network. If it has, set the action to be executed as the final action to be executed; if it has not, let k = k + 1 and return to step 303.

4. The heterogeneous UAV conflict resolution method according to claim 3, characterized in that, The specific steps of step 5 are as follows: Step 501: Based on the conflict situation, trajectory offset, and number of hovering after the final action to be executed according to the conflict resolution model, calculate the reward income through the reward function, and store it together with the state at time t, the selected action, and the state at time t+1 as an experience sample and store it in the experience pool. Step 502: After reaching the preset number of training rounds, after each time step has completed the action, the conflict resolution model selects a batch of samples from the experience pool. Step 503: Calculate the loss of the collected experience samples using an action decision network; Step 504: Update the parameters of the conflict resolution model using the gradient descent method to gradually reduce the loss.

Images