Multi-uav airspace conflict resolution and scheduling method based on counterfactual causal reasoning

Abstract

This invention discloses a method for resolving and scheduling airspace conflicts among multiple unmanned aerial vehicles (UAVs) based on counterfactual causal reasoning, comprising the following steps: S1, constructing a multi-leader-multi-follower Stackelberg game model; S2, performing equilibrium analysis on the Stackelberg game model using backward induction; S3, modeling the Stackelberg game model as a partially observable Markov decision process; S4, obtaining the counterfactual individual contribution of each UAV within the framework of the partially observable Markov decision process; S5, incorporating the counterfactual individual contribution as an intrinsic reward into the policy optimization objective; S6, obtaining the neuron importance ranking and dynamic pruning threshold; S7, updating the pruning binary mask and reconstructing the policy network; S8, obtaining a lightweight neural network model adapted for real-time deployment on UAV airborne platforms. This invention has the advantages of strong game modeling capabilities, interpretable causal contributions, and ease of real-time airborne deployment.

Description

Technical Field

[0001] This invention relates to the field of air vehicle traffic control system technology, specifically to a method for resolving and scheduling airspace conflicts among multiple unmanned aerial vehicles (UAVs) based on counterfactual causal reasoning. Background Technology

[0002] With the booming development of the low-altitude economy, the number of drones in urban airspace has increased dramatically, making the resolution of airspace conflicts among multiple drones a core challenge for air traffic control systems. In dense airspace environments, multiple drones need to perform real-time route planning, conflict detection, and active obstacle avoidance under the coordination of the air traffic control center to prevent mid-air collisions. Due to the high speed, high maneuverability, and dynamic changes in the flight environment of drones, traditional methods based on fixed airspace allocation are insufficient to meet the real-time scheduling needs of high-density airspace.

[0003] Multi-UAV airspace conflict resolution involves a game of airspace usage rights allocation and dynamic route scheduling between air traffic control centers and UAV formations. Existing methods mainly rely on rule-based conflict detection and resolution algorithms, such as TCAS (Traffic Collision Avoidance System) and ADS-B (Automatic Dependent Surveillance-Broadcast), which lack the ability to model multi-UAV cooperative game behavior and struggle to handle complex conflict scenarios involving large-scale UAV formations. While deep reinforcement learning methods can handle complex decisions, they suffer from two main problems: first, it is difficult to quantify the causal impact of individual UAV decisions on the overall safety posture of the formation in a high-dimensional state-action space; second, neural network models have a large number of parameters and high computational cost, making it difficult to meet the millisecond-level response requirements for airspace conflict resolution.

[0004] Existing multi-agent deep reinforcement learning algorithms mostly employ random exploration strategies, such as ε-greedy or Gaussian noise exploration, making it difficult to identify key obstacle avoidance actions that have a significant positive impact on the overall formation safety situation through counterfactual reasoning. Neural network pruning methods typically employ a uniform sparsity strategy, failing to adaptively adjust according to the UAV's contribution to formation safety, which may lead to the mispruning of safety-critical neurons. In summary, the existing technologies have the following shortcomings: (1) lack an effective game modeling method to characterize the airspace resource allocation game between the air traffic control center and the UAV formation; (2) make it difficult to quantify the causal contribution of individual UAV obstacle avoidance decisions to the overall formation safety situation; and (3) make it difficult to deploy neural network models in real time on UAV airborne platforms. Summary of the Invention

[0005] To overcome the shortcomings of existing technologies, the present invention aims to provide a method for resolving and scheduling airspace conflicts among multiple unmanned aerial vehicles (UAVs) based on counterfactual causal reasoning. It achieves the optimal strategy solution for airspace resource allocation and route scheduling by constructing a Stackelberg game model, accurately quantifies the causal contribution of individual UAV obstacle avoidance decisions to the overall safety situation of the formation through a counterfactual causal reasoning mechanism, and realizes lightweight deployment of neural networks through an adaptive pruning strategy. It has the advantages of strong game modeling ability, interpretable causal contribution, and easy real-time airborne deployment.

[0006] To achieve the objective of this invention, the following solution is adopted:

[0007] A method for resolving and scheduling airspace conflicts among multiple unmanned aerial vehicles (UAVs) based on counterfactual causal reasoning includes the following steps:

[0008] S1. Construct a multi-leader-multi-follower Stackelberg game model, define the drone utility function and the air traffic control sector utility function. The drone utility function includes an airspace safety violation penalty term, and the control sector utility function includes an airspace conflict penalty term. The game relationship between airspace pricing strategy and airspace use demand strategy is obtained through the Stackelberg game model.

[0009] S2. Using backward induction, the Stackelberg game model is analyzed for equilibrium to find the optimal airspace use demand strategy for UAVs and the optimal airspace pricing strategy for controlled sectors, thus obtaining the Stackelberg equilibrium solution.

[0010] S3. The Stackelberg game model is modeled as a partially observable Markov decision process. The global state space, action space and reward function are defined using the Stackelberg equilibrium solution to obtain a mathematical description of the multi-agent decision process.

[0011] S4. Within the framework of the partially observable Markov decision process, a counterfactual individual contribution measurement mechanism based on the counterfactual distribution difference is introduced. A conditional variational autoencoder is used to estimate the difference in the security situation distribution between the factual scenario and the counterfactual scenario, quantify the causal impact of single-drone obstacle avoidance actions on the overall security situation of the formation, and obtain the counterfactual individual contribution of each UAV.

[0012] S5. An improved multi-agent proximal policy optimization algorithm is adopted, incorporating the counterfactual individual contribution as an intrinsic reward into the policy optimization objective, and performing policy optimization on the actions in the action space to obtain optimized policy network parameters.

[0013] S6. Design a neuron importance metric based on time window decay, and design an adaptive pruning threshold based on the counterfactual individual contribution, evaluate the importance of neurons in the policy network corresponding to the policy network parameters, and obtain the neuron importance ranking and dynamic pruning threshold.

[0014] S7. Based on the neuron importance ranking and the dynamic pruning threshold, update the pruning binary mask and reconstruct the policy network to obtain a lightweight neural network structure.

[0015] S8. The network parameters in the reconstructed lightweight neural network structure are iteratively updated using the stochastic gradient method to obtain a lightweight neural network model adapted to real-time deployment on the UAV airborne platform.

[0016] Further, in step S1, the utility function of the UAV is:

[0017]

[0018] The utility function of the controlled sector is:

[0019]

[0020] in, For drone swarm assembly, For the control sector set, For drones The spatial domain uses the demand vector. This vector represents the airspace usage needs of other drones. For the pricing strategy vector of the regulated sector, To remove sectors Pricing strategy vectors for other regulated sectors A matrix of airspace usage requirements for all drones. For drones To sector The requested airspace usage weight, For drones Select sector The matching probability, For sectors Service quality parameters For sectors airspace usage price, The task satisfaction coefficient. The base of the natural constant is... The delay sensitivity coefficient, To minimize the delay in conflict resolution, For formation coordination safety factors, For sector route complementarity parameters, The penalty coefficient for airspace safety violations. For sectors For drones The cost of providing services The sector airspace conflict penalty coefficient, For drones With drones Real-time distance between them For safe interval distance, For indicator functions, in the conflict penalty term Constraints are used to avoid conflicting and redundant calculations between drone pairs.

[0021] Further, in step S4, the counterfactual individual contribution is defined as:

[0022]

[0023] in, For Kullback-Leibler divergence, As a potential security situation variable, For time step The global state, For the motion vectors of all drones, To eliminate drones The motion vectors of other drones, For including drones Prior distribution of the formation safety situation of the action. To rule out drones The counterfactual security posture prior distribution of actions combines the individual contribution of counterfactual actions with external rewards to form a hybrid reward system. ,in As a reward for the external environment, This is the hyperparameter for intrinsic reward weighting.

[0024] Further, in step S4, the conditional variational autoencoder includes:

[0025] Fact Encoder The input includes the target UAV's state-action pair and the state at the next time step. Output potential security situation variables The posterior distribution of;

[0026] Counterfact encoder Input the state-action pair and the state at the next time step of the excluded target UAV actions, and output the counterfactual posterior distribution;

[0027] Fact Prior Network Estimate the prior distribution of the security situation including the actions of the target UAV;

[0028] Counterfactual priori networks Estimate the prior distribution of the safe situation when excluding the actions of the target UAV;

[0029] Security posture decoder Input latent variables Output the reconstructed security posture;

[0030] in, , , These are the learnable parameters for the two encoders and decoders, respectively. , These are the learnable parameters of the two prior networks;

[0031] The conditional variational autoencoder is trained by maximizing the variational lower bound, which includes the KL divergence term of the encoder and the corresponding prior network, and the reconstruction error term of the decoder.

[0032] Further, in step S5, the loss function of the improved multi-agent proximal policy optimization algorithm is:

[0033]

[0034] in, The objective function for the actor strategy iteration. For the critic loss function, For actor network parameters, These are the parameters for the critic network. For policy entropy, For entropy regularization weights, For the critic loss weight, Sigmoid annealing weights are used , For annealing rate, This represents the current number of training steps. For annealing center steps, counterfactual individual contribution is incorporated into the critic value assessment through a hybrid reward system.

[0035] Further, in step S6, the neuron importance metric based on time window decay is:

[0036]

[0037] in, For time step Time Layer The current importance of each neuron. For the first Layer The first neuron and the second Layer Connection weights between neurons For the first Layer The first neuron and the second Layer Connection weights between neurons and These are the indices of neurons in the previous and next layers, respectively. As the attenuation factor, For time step The binary mask, This represents the width of the time window.

[0038] Furthermore, the adaptive pruning threshold in step S6 is: all neurons are ranked according to importance. Sort in ascending order, take the first... The importance value of each neuron is used as a threshold. ,in, This is the floor function. The total number of neurons. To adapt to the pruning rate, For sensitivity hyperparameters, For the normalized counterfactual contribution, among which For the prior distribution of security posture in real-world scenarios, For the prior distribution of security posture in counterfactual scenarios, It is a mixed distribution. The Jensen-Shannon divergence, when using the base-2 logarithm, has a range of values ​​of [value missing]. , The original progressive pruning rate, of which The initial sparsity, For target sparsity, This is the starting round for pruning. This represents the total number of pruning sessions. For pruning frequency, when the counterfactual contribution of drones is high Reduce the pruning rate to decrease the counterfactual contribution, and increase the pruning rate when the counterfactual contribution is low.

[0039] Furthermore, in step S3, the state transition function of the partially observable Markov decision process is determined by the UAV kinematic model, specifically as follows:

[0040]

[0041] in, For drones At time step The position vector, For drones At time step The position vector, For drones The obstacle avoidance velocity vector, For the time step, take s, It is the Euclidean norm.

[0042] Furthermore, after step S5 and before step S6, step S5a is also included: introducing equilibrium prior action-level guidance before action execution, projecting the regulated sector pricing action and the UAV demand action onto the direction of the UAV optimal airspace use demand policy in the Stackelberg equilibrium solution according to the projection ratio, and triggering recovery enhancement when the equilibrium ratio is lower than the threshold, and using the projected action as the actual execution action for policy optimization by the improved multi-agent proximal policy optimization algorithm.

[0043] Further, in step S5a, the action-level guidance satisfies:

[0044]

[0045] in, and These are the original pricing action and the original demand action output by the strategy network, respectively. and These are the actions after projection. For reference pricing, For the follower's optimal response under projection pricing, This is the projection scale.

[0046] Furthermore, the projection scale adopts a joint mechanism of schedule scheduling and balanced recovery:

[0047]

[0048]

[0049] in, This represents the equilibrium ratio at the previous time step. To restore the threshold, To restore the magnitude of the enhancement.

[0050] Furthermore, safe interval distance Set the m based on the type of drone: 50m for multi-rotor drones, 100m for fixed-wing drones, and 80m for unmanned helicopters.

[0051] Furthermore, in step S7, when reconstructing the policy network, the Lagrange multiplier method is used to transform the constrained optimization problem of retaining the number of neurons into an unconstrained optimization problem. The reconstructed actor loss function is:

[0052]

[0053] in, Importance ratio, For the clipping function, For generalized advantage estimation, For Lagrange multipliers, regularization terms The importance of pruned neurons is summed to punish mispruning behavior.

[0054] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0055] 1. This invention constructs a multi-leader, multi-follower Stackelberg game model, effectively characterizing the game relationship of airspace resource allocation. Addressing the lack of modeling capabilities for multi-aircraft cooperative game behavior in existing methods, this invention solves the Stackelberg equilibrium solution through backward induction, accurately characterizing the game relationship of airspace usage rights allocation and dynamic route scheduling between air traffic control centers and UAV formations, providing a theoretically optimal equilibrium strategy benchmark for subsequent decision-making processes.

[0056] 2. This invention introduces a counterfactual individual contribution measurement mechanism to quantify the causal impact of individual obstacle avoidance decisions on the overall safety posture of the formation. Addressing the difficulty of quantifying the causal contribution of individual decisions using deep reinforcement learning methods, this invention employs a conditional variational autoencoder to estimate the difference in safety posture distribution between the factual and counterfactual scenarios. It calculates the counterfactual individual contribution using KL divergence, accurately identifying obstacle avoidance actions with a key positive impact on the formation's safety posture, thus overcoming the technical deficiency of existing methods in quantifying individual contributions.

[0057] 3. This invention incorporates counterfactual contribution into policy optimization, improving the performance of multi-agent collaborative decision-making. This invention integrates counterfactual individual contribution as an intrinsic reward into an improved multi-agent proximal policy optimization algorithm, enabling the policy network to learn obstacle avoidance behaviors that positively contribute to formation safety. Compared to the MAPPO baseline, the equilibrium ratio is improved by approximately 22.6%, significantly enhancing the performance and stability of airspace conflict resolution.

[0058] 4. This invention designs an adaptive pruning strategy, achieving lightweight deployment. Addressing the issue of large parameter counts in neural network models, making it difficult to meet millisecond-level response requirements, this invention designs a neuron importance metric based on time window decay and dynamically adjusts the pruning threshold according to counterfactual contribution. The policy network is then reconstructed, and a lightweight neural network model is obtained through iterative updates using the stochastic gradient method. This significantly reduces computational overhead while maintaining conflict resolution performance. After training, the CVAE module can be removed, retaining only the lightweight actor network, adapting to the real-time deployment requirements of UAV-borne heterogeneous computing platforms.

[0059] 5. This invention forms a complete technical closed loop, collaboratively solving the core problem of resolving airspace conflicts among multiple UAVs. From game theory modeling, causal reasoning, strategy optimization to model pruning, each step of this invention is interconnected, jointly solving three major technical problems in existing technologies: "lack of game theory modeling, difficulty in quantifying causal contributions, and inability to deploy in real time." This provides a complete technical solution for UAV traffic control in dense airspace under low-altitude economic scenarios. Attached Figure Description

[0060] Figure 1 This is a flowchart of a multi-UAV airspace conflict resolution and scheduling method based on counterfactual causal reasoning in an embodiment of the present invention;

[0061] Figure 2 This is a schematic diagram of the multi-UAV airspace conflict resolution and scheduling method based on counterfactual causal reasoning in an embodiment of the present invention.

[0062] Figure 3 This is an architecture diagram of the TinyMA-CF-PPO algorithm in an embodiment of the present invention. Detailed Implementation

[0063] The present invention will now be further described in conjunction with the accompanying drawings and specific embodiments. It should be noted that, without conflict, the various embodiments or technical features described below can be arbitrarily combined to form new embodiments.

[0064] Explanation of key symbols:

[0065]

[0066]

[0067] like Figure 1-3 As shown, this invention provides a method for resolving and scheduling airspace conflicts among multiple unmanned aerial vehicles (UAVs) based on counterfactual causal reasoning, comprising the following steps:

[0068] S1. Construct a multi-leader-multi-follower Stackelberg game model, define the drone utility function and the air traffic control sector utility function. The drone utility function includes an airspace safety violation penalty term, and the control sector utility function includes an airspace conflict penalty term. The game relationship between airspace pricing strategy and airspace usage demand strategy is obtained through the Stackelberg game model.

[0069] In this embodiment, the specific details of constructing the multi-leader-multi-follower Stackelberg game model are as follows:

[0070] Define drone swarm set and air traffic control sectors Multi-UAV airspace conflict resolution comprises six stages: situational awareness, conflict detection, obstacle avoidance decision-making, airspace allocation, route execution, and state synchronization. (Definition of UAVs) Controlled sectors Matching probability ,satisfy Since all drones face the same airspace service quality and price information, the matching probability applies to all drones. Same (i.e.) Only depend on ), using a price-quality driven selection model:

[0071]

[0072] The overall conflict resolution delay is ,in Due to the delay in conflict detection, To avoid obstacles, decision-making was delayed. For the delay of mobile execution, This is for state synchronization delay.

[0073] drones The utility function is:

[0074]

[0075] in, For drones The weight vector for airspace usage requested from each controlled sector. For drones To the controlled sector The requested airspace usage weight, For controlled sectors The unit airspace usage price, For airspace quality of service parameters, The task satisfaction coefficient. The delay sensitivity coefficient, To minimize the delay in conflict resolution, For formation coordination safety factors, For sector route complementarity / conflict parameters, Penalty coefficient for airspace safety violations (typical value) ), For drones With drones Real-time distance between them For safe interval distance, For indicator functions, The base of the natural constant ( ).

[0076] Air traffic control sector The utility function is:

[0077]

[0078] in, To remove sectors Pricing strategy vectors for other regulated sectors A matrix of airspace usage requirements for all drones. For controlled sectors For drones The cost of providing airspace services For sectors Airspace conflict penalty coefficient (typical value) ). Conflict penalty items adopt The constraint avoids double-counting of conflicts between drone pairs, meaning that conflicts between each drone pair are only counted once.

[0079] In a two-stage Stackelberg game, the controlled sector in stage I determines the airspace use pricing strategy. (in To minimize service costs, (Maximum permissible pricing), Phase II UAV airspace usage demand strategy The constraints are and .

[0080] S2. Using backward induction, the Stackelberg game model is analyzed for equilibrium to find the optimal airspace use demand strategy for UAVs and the optimal airspace pricing strategy for controlled sectors, thus obtaining the Stackelberg equilibrium solution.

[0081] In this embodiment, the specific details of the Stackelberg equilibrium analysis are as follows:

[0082] The analysis employs backward induction. For follower-level games, the first-order optimality condition is used... Obtain the drone The optimal response:

[0083]

[0084] in, The sum of formation coordination and sector complementarity effects, (must meet) To ensure logarithmic parameters The optimal response function must satisfy the non-negativity constraint. ,and This ensures that the cost of airspace use exceeds the synergistic effect. It is proven that the optimal response function satisfies the three properties of the standard function—positivity, monotonicity, and scalability—ensuring the existence and uniqueness of the follower-level Nash equilibrium.

[0085] For leader-level games, the follower-optimal strategy is substituted into the utility function of the regulated sector to obtain the optimal airspace pricing strategy of the regulated sector. This proves that the leader-level Nash equilibrium exists and is unique, thus confirming that the Stackelberg equilibrium exists and is unique.

[0086] Theorem 1 (Existence and Uniqueness of Follower-Level Nash Equilibrium): Given a pricing strategy for a regulated sector... Under these conditions, the game of airspace use demand for drone formations has a unique Nash equilibrium. .

[0087] Proof points: Verify the optimal response function Satisfying the three properties of the standard function: (1) Positiveness: when hour, (2) Monotonicity: If ,but (3) Scalability: For , (in This is the scaling factor, to distinguish it from the delay sensitivity factor. and annealing rate According to the standard function fixed-point theorem, there exists a unique fixed point, namely Nash equilibrium.

[0088] Theorem 2 (Existence and Uniqueness of Stackelberg Equilibrium): A unique equilibrium exists in a multi-leader, multi-follower Stackelberg game. .

[0089] Proof points: The optimal response of the follower Substitute into the leader utility function and verify. about Strictly concave (second derivative) Thus, there exists a unique leader-level Nash equilibrium.

[0090] S3. The Stackelberg game model is modeled as a partially observable Markov decision process. The global state space, action space, and reward function are defined using the Stackelberg equilibrium solution to obtain a mathematical description of the multi-agent decision process.

[0091] In this embodiment, the specific details of constructing a partially observable Markov decision process (POMDP) ​​model are as follows:

[0092] Define POMDP quintuple ,in For the global state space, For observation space, For the action space, This is the state transition function. The reward function (the subscript ew is used to distinguish the set of controlled sectors) ):

[0093] (1) Global state space: This includes airspace pricing strategies, airspace usage demand, and arrival rates in conflict situations. Conflict resolution rate Controlled sector rewards Drone rewards UAV spacing matrix .

[0094] (2) Partial observation space: Controlled sectors Observations drones Observations ,in For sectors Internal UAV spacing sub-matrix For sectors Internal drone collection, The historical observation window length is taken as... .

[0095] (3) Action space: Controlled sector actions Drone Actions ,in For airspace usage requirements, This is the three-dimensional obstacle avoidance velocity vector.

[0096] (4) State transition function: Describe the current state Execute joint actions Then transition to the next state The probability distribution. State transitions are determined by the UAV kinematic model and the dynamics of the controlled sector resources: , ,in For drones At time step The position vector, The time step (take) s).

[0097] (5) Reward function: Controlled sector reward Drone rewards .

[0098] S4. Within the framework of the partially observable Markov decision process, a counterfactual individual contribution measurement mechanism based on the counterfactual distribution difference is introduced. A conditional variational autoencoder is used to estimate the difference in the security situation distribution between the factual scenario and the counterfactual scenario, quantify the causal impact of single-drone obstacle avoidance actions on the overall security situation of the formation, and obtain the counterfactual individual contribution of each UAV.

[0099] In this embodiment, the specific details of the counterfactual individual contribution measurement mechanism based on counterfactual distribution dissimilarity (a variant of Bayesian surprise) are as follows:

[0100] For any drone Define its time step Execute action The counterfactual individual contribution is the KL divergence between the security posture distributions of the factual and counterfactual scenarios. This metric quantifies the causal impact of individual drone decisions on the overall security posture of the formation by constructing counterfactual scenarios of "whether or not the drone took action".

[0101]

[0102] in, For Kullback-Leibler divergence, For including drones Prior distribution of formation safety situation during maneuvers (factual scenario). To rule out drones Prior distribution of formation safety posture during maneuvers (counterfactual scenario, i.e., assuming drones) No action was performed or the default action was performed. For time step The motion vectors of all drones To eliminate drones The motion vectors of other drones. The core idea of ​​this counterfactual causal reasoning mechanism is: if the obstacle avoidance actions of a certain drone significantly change the safety situation of the formation (the difference between factual and counterfactual distributions is large), then the drone makes a key positive contribution to the resolution of airspace conflicts.

[0103] Conditional variational autoencoders (CVAEs) are used to estimate counterfactual individual contributions. A CVAE consists of two encoders. and Two prior networks and and decoder The training objective is to maximize the variational lower bound:

[0104]

[0105] in, For the posterior distribution of the fact encoder, For the posterior distribution of the counterfactual encoder, For the fact prior distribution, For the counterfactual prior distribution, the first two terms are the KL divergence terms between the encoder posterior distribution and the prior network prior distribution (negated to maximize), and the last two terms are the expected reconstruction error of the decoder (log-likelihood terms).

[0106] Combining counterfactual individual contributions as intrinsic and extrinsic rewards creates a hybrid reward system.

[0107]

[0108] in, For drones At time step External environmental rewards (task utility and safety penalties). Contribution to counterfactual individuals To balance the hyperparameters of intrinsic motivation and extrinsic reward (take...) ).

[0109] S5. An improved multi-agent proximal policy optimization algorithm is adopted, incorporating the counterfactual individual contribution as an intrinsic reward into the policy optimization objective, and performing policy optimization on the actions in the action space to obtain optimized policy network parameters.

[0110] In this embodiment, the specific details of the improved Multi-Agent Proximity Policy Optimization (MAPPO) algorithm are as follows:

[0111] For any intelligent agent (drone or controlled sector), the parameters are as follows: Parameterized policies in actor networks (policy networks) The parameters are Parameterized value function of the critic network (value network) With drones For example, based on hybrid rewards The critic loss function is:

[0112]

[0113] The iteration goal of the actor strategy is:

[0114]

[0115] in, Importance ratio, For the clipping function ( (PPO (Proximal Policy Optimization) parameter trimming). For drones At time step Generalized Advantage Estimation (GAE) ,in The total number of time steps in a single episode. Temporal Difference (TD) error (here) The TD error is compared with the task satisfaction coefficient in step S1. For different physical quantities, use time superscript distinguish), For GAE parameters, This is the discount factor.

[0116] Improve the PPO loss function by incorporating policy entropy regularization:

[0117]

[0118] in, For policy entropy, To control the hyperparameters of the entropy regularization weights, For the critic loss weight, These are entropy-regularized weights. Training employs a step-by-step optimization approach: minimizing the actor network.

[0119]

[0120] critic network minimized Counterfactual individual contribution Through mixed rewards Value assessments incorporating critics implicitly influence the advantage function. Calculation of annealing weights. ,in For the annealing rate hyperparameter (take... ), This represents the current number of training steps. The number of annealing center steps (take) The sigmoid (S-shaped function) annealing mechanism makes the initial training phase ( hour, ) Approaching 1 encourages strategy exploration, in the later stages of training ( hour, ) Approaching 0 is a stable strategy.

[0121] Step S5a: Introduce an action-level guidance mechanism based on Stackelberg equilibrium priors. Step S5a is an optional enhancement step.

[0122] To reduce invalid exploration in the early stages of training and improve convergence stability, at time step... Perform a projection transformation before executing the action. Let the original price output by the actor be... The original demand is Define the action after projection as:

[0123]

[0124]

[0125] in, This is a reference price (it may be a fixed value or updated according to the scenario). For pricing in step S2 The optimal response of the follower. These represent the projection ratios of demand and price, respectively.

[0126] The projection ratio can be jointly scheduled using a combination of course-based scheduling and anomaly recovery:

[0127]

[0128]

[0129] in, For the training progress, Equilibrium Ratio represents the equilibrium ratio of the previous time step. To restore the threshold, To restore the enhancement magnitude, this mechanism temporarily enhances the projection intensity when the equilibrium ratio drops, in order to suppress performance drops and improve training stability.

[0130] S6. Design a neuron importance metric based on time window decay, and design an adaptive pruning threshold based on the counterfactual individual contribution, to evaluate the importance of neurons in the policy network corresponding to the policy network parameters, and obtain the neuron importance ranking and dynamic pruning threshold.

[0131] In this embodiment, the specific details of designing a neuron importance metric based on time window decay are as follows:

[0132] For a fully connected actor network, the first Layer At time step, one neuron Its current importance is:

[0133]

[0134] in, For the first Layer The first neuron and the second Layer Connection weights between neurons ( For the first Layer neuron index). For the first Layer The first neuron and the second Layer Connection weights between neurons ( For the first Layer neuron index). Here and This represents the neural network connection weights (to distinguish them from the matching probabilities in step S1). ), Indicates the width of the time window (to distinguish it from the connection weight subscript). The weights in the formula are taken from the time step. The value at time. This metric takes into account both the input and output connectivity strength of the neuron.

[0135] The importance measure of dynamically decaying neurons within a time window is:

[0136]

[0137] in, For time step Time Layer The current importance of each neuron (calculated using the formula above). The width of the time window. As the attenuation factor, For time step The binary mask (excluding historical contributions from pruned neurons). Exponent Ensure that the recent time step weight is In the long term, the weights decay exponentially over time. Early training... When the summation range is automatically shortened to .

[0138] In this embodiment, the specific details of designing a dynamic pruning threshold that adapts to the contribution of counterfactual individuals are as follows:

[0139] The dynamic pruning threshold is defined as: all neurons are ranked by importance. Sort in ascending order, take the first... The importance value of each neuron is used as a threshold. ,in This is the floor function. The total number of neurons. For the first Number of neurons in a layer. Adaptive pruning rate. Defined as:

[0140]

[0141] in, For the sensitivity hyperparameter (take When counterfactual contribution At higher levels, Reduce, thereby lowering the pruning rate. To preserve more neurons. Original progressive pruning rate. Adopting a cubic decay strategy:

[0142]

[0143] in, The initial sparsity (no pruning during the initial training phase, taking...) ), For target sparsity (taken in 300 rounds fast caliber) A longer budget could be increased to ), For the starting round of pruning (taken as caliber after 300 rounds) Long-term budgets can be moved forward to ), The total number of pruning operations (taken as 300 rounds). Longer budgets are acceptable ), Pruning frequency (every time) (Prune once per round). At that time, due to Therefore , function take Its cube is still greater than 1, therefore Subject to nonnegativity constraints, the actual value is taken That is, pruning is not performed; when hour, , The internal value becomes 0, thus Achieve the target sparsity.

[0144] Transform the counterfactual individual contribution using Jensen-Shannon divergence:

[0145]

[0146] in, It is a mixed distribution. For including drones Prior distribution of the safety situation of the formation of actions (factual scenario). To exclude drones The prior distribution of the counterfactual formation security situation of the action. For Jensen-Shannon divergence. When drones When making a key contribution to the safety of the formation ( When the contribution is high, reduce the pruning rate to retain more neurons and maintain safe decision-making ability; when the contribution is low, increase the pruning rate to remove redundant neurons. Note that... Contribution to the original counterfactual facts (calculated using KL divergence). The normalized counterfactual contribution (using JS divergence transform, with a range of values ​​when using base-2 logarithm) is... The latter is used for dynamic pruning threshold adjustment.

[0147] S7. Based on the neuron importance ranking and the dynamic pruning threshold, update the pruning binary mask and reconstruct the policy network to obtain a lightweight neural network structure.

[0148] In this embodiment, the specific steps for updating the pruned binary mask and reconstructing the neural network are as follows:

[0149] No. Layer output is ,in For the first The nonlinear activation function of the layer, For the first The weight matrix of the layer (elements are) Bold text to distinguish it from the time window width ), For the first The output of the layer, For the first The binary mask vector of the layer, This is an element-wise multiplication (Hadamard product). The mask is applied to the output of the activated neuron, setting the output of the pruned neuron to zero.

[0150] Integrating binary masks into the actor network optimizes the following problem:

[0151]

[0152] in, This represents the total number of layers in the actor network (including input, hidden, and output layers). Indicates the hidden layer index. It is the zero norm (representing the number of non-zero elements). This represents the maximum number of neurons that can be retained.

[0153] Construct a sparse regularizer for neuron importance groups and reconstruct the actor loss function:

[0154]

[0155] in, Importance ratio, Let be the pruning function (defined in step S5). The first term is the negative policy objective (minimizing the negative value is equivalent to maximizing the original value), and the second term is the negative entropy regularization term (minimizing the negative entropy is equivalent to maximizing the entropy). For the estimation of the advantage function, This is the Lagrange multiplier. Regularization term. For pruned neurons ( hour The importance of neurons is summed; if a highly important neuron is pruned, this term increases, thus penalizing mispruning and guiding network weight adjustments to reduce the neuron's importance or retain it in the next round of pruning. The mask update rule is:

[0156]

[0157] in, Indicates the removal of the first Layer One neuron, This indicates that the neuron is retained. This measures the importance of the decay of the neuron's time window. This is the dynamic pruning threshold.

[0158] S8. The network parameters in the reconstructed lightweight neural network structure are iteratively updated using the stochastic gradient method to obtain a lightweight neural network model adapted to real-time deployment on the UAV airborne platform.

[0159] In this embodiment, the specific details of iteratively updating the network parameters using the stochastic gradient method are as follows:

[0160] actor network parameter update (minimize) ):

[0161]

[0162] in, For the first Layer The first neuron and the second Layer Connection weights between neurons For the first Layer A binary mask for neurons, when When the gradient is zero, the weights of pruned neurons are no longer updated.

[0163] critic network parameter update (minimize) ):

[0164]

[0165] in, and This is the learning rate.

[0166] The training process of this invention iteratively executes steps S1 to S8, and the overall algorithm architecture is detailed below:

[0167] Algorithm 1: TinyMA-CF-PPO (Tiny Multi-Agent Counterfactual PPO) Training Algorithm

[0168] Input: Number of drones Number of controlled sectors Maximum number of training rounds

[0169] Output: Lightweight actor network parameters

[0170] Line 1: Initialize the actor network critic network CVAE parameters ,

[0171] Line 2: Initialize the binary mask (Preserve all neurons)

[0172] Line 3: Loop to

[0173] Line 4: Reset the environment and get the initial state.

[0174] Line 5: Loop to

[0175] Line 6: Controlled sectors based on observations Determine pricing strategy

[0176] Line 7: The drone, based on observations Determine airspace requirements and obstacle avoidance speed

[0177] Line 8: Perform an equilibrium prior projection on pricing and demand actions according to step S5a, to obtain... and

[0178] Line 9: Perform the projection action and obtain the next state. and rewards

[0179] Line 10: Storage Experience To playback buffer

[0180] Line 11: Calculate the counterfactual individual contribution according to step S4.

[0181] Line 12: Calculate mixed rewards

[0182] Line 13: End the inner time step loop.

[0183] Line 14: Sample batch data from the buffer to update CVAE parameters

[0184] Line 15: Calculate the GAE dominance estimate according to step S5.

[0185] Line 16: Update actor network parameters according to step S8. (with mask gradient)

[0186] Line 17: Update the critic network parameters

[0187] Line 18: If and

[0188] Line 19: Calculate neuron importance according to step S6.

[0189] Line 20: Calculate the normalized counterfactual contribution.

[0190] Line 21: Calculate the adaptive pruning rate

[0191] Line 22: Update the pruning threshold according to step S7. and binary mask

[0192] Line 23: Termination condition check

[0193] Line 24: End training round loop

[0194] Line 25: Remove the CVAE module and output a lightweight actor network.

[0195] Initialize the actor and critic networks for all drones, and initialize the CVAE parameters. For each training round, reset the environment to obtain the initial state. For each time step, the controlled sector determines the airspace pricing strategy based on observations, the drone determines its airspace usage demand and obstacle avoidance speed strategy, and executes the actions after an adjustable projection onto the Stackelberg equilibrium prior direction according to step S5a. Calculate the utility function and reward, and collect data. Samples were used for CVAE training (counterfactual scenarios involving target drones). action Set the vector to zero or keep the default hover action to construct a counterfactual action vector. The process involves calculating counterfactual individual contributions and mixed rewards, calculating neuron importance, updating CVAE parameters, updating actor and critic network parameters, updating the pruning binary mask, removing neurons with importance below a threshold, and reconstructing the lightweight network until convergence is achieved and the trained model is output.

[0196] Hyperparameter settings adopt a unified approach of "basic mode + enhanced mode": In basic mode , , , , , , , , , , , , ; and adopt , , Dispatch disabled (none), CF gating disabled, CF signal limiting disabled. Allowed in enhanced mode. , , , , To accommodate different training budgets. Action-level guidance parameters can be set. , , , , , The remaining parameters are: , , , , , round, Time step, Bonus clipping is off by default; Engineering Enhancement Mode can be optionally enabled and uses UAV / sector clipping ranges. .

[0197] The multi-UAV airspace conflict resolution and scheduling method based on counterfactual causal reasoning in this invention has the following advantages:

[0198] 1. This invention constructs a multi-leader-multi-follower Stackelberg game model, which effectively characterizes the game relationship between air traffic control centers and drone formations in terms of airspace usage rights allocation and dynamic route scheduling, and proves that the Stackelberg equilibrium exists and is unique;

[0199] 2. This invention proposes a counterfactual individual contribution measurement mechanism based on the counterfactual distribution difference degree. It models the security situation difference between factual and counterfactual scenarios through a conditional variational autoencoder, quantifies the causal impact of single-machine obstacle avoidance decisions on the overall security situation of the formation, and can be linked with strategy optimization and pruning threshold.

[0200] 3. This invention proposes a balanced prior action-level guidance and recovery enhancement mechanism, which adjustably projects pricing and demand actions toward the follower's optimal response direction during the training phase, and adaptively increases the projection intensity when the balance ratio falls back, thereby improving convergence efficiency and later-stage stability.

[0201] 4. This invention designs a dynamic pruning strategy that adapts to counterfactual contribution, achieving lightweight deployment while maintaining conflict resolution performance. After training, the CVAE module can be removed, leaving only the actor network for onboard inference.

[0202] The core technical indicators are summarized below:

[0203] index numerical values illustrate Quick training budget 300 rounds 200 time steps per round Key evaluation indicators EqRatio@100 Average of the last 100 rounds relative improvement of MAPPO Approximately 22.4%~22.6% Three average calibers Project goals achieved ≥10% Improvement relative to PPO baseline Deployable form Only keep actor Remove CVAE after training

[0204] The following is a detailed implementation process of the multi-UAV airspace conflict resolution and scheduling method based on counterfactual causal reasoning according to an embodiment of the present invention:

[0205] Consider a scenario involving dense airspace conflict resolution with 5 drones and 2 air traffic control sectors. Safe separation distance. The settings are based on the type of UAV: ​​50m for multi-rotor UAVs, 100m for fixed-wing UAVs, and 80m for unmanned helicopters. The process of resolving airspace conflicts among multiple UAVs includes: (1) Situational awareness: UAVs obtain surrounding airspace situational information through airborne sensors and ADS-B; (2) Conflict detection: Real-time calculation of distances to other UAVs to detect potential collision risks; (3) Obstacle avoidance decision: UAVs determine the direction and speed of obstacle avoidance maneuvers based on the policy network driven by counterfactual contribution; (4) Airspace allocation: Control sectors dynamically allocate airspace usage rights based on real-time conflict situation; (5) Route execution: UAVs perform obstacle avoidance maneuvers while maintaining a safe distance; (6) Status synchronization: Status information is broadcast and synchronized after the maneuver is completed.

[0206] drones Controlled sectors Matching probability Service quality parameters determined based on factors such as drone location, sector coverage, and conflict situation. And calculate the total conflict resolution delay according to the price-quality model in step S1. Including: Collision detection delay Depends on sensor refresh rate and data fusion speed; decision latency ,in The computational cost of neural network inference (unit: number of floating-point operations). Computing power of the airborne computing platform (unit: FLOPS, i.e., floating-point operations per second); maneuver latency ,in The change in velocity (unit: m / s). Maximum acceleration (unit: m / s²); State synchronization delay It depends on the quality of the communication link.

[0207] The first term of the drone utility function This represents the satisfaction level of conflict resolution based on delay constraints. The second term... This indicates the safety effect of formation coordination; multiple drones coordinating obstacle avoidance generates positive safety externalities. (Third item) Indicates the complementarity / conflictability of sector routes. Indicates complementarity (such as airspace splitting). Indicates a conflict (such as a sector boundary contention). Item 4 Airspace usage fees. Item 5. Penalties for airspace security violations, and sectors It is irrelevant, so in terms of expected utility, it is equivalent to directly subtracting the penalty term, which produces negative utility when the distance from other drones is less than the safe interval.

[0208] The revenue term of the regulated sector utility function is: airspace usage fees Subtract the cost of providing airspace services Matching probability Characterize the probability distribution of the drone selecting a controlled sector (definition see step S1). Conflict penalty term. Reflecting the responsibility of controlled sectors for airspace safety incidents, using Constraints should be imposed to prevent drones from performing redundant calculations.

[0209] The Stackelberg equilibrium is analyzed using inverse induction. The first derivative of the UAV utility function is calculated (note: airspace safety violation penalty term). and (Irrelevant, therefore the derivative is zero)

[0210]

[0211] Note: The second item corresponds to... right The derivative is The same applies to the third item.

[0212] make After sorting, we get:

[0213]

[0214] make Obtain the drone The optimal response:

[0215]

[0216] Substituting the optimal drone strategy into the utility function of the controlled sector, let This study analyzes the impact of pricing strategies for regulated sectors on utility. Because... yes The decreasing function means that regulatory sectors need to balance the revenue from airspace pricing with the decline in demand for drones. Through analysis of... By taking the first derivative and setting it to zero, we obtain the optimal airspace pricing strategy for the regulated sector. We verify that the second derivative is strictly negative, ensuring that the leader's utility function is strictly concave with respect to the pricing strategy. This confirms the existence and uniqueness of the leader-level Nash equilibrium, and consequently, the existence and uniqueness of the Stackelberg equilibrium.

[0217] CVAE consists of two encoders, two prior networks, and one decoder. Encoder 1 takes the current state as input. All drone actions Next state Output potential security situation variables posterior distribution (Real-world scenario). Encoder 2 inputs the current state. Except for drones Other drone actions Next state Output the counterfactual posterior distribution (Counterfactual scenario). Prior network 1 output. Prior network 2 output Decoder input latent variables Output reconstructed security posture .

[0218] CVAE training maximizes the variational lower bound, including the KL divergence terms of the two encoders and their corresponding prior distributions, and the reconstruction error term of the decoder (based on latent variables sampled from the two encoders, respectively). After training, the learned prior network is used to estimate the prior distribution of the factual scene. and prior distribution of counterfactual scenarios Calculate the individual contribution of counterfactual facts This contribution quantifies the capabilities of drones. The causal impact of obstacle avoidance maneuvers on the overall safety posture of the formation.

[0219] A dynamic decay mechanism within the time window assigns higher weights to recent time steps and exponentially decays the weights of longer time steps. This is achieved by multiplying by the binary mask of each time step. This ensures that the historical contributions of pruned neurons are not included in the importance assessment.

[0220] Adaptive pruning strategy: When the counterfactual contribution of a UAV is high (indicating that the UAV makes a key contribution to formation safety), the pruning rate is reduced to retain more neurons to maintain the ability to express safety decisions; when the counterfactual contribution is low, the pruning rate is increased to remove redundant neurons and reduce computational overhead. The JS divergence is symmetric, and its range is determined by using a base-2 logarithm. It is suitable as a normalized counterfactual contribution indicator.

[0221] At the start of training, the binary mask is initialized to 1, retaining all neurons. As training progresses, the binary mask is progressively updated based on neuron importance metrics and a dynamic pruning threshold. The Lagrange multiplier method is used to transform constrained optimization into unconstrained optimization. After calculating the pruning threshold, neurons are sorted in ascending order of importance; neurons with importance below the threshold are removed (mask set to 0), while those with importance above the threshold are retained (mask set to 1). When a neuron is removed, all weights connected to it are also removed.

[0222] Experiments were conducted on the NVIDIA Jetson Orin Nano UAV-borne heterogeneous computing platform, using Ubuntu 22.04, PyTorch 2.3.0 framework, and CUDA 12.2 acceleration. Network architecture settings: The actor network is a 3-layer fully connected network with 256-128-64 neurons in each layer, using ReLU activation function, and Tanh mapping to bounded space in the output layer; the critic network has the same structure as the actor network, with no activation function in the output layer; the CVAE encoder is a 2-layer fully connected network (128-64 neurons), with latent variables... The decoder is a two-layer fully connected network (64-128 neurons). The example uses 200 time steps per round, a batch size of 64, and an experience replay buffer capacity of [missing information]. and support to Round training budget; reward pruning is off by default, but can be turned off if necessary. Enable stability protection for the range.

[0223] Under a rapid training caliber of 300 rounds, a stable high balance ratio can be achieved by employing a joint strategy of "counterfactual reward + adaptive pruning + balanced prior action guidance". Using the average balance ratio of the most recent 100 rounds (EqRatio@100) as the evaluation metric, the MAPPO baseline is approximately 1.188, 1.221, and 1.237 under three random seeds; the method of this invention can achieve approximately 1.493, 1.483, and 1.493 under the corresponding seeds, with an average relative improvement of approximately 22.6%, and meets the engineering objective of "improvement of more than 10% relative to the PPO baseline".

[0224] The pruning strategy can be applied with varying intensities under different training budgets: in the 300-round fast convergence mode, sparsity is typically controlled to a light to moderate level to reduce fluctuations in the later stages; under longer training budgets, the target sparsity can be further increased. After training, the CVAE counterfactual reasoning module can be removed, retaining only the lightweight actor network for airborne deployment. This invention significantly reduces deployment complexity while maintaining conflict resolution performance, making it suitable for resource-constrained UAV platforms.

[0225] The method of this invention is compared with the MAPPO baseline (EqRatio@100).

[0226] algorithm seed42 seed43 seed44 Three seed averages relative improvement of MAPPO MAPPO baseline 1.188 1.221 1.237 1.216 - This invention (basic high-performance mode) 1.493 1.483 1.493 1.490 +22.6% This invention (including restored projection enhancement) 1.482 1.489 1.494 1.488 +22.4%

[0227] Significant differences exist in the distribution of counterfactual contributions from various UAVs across different conflict scenarios, validating the ability of the counterfactual causal reasoning mechanism to distinguish key obstacle avoidance behaviors. Ablation analysis reveals that removing the equilibrium prior action guidance easily degenerates the model to near MAPPO levels; removing counterfactual rewards or adaptive pruning both decrease performance and stability; retaining only the equilibrium recovery mechanism can reduce the number of late-stage descents in some seeds. This indicates that the counterfactual contribution modeling, equilibrium prior guidance, and adaptive pruning in this invention are complementary.

[0228] Extended application scenarios: The method of this invention can be extended to the following scenarios: (1) Urban Air Mobility (UAM): resolving airspace conflicts of new aircraft such as air taxis and eVTOL (electric vertical take-off and landing aircraft); (2) Unmanned aerial vehicle (UAV) logistics delivery: route coordination and obstacle avoidance for multi-UAV express delivery; (3) Agricultural plant protection UAV formation: multi-aircraft collaboration and safe spacing maintenance for large-scale plant protection operations; (4) Emergency rescue UAV: ​​dynamic route planning for multi-UAV collaborative search and rescue in disaster scenarios.

[0229] The following table compares and analyzes the multi-UAV airspace conflict resolution and scheduling method based on counterfactual causal reasoning of the present invention with existing technical solutions. In the table, QMIX is Q-value Mixing Network, MAPPO is Multi-Agent Proximal Policy Optimization, COMA is Counterfactual Multi-Agent Policy Gradients, and CVAE is Conditional Variational Autoencoder.

[0230] Comparative analysis of existing technologies

[0231]

[0232] This invention also provides a multi-UAV airspace conflict resolution and scheduling system based on counterfactual causal reasoning, used to implement the above-mentioned multi-UAV airspace conflict resolution and scheduling method based on counterfactual causal reasoning, including:

[0233] The game modeling module is used to construct a Stackelberg game model based on steps S1 and S2 and calculate the equilibrium strategy, outputting the optimal airspace use demand strategy for drones and the optimal airspace pricing strategy for controlled sectors.

[0234] The POMDP modeling module, connected to the game modeling module, is used to construct a partially observable Markov decision process model according to step S3, receive the environmental state, and use the equilibrium strategy output by the game modeling module to define the state space, action space, and reward function, and output agent observations.

[0235] The counterfactual causal reasoning module, connected to the POMDP modeling module, includes a fact encoder, a counterfactual encoder, a fact prior network, a counterfactual prior network, and a security posture decoder. It is used to calculate the counterfactual individual contribution based on step S4 and combine the counterfactual individual contribution with external rewards to form a hybrid reward output.

[0236] The strategy network module, connected to the counterfactual causal reasoning module, includes an actor network and a critic network, and is used to optimize the strategy based on the hybrid reward in step S5 and output an obstacle avoidance decision.

[0237] An adaptive pruning module, connected to the policy network module, is used to dynamically adjust the pruning threshold and update the binary mask based on the neuron importance metric according to steps S6 and S7, and to prune and reconstruct the actor network in the policy network module.

[0238] The parameter update module, connected to the policy network module and the adaptive pruning module, is used to iteratively update the pruned and reconstructed actor network parameters and critic network parameters according to step S8, so as to obtain a lightweight neural network model that is adapted to the real-time deployment of the UAV airborne platform.

[0239] Furthermore, the system is deployed on an airborne heterogeneous computing platform of an unmanned aerial vehicle (UAV) and runs in a deep learning inference environment that supports GPU acceleration. The lightweight actor network performs online decision-making under the condition of satisfying the real-time constraint of airspace conflict resolution.

[0240] Furthermore, the actor network of the policy network module is a 3-layer fully connected neural network with 256-128-64 neurons in each layer. The hidden layer activation function is ReLU, and the output layer uses the Tanh activation function to map the obstacle avoidance action to the bounded space. The lightweight actor network is obtained through adaptive pruning in steps S6 and S7.

[0241] Furthermore, the counterfactual causal reasoning module is only used during the training phase. After training is completed, the counterfactual causal reasoning module (CVAE module, approximately 0.13M parameters) is removed, and only the lightweight actor network is retained for actual deployment.

[0242] The above is a detailed description of the preferred embodiments of the present invention. However, the present invention is not limited to the embodiments described. Those skilled in the art can make various equivalent modifications or substitutions without departing from the spirit of the present invention. All such equivalent modifications or substitutions are included within the scope defined by the claims of this application.

Claims

1. A method for resolving and scheduling airspace conflicts among multiple unmanned aerial vehicles (UAVs) based on counterfactual causal reasoning, characterized in that: Includes the following steps: S1. Construct a multi-leader-multi-follower Stackelberg game model, define the drone utility function and the air traffic control sector utility function. The drone utility function includes an airspace safety violation penalty term, and the control sector utility function includes an airspace conflict penalty term. The game relationship between airspace pricing strategy and airspace use demand strategy is obtained through the Stackelberg game model. S2. Using backward induction, the Stackelberg game model is analyzed for equilibrium to find the optimal airspace use demand strategy for UAVs and the optimal airspace pricing strategy for controlled sectors, thus obtaining the Stackelberg equilibrium solution. S3. The Stackelberg game model is modeled as a partially observable Markov decision process. The global state space, action space and reward function are defined using the Stackelberg equilibrium solution to obtain a mathematical description of the multi-agent decision process. S4. Within the framework of the partially observable Markov decision process, a counterfactual individual contribution measurement mechanism based on the counterfactual distribution difference is introduced. A conditional variational autoencoder is used to estimate the difference in the security situation distribution between the factual scenario and the counterfactual scenario, quantify the causal impact of single-drone obstacle avoidance actions on the overall security situation of the formation, and obtain the counterfactual individual contribution of each UAV. The counterfactual individual contribution is defined as follows: in, For Kullback-Leibler divergence, As a potential security situation variable, For time step The global state, For the motion vectors of all drones, To eliminate drones The motion vectors of other drones, For including drones Prior distribution of the formation safety situation of the action. To rule out drones The counterfactual security posture prior distribution of actions combines the individual contribution of counterfactual actions with external rewards to form a hybrid reward system. ,in As a reward for the external environment, This is a hyperparameter for intrinsic reward weighting; S5. An improved multi-agent proximal policy optimization algorithm is adopted, incorporating the counterfactual individual contribution as an intrinsic reward into the policy optimization objective, and performing policy optimization on the actions in the action space to obtain optimized policy network parameters. S6. Design a neuron importance metric based on time window decay, and design an adaptive pruning threshold based on the counterfactual individual contribution, evaluate the importance of neurons in the policy network corresponding to the policy network parameters, and obtain the neuron importance ranking and dynamic pruning threshold. The neuron importance metric based on time window decay is: in, For time step Time Layer The current importance of each neuron. For the first Layer The first neuron and the second Layer Connection weights between neurons For the first Layer The first neuron and the second Layer Connection weights between neurons and These are the indices of neurons in the previous and next layers, respectively. As the attenuation factor, For time step The binary mask, The width of the time window; S7. Based on the neuron importance ranking and the dynamic pruning threshold, update the pruning binary mask and reconstruct the policy network to obtain a lightweight neural network structure. S8. The network parameters in the reconstructed lightweight neural network structure are iteratively updated using the stochastic gradient method to obtain a lightweight neural network model adapted to real-time deployment on the UAV airborne platform.

2. The method for resolving and scheduling multi-UAV airspace conflicts based on counterfactual causal reasoning according to claim 1, characterized in that, In step S1, the utility function of the UAV is: The utility function of the controlled sector is: in, For drone swarm assembly, For the control sector set, For drones The spatial domain uses the demand vector. Vector of airspace usage needs for other drones, For the pricing strategy vector of the regulated sector, To remove sectors Pricing strategy vectors for other regulated sectors A matrix of airspace usage requirements for all drones. For drones To sector The requested airspace usage weight, For drones Select sector The matching probability, For sectors Service quality parameters For sectors airspace usage price, The task satisfaction coefficient. The base of the natural constant is... The delay sensitivity coefficient, To minimize the delay in conflict resolution, For formation coordination safety factors, For sector route complementarity parameters, The penalty coefficient for airspace safety violations. For sectors For drones The cost of providing services The sector airspace conflict penalty coefficient, For drones With drones Real-time distance between them For safe interval distance, For indicator functions, in the conflict penalty term Constraints are used to avoid conflicting and redundant calculations between drone pairs.

3. The method for resolving and scheduling airspace conflicts among multiple unmanned aerial vehicles based on counterfactual causal reasoning according to claim 2, characterized in that, In step S5, the loss function of the improved multi-agent proximal policy optimization algorithm is: in, The objective function for the actor strategy iteration. For the critic loss function, For actor network parameters, These are the parameters of the critic network. For policy entropy, For entropy regularization weights, For the critic loss weight, Sigmoid annealing weights are used , For annealing rate, This represents the current number of training steps. For annealing center steps, counterfactual individual contribution is incorporated into the critic value assessment through a hybrid reward system.

4. The method for resolving and scheduling multi-UAV airspace conflicts based on counterfactual causal reasoning according to claim 3, characterized in that, The adaptive pruning threshold in step S6 is: all neurons are ranked by importance. Sort in ascending order, take the first... The importance value of each neuron is used as a threshold. ,in, This is the floor function. The total number of neurons. To adapt to the pruning rate, For sensitivity hyperparameters, For the normalized counterfactual contribution, among which For the prior distribution of security posture in real-world scenarios, For the prior distribution of security posture in counterfactual scenarios, It is a mixed distribution. The Jensen-Shannon divergence, when using the base-2 logarithm, has a range of values ​​of [value missing]. , The original progressive pruning rate, of which The initial sparsity, For target sparsity, This is the starting round for pruning. This represents the total number of pruning sessions. For pruning frequency, when the counterfactual contribution of drones is high Reduce the pruning rate to decrease the counterfactual contribution, and increase the pruning rate when the counterfactual contribution is low.

5. The method for resolving and scheduling multi-UAV airspace conflicts based on counterfactual causal reasoning according to claim 4, characterized in that, In step S3, the state transition function of the partially observable Markov decision process is determined by the UAV kinematic model, specifically as follows: in, For drones At time step The position vector, For drones At time step The position vector, For drones The obstacle avoidance velocity vector, For the time step, take s, It is the Euclidean norm.

6. The method for resolving and scheduling multi-UAV airspace conflicts based on counterfactual causal reasoning according to claim 5, characterized in that, After step S5 and before step S6, step S5a is also included: before the action is executed, an equilibrium prior action-level guidance is introduced, the pricing action of the control sector and the demand action of the UAV are projected onto the direction of the UAV's optimal airspace use demand policy in the Stackelberg equilibrium solution according to the projection ratio, and when the equilibrium ratio is lower than the threshold, recovery enhancement is triggered, and the projected action is used as the actual execution action for policy optimization by the improved multi-agent proximal policy optimization algorithm.

7. The method for resolving and scheduling airspace conflicts among multiple unmanned aerial vehicles based on counterfactual causal reasoning according to claim 6, characterized in that, In step S5a, the action-level guidance satisfies: in, and These are the original pricing action and the original demand action output by the strategy network, respectively. and These are the actions after projection. For reference pricing, For the follower's optimal response under projected pricing, This is the projection scale.

8. The method for resolving and scheduling airspace conflicts among multiple unmanned aerial vehicles based on counterfactual causal reasoning according to claim 7, characterized in that, In step S7, when reconstructing the policy network, the Lagrange multiplier method is used to transform the constrained optimization problem of retaining the number of neurons into an unconstrained optimization problem. The reconstructed actor loss function is: in, Importance ratio, For the clipping function, For generalized advantage estimation, For Lagrange multipliers, regularization terms The importance of pruned neurons is summed to punish mispruning behavior.

Images