A weak communication unmanned aerial vehicle cluster pursuit and evasion decision-making method and system based on trajectory prediction and unified efficient game

Abstract

The application belongs to the technical field of unmanned aerial vehicle cluster autonomous decision, multi-agent reinforcement learning, experience game analysis and trajectory prediction cross technology, and particularly relates to a weak communication unmanned aerial vehicle cluster pursuit and evasion decision method and system based on trajectory prediction and unified efficient game, the method is divided into a centralized training stage and a decentralized decision stage, wherein the centralized training stage reconstructs missing flight paths through recursive polynomial fitting and calculates analytical uncertainty, generates fixed dimension belief state irrelevant to cluster size by using double flow independent permutation invariant coding, constructs a factorized mixed action strategy constrained by legal targets, combines confidence weighted UEP heuristic regularization and parameter sharing HAPPO update strategy, and iteratively updates the strategy pool based on zero-sum meta-game with no reshaping benefits and double threshold rules. The application can realize decentralized autonomous decision under weak communication and quantity asymmetric conditions, while considering training stability and confrontation performance.

Description

Technical Field

[0001] This invention belongs to the technical fields of autonomous decision-making in UAV swarms, multi-agent reinforcement learning, empirical game analysis, and trajectory prediction. Specifically, it relates to a weak-communication UAV swarm pursuit and escape decision-making method and system based on trajectory prediction and unified efficient game theory. Background Technology

[0002] Drone swarm pursuit is a typical multi-agent adversarial task, with significant application value in scenarios such as airspace surveillance, target interception, and coordinated penetration. In actual missions, drone swarms often face challenges such as limited communication distance, obstructed lines of sight, and intermittent loss of messages from teammates due to electromagnetic interference. Meanwhile, the adversary can only observe locally through onboard sensors, and the number of drones on both sides is unequal, with the number of nodes potentially changing dynamically with battle losses. This places extremely high demands on the robustness, adaptability, and distributed execution capability of decision-making algorithms.

[0003] A mainstream approach to swarm pursuit decision-making relies on multi-agent deep reinforcement learning. This typically models the task as a multi-agent Markov decision process, employing algorithms like MAPPO and MADDPG to learn cooperative strategies through centralized training and distributed execution. Some approaches introduce graph neural networks to aggregate neighbor states to enhance cooperation. For example, related literature proposes a multi-UAV pursuit game method based on online motion planning, improving decision-making performance in adversarial scenarios. However, these approaches rely on current observation information and lack interpretable compensation mechanisms for target states experiencing communication loss or sensor loss. This can lead to misinterpreting communication interruptions as target disappearance, resulting in a sharp performance drop in weak communication scenarios. Furthermore, the policy network input uses fixed-dimensional concatenation, relying on a preset number of UAVs. When the number of nodes changes due to losses or reinforcements, the network structure needs modification and retraining, making it unsuitable for scenarios with asymmetric numbers and dynamic scales. Additionally, this approach struggles to simultaneously output discrete tactical patterns and continuous maneuver commands; simple action concatenation can lead to the invalidation of action validity constraints, affecting training stability.

[0004] Another approach combines communication enhancement, trajectory prediction, and game theory strategies to improve decision-making performance: it compensates for missing information through anti-interference communication allocation and target trajectory prediction, and then combines empirical game analysis and strategy pool methods to enhance adversarial diversity. However, this type of technical solution does not distinguish between the different mechanisms of teammate communication failure and opponent sensor failure, treating unreliable information and reliable information equally, and the credibility of information cannot be perceived by the strategy; learning-based trajectory prediction requires additional training data and independent network parameters, making airborne online deployment highly complex, and the prediction uncertainty cannot be dynamically matched with the duration of disconnection and the quality of fitting; the diversity training of the game theory strategy pool does not consider the reliability of information, and still forcibly amplifies the strategy differences when the state reconstruction is unreliable, which can easily amplify observation errors and cause training fluctuations; at the same time, most game theory methods rely on the global real state or a pre-set alliance structure, and cannot be directly applied to decentralized execution scenarios that only rely on local observation.

[0005] In summary, existing drone swarm pursuit decision-making technologies cannot simultaneously meet the requirements of robustness in weak communication, dynamic scale adaptability, legality of mixed actions, and diversity of adversaries. There is an urgent need for a solution that can achieve decentralized autonomous decision-making under conditions of weak communication and asymmetric numbers, while taking into account training stability and adversarial performance. Summary of the Invention

[0006] The purpose of this invention is to provide a decision-making method and system for pursuing and escaping unmanned aerial vehicle (UAV) swarms based on trajectory prediction and unified efficient game theory. This method enables decentralized autonomous decision-making under conditions of weak communication and asymmetric numbers, while also taking into account training stability and adversarial performance.

[0007] The specific technical solution adopted by this invention is as follows: A decision-making method for pursuing and escaping unmanned aerial vehicle (UAV) swarms with weak communication based on trajectory prediction and unified efficient game theory consists of five coupled layers: weak information pursuit and escape environment, weak information belief state construction, confidence-weighted UEP heuristic regular approximate optimal response, target perception parameter sharing HAPPO, and non-shaping gain empirical meta-game theory. During the offline centralized training phase, the weak communication UAV swarm pursuit decision method employs five coupled layers: weak information pursuit environment, weak information belief state construction, confidence-weighted UEP heuristic regular approximate optimal response, target perception parameter sharing HAPPO, and non-shaping gain empirical meta-game. During the decentralized decision-making phase, the weak communication UAV swarm pursuit decision-making method only enables the forward inference function of the two layers of weak information pursuit environment and weak information belief state construction, and the target perception parameter sharing HAPPO layer. It disables the training and update functions of the two layers of confidence-weighted UEP heuristic regular approximate optimal response and non-shaping gain empirical meta-game, as well as the target perception parameter sharing HAPPO layer.

[0008] Specifically, the following steps are included: Step S1: Modeling the environment and weak communication of pursuit by asymmetric fixed-wing UAVs For ease of understanding, the following is stated in this instruction manual: Represents a discrete environment moment or time step; An index representing the drone currently being observed or making a decision. Indicated by drones Maintained teammate or opponent index; Indicates the number of updates to the outer policy pool; Represents the faction index, where Indicates the escapee, Indicates the pursuing party; This represents the finite time-domain length of a single round. For simplicity and to avoid ambiguity, the explicit time parameters of some state variables are used. This can be omitted. and These represent the ensemble of escaping drones and the ensemble of pursuing drones, respectively.

[0009] Let the set of drones be ,in and These represent the pursuing and fugitive sides, respectively. (Any drone) The state of a three-degree-of-freedom particle is , in The three-dimensional position in the inertial coordinate system. Airspeed, For the inclination angle of the flight path, For the heading angle. The continuous control quantity is... , These correspond to longitudinal acceleration control, normal overload, and roll angle, respectively.

[0010] Fixed-wing kinematics adopts ; In the above fixed-wing kinematic equations, the variables above... This indicates taking the derivative with respect to time; It represents the acceleration due to gravity. and These represent the minimum and maximum permissible airspeeds, respectively. This indicates the upper limit of the absolute value of the roll angle. Indicates the upper limit of normal overload. This represents the set of physically feasible states that satisfy the above flight envelope constraints.

[0011] The flight conditions satisfy the speed, roll angle, and overload boundaries, for example... , , Predicted states that exceed the boundary are mapped back to the physically feasible set via a projection operator. .

[0012] Each drone's hybrid actions consist of tactical modes, adversary targets, and continuous maneuver commands: , in The target selection set is the union of the opponent's index set and the empty target set; continuous actions. Then, the physical control quantity is obtained through linear mapping.

[0013] This invention models teammate communication and opponent sensing separately. For teammates... The physical communication adjacency relationship is ; In the formula, express Time Drone With drones Spatial distance between them; Indicates the maximum effective communication radius between teammates; The line-of-sight distance between two aircraft can be indicated by an indicative quantity. This indicates that there is an effective line of sight; This indicates the physical communication adjacency indicator, which is determined by both communication distance and line-of-sight conditions.

[0014] Sudden packet loss is handled using a two-state Gilbert-Elliott channel, with the transition matrix being: . In the Gilbert-Elliott channel and These represent the two channel states: "Good" and "Bad," respectively. Indicates that the channel is State transition The probability of a state. Indicates by State transition The probability of a state; and They respectively indicate that they are kept in status and remain The probability of a state; Indicates order of states The two-state transition probability matrix is ​​obtained by arranging the states.

[0015] make The effective teammate communication mask indicates whether the data packet is available. .

[0016] For the opponent Constructed from sensing distance, line of sight, and detection probability . In the formula, This indicates an indicator function; it takes the value 1 if all conditions within the parentheses are met, and 0 otherwise. Indicates the effective sensing range of the airborne sensor; express Time Drone against the opponent The instantaneous detection probability; This represents the minimum detection probability threshold required to classify a detection result as a valid track. This indicates that the detection probability is not lower than the minimum confidence threshold; This indicates that the opponent can perceive the available mask.

[0017] A mask can be used to further define the unified information. ; Teammate communication availability Perceived availability of the opponent They will be counted separately, and will not be confused as the same signal in the future.

[0018] This task is modeled as a weakly informed, distributed, partially observable Markov decision process. During the centralized training phase, team-level value networks and team-level shaping variables are allowed; during the distributed execution phase, the individual policy uses only the local state, teammate messages, locally maintained adversary tracks, and trajectory prediction compensation states.

[0019] Step S2: State normalization and recursive track maintenance The dimensions of components in different states vary considerably. Firstly, the mean of the training rolling data is used... and diagonal scaling matrix Normalization of the state: . The angular components are expanded within each fitting window, predicted, and then mapped to the principal value interval.

[0020] Observer For each teammate or opponent Maintaining normalized flight paths and track age : , In the formula, Represents the observer Locally maintained drones Normalized track status; This refers to the currently directly obtained and normalized drones. state; This indicates that when the current information is unavailable, the observer... Based on local historical windows for drones The normalized predicted state is obtained by polynomial extrapolation. Therefore, Direct observation was used at that time. Local prediction compensation is used at that time.

[0021] ; When a message is received again or a target is detected again, the track age is reset to zero and the direct observation is written into the history; when there is continuous loss of contact, the polynomial estimate of the previous moment is written back to the sliding window as the current local track sample, and then the updated history is used to refit the track. As an explicit expiration marker, expired tracks cannot participate in target selection and can only enter the set encoder via shared empty markers.

[0022] Length is The local history records simultaneously store the state, available mask, and normalized track age. The entire recursive process does not insert real-world states invisible to the simulator, ensuring auditable information boundaries during execution.

[0023] Step S3: Adaptive sliding window weighted polynomial reconstruction Let the current number of valid historical samples be The polynomial order is adaptively chosen as . In the formula, Represents the observer For flight path exist The order of the polynomial is adaptively selected at any given time. This indicates the number of valid historical samples within the current sliding window; This indicates the maximum allowed order of the lower-order polynomial. This ensures... The number of effective samples should not exceed 1 minus the number of effective samples to avoid underfitting when there are insufficient historical samples.

[0024] When there is only one sample, it degenerates into zero-order preservation; when there are no valid samples, an empty label is used. For valid historical moments... Define dimensionless local time and basis vectors . Direct observations are assigned unit weights, and recursive predictions are used to predict sample decay based on track age. . In the formula, Representing historical moments The weights of the corresponding samples in the weighted polynomial fitting; Represents the basic weights of the recursive prediction samples; The rate coefficient that decreases with increasing track age; This indicates the age of the track corresponding to the sample; This indicates the maximum valid track age allowed for prediction. Direct observation sample. The time weight is 1, and the recursive prediction samples decay exponentially according to the track age.

[0025] Stack the basis vectors into a Vandermonde matrix. Stacking historical states into The weight matrix is and set Online solution for weighted ridge polynomial coefficients: . At the outreach point Normalized predictions are obtained And restore to the physical state: . In the formula, Represents the set of physically feasible states The deterministic projection operator is used to perform amplitude limiting or angle wrapping on the extrapolated state components such as velocity, angle and overload, so that the prediction results satisfy the fixed-wing flight envelope. and These are used to restore the normalized prediction to the original physical quantity scale.

[0026] The polynomial coefficients are solved analytically each time based on the current local window. They are not part of the trainable neural network parameters and do not need to be independently predicted by the network, labels, or loss.

[0027] This invention does not simply extrapolate using the last state, but recursively writes the prediction result from the previous step back into the local history, and then refits the updated sliding window; at the same time, it explicitly controls the prediction availability period through track age and expiration flags. This process is entirely based on locally available information, without reading invisible real states, and can achieve recursive polynomial track reconstruction under conditions of continuous loss of contact.

[0028] Step S4: Analyzing Uncertainty Based on Residuals, Leverage, and Track Age Regarding the first For each state component, the error scale is calculated based on the weighted fitting residual: . In the residual calculation formula, Indicates local flight path The One state component; Represents the polynomial coefficient matrix The List; This indicates the use of the current polynomial model to calculate historical moments. First The fitted values ​​obtained for each state component; the difference between the two is the fitting residual for that component. Use it as a weighted error scale.

[0029] The leverage of the extrapolation point is . In the leverage calculation formula, The ridge regularization coefficients represent the weighted ridge polynomial fitting, used to improve the numerical conditions when inverting a matrix. represents a ridge regular matrix, where constant terms are not penalized; Represents the polynomial basis vector of the current extrapolation point; This indicates the leverage of the extrapolation point; the larger the value, the less stable the extrapolation conditions.

[0030] The component reconstruction scale is obtained by combining residuals, leverage, and track age. Then, by averaging across all dimensions and truncating, the scalar uncertainty is obtained. When recent motion is difficult to fit, matrix extrapolation conditions worsen, or the duration of continuous disconnection increases, Automatically increases in size.

[0031] The compensation state actually used by the observer is . The average uncertainty of the machine is obtained by averaging the uncertainties of all currently unavailable tracks. This quantity is an analytical reliability metric in the normalized state space and is not claimed to be a rigorously calibrated probability confidence interval.

[0032] This invention utilizes the joint analysis of uncertainty by residual-leverage-track age. Specifically, it does not train an additional uncertainty network, but instead constructs an interpretable reliability index using the current weighted fitting residual, extrapolation point leverage, and continuous disconnection duration, and applies this index to both belief state and UEP canonical intensity modulation.

[0033] Step S5: Teammate-Opponent Dual-Stream Variable Cardinality Set Encoder For the observer Establish separate teammate groups Gather with the opponent Each object tag includes relative compensation state, information availability, reconstruction uncertainty, normalized track age, and role embedding: . Teammates and opponents each use a shared encoder , Obtain object embedding, native state via Get the local query vector Perform masked multi-head attention on both sets using a single native query: . The final fixed-dimensional belief state is . This invention does not use unique agent ID embedding or sequence position encoding; it shares an object encoder within the same stream, removes illegal items only through binary masks, and aggregates sets through single-query attention or symmetric pooling. Therefore, arbitrarily changing the input order of teammates or opponents does not change the belief state, and the input dimension of the policy network does not change with the cluster size.

[0034] This invention implements permutation-invariant dual-stream coding that separates the communication stream and the perception stream. Specifically, teammate status comes from communication, while opponent status comes from onboard perception, and the two types of information have different failure mechanisms. This invention encodes and calculates availability separately, without using unique agent IDs or sequential position coding, and structurally supports input object rearrangement and changes in the number of nodes.

[0035] Step S6: Local target effectiveness, tactical mode, and masked target selection The global target survival flag of the simulator is not read during the execution phase. The observer maintains an "undetermined destruction" indicator for each adversary. A target is marked as destroyed only when the effectiveness of the weapon is confirmed or a certified teammate reports it; simply losing awareness of a target will not mark it as destroyed.

[0036] The set of effective adversary tracks is . Construct a locking set based on distance, line-of-sight angle, previous lock-on status, and remaining ammunition. With firing assembly In cruise mode, the target is fixed to empty; in lock-on mode, only targets from... Select; firing mode is only allowed from Select. If the set of legitimate targets for a non-cruise mode is empty, then that mode is directly masked before sampling in the tactical mode.

[0037] The target pointer is calculated for each legitimate opponent. , In the target pointer format, and These represent the trainable query projection matrix and key projection matrix, respectively, used to project local fixed-dimensional belief states. Embedding with adversary objects Mapped to the same attention space; This represents the dimension of the projected key vector. Used for scale normalization; Indicates a legitimate target Unnormalized target selection score.

[0038] Softmax is performed only on the valid subset corresponding to the pattern. Set aggregation remains unchanged with respect to the input permutation, while target pointers are equally variable with respect to the opponent permutation. That is, changing the opponent's input order will only change the target probability permutation in the same way, without changing the probability value of any physical target.

[0039] Step S7: Target perception parameter sharing hybrid action strategy For homogeneous drones belonging to the same party, a team-shared policy network is used during deployment. Discrete branches are further decomposed into pattern distribution and conditional objective distribution, with the complete hybrid policy being... . The continuous branches are subjected to a diagonal Gaussian distribution followed by a bounded compression transformation. The joint logarithmic probability equals the pattern distribution. Conditional target distribution and continuous action distribution The sum of logarithmic probabilities; the mixed entropy is the weighted sum of the entropies of the three. This structure clearly distinguishes between "which tactical mode to adopt," "which opponent the tactics are applied to," and "how to maneuver continuously."

[0040] This invention implements a three-level factorized hybrid action of mode-target-continuous control. Specifically, it first masks the non-executable lock or fire modes based on the set of legal targets, then performs target pointer selection on the set of legal mode conditions, and finally outputs continuous maneuvering. During training, the sampling mask is used to calculate the probabilities of the old and new strategies to avoid illegal actions and changes in the probability support set from polluting the importance sampling ratio.

[0041] Meanwhile, this invention implements parameter sharing HAPPO and variable-scale deployment interface. Specifically, homogeneous drones share actor parameters, and during training, they are updated sequentially according to subsets of agent indexes. During deployment, the same strategy can be replicated to different numbers of drones. The dual-stream ensemble encoder decouples the network input dimension from the cluster size.

[0042] Step S8: Centralized Value Network and Parameter Sharing HAPPO Update During the intensive training phase, each side uses a team-level feature obtained by permutation-invariant pooling of the belief states of the active agents. and by value network Estimate the returns. The generalized advantage is estimated as follows: , . In the above generalized advantage estimation, Indicates training camp exist Instant training rewards used for updating candidate policies at all times; Indicates the discount factor; The trace parameter represents the generalized advantage estimation; This represents the cumulative number of steps taken from the current moment forward. This represents the one-step time-series difference residual. This indicates that multiple time-series difference residuals are calculated according to... The generalized advantage estimate obtained by weighted summation; For camp Centralized value networks for team-level characteristics Value estimate.

[0043] During rolling sampling, the valid pattern mask and the pattern condition target mask are stored together with the action. During the optimization phase, both the new and old strategies use the same set of stored masks to calculate probabilities, ensuring a consistent domain for the importance sampling ratio.

[0044] For the current subset of samples divided by agent index, the probability ratio of mixed actions is . In the probability ratio of mixed actions, , and These represent the likelihood ratios of the new strategy probabilities relative to the old strategy probabilities in the tactical mode branch, conditional target selection branch, and continuous maneuver branch, respectively; the product of these three probabilities... The joint importance sampling ratio that constitutes the current hybrid action.

[0045] Perform HAPPO-style sequential sub-updates on a randomly arranged subset of agents, using the cumulative correction coefficients formed by previous sub-updates. The target of the cropping is... . In the cropping target, The importance of PPO / HAPPO is indicated by the sampling ratio and the clipping radius. This means limiting the probability ratio to 1. Within the range; This represents the cumulative correction coefficient formed by the previously updated subset of agents in the current HAPPO sequential update; This represents the expectation of the rolling sample time step.

[0046] The total loss consists of the policy loss, value loss, and mixture entropy regularization for each agent subset. After each update, the probability of the shared policy for the unprocessed subset is recalculated. This implementation is a HAPPO-style update adapted to shared parameters, without directly extending the monotonic improvement guarantee of the completely independent policy case to the shared parameter case.

[0047] Step S9: No-Shaping Turn Gains, Experience Element Game, and Strategy Pooling Candidate policy training and policy pool evaluation are separated. The environment-native reward includes components such as the first neutralization event, player losses, radar exposure, mission completion, and existing smoothness in the environment. Empirical meta-games only use turn payoffs without UEP intrinsic terms. . In the formula, This represents the escapee's intrinsic shaping round gain without UEP over a full round. This indicates the turn-based benefit for the pursuing side; express The immediate, unshaped environmental reward is directly returned to the fleeing player by the environment. When using a zero-sum reward definition, the pursuer's immediate reward is opposite to the fleeing player's. .

[0048] No. The policy pools of both sides in the next outer iteration are respectively , Estimate the payoff matrix for all strategy pairs using a uniform seed set and evaluation protocol. . In the payoff matrix, Indicates the first The strategy pair payoff matrix constructed in the next outer iteration is based on the escaper's payoff. Represents the payoff matrix for the pursuer, and ; Indicates the first in the escape strategy pool One strategy, Indicates the first strategy in the pursuing side's strategy pool A strategy. Symbols This refers to the decision-making strategy of drone swarms based on the current belief state and the probability distribution of mixed actions.

[0049] Solve finite zero-sum matrix games using linear programming to obtain the meta-strategies of both players. , Game value . The candidate strategy improves upon the current opponent's hybrid unshaping payoff by: Simultaneously, policy fingerprints are computed using a jointly frozen embedding encoder and a common reference dataset, with the minimum fingerprint distance between a candidate and the existing policy pool used as novelty. Only when ; In the criteria for inclusion in the pool, Indicates the first The camp during the second outermost iteration The improvement in unshaping gains obtained by the candidate strategy relative to the current opponent's meta-strategy mixture; This represents the minimum policy fingerprint distance between the candidate policy and the existing policy pool of the player. This represents the minimum non-shaping benefit improvement threshold that the candidate strategy must achieve. This represents the minimum fingerprint novelty threshold that a candidate strategy must achieve.

[0050] Candidate policies are added to the policy pool only when both conditions are met; rejected candidates do not change the current meta-policy.

[0051] This invention uses the “experience strategy pool return gap” as an approximate oracle diagnostic quantity, but does not call this quantity strict NashConv, nor does it claim to have proven the convergence of the full-space Nash equilibrium based on this.

[0052] This invention achieves strict separation between candidate training rewards and policy pool evaluation rewards. Specifically, the UEP heuristic intrinsic term is only used for candidate policy training; the policy pair reward matrix, meta-game solution, reward improvement, and policy pooling all use the unshaping environment round rewards, thereby avoiding changes in the definition of zero-sum rewards due to intrinsic rewards or causing incomparability of cross-generational evaluations.

[0053] Step S10: Heuristic approximation of optimal response using cross-generational comparable fingerprints and confidence-weighted UEP. At the start of each outer iteration, a snapshot of the policy fingerprint encoder is frozen, and a common reference dataset is generated using a fixed set of scenarios, a fixed set of random seeds, and a fixed opponent evaluation protocol. All historical policies in the current policy pool are re-evaluated and re-encoded under the same frozen encoder, avoiding direct comparison of fingerprints generated by different generations of encoders.

[0054] The availability of local information is . In the formula for information availability, and These are non-negative weights, used to measure the contribution of teammate communication information and opponent perception information to the overall information availability, respectively, and satisfying the following conditions: ; Indicates drone Teammate communication availability Indicates drone The opponent's perceived availability rate; This indicates the overall availability of local information obtained by weighting the two values.

[0055] Local confidence level is formed by combining average reconstruction uncertainty. . In the local confidence formula, This represents the sensitivity coefficient of reconstruction uncertainty to confidence level; Indicates drone The average reconstruction uncertainty calculated for currently unavailable but not expired tracks; This represents the exponential decay factor resulting from reconstruction uncertainty; This means truncating the result to... Interval. A larger value indicates more complete local information and a more reliable trajectory reconstruction. This applies to drones operating within the same faction. Calculate the average team confidence level and to The average team restructuring uncertainty is obtained by averaging. .

[0056] The team confidence level is obtained by averaging the activity of drones on the same side. The training reward for candidate strategies is... . In the candidate strategy training reward system, Indicates camp exist The regularized immediate reward used to train the candidate policy at each step This indicates the immediate reward for the non-shaping environment of the corresponding faction; Indicates the team's confidence level; , and All are non-negative weight coefficients, which respectively control the strength of strategy diversity incentives, reference strategy constraints, and uncertainty penalties; This represents the minimum distance between the current tactical embedding and the existing strategy fingerprints in the player's strategy pool. This represents the distance between the current tactical embedding and the stable reference policy fingerprint; This represents the team's average restructuring uncertainty. Therefore, high confidence strengthens the diversity term while maintaining reference constraints, while low confidence weakens the diversity driver and strengthens the uncertainty penalty.

[0057] When direct information is sufficient and predictions are reliable, the behavioral difference between the policy and the existing policy pool is increased, while constraining it from deviating excessively from the stable reference policy; when information is lacking or predictions are unreliable, diversity-driven mechanisms are automatically weakened and uncertainty penalties are increased. The candidate policy is defined as a regularized approximate optimal response by the following formula: . In the regular approximate optimal response, Indicates the first In the next outermost iteration, the camp The resulting regularized near-optimal response candidate strategy; Indicates that there are trainable parameters The current strategy is defined; Indicates the first In the next iteration, the opponent's freezing meta-strategy is mixed; This represents the expectation of the random round trajectory and environmental randomness generated by the strategy interaction; Indicates the first Discount weight of each step; This represents maximizing the expected cumulative discount training gain within the range of candidate policy parameters. The trajectory sampling symbol in the formula... This represents a round-based interaction trajectory, and is related to the length of the local history window in step S2. They have different meanings.

[0058] When updating one side, the opponent's policy mix remains frozen; after training is complete, candidates are re-evaluated using non-shaping gains before switching to the other side. Trajectory prediction coefficients are always solved analytically by the local window and are not included in the set of parameters to be trained.

[0059] This invention implements a confidence-weighted UEP heuristic regularization, which encourages differences from existing behaviors in the policy pool when information is reliable, while constraining candidates from deviating excessively from stable references; when information is unreliable, it reduces diversity-driven behavior and increases uncertainty penalties to avoid blindly pursuing policy differences in erroneous reconstruction states.

[0060] Meanwhile, this invention achieves cross-generational comparable policy fingerprints and dual-threshold pooling. Specifically, each generation uses the same frozen encoder and common reference data to re-encode all historical policies; candidates are only pooled when both the no-shaping-profit improvement and fingerprint novelty exceed the threshold, reducing the entry of repetitive policies with no performance improvement into the population.

[0061] A complete outer layer iteration of the present invention includes: (1) Initialize the strategy pools of both parties, the team-shared hybrid action actors, the centralized value network, the local track buffer, the online belief encoder, and the fingerprint encoder; (2) Freeze the fingerprint encoder and recalculate all historical policy fingerprints on the common reference dataset; (3) Use the unshaping round payoff to evaluate all policy pairs, construct a zero-sum payoff matrix and solve for the meta-policy; (4) Fix one side as the learner and freeze the other side as the opponent; the learner collects rolling data in a weak information environment; (5) In each environmental step, update the teammate communication mask and the opponent perception mask respectively, perform recursive polynomial trajectory reconstruction, calculate uncertainty, and construct dual-stream fixed-dimensional belief state; (6) Generate motion masks based on local effective targets, lock-on conditions, firing conditions and ammunition status, and sample tactical patterns, targets and continuous maneuvering actions by shared actors; (7) Calculate the confidence-weighted shaping reward and update the current learner using parameter-sharing HAPPO order; (8) After training, candidates are evaluated based on the unshaping round gains and common seed set, and whether to include them in the pool is determined based on the gains improvement and fingerprint novelty. (9) Switch learning method and repeat steps (4) to (8); (10) Update the next-generation fingerprint encoder snapshot to form a new two-way policy pool and meta-policy; (11) During the deployment phase, the centralized value network, meta-game solving and candidate training modules are removed. Each UAV retains only the local track buffer, recursive polynomial reconstruction, dual-stream set encoder, shared hybrid actor and local target mask logic.

[0062] A decision-making system for pursuing and escaping unmanned aerial vehicle (UAV) swarms based on trajectory prediction and unified efficient game theory, comprising: The weak communication information access module is used to collect local status, teammate messages, opponent sensor data, communication availability flags, and sensor availability flags; The local track buffer and recursive polynomial reconstruction module is used to maintain track age, perform adaptive order weighted ridge fitting, extrapolate missing states, and project physical boundaries. The uncertainty calculation module is used to output track reliability based on the fitting residual, extrapolation leverage, and track age. The teammate-opponent dual-stream set encoding module is used to encode a variable number of objects into fixed-dimensional belief states; The local tactical legitimacy module is used to maintain the target's unconfirmed destruction status, locked status, ammunition status, and mode condition target mask; The parameter-sharing hybrid action actor module is used to output tactical modes, masked targets, and continuous flight control quantities; A centralized training module is used to perform team-level value estimation, generalized advantage calculation, and HAPPO-style sequential updates; The Experience Strategy Pool and Meta-Game Module are used to construct the non-shaping payoff matrix, solve the meta-policy, sample opponents, mix them, and update the strategy pool. The confidence-weighted UEP regularization module is used to dynamically adjust the strategy diversity, reference deviation, and uncertainty penalty based on the current information availability and trajectory reconstruction uncertainty. The deployment interface module is used to replicate the same shared strategy to any number of homogeneous drones and to connect to teams and adversaries of different sizes without changing the network input dimension.

[0063] The technical effects achieved by this invention are as follows: (1) The weak communication UAV swarm pursuit decision-making method and system based on trajectory prediction and unified efficient game theory of the present invention can effectively improve the task benefits and the all-hit rate in a standard number asymmetric scenario: In a standard scenario of 5 blue team pursuit UAVs with learning control against 8 red team evasion UAVs, the smooth evaluation reward after training is stable at about 12,000 to 12,500, and the all-hit success rate in the mature stage is maintained above 90% for most of the time, indicating that the strategy does not rely on individual favorable rounds, but forms repeatable tactical behavior.

[0064] (2) The weak-communication UAV swarm pursuit and escape decision-making method and system based on trajectory prediction and unified efficient game theory of the present invention has higher comprehensive environmental benefits compared with the compared iterative game baseline: under the unified environment, reset geometry, radar parameters, round length and no shaping reward definition, the average reported reward of TP-UEP is 17011; Obs-ZeroSumDiff-PE, Oracle-ZeroSum Diff-PE, and Coalition PE-Game are 14202, 13995, and 13853 respectively, and Dynamic Stackelberg is -9102. This result supports that the present invention has better comprehensive benefits under the adopted scenario and implementation conditions.

[0065] (3) The present invention provides a weak communication UAV swarm pursuit and escape decision-making method and system based on trajectory prediction and unified efficient game theory. The polynomial reconstruction and UEP regularization have complementary effects: ablation results show that the complete method has higher and more stable rewards in the later stages; after removing UEP regularization, the volatility increases, and after removing trajectory prediction, although some intervals are stable, it converges to a lower reward. These results indicate that trajectory reconstruction mainly compensates for short-term missing information, while UEP regularization mainly improves the stability of population-level learning.

[0066] (4) The present invention provides a weak communication UAV swarm pursuit and escape decision-making method and system based on trajectory prediction and unified efficient game theory. Under moderate communication degradation, it maintains basic performance and exhibits smooth degradation under extreme disconnection: from normal communication to Weak-L2, the average reward, total neutralization rate, and neutralization rate remain basically unchanged; when the effective link ratio drops to 0.325 in Weak-L4, the neutralization rate is still 0.963, but the total neutralization rate drops to 0.70; when the effective link ratio is 0 in Weak-L6, the neutralization rate is still 0.931, the total neutralization rate is 0.45, and the radar exposure rate rises to 0.145. The results show that the present invention retains a strong partial neutralization capability, but it is not completely robust under near-complete disconnection, and performance and radar security will still decline.

[0067] (5) The present invention provides a weak communication UAV swarm pursuit decision-making method and system based on trajectory prediction and unified efficient game theory, which supports large-scale migration without retraining: the fixed checkpoints trained in 5-to-8 scenarios are directly deployed to 10-to-15 and 15-to-30 scenarios without retraining, fine-tuning, reward modification, or environment adaptation. The neutralization rate and PQS of TP-UEP in 10-to-15 are 0.657 and 0.409, respectively, and in 15-to-30 they are 0.414 and 0.350, respectively, all significantly higher than the ablation version that removes trajectory prediction or UEP regularization. This result proves that the network interface can be directly implemented to a larger scale, but also shows that the absolute performance will decrease with the increase of scale.

[0068] (6) The trajectory and fingerprint distribution of different strategy generations of the present invention based on trajectory prediction and unified efficient game theory for weak communication UAV swarm pursuit and escape decision-making presents distinguishable tactics, forming interpretable tactical evolution and strategy diversity: early strategies form basic circling around the radar area, mid-term strategies form more organized formations and opportunities for coordinated combat, and late-term strategies generate more intensive local maneuvers. The strategy fingerprints occupy different regions in the low-dimensional projection, indicating that the strategy pool does not simply repeat the same behavior.

[0069] (7) The present invention provides a weak communication UAV swarm pursuit and escape decision-making method and system based on trajectory prediction and unified efficient game theory. The computational link in the execution stage is clear and the system is well deployable: the polynomial coefficients are solved online analytically, without the need for an additional prediction network; homogeneous UAVs share actor parameters; when deploying, there is no need for a centralized value network and meta-game solver, only the local trajectory maintenance, set encoding, legality mask and mixed action actors need to be retained, which reduces the complexity of single-machine deployment. Attached Figure Description

[0070] Figure 1This is the overall framework of the invention; the upper part of the diagram shows the two-way policy pool based on unshaping round payoffs, the policy pair payoff matrix, the solution of the finite zero-sum game, the opponent mixed sampling, and the inclusion of candidate policies into the pool; the middle part, from left to right, shows weak information input and recursive polynomial trajectory reconstruction, dual-stream set belief states, confidence-weighted UEP heuristic regularized approximate optimal response, and target perception parameter sharing HAPPO; the lower part shows the quantity asymmetric pursuit environment, radar threat, communication map, and environment output. The environment output updates the local history, communication mask, and perception mask in reverse, and the mixed actions are returned to the environment to form a closed loop.

[0071] Figure 2 This invention provides a standard 5-to-8 scenario training evaluation curve, demonstrating the changes in the original reward, smoothed reward, and full neutralization success rate during the TP-UEP training process with training iterations. This curve is used to illustrate the convergence phase and stability of the mature strategy of this invention. Figure 3 This is a comparison of the checkpoint rewards of TP-UEP, MAPPO, MATD3 and GNN-MARL under the Default and Weak-L5 conditions of this invention. The horizontal axis is the normalized checkpoint progress of each method and the vertical axis is the return reward of the unified environment. This is used to illustrate the learning stability of this invention under full communication and severely weak communication. Figure 4 The invention describes the communication availability decline trend and the changes in reward and success rate between the complete method and the ablation method, which are composed of the effective communication link trend and the reward-success rate robustness curve; the left figure shows that ELR decreases monotonically from Normal to Weak-L6; the right figure shows the changes in reward and success rate of the complete method and the two ablation methods when communication degrades, which is used to illustrate the boundary of the role of the prediction module and UEP regularization under moderate and severe disconnection conditions; Figure 5 This invention provides a representative trajectory for direct deployment of fixed 5-to-8 training checkpoints to 10-to-15 and 15-to-30 scenarios, including a three-dimensional trajectory and XZ projection, to illustrate that the variable cardinality dual-stream ensemble encoder can access a larger number of teammates and opponents without changing the input dimension of the policy network. Figure 6 This invention presents the three-dimensional tracks and XZ projections of the three strategy generations Gen15, Gen35, and Gen119. The left column shows the three-dimensional tracks, and the right column shows the XZ projections. These are used to illustrate the tactical evolution from basic circling to formation coordination and then to dense local maneuvers during the strategy pool iteration process. Detailed Implementation

[0072] To make the objectives and advantages of this invention clearer, the invention will be specifically described below with reference to embodiments. It should be understood that the following text is merely used to describe one or more specific embodiments of the invention and does not strictly limit the scope of protection specifically claimed by the invention.

[0073] A decision-making method for pursuing and escaping unmanned aerial vehicle (UAV) swarms with weak communication, based on trajectory prediction and unified efficient game theory, is proposed. This method is applicable to scenarios where communication distance is limited, line-of-sight is obstructed, sudden packet loss occurs, interference or suppression leads to intermittent loss of teammate messages, and the opponent can only observe locally through onboard sensors. The system needs to simultaneously complete tactical mode selection, target selection, and continuous flight maneuver control under partially observable conditions, and be able to handle problems such as unequal numbers of both sides and changes in the number of nodes due to battle losses.

[0074] Example 1: Standard 5-to-8 Fixed-Wing UAV Swarm Pursuit Training and Evaluation This embodiment sets up 5 blue team learning-controlled pursuit drones and 8 red team script-evading drones in a 3D point-mass fixed-wing simulation environment. The nominal horizontal map size is 4000m × 4000m, and the actual boundary is... , The integration step size is 0.1s, the control frequency is 10Hz, and the maximum number of steps per round is 1200, corresponding to 120s.

[0075] The initial formation center of the Blue team is (300, 300, 600) m, and the horizontal offsets of the five UAVs are (0, 0), (-30, 30), (-30, -30), (-60, 60), and (-60, -60) m respectively. The initial formation center of the Red team is (2800, 2800, 600) m, and eight different sets of three-dimensional offsets are used. The initial velocity of the Blue team is 30 m / s, the track inclination angle is 0, and the heading angle is... Red team's speed 20m / s, heading angle The speed limit for the blue team is 25 to 38 m / s, the normal overload is 0 to 2g, and the roll and track inclination angles are limited to [missing information]. .

[0076] The radar is located at (1500, 1500, 0) m, and its power parameters are as follows: Radar detection probability is calculated based on the fourth power of the range and the action-related RCS. The RCS for cruise, lock-on, and firing are set to 1, 3, and 8, respectively. A detection probability exceeding 0.95 indicates a loss of the Blue Force UAV. Threat exposure is recorded when any active Blue Force UAV enters the radar's 800m range. The Blue Force neutralizes Red Force targets only when choosing to fire, the target distance is less than 800m, and the speed-line-of-sight alignment is greater than 0.4.

[0077] Trajectory prediction retains 5 frames of history; the standard implementation uses 3 target position frames, a second-order polynomial, and a 0.5s prediction time domain. Hybrid actions include 3 normalized continuous actions and 3 discrete modes: cruise, lock-on, and fire. Evaluation uses a fixed Gen35 strategy with a probability of 0.937049 in the pre-formed meta-strategy; checkpoint selection is not based on the evaluation results in this section.

[0078] Depend on Figure 2 It can be seen that the smooth reward rises rapidly from about 10,000 in the early stage of training; after about 30 training iterations, it enters a stable range of 12,000 to 12,500, and in the mature stage, the success rate of all hits is above 90% for most of the time.

[0079] Example 2: Unified Evaluation of Multi-Agent Learning and Iterative Game Baselines Under both Default and Weak-L5 communication conditions, TP-UEP was compared with MAPPO, MATD3, and GNN-MARL after normalization according to their respective checkpoint progress. All methods used the same environment to return rewards and the same reset, radar, and round configurations.

[0080] Depend on Figure 3 It can be seen that TP-UEP maintains positive rewards under both conditions, and stabilizes at approximately [value missing] in the later stages. ; Default conditions can be achieved in the early stages. The Weak-L5 condition first decreased and then recovered to approximately [value missing]. Subsequently, it tends to approach the level of the Default. All three traditional learning baselines remain in the negative reward zone within the reporting checkpoint range of this implementation.

[0081] In the baseline comparison of iterative game theory, the unified evaluation results are as follows:

[0082] This embodiment illustrates that the present invention balances neutralization capability, radar risk, and execution efficiency under the simulation environment and unified evaluation protocol adopted.

[0083] Example 3: Progressive Weak Communication Robustness Test Effective communication links were progressively reduced from Normal to Weak-L6, and environmental rewards, total neutralization rate (FNSR), neutralization rate (KR), radar exposure rate (TER), and effective link ratio (ELR) were recorded. The results are shown in the table below.

[0084]

[0085] Normal to Weak-L2 has almost no impact on performance; after Weak-L4, the overall neutralization rate and reward decrease significantly; although Weak-L6 lacks effective teammate links, the neutralization rate is still 0.931, indicating that local recursive reconstruction retains strong partial task capabilities. At the same time, the large reward variance, low FNSR, and high TER indicate that complete disconnection cannot be completely offset by the prediction module.

[0086] Example 4: 10-to-15 and 15-to-30 zero-sample-scale transfer The fixed policies trained in the 5-to-8 scenario are directly deployed to the 10-to-15 and 15-to-30 scenarios. Each method-scale combination uses 5 random seeds, with 30 rounds per seed, for a total of 150 rounds; different methods use matching reset states and round keys. No retraining, fine-tuning, reward modification, or environment adaptation is performed throughout the process.

[0087]

[0088] Depend on Figure 5 It is evident that TP-UEP can run directly on both unseen scales and maintains higher performance than the two ablation versions of KR and PQS. However, as the scale increases, many-to-many interactions become more intensive, making full neutralization and self-survival significantly more difficult. Therefore, zero-sample "deployability" does not equate to no performance degradation.

[0089] Example 5: Strategic Intergenerational Tactical Evolution Gen15, Gen35, and Gen119 were selected to represent the early effective strategy, the stable mid-term strategy, and the late-term strategy, and their three-dimensional tracks and XZ projections were compared.

[0090] like Figure 6 As shown, Gen15 has established basic coordinated maneuvering around the radar area; Gen35's formation is more organized and its engagement timing is more coordinated; Gen119 generates more concentrated local maneuvers near the engagement zone, but does not demonstrate a clear global advantage over Gen35. These results indicate that strategy pool learning has resulted in distinguishable tactics, while mid-term strategies can achieve a relatively balanced combination between formation consistency and local engagements.

[0091] The above description is merely a preferred embodiment of the present invention. It should be noted that those skilled in the art can make various improvements and modifications without departing from the principles of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention. Structures, devices, and operating methods not specifically described or explained in this invention are implemented according to conventional methods in the art unless otherwise specified or limited.

Claims

1. A decision-making method for pursuing and escaping swarms of unmanned aerial vehicles (UAVs) with weak communication based on trajectory prediction and unified efficient game theory, characterized in that: It includes a centralized training phase and a decentralized decision-making phase: The intensive training phase includes the following steps: Step A1: Generate teammate communication availability masks and opponent sensor availability masks respectively, and independently calculate the availability of the two types of information; maintain local track buffers for each teammate and opponent, and reconstruct the missing state by using recursive multinomial fitting extrapolation when information is missing, and calculate the reconstruction uncertainty index based on the fitting residual, extrapolation leverage and track age; construct object features for teammates and opponents respectively, and generate fixed-dimensional belief states that are independent of cluster size through dual-stream independent permutation invariant aggregation encoding. Step A2: Construct a factorized hybrid action strategy, wherein the hybrid actions include tactical mode selection, conditional target selection, and continuous maneuver commands; based on a centralized value network and generalized advantage estimation, update the team-shared strategy using a parameter-sharing HAPPO mechanism; Step A3: Combine information availability and reconstruction uncertainty to calculate team confidence, and dynamically adjust the incentive strength for strategy diversity and the penalty strength for uncertainty in the training rewards based on the confidence to construct a regularized approximate optimal response training objective; Step A4: Construct the policy pair payoff matrix using the original round payoffs of the environment without shaping terms, solve the zero-sum game to obtain the meta-policies of both parties, and complete the candidate policy entry into the pool based on the dual thresholds of payoff improvement amount and policy fingerprint novelty, and iteratively update the policy pool. In the decentralized decision-making phase, each UAV loads the shared strategy completed during training and independently completes the calculation of belief state and output of mixed decision actions based solely on the local track buffer maintained by the UAV, the recursive fitting compensation results, and the belief state.

2. The weak communication UAV swarm pursuit and escape decision-making method according to claim 1, characterized in that: The teammate communication availability mask is determined by the physical communication adjacency relationship and the Gilbert-Elliott burst packet loss channel model. The opponent sensor availability mask is determined by the sensing distance constraint, the line-of-sight constraint, and the target detection probability; the availability rates of the two types of availability masks are calculated separately and are not mixed.

3. The weak communication UAV swarm pursuit and escape decision-making method according to claim 1, characterized in that: The local track buffer is used to maintain normalized track data and track age: when a message is received again or a target is detected again, the direct observation state is written into the buffer and the track age is cleared; when there is continuous loss of contact, the predicted compensation state of the previous moment is written into the buffer as the current local track sample and then refitted with the updated history; when the track age exceeds the preset maximum threshold, the corresponding track is marked as expired and expired tracks cannot participate in target selection.

4. The weak communication UAV swarm pursuit and escape decision-making method according to claim 3, characterized in that: The recursive polynomial fitting employs an adaptive-order weighted ridge polynomial fitting: The directly observed samples are assigned unit weights, and the recursive predicted samples decay according to the track age. The polynomial coefficients are solved, and the predicted compensation state at the current moment is extrapolated and restored to the physical state.

5. The weak communication UAV swarm pursuit and escape decision-making method according to claim 1, characterized in that: The reconstruction uncertainty index is calculated by combining the weighted fitting residual, the extrapolation point leverage, and the track age. When the fitting residual increases, the extrapolation distance increases, or the continuous disconnection time is prolonged, the reconstruction uncertainty index automatically increases. This index is used for both belief state feature input and training regularization intensity modulation.

6. The weak communication UAV swarm pursuit and escape decision-making method according to claim 1, characterized in that: The flow-independent permutation-invariant aggregation encoding includes the following steps: Teammate and opponent object features are respectively encoded by independent shared encoders to obtain object embeddings, and local query vectors are obtained by local state encoding. Masked multi-head attention is performed on the teammate set and opponent set respectively using a single local query vector to obtain the final fixed-dimensional belief state. The encoding process does not use agent unique number embedding and sequence position encoding. Changing the input order of teammates or opponents does not change the belief state, and the input dimension of the policy network does not change with the cluster size.

7. The weak communication UAV swarm pursuit and escape decision-making method according to claim 6, characterized in that: The joint log probability of the factorized hybrid action strategy is equal to the sum of the log probabilities of the pattern distribution, the conditional target distribution, and the continuous action distribution; During training, the sampling mask is used to calculate the probabilities of the old and new policies.

8. The weak communication UAV swarm pursuit and escape decision-making method according to claim 1, characterized in that: During intensive training, each party uses a team-level feature obtained by permutation-invariant pooling of the belief states of the active agents, estimates the reward and calculates the generalized advantage through a value network, and performs HAPPO sequential sub-updates according to a randomly permuted subset of agents. The total loss consists of policy loss, value loss and hybrid entropy regularization.

9. The weak communication UAV swarm pursuit and escape decision-making method according to claim 1, characterized in that: The information availability is obtained by weighted summation of the teammate communication availability and the opponent's sensor availability. The local confidence is formed by combining the average reconstruction uncertainty. The team confidence is obtained by averaging the activity of the same party's drones. When the team confidence is high, the incentive for strategy diversity is increased, while the deviation of the strategy from the reference strategy is constrained. When the confidence is low, the diversity drive is weakened and the uncertainty penalty is increased.

10. A decision-making system for pursuing and escaping a swarm of unmanned aerial vehicles (UAVs) with weak communication based on trajectory prediction and unified efficient game theory, employing the decision-making method for pursuing and escaping a swarm of unmanned aerial vehicles with weak communication as described in any one of claims 1-9, characterized in that: include: The weak communication information access module is used to collect local status, teammate messages, opponent sensor data, communication availability flags, and sensor availability flags; The local track buffer and recursive polynomial reconstruction module is used to maintain track age, perform adaptive order weighted ridge fitting, extrapolate missing states, and project physical boundaries. The uncertainty calculation module is used to output track reliability based on the fitting residual, extrapolation leverage, and track age. The teammate-opponent dual-stream set encoding module is used to encode a variable number of objects into fixed-dimensional belief states; The local tactical legitimacy module is used to maintain the target's unconfirmed destruction status, locked status, ammunition status, and mode condition target mask; The parameter-sharing hybrid action actor module is used to output tactical modes, masked targets, and continuous flight control quantities; A centralized training module is used to perform team-level value estimation, generalized advantage calculation, and HAPPO-style sequential updates; The Experience Strategy Pool and Meta-Game Module are used to construct the non-shaping payoff matrix, solve the meta-policy, sample opponents, mix them, and update the strategy pool. The confidence-weighted UEP regularization module is used to dynamically adjust the strategy diversity, reference deviation, and uncertainty penalty based on the current information availability and trajectory reconstruction uncertainty. The deployment interface module is used to replicate the same shared strategy to any number of homogeneous drones and to connect to teams and adversaries of different sizes without changing the network input dimension.

Images