Unmanned aerial vehicle cluster cooperative game method and system for adaptive communication and directional interference

Abstract

The application discloses a UAV cluster cooperative game method and system for adaptive communication and directional interference, and belongs to the technical field of UAV cluster control, wireless communication and electromagnetic protection. The application constructs a UAV cluster game scenario and a basic model, designs an adaptive dynamic communication topology and a power control mechanism, and realizes dynamic adjustment of communication radius and transmission power through a quantitative formula. An omnidirectional / directional switchable adaptive interference model is established to accurately quantify the correlation between directional interference gain and competitor situation awareness error, and a cluster frequency hopping anti-interference communication mechanism is synchronously designed. Finally, a constrained MADDPG game decision model integrating a multi-agent attention mechanism is constructed to realize end-to-end linkage optimization of communication cooperation, active interference and game decision. The application greatly improves the cluster game task completion rate, anti-interference capability and scene generalization, and can be widely applied to civil scenes such as multi-aircraft cooperative operation, airspace resource scheduling, complex environment inspection and emergency rescue.

Description

Technical Field

[0001] This invention belongs to the fields of unmanned aerial vehicle (UAV) swarm control, wireless communication and electromagnetic protection technology, and particularly relates to a UAV swarm cooperative game method and system with adaptive communication and directional interference. Background Technology

[0002] With the rapid development of drone technology, drone swarms, with their advantages of distributed collaboration, high fault tolerance, and strong mission scalability, have been widely applied in various fields such as multi-drone collaborative operations, airspace resource scheduling, complex environment inspection, and emergency rescue. In drone swarm collaborative game scenarios, wireless communication collaboration is the core foundation for realizing multi-drone collaborative operations, while electromagnetic interference protection and active interference capabilities are key means to gain an advantage in operational information and ensure the successful completion of tasks.

[0003] Existing technologies for collaborative game theory involving UAV swarms suffer from the following core shortcomings: First, swarm communication mechanisms often employ idealized designs with fixed communication radii, failing to meet the connectivity requirements of dynamic game scenarios. Fixed high-power communication is easily intercepted and located by third parties, while fixed low-power communication is prone to swarm disconnection. Furthermore, no anti-interference communication mechanism has been designed for the swarm, making its own communication susceptible to external electromagnetic interference. Second, active interference schemes often adopt fixed omnidirectional interference modes. Indiscriminate omnidirectional interference has high power consumption and poor concealment, making it extremely easy for third parties to detect and locate. Moreover, it fails to achieve adaptive linkage between interference parameters and the game scenario, resulting in low interference efficiency. Third, traditional deep reinforcement learning decision-making models do not deeply integrate the constraints and optimization of communication and interference resources into the decision-making process. They exhibit poor generalization when the swarm size changes, failing to achieve end-to-end linkage optimization of communication collaboration, active interference, and task execution, resulting in insufficient engineering feasibility and practical operational performance.

[0004] Therefore, there is an urgent need to propose a collaborative game-playing scheme for drone swarms that possesses adaptive communication capabilities, directional and efficient interference capabilities, and strong generalization intelligent decision-making capabilities, in order to address the core pain points of existing technologies. Summary of the Invention

[0005] The purpose of this invention is to overcome the shortcomings of existing technologies, such as the susceptibility of fixed communication to eavesdropping and loss of connection, low efficiency and poor concealment of omnidirectional interference, and poor generalization of decision-making models. It provides a collaborative game method and system for UAV swarms with adaptive communication and directional interference, which realizes adaptive adjustment of communication parameters, directional interference, and multi-dimensional action linkage decision-making, while significantly improving the actual operational performance and engineering feasibility of UAV swarms.

[0006] To achieve the above-mentioned objectives, the present invention adopts the following technical solution:

[0007] A cooperative game theory method for UAV swarms based on adaptive communication and directional interference includes the following steps:

[0008] S1 Constructing a Cluster Game Scenarios and Basic Model: Delineating a three-dimensional operational airspace, configuring our own operational drone cluster and the competitor's drone cluster, and equipping each drone with a flight power module, situational awareness module, frequency hopping anti-interference wireless communication module, omnidirectional / directional switchable active interference module, and task execution module;

[0009] S2 constructs a cluster adaptive distributed communication mechanism: sets a dynamically adjustable range for the communication radius of our UAVs, adjusts the communication radius and transmission power of a single UAV in real time based on the adaptive communication parameter optimization formula, and realizes encrypted sharing of situational information of friendly UAVs within the communication range by combining connectivity determination rules, while realizing cluster anti-interference communication through synchronous frequency hopping sequences.

[0010] S3 constructs an adaptive active jamming and electromagnetic protection model: sets a dynamically adjustable range for the jamming radius of UAVs, supports switching between omnidirectional and directional jamming modes, and quantifies the impact of directional jamming gain and jamming signal on the situational awareness distance error of competitor UAVs;

[0011] S4 Constructs a game-theoretic decision-making model that integrates attention mechanisms: taking real-time UAV situational information as state input, flight actions, communication parameter actions, interference parameter actions, and task triggering commands as action outputs, and communication-interference linkage constraint-type reward function as optimization objective, adopting the MADDPG algorithm that integrates multi-agent attention mechanisms, and completing model training based on a centralized training-distributed execution architecture;

[0012] S5 online game task execution: The trained model is deployed to each of our drones. Based on real-time perception and shared information, the model outputs decision commands and simultaneously executes adaptive communication networking, directional interference suppression and task execution actions to complete the cluster game task.

[0013] Furthermore, in step S2, the shared situational information includes the UAV's own three-dimensional position, flight speed, heading, remaining battery power, jamming module status, and the detected position and movement information of competitor UAVs; our cluster adopts a decentralized distributed communication architecture, with each UAV only interacting encrypted with friendly neighbor nodes within its communication range; when a UAV exceeds the communication range of all friendly neighbors, it switches to an autonomous independent operation mode, and automatically synchronizes the frequency hopping sequence and resumes cooperative communication after re-entering the communication range.

[0014] Furthermore, in step S2, the dynamically adjustable range of the communication radius is 200m~800m, and the adaptive communication parameter optimization formula is calculated based on: the distance to the nearest friendly node, the current channel eavesdropping signal-to-noise ratio, the upper and lower limits of the transmission power, and the third-party eavesdropping risk level; the connectivity determination rule is based on the distance between friendly UAVs and the communication radius; the frequency hopping sequence is generated using a pseudo-random sequence, and the nodes in the cluster achieve precise alignment of the frequency hopping time and frequency point through synchronization beacons.

[0015] Further, in step S3, the dynamically adjustable range of the interference radius is 100m~500m, and the maximum interference radius does not exceed the current communication radius; the directional interference mode adopts beamforming technology, and the adjustable range of the interference beamwidth is 15°~90°, which only performs directional suppression on the selected competitor target. The directional interference gain is calculated based on the omnidirectional interference base gain, the effective aperture area of ​​the antenna, the wavelength of the interference signal, and the angle between the interference beam axis and the line connecting the UAV to the competitor target; the influence of situational awareness distance error is calculated based on the directional interference gain, the UAV interference power, the base error corresponding to the full power unit distance, the position coordinates of the UAV and the competitor target, and the interference coverage determination factor. When multiple friendly UAVs carry out coordinated interference on the same target, the interference error is linearly superimposed and does not exceed the preset maximum error limit.

[0016] Furthermore, in step S4, the multi-agent attention mechanism is used to assign weights to the shared neighbor information and the perceived competitor target information; in the offline training stage, the policy network and evaluation network are iteratively optimized based on global situational information; in the online execution stage, a single UAV outputs decisions based solely on its own perception and locally shared information.

[0017] Further, in step S4, the state input includes: ① its own state, including three-dimensional coordinates, three-dimensional velocity, heading angle, and remaining battery power; ② the state of neighboring UAVs within the communication range, including relative three-dimensional coordinates, relative three-dimensional velocity, heading angle, and remaining battery power; ③ the relative motion state of competing UAVs within the perception range, including relative three-dimensional coordinates and relative three-dimensional velocity; ④ the channel environment and eavesdropping risk state, including channel signal-to-noise ratio and eavesdropping risk level; wherein the perception range is a fixed detection radius of 300m for the airborne sensor, and the communication range and interference range are dynamically adjusted values;

[0018] The action outputs include: ① Flight actions, including flight speed adjustment and heading adjustment; ② Communication parameter actions, including communication radius adjustment and transmit power adjustment, which are dynamically adjusted based on the real-time adjustment of the communication radius and transmit power of a single UAV in S2; ③ Interference parameter actions, including interference mode switching, interference power adjustment, interference azimuth angle, interference pitch angle, and beamwidth adjustment; ④ Mission trigger command, which is a binary mission start / stop control signal.

[0019] Further, in step S4, the communication-interference linkage constraint-type reward function is:

[0020]

[0021] Instant rewards The final reward is calculated once per decision step. Calculated only once at the end of the game round, and added to the total reward included in the final step. ;

[0022]

[0023] In the formula These are the weighting coefficients. For interference benefits, interference is considered effective if the target is within the interference radius and beam coverage area. Effective directional interference earns +3 base reward, and effective omnidirectional interference earns +1 base reward. For communication connectivity rewards, the cluster connectivity rate is calculated based on the ratio of the number of drones actually connected to the total number of drones. A +1 reward is given when the connectivity rate is ≥80%, and 0 is given when it is below 80%. In addition to the +1 or 0 reward based on the cluster connectivity rate, 1 point is deducted for each single node that is completely disconnected from all neighboring nodes. The points are accumulated for multiple disconnected nodes. For target tracking rewards, if a competitor's target remains within our perception range for three consecutive decision steps, it is considered stable tracking, and we receive +1 reward for each step; if the target's interference error continuously exceeds the threshold and it is determined to have lost its operational capability, we receive a one-time reward of +50. To enforce penalties, drones exceeding the operational airspace boundary are considered to have crossed the boundary; drones within 5 meters of each other are considered to have collided; each instance results in a deduction of 100 points. Communication / interference parameters exceeding the adjustable range are considered to have crossed the boundary constraint; each instance results in a deduction of 5 points.

[0024]

[0025] , These are the weighting coefficients. The winning bonus is triggered only at the end of the game round; the competitor's drones lose all operational capability and the player's drones lose all operational capability and the player loses 200; this bonus is not triggered if only some drones lose operational capability. The number of drones that maintain our operational capability will be awarded a corresponding weighted reward for each drone maintained, with points calculated in real time based on the actual number. The number of drones that lose operational capability for the competitor is used to determine the corresponding weighted reward for each drone that fails, and the score is calculated in real time based on the actual number. The loss of operational capability is determined by a symmetrical judgment rule: when the perception error caused by interference continuously exceeds the preset threshold, or when a boundary crossing or collision occurs, the operational capability is determined to be lost. The interference effect of our drones on the competitor is calculated according to the rule symmetrical to S3. The interference parameters of the competitor are fixed baseline values ​​in the simulation environment. The competitor's drone swarm adopts a traditional baseline operation strategy, with a fixed communication radius of 500m, a fixed omnidirectional interference mode, and a maneuver rule of preset area cruise and target tracking logic, without autonomous learning and parameter iteration capabilities.

[0026] An adaptive communication and directional jamming UAV swarm cooperative game system is used to execute the adaptive communication and directional jamming UAV swarm cooperative game method described above, including a UAV swarm of our own and a training and deployment module; each UAV in the UAV swarm is equipped with a flight power module, a situational awareness module, a frequency hopping anti-jamming wireless communication module, an omnidirectional / directional switchable active jamming module, a mission execution module and an airborne decision-making module.

[0027] The frequency hopping anti-interference wireless communication module is used to execute an adaptive distributed communication mechanism to complete the encrypted sharing of friendly neighbor situation information and cluster anti-interference communication;

[0028] The omnidirectional / directional switchable active jamming module is used to execute an adaptive active jamming model to complete directional / omnidirectional jamming suppression;

[0029] The training and deployment module is used to perform offline training and airborne deployment of a game decision-making model that incorporates an attention mechanism;

[0030] The airborne decision module is used to output real-time decision commands through the trained model to control the UAV to complete the game task.

[0031] Compared with the prior art, the present invention has the following advantages:

[0032] (1) The present invention designs an adaptive dynamic communication topology and power control mechanism, which replaces the traditional fixed communication radius scheme. It can dynamically adjust the communication parameters according to the game scenario. While ensuring the cluster connectivity, it greatly reduces the probability of the communication signal being eavesdropped by a third party. The synchronously designed frequency hopping anti-interference communication mechanism can effectively resist external electromagnetic interference, solving the pain point that the traditional scheme cannot balance connectivity and concealment.

[0033] (2) This invention constructs an omnidirectional / directional switchable adaptive interference model, which replaces the traditional fixed omnidirectional interference scheme. It achieves directional and precise interference against the competitor's target through beamforming technology, improving the interference efficiency by more than 3 times while reducing power consumption by 70%. It greatly improves the concealment and anti-location capability of the interference action and realizes the adaptive linkage of interference power, direction, mode and game scenario.

[0034] (3) This invention constructs a constrained MADDPG decision model that integrates multi-agent attention mechanism. It deeply integrates the constraints and optimization of communication resources and interference resources into the reward function and network structure, solves the problem of poor generalization of traditional model clusters, realizes end-to-end linkage optimization of communication coordination, active interference and task execution, and enables the model to learn the optimal coordination strategy of multi-dimensional actions autonomously. It has a strong adaptive ability in dynamic and complex game scenarios.

[0035] (4) The technical solution of the present invention can be directly and modularly deployed on existing industrial-grade drone platforms without major modifications to the hardware architecture. It can be quickly implemented in various civilian operation scenarios and has extremely high engineering application value. Attached Figure Description

[0036] Figure 1. Overall flowchart of the collaborative game method of the present invention;

[0037] Figure 2. Schematic diagram of modeling the adaptive communication range and directional interference range of UAVs;

[0038] Figure 3. Schematic diagram of the game decision-making model architecture that integrates multi-agent attention mechanism. Detailed Implementation

[0039] The present invention will be further described in detail below with reference to specific embodiments and formulas.

[0040] This embodiment provides a method and system for cooperative game theory in UAV swarms, based on adaptive communication and directional interference. Figure 1 As shown, the complete steps are as follows:

[0041] Step S1: Construction of Cluster Game Scenarios and Basic Models

[0042] A 1000m×1000m×100m three-dimensional operational airspace was defined, and airspace boundary constraints were set. Drones exceeding the boundaries were deemed to have exited the mission. A swarm of our operational drones and a swarm of the competitor's drones were configured, each swarm consisting of 6 drones. Basic drone parameters were: maximum flight speed 20m / s, maximum acceleration 5m / s², maximum single-drone situational awareness detection radius 300m, and maximum endurance 20min.

[0043] Each drone is equipped with five core functional modules:

[0044] Flight propulsion module: used to perform flight maneuvers, speed and heading adjustments for the UAV;

[0045] Situational awareness module: Based on airborne radar and electro-optical pods, it acquires situational information about itself and other UAVs within its perception range;

[0046] Frequency-hopping anti-interference wireless communication module: supports dynamic adjustment of communication radius and transmission power, supports pseudo-random frequency-hopping communication, and is used for encrypted information sharing between friendly UAVs;

[0047] Omnidirectional / directional switchable active jamming module: Based on a phased array antenna, it realizes the switching between omnidirectional and directional jamming modes, supports dynamic adjustment of jamming power, direction and beamwidth, and is used to jam the situational awareness and communication capabilities of competitors' UAVs;

[0048] Task execution module: Used to perform operational tasks in the target area, interfere with competing drones, and preempt task priorities.

[0049] The competitor's drone swarm adopts a traditional baseline operation strategy, with a fixed communication radius of 500m and a fixed omnidirectional jamming mode. Its maneuver rules are preset area cruise and target tracking logic, and it has no autonomous learning or parameter iteration capabilities. The entire process serves as a benchmark to verify the performance of our solution in an adversarial environment.

[0050] Step S2: Construction of Cluster Adaptive Distributed Communication Mechanism

[0051] The dynamically adjustable range for setting the communication radius for friendly drones is: The adjustable range of communication transmission power is: .

[0052] Communication connectivity determination formula: Let the three-dimensional position coordinates of the friendly UAV i be... The three-dimensional position coordinates of the friendly drone j are: Then the formula for determining the communication connectivity between the two is:

[0053] (1)

[0054] in This indicates that the two can communicate bidirectionally. This indicates that communication is impossible. Let be the current communication radius of UAV i.

[0055] Adaptive Communication Parameter Optimization Formula: The baseline calculation formula for adaptive communication parameters is as follows: With connectivity assurance, enhanced stealth, and power consumption control as constraints, a baseline calculation rule for communication radius and transmission power is constructed, serving as the foundational parameters for adaptive communication. The decision network can output incremental adjustments based on this baseline value to adapt to the real-time requirements of dynamic game scenarios. Adaptive adjustment formulas for communication radius and transmission power:

[0056] (2)

[0057] in, Let be the distance from drone i to its nearest friendly node. The signal-to-noise ratio of the signal received by drone i. Risk levels are determined by the detection intensity of unknown signals in the surrounding area and mapped to a continuous value of 0 to 1 according to preset rules, where 0 represents no risk and 1 represents the highest risk. These are weighting coefficients (in this embodiment, they are taken as 0.3 and 0.8 respectively). These are the upper and lower limits of the communication radius. These represent the upper and lower limits of the transmission power.

[0058] when At the same time, UAVs i and j share situational information in real time through an encrypted channel, including their own three-dimensional position, flight speed, heading, remaining battery power, jamming module status, and the position and movement information of detected competitor UAVs. This embodiment adopts a decentralized distributed communication architecture, where a single UAV only interacts with neighboring nodes within its communication range. Simultaneously, synchronization beacons are used to achieve full cluster frequency hopping sequence alignment with a 10ms hopping interval, effectively resisting external electromagnetic interference. When a UAV exceeds the communication range of all neighboring UAVs, it switches to an autonomous independent operation mode, automatically synchronizing the frequency hopping sequence and resuming cooperative communication upon re-entering the communication range.

[0059] Step S3: Construction of Adaptive Active Interference and Electromagnetic Protection Model

[0060] like Figure 2 As shown, the dynamically adjustable range for setting the interference radius for each drone is... And the maximum interference radius does not exceed the current communication radius. It supports switching between omnidirectional and directional jamming modes. In directional jamming mode, the beamwidth is adjustable from 15° to 90°.

[0061] Directional jamming gain formula: In directional jamming mode, the jamming gain for a target competitor's UAV k is calculated using the following formula:

[0062] (3)

[0063] in For the omnidirectional interference base gain, The effective aperture area of ​​the antenna, For the wavelength of the interference signal, To determine the angle between the interference beam axis and the line connecting UAV i to target k, when When the beamwidth is outside the range, the gain attenuation is 0.

[0064] Interference effect quantification formula: Let the interference power of the interference source UAV i account for the proportion of full power as follows: (Value range 0~1, 1 represents full power), the position of competitor drone k is... Then the distance measurement error experienced by the competitor's drone k The calculation formula is:

[0065] (4)

[0066] in To gather friendly drone swarms, This represents the baseline error corresponding to a unit distance at full power. The interference coverage determination factor satisfies:

[0067] (5)

[0068] Set the interference radius of the interference source drone i, and simultaneously set the maximum error upper limit. ,Right now When multiple friendly drones coordinate to interfere with the same target, the errors are linearly superimposed.

[0069] Step S4: Construction of a game-theoretic decision-making model incorporating attention mechanisms

[0070] Formula (1) is used to determine the communication connectivity status of nodes. Formula (2) outputs the baseline value of communication parameters, which together with the incremental value of the network output to obtain the final effective parameters. Formulas (3)-(5) are used to quantify the actual effect of interference. All calculation results together constitute the input of the reward function, which serves as the feedback basis for model training.

[0071] like Figure 3As shown, this embodiment employs the MADDPG algorithm, which integrates a multi-agent attention mechanism. It uses an MLP architecture and includes two types of networks: a policy network and an evaluation network. The policy network consists of two fully connected hidden layers, each with 128 neurons. The hidden layer activation function is ReLU, and the output layer activation function is Tanh, outputting normalized continuous actions. The evaluation network also consists of two fully connected hidden layers, each with 128 neurons. The hidden layer activation function is ReLU, and the output layer uses linear activation, outputting a one-dimensional state-action Q-value. A centralized training-distributed execution architecture is adopted, with the core structure as follows:

[0072] (1) State Space: The state input of a single UAV includes four categories: ① its own state, including three-dimensional coordinates, three-dimensional velocity, heading angle, and remaining battery power; ② the state of neighboring UAVs within the communication range, including relative three-dimensional coordinates, relative three-dimensional velocity, heading angle, and remaining battery power; ③ the relative motion state of competing UAVs within the perception range, including relative three-dimensional coordinates and relative three-dimensional velocity; ④ the channel environment and eavesdropping risk state, including the channel signal-to-noise ratio and eavesdropping risk level. Among them, the perception range is a fixed detection radius of 300m for the airborne sensor, while the communication range and interference range are dynamically adjusted values.

[0073] (2) Action Space: The action output of a single UAV is a continuous action, including four types: ① Flight action, including flight speed adjustment and heading adjustment; ② Communication parameter action, including communication radius adjustment and transmit power adjustment, which are dynamically fine-tuned based on the baseline parameters calculated by formula (2); ③ Interference parameter action, including interference mode switching, interference power adjustment, interference azimuth angle, interference pitch angle, and beamwidth adjustment; ④ Mission trigger command, which is a binary mission start and stop control signal.

[0074] (3) Multi-agent attention mechanism: An attention module is introduced into the policy network to assign weights to the input neighbor information and the competitor's target information. The weight calculation formula is as follows:

[0075]

[0076] in The feature vector is obtained by mapping the UAV's own state through a shared-coding fully connected layer. Let W be the feature vector obtained by mapping the state of the m-th external node (friendly neighbor / competitor) through the same encoding layer; W is the learnable weight matrix. The attention weight (scalar) of the m-th node is given; the summation term in the denominator iterates through all external nodes involved in the computation, and normalizes all of them.

[0077] The attention module operates between the input layer of the policy network and the first hidden layer: the feature vectors of each external node are weighted and summed according to their corresponding weights, then concatenated with their own feature vectors and fed into the subsequent hidden layer for calculation, so as to focus on the information of high-value nodes.

[0078] (4) Formula for the communication-interference linkage constraint-type reward function: Total reward From instant rewards and endgame rewards The composition, the formula is:

[0079] (6)

[0080] Instant rewards The final reward is calculated once per decision step. Calculated only once at the end of the game round, and added to the total reward included in the final step. Instant reward. To determine the benefits and penalties for single-step decision-making, the constraints of communication and interference are deeply integrated into the reward system. Interference is deemed effective when the target is within the interference radius and beam coverage area; operational capability is deemed lost when the interference error continuously exceeds a threshold. The calculation formula is as follows:

[0081] (7)

[0082] In the formula These are weighting coefficients (in this embodiment, they are taken as 1.2, 0.6, 1, and 1 respectively). To determine the interference benefit, interference is considered effective if the target is within the interference radius and beam coverage area. Effective directional interference earns a +3 base reward, while effective omnidirectional interference earns a +1 base reward. The final benefit is positively correlated with the measurement error caused by the interference. For communication connectivity rewards, the cluster connectivity rate is obtained by calculating the ratio of the actual number of connected drones to the total number of drones based on formula (1). A reward of +1 is obtained when the connectivity rate is ≥80%, and 0 is obtained when it is below 80%. Based on the +1 or 0 reward obtained from the cluster connectivity rate, 1 point is deducted for each single node that is completely disconnected from all friendly nodes. The points are accumulated for multiple disconnected nodes. For target tracking rewards, if a competitor's target remains within our perception range for three consecutive decision steps, it is considered stable tracking, and we receive +1 reward for each step; if the target's interference error continuously exceeds the threshold and it is determined to have lost its operational capability, we receive a one-time reward of +50. To enforce penalties, drones exceeding the operational airspace boundary are considered to have crossed the boundary; drones within 5 meters of each other are considered to have collided; each instance results in a deduction of 100 points. Communication / interference parameters exceeding the adjustable range are considered to have crossed the boundary constraint; each instance results in a deduction of 5 points.

[0083] Endgame Rewards The global reward at the end of the game round is calculated using the following formula:

[0084] (8)

[0085] In the formula This is a winning reward, triggered only at the end of the game round; the competitor gains +200 when all their drones are inoperable, and loses 200 when all of our drones are inoperable; this reward does not trigger if only some of our drones are inoperable. The number of drones that maintain our operational capability will be awarded a corresponding weighted reward for each drone maintained, with points calculated in real time based on the actual number. The number of drones that a competitor loses operational capability is used to determine the corresponding weighted reward for each drone that fails, and the score is calculated in real time based on the actual number. , The weighting coefficients are 15 and 15 respectively in this embodiment. The loss of operational capability is determined by a symmetrical judgment rule: when the perception error caused by interference continuously exceeds the preset threshold, or when a boundary crossing or collision occurs, the operational capability is determined to be lost. The interference effect on our side is calculated according to the rule symmetrical to equations (3)-(5). The interference parameters of the competitor are the fixed baseline values ​​of the simulation environment. The policy network only outputs the control actions of our UAV. During the training phase, the parameters of the competitor are the globally known information of the environment. During the online execution phase, the observable state of the competitor is only obtained through the perception module.

[0086] The above rewards and model training follow the standard MADDPG mechanism: the evaluation network outputs a one-dimensional Q value, which represents the expected value of the future cumulative reward corresponding to the current state-action pair; during training, the immediate reward and final reward calculated by formulas (6)-(8) are used as real supervision signals to construct the target Q value based on the Bellman equation, and the evaluation network parameters are updated by minimizing the prediction error.

[0087] (5) Model training: A simulation environment was built based on the Python and PyTorch frameworks to train hyperparameters: discount factor. Experience replay pool capacity 1,000,000, batch size 1024, policy network learning rate 1e-4, evaluation network learning rate 1e-3, soft update coefficient. The maximum number of training epochs is 50,000. During the offline training phase, the network is iteratively optimized based on global situational information. After training, the policy network is deployed to the onboard device, and during the online execution phase, only local information is needed to output decisions.

[0088] Step S5: Online execution of cluster game

[0089] Mission initialization: Our drone cluster and the competitor's drone cluster take off from the preset initial position, complete module self-test, frequency hopping sequence synchronization and communication networking;

[0090] Situational awareness and information sharing: The UAV acquires environmental and target information in real time and shares situational information with neighboring nodes within the communication range through an adaptive communication mechanism.

[0091] Intelligent decision output: The airborne decision module inputs situational information into the trained model and outputs real-time decision commands for flight, communication, jamming, and mission execution;

[0092] Action execution and closed-loop iteration: Each module executes corresponding instructions, synchronously completes adaptive communication adjustment, directional interference and core task execution actions, updates situation information in real time, enters the next decision cycle, until the game task ends.

[0093] The above description is only a preferred embodiment of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A cooperative game theory method for UAV swarms based on adaptive communication and directional interference, characterized in that, Includes the following steps: S1 Constructing a Cluster Game Scenarios and Basic Model: Delineating a three-dimensional operational airspace, configuring our own operational drone cluster and the competitor's drone cluster, and equipping each drone with a flight power module, situational awareness module, frequency hopping anti-interference wireless communication module, omnidirectional / directional switchable active interference module, and task execution module; S2 constructs a cluster adaptive distributed communication mechanism: sets a dynamically adjustable range for the communication radius of our UAVs, adjusts the communication radius and transmission power of a single UAV in real time based on the adaptive communication parameter optimization formula, and realizes encrypted sharing of situational information of friendly UAVs within the communication range by combining connectivity determination rules, while realizing cluster anti-interference communication through synchronous frequency hopping sequences. S3 constructs an adaptive active jamming and electromagnetic protection model: sets a dynamically adjustable range for the jamming radius of UAVs, supports switching between omnidirectional and directional jamming modes, and quantifies the impact of directional jamming gain and jamming signal on the situational awareness distance error of competitor UAVs; S4 Constructs a game-theoretic decision-making model that integrates attention mechanisms: taking real-time UAV situational information as state input, flight actions, communication parameter actions, interference parameter actions, and task triggering commands as action outputs, and communication-interference linkage constraint-type reward function as optimization objective, adopting the MADDPG algorithm that integrates multi-agent attention mechanisms, and completing model training based on a centralized training-distributed execution architecture; S5 online game task execution: The trained model is deployed to each of our drones. Based on real-time perception and shared information, the model outputs decision commands and simultaneously executes adaptive communication networking, directional interference suppression and task execution actions to complete the cluster game task.

2. The UAV swarm cooperative game method with adaptive communication and directional interference as described in claim 1, characterized in that, In step S2, the shared situational information includes the UAV's own three-dimensional position, flight speed, heading, remaining battery power, jamming module status, and the detected position and movement information of competitor UAVs; our cluster adopts a decentralized distributed communication architecture, with each UAV only interacting encrypted with friendly neighbor nodes within its communication range; when a UAV exceeds the communication range of all friendly neighbors, it switches to an autonomous independent operation mode, and automatically synchronizes the frequency hopping sequence and resumes cooperative communication after re-entering the communication range.

3. The UAV swarm cooperative game method based on adaptive communication and directional interference according to claim 1, characterized in that, In step S2, the dynamically adjustable range of the communication radius is 200m~800m, and the adaptive communication parameter optimization formula is calculated based on: the distance to the nearest friendly neighbor node, the current channel eavesdropping signal-to-noise ratio, the upper and lower limits of the transmission power, and the third-party eavesdropping risk level; the connectivity determination rule is based on the relationship between the distance between friendly UAVs and the communication radius. The frequency hopping sequence is generated using a pseudo-random sequence, and nodes within the cluster achieve precise alignment of the frequency hopping time with the frequency point through a synchronization beacon.

4. The UAV swarm cooperative game method with adaptive communication and directional interference as described in claim 1, characterized in that, In step S3, the dynamically adjustable range of the interference radius is 100m~500m, and the maximum interference radius does not exceed the current communication radius; the directional interference mode adopts beamforming technology, and the adjustable range of the interference beamwidth is 15°~90°, which only performs directional suppression on the selected competitor target. The directional interference gain is calculated based on the omnidirectional interference base gain, the effective aperture area of ​​the antenna, the wavelength of the interference signal, and the angle between the interference beam axis and the line connecting the UAV to the competitor target; the influence of situational awareness distance error is calculated based on the directional interference gain, the UAV interference power, the base error corresponding to the full power unit distance, the position coordinates of the UAV and the competitor target, and the interference coverage determination factor. When multiple friendly UAVs carry out coordinated interference on the same target, the interference error is linearly superimposed and does not exceed the preset maximum error limit.

5. The UAV swarm cooperative game method with adaptive communication and directional interference according to claim 1, characterized in that, In step S4, the multi-agent attention mechanism is used to assign weights to the shared neighbor information and the perceived competitor target information; in the offline training stage, the policy network and evaluation network are iteratively optimized based on global situation information; in the online execution stage, a single UAV outputs decisions based solely on its own perception and locally shared information.

6. The UAV swarm cooperative game method based on adaptive communication and directional interference according to claim 1, characterized in that, In step S4, the status input includes: ① its own status, including three-dimensional coordinates, three-dimensional velocity, heading angle, and remaining battery power; ② the status of neighboring UAVs within the communication range, including relative three-dimensional coordinates, relative three-dimensional velocity, heading angle, and remaining battery power; ③ the relative motion status of competing UAVs within the perception range, including relative three-dimensional coordinates and relative three-dimensional velocity; ④ the channel environment and eavesdropping risk status, including channel signal-to-noise ratio and eavesdropping risk level; wherein the perception range is a fixed detection radius of 300m for the airborne sensor, and the communication range and interference range are dynamically adjusted values; The action outputs include: ① Flight actions, including flight speed adjustment and heading adjustment; ② Communication parameter actions, including communication radius adjustment and transmit power adjustment, which are dynamically adjusted based on the real-time adjustment of the communication radius and transmit power of a single UAV in S2; ③ Interference parameter actions, including interference mode switching, interference power adjustment, interference azimuth angle, interference pitch angle, and beamwidth adjustment; ④ Mission trigger command, which is a binary mission start / stop control signal.

7. The UAV swarm cooperative game method based on adaptive communication and directional interference according to claim 1, characterized in that, In step S4, the communication-interference linkage constraint-type reward function is: ； Instant rewards The final reward is calculated once per decision step. Calculated only once at the end of the game round, and added to the total reward included in the final step. ; ； In the formula These are the weighting coefficients. For interference benefits, interference is considered effective if the target is within the interference radius and beam coverage area. Effective directional interference earns +3 base reward, and effective omnidirectional interference earns +1 base reward. For communication connectivity rewards, the cluster connectivity rate is calculated based on the ratio of the number of drones actually connected to the total number of drones. A +1 reward is given when the connectivity rate is ≥80%, and 0 is given when it is below 80%. In addition to the +1 or 0 reward based on the cluster connectivity rate, 1 point is deducted for each single node that is completely disconnected from all neighboring nodes. The points are accumulated for multiple disconnected nodes. For target tracking rewards, if a competitor's target remains within our perception range for three consecutive decision steps, it is considered stable tracking, and we receive +1 reward for each step; if the target's interference error continuously exceeds the threshold and it is determined to have lost its operational capability, we receive a one-time reward of +50. To enforce penalties, drones exceeding the operational airspace boundary are considered to have crossed the boundary; drones within 5 meters of each other are considered to have collided; each instance results in a deduction of 100 points. Communication / interference parameters exceeding the adjustable range are considered to have crossed the boundary constraint; each instance results in a deduction of 5 points. ； , These are the weighting coefficients. The winning reward is triggered only at the end of the game round; the competitor's drones lose all operational capability and the player's drones lose all operational capability and the player loses 200; this reward is not triggered if only some drones lose operational capability. The number of drones that maintain our operational capability will be awarded a corresponding weighted reward for each drone maintained, with points calculated in real time based on the actual number. The number of drones that lose operational capability for the competitor is used to determine the corresponding weighted reward for each drone that fails, and the score is calculated in real time based on the actual number. The loss of operational capability is determined by a symmetrical judgment rule: when the perception error caused by interference continuously exceeds the preset threshold, or when a boundary crossing or collision occurs, the operational capability is determined to be lost. The interference effect of our drones on the competitor is calculated according to the rule symmetrical to S3. The interference parameters of the competitor are fixed baseline values ​​in the simulation environment. The competitor's drone swarm adopts a traditional baseline operation strategy, with a fixed communication radius of 500m, a fixed omnidirectional interference mode, and a maneuver rule of preset area cruise and target tracking logic, without autonomous learning and parameter iteration capabilities.

8. A collaborative game-playing system for UAV swarms with adaptive communication and directional interference, characterized in that, The method for implementing the adaptive communication and directional jamming collaborative game of UAV swarms according to any one of claims 1-7 specifically includes our UAV swarm and training and deployment module; each UAV in the our UAV swarm is equipped with a flight power module, a situational awareness module, a frequency hopping anti-jamming wireless communication module, an omnidirectional / directional switchable active jamming module, a mission execution module and an airborne decision-making module; The frequency hopping anti-interference wireless communication module is used to execute an adaptive distributed communication mechanism to complete the encrypted sharing of friendly neighbor situation information and cluster anti-interference communication; The omnidirectional / directional switchable active jamming module is used to execute an adaptive active jamming model to complete directional / omnidirectional jamming suppression; The training and deployment module is used to perform offline training and airborne deployment of a game decision-making model that incorporates an attention mechanism; The airborne decision module is used to output real-time decision commands through the trained model to control the UAV to complete the game task.

Images