Control method and system based on cooperative game confrontation of multiple unmanned aerial vehicles

Abstract

The invention discloses a control method and system based on multi-unmanned aerial vehicle cooperative game confrontation, and belongs to the technical field of unmanned aerial vehicle control, and the method comprises the steps: S1, game environment modeling and simulation, S2, game strategy generation and optimization, S3, dynamic task allocation and cooperative control, S4, communication and information interaction optimization, and S5, efficiency evaluation and adaptive adjustment. On the basis of realizing cooperative game confrontation of multiple unmanned aerial vehicles, task allocation can be efficiently carried out on the unmanned aerial vehicles, and the autonomous decision-making capability of the unmanned aerial vehicles can be improved.

Description

Technical Field

[0001] The present invention relates to the field of drone control technology, and more specifically, to a control method and system based on multi-drone collaborative game confrontation. Background Art

[0002] Unmanned aerial vehicles (UAVs), also known as drones, are unmanned aircraft controlled by radio remote control and self-contained programmable controls. Drones are a general term for unmanned aerial vehicles (UAVs), which can be technically categorized into: unmanned helicopters, unmanned fixed-wing aircraft, unmanned multi-rotor aircraft, unmanned airships, and unmanned paragliders.

[0003] With advances in battlefield detection technology, data link technology, and intelligent algorithms, air combat has become more complex and multi-dimensional. Traditional single-aircraft combat is difficult to cope with dynamic confrontation environments due to information perception and firepower limitations. Drone swarms, through distributed collaboration and intelligent decision-making, have become a key direction for improving combat effectiveness. However, existing technologies have the following bottlenecks:

[0004] High communication pressure: The internal communication network of large-scale clusters is susceptible to interference, resulting in information delay or loss, affecting collaboration efficiency;

[0005] Inefficient task allocation: The traditional zero-sum game model has high computational complexity and is difficult to meet real-time requirements;

[0006] Lack of intelligence: Homogeneous clustering algorithms are difficult to adapt to heterogeneous scenarios and rely on manual rule design, resulting in poor flexibility. Summary of the Invention

[0007] 1. Technical problems to be solved

[0008] In response to the problems existing in the prior art, the purpose of the present invention is to provide a control method and system based on collaborative game confrontation of multiple UAVs. On the basis of realizing collaborative game confrontation of multiple UAVs, the present invention can also efficiently assign tasks to UAVs and increase the autonomous decision-making ability of UAVs.

[0009] 2. Technical solution

[0010] In order to solve the above problems, the present invention adopts the following technical solutions:

[0011] The control method based on multi-UAV cooperative game confrontation includes:

[0012] S1. Game environment modeling and simulation;

[0013] S2, game strategy generation and optimization;

[0014] S3, dynamic task allocation and collaborative control;

[0015] S4, Optimization of Communication and Information Interaction;

[0016] S5, Performance Evaluation and Adaptive Adjustment.

[0017] As a preferred embodiment of the present invention, the step S1 of game environment modeling and simulation includes:

[0018] S101, Definition of Adversarial Scenarios and Parameter Setting: Construct a heterogeneous UAV swarm confrontation simulation environment, clarify the UAV types of the red and blue sides, the differences in maneuverability, and the weight of the victory and defeat influence, define the three-dimensional space range, obstacle distribution, and physical constraints of dynamic interference, and set the initial positions, target points, and communication radii of the UAVs;

[0019] S102, Modeling of State Space and Action Space: The state space includes the UAV position, speed, remaining energy, enemy position, obstacle information, and target point state. The action space is designed with discrete or continuous actions including three-dimensional movement directions, attack, and interference instructions, which need to comply with physical dynamics constraints;

[0020] S103, Design of Reward Function; Adopt a sparse reward mechanism, combine goal achievement (getting closer to the target point +100 points), obstacle avoidance (collision -50 points), and confrontation effectiveness (weight of destroying the enemy), introduce Curriculum Learning to gradually increase the environmental complexity, and optimize the difficulty of reward function design.

[0021] The algorithms in the step S1 of game environment modeling and simulation include MARL multi-agent reinforcement learning, game theory methods, and distributed optimization algorithms;

[0022] The MARL multi-agent reinforcement learning includes the MAPPO algorithm and the Self-play algorithm. The MAPPO algorithm is based on a centralized training - decentralized execution framework, outputs actions through a policy network, and a critic network evaluates the global state, which is suitable for heterogeneous UAV cooperative control. The Self-play algorithm is optimized through adversarial iteration in the AlphaDogfight experiment;

[0023] The game theory methods include Nash equilibrium solution and mixed strategy game. The Nash equilibrium solution uses heuristic search to traverse action combinations and selects the strategy with the highest expected reward. The strategy can be Nash-Q learning. The mixed strategy game combines dynamic programming and Markov game to handle multi-objective conflicts, and the multi-objective conflicts include resource allocation and task priority.

[0024] The distributed optimization algorithm includes a genetic algorithm, a particle swarm optimization, and a contract net - Hungarian algorithm. The genetic algorithm and the particle swarm optimization are used for multi - objective trajectory planning to optimize the voyage, threat avoidance, and formation spacing. The contract net - Hungarian algorithm is used to realize the re - allocation of tasks for damaged UAVs, reduce communication costs, and improve response speed.

[0025] As a preferred embodiment of the present invention, the generation and optimization of the game strategy in step S2 include:

[0026] s201. Adversarial scenario modeling: Construct an adversarial environment including heterogeneous UAVs of the red and blue sides, define the UAV types, kinematic equations, and physical constraints. The UAV types include the leader, wingman, and jammer. Typical scenarios include central key - area defense and cooperative interception, and simulate battlefield detection, communication links, and weapon system characteristics;

[0027] s202. State - space design: The state - space includes the UAV position, speed, survival status, ammunition quantity (individual characteristics), enemy distribution, and base location (global information). Adopt a method of fusing local observations and global information, and screen key features through an attention mechanism;

[0028] s203. Strategy - generation algorithm design: Use a multi - agent reinforcement learning framework driven by game theory;

[0029] Among them, the multi - agent reinforcement learning framework includes:

[0030] MAPPO algorithm: Based on a centralized training - decentralized execution architecture, output action values through a policy network, and gradually increase the adversarial complexity in combination with curriculum learning;

[0031] Hierarchical reinforcement learning: Adopt a total - part - total structure. The top - level policy network generates global control information, and the bottom - level network makes action decisions in combination with individual perception to solve the problem of balancing individual and overall values;

[0032] MADDPG improvement: Introduce the idea of trembling - hand perfect equilibrium, enhance robustness through policy perturbation, and avoid local - optimal traps;

[0033] Among them, the game - theory - driven methods include:

[0034] Nash - equilibrium solution: Convert formation control into a distributed game problem, and realize UAV state synchronization through gradient descent and a consensus protocol;

[0035] Dynamic Bayesian network: Model the opponent's strategy - dependence relationship, predict the enemy's behavior, and adjust one's own strategy. As a preferred embodiment of the present invention, the dynamic task allocation and cooperative control in step S3 include:

[0036] S301. Task initialization and priority division: Read task information and UAV capability parameters, perform initial task allocation through the ISOM self-organizing mapping algorithm or coalition game model, and combine the attention mechanism for priority division to dynamically adjust weights according to the distance between the task points and the UAVs, ensuring that high-priority tasks are executed first;

[0037] S302. Distributed task allocation and conflict detection: Adopt a distributed coalition game framework, evaluate the utility of UAVs for tasks by calculating the marginal contribution of the Shapley value, achieve dynamic task reallocation, and use conflict detection algorithms to identify path conflicts or resource competition among UAVs;

[0038] S303. Dynamic adjustment and fault tolerance mechanism: Introduce the attention mechanism and LSTM network to process local observation information under limited communication, compensate for environmental changes during task execution, and optimize the allocation scheme in real time through the hybrid particle swarm optimization algorithm.

[0039] As a preferred solution of the present invention, the communication and information interaction optimization in step S4 includes:

[0040] S401. Dynamic hierarchical network communication architecture: Adopt a hierarchical communication mode, process time series observation data through the LSTM network, generate compact communication messages, and reduce redundant information transmission;

[0041] S402. Ad hoc network protocol: Support dynamic routing and multiple access, combine Q-learning to optimize route selection, and cope with high-dynamic topology changes;

[0042] S403. Information compression and efficient transmission: Use the LSTM network to encode local observation data, generate low-dimensional feature vectors, and screen key information through the attention mechanism. Introduce an explicit communication mechanism, such as a game theory-based intention sharing model, to only transmit the core parameters required for decision-making;

[0043] S404. Dynamic topology management and anti-interference optimization: Adopt the Raft protocol to achieve rapid topology reconstruction. When some nodes fail, maintain the communication link through redundant design, combine edge computing to reduce the data transmission volume, and use dynamic spectrum access technology to avoid electromagnetic interference.

[0044] As a preferred solution of the present invention, the effectiveness evaluation and adaptive adjustment in step S5 includes:

[0045] S501. Multidimensional index design: Establish a four-level quantitative index system including task effectiveness, adaptability, survivability, and cooperation. The specific indexes include: task completion speed, target strike accuracy, communication stability, cluster loss rate, and cross-domain data delay;

[0046] S502. Data Acquisition and Feature Modeling: The Mahalanobis distance modeling method is adopted to solve the problem that the traditional Euclidean distance is insensitive to data dispersion. By calculating the covariance matrix distance between the effectiveness of each node and the expected value, the effectiveness distribution characteristics of the heterogeneous network are characterized. A multi-source fusion database is constructed by combining battlefield environment sensor data, where the sensors include radar and infrared cameras;

[0047] S503. Evaluation Model Training Using the PSO-BP Neural Network Architecture:

[0048] S504. Dynamic Effectiveness Monitoring and Situation Awareness:

[0049] S50401. Real-time Data Stream Processing: An LSTM network is deployed to process time-series observation data. Information loss caused by communication delay is compensated through time-series prediction. Each UAV maintains a local observation memory bank to store sensor data of the last 50 time nodes;

[0050] S50402. Distributed Situation Awareness Fusion Using the Multi-Agent System (MAS) Architecture;

[0051] ]>S505. Adaptive Control Strategy Generation:

[0052] S50501. Constructing a Total-Subtotal-Total Strategy Network:

[0053] S50502. Introducing an improved NSGA-Ⅲ algorithm for dynamic task reallocation, where the objective function is min{f 1( (energy consumption), f 2( (time), f 3( (risk)}, and the constraint conditions are that the maximum turning angle of the UAV ≤ 30°, and the speed mutation threshold ≤ 5 m / s 2 , and the optimal allocation scheme is selected through the Pareto front solution set.

[0054] As a preferred embodiment of the present invention, the PSO-BP neural network architecture in step S503 includes:

[0055] Input layer: The hierarchical network effectiveness value calculated by Mahalanobis distance;

[0056] Hidden layer: Optimizing the weight threshold by the particle swarm algorithm to solve the problem of slow convergence of the traditional BP network;

[0057] Output layer: Generating the overall effectiveness evaluation value;

[0058] During the training process, the battlefield space is segmented by the Voronoi diagram, and the sample acquisition path is optimized by combining with the Dijkstra algorithm.

[0059] As a preferred embodiment of the present invention, the multi-agent system MAS architecture in step S50402 includes:

[0060] Interference UAV node: responsible for electromagnetic spectrum situation awareness;

[0061] Strike UAV node: execute target motion trajectory prediction;

[0062] Communication relay node: build a blue-green laser VHF hybrid communication link;

[0063] Cross-domain data fusion is achieved through federated learning to reduce communication bandwidth requirements.

[0064] As a preferred embodiment of the present invention, the overall-partial-overall type policy network includes:

[0065] Top-level policy network: generate collaborative constraints based on the global state;

[0066] Middle-level decision-making network: use the MADDPG algorithm to process local observation information;

[0067] Bottom-level execution network: output specific control instructions including heading angle and speed;

[0068] The reward function design includes a time penalty term: R t = α·S task - β·Δt.

[0069] A control system based on multi-UAV cooperative game confrontation includes: a perception layer, a decision-making layer, an execution layer, a communication layer, and a simulation verification platform;

[0070] The perception layer includes a perception fusion module and a dynamic threat assessment module. The perception fusion module realizes environmental perception through multi-modal sensor fusion, and the dynamic threat assessment module analyzes the behavior of enemy targets in real time based on machine learning;

[0071] The decision-making layer includes a game strategy engine and a task allocation module. The game strategy engine integrates swarm intelligence algorithms to generate cooperative confrontation strategies, and the task allocation module uses a distributed optimization algorithm to process multi-objective allocation problems and supports dynamic priority adjustment;

[0072] The execution layer includes a flight control module and a weapon interference module. The flight control module realizes accurate attitude and trajectory tracking based on a robust control algorithm, and the weapon / interference system integrates electromagnetic interference and laser weapon confrontation means;

[0073] The communication layer includes a distributed communication network and an information fusion center. The distributed communication network supports low-latency data exchange and a fault tolerance mechanism, and the information fusion center is used to process multi-source heterogeneous data and generate a global situation map;

[0074] The simulation and verification platform includes an adversarial simulation environment and an effectiveness evaluation module. The adversarial simulation environment is used to test the algorithm performance. The adversarial simulation environment is the linkage of Matlab / Flightgear. The effectiveness evaluation system is used to quantitatively analyze the task completion rate and energy consumption index.

[0075] 3. Beneficial effects

[0076] Compared with the prior art, the advantages of the present invention are as follows:

[0077] Through the distributed cooperative control and game model, the cluster can dynamically adjust the formation, adapt to the complex battlefield environment, reduce the risk of single-point failure. The self-game training based on reinforcement learning simplifies the design of the reward function and improves the decision-making response speed.

[0078] Through the hybrid control structure combined with redundant design, the function can still be maintained when 80% of the communication links are lost. The leader-follower architecture realizes complementary functions and improves the overall survival rate. Through intelligent task allocation, redundant UAV deployment is reduced, combat costs are saved, and autonomous situation assessment and path planning reduce the operation complexity, especially with significant advantages in the electromagnetic interference environment. Brief description of the drawings

[0079] Figure 1 It is a flowchart of the control method based on multi-UAV cooperative game and confrontation of the present invention;

[0080] Figure 2 It is a schematic diagram of the modules of the control system based on multi-UAV cooperative game and confrontation of the present invention. Specific embodiments

[0081] The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0082] Embodiment:

[0083] Please refer to Figure 1 , the control method based on multi-UAV cooperative game and confrontation includes:

[0084] S1. Game environment modeling and simulation;

[0085] S2. Game strategy generation and optimization;

[0086] S3. Dynamic task allocation and cooperative control;

[0087] S4. Communication and Information Interaction Optimization;

[0088] S5. Performance Evaluation and Adaptive Adjustment.

[0089] In a specific embodiment of the present invention, through a distributed cooperative control and game model, the cluster can dynamically adjust the formation, adapt to complex battlefield environments, reduce the risk of single-point failure. The self-play training based on reinforcement learning simplifies the design of the reward function and improves the decision-making response speed. Through a hybrid control structure combined with redundant design, the function can still be maintained when 80% of the communication links are lost. The leader-follower architecture realizes complementary functions and improves the overall survival rate. Through intelligent task allocation, redundant UAV deployments are reduced, combat costs are saved, and autonomous situation assessment and path planning reduce the operation complexity, especially with significant advantages in an electromagnetic interference environment.

[0090] Specifically, the game environment modeling and simulation in step S1 includes:

[0091] S101. Adversarial Scenario Definition and Parameter Setting: Construct a heterogeneous UAV cluster confrontation simulation environment, clarify the differences in UAV types, maneuverability, and the weight of victory and defeat influence between the red and blue sides, define the three-dimensional space range, obstacle distribution, and physical constraints of dynamic interference, and set the initial positions, target points, and communication radii of the UAVs;

[0092] S102. State Space and Action Space Modeling: The state space includes UAV position, speed, remaining energy, enemy position, obstacle information, and target point status. The action space is designed with discrete or continuous actions including three-dimensional movement directions, attack, and interference instructions, which need to conform to physical dynamics constraints;

[0093] S103. Reward Function Design; Adopt a sparse reward mechanism, combine goal achievement (such as approaching the target point +100 points), obstacle avoidance (collision -50 points), and confrontation effectiveness (weight of destroying the enemy), and introduce Curriculum Learning to gradually increase the environmental complexity and optimize the difficulty of reward function design.

[0094] The algorithms in the game environment modeling and simulation of step S1 include MARL multi-agent reinforcement learning, game theory methods, and distributed optimization algorithms;

[0095] MARL multi-agent reinforcement learning includes the MAPPO algorithm and the Self-play algorithm. The MAPPO algorithm is based on a centralized training - decentralized execution framework, outputs actions through a policy network, and the critic network evaluates the global state, which is suitable for heterogeneous UAV cooperative control. The Self-play algorithm optimizes through adversarial iteration in the AlphaDogfight experiment to improve the robustness of the policy;

[0096] The game theory method includes Nash equilibrium solving and mixed strategy games. Nash equilibrium solving uses heuristic search to traverse action combinations and selects the strategy with the highest expected reward, where the strategy can be Nash-Q learning. The mixed strategy game combines dynamic programming and Markov games to handle multi-objective conflicts, which include resource allocation and task priority.

[0097] The distributed optimization algorithms include genetic algorithms, particle swarm optimization, and contract net-Hungarian algorithms. Genetic algorithms and particle swarm optimization are used for multi-objective trajectory planning to optimize the voyage, threat avoidance, and formation spacing. The contract net-Hungarian algorithm is used to achieve the reallocation of tasks for damaged UAVs, reducing communication costs and improving response speed.

[0098] Specifically, step S2, game strategy generation and optimization, includes:

[0099] s201, adversarial scenario modeling: Construct an adversarial environment containing heterogeneous UAVs of the red and blue sides, define UAV types, kinematic equations, and physical constraints. The UAV types include leader aircraft, wingman aircraft, and jammer aircraft. Typical scenarios include central key area defense and cooperative interception, and simulate battlefield detection, communication links, and weapon system characteristics;

[0100] s202, state space design: The state space includes UAV position, speed, survival status, ammunition quantity, individual characteristics, enemy distribution, and global information of the base location. Adopt a method of fusing local observations and global information, and screen key features through an attention mechanism;

[0101] s203, strategy generation algorithm design: Use a multi-agent reinforcement learning framework and be driven by game theory;

[0102] Among them, the multi-agent reinforcement learning framework includes:

[0103] MAPPO algorithm: Based on a centralized training - decentralized execution architecture, output action values through a policy network, and gradually increase the adversarial complexity in combination with curriculum learning;

[0104] Hierarchical reinforcement learning: Adopt a total - sub - total structure. The top - layer policy network generates global control information, and the bottom - layer network makes action decisions in combination with individual perception to solve the problem of balancing individual and overall values;

[0105] MADDPG improvement: Introduce the idea of trembling hand perfect equilibrium, enhance robustness through policy perturbation, and avoid local optimal traps;

[0106] Among them, the game theory - driven methods include:

[0107] Nash equilibrium solving: Convert formation control into a distributed game problem, and achieve UAV state synchronization through gradient descent and consensus protocol;

[0108] Dynamic Bayesian Network: Modeling the opponent's strategy dependencies, predicting the enemy's behavior, and adjusting one's own strategy.

[0109] Specifically, step S3 of dynamic task allocation and collaborative control includes:

[0110] S301. Task initialization and priority division: Read the task information and the drone's capability parameters, perform initial task allocation through the ISOM self-organizing mapping algorithm or the coalition game model, and combine the attention mechanism for priority division. Dynamically adjust the weights according to the distance between the task points and the drones to ensure that high-priority tasks are executed first;

[0111] S302. Distributed task allocation and conflict detection: Adopt a distributed coalition game framework, evaluate the utility of drones for tasks by calculating the marginal contribution of the Shapley value, achieve dynamic task reallocation, and use conflict detection algorithms to identify path conflicts or resource competition among drones;

[0112] S303. Dynamic adjustment and fault tolerance mechanism: Introduce the attention mechanism and the LSTM network to process local observation information under limited communication, compensate for environmental changes during task execution, and optimize the allocation scheme in real time through the hybrid particle swarm optimization algorithm.

[0113] Specifically, step S4 of communication and information interaction optimization includes:

[0114] S401. Dynamic hierarchical network communication architecture: Adopt a hierarchical communication mode, process time series observation data through the LSTM network, generate compact communication messages, and reduce redundant information transmission;

[0115] S402. Ad hoc network protocol: Support dynamic routing and multiple access, optimize routing selection by combining Q-learning to cope with high-dynamic topology changes;

[0116] S403. Information compression and efficient transmission: Use the LSTM network to encode local observation data, generate low-dimensional feature vectors, and screen key information through the attention mechanism. Introduce an explicit communication mechanism, such as a game theory-based intention sharing model, and only transmit the core parameters required for decision-making;

[0117] S404. Dynamic topology management and anti-interference optimization: Adopt the Raft protocol to achieve fast topology reconstruction. When some nodes fail, maintain the communication link through redundant design, reduce the data transmission volume by combining edge computing, and use dynamic spectrum access technology to avoid electromagnetic interference.

[0118] Specifically, step S5 of effectiveness evaluation and adaptive adjustment includes:

[0119] S501. Multi-dimensional Indicator Design: Establish a four-level quantitative indicator system encompassing mission effectiveness (40%), adaptability (20%), survivability (20%), and collaboration (20%). Specific indicators include: mission completion speed (number of coordinated attacks), target strike accuracy (≥95% for excellence), communication stability (anti-interference capability), cluster loss rate (≤10% for pass), and cross-domain data latency (≤200ms for excellence).

[0120] S502, Data Collection and Feature Modeling: The Mahalanobis distance modeling method is used to address the problem that traditional Euclidean distance is insensitive to data dispersion. By calculating the covariance matrix distance between each node's performance and the expected value, the performance distribution characteristics of the heterogeneous network are characterized. A multi-source fusion database is constructed by combining battlefield environment sensor data (radar, infrared, etc.), where the sensors include radar and infrared cameras.

[0121] S503. Use the PSO-BP neural network architecture to evaluate model training:

[0122] S504, Dynamic Performance Monitoring and Situational Awareness:

[0123] S50401, Real-time Data Stream Processing: Deploy an LSTM network to process time-series observation data, using time series prediction to compensate for information loss caused by communication delays. Each drone maintains a local observation memory, storing sensor data from the most recent 50 time points.

[0124] S50402, using the multi-agent system MAS architecture distributed situational awareness fusion;

[0125] S505, adaptive control strategy generation:

[0126] S50501. Construct a master-detail-master strategy network:

[0127] S50502, introduce the improved NSGA-Ⅲ algorithm for dynamic task reallocation, where the objective function is min{f 1( Energy consumption),f 2( time),f 3( The constraints are that the maximum turning angle of the drone is ≤30° and the speed mutation threshold is ≤5m / s. 2 And screen the optimal allocation plan through the Pareto frontier solution set.

[0128] Specifically, the PSO-BP neural network architecture in step S503 includes:

[0129] Input layer: hierarchical network performance value calculated by Mahalanobis distance;

[0130] Hidden layer: The particle swarm optimization algorithm is used to optimize the weight thresholds to solve the problem of slow convergence of traditional BP networks;

[0131] Output layer: Generate the overall effectiveness evaluation value;

[0132] During the training process, the Voronoi diagram is used to divide the battlefield space, and the Dijkstra algorithm is combined to optimize the sample collection path.

[0133] Specifically, the multi-agent system MAS architecture in step S50402 includes:

[0134] Jamming UAV node: Responsible for electromagnetic spectrum situation awareness;

[0135] Strike UAV node: Execute target motion trajectory prediction;

[0136] Communication relay node: Construct a blue-green laser VHF hybrid communication link;

[0137] Cross-domain data fusion is achieved through federated learning, reducing the communication bandwidth requirement by more than 40%.

[0138] Specifically, the overall-to-detail policy network includes:

[0139] Top-level policy network: Generate collaborative constraints based on the global state;

[0140] Middle-level decision-making network: Use the MADDPG algorithm to process local observation information;

[0141] Bottom-level execution network: Output specific control instructions including heading angle and speed;

[0142] The reward function design includes a time penalty term: R t = α·S task - β·Δt.

[0143] Please refer to Figure 2 , The control system based on multi-UAV collaborative game confrontation includes: a perception layer, a decision-making layer, an execution layer, a communication layer, and a simulation verification platform;

[0144] The perception layer includes a perception fusion module and a dynamic threat assessment module. The perception fusion module realizes environmental perception through multi-modal sensor fusion, and the dynamic threat assessment module analyzes the behavior of enemy targets in real time based on machine learning;

[0145] The decision-making layer includes a game strategy engine and a task allocation module. The game strategy engine integrates swarm intelligence algorithms to generate collaborative confrontation strategies, and the task allocation module uses a distributed optimization algorithm to handle multi-objective allocation problems and supports dynamic priority adjustment;

[0146] The execution layer includes a flight control module and a weapon interference module. The flight control module achieves precise attitude and trajectory tracking based on a robust control algorithm, and the weapon / interference system integrates electromagnetic interference and laser weapon countermeasure means;

[0147] The communication layer includes a distributed communication network and an information fusion center. The distributed communication network supports low-latency data exchange and a fault-tolerant mechanism, and the information fusion center is used to process multi-source heterogeneous data and generate a global situation map;

[0148] The simulation verification platform includes a confrontation simulation environment and an effectiveness evaluation module. The confrontation simulation environment is used to test the algorithm performance. The confrontation simulation environment is a linkage between Matlab / Flightgear, and the effectiveness evaluation system is used to quantitatively analyze the task completion rate and energy consumption indicators.

[0149] The above is only a preferred specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art within the technical scope disclosed by the present invention, according to the technical solution of the present invention and its improved concept, makes equivalent substitutions or changes, and should be covered by the protection scope of the present invention.

Claims

1. A control method based on multi-UAV collaborative game confrontation, characterized in that Including: S1, Game environment modeling and simulation; S2, Game strategy generation and optimization; S3, Dynamic task allocation and cooperative control; S4, Communication and information interaction optimization; S5, Effectiveness evaluation and adaptive adjustment.

2. The control method based on multi-UAV collaborative game confrontation according to claim 1, wherein The step S1 of game environment modeling and simulation includes: S101, Adversarial scenario definition and parameter setting: Construct a heterogeneous UAV cluster confrontation simulation environment, clarify the differences in UAV types, maneuverability, and the weight of victory and defeat influence between the red and blue sides, define the three-dimensional space range, obstacle distribution, and dynamic interference physical constraints, and set the initial positions, target points, and communication radii of the UAVs; S102, State space and action space modeling: The state space includes UAV position, speed, remaining energy, enemy position, obstacle information, and target point state, and the action space designs discrete or continuous actions including three-dimensional movement directions, attack, and interference instructions, which need to comply with physical dynamics constraints; S103, Reward function design; Adopt a sparse reward mechanism, combine goal achievement (getting closer to the target point +100 points), obstacle avoidance (collision -50 points), and confrontation effectiveness (weight of destroying the enemy), introduce Curriculum Learning to gradually increase the environmental complexity, and optimize the difficulty of reward function design. The algorithms in the step S1 of game environment modeling and simulation include MARL multi-agent reinforcement learning, game theory methods, and distributed optimization algorithms; The MARL multi-agent reinforcement learning includes the MAPPO algorithm and the Self-play algorithm. The MAPPO algorithm is based on the centralized training - decentralized execution framework, outputs actions through the policy network, and the critic network evaluates the global state, and is applicable to heterogeneous UAV cooperative control. The Self-play algorithm is optimized through adversarial iteration in the AlphaDogfight experiment; The game theory methods include Nash equilibrium solution and mixed strategy game. The Nash equilibrium solution uses heuristic search to traverse action combinations and selects the strategy with the highest expected reward. The strategy can be Nash-Q learning. The mixed strategy game combines dynamic programming and Markov game to handle multi-objective conflicts, and the multi-objective conflicts include resource allocation and task priority. The distributed optimization algorithms include genetic algorithm and particle swarm optimization and contract net - Hungarian algorithm. The genetic algorithm and particle swarm optimization are used for multi-objective trajectory planning to optimize the flight range, threat avoidance, and formation spacing. The contract net - Hungarian algorithm is used to realize the reallocation of tasks of damaged UAVs, reduce communication costs, and improve response speed.

3. The control method based on multi-UAV collaborative game confrontation according to claim 2, wherein The step S2 of game strategy generation and optimization includes: s201, Adversarial scenario modeling: Construct an adversarial environment containing heterogeneous UAVs of the red and blue sides, define UAV types, kinematic equations, and physical constraints. The UAV types include leader aircraft, wingman aircraft, and jammer aircraft. Typical scenarios include central key area defense and cooperative interception, and simulate battlefield detection, communication links, and weapon system characteristics; S202. State Space Design: The state space includes the UAV's position, speed, survival status, individual characteristics of ammunition quantity, global information of enemy distribution and base location. A local observation and global information fusion method is adopted, and key features are screened through an attention mechanism. S203. Policy Generation Algorithm Design: Use a multi-agent reinforcement learning framework and be driven by game theory. Among them, the multi-agent reinforcement learning framework includes: MAPPO Algorithm: Based on a centralized training - decentralized execution architecture, the action value is output through a policy network, and the adversarial complexity is gradually increased by combining curriculum learning. Hierarchical Reinforcement Learning: Adopt a total - sub - total structure. The top - level policy network generates global control information, and the bottom - level network makes action decisions in combination with individual perception to solve the problem of balancing individual and overall value. MADDPG Improvement: Introduce the idea of trembling - hand perfect equilibrium, enhance robustness through policy perturbation, and avoid local optimal traps. Among them, the game - theory - driven methods include: Nash Equilibrium Solving: Transform formation control into a distributed game problem, and achieve UAV state synchronization through gradient descent and consensus protocol. Dynamic Bayesian Network: Model the opponent's strategy dependence relationship, predict the enemy's behavior, and adjust one's own strategy.

4. The control method based on multi-UAV collaborative game confrontation according to claim 3, wherein The step S3 of dynamic task allocation and cooperative control includes: S301. Task Initialization and Priority Division: Read task information and UAV capability parameters, perform initial task allocation through the ISOM self - organizing mapping algorithm or coalition game model, and combine the attention mechanism for priority division. Dynamically adjust the weight according to the distance between the task point and the UAV to ensure that high - priority tasks are executed first. S302. Distributed Task Allocation and Conflict Detection: Adopt a distributed coalition game framework, evaluate the utility of UAVs for tasks by calculating the marginal contribution of the Shapley value, achieve dynamic task re - allocation, and use conflict detection algorithms to identify path conflicts or resource competition among UAVs. S303. Dynamic Adjustment and Fault - Tolerance Mechanism: Introduce the attention mechanism and LSTM network to process local observation information under limited communication, compensate for environmental changes during task execution, and optimize the allocation scheme in real - time through a hybrid particle swarm optimization algorithm.

5. The control method based on multi-UAV collaborative game confrontation according to claim 4, characterized in that The step S4 of communication and information interaction optimization includes: S401. Dynamic Hierarchical Network Communication Architecture: Adopt a hierarchical communication mode, process time - series observation data through the LSTM network, generate compact communication messages, and reduce redundant information transmission. S402. Ad Hoc Network Protocol: Support dynamic routing and multiple access, optimize route selection by combining Q - learning, and cope with high - dynamic topology changes. S403. Information Compression and Efficient Transmission: Use the LSTM network to encode local observation data, generate low - dimensional feature vectors, and screen key information through the attention mechanism. Introduce an explicit communication mechanism, such as a game - theory - based intention sharing model, and only transmit the core parameters required for decision - making. S404. Dynamic Topology Management and Anti - Interference Optimization: Adopt the Raft protocol to achieve fast topology reconstruction. When some nodes fail, maintain the communication link through redundant design, reduce the data transmission volume by combining edge computing, and use dynamic spectrum access technology to avoid electromagnetic interference.

6. The control method based on multi-UAV collaborative game confrontation according to claim 5, wherein The performance evaluation and adaptive adjustment in step S5 include: S501, Multi-dimensional index design: Establish a four-level quantitative index system including mission effectiveness, adaptability, survivability, and cooperation. The specific indicators include: mission completion speed, target strike accuracy, communication stability, cluster loss rate, and cross-domain data latency; S502, Data collection and feature modeling: Use the Mahalanobis distance modeling method to solve the problem that the traditional Euclidean distance is insensitive to data dispersion. By calculating the covariance matrix distance between the effectiveness of each node and the expected value, the effectiveness distribution characteristics of heterogeneous networks are characterized. Combine battlefield environment sensor data to construct a multi-source fusion database, where the sensors include radar and infrared cameras; S503, Use the PSO-BP neural network architecture to evaluate model training: S504, Dynamic effectiveness monitoring and situation awareness: S50401, Real-time data stream processing: Deploy an LSTM network to process time-series observation data, compensate for information loss caused by communication delay through time-series prediction. Each UAV maintains a local observation memory bank to store sensor data of the last 50 time nodes; S50402, Adopt the multi-agent system MAS architecture for distributed situation awareness fusion; S505, Generation of adaptive control strategies: S50501, Construct a total-subtotal-total type policy network: S50502. Introduce the improved NSGA-Ⅲ algorithm for dynamic task reallocation, where the objective function is min{f 1( energy consumption), f 2( time), f 3( risk)}, and the constraint conditions are that the maximum turning angle of the UAV ≤ 30°, and the speed mutation threshold ≤ 5 m / s 2 , and screen the optimal allocation scheme through the Pareto front solution set.

7. The control method based on multi-UAV collaborative game confrontation according to claim 6, wherein The PSO-BP neural network architecture in step S503 includes: Input layer: The hierarchical network effectiveness value calculated by Mahalanobis distance; Hidden layer: The particle swarm algorithm optimizes the weight threshold to solve the problem of slow convergence of the traditional BP network; Output layer: Generate the overall effectiveness evaluation value; In the training process, the battlefield space is segmented by the Voronoi diagram, and the sample acquisition path is optimized by combining the Dijkstra algorithm.

8. The control method based on multi-UAV collaborative game confrontation according to claim 7, wherein The multi-agent system MAS architecture in step S50402 includes: Interference UAV node: Responsible for electromagnetic spectrum situation awareness; Strike UAV node: Execute target motion trajectory prediction; Communication relay node: Construct a blue-green laser VHF hybrid communication link; Cross-domain data fusion is achieved through federated learning to reduce communication bandwidth requirements.

9. The control method based on multi-UAV collaborative game confrontation according to claim 7, characterized in that, The total-subtotal-total type policy network includes: Top-level policy network: Generate cooperative constraints based on the global state; Middle-level decision-making network: Use the MADDPG algorithm to process local observation information; Bottom-level execution network: Output specific control instructions including heading angle and speed; The reward function design includes a time penalty term: R t = α·S task - β·Δt.

10. A control system based on multi-UAV collaborative game confrontation, applied to the control method based on multi-UAV collaborative game confrontation described in claims 1-9, characterized in that, Includes a perception layer, a decision-making layer, an execution layer, a communication layer, and a simulation verification platform; The perception layer includes a perception fusion module and a dynamic threat assessment module. The perception fusion module realizes environmental perception through multi-modal sensor fusion, and the dynamic threat assessment module analyzes the behavior of enemy targets in real time based on machine learning; The decision-making layer includes a game strategy engine and a task allocation module. The game strategy engine integrates swarm intelligence algorithms to generate cooperative confrontation strategies, and the task allocation module uses a distributed optimization algorithm to process multi-target allocation problems and supports dynamic priority adjustment; The execution layer includes a flight control module and a weapon interference module. The flight control module realizes precise attitude and trajectory tracking based on a robust control algorithm. The weapon / jamming system integrates electromagnetic interference and laser weapon countermeasure means; The communication layer includes a distributed communication network and an information fusion center. The distributed communication network supports low-latency data exchange and a fault-tolerant mechanism. The information fusion center is used to process multi-source heterogeneous data and generate a global situation map; The simulation and verification platform includes a confrontation simulation environment and an effectiveness evaluation module. The confrontation simulation environment is used to test the algorithm performance. The confrontation simulation environment is a linkage of Matlab / Flightgear. The effectiveness evaluation system is used to quantitatively analyze the task completion rate and energy consumption indicators.

Images