Distributed unmanned aerial vehicle cooperative defense intention coordination method and training method

The distributed UAV cooperative defense method addresses inefficiencies in existing systems by using a distributed Markov process and neural networks with attention mechanisms for intent processing, enhancing scalability and conflict resolution in UAV coordination.

CN119065398APending Publication Date: 2024-12-03TONGJI UNIV
View PDF 0 Cites 2 Cited by

Patent Information

Application Number
CN202411084350.0
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-08
Publication Date
2024-12-03

AI Technical Summary

Technical Problem

Existing multi-UAV cooperative defense systems face challenges in coordinating drone intentions efficiently, particularly in large-scale UAV clusters, and require scalable solutions that can handle complex dynamic battlefield environments while minimizing computational demands on central nodes.

Method used

A distributed UAV cooperative defense intention coordination method is developed, utilizing a distributed Markov process to decouple target allocation and one-to-one interception strategies, incorporating a neural network with an attention mechanism for intent processing and a conflict resolution mechanism, trained through reinforcement and imitation learning to enhance scalability and generalization.

Benefits of technology

The method improves target allocation conflict resolution capabilities and reduces computational demands on central nodes, ensuring efficient coordination of UAVs in dynamic environments with better generalization performance.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119065398A_ABST
    Figure CN119065398A_ABST
Patent Text Reader

Abstract

The invention relates to a distributed unmanned aerial vehicle cooperative defense intention coordination method and training method, and the method comprises the steps: modeling a multi-unmanned aerial vehicle cooperative defense problem into a distributed Markov process, and training a bottom layer one-to-one strategy through deep reinforcement learning, and taking the bottom layer one-to-one strategy as a bottom layer strategy of a target distributor concerned by the method; a neural network which is based on a self-attention mechanism and is combined with a conflict resolution mechanism is adopted as a centralized target distributor, and a centralized strategy with conflict resolution capability is obtained through reinforcement learning training; extensible distributed intention fusion based on the attention mechanism is constructed; based on data generated by a centralized policy, distributed target allocation is trained by imitation learning. Compared with the prior art, the method has the advantages that the problem of expandability of the number of the unmanned aerial vehicles is solved, so that the method has better generalization performance, the target allocation conflict resolution capability under the distributed condition is remarkably improved, and the overall performance is improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention belongs to the field of multi-UAV cooperative control, and in particular relates to a distributed UAV cooperative defense intention coordination method and training method. Background Art

[0002] Drone coordinated defense is an effective means of countering drone swarm attacks. Its task is to control the drone's posture and shoot down the invading drone before it enters the target area. The key to this problem is to coordinate the drone's intention through implicit or explicit communication, and then control the drone's maneuvering according to the intention. The difficulty lies in taking into account all invading drones, maximizing the efficiency of drone coordination, and completing real-time reasoning to cope with highly dynamic battlefield environments.

[0003] The problem of multi-UAV cooperative control in confrontation scenarios can be divided into four technical routes, namely game theory, expert strategy, intelligent optimization algorithm and deep multi-agent reinforcement learning. The technical route based on game theory can solve the optimal lower bound, but cannot handle complex dynamic models; the method based on expert strategy can integrate prior knowledge, but lacks existing expert knowledge for the coordination of large-scale UAV clusters and lacks the generalization performance of the scene; the method based on intelligent optimization algorithm parameterizes the strategy and optimizes the strategy through certain heuristic rules. It is a data-driven algorithm that can be effectively combined with expert strategy, but it does not utilize the Markov property of the problem and is less efficient.

[0004] In the UAV collaborative defense problem, because UAVs may be damaged during combat, the design algorithm needs to be scalable to the number of UAVs. In order to cope with the complexity of the problem, a layered technical architecture is usually adopted, that is, the upper layer adopts target allocation and the lower layer adopts a one-to-one strategy. Target allocation methods can be divided into centralized and distributed methods. The centralized architecture is conducive to conflict resolution. In the prior art, Zhao et al. proposed a centralized target allocation method in their paper "Learning Scalable Task Assignment with Imperative-Priori Conflict Resolution in Multi-UAVAdversarial Swarm Defense Problem". However, the centralized architecture needs to consider the switching of the central node and has high requirements on the computing power of the central node. Summary of the invention

[0005] The purpose of the present invention is to overcome the defects of the above-mentioned prior art and provide a distributed UAV collaborative defense intention coordination method and training method.

[0006] The purpose of the present invention can be achieved by the following technical solutions:

[0007] The present invention provides a distributed UAV cooperative defense intention coordination method, comprising the following steps:

[0008] The present invention provides a distributed UAV cooperative defense intention coordination method, comprising the following steps:

[0009] The collaborative defense problem is modeled as a distributed Markov process. Under the distributed Markov process, the UAV distributed control strategy is decoupled into a distributed target allocation strategy and an underlying one-to-one interception strategy. The observation information of all defending UAVs is preprocessed to obtain observation pairs. All observation pairs are input into a distributed target allocation strategy network based on an attention mechanism for processing. The distributed target allocation strategy network outputs the unique target intruder ID of the current defending UAV. The underlying one-to-one interception strategy makes decisions based on the unique target intruder ID, and the defending UAV defends based on the decision.

[0010] The distributed Markov process includes a state space, an action space and a state transfer function, wherein the state transfer function is used to update the drone posture according to kinematic constraints and to update the drone survival state according to the countermeasure influence rule.

[0011] The state space

[0012] in Indicates the drone's survival status. Indicates the spatial position, represents the velocity component, the action space Defined in discrete form, the reward function Represents the group reward function, which is set as follows: Base destruction penalty: When the base is destroyed, a penalty of -100n is given, where n is the number of invaders. Elimination of invaders reward: Eliminate any new attacker and get a +100 reward. Invader proximity penalty: Each time step is based on the distance of the invader to the base. The reward, where R attacker,init is the average distance from the attacker’s initial position to the base, R base is the radius of the base, dist(j,b) is the distance between attacker j and base b, Δr j is the change in distance to the base in the previous time step, and the state transfer function is defined as It includes updating the drone's posture according to kinematic constraints and updating the drone's survival status according to countermeasure impact rules.

[0013] The defensive drone communication is set as broadcast communication. At each time step, each drone can receive communication information from all friendly drones.

[0014] The observation information preprocessing process includes: processing the observation information into an observation pair including the relative positions of the defender, the invader and the near-ground airspace, designing an air-to-air situation assessment function for the UAV, and adding the situation assessment to the observation pair based on the function.

[0015] All the observation pairs are input into a distributed target allocation strategy network based on an attention mechanism for processing, and the processing process includes: intent generation, friendly machine intent aggregation, intent coordination and target allocation.

[0016] The intention generation includes: a The observation pairs go through N a The fully connected neural network with the same input parameters is obtained by a intermediate vectors, and then processed by the self-attention mechanism module, after N a The output end of the neural network with the same parameters is fully connected to obtain N a vector, this N a The vectors together constitute the UAV's intention tensor.

[0017] The friendly machine intention aggregation includes: d -1 The intention tensor of a friendly machine is used as the key and value input of the attention mechanism, and the intention tensor of the local machine is used as the query input of the attention mechanism. After being processed by the attention mechanism, the friendly machine intention aggregation tensor is obtained.

[0018] The intention coordination includes: splitting the friendly machine intention aggregation tensor into N a vectors, as the query input of the attention mechanism, splitting the native intent tensor into N a vectors, as the key and value input of the attention mechanism, are processed by the attention mechanism to obtain N a vectors are embedded to form a coordination intention tensor.

[0019] The target allocation includes: a The vectors are passed through the fully connected neural network to obtain N a value, N a The values ​​are sampled to obtain a unique target intruder ID.

[0020] The present invention also provides a distributed UAV target allocation strategy network training method, which is characterized in that it includes the following steps: using a neural network based on a self-attention mechanism and combined with a conflict resolution mechanism as a centralized target allocator, using reinforcement learning training to obtain a centralized strategy with conflict resolution capabilities, constructing a scalable distributed intent fusion based on an attention mechanism, and using imitation learning to train a distributed target allocation strategy network based on data generated by the centralized strategy.

[0021] Compared with the prior art, the present invention has the following advantages:

[0022] (1) The present invention adopts a distributed architecture, which solves the problem that the centralized architecture needs to consider the switching of the central node and has high requirements on the computing power of the central node.

[0023] (2) Compared with the existing collaborative strategies in the context of multi-UAV air combat, the intention coordination mechanism based on the attention mechanism proposed in the present invention is invariant to the observation length, solves the scalability problem of the number of UAVs, and thus has better generalization performance.

[0024] (3) The present invention significantly improves the target allocation conflict resolution capability in a distributed situation by learning from a centralized expert strategy with conflict resolution capability, thereby improving the overall performance. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] Figure 1 Assigning policy network training graphs to distributed targets;

[0026] Figure 2 Schematic diagram of drone motion discretization;

[0027] Figure 3 Schematic diagram of the attitude angle of the UAV;

[0028] Figure 4 A diagram of the attack model of a defense drone against an invading drone;

[0029] Figure 5 This is a diagram of the attack model of the invading drone on the base;

[0030] Figure 6 is the attention mechanism diagram;

[0031] Figure 7 Assign architecture diagrams for distributed targets. DETAILED DESCRIPTION

[0032] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0033] This embodiment provides a method for coordinating the intention of coordinated defense of distributed UAVs, which includes: modeling the problem of coordinated defense of multiple UAVs as a distributed Markov process, stipulating the kinematic constraints, boundary constraints, observation models and confrontation influence rules of UAVs; on this basis, designing a parallel accelerated UAV cluster air combat simulation scene, which can realize GPU acceleration; using deep reinforcement learning to train the underlying one-to-one strategy as the underlying strategy of the target allocator concerned by the present invention; using a neural network based on a self-attention mechanism and a conflict resolution mechanism as a centralized target allocator, and using reinforcement learning training to obtain a centralized strategy with conflict resolution capabilities; constructing an extensible distributed intention fusion based on an attention mechanism; based on the data generated by the centralized strategy, using imitation learning to train distributed target allocation. This method can significantly improve the target allocation conflict resolution capability in a distributed situation by learning from a centralized expert strategy with conflict resolution capabilities, thereby improving the overall performance.

[0034] In order to solve the problems of lack of effective distributed intent coordination mechanism and difficulty in training in the prior art, the present invention provides a distributed intent coordination method based on attention mechanism, which uses imitation learning to optimize strategy parameters and improves implicit conflict resolution capabilities by learning from centralized strategies. The technical problem to be solved by the present invention is achieved through the following technical solutions:

[0035] The UAV collaborative defense problem is established as a distributed Markov process, and the three elements of the system are specified, including the subject, the environment and the task. The subject is the defending UAV, and the environment is the near-ground target area, the invading UAV, the 20km*20km*10km airspace and the dynamic model followed. The task is to prevent the invading UAV from entering the near-ground target area.

[0036] The observations are processed into observation pairs containing the relative relationship between the defender, the attacker and the target area, and the situation function is added to the observation pairs to integrate expert knowledge and provide high-level information for target allocation decisions, which is beneficial to target allocation.

[0037] In order to reduce the complexity of the problem, the distributed strategy is decoupled into the underlying one-to-one interception strategy and the upper-level target allocation strategy.

[0038] A fully connected neural network is used to build the underlying interception strategy, and the proximal policy optimization (PPO) algorithm is used to train the underlying strategy.

[0039] like Figure 1 As shown, a scalable centralized target allocator with conflict resolution capability is constructed. The centralized allocator includes a scalable backbone network based on a self-attention mechanism and a conflict resolution mechanism. The target allocator is trained using a variant of the PPO algorithm.

[0040] The key to UAV collaborative decision-making is to build a learnable intention coordination mechanism, which requires the mechanism to be scalable. In order to achieve the above purpose, a distributed intention coordination architecture based on attention mechanism is constructed, including four steps: intention generation, friendly aircraft intention fusion, intention coordination, and target allocation.

[0041] In the intention generation stage, N a The observation pairs go through N a The fully connected neural network with the same input parameters is obtained by a intermediate vectors, and then processed by the self-attention mechanism module, and finally by N a The output end of the neural network with the same parameters is fully connected to obtain N a vector, this N a The vectors together constitute the UAV's intention tensor.

[0042] In the friendly aircraft intention fusion stage, N d -1 The intention tensor of a friendly machine is used as the key and value input of the attention mechanism, and the intention tensor of the local machine is used as the query input of the attention mechanism. After being processed by the attention mechanism, the friendly machine intention aggregation tensor is obtained.

[0043] In the intention coordination phase, the friendly machine intention aggregation tensor is split into N a vectors, as the query input of the attention mechanism, splitting the native intent tensor into N a vectors, as the key and value input of the attention mechanism, are processed by the attention mechanism to obtain N a vectors are embedded to form a coordination intention tensor.

[0044] In the target assignment phase, the N a The vectors are respectively passed through a fully connected neural network with the same parameters to obtain N a logits values, sampling can get the final target distribution.

[0045] Distributed strategies are trained inefficiently by trial and error through interaction with the environment, and are prone to fall into the minimum state, making it difficult to learn effective conflict resolution capabilities. In order to overcome the above shortcomings, a training architecture based on imitation learning is designed, using a centralized strategy to generate labeled data, and then using these labeled data to supervise the learning of distributed intent coordination strategies.

[0046] In order to overcome the distribution shift problem caused by the difference between centralized and distributed strategies, the Dagger method based on data augmentation is used for imitation learning.

[0047] The steps for problem modeling and strategy architecture design are as follows:

[0048] Step 1: Establish the problem model. In the battlefield, which is a rectangular area of ​​20km*20km*10km, there is a hemispherical near-ground airspace Ω with a radius of 2km in the middle of the horizontal plane with a height of 0. B , initial N d Guard drones and N a The intruder drones are randomly distributed in the hemispherical shell areas with smaller and larger radii. The intruder tries to enter Ω B This study focuses on the cooperative strategy of the defending drones, whose task is to coordinate the maneuvers of the defending drones to make the invading drones enter the attack range before they enter the near-ground airspace. B The coordinated defense problem is modeled as a distributed Markov process (Dec-MDP): in are the state spaces of a single UAV. Since the goals of all defenders are the same, there is a unified reward function

[0049] Step 1-1: The state space

[0050] in Indicates the drone's survival status. Indicates the spatial position, Represents the velocity component.

[0051] Step 1-2: The reward function Represents the group reward function, and the reward function is set as follows:

[0052] (a) Base Destruction Penalty: When a base is destroyed, a penalty of -100n is given, where n is the number of attackers.

[0053] (b) Reward for Eliminating Invaders: Once any new attacker is eliminated, a +100 reward will be given.

[0054] (c) Intruder proximity penalty: Different from the above two event-triggered rewards, this is a dense reward that gives a penalty based on the distance of the intruder to the base at each time step. The reward, where R attacker,init is the average distance from the attacker’s initial position to the base, Rbase is the radius of the base, dist(j,b) is the distance between attacker j and base b, Δr j is the change in distance to the base in the previous time step. The reason for designing this reward is to make the cumulative reward of the attacker moving from the initial position to the base approximately -100, so as to make the total reward function smoother.

[0055] The ultimate goal of collaborative defense is to find an optimal strategy π * , so that the cumulative reward E π [∑ t γ t r t ] is maximized, where γ is the reward discount factor, is the reward at step t.

[0056] Steps 1-3: Action Space Defined as a discrete form, such as Figure 2 There are 15 actions in total, and the underlying continuous control quantity input corresponding to each action is shown in the table. x ∈[-1,2] represents the throttle acceleration coefficient, n z ∈[0,8] is the radial acceleration coefficient, and μ∈[-π,π] represents the roll angle.

[0057] Table 1. Correspondence between the UAV's discrete motion and the underlying continuous control quantity

[0058]

[0059] Step 1-4: The state transfer function is defined as It includes updating the drone's posture according to kinematic constraints and updating the drone's survival status according to countermeasure impact rules.

[0060] Step 1-4-1: Update the drone's pose according to the kinematic constraints. The kinematic constraints of the intruder and defender are the same, defined as shown in formula (3) (4):

[0061]

[0062] like Figure 2 As shown, γ represents the pitch angle, ψ represents the heading angle,

[0063] Step 1-4-2: Update the drone survival status according to the confrontation impact rule. The confrontation impact rule is defined as the relationship between the intruder, defender, and base, which depends on the relative posture and distance. Figure 3 , 4 As shown, the intruder can only attack Ω BIf an intruder enters the near-ground airspace Ω B (Radius R B The mission fails if the intruder is within the hemispherical area composed of the sphere and the ground. The intruder cannot attack the defender. The defender can attack the intruder, and the antenna steering angle α is defined ij ∈[0,2π] is the angle between the speed direction of the defender and the sight vector r of the two aircraft, and the target azimuth β is defined ij ∈[0,2π] is the angle between the target velocity direction and r. When the states of the two aircraft meet the following conditions, the intruder is considered to be shot down. (1) The target is in the cone area Ω in front of the nose of the aircraft F The antenna steering angle (ATA) meets |α ij |<α F , and the distance satisfies ρ ij < f (2) The target azimuth (AA) satisfies |β ij |<β F . Among them α F =π / 6,β F =π / 2.

[0064] Step 1-5: Observation preprocessing. Assuming that each drone can observe the information of all drones at any time, in order to facilitate the neural network to better estimate the situation, the observation is processed into a Intruder and near-ground airspaceΩ B In order to provide more advanced situation information, a UAV air-to-air situation assessment function is designed. Add to the observation pair, so the observation is defined as in in Respectively represent the situation functions about antenna steering angle, target azimuth and relative distance, where

[0065]

[0066] Step 1-6: The UAV communication is set as broadcast communication, that is, at each time step, each UAV can receive the communication information of all friendly UAVs. Ignore the communication delay.

[0067] Step 1-7: Figure 4 As shown in the figure, the mission termination conditions are defined as: (1) all invaders are shot down; (2) or the base is destroyed by invaders; (3) the mission times out. If the time exceeds a certain limit, the mission is automatically terminated. When one of the above conditions is met, the mission is terminated.

[0068] Step 2: Establish a hierarchical control architecture, that is, decouple the UAV distributed control strategy into target allocation and underlying one-to-one interception strategy. The target allocation strategy outputs the unique target intruder IDa of the current defender ij The task of the lower-level controller is to prevent the intruder drone from entering the near-ground airspace. B .

[0069] Step 3: Establish the underlying one-to-one interception strategy. For the sake of simplicity, the formulas in this section omit the drone subscript. Based on the basic maneuver library, establish a Markov model for the 1v1 interception problem. in

[0070] Step 4: Use the proximal policy optimization (PPO-clip) algorithm to build and train the maneuver controller. Two neural networks of size 16*3 (3 hidden layers, each layer width 16) are used as the policy network (actor) and value network (critic) The policy network is used to predict actions, and the value network is used to evaluate situations. The algorithm shown in Algorithm 1 is used to train these two networks simultaneously.

[0071] Algorithm 1 Interception strategy training algorithm:

[0072] Input: Initial policy network parameters and value network parameters Output: Policy network parameters

[0073] 1:fork=0,1,2,...do

[0074] 2: Run the strategy in the environment Collecting Trajectory Sets in

[0075]

[0076] 3: Calculate reward-to-go

[0077] 4: Based on the current value function Calculating the odds estimate

[0078] 5: Maximize the PPO-Clip objective function, use the Adam algorithm for stochastic gradient ascent, and update the policy network

[0079]

[0080] 6: Minimize the mean square error, use the Adam algorithm for stochastic gradient descent, and update the value network φ k :

[0081]

[0082] 7: end for

[0083] Step 5: Define the basic attention module. In order to handle variable length inputs in observations and communications, a large number of attention mechanisms are used in the target allocation. The attention mechanism borrows the structure of database retrieval, such as Figure 6 As shown in the figure, we input a query vector (query), a key vector (key), and a value vector (value), where the key vector and the value vector are one-to-one corresponding. By calculating the similarity between the query vector and the key vector, we get the weight of each pair (key vector, value vector), and then perform weighted summation on the value vector to get the aggregate vector. Define the similarity function σ(q,k j )=a T (q||k j ), for simplicity, symbol reuse is performed, and the operation on multiple vectors is defined as σ(q,K)=(σ(q,k1),…,σ(q,k N )), where K = (k1,…,k N ). The output of the attention mechanism can be expressed as: h = Att(q,K,V) = Sum(Softmax(σ(q,K))⊙V), where q = W q y,k j =W k x j ,v j =W v x j , a t ,W q ,W k ,W v is the learnable parameter of the attention mechanism, and ⊙ represents element-wise multiplication. When the query vector y is a component vector in the matrix X, the self-attention mechanism is obtained.

[0084] Step 6: Based on the underlying 1v1 interceptor described in step 3, establish a distributed target allocator based on the attention mechanism. In order to create conditions for conflict resolution for distributed target allocation, it is necessary to design a reasonable coordination mechanism based on communication, while maintaining the scalability of input, which includes observation and communication information, where communication information is the intention information of teammates. Figure 7 As shown, the distributed target allocator consists of four stages: (1) intent generation, (2) friendly machine intent aggregation, (3) intent coordination, and (4) target allocation.

[0085] Step 6-1: During combat, as combat casualties increase, the number of observed enemy aircraft will change. In order to cope with this change, the first step is to generate inputs of variable length using a self-attention mechanism.

[0086] Step 6-1-1: As described in step 1-3, module input observation pairs, each of which corresponds to a potential assignment. A fully connected neural network with the same parameters is obtained vector embedding.

[0087] Step 6-1-2: Then process it through the self-attention mechanism vector embedding.

[0088] Step 6-1-3: Finally, these vectors are embedded through The fully connected network outputs with the same parameters vector embedding, forming a dimension The intention tensor h i , each component h ij corresponds to a potential allocation.

[0089] Step 6-2: Next, the friendly aircraft will be integrated. At each moment, the drone receives N d -1 friendly aircraft’s broadcast communication. To reduce the complexity, the attention mechanism is used to fuse the intention matrices of all friendly aircraft. Specifically, the current drone’s intention tensor is used as the query vector, and the friendly aircraft’s intention tensor is used as the key and value to find the intention aggregation h of the friendly aircraft. f .

[0090] Step 6-3: Then proceed with intention coordination. The key to this stage is to create conditions for conflict resolution. To achieve this goal, consider the friendly aircraft intention tensor as N a One-dimensional tensors, corresponding to N a The intruder is used as the query vector of the attention mechanism; the current drone’s intention tensor is processed in the same way as the key and value of the attention mechanism. After the attention mechanism is processed, the coordinated intention tensor is obtained.

[0091] Step 6-4: The final step is to assign the target, which coordinates the N of the intention tensor a The vectors are respectively passed through a fully connected neural network with the same parameters to obtain N a logits values, and finally the assigned ID is obtained through Softmax sampling.

[0092] like Figure 1As shown, this embodiment also provides a distributed UAV target allocation strategy network training method, including: constructing and training a centralized allocator as an expert strategy Based on the underlying one-to-one interception strategy mentioned above and the centralized allocator as an expert strategy Training the distributed allocator π with the Dagger framework θ Specifically, using a centralized target allocator Generate trajectory sets through Monte Carlo experiments where τ=(s0,a0,r0,s1,…,s T ), extract all state-action pairs as expert demonstration dataset Specifically, the defender and invader are initialized in the hemispherical shells S(2km,2km) and S(8km,2km) respectively. The hemispherical shell S(r,w) represents the upper hemispherical shell formed by the spherical shell with inner radius r and thickness w and the z=0 plane.

[0093] In order to alleviate the distribution deviation problem caused by the difference between the centralized expert strategy and the current distributed strategy, the DAgger method based on data augmentation is adopted. The specific algorithm flow is shown in Algorithm 2.

[0094] Algorithm 2 Distributed Allocator Imitation Learning Algorithm:

[0095] Input: Initialize the demonstration dataset Initialize the distributed target allocation network θ=θ0, Output: Policy network parameters θ1: According to the data set Train the policy π using supervised learning θ :

[0096]

[0097] Where L cross is the cross entropy loss function; 2: By θ Interact with the environment to get a new set of observations And record the average cumulative return 3: Artificial The state annotation action in the , and get the labeled demonstration data set 4: Aggregate data: 5: If Return θ, otherwise And execute step 1.

[0098] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed by the present invention, and these modifications or replacements should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention shall be based on the protection scope of the claims.

Claims

1. A distributed UAV cooperative defense intention coordination method, characterized in that: The following steps are involved: The collaborative defense problem is modeled as a distributed Markov process. Under the distributed Markov process, the UAV distributed control strategy is decoupled into a distributed target allocation strategy and an underlying one-to-one interception strategy. The observation information of all defending UAVs is preprocessed to obtain observation pairs. All observation pairs are input into a distributed target allocation strategy network based on an attention mechanism for processing. The distributed target allocation strategy network outputs the unique target intruder ID of the current defending UAV. The underlying one-to-one interception strategy makes decisions based on the unique target intruder ID, and the defending UAV defends based on the decision.

2. A distributed UAV cooperative defense intention coordination method according to claim 1, characterized in that: The distributed Markov process includes a state space, an action space, a reward function and a state transfer function.

3. A distributed UAV cooperative defense intention coordination method according to claim 2, characterized in that: The state space in Indicates the drone's survival status. Indicates the spatial position, represents the velocity component, the action space Defined in discrete form, the reward function Represents the group reward function, which is set as follows: Base destruction penalty: When the base is destroyed, a penalty of -100n is given, where n is the number of invaders. Elimination of invaders reward: Eliminate any new attacker and get a +100 reward. Invader proximity penalty: Each time step is based on the distance of the invader to the base. The reward, where R attacker,init is the average distance from the attacker’s initial position to the base, R base is the radius of the base, dist(j, b) is the distance between attacker j and base b, Δr j is the change in distance to the base in the previous time step, and the state transfer function is defined as It includes updating the drone's posture according to kinematic constraints and updating the drone's survival status according to countermeasure impact rules.

4. A distributed UAV cooperative defense intention coordination method according to claim 1, characterized in that: The observation information preprocessing process includes: processing the observation information into an observation pair including the relative positions of the defender, the invader and the near-ground airspace, designing an air-to-air situation assessment function for the UAV, and adding the situation assessment to the observation pair based on the function.

5. A distributed UAV cooperative defense intention coordination method according to claim 1, characterized in that: All the observation pairs are input into a distributed target allocation strategy network based on an attention mechanism for processing, and the processing process includes: intent generation, friendly machine intent aggregation, intent coordination and target allocation.

6. A distributed UAV cooperative defense intention coordination method according to claim 5, characterized in that: The intention generation includes: a The observation pairs go through N a The fully connected neural network with the same input parameters is obtained by a intermediate vectors, and then processed by the self-attention mechanism module, after N a The output end of the neural network with the same parameters is fully connected to obtain N a vector, this N a The vectors together constitute the UAV's intention tensor.

7. A distributed UAV cooperative defense intention coordination method according to claim 5, characterized in that: The friendly machine intention aggregation includes: d -1 The intention tensor of a friendly machine is used as the key and value input of the attention mechanism, and the intention tensor of the local machine is used as the query input of the attention mechanism. After being processed by the attention mechanism, the friendly machine intention aggregation tensor is obtained.

8. A distributed UAV cooperative defense intention coordination method according to claim 5, characterized in that: The intention coordination includes: splitting the friendly machine intention aggregation tensor into N a vectors, as the query input of the attention mechanism, splitting the native intent tensor into N a vectors, as the key and value input of the attention mechanism, are processed by the attention mechanism to obtain N a vectors are embedded to form a coordination intention tensor.

9. A distributed UAV cooperative defense intention coordination method according to claim 5, characterized in that: The target allocation includes: a The vectors are passed through the fully connected neural network to obtain N a value, N a The values ​​are sampled to obtain a unique target intruder ID.

10. A distributed target allocation strategy network training method for distributed unmanned aerial vehicles, characterized in that: The following steps are involved: A neural network based on self-attention mechanism and combined with conflict resolution mechanism is used as the centralized target allocator. Reinforcement learning is used to train a centralized strategy with conflict resolution ability. A scalable distributed intention fusion based on attention mechanism is constructed. Based on the data generated by the centralized strategy, imitation learning is used to train the distributed target allocation strategy network.

Citation Information

Cited By

  • Multi-unmanned aerial vehicle adaptive intelligent decision reinforcement learning method and system

    CN119690101A

  • Multi-unmanned aerial vehicle collaborative decision-making method, device and equipment based on graph attention network

    CN120722955A