A multi-UAV formation cluster control method based on multi-agent deep reinforcement learning

Abstract

The present invention relates to the field of intelligent collaborative control of multiple unmanned aerial vehicles (UAVs). It provides a method for autonomous clustering of multiple UAV formations based on multi-agent deep reinforcement learning. The method mainly comprises the following steps: 1) modeling the motion process of a single UAV; 2) modeling the clustering process of heterogeneous multiple UAVs as a Markov game process; 3) training the clustering process based on the MADDPG algorithm; 4) introducing an attention mechanism into multi-agent deep reinforcement learning to optimize the learning process of the UAVs; and 5) using the learned strategy to control the multiple UAVs. The present invention proposes a method for clustering multiple UAV formations based on multi-agent deep reinforcement learning. By modeling the clustering problem of multiple UAVs as a Markov game process, the UAVs are autonomously trained to obtain corresponding control models. The introduction of an attention mechanism accelerates training and improves training effectiveness. Finally, the clustering of multiple different formations is completed by transforming the coordinates.

Description

Technical Field

[0001] The present invention belongs to the technical field of intelligent collaboration of multiple UAV formation clusters, and in particular relates to a multi-UAV formation cluster control method based on multi-agent deep reinforcement learning. Background Art

[0002] With the advancement of drone technology, research on multi-drone formation swarms has garnered increasing attention. Drones working together to form a swarm can leverage advantages over individual drones, making them more capable of handling complex, multi-task scenarios. Currently, reinforcement learning methods are increasingly being used to control drone formation systems. However, most reinforcement learning-based drone control methods are only suitable for situations with a small number of drones. As the number of drones in a swarm increases, training becomes more difficult, convergence slows, and the results after convergence deteriorate. Furthermore, other reinforcement learning training methods designed for larger numbers of drones struggle to form precise swarm formation structures, and can only achieve consistent drone movement. Summary of the Invention

[0003] In view of this, the present invention discloses a multi-UAV formation cluster control method based on multi-agent deep reinforcement learning, which can achieve more competent work in complex, multi-task and other scenarios.

[0004] The technical solution of the present invention is a multi-UAV formation cluster control method based on multi-agent deep reinforcement learning, comprising the following steps:

[0005] Step 1: Model the motion process of a single drone;

[0006] Step 2: Model the aggregation problem between multiple drones as a Markov game process;

[0007] Step 3: Use multi-agent deep reinforcement learning methods for training, use the actor network to learn the aggregation strategy, and use the critic network to evaluate the learning effect to guide the drone aggregation;

[0008] Step 4: Introduce the attention mechanism into the critic network during training to speed up training and improve training results;

[0009] Step 5: Obtain the mutual aggregation model of multiple drones and complete the aggregation of corresponding sub-formations;

[0010] Step 6: Based on coordinate transformation, multiple different formations are unified together to form a drone cluster.

[0011] Preferably, the modeling of the single UAV motion process in step 1 is specifically as follows:

[0012] First, define a vector set p=[p1,p2,…,p n ] T , which is the position of all drones in the cluster in the Cartesian coordinate system, represented by the vector p i =(x i ,y i ,z i ) T The position of UAV i in space can be determined, and the motion process of a single UAV can be modeled using the following formula:

[0013]

[0014] where v i is the flight speed of UAV i, θ i and φ i are the pitch angle and heading angle of UAV i respectively. During the flight of the UAV, the flight of the UAV is guided by applying control input. The following control input formula is given:

[0015]

[0016] where p i and V i are the position and velocity of UAV i, are the control inputs in three directions.

[0017] Preferably, in step 2, the aggregation problem among multiple drones is modeled as a Markov game process, and the drones are corresponding to intelligent agents. Given the corresponding state space, action space and reward function settings:

[0018] The state space is described as follows: the formation contains leader drones and follower drones. Here, the state space is classified from the perspective of leader and follower, and the corresponding leader drone s l and the status of the follower drone s f ; Define the state space of the leader drone as:

[0019] s t =(p l ,p tar ,v l )

[0020] The state space of the follower drone is:

[0021] s f =(p f ,p l ,v l ,v f )

[0022] In general, the leader drone moves towards its own target position p tar =(x tar ,y tar ,z tar ) movement, the follower UAV obtains the position information of the leader UAV, obtains its relative position relationship with it and maintains the formation; l =(x l ,y l ,z l ) is the coordinate of the leader UAV in the Cartesian coordinate system, p f =(x f ,y f ,z f ) is the position coordinate of the follower drone, v l is the speed of the leader drone, v f is the velocity of the follower drone; finally there is a joint state space:

[0023] s=(p l ,p f ,p tar ,v l ,v f )

[0024] Action space: The control of the drone is mainly achieved by changing the speed and angle; here the action space is set as:

[0025] a=(u v ,u r ,u p )

[0026] where a v is the control input for the UAV speed, and the control change of the speed satisfies the following conditions:

[0027]

[0028] The control amount of angle includes u r and u p , which respectively control the change of heading angle and pitch angle. For heading angle:

[0029]

[0030] Where [-r φ ,r φ ] is the allowable steering range of the heading angle, and the same is true for the pitch angle [-p θ ,p θ ], the control input is similar to the above formula;

[0031] The reward function is (1) the reward of the leader drone:

[0032]

[0033] where R tar >0 is the radius of the target area. When the distance between the leader drone and the target point is less than this radius, it is considered to have reached the target point and a positive reward α is given. l On the contrary, it will be punished, and the degree of punishment is proportional to the distance to the target point, that is, the farther the distance, the greater the punishment;

[0034] (2) Rewards for follower drones:

[0035]

[0036] The reward setting for the follower is similar to that of the leader drone. When the follower drone is in the expected position area, a positive reward α is given. f ; On the contrary, give negative rewards as punishment;

[0037] (3) Rewards for avoiding collisions between drones:

[0038]

[0039] (4) Global Rewards:

[0040]

[0041] Preferably, the improvement of introducing the attention mechanism into the critic network during the training process in step 4 is as follows: an attention module is added to the evaluation function part; the newly added attention module is used to aggregate the input information of the evaluation function part (x, a1, ..., a n ), where x=(o1,...,o n ) is the set of observation information of all drones. The value function after adding the attention mechanism can be expressed as:

[0042]

[0043] Among them, f i is a two-layer MLP network used to estimate the action state value function of the drone; g i It is a one-layer MLP network used to encode the state observation value and action value of the drone itself; c i To measure the impact of other drones on drone i:

[0044]

[0045] Where h is a nonlinear ReLu function, and the attention weight α is used j represents the impact of UAV j on UAV i; ci It is concluded that the greater the weight of drone j, the greater its influence on drone i, and the greater the attention weight α j Perform SoftMax normalization, we have:

[0046]

[0047] The evaluation function of the corresponding i-th drone is updated as follows:

[0048]

[0049] Where y is the value function of the target network:

[0050]

[0051] At the same time, the action value function of the i-th drone is updated as follows:

[0052]

[0053] Preferably, in step 5, the coordinate systems of multiple different sub-formations are converted to the same cluster coordinate system to complete the unification of the cluster:

[0054] There is a rotation relationship between the two coordinate systems on the corresponding axes. Considering the angle changes of the formation in the horizontal heading angle and longitudinal pitch angle, first, the formation coordinate system O f X f Y f Z f O around the ground coordinate system g Z g The axis rotation gets a horizontal angle of -φ, and a transition rotation matrix can be obtained:

[0055]

[0056] Secondly, around the ground coordinate system O g Y g The axis rotation obtains the pitch angle θ about the longitudinal direction perpendicular to the horizontal plane, and a transition rotation matrix can also be obtained:

[0057]

[0058] From this, we can obtain the rotation matrix of the formation coordinate system relative to the ground coordinate system:

[0059]

[0060] Preferably, in step six, multiple different formations are unified together to form a drone cluster based on coordinate transformation; specifically:

[0061] Assume that a formation moves dx, dy and dz in the X, Y and Z directions respectively, and any of the drones has p t =p+t=[x+dx,y+dy,z+dz], where t=[dx,dy,dz]. The translation process can also be expressed in the form of a matrix. To facilitate calculation, a homogeneous coordinate w is introduced, and w=1 is taken; for the UAV, p w =[x,y,z,w], so the homogeneous coordinates after translation can be expressed as:

[0062]

[0063] in To obtain the homogenized translation matrix, combined with the possible rotation R, we can finally obtain the coordinate transformation of any UAV in the formation in the ground coordinate system:

[0064]

[0065] Where p' is the position of the UAV in the formation coordinate system after rotation and translation in the ground coordinate system; different formations move towards the same gathering point and are unified into the same cluster coordinate system through the above coordinate transformation relationship to complete the clustering.

[0066] This paper proposes a multi-UAV formation swarm control method based on multi-agent deep reinforcement learning. By modeling the multi-UAV clustering problem using a Markov game process, the UAVs are autonomously trained to obtain the corresponding control model. An attention mechanism is introduced to accelerate training and improve training effectiveness. Finally, coordinate transformation is performed to achieve clustering of multiple different formations.

[0067] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the disclosure of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0068] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the invention and, together with the description, serve to explain the principles of the invention.

[0069] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0070] Figure 1 This is a flow chart of a multi-UAV formation cluster control method based on multi-agent deep reinforcement learning provided by the present invention;

[0071] Figure 2 Schematic diagram of the multi-agent deep reinforcement learning method after adding the attention mechanism in the embodiment disclosed in the present invention;

[0072] Figure 3 Schematic diagram of the correspondence between the ground coordinate system and the formation coordinate system disclosed in an embodiment of the present invention;

[0073] Figure 4 is a schematic diagram of the cluster disclosed in the embodiment of the present invention;

[0074] Figure 5 is a graph showing changes in the global reward curve during the training process disclosed in an embodiment of the present invention;

[0075] Figure 6 This is a comparison chart of follower drone rewards disclosed in an embodiment of the present invention;

[0076] Figure 7 It is a cluster fit change diagram disclosed in an embodiment of the present invention. DETAILED DESCRIPTION

[0077] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. In the following description, when referring to the drawings, like numbers in different figures represent the same or similar elements unless otherwise indicated. The embodiments described in the following exemplary embodiments are not intended to represent all possible embodiments consistent with the present invention. Rather, they are merely examples of systems consistent with certain aspects of the present invention, as detailed in the appended claims.

[0078] The reinforcement learning-based formation control of the present invention mainly obtains a control strategy that meets the requirements by giving control inputs in the training phase. Finally, the obtained control strategy can be applied to the drones in the testing and application phase. This method does not require a large amount of complex cluster system modeling and has certain advantages.

[0079] This paper provides a multi-UAV formation swarm control method based on multi-agent deep reinforcement learning. By modeling the multi-UAV clustering problem using a Markov game process, the UAVs are autonomously trained to obtain the corresponding control model. An attention mechanism is introduced to accelerate training speed and improve training effectiveness. Finally, coordinate transformation is performed to achieve clustering of multiple different formations.

[0080] like Figure 1-4 As shown, the above method specifically includes the following steps:

[0081] Step 1: Model the motion process of a single drone;

[0082] First, define a vector set p=[p1,p2,…,p n ] T , which is the position of all drones in the cluster in the Cartesian coordinate system, represented by the vector p i =(x i ,y i ,z i ) T The position of UAV i in space can be determined. The motion process of a single UAV is modeled using the following formula:

[0083]

[0084] where v i is the flight speed of UAV i, θ i and φ i are the pitch angle and heading angle of UAV i, respectively. During the flight of the UAV, the flight of the UAV is guided by applying control input. Given the following control input formula:

[0085]

[0086] where p i and V i are the position and velocity of UAV i respectively. is the control input in three directions, by i The input control can change the speed and angle of the drone and other parameters.

[0087] Step 2: Model the aggregation problem between multiple drones as a Markov game process;

[0088] The drone corresponds to the intelligent agent, and the corresponding state space, action space and reward function settings are given:

[0089] State space: The formation contains leader drones and follower drones. Here, the state space is classified from the perspective of leader and follower. The corresponding leader drones are s l and the status s of the follower drone f The state space of the leader drone is defined as:

[0090] s t =(p l ,p tar ,v l )

[0091] The state space of the follower drone is:

[0092] s f =(p f ,p l ,v l,v f )

[0093] In general, the leader drone moves towards its own target position p tar =(x tar ,y tar ,z tar ) movement, the follower UAV obtains the position information of the leader UAV, obtains its relative position relationship with it and maintains the formation. l =(x l ,y l ,z l ) is the coordinate of the leader UAV in the Cartesian coordinate system, p f =(x f ,y f ,z f ) is the position coordinate of the follower drone, v l is the speed of the leader drone, v f is the velocity of the follower drone. Finally, there is a joint state space:

[0094] s=(p l ,p f ,p tar ,v l ,v f )

[0095] Action space: The control of the drone is mainly achieved by changing the speed and angle. Here the action space is set as:

[0096] a=(u v ,u r ,u p )

[0097] where a v is the control input for the UAV speed, and the control change of the speed satisfies the following conditions:

[0098]

[0099] The control amount of angle includes u r and u p , which respectively control the change of heading angle and pitch angle. For heading angle:

[0100]

[0101] Where [-r φ ,r φ ] is the allowable steering range of the heading angle, and the same is true for the pitch angle [-p θ ,p θ ], the control input is similar to the above formula;

[0102] Reward function: (1) Reward for the leader drone:

[0103]

[0104] where R tar >0 is the radius of the target area. When the distance between the leader drone and the target point is less than this radius, it is considered to have reached the target point and a positive reward α is given. l On the contrary, it will be punished, and the degree of punishment is proportional to the distance to the target point, that is, the farther the distance, the greater the punishment.

[0105] (2) Rewards for follower drones:

[0106]

[0107] The reward setting for the follower is similar to that of the leader drone. When the follower drone is in the expected position area, a positive reward α is given. f On the contrary, negative rewards are given as punishment.

[0108] (3) Rewards for avoiding collisions between drones:

[0109]

[0110] (4) Global Rewards:

[0111]

[0112] Step 3: Use multi-agent deep reinforcement learning methods for training, use the actor network to learn the aggregation strategy, and use the critic network to evaluate the learning effect to guide the drone aggregation;

[0113] Step 4: Introduce the attention mechanism into the critic network during training to speed up training and improve training results;

[0114] The improvements to the critic network input are as follows:

[0115] An attention module is added to the evaluation function. More specifically, the newly added attention module is used to aggregate the input information of the evaluation function (x, a1, ..., a n ), where x=(o1,...,o n ) is the set of observation information of all drones. The value function after adding the attention mechanism can be expressed as:

[0116]

[0117] Among them, f iIt is a two-layer MLP network used to estimate the action state value function of the drone. i It is a one-layer MLP network used to encode the state observation value and action value of the drone itself. i To measure the impact of other drones on drone i:

[0118]

[0119] Where h is a nonlinear ReLu function, and the attention weight α is used j represents the impact of drone j on drone i. i It can be seen that the greater the weight of drone j, the greater its influence on drone i. j Perform SoftMax normalization, we have:

[0120]

[0121] Therefore, different drones use their similarities to determine their corresponding attention and obtain key information based on the size of the attention weight. By changing the input method of the drone's centralized evaluation function, different reward feedback is obtained, ultimately guiding the drone to take actions that will earn more rewards.

[0122] The evaluation function of the corresponding i-th drone is updated to:

[0123]

[0124] Where y is the value function of the target network:

[0125]

[0126] At the same time, the action value function update formula of the i-th drone is:

[0127]

[0128] Step 5: Obtain the mutual aggregation model of multiple drones and complete the aggregation of corresponding sub-formations;

[0129] Step 6: Based on coordinate transformation, multiple different formations are unified together to form a drone cluster;

[0130] The cluster is unified by converting multiple different sub-formation coordinate systems into the same cluster coordinate system:

[0131] There is a rotation relationship between the two coordinate systems on the corresponding axes. Here we consider the angle changes of the formation in the horizontal heading angle and longitudinal pitch angle. First, the formation coordinate system O f Xf Y f Z f O around the ground coordinate system g Z g The axis rotation gets a horizontal angle of -φ, and a transition rotation matrix can be obtained:

[0132]

[0133] Secondly, around the ground coordinate system O g Y g The axis rotation obtains the pitch angle θ about the longitudinal direction perpendicular to the horizontal plane, and a transition rotation matrix can also be obtained:

[0134]

[0135] From this, we can obtain the rotation matrix of the formation coordinate system relative to the ground coordinate system:

[0136]

[0137] In addition to the rotation, the sub-formation may also undergo translation relative to the ground coordinate system. Suppose a formation moves dx, dy, and dz in the X, Y, and Z directions respectively. For any of the drones in the formation, there is a p t =p+t=[x+dx,y+dy,z+dz], where t=[dx,dy,dz]. The translation process can also be expressed in the form of a matrix. For the convenience of calculation, a homogeneous coordinate w is introduced, and w=1 is taken. For this drone, there is p w =[x,y,z,w], so the homogeneous coordinates after translation can be expressed as:

[0138]

[0139] in is the translation matrix after homogenization. Combined with the possible rotation R, the coordinate transformation of any UAV in the formation in the ground coordinate system can be obtained:

[0140]

[0141] Where p' is the position of the drone in the formation coordinate system after rotation and translation in the ground coordinate system. Different formations converge toward the same gathering point and are unified into the same cluster coordinate system through the above coordinate transformation relationship to complete the clustering.

[0142] Applying the method provided by the present invention, Figure 5The following graph shows the evolution of the global reward curve during training. Formation aggregation is a cooperative scenario, so the global reward includes all drones, including the leader and followers. It can be seen that both algorithms undergo an exploration phase in the early stages of training, with large fluctuations in reward values. Subsequently, with increasing training experience, the reward values gradually increase. The A-MADDPG algorithm, with the addition of the attention module, converges faster, reaching convergence after approximately 4,000 episodes, while MADDPG only reaches a stable state after approximately 6,000 episodes. When the rewards reach equilibrium, the attention-based A-MADDPG algorithm has a higher reward value, indicating better training results and, consequently, a higher reward. Higher rewards also indicate better formation aggregation.

[0143] Figure 6 The figure shows the changes in the reward values for all follower drones in a sub-formation during training. It can be seen that the reward values for follower drones trained using the A-MADDPG method converge faster, reaching a stable state after approximately 3,500 episodes, while MADDPG only reaches a stable state after approximately 6,000 episodes. When reaching a stable state, the rewards for the A-MADDPG method are also higher. Furthermore, it can be seen that the range of variation of the rewards for the A-MADDPG method is smaller than that for MADDPG, and the lower bound of its reward range is hidden within the MADDPG reward range, indicating that the method with the attention mechanism has better training results. Because the follower drones in the A-MADDPG method are able to pay more attention to the status information of the leader drone during learning, they engage in less ineffective exploration, which is reflected in the smaller fluctuation range of the reward values.

[0144] Figure 7 The curve showing the change in the degree of fit between the drones and the desired formation structure during the clustering process is shown. An objective function f(x) is defined. The smaller the value of f(x), the higher the degree of fit.

[0145]

[0146] Where n is the number of drones, d(·) is the distance, which is used to measure the cost of the i-th drone to the corresponding point in its virtual structure. is the position of the UAV in ground coordinates, is its corresponding expected position in the virtual structure, = is the transformation relationship between the corresponding drones in the formation coordinate system and the ground coordinate system. It can be seen that in the initial stage of aggregation, due to the random initial positions of the drones, the initial fit of the two methods is different. However, the attention-based A-MADDPG converges faster, and therefore improves the fit faster. This is reflected in the faster rate of decrease in the function value in the figure, and the fit with the set formation is also higher when convergence is finally achieved.

[0147] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the technical principles of the present invention. These changes and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A multi-UAV formation cluster control method based on multi-agent deep reinforcement learning, characterized by: The steps include: Step 1: Model the motion process of a single drone; Step 2: Model the aggregation problem between multiple drones as a Markov game process; Step 3: Use multi-agent deep reinforcement learning methods for training, use the actor network to learn the aggregation strategy, and use the critic network to evaluate the learning effect to guide the drone aggregation; Step 4: Introduce the attention mechanism into the critic network during training to speed up training and improve training results; Step 5: Obtain the mutual aggregation model of multiple drones and complete the aggregation of corresponding sub-formations; Step 6: Based on coordinate transformation, multiple different formations are unified together to form a drone cluster; The specific modeling of the single UAV motion process in step 1 is as follows: First, define a vector set p=[p1,p2,…,p n ] T , which is the position of all drones in the cluster in the Cartesian coordinate system, represented by the vector p i =(x i ,y i ,z i ) T The position of UAV i in space can be determined, and the motion process of a single UAV can be modeled using the following formula: where v i is the flight speed of UAV i, θ i and φ i are the pitch angle and heading angle of UAV i respectively. During the flight of the UAV, the flight of the UAV is guided by applying control input. The following control input formula is given: where p i and V i are the position and velocity of UAV i, is the control input in three directions; In step 2, the aggregation problem among multiple drones is modeled as a Markov game process, with the drones acting as agents. Given the corresponding state space, action space, and reward function settings: The state space is described as follows: the formation contains leader drones and follower drones. Here, the state space is classified from the perspective of leader and follower, and the corresponding leader drone s l and the status of the follower drone s f ; Define the state space of the leader drone as: s l =(p l ,p tar ,v l ) The state space of the follower drone is: s f =(p f ,p l ,v l ,v f ) In general, the leader drone moves towards its own target position p tar =(x tar ,y tar ,z tar ) movement, the follower UAV obtains the position information of the leader UAV, obtains its relative position relationship with it and maintains the formation; l =(x l ,y l ,z l ) is the coordinate of the leader UAV in the Cartesian coordinate system, p f =(x f ,y f ,z f ) is the position coordinate of the follower drone, v l is the speed of the leader drone, v f is the velocity of the follower drone; finally there is a joint state space: s=(p l ,p f ,p tar ,v l ,v f ) Action space: The control of the drone is mainly achieved by changing the speed and angle; here the action space is set as: a=(u v ,u r ,u p ) where u v is the control input for the UAV speed, and the control change of the speed satisfies the following conditions: The control amount of angle includes u r and u p , which respectively control the change of heading angle and pitch angle. For heading angle: Where [-r φ ,r φ ] is the allowable steering range of the heading angle, and the same is true for the pitch angle [-p θ ,p θ ], the control input is similar to the above formula; The reward function is (1) the reward of the leader drone: where R tar >0 is the radius of the target area. When the distance between the leader drone and the target point is less than this radius, it is considered to have reached the target point and a positive reward α is given. l On the contrary, it will be punished, and the degree of punishment is proportional to the distance to the target point, that is, the farther the distance, the greater the punishment; (2) Rewards for follower drones: The reward setting for the follower is similar to that of the leader drone. When the follower drone is in the expected position area, a positive reward α is given. f ; On the contrary, give negative rewards as punishment; (3) Rewards for avoiding collisions between drones: (4) Global Rewards: The improvement of introducing the attention mechanism into the critic network during the training process in step 4 is as follows: an attention module is added to the evaluation function part; the newly added attention module is used to aggregate the input information of the evaluation function part (x, a1, ..., a n ), where x=(o1,...,o n ) is the set of observation information of all drones. The value function after adding the attention mechanism can be expressed as: Among them, f i is a two-layer MLP network used to estimate the action state value function of the drone; g i It is a one-layer MLP network used to encode the state observation value and action value of the drone itself; c i To measure the impact of other drones on drone i: Where h is a nonlinear ReLu function, and the attention weight α is used j represents the impact of UAV j on UAV i; c i It is concluded that the greater the weight of drone j, the greater its influence on drone i, and the greater the attention weight α j Perform SoftMax normalization, we have: The evaluation function of the corresponding i-th drone is updated as follows: Where y is the value function of the target network: At the same time, the action value function of the i-th drone is updated as follows: In step 5, the coordinate systems of multiple sub-formations are converted to the same cluster coordinate system to unify the cluster: There is a rotation relationship between the two coordinate systems on the corresponding axes. Considering the angle changes of the formation in the horizontal heading angle and longitudinal pitch angle, first, the formation coordinate system O f X f Y f Z f O around the ground coordinate system g Z g The axis rotation gets a horizontal angle of -φ, and a transition rotation matrix can be obtained: Secondly, around the ground coordinate system O g Y g The axis rotation obtains the pitch angle θ about the longitudinal direction perpendicular to the horizontal plane, and a transition rotation matrix can also be obtained: From this, we can obtain the rotation matrix of the formation coordinate system relative to the ground coordinate system: In step 6, multiple different formations are unified together to form a drone cluster based on coordinate transformation; specifically: Assume that a formation moves dx, dy and dz in the X, Y and Z directions respectively, and any of the drones has p t =p+t=[x+dx,y+dy,z+dz], where t=[dx,dy,dz]. The translational motion process can also be expressed in the form of a matrix. For the convenience of calculation, a homogeneous coordinate w is introduced, and w=1 is taken. For the UAV, there is p w =[x,y,z,w], so the homogeneous coordinates after translation can be expressed as: in To obtain the homogenized translation matrix, combined with the possible rotation R, we can finally obtain the coordinate transformation of any UAV in the formation in the ground coordinate system: Where p' is the position of the UAV in the formation coordinate system after rotation and translation in the ground coordinate system; different formations move towards the same gathering point and are unified into the same cluster coordinate system through the above coordinate transformation relationship to complete the clustering.

Images