Unmanned aerial vehicle cooperative attack method based on reinforcement learning and dynamic adjustment artificial potential field

Abstract

The application discloses a kind of based on reinforcement learning and dynamic adjustment artificial potential field UAV cooperative attack method, first according to demand establishes multiple unmanned aerial vehicle cooperative attack task model and carries out initialization, then designs dynamic adjustment artificial potential field method, and dynamic adjustment artificial potential field method is combined with MADDPG algorithm, respectively to state space and reward function are designed, finally algorithm iteration is carried out, applied to specific scene, complete mountain area fixed height condition under multiple unmanned aerial vehicle cooperative attack task.The method of the application not only can effectively solve multiple unmanned aerial vehicle cooperative attack problem, but also under different environment and initial condition, ensure the stability and high efficiency of unmanned aerial vehicle system, by combining reinforcement learning algorithm and dynamic adjustment artificial potential field method, provide a more intelligent and flexible multiple unmanned aerial vehicle cooperative attack algorithm, with wide application prospect.

Description

Technical Field

[0001] This invention relates to the field of unmanned aerial vehicle (UAV) swarm technology, specifically to a UAV cooperative strike method based on reinforcement learning and dynamic adjustment of artificial potential fields. Background Technology

[0002] With the continuous advancement and improvement of drone technology, drones, with their advantages of low cost, high maneuverability, scalability, and hovering capabilities, have demonstrated broad application potential in various fields. Drone swarm technology, through collaborative cooperation among drones, can perform a variety of complex combat missions, including target strikes, area defense, and reconnaissance. The synergistic effect of swarm drones enables them to address complex tactical challenges and execute more precise strike missions.

[0003] In strike missions against adversary targets, the collaborative combat capability of multiple drones is crucial. These missions require drones not only to possess high precision but also to maintain efficient cooperation and coordination in dynamically changing adversarial environments. During strike missions, drones need to deploy rapidly and strike precisely based on target locations while avoiding mutual interference and collisions; therefore, collaborative strike capability is particularly important.

[0004] Reinforcement learning is a computational method that enables intelligent agents to achieve goals through interaction with their environment. It optimizes strategies through trial and error mechanisms and delayed feedback, without relying on external models or prior knowledge. Reinforcement learning enables unmanned aerial vehicles (UAVs) to exhibit strong adaptability and flexibility in the face of unknown or dynamically changing battlefield environments. Furthermore, reinforcement learning models typically possess good generalization capabilities after training, enabling them to handle novel problems and scenarios not previously encountered. In multi-UAV cooperative combat systems, reinforcement learning can promote collaboration and coordination among UAVs to cope with large-scale strike missions.

[0005] Although existing multi-agent reinforcement learning methods such as MADDPG (Multi-Agent Deep Deterministic Policy Gradient) have achieved certain results in multi-UAV cooperative tasks, they still have problems such as unstable path planning and low task execution efficiency when performing complex tasks. Summary of the Invention

[0006] To address the aforementioned technical problems, this invention provides a UAV cooperative strike method based on reinforcement learning and dynamically adjusted artificial potential field. Combining the advantages of the artificial potential field method, it aims to improve the efficiency and stability of UAV strike missions, avoid collisions between UAVs, and enhance the robustness of mission completion.

[0007] The technical solution adopted in this invention is: a UAV cooperative strike method based on reinforcement learning and dynamic adjustment of artificial potential field, the specific steps of which are as follows:

[0008] S1. Establish and initialize a multi-UAV collaborative strike mission model;

[0009] S11. Initialize the task scenario settings;

[0010] The mission scenario is set as a square area with a side length of 'a' kilometers. The number of particles of each type—drone, obstacle, and target point—is set to 'n'. The square area is divided into three equal regions, and particles of the same type are placed in the same region. At the start of each round, the positions of all particles are randomly initialized within their respective regions to cope with different environments. During each round's initialization, drone particles and target particles must avoid collisions.

[0011] S12. Establish a lidar detection model for fixed-wing UAVs;

[0012] m laser radars of equal spacing are arranged around each fixed-wing UAV, and each laser radar has a length of L. When an obstacle approaches the UAV, the length of the laser radar is shortened to l∈[0,L].

[0013] S2. Based on step S1, design a method for dynamically adjusting the artificial potential field;

[0014] S3. Based on step S2, the dynamic adjustment of the artificial potential field method and the MADDPG algorithm are combined to construct a multi-UAV cooperative multi-target strike algorithm, and the state space of the algorithm is designed.

[0015] S4. Based on the multi-UAV cooperative multi-target strike algorithm constructed in step S3, design the algorithm's reward function;

[0016] S5. Based on the algorithm designed in step S4, train the neural network model, and then apply the neural network model to specific scenarios for iteration to complete the UAV collaborative strike mission under fixed altitude conditions in mountainous areas.

[0017] Furthermore, step S2 is specifically as follows:

[0018] S21. Define the artificial potential field method;

[0019] First, define the gravitational field U for the artificial potential method. att (q) and gravity F att (q), the expression is as follows:

[0020]

[0021] F att (q)=-▽U att (q)=Katt (q goal -q)

[0022] Among them, K att Let q represent the gravitational scale factor, and q represent the UAV coordinates. goal Represents the coordinates of the target point, ρ(q,q goal () indicates the distance between the drone and the target point.

[0023] Redefining the repulsive field U of the artificial potential field method rep (q) and repulsive force F rep (q), the expression is as follows:

[0024]

[0025] Among them, K rep q represents the repulsive force scale factor. obs Represents the coordinates of the obstacle, ρ(q,q) obs ) represents the distance between the drone and the obstacle, and ρ0 represents the range of the repulsive force. The repulsive force is generated within the range and is zero outside the range.

[0026] S22. Based on the artificial potential field method in step S21, define a dynamically adjustable artificial potential field method;

[0027] First, the gravitational field and gravitational function are corrected, as shown in the following expressions:

[0028]

[0029] Where d represents the gravitational distance factor.

[0030] Then, the repulsive field function is modified by adding the square power of the distance between the target and the drone, as shown in the following expression:

[0031]

[0032] When the drone is detected to be in a local optimum, a random perturbation is applied to it to cause it to move out of the local optimum. Finally, the gravitational and repulsive fields are added together to obtain the total potential field, and the total force is obtained similarly, as expressed below:

[0033] U total =U att +U rep

[0034] F total =F att +F rep

[0035] Furthermore, step S3 is specifically as follows:

[0036] For the n unmanned aerial vehicles (UAVs) mentioned in S11, the state space of UAV 1 includes: position information, velocity information, radar information, and artificial potential force information. Specifically:

[0037] (1) The position information in the state space includes: the position of UAV 1. agent.pos The relative positions of other drones other.pos The relative positions of n target points o landmark.pos .

[0038] (2) The velocity information in the state space includes: the velocity of UAV 1 o agent.vel .

[0039] (3) Radar information in the state space includes: the radar length of UAV 1. L .

[0040] (4) Artificial potential force information in state space force .

[0041] For each of the UAVs 2, 3, ..., n, steps (1)-(4) are designed separately.

[0042] In calculating the dynamic adjustment of artificial potential field forces, for n target points, n resultant force information is calculated, which is the sum of the repulsive force of all obstacles and the attractive force of a single target point. This ensures that each resultant force information can guide the UAV to the corresponding target point.

[0043] Furthermore, step S4 is specifically as follows:

[0044] The reward function of the algorithm is designed to include: reward for the drone hitting the target, penalty for collision between the drone and obstacles, penalty for collision between drones, penalty for the drone going out of bounds, reward for the drone's target guidance, reward for the artificial potential field, reward for the drone's speed, and reward for the drone's turning, as detailed below:

[0045] 1) Reward R for drone strike targets cover When ρ(q,q) is satisfied goal )≤d(q,q goal When the drone strikes the target point, it can be determined that the drone has completed the strike.

[0046] Where d(q,q) goal This represents the distance threshold at which the drone can strike the target point.

[0047] 2) Collision penalty between drones and obstacles R collide.o When ρ(q,q) is satisfied obs )≤d(q,q obsWhen this happens, it can be determined that a collision has occurred between the drone and the obstacle.

[0048] Where d(q,q) obs ) represents the distance threshold at which the drone collides with an obstacle.

[0049] 3) Collision penalty between drones R collide.a .

[0050] 4) Out-of-bounds penalty for drones R bound When the drone's current position deviates from the predetermined boundary, a penalty value will be applied to it.

[0051] 5) Target-guided reward for drones R dist The reward value increases as the drone gets closer to the target, and R dist =-ρ(q,q goal ).

[0052] 6) Artificial potential field reward R APF At the start of each round, the obstacle avoidance path of the drone to different target points is obtained by dynamically adjusting the artificial potential field method based on the positions of the drone, obstacles, and target points. The expression is as follows:

[0053]

[0054] Among them, dist agent.path This indicates the distance from the drone to the obstacle avoidance path. The closer the drone is to the obstacle avoidance path, the higher the reward value.

[0055] 7) Drone speed bonus R v The drone's speed is set with upper and lower limits. Exceeding the limits will result in a penalty value, expressed as follows:

[0056]

[0057] Where V represents the current velocity scalar of the drone, K1 and K2 are both coefficients greater than 0, and B1 and B2 are both constants. t This indicates the speed related to the speed limit of fixed-wing drones.

[0058] 8) Drone Turning Reward R r The drone is also subject to maneuverability limits. When the maneuverability is exceeded, a penalty value is applied, expressed as follows:

[0059] R r =-K r θ, θ>θ0

[0060] Among them, K r>0, θ represents the angle between the velocity vectors at the current moment and the previous moment, and θ0 represents the angle value related to the turning characteristics of the fixed-wing UAV.

[0061] The beneficial effects of this invention are as follows: First, the method of this invention establishes and initializes a multi-UAV cooperative strike mission model according to requirements. Then, it designs a dynamically adjusted artificial potential field method and combines this method with the MADDPG algorithm, designing the state space and reward function respectively. Finally, it iterates the algorithm and applies it to a specific scenario to complete a multi-UAV cooperative strike mission under fixed altitude conditions in mountainous areas. By designing and optimizing the algorithm's state space and reward function, the method of this invention enables the UAV system to avoid obstacles and efficiently complete strike missions in dynamically changing environments. It solves the problems of easily getting trapped in local optima and collisions in existing methods. It not only effectively solves the multi-UAV cooperative strike problem but also ensures the stability and efficiency of the UAV system under different environments and initial conditions. By combining reinforcement learning algorithms and the dynamically adjusted artificial potential field method, it provides a more intelligent and flexible multi-UAV cooperative strike algorithm with broad application prospects. Attached Figure Description

[0062] Figure 1 This is a flowchart of a UAV cooperative strike method based on reinforcement learning and dynamic adjustment of artificial potential field according to the present invention.

[0063] Figure 2 This is a schematic diagram of a fixed-wing UAV lidar detection model in an embodiment of the present invention.

[0064] Figure 3 This is a schematic diagram of the round reward curve obtained from training the multi-UAV cooperative multi-target attack algorithm described in this embodiment of the invention.

[0065] Figure 4 This is a trajectory diagram of multiple UAVs coordinating three-dimensional strikes against multiple targets in an embodiment of the present invention. Detailed Implementation

[0066] The method of the present invention will be further described below with reference to the accompanying drawings and embodiments.

[0067] like Figure 1 The flowchart shown is a method for cooperative UAV strike based on reinforcement learning and dynamically adjusting artificial potential fields according to the present invention. The specific steps are as follows:

[0068] S1. Establish and initialize a multi-UAV collaborative strike mission model;

[0069] S11. Initialize the task scenario settings;

[0070] In this embodiment, the mission scenario is set as a square area with a side length of a = 8 kilometers. The number of particles of each type—drone, obstacle, and target point—is set to n = 3. The square area is divided into three equal regions, and particles of the same type are placed in the same region. At the start of each round, the positions of all particles are completely randomly initialized within their respective regions to cope with different environments. Furthermore, during each round's initialization, drone particles and target particles must avoid collisions.

[0071] S12. Establish a lidar detection model for fixed-wing UAVs;

[0072] like Figure 2 As shown, m = 16 lidars are arranged at equal intervals around each fixed-wing UAV, and each lidar has a length of L. When an obstacle approaches the UAV, the length of the lidar is shortened to l∈[0,L].

[0073] S2. Based on step S1, design a method for dynamically adjusting the artificial potential field;

[0074] S3. Based on step S2, the dynamic adjustment of the artificial potential field method and the MADDPG algorithm are combined to construct a multi-UAV cooperative multi-target strike algorithm, and the state space of the algorithm is designed.

[0075] S4. Based on the multi-UAV cooperative multi-target strike algorithm constructed in step S3, design the algorithm's reward function;

[0076] S5. Based on the algorithm designed in step S4, train the neural network model, and then apply the neural network model to specific scenarios for iteration to complete the UAV collaborative strike mission under fixed altitude conditions in mountainous areas.

[0077] In this embodiment, step S2 is specifically as follows:

[0078] S21. Define the artificial potential field method;

[0079] First, define the gravitational field U for the artificial potential method. att (q) and gravity F att (q), the expression is as follows:

[0080]

[0081] F att (q)=-▽U att (q)=K att (q goal -q)

[0082] Among them, K att Let q represent the gravitational scale factor, and q represent the UAV coordinates. goal Represents the coordinates of the target point, ρ(q,q goal() indicates the distance between the drone and the target point.

[0083] Redefining the repulsive field U of the artificial potential field method rep (q) and repulsive force F rep (q), the expression is as follows:

[0084]

[0085]

[0086] Among them, K rep q represents the repulsive force scale factor. obs Represents the coordinates of the obstacle, ρ(q,q) obs ) represents the distance between the drone and the obstacle, and ρ0 represents the range of the repulsive force. The repulsive force is generated within the range and is zero outside the range.

[0087] The artificial potential field method described in step S21 is based on the definition of classical artificial potential field theory. The shortcomings of this algorithm are as follows:

[0088] A: When the drone is far from the target point, according to the gravity formula, the magnitude of gravity is proportional to the distance between the drone and the target point. At this time, the gravity will become very large, while the repulsive force will be relatively small. Therefore, the drone is prone to colliding with obstacles while moving.

[0089] B: As the drone approaches the target point, the attraction of the target point will be significantly reduced. At the same time, if there are obstacles around the target point, the repulsive force generated by the obstacles will increase sharply. This repulsive force is inversely proportional to the distance between the agent and the obstacle. In this case, the drone will face the problem of not being able to reach the target point.

[0090] C: During the drone's travel path, the resultant force of the repulsive and attractive forces acting on the drone at a certain position may be zero, thus causing the drone to fall into a "local optimum".

[0091] The corresponding improvement methods are given for the three shortcomings, namely the dynamic adjustment of the artificial potential field method.

[0092] S22. Based on the artificial potential field method in step S21, define a dynamically adjustable artificial potential field method;

[0093] First, the gravitational field and gravitational function are corrected, as shown in the following expressions:

[0094]

[0095] Here, d represents the gravitational distance factor, which reduces the gravitational force at greater distances. By modifying the gravitational field and gravitational function, the gravitational force experienced by the UAV should decrease when it is far from the target point, with the reduction depending on the distance.

[0096] Then, the repulsive field function is modified by adding the square power of the distance between the target and the drone, as shown in the following expression:

[0097]

[0098]

[0099] As the drone approaches the target, although the intensity of the repulsive field it experiences increases, the actual distance between the drone and the target decreases. Therefore, the rate at which the repulsive field increases as the drone approaches the target is slower than before.

[0100] For problem C, when the drone is detected to be in a local optimum, a random perturbation is given to the drone (specifically, the resultant force at this time is a random value, the magnitude of which depends on the determination of specific parameters) to make the drone escape from the local optimum.

[0101] Finally, the gravitational and repulsive fields are added together to obtain the total potential field, and the total force is obtained similarly, as expressed below:

[0102] U total =U att +U rep

[0103] F total =F att +F rep

[0104] In this embodiment, step S3 is specifically as follows:

[0105] For the n unmanned aerial vehicles (UAVs) mentioned in S11, the state space of UAV 1 includes: position information, velocity information, radar information, and artificial potential force information. Specifically:

[0106] (1) The position information in the state space includes: the position of UAV 1. agent.pos The relative positions of other drones other.pos The relative positions of n target points o landmark.pos .

[0107] (2) The velocity information in the state space includes: the velocity of UAV 1 o agent.vel .

[0108] (3) Radar information in the state space includes: the radar length of UAV 1. LThe distance of the LiDAR sensor can be used to determine whether an obstacle is close to the drone. Therefore, for the same scene with varying numbers of obstacles, there is no need to repeatedly train the model, because the drone's state space does not directly contain the obstacle positions. This greatly improves the generalization ability of the trained model.

[0109] (4) Artificial potential force information in state space force After introducing a dynamically adjustable artificial potential field, the potential field values ​​at each point in the environmental model are different. Based on these differences in potential field values, a path can be formed that avoids obstacles and runs from the starting point to the end point. The force applied to the drone by the dynamically adjusted artificial potential field maps the positional relationship between the particles to a certain extent.

[0110] For both UAVs 2 and 3, steps (1)-(4) are designed separately.

[0111] In calculating the dynamic adjustment of artificial potential forces, for n=3 target points, n resultant force information is calculated, which is the sum of the repulsive forces of all obstacles and the attractive forces of a single target point. This ensures that each resultant force information can guide the UAV to the corresponding target point.

[0112] In this embodiment, step S4 is specifically as follows:

[0113] The reward function of the algorithm is designed to include: reward for the drone hitting the target, penalty for collision between the drone and obstacles, penalty for collision between drones, penalty for the drone going out of bounds, reward for the drone's target guidance, reward for the artificial potential field, reward for the drone's speed, and reward for the drone's turning, as detailed below:

[0114] 1) Reward R for drone strike targets cover When ρ(q,qgoal)≤d(q,qgoal), it can be determined that the UAV has completed the strike on the target point.

[0115] Where d(q,q) goal ) represents the distance threshold at which the drone can strike the target point. In this embodiment, R cover =8.

[0116] 2) Collision penalty between drones and obstacles R collide.o When ρ(q,q) is satisfied obs )≤d(q,q obs When this happens, it can be determined that a collision has occurred between the drone and the obstacle.

[0117] Where d(q,q) obs ) represents the distance threshold at which the drone collides with an obstacle. In this embodiment, R collide.o = -30.

[0118] 3) Collision penalty between drones R collide.a In this embodiment, R collide.a =-7.

[0119] 4) Out-of-bounds penalty for drones R bound If no scene area is defined, the drone may deviate from the predetermined task execution area. This could prevent the drone from accumulating effective experience, causing it to become stuck in its current state, affecting training efficiency, and even stagnate the training process. To prevent this, a penalty value is applied when the drone's current position deviates from the predetermined boundary. In this embodiment, R... bound =-2.

[0120] 5) Target guidance reward for drones R dist When drones perform cooperative strike missions, if they rely solely on strike and collision rewards while ignoring the interaction and feedback with the environment during exploration, they will struggle to accumulate effective experience, thus affecting their subsequent learning efficiency. To accelerate reinforcement learning training, a target-guided reward is introduced. This reward value increases as the agent approaches the target, and R... dist =-ρ(q,q goal ).

[0121] 6) Artificial potential field reward R APF At the start of each round, the obstacle avoidance path of the drone to different target points is obtained by dynamically adjusting the artificial potential field method based on the positions of the drone, obstacles, and target points. The expression is as follows:

[0122]

[0123] Among them, dist agent.path This indicates the distance from the drone to the obstacle avoidance path. The closer the drone is to the obstacle avoidance path, the higher the reward value.

[0124] 7) Drone speed bonus R v The drone's speed is set with upper and lower limits. Exceeding the limits will result in a penalty value, expressed as follows:

[0125]

[0126] Where V represents the current speed scalar of the drone, K1 and K2 in this embodiment both represent coefficients greater than 0, namely 0.2 and 0.1, and B1 and B2 are constants, namely -1 and 2, respectively. t This represents the speed related to the speed limit of the fixed-wing UAV; in this embodiment, the value is 10.

[0127] 8) Drone Turning Reward R r The drone is also subject to maneuverability limitations; when these limitations are exceeded, i.e. Given a penalty value, the expression is as follows:

[0128] R r =-K r θ, θ>θ0

[0129] Among them, K r >0, in this embodiment the value is 10, θ represents the angle between the velocity vectors at the current moment and the previous moment. This represents the angle value related to the turning characteristics of a fixed-wing UAV.

[0130] In this embodiment, step S5 is specifically as follows:

[0131] In this embodiment, the number of neurons in the Actor and Critic networks is set to 64, the number of hidden layers in both networks is 3, the learning rate is 0.005, the discount factor is 0.95, the experience replay pool capacity is 100,000, the batch size is 1024, the total number of iterations is 80,000, and the maximum step size per iteration is 40. Substituting these parameter values, training can begin, resulting in the following... Figure 3 The reward curve shown is from... Figure 3 It can be seen that the multi-UAV cooperative multi-target attack algorithm converges in approximately 65,000 rounds, with a reward value of about 1500. The trained neural network model can obtain the UAV's path points with a single inference, and plot the path as shown below. Figure 4 The diagram shows the trajectory of a multi-UAV coordinated strike against multiple targets in three dimensions. This verifies the effectiveness of the method described in this invention in the field of multi-UAV coordinated strikes.

[0132] In summary, the method of this invention, in terms of collaborative decision-making and environmental adaptability, introduces the dynamically adjusted artificial potential field method into the MADDPG reinforcement learning algorithm. This not only enables decision-making through reinforcement learning but also leverages the power of the artificial potential field to achieve more efficient path planning. This combination enhances the algorithm's adaptability and robustness in complex dynamic environments. Regarding path planning, existing MADDPG algorithms may be inefficient in environments with dense obstacles or complex environments. The dynamically adjusted artificial potential field method guides the UAV to avoid obstacles, ensuring the safety and effectiveness of path planning. In terms of collaborative strike efficiency, the method of this invention, through the combination of the MADDPG algorithm and the dynamically adjusted artificial potential field method, enables UAVs to achieve more comprehensive target strikes in collaborative work, reducing duplicate strikes or missed strikes, and improving the overall mission completion quality. In terms of scalability, the method of this invention has good scalability and can be customized and optimized according to specific needs, making it suitable for UAV collaborative missions of different scales and complexities.

[0133] Those skilled in the art will recognize that the embodiments described herein are intended to help the reader understand the principles of the invention, and should be understood that the scope of protection of the invention is not limited to such specific statements and embodiments. Those skilled in the art can make various other specific modifications and combinations based on the technical teachings disclosed in this invention without departing from the spirit of the invention, and these modifications and combinations are still within the scope of protection of this invention.

Claims

1. A method for cooperative drone strikes based on reinforcement learning and dynamically adjusting the artificial potential field, the specific steps of which are as follows: S1. Establish and initialize a multi-UAV collaborative strike mission model; S11. Initialize the task scenario settings; The task scenario is set as a square area with a side length of a kilometers. The number of particles of the three types—drones, obstacles, and target points—is set to n. The square area is divided into three equal areas, and particles of the same type are placed in the same area. At the beginning of each round, the positions of all particles are completely randomly initialized in their respective areas to cope with different environments. During the initialization of each round, each drone particle and target particle must avoid collisions. S12. Establish a lidar detection model for fixed-wing UAVs; m laser radars of equal spacing are arranged around each fixed-wing UAV, and each laser radar has a length of L. When an obstacle approaches the UAV, the length of the laser radar is shortened to l∈[0,L]. S2. Based on step S1, design a method for dynamically adjusting the artificial potential field; S3. Based on step S2, the dynamic adjustment of the artificial potential field method and the MADDPG algorithm are combined to construct a multi-UAV cooperative multi-target strike algorithm, and the state space of the algorithm is designed. S4. Based on the multi-UAV cooperative multi-target strike algorithm constructed in step S3, design the algorithm's reward function; S5. Based on the algorithm designed in step S4, train the neural network model, and then apply the neural network model to specific scenarios for iteration to complete the UAV collaborative strike mission under fixed altitude conditions in mountainous areas.

2. The UAV cooperative strike method based on reinforcement learning and dynamically adjusted artificial potential field according to claim 1, characterized in that, Step S2 is as follows: S21. Define the artificial potential field method; First, define the gravitational field U for the artificial potential method. att (q) and gravity F att (q), the expression is as follows: F att (q)=-▽U att (q)=K att (q goal -q) Among them, K att Let q represent the gravitational scale factor, and q represent the UAV coordinates. goal Represents the coordinates of the target point, ρ(q,q goal This indicates the distance between the drone and the target point; Redefining the repulsive field U of the artificial potential field method rep (q) and repulsive force F rep (q), the expression is as follows: Among them, K rep q represents the repulsive force scale factor. obs Represents the coordinates of the obstacle, ρ(q,q) obs ) represents the distance between the drone and the obstacle, and ρ0 represents the range of the repulsive force. The repulsive force is generated within the range and is zero outside the range. S22. Based on the artificial potential field method in step S21, define a dynamically adjustable artificial potential field method; First, the gravitational field and gravitational function are corrected, as shown in the following expressions: Where d represents the gravitational distance factor; Then, the repulsive field function is modified by adding the square power of the distance between the target and the drone, as shown in the following expression: When the drone is detected to be in a local optimum, a random perturbation is given to the drone to cause it to move out of the local optimum. Finally, the gravitational and repulsive fields are added together to obtain the total potential field, and the total force is obtained similarly, as expressed below: IN total =U att +U rep F total =F att +F rep 。 3. The UAV cooperative strike method based on reinforcement learning and dynamically adjusted artificial potential field according to claim 1, characterized in that, Step S3 is as follows: For the n UAVs mentioned in S11, the state space of UAV 1 includes: position information, velocity information, radar information, and artificial potential force information in the state space; specifically as follows: (1) The position information in the state space includes: the position of UAV 1. agent.pos The relative positions of other drones other.pos The relative positions of n target points o landmark.pos ; (2) The velocity information in the state space includes: the velocity of UAV 1 o agent.vel ; (3) Radar information in the state space includes: the radar length of UAV 1. L ; (4) Artificial potential force information in state space force ; For each of the UAVs 2, 3, ..., n, steps (1)-(4) are designed separately. In calculating the dynamic adjustment of artificial potential field forces, for n target points, n resultant force information is calculated, which is the sum of the repulsive force of all obstacles and the attractive force of a single target point. This ensures that each resultant force information can guide the UAV to the corresponding target point.

4. The UAV cooperative strike method based on reinforcement learning and dynamically adjusted artificial potential field according to claim 1, characterized in that, Step S4 is as follows: The reward function of the algorithm is designed to include: reward for the drone hitting the target, penalty for collision between the drone and obstacles, penalty for collision between drones, penalty for the drone going out of bounds, reward for the drone's target guidance, reward for the artificial potential field, reward for the drone's speed, and reward for the drone's turning, as detailed below: 1) Reward R for drone strike targets cover When ρ(q,q) is satisfied goal )≤d(q,q goal When the drone strikes the target point, it can be determined that the drone has completed the strike. Where d(q,q) goal This indicates the distance threshold at which the drone can strike the target point; 2) Collision penalty between drones and obstacles R collide.o When ρ(q,q) is satisfied obs )≤d(q,q obs When this happens, it can be determined that a collision has occurred between the drone and the obstacle; Where d(q,q) obs This indicates the distance threshold at which a drone collides with an obstacle; 3) Collision penalty between drones R collide.a ; 4) Out-of-bounds penalty for drones R bound When the drone's current position deviates from the predetermined boundary, a penalty value will be applied to it; 5) Target-guided reward for drones R dist The reward value increases as the drone gets closer to the target, and R dist =-ρ(q,q goal ); 6) Artificial potential field reward R APF At the start of each round, the obstacle avoidance path of the drone to different target points is obtained by dynamically adjusting the artificial potential field method based on the positions of the drone, obstacles, and target points. The expression is as follows: Among them, dist agent.path This indicates the distance from the drone to the obstacle avoidance path. The closer the drone is to the obstacle avoidance path, the higher the reward value. 7) Drone speed bonus R v The drone's speed is set with upper and lower limits. Exceeding the limits will result in a penalty value, expressed as follows: Where V represents the current velocity scalar of the drone, K1 and K2 are both coefficients greater than 0, and B1 and B2 are both constants. t This indicates the speed related to the speed limit of fixed-wing drones; 8) Drone Turning Reward R r The drone is also subject to maneuverability limits. When the maneuverability is exceeded, a penalty value is applied, expressed as follows: R r =-K r θ,θ>θ0 Among them, K r >0, θ represents the angle between the velocity vectors at the current moment and the previous moment, and θ0 represents the angle value related to the turning characteristics of the fixed-wing UAV.

Images