A method for UAV swarm trajectory planning in urban environment

Abstract

The present invention discloses a method for planning the trajectory of a drone swarm in an urban environment, including three-dimensional modeling of the urban environment through a grid method, accurately expressing the threat of the enemy's weapons through mathematical expressions in the three-dimensional modeling by mathematical modeling, and continuously exploring the constructed environment by using a reinforcement learning algorithm, and finally finding an optimal trajectory to avoid tall buildings in the city and the enemy's defensive fire attack range, and accurately strike the mission target. The technical solution of the present invention, the drone swarm, supported by situational awareness and collaborative combat technology, finds out the enemy's deployment, understands the trend, and predicts the deployment, accurately analyzes the strike target, avoids the enemy's fire detection range, building obstacles, etc., finds the optimal strike path, and implements accurate and efficient strikes on the mission target to achieve the tactical purpose.

Description

Technical Field

[0001] The present invention belongs to the technical field of unmanned aerial vehicles, and in particular relates to a method for planning the trajectory of an unmanned aerial vehicle cluster in an urban environment. Background Art

[0002] As drone swarms demonstrate great efficiency and application potential, drones are widely used in various fields, among which the application of drone swarms in urban environments is the most promising. With the continuous evolution of war forms and the continuous improvement of artificial intelligence technology, modern battlefields are showing characteristics such as intelligence and unmanned operations. Inspired by the "biological swarms" in nature, combined with the many characteristics of drones such as low cost, large scale, high anti-destruction, and high mobility, drones often perform reconnaissance and surveillance, fire strikes, communication relays, information confrontation and other tasks in combat in a swarm manner. They have the greatest overall efficiency and are widely used. In view of the problems of limited equipment performance, extremely complex environment, and difficult trajectory planning in urban environments, combined with the advantages of drone swarms such as good stealth, strong mobility, and sufficient attack firepower, it is very urgent and necessary to build a low-cost, low-cost, and high-efficiency drone swarm urban trajectory planning mechanism. Summary of the invention

[0003] In view of the above-mentioned deficiencies in the prior art, the present invention invents a method for UAV cluster trajectory planning based on an urban environment.

[0004] In order to achieve the above object, the present invention adopts the following technical solution:

[0005] A method for UAV cluster trajectory planning in an urban environment includes the following steps:

[0006] Step 1: The drone uses the grid method to perform three-dimensional modeling of the urban environment, depicting the complexity of the urban environment;

[0007] Step 2: Mark and construct an urban environment model in the 3D modeling based on the pre-set mission objectives and pre-detected weapon threat centers;

[0008] Step 3: During the flight of the drone cluster, mathematical modeling is used to calculate the distance between the mth drone and the ith weapon threat center, and the lethality of weapon fire attack on drones in an urban environment is analyzed;

[0009] The straight-line distance from the mth UAV to the ith weapon threat center for:

[0010]

[0011] The mathematical model of the weapon threat is expressed as:

[0012]

[0013] in, To compare the judged weapon threat utility, represents the threat utility value of the mth UAV in the ith weapon threat center area, R i is the weapon threat radius, (x m ,y m ,z m ) is the real-time coordinate of the mth UAV, is the coordinate of the ith weapon threat center;

[0014] Step 4: The UAV uses the reinforcement learning algorithm to calculate in real time the optimal trajectory of the mth UAV to reach the mission target under the conditions of urban environment and weapon threat;

[0015] Step 5: The drones select their respective mission targets to attack based on the calculated optimal trajectory, and update the optimal trajectory based on the real-time updated environmental information in the urban environment model.

[0016] The technical solution of the present invention is that the drone cluster, with the support of situational awareness and collaborative combat technology, can identify the enemy's deployment, understand its trends, predict its deployment, accurately analyze the target, avoid the enemy's fire detection range, building obstacles, etc., find the optimal strike path, and implement accurate and efficient strikes on the mission target to achieve tactical goals.

[0017] Furthermore, the grid method divides the urban environment into several identical grids according to the proportion and segmentation requirements, sets environmental parameters for each grid, sets obstacles in each grid according to the actual or hypothetical environment, and maps the constraints that the algorithm needs to deal with with the obstacles in the physical environment one by one, thereby providing an urban environment model for the navigation of the drone.

[0018] Furthermore, the weapon threats include anti-aircraft artillery threats, tank threats, electromagnetic gun threats, mobile fortress threats and / or missile threats, and any threat that causes damage to a drone is considered a weapon threat.

[0019] Furthermore, the environmental information in step 5 is a newly appeared obstacle and / or a newly appeared weapon threat center.

[0020] Furthermore, the reinforcement learning calculation method is:

[0021] S1: The drone explores the constructed urban environment model and obtains the drone observation status;

[0022] S2: Make a judgment based on the currently acquired environmental information, select an action with a high probability and execute it;

[0023] S3: Execute actions and enter the next observation state according to the changes in the urban environment. Each action performed will receive a corresponding reward;

[0024] S4: The current state, the currently executed action, the next observed state, and the return reward are stored as a piece of information in the experience pool. When there are many pieces of information in the experience pool, random sampling is used to extract pieces of information from the system experience pool to train the neural network and optimize the neural network parameters until the next state is the set termination state and the number of iterative training reaches the threshold. Otherwise, the state of the drone is reset and training is repeated to finally obtain the optimal trajectory of the drone.

[0025] Furthermore, if the action currently performed by the drone is identified as an incorrect action, the system experience pool will give a negative reward for this incorrect action; if the action is identified as a correct action, the system experience pool will give a positive reward for this correct action. The track with the highest positive reward is the optimal track.

[0026] Furthermore, the observation state is constrained by the roll angle, pitch angle and yaw angle of the drone, which is more in line with the actual flight state of the drone.

[0027] The advantages of the present invention are as follows: the urban environment trajectory planning of the present invention firstly performs three-dimensional modeling of the urban environment through the grid method to depict the complexity of the urban environment, and at the same time uses mathematical modeling to accurately express the threat of enemy weapons in the form of mathematical expressions to depict the range of the threat area, and then uses the reinforcement learning algorithm to continuously explore the constructed environment, obtain observation values, and automatically optimize by maximizing the cumulative reward, and finally finds an optimal trajectory to avoid the tall buildings in the city and the enemy's fire attack range, and accurately strikes the mission target, ensuring that our combat objectives are achieved at the lowest cost. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] Figure 1 It is a schematic diagram of a UAV cluster trajectory planning method based on an urban environment according to the present invention. DETAILED DESCRIPTION

[0029] The present invention is described in detail below with reference to the accompanying drawings:

[0030] like Figure 1 As shown, a method for UAV cluster trajectory planning in an urban environment includes the following steps:

[0031] Step 1: The drone uses the grid method to perform three-dimensional modeling of the urban environment, depicting the complexity of the urban environment, that is, simulating the real scene, replicating the physical space into the abstract space, and establishing an environmental model that can and is easy for computers to process. Its essence is to map the constraints that the algorithm needs to deal with with the obstacles in the physical environment one by one, and to make the path search situation intuitively displayed in the virtual scene, so as to conveniently and intuitively show the path planning scene and planning results.

[0032] Step 2: Mark and construct the urban environment model in the 3D modeling according to the pre-set mission objectives and the pre-detected weapon threat centers, and add factors that affect the trajectory planning in the 3D modeling to ensure that the simulated urban environment is close to the real situation. The weapon threats include anti-aircraft artillery threats, tank threats, electromagnetic gun threats, mobile fortress threats and / or missile threats. Any unit that poses a threat to the drone can be identified as a weapon threat center.

[0033] Step 3: During the flight of the drone cluster, mathematical modeling is used to calculate the distance between the mth drone and the ith weapon threat center, and the lethality of weapon fire attack on drones in an urban environment is analyzed;

[0034] The straight-line distance from the mth UAV to the ith weapon threat center for:

[0035]

[0036] The mathematical model of the weapon threat is expressed as:

[0037]

[0038] in, To compare the judged weapon threat utility, represents the threat utility value of the mth UAV in the ith weapon threat center area, R i is the weapon threat radius, (x m ,y m ,z m ) is the real-time coordinate of the mth UAV, is the coordinate of the ith weapon threat center;

[0039] Step 4: The UAV uses the reinforcement learning algorithm to calculate in real time the optimal trajectory of the mth UAV to reach the mission target under the conditions of urban environment and weapon threat.

[0040] Step 5: The drones select their respective mission targets to attack based on the calculated optimal trajectory, and update the optimal trajectory based on the real-time updated environmental information in the urban environment model; the environmental information is the newly emerged obstacles and / or the newly emerged weapon threat center, and the model variables are updated in real time. For example, when the mth drone passes a certain location and is shot down by the newly emerged weapon threat center, the newly emerged weapon threat center will be transmitted back to the drone cluster before the drone is shot down. The drone cluster will automatically update the urban environment model to provide a new urban environment model for the next drone to perform a mission until all pre-set tasks are completed or a command to stop the mission is received.

[0041] The grid method is to divide the urban environment into several identical grids according to the proportion and segmentation requirements, set environmental parameters for each grid, and set obstacles in each grid according to the actual or hypothetical environment. The grid method is used to divide the urban planning space into regularly arranged and equally sized discrete cells, simplify urban buildings into a single cuboid or a combination of multiple cuboids, and establish a three-dimensional urban combat space model based on gridding, so that the location of obstacles (weapon threat center) and feasible areas can be more intuitively displayed, and each location information can be significantly identified by rows and columns, truly depicting the complexity of the urban environment.

[0042] The mathematical modeling is that in the case of complex and harsh combat environment, disordered spectrum resources, etc., the radar is first used to detect and lock the location of the enemy's weapon threat center, weapon type and other information, and then the radar processes the relevant information of the drone's navigation status and combines it with the detected weapon threat center to calculate the avoidance plan. There are many factors that affect missile strikes on drones, such as radar infrastructure, wide-mouth tilt direction, information processing capabilities, and the range of threat capture. Therefore, it is particularly important to have a simple and accurate mathematical model that reflects missile performance and threat range to provide avoidance measures for drones.

[0043] The reinforcement learning calculation method is:

[0044] S1: The drone explores the constructed urban environment model and obtains the drone’s observation state, which is constrained by the drone’s roll angle, pitch angle, and yaw angle;

[0045] S2: Make a judgment based on the currently acquired environmental information, select an action with a high probability and execute it;

[0046] S3: Execute actions and enter the next observation state according to the changes in the urban environment. Each action performed will receive a corresponding reward;

[0047] S4: The current state, the currently executed action, the next observed state, and the return reward are stored as a piece of information in the experience pool. If the action currently executed by the drone is identified as an incorrect action, the system experience pool will give a negative reward to this incorrect action. If the action currently executed by the drone is identified as a correct action, the system experience pool will give a positive reward to this correct action (that is, if the drone passes through a node every time and is not shot down by the enemy, the flight state passing through this node will receive a positive reward). When there are many pieces of information in the experience pool, random sampling is used to extract pieces of information from the system experience pool to train the neural network and optimize the neural network parameters until the next state is the set termination state and the number of iterative training reaches the threshold. Otherwise, the state of the drone is reset and the training is repeated to finally obtain the optimal trajectory of the drone.

[0048] In the process of environmental situation perception, the UAV swarm trajectory planning method of the present invention can make full use of UAV swarm technology to achieve covert reconnaissance and all-round detection, and effectively analyze the enemy's deployment, movements, intentions, etc., providing a basis for finding the optimal trajectory planning in the next step.

[0049] Although the specific embodiments of the present invention have been described and illustrated in detail above, it should be noted that various changes and modifications may be made to the above embodiments without departing from the spirit of the present invention and the scope of the appended claims.

Claims

1. A method for UAV cluster trajectory planning in an urban environment, characterized in that: The steps include: Step 1: The drone uses the grid method to perform three-dimensional modeling of the urban environment, depicting the complexity of the urban environment; Step 2: Mark and construct an urban environment model in the 3D modeling based on the pre-set mission objectives and pre-detected weapon threat centers; Step 3: During the flight of the drone cluster, mathematical modeling is used to calculate the distance between the mth drone and the ith weapon threat center, and the lethality of weapon fire attack on drones in an urban environment is analyzed; The straight-line distance from the mth UAV to the ith weapon threat center for: The mathematical model of the weapon threat is expressed as: in, To compare the judged weapon threat utility, represents the threat utility value of the mth UAV in the ith weapon threat center area, R i is the weapon threat radius, (x m ,y m ,z m ) is the real-time coordinate of the mth UAV, is the coordinate of the ith weapon threat center; Step 4: The UAV uses the reinforcement learning algorithm to calculate in real time the optimal trajectory of the mth UAV to reach the mission target under the conditions of urban environment and weapon threat; Step 5: The drones select their respective mission targets to attack based on the calculated optimal trajectory, and update the optimal trajectory based on the real-time updated environmental information in the urban environment model.

2. The method for swarm trajectory planning of unmanned aerial vehicles in an urban environment according to claim 1 is characterized in that: The grid method divides the urban environment into a number of identical grids according to the proportion and segmentation requirements, sets environmental parameters for each grid, and sets obstacles in each grid according to the actual or assumed environment.

3. The method for UAV cluster trajectory planning in an urban environment according to claim 1 is characterized in that: The weapon threats include anti-aircraft artillery threats, tank threats, electromagnetic gun threats, mobile fortress threats and / or missile threats.

4. The method for swarm trajectory planning of unmanned aerial vehicles in an urban environment according to claim 1 is characterized in that: The environmental information in step 5 is a newly appeared obstacle and / or a newly appeared weapon threat center.

5. The method for UAV cluster trajectory planning in an urban environment according to claim 1 is characterized in that: The reinforcement learning calculation method is: S1: The drone explores the constructed urban environment model and obtains the drone observation status; S2: Make a judgment based on the currently acquired environmental information, select an action with a high probability and execute it; S3: Execute actions and enter the next observation state according to the changes in the urban environment. Each action performed will receive a corresponding reward; S4: The current state, the currently executed action, the next observed state, and the return reward are stored as a piece of information in the experience pool. When there are many pieces of information in the experience pool, random sampling is used to extract pieces of information from the system experience pool to train the neural network and optimize the neural network parameters until the next state is the set termination state and the number of iterative training reaches the threshold. Otherwise, the state of the drone is reset and training is repeated to finally obtain the optimal trajectory of the drone.

6. The method for UAV cluster trajectory planning in an urban environment according to claim 5 is characterized in that: If the action currently performed by the drone is identified as an incorrect action, the system experience pool will give a negative reward for this incorrect action.

7. The method for UAV cluster trajectory planning in an urban environment according to claim 5 is characterized in that: The observation state is constrained by the roll angle, pitch angle and yaw angle of the drone.

Images