A UAV swarm escort method based on state machine

Abstract

The present invention relates to a drone swarm escort method based on a state machine, and belongs to the technical field of drone swarm collaborative control. The present invention comprises the following steps: setting a list of path points of a protected target's route; calculating the relative position of each drone in the drone swarm and the protected target; constructing an initialization function; constructing a travel function; constructing a reconnaissance perception function; constructing a strike function; constructing a drone swarm escort state machine, and controlling according to the instructions of the state machine until the protected target moves to the end of the path point list of the route, thereby realizing the drone swarm's escort of the protected target. The present invention is oriented to the objective needs of drone swarm escort. The drone swarm and the protected target keep their relative positions still and form a formation, and move along the path navigation points with the protected target. By constructing a drone swarm escort state machine, the drone swarm and the protected target are dynamically switched in the moving, reconnaissance and strike states, destroying the opponent's interception equipment, and covering the protected target to reach the designated position.

Description

Technical Field

[0001] The present invention belongs to the technical field of cooperative control of unmanned aerial vehicle swarms, and in particular to a state machine-based unmanned aerial vehicle swarm escort method. Background Art

[0002] With the widespread application of drone swarms in the fields of collaborative positioning and relay communication, drone swarm control technology has become a current research focus. The escort capability of drone swarms is a key component of the adversarial game technology, which can ensure the delivery of resources and forces, thereby providing key support for achieving goals. However, there are many entities that need to be controlled in the drone swarm, which not only requires the coordination between drone groups, but also requires the ability to provide cover for the protection of targets, which brings challenges to the drone swarm collaborative control technology.

[0003] In the patent "A Variable Granularity Collaborative Search and Rescue Method for Unmanned Marine Formation Based on Reinforcement Learning" applied by Shandong University of Science and Technology, with the authorization announcement number CN117420849B, a secondary search space is constructed, and the collaborative lawnmower algorithm is first used to search in a coarse-grained environment, and then fine-grained capture is performed based on the coarse granularity, forming the cooperative search and rescue capability of unmanned aerial vehicles. Although this method greatly reduces the computational complexity by constructing a secondary search space, the difficulty of training the Actor-Critic framework still poses a challenge to the generation of collaborative search and rescue capabilities.

[0004] The paper "Air Combat Intelligent Decision-making Method Based on Self-Game Deep Reinforcement Learning" published by Shan Shengzhe and others from Northwestern Polytechnical University in the Journal of Aeronautics and Astronautics is based on the deep reinforcement learning method, combined with domain knowledge to improve the observation, action and reward information used for decision-making, and based on the self-game training method to form intelligent decision-making capabilities, overcoming the difficulty of traditional single-agent training methods to form high-level air combat decision-making capabilities. However, this method based on deep reinforcement learning not only requires the construction of a training platform, but also has high requirements for the training strategy of deep neural networks.

[0005] The paper "Research on Multi-aircraft Cooperative Air Combat Method Based on Deep Reinforcement Learning" published by Shi Wei and others from the National University of Defense Technology in the journal "Acta Automatica Sinica" proposed a multi-level cooperative air combat decision-making process framework based on deep reinforcement learning, and proposed an enhancement mechanism in combination with the proximal strategy optimization algorithm, which achieved multi-aircraft coordination and made the confrontation data interpretable afterwards. However, this method still does not get rid of the shortcomings of traditional reinforcement learning methods, that is, it requires a training platform and high-quality data for training.

[0006] In the patent "A multi-UAV collaborative penetration method based on heterogeneous multi-agent reinforcement learning" applied by the 54th Institute of China Electronics Technology Group Corporation, with the authorization announcement number CN113822409B, the generation of multi-UAV collaborative penetration capability is achieved through traction game training scenarios and evaluation game training scenarios. Although this method is based on course learning to generate traction capabilities, it reduces the demand for training data, but it still needs to build a training environment to generate strategies. Summary of the invention

[0007] In order to quickly form the ability of drone swarm escort, the present invention provides a drone swarm escort method based on a state machine. In the present invention, the drone swarm and the protected target remain stationary relative to each other and form a formation, move along the path navigation point with the protected target, and construct a drone swarm escort state machine, so that the drone swarm and the protected target can dynamically switch between the mobile, reconnaissance and strike states, destroy the opponent's interception equipment, and cover the protected target to reach the designated position.

[0008] The technical solution adopted by the present invention is:

[0009] A state machine-based drone swarm escort method is used to control the movement of a drone swarm and a protected target so that the drone swarm provides protection to the protected target, comprising the following steps:

[0010] Step 1: Set a list of path points of the protected target's route, and the protected target moves according to the coordinates of the path points in the list;

[0011] Step 2: Calculate the relative position of each drone in the drone group and the protected target;

[0012] Step 3: Construct an initialization function to control the protected target and each drone in the drone group to move to their respective initial positions according to the starting point of the route path point list and the relative position of each drone in the drone group and the protected target;

[0013] Step 4: Construct the march function;

[0014] Step 5: Construct reconnaissance perception function;

[0015] Step 6: Construct the attack function;

[0016] Step 7: Construct a drone swarm escort state machine. According to the instructions of the state machine, call the initialization function, travel function, reconnaissance perception function or strike function respectively to control the movement and action of the protected target and each drone in the drone swarm until the protected target moves to the end point of the route path point list, thereby realizing the escort of the drone swarm to the protected target.

[0017] Furthermore, in step 1, the list of path points along the route The coordinates of each path point in the two-dimensional scene are arranged in path order, that is:

[0018] =

[0019] N is the number of path points.

[0020] Furthermore, in step 2, the two-dimensional relative position of UAV m relative to the protected target ( , )for:

[0021]

[0022] Among them, m=1,2,...,M, M is the number of drones in the drone swarm, and R is the distance between the drone and the protected target.

[0023] Further, the initialization function works as follows:

[0024] Control the protected target to move to the first position (x1, y1) in the path point list of the route, and complete the protected target position ( , ) initialization;

[0025] Control the drone m to move to position ( , ), complete the position of UAV m ( , ), m=1,2,...,M.

[0026] Furthermore, the marching function is used to obtain the current two-dimensional position of the protection target ( , ), and matches and updates each path point in the path point list of the route. and When , the protected target is controlled to move to the coordinates of the n+1th path point in the list ( , ),renew( , ) = ( , ) and control the drone m to move to a new position ( , ), m=1,2,...,M.

[0027] Furthermore, the reconnaissance perception function is used to make each drone within its own reconnaissance perception range. The opponent's interception equipment is sensed and summarized into a list of the opponent's interception equipment positions. When there is no opponent's interception equipment within the reconnaissance perception range of each drone, the opponent's interception equipment position list is empty.

[0028] Furthermore, the attack function is used to destroy the opponent interception device at the corresponding position in the opponent interception device position list, and delete its position coordinates from the opponent interception device position list until the opponent interception device position list is empty.

[0029] Furthermore, in step 7, the state of the drone group escort state machine includes initialization, reconnaissance, attack, and movement; the control method is:

[0030] Set the state to 'initialize', call the initialization function, and then set the state to 'scout';

[0031] When the state is 'reconnaissance', call the reconnaissance perception function. If the opponent's interception device location list is empty, change the state to 'movement', otherwise change the state to 'attack';

[0032] When the state is 'moving', call the march function and then set the state to 'scout';

[0033] When the state is 'hit', call the hit function and then set the state to 'move'.

[0034] The beneficial effects of the present invention are:

[0035] 1. The method of the present invention is simple, the processing flow is clear, and the state machine is simple and easy to implement.

[0036] 2. The present invention does not require the construction of a simulation environment and the training of a deep learning network, which effectively reduces the difficulty of generating UAV escort capabilities. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] Figure 1 The figure is a flow chart of a method according to an embodiment of the present invention.

[0038] Figure 2 A schematic diagram of the route points.

[0039] Figure 3 Initial schematic diagram for escorting a swarm of drones.

[0040] Figure 4 Schematic diagram of the escort movement of a drone swarm.

[0041] Figure 5 Schematic diagram of escorting a swarm of drones to destroy the enemy's interception equipment.

[0042] Figure 6 Schematic diagram of escorting a swarm of drones to the destination. DETAILED DESCRIPTION

[0043] The present invention will be further described below in conjunction with the accompanying drawings and specific embodiments.

[0044] A drone swarm escort method based on a state machine, the overall process is as follows Figure 1 As shown, the specific steps include:

[0045] Step 1: Build a list of route points;

[0046] The number of path points along the route is N, and each path point corresponds to a coordinate in the two-dimensional scene ( , ), n=1,2,...,N. Summarize the path points in order and build a list of path points along the route for , the values ​​are [(10, 10), (12, 12), (12, 14), (14, 14), (16,16), (16, 18), (19, 20), (24, 20), (24, 23), (27, 25), (29, 26), (29, 30),(32, 33), (35, 34), (35, 37), (38, 40), (40, 41), (43, 43), (44, 45), (46,48), (48, 48)], and the route is as follows Figure 2 shown.

[0047] Step 2: Calculate the relative position of each drone in the drone group and the protected target;

[0048] The function name is circular_arragement, and the function input is the distance R between the drone and the protected target and the number of drones M. The two-dimensional relative position of each drone m relative to the protected target ( , ) as the output of the function, where m=1,2,...,M, then

[0049]

[0050] Step 3: Construct the initialization function;

[0051] The two-dimensional position of the protected target is ( , ), the initialization function assigns the first position in the list of path points of the route to the protection target position, that is, and , where the first position is [10, 10]. The position of each drone m in the drone swarm is ( , ), set the distance R between the drone and the protection target to 5m and the number of drones M=8, and obtain the relative position of each drone to the protection target as [(5.0, 0.0), (3.54, 3.54), (0.0, 5.0), (-3.54, 3.54), (-5.0, 0.0), (-3.54, -3.54), (-0.0, -5.0), (3.54, -3.54)], and update the drone position, that is, and .

[0052] Step 4: Build the march function:

[0053] The function name is moving_function, in which the current two-dimensional position of the protection target is obtained ( , ), matches and updates each path point in the path point list of the route, that is, when and , assign the n+1th path point in the list to the protection target, that is, and , and update the drone position, i.e. and .

[0054] Step 5: Construct the reconnaissance perception function:

[0055] The function name is countermeasure_function, which sets the drone reconnaissance perception range =5. When there is an interception device of the other party within the reconnaissance perception range, the function outputs a list of the location of the interception device of the other party; when there is no interception device of the other party within the reconnaissance perception range, the function outputs an empty list.

[0056] Step 6: Construct the attack function:

[0057] The function name is attack_function. It uses the opponent's interception device location list output by the reconnaissance perception function countermeasure_function to destroy the opponent's interception device and delete its location from the opponent's interception device location list.

[0058] Step 7: Construct the drone swarm escort state machine, and implement the drone swarm escort to the protected target according to the state machine:

[0059] The state machine states include initialization, reconnaissance, attack, and movement; the control methods are:

[0060] Set the state to 'initialization', call the initialization function, set the initial position of the drone group based on the protection target, and then set the state to 'reconnaissance', such as Figure 3 shown.

[0061] When the state is 'scout', call the countermeasure_function function. If the list is empty, change the state to 'move', call moving_function to update the position, and then set the state to 'scout', such as Figure 4 As shown; if the list is not empty, set the state to 'attack', call attack_function to destroy the opponent's interception device, then set the state to 'move', call moving_function to update the position, and then set the state to 'reconnaissance', as shown Figure 5 Until the current protection target reaches the last path point, as shown in Figure 6 As shown, the state machine operation ends.

[0062] By constantly switching between initialization, movement, reconnaissance and strike, the drone swarm covers the target to reach the designated location.

[0063] In summary, the present invention is aimed at the objective demand of drone swarm escort. The drone swarm and the protected target keep their relative positions still and form a formation, and move along the path navigation points with the protected target. By constructing a drone swarm escort state machine, the drone swarm and the protected target can dynamically switch between mobile, reconnaissance and strike states, destroy the opponent's intercepting equipment, and cover the protected target to reach the designated position.

Claims

1. A drone swarm escort method based on a state machine, characterized in that: The method is used to control the movement of the drone swarm and the protected target so that the drone swarm can provide protection to the protected target, including the following steps: Step 1: Set the path point list of the protected target. The protected target moves according to the path point coordinates in the list. The coordinates of each path point in the two-dimensional scene are arranged in path order, that is: = ； N is the number of waypoints; Step 2: Calculate the relative position of each drone in the drone group and the protected target; m Two-dimensional relative position relative to the protected target ( , )for: ； in, m =1,2,..., M , M is the number of drones in the drone swarm, R is the distance between the drone and the protected target; Step 3: Construct an initialization function to control the protected target and each drone in the drone swarm to move to their respective initial positions according to the starting point of the route path point list and the relative position of each drone in the drone swarm and the protected target; the working method of the initialization function is: Control the protected target to move to the first position in the path point list of the route ( x 1, y 1), complete the protection of the target location ( , ) initialization; Controlling the Drone m Move to position ( , ), complete the UAV m Location( , ), m =1,2,..., M ; Step 4: Construct a marching function; the marching function is used to obtain the current two-dimensional position of the protection target ( , ), and matches and updates each path point in the path point list of the route. and When the protection target moves to the first n +1 waypoint coordinates ( , ),renew( , ) = ( , ) and control the drone m Move to a new position ( , ), m =1,2,..., M ; Step 5: Construct a reconnaissance perception function; the reconnaissance perception function is used to make each drone within its own reconnaissance perception range. The other party's interception equipment is sensed and summarized into a list of the other party's interception equipment positions. When there is no other party's interception equipment within the reconnaissance perception range of each drone, the list of the other party's interception equipment positions is empty; Step 6: Construct an attack function; the attack function is used to destroy the opponent's interception device at the corresponding position in the opponent's interception device position list, and delete its position coordinates from the opponent's interception device position list until the opponent's interception device position list is empty; Step 7: Construct the drone swarm escort state machine. According to the instructions of the state machine, call the initialization function, travel function, reconnaissance perception function or strike function respectively to control the movement and action of the protected target and each drone in the drone swarm until the protected target moves to the end of the path point list of the travel route, so as to realize the escort of the drone swarm to the protected target; the state of the drone swarm escort state machine includes initialization, reconnaissance, strike, and movement; the control method is: Set the state to 'initialize', call the initialization function, and then set the state to 'scout'; When the state is 'reconnaissance', call the reconnaissance perception function. If the opponent's interception device location list is empty, change the state to 'movement', otherwise change the state to 'attack'; When the state is 'moving', call the march function and then set the state to 'scout'; When the state is 'hit', call the hit function and then set the state to 'move'.

Images