Unmanned aerial vehicle collaborative navigation method, system and equipment based on joint perception

Abstract

The invention discloses an unmanned aerial vehicle cooperative navigation method, system and device based on joint perception, and the method comprises the steps: constructing a multi-unmanned aerial vehicle joint state description through combining the original vision of an unmanned aerial vehicle, the semantic information of key elements, the geometric data of the distance between the unmanned aerial vehicles and the execution action at the current moment; an end-to-end multi-unmanned aerial vehicle collaborative decision-making network is constructed, mapping from joint state description to a multi-unmanned aerial vehicle joint action value is formed, and an executable unmanned aerial vehicle motion control instruction is generated by using an action selection strategy and a value function. The functions of exploration, target acquisition, dynamic target allocation, obstacle avoidance and the like of a multi-unmanned aerial vehicle system in an unknown environment are realized, and the task that each unmanned aerial vehicle reaches the respective target without collision is completed; according to the method, cooperative control based on multi-agent deep reinforcement learning is combined with a time sequence prediction method, so that the unmanned aerial vehicle autonomously completes a cooperative navigation task of multiple unmanned aerial vehicles in an unknown environment without manual intervention, and the autonomy and intelligence of the unmanned aerial vehicle are improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of unmanned aerial vehicle navigation, and in particular relates to a method, system and equipment for cooperative navigation of unmanned aerial vehicles based on joint perception. Background Art

[0002] With the development of technologies such as robot control and artificial intelligence in the 21st century, multi-UAV systems have gradually become a research focus in the fields of robotics and artificial intelligence, and have been widely used in military reconnaissance, collaborative rescue, logistics distribution, forest or power inspection, etc. The development of UAV collaborative navigation technology has greatly improved the efficiency of multiple UAV mission execution. At the same time, the application of autonomous and intelligent technologies has reduced the dependence on human operation and provided important protection for the personal safety of practitioners.

[0003] UAV cooperative navigation refers to the process of UAVs coordinating their own behaviors by perceiving environmental information to achieve collision-free movement to target areas or target objects. Traditional multi-UAV systems usually include two main steps: task allocation and task execution. When multiple homogeneous UAVs perform tasks to reach multiple target locations, the main steps of UAV cooperation are target allocation and navigation. Usually, the UAV target allocation algorithm allocates targets to UAVs to ensure the overall optimal target allocation and reduce the total cost. On the other hand, the navigation part usually uses the UAV path planning algorithm to plan paths for UAVs to avoid collisions between UAVs or with other obstacles. However, since these two algorithms are decoupled, most methods are difficult to dynamically adjust in dynamic obstacles, dynamic targets or partially observable environments. The UAV cooperative navigation method based on reinforcement learning is an end-to-end multi-agent cooperative control method. It uses the interaction between UAVs and the environment to enable UAVs to complete navigation tasks autonomously through continuous learning and optimization. According to the different perception ranges of the agent to the environmental state, the multi-machine cooperative navigation method based on reinforcement learning can be divided into two categories: cooperative navigation method based on global perception and reactive navigation method based on local perception. The method based on global perception relies on the global information of the entire environment to make decisions. It can provide the globally optimal path planning, but it requires a lot of computing resources in large-scale complex environments and is difficult to adapt to real complex environments. The method based on local perception is more suitable for partially observable environments and has lower requirements for computing resources. It is more suitable for real complex environments, but it is difficult to take into account autonomous target allocation. It often adopts the method of allocation first and then navigation. There are challenges in navigation efficiency and performance, coordination and cooperation between drones, and allocation of multiple dynamic targets. Summary of the invention

[0004] In order to solve the problems existing in the prior art, the present invention provides a UAV collaborative navigation method based on joint perception, which reconstructs the semantic information of key elements extracted from the original visual perception, integrates the geometric data of the distance between UAVs and the execution actions of the UAVs at the current moment to obtain a joint state description, and designs an efficient collaborative decision-making model in combination with the TransFormer network structure to realize adaptive target allocation and collaborative navigation of UAVs in dynamic environments.

[0005] In order to achieve the above object, the technical solution adopted by the present invention is: a UAV collaborative navigation method based on joint perception, comprising the following steps:

[0006] Build and train a visual reconstruction network to extract the location features and semantic features of targets and obstacles from the original vision of the drone and reconstruct them, and process the reconstructed images to obtain a joint image;

[0007] The geometric data of the distance between drones and the execution action data of the drones at the current moment are mapped together into the reconstructed joint image to construct the joint perception result, and the joint perception results at multiple moments are stacked to construct the joint state description;

[0008] Construct an end-to-end multi-UAV collaborative decision-making network to form a mapping from the joint state description to the value of the joint action of the UAVs;

[0009] According to the value of the UAV joint action, the direction control command and speed control command of the UAV are determined by using the action selection strategy;

[0010] According to the direction control instructions and speed control instructions, each UAV is controlled to fly to the target position to achieve multi-UAV collaborative navigation.

[0011] Furthermore, a visual reconstruction network is used to extract positional features and semantic features of targets and obstacles in the original vision, and the positional features and semantic features are reconstructed into a general visual description, and the visual description is subjected to dimensionality reduction, cropping, and splicing.

[0012] Furthermore, in step 2, the geometric data of the distance between drones and the execution action data of the drones at the current moment are mapped to grayscale values ​​between 0 and 255 and integerized. The integerized data are spliced ​​to the bottom of the joint image to construct a joint perception result, and multiple joint perception results in continuous time are stacked to construct a joint state description.

[0013] Furthermore, the multi-UAV collaborative decision-making model combines the temporal feature extraction based on TransFormer and the multi-agent reinforcement learning algorithm based on the value decomposition idea, including input data processing, temporal feature extraction part and action value calculation. The input data processing part divides the joint state description into several patches, uses convolution operation to map each patch to a feature vector of dim dimension, adds position encoding and adds category labeling parameters; the temporal feature extraction part adopts 3 TransFormer layers, each layer contains 6 attention heads, and the input is residually connected with the attention output, and after layer regularization and feedforward neural network processing, it is fused with the features before regularization to output the temporal feature vector; the action value calculation part has two fully connected layers, the first fully connected layer further processes the joint features, and the second fully connected layer is a multi-branch structure, each UAV corresponds to a branch, each branch contains m outputs, and a greedy strategy is used on the branch for action selection, and the results of each branch are summed up to finally obtain m n dimensional joint action value, where m is the dimension of the action space and n is the number of drones.

[0014] Furthermore, training the multi-UAV collaborative decision network includes: initializing the collaborative decision network, the collaborative decision network obtains the joint state description, and uses the joint state description as the state information s union , the collaborative decision network outputs a value function, selects the best action and executes it;

[0015] At each step, the drone will detect whether the termination condition is reached, and if so, stop the loop; otherwise, it will update the reward information and convert the state to a new state, use the new state as the input of the collaborative decision network, calculate and select a new action, and store <state, action, reward, new state, termination flag> in the experience replay pool;

[0016] The experience replay pool samples <state, action, reward, new state, abort flag>. During each iteration, the collaborative decision network obtains and Calculating the state value function θ is the collaborative decision network parameter, using the executed actions Choose a state-action-value function Will As the next moment state calculation to obtain the state value function, right Select the action with the maximum Q value respectively Finally, using the discount function γ and reward r t Calculate the state-action value function y at time t.

[0017] Furthermore, when confirming the direction control command and the speed control command: according to the obtained joint action value, the UAV joint action with the highest value is selected.

[0018] Based on the concept of the above method, the present invention provides a UAV collaborative navigation system based on joint perception, including a collaborative perception module, a collaborative decision module, a control instruction output module and a control module;

[0019] The collaborative perception module is used to reconstruct, splice, grayscale and reduce the dimension of the original visual data; the geometric data of the distance between drones and the execution action data of the drones at the current moment are mapped to the joint image to construct the joint perception result, and the joint perception results at multiple moments are stacked to form a universal joint state description;

[0020] The collaborative decision-making module is used to map the joint state description to the value of the joint action of the UAVs and obtain the optimal joint action using the action selection strategy;

[0021] A control command output module is used to determine the direction control command and speed control command of the UAV according to the optimal joint action of the UAV;

[0022] The control module is used to control the UAV to fly to the target position according to the direction control instructions and the speed control instructions, so as to realize the collaborative navigation of multiple UAVs.

[0023] On the other hand, the present invention can provide a UAV formation, which navigates based on the above-mentioned UAV collaborative navigation method based on joint perception.

[0024] The present invention can also provide a computer device, including a processor and a memory, the memory is used to store a computer executable program, the processor reads the computer executable program from the memory and executes it, and when the processor executes the computer executable program, the drone collaborative navigation method based on joint perception described in the present invention can be implemented.

[0025] At the same time, a computer-readable storage medium is provided, in which a computer program is stored. When the computer program is executed by a processor, the method for cooperative navigation of unmanned aerial vehicles based on joint perception described in the present invention can be implemented.

[0026] Compared with the prior art, the present invention has at least the following beneficial effects: the present invention discloses a cooperative navigation method and system of unmanned aerial vehicles based on joint perception, which realizes that in the absence of a priori maps and target positions, multiple unmanned aerial vehicles can cooperatively explore unknown environments, actively obtain multiple target positions and adaptively allocate navigation targets, and complete cooperative autonomous navigation movement from the current position to the target object; the method first combines the original vision of the unmanned aerial vehicle, the semantic information of key elements, the geometric data of the distance between the aircraft, and the execution action at the current moment to construct a joint state description of multiple unmanned aerial vehicles. Subsequently, an end-to-end multi-unmanned aerial vehicle cooperative decision-making network is constructed to form a mapping from the joint state description to the value of the joint action of multiple unmanned aerial vehicles. Finally, the action selection strategy and the value function are used to generate executable unmanned aerial vehicle motion control instructions, so as to realize the functions of exploration, target acquisition, dynamic target allocation, obstacle avoidance, etc. of the multi-unmanned aerial vehicle system in an unknown environment, and complete the task of each unmanned aerial vehicle reaching its own target without collision; the method combines the cooperative control based on multi-agent deep reinforcement learning with the time series prediction method, so that the unmanned aerial vehicle can autonomously complete the cooperative navigation task of multiple unmanned aerial vehicles in an unknown environment without human intervention, which greatly improves the autonomy and intelligence of the unmanned aerial vehicle. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Figure 1 It is a system overall block diagram of the multi-UAV cooperative navigation method of the present invention.

[0028] Figure 2 It is a schematic diagram of the visual perception reconstruction framework of the present invention.

[0029] Figure 3 It is a schematic diagram of distance information mapping between drones of the present invention.

[0030] Figure 4 It is a schematic diagram of the drone action information mapping of the present invention.

[0031] Figure 5 It is a schematic diagram of generating the joint state description of the present invention.

[0032] Figure 6 It is a schematic diagram of the collaborative decision-making network structure of the present invention.

[0033] Figure 7 It is a schematic diagram of the collaborative decision-making network training framework of the present invention.

[0034] Figure 8 It is a schematic flow chart of an implementable method of the present invention. DETAILED DESCRIPTION

[0035] The present invention will be further described in detail below in conjunction with the accompanying drawings, which are intended to explain the present invention rather than to limit it.

[0036] See also Figure 1 and Figure 8 , a UAV cooperative navigation method based on joint perception, comprising the following steps:

[0037] Step 1: Build and train a visual reconstruction network to extract the location features and semantic features of the target and obstacles from the original vision of the drone and reconstruct them, and process the reconstructed image to obtain a joint image;

[0038] refer to Figure 2 Specifically, two sets of visual reconstruction networks are built, which are applied to the simulation environment and the real environment respectively. During the entire algorithm training period, the visual reconstruction network 1 works, which converts the first-person image information img obtained by the drone in the simulation environment sim After target recognition and classification, the image is reconstructed into a highly general semantic description that is easy to distinguish between targets, obstacles, and drones based on the detection results of the image. in The generation process can be expressed as in The network model for visual reconstruction network 1, is the network parameter of the visual reconstruction network 1, and its superscript g indicates a generalized state description. During the migration deployment, the visual reconstruction network 2 works to convert the image information img of the first-person perspective of the drone in the real environment real After the visual reconstruction network 2, Reconstruction results with similar features in The network model for visual reconstruction network 2, is the network parameter of visual reconstruction network 2, and the final visual reconstruction result This will reduce the difference between the simulation training environment and the actual application environment and ensure the algorithm's migration capability.

[0039] The visual reconstruction network is built on the existing single-stage target detection model YoloV5 based on deep learning. The models of people, fire hydrants, drones, etc. in different simulation scenes are collected through the front view of the drone in the Gazebo simulation environment to establish a data set for the simulation navigation environment and annotate it. The YOLOv5 model is trained with the prepared data set for visual reconstruction in the simulation environment. For the real environment, the drone is used to collect and annotate samples in the real environment, and the visual reconstruction network 2 is trained for visual information reconstruction in the real environment.

[0040] The reconstructed image is subjected to dimensionality reduction and cropping processing. Specifically, grayscale operation is first performed to obtain a grayscale image of size 640×480, and then further compressed to an image of size 32×24; then stitching processing is performed to horizontally stitch the compressed images collected by multiple drones to obtain a joint image. For two drones, the size of the stitching result is 64×24;

[0041] Step 2: Map the geometric data of the distance between drones and the execution action data of the drones at the current moment into the reconstructed joint image to construct a joint perception result, and stack the joint perception results at multiple moments to construct a joint state description;

[0042] refer to Figure 3 and Figure 4 Specifically, the geometric data of the distance between continuous drones are normalized, and then mapped to grayscale values ​​between 0 and 255, integerized, and spliced ​​to the bottom of the joint image. The grayscale value of 0 is filled between the joint image, the geometric data of the distance between drones, and the execution action data of the drone at the current moment to separate them, so as to achieve the splicing fusion of distance data and visual perception; for the execution action data of the drone at the current moment, it is also mapped to between 0 and 255, and the motion information of different drones is distinguished by different pixel positions.

[0043] According to the above method, the distance and motion data between drones are spliced ​​into a 64×8 all-zero matrix, and then the result is vertically spliced ​​with the joint image to obtain a 64×32 joint perception result. For two drones, the images of 8 consecutive time steps are stacked to obtain a 512×32 joint state description s union , which is used as the input for the following collaborative decision-making part, see Figure 5 .

[0044] Step 3: Build an end-to-end multi-UAV collaborative decision-making network to form a mapping from joint state description to UAV joint action value;

[0045] refer to Figure 6 The multi-UAV collaborative decision-making model is designed by combining the temporal feature extraction based on TransFormer and the multi-agent reinforcement learning algorithm based on the value decomposition idea. The network can be divided into three parts, namely input data processing, temporal feature extraction, and action value calculation.

[0046] The input data processing part divides the joint state description into several patches, uses convolution operation to map each patch into a feature vector of dim dimension, and then adds position encoding and category tag parameters. The dimension of position encoding is (1, num patch+1, dim), and the category label is (1, 1, dim), where both the position encoding and the category label are learnable parameters. For two drones, when the perception sequence is 8 consecutive images, the input size is 521×32, which needs to be divided into 8 patches, each of which is 64×32 in size, and the mapping dimension dim=256.

[0047] The temporal feature extraction part uses 3 TransFormer layers, each layer contains 6 attention heads, and then the input is residually connected with the attention output, and then it is regularized by the layer and processed by the feedforward neural network. At the same time, it is fused with the features before regularization and outputs the temporal feature vector to provide input for the subsequent action value calculation part. In the actual design, the feedforward neural network has 512 neurons, the output feature vector dimension is 256, and the dropout value is 0.1.

[0048] The action value calculation part is to calculate the value of each possible action of each drone in the joint state according to the features in the joint state description extracted by the backbone network, and provide a basis for action selection. This part consists of two fully connected layers. The first fully connected layer is a further processing of the joint features, and the second fully connected layer is a multi-branch structure. Each drone corresponds to a branch, and each branch contains m outputs. When the number of drones changes, only the number of branches needs to be adjusted. Therefore, its action selection branch contains m×n outputs. The greedy strategy is used on this branch for action selection. Finally, the results of each branch are summed up to get m n dimensional joint action value, which is used for subsequent training using the joint reward of the environment, where m is the dimension of the action space and n is the number of drones.

[0049] During training, at each step, the training protocol is as follows Figure 7 First, the collaborative decision network is initialized, and then the collaborative decision network obtains the joint state description and uses it as the state information s union , the collaborative decision network obtains the state information and outputs the value function, then selects the best action and executes it. In each step, the drone will detect whether the termination condition is reached. If so, the loop will be stopped; otherwise, the reward information will be updated and the state will be converted to a new state. Finally, the new state will be used as the input of the collaborative decision network to calculate and select a new action. Store <state, action, reward, new state, termination flag> in the experience replay pool.

[0050] refer to Figure 7 , obtained by sampling from the experience replay pool in is the state code obtained by the collaborative decision-making network at time t, The joint action selected for the drone, rt To evaluate the selected action, is the next moment state, and done is the termination flag. In each iteration, the collaborative decision network first obtains and Calculating the state value function θ is the collaborative decision network parameter. Then, using the executed actions Choose a state-action-value function Will As the next moment state calculation to obtain the state value function, right Select the action with the maximum Q value respectively Finally, using the discount function γ and reward r t Calculate the state action value function y at time t, y is as shown in the formula:

[0051]

[0052] Based on the above formula, the loss function formula of the collaborative decision-making network is:

[0053]

[0054] The Adam optimizer is used to adjust the network parameters.

[0055] Step 4: According to the UAV joint action value, the direction control command and speed control command of the UAV are determined by using the action selection strategy;

[0056] Specifically, three high-level instructions are defined for drones, namely, for drone i, its action space A i ={a1, a2, a3}, where a1 means going straight, a2 means turning left on the spot, and a3 means turning right on the spot. Set the linear velocity of the drone to 1m / s, the angular velocity to ±1.3rad / s, and the instruction duration to 200ms. The specific values ​​of the high-order instructions are finally obtained as a1=[0.2 0], a2=[0 0.26], a3=[0 -0.26], where the first component is the displacement of the drone along the nose direction, in meters (m), and the second component is the angle change in the rotation direction, in rad.

[0057] According to the action selection strategy, the joint action with the largest joint action value is selected, and the direction control action and speed control action of the UAV are generated.

[0058] Step 5: Control each UAV to fly to the target position according to the direction control command and the speed control command.

[0059] Based on the above results, the direction and speed control instructions of the UAV are finally output to complete the collaborative navigation task of multiple UAVs.

[0060] The present invention also provides a system for a UAV collaborative navigation method based on joint perception, comprising a collaborative perception module, a collaborative decision-making module, a control instruction output module and a control module.

[0061] The collaborative perception module is used to reconstruct, splice, grayscale and reduce the dimension of the original visual data; the geometric data of the distance between drones and the execution action data of the drones at the current moment are mapped to the joint image to construct the joint perception result, and the joint perception results at multiple moments are stacked to form a universal joint state description;

[0062] The collaborative decision-making module is used to map the joint state description into the value of the joint action of UAVs and obtain the optimal joint action using the action selection strategy.

[0063] A control command output module is used to determine the direction control command and speed control command of the UAV according to the optimal joint action of the UAV;

[0064] The control module is used to control the UAV to fly to the target position according to the direction control command and the speed control command.

[0065] On the other hand, the present invention provides a computer-readable storage medium, in which a computer program is stored. When the computer program is executed by a processor, the method for cooperative navigation of unmanned aerial vehicles based on joint perception described in the present invention can be implemented.

[0066] The computer device may be a laptop computer, a desktop computer or a workstation.

[0067] The present invention can also provide a computer device, including a processor and a memory, the memory is used to store a computer executable program, the processor reads the computer executable program from the memory and executes it, and when the processor executes the computer executable program, it can implement the drone collaborative navigation method based on joint perception described in the present invention.

[0068] The processor may be a central processing unit (CPU), a digital signal processor (DSP), an application specific integrated circuit (ASIC), or an off-the-shelf field programmable gate array (FPGA).

[0069] The memory of the present invention may be an internal storage unit of a laptop computer, a desktop computer or a workstation, such as a memory or a hard disk; or an external storage unit, such as a mobile hard disk or a flash memory card.

[0070] Computer-readable storage media may include computer storage media and communication media. Computer storage media include volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information such as computer-readable instructions, data structures, program modules or other data. Computer-readable storage media may include: read-only memory (ROM), random access memory (RAM), solid-state drives (SSD) or optical disks, etc. Among them, random access memory may include resistance random access memory (ReRAM) and dynamic random access memory (DRAM)

[0071] In summary, the present invention discloses a joint perception-based UAV collaborative navigation method and system, which realizes that in the absence of a priori maps and target positions, multiple UAVs can collaboratively explore unknown environments, actively obtain multiple target positions and adaptively allocate navigation targets, and complete collaborative autonomous navigation movements from the current position to the target object; the method first combines the original vision of the UAV, the semantic information of key elements, the geometric data of the distance between the machines, and the execution action at the current moment to construct a multi-UAV joint state description. Subsequently, an end-to-end multi-UAV collaborative decision network is constructed to form a mapping from the joint state description to the value of the multi-UAV joint action. Finally, the action selection strategy and the value function are used to generate executable UAV motion control instructions, realize the functions of exploration, target acquisition, dynamic target allocation, obstacle avoidance, etc. of the multi-UAV system in an unknown environment, and complete the task of each UAV reaching its own target without collision; the method combines the collaborative control based on multi-agent deep reinforcement learning with the time series prediction method, so that the UAV can autonomously complete the collaborative navigation task of multiple UAVs in an unknown environment without human intervention, which greatly improves the autonomy and intelligence of the UAV.

[0072] The above contents are only for explaining the technical idea of ​​the present invention and cannot be used to limit the protection scope of the present invention. Any changes made on the basis of the technical solution in accordance with the technical idea proposed by the present invention shall fall within the protection scope of the claims of the present invention.

Claims

1. A UAV collaborative navigation method based on joint perception, characterized in that: The following steps are involved: Build and train a visual reconstruction network to extract the location features and semantic features of targets and obstacles from the original vision of the drone and reconstruct them, and process the reconstructed images to obtain a joint image; The geometric data of the distance between drones and the execution action data of the drones at the current moment are mapped together into the reconstructed joint image to construct the joint perception result, and the joint perception results at multiple moments are stacked to construct the joint state description; Construct an end-to-end multi-UAV collaborative decision-making network to form a mapping from the joint state description to the value of the joint action of the UAVs; According to the value of the UAV joint action, the direction control command and speed control command of the UAV are determined by using the action selection strategy; According to the direction control instructions and speed control instructions, each UAV is controlled to fly to the target position to achieve multi-UAV collaborative navigation.

2. The method for cooperative navigation of unmanned aerial vehicles based on joint perception according to claim 1, characterized in that: In step 1, a visual reconstruction network is used to extract the position features and semantic features of the target and obstacles in the original vision, and the position features and semantic features are reconstructed into a general visual description, and the visual description is subjected to dimensionality reduction, cropping, and splicing.

3. The method for cooperative navigation of unmanned aerial vehicles based on joint perception according to claim 1, characterized in that: In step 2, the geometric data of the distance between drones and the execution action data of the drones at the current moment are mapped to grayscale values ​​between 0 and 255 and integerized. The integerized data are spliced ​​to the bottom of the joint image to construct the joint perception result. Multiple joint perception results in continuous time are stacked to construct a joint state description.

4. The method for cooperative navigation of unmanned aerial vehicles based on joint perception according to claim 1, characterized in that: The multi-UAV collaborative decision-making model combines the temporal feature extraction based on TransFormer and the multi-agent reinforcement learning algorithm based on the value decomposition idea, including input data processing, temporal feature extraction part and action value calculation. The input data processing part divides the joint state description into several patches, uses convolution operation to map each patch to a feature vector of dim dimension, adds position encoding and adds category labeling parameters; the temporal feature extraction part adopts 3 TransFormer layers, each layer contains 6 attention heads, and the input is residually connected with the attention output, and after layer regularization and feedforward neural network processing, it is fused with the features before regularization to output the temporal feature vector; the action value calculation part has two fully connected layers, the first fully connected layer further processes the joint features, and the second fully connected layer is a multi-branch structure, each UAV corresponds to a branch, each branch contains m outputs, and a greedy strategy is used on the branch for action selection, and the results of each branch are summed up to finally obtain m n dimensional joint action value, where m is the dimension of the action space and n is the number of drones.

5. The method for cooperative navigation of unmanned aerial vehicles based on joint perception according to claim 1, characterized in that: Training a multi-UAV collaborative decision-making network includes: initializing the collaborative decision-making network, the collaborative decision-making network obtains a joint state description, and uses the joint state description as state information s union , the collaborative decision network outputs a value function, selects the best action and executes it; At each step, the drone will detect whether the termination condition is reached, and if so, stop the loop; otherwise, it will update the reward information and convert the state to a new state, use the new state as the input of the collaborative decision network, calculate and select a new action, and store <state, action, reward, new state, termination flag> in the experience replay pool; The experience replay pool samples <state, action, reward, new state, abort flag>. During each iteration, the collaborative decision network obtains and Calculating the state value function θ is the collaborative decision network parameter, using the executed actions Choose a state-action-value function Will As the next moment state calculation to obtain the state value function, right Select the action with the maximum Q value respectively Finally, using the discount function γ and reward r t Calculate the state-action value function y at time t.

6. The method for cooperative navigation of unmanned aerial vehicles based on joint perception according to claim 1, characterized in that: When confirming the direction control command and speed control command: according to the obtained joint action value, select the UAV joint action with the highest value.

7. A UAV collaborative navigation system based on joint perception, characterized in that: It includes a collaborative perception module, a collaborative decision-making module, a control instruction output module and a control module; The collaborative perception module is used to reconstruct, splice, grayscale and reduce the dimension of the original visual data; the geometric data of the distance between drones and the execution action data of the drones at the current moment are mapped to the joint image to construct the joint perception result, and the joint perception results at multiple moments are stacked to form a universal joint state description; The collaborative decision-making module is used to map the joint state description to the value of the joint action of the UAVs and obtain the optimal joint action using the action selection strategy; A control command output module is used to determine the direction control command and speed control command of the UAV according to the optimal joint action of the UAV; The control module is used to control the UAV to fly to the target position according to the direction control instructions and the speed control instructions, so as to realize the collaborative navigation of multiple UAVs.

8. A drone formation, characterized in that: Navigation based on the UAV collaborative navigation method based on joint perception as described in any one of claims 1-6.

9. A computer device, characterized in that: It includes a processor and a memory, the memory is used to store a computer executable program, the processor reads part or all of the computer executable program from the memory and executes it, and when the processor executes part or all of the computer executable program, it can implement the joint perception-based UAV collaborative navigation method described in any one of claims 1-6.

10. A computer-readable storage medium, characterized in that: A computer program is stored in a computer-readable storage medium, and when the computer program is executed by a processor, it can implement the UAV collaborative navigation method based on joint perception as described in any one of claims 1 to 6.

Images