Multi-unmanned aerial vehicle cooperative tracking method for non-cooperative target, product, medium and equipment

Abstract

The invention discloses a multi-unmanned aerial vehicle cooperative tracking method for a non-cooperative target, a product, a medium and equipment, and relates to the technical field of unmanned aerial vehicles, and the method comprises the steps: obtaining the state of each unmanned aerial vehicle at a future moment, carrying out the modeling of the control of each unmanned aerial vehicle, and obtaining the control quantity of each unmanned aerial vehicle; inputting the non-cooperative target image obtained by each unmanned aerial vehicle into the target position detection model to obtain the coordinate of the non-cooperative target; performing dynamic modeling based on the coordinates of the non-cooperative target, and determining a final state estimation result of the non-cooperative target; modeling pod control of each unmanned aerial vehicle according to the pod state of each unmanned aerial vehicle at the current moment to obtain the control quantity of each unmanned aerial vehicle pod; and performing multi-unmanned aerial vehicle cooperative tracking of the non-cooperative target based on the final state estimation result of the non-cooperative target, the control quantity of each unmanned aerial vehicle and the control quantity of each unmanned aerial vehicle pod. According to the invention, the stability and robustness of multi-unmanned aerial vehicle cooperative tracking for the non-cooperative target can be effectively enhanced.

Description

Technical Field

[0001] The present application relates to the field of drone technology, and in particular to a method, product, medium and device for collaborative tracking of multiple drones of non-cooperative targets. Background Art

[0002] With the rapid development of drone technology, single drone-based systems have been widely used in agriculture, monitoring, and map construction, among which the tracking system as the core also plays an important role. However, as the scene expands, a single drone often encounters interference, occlusion, false detection, and missed detection, which leads to the problem of target tracking loss, and it is difficult to complete the continuous tracking of the target. In order to solve this problem, drones are usually required to have more powerful perception performance or to use multiple drones to observe the scene collaboratively. The enhancement of perception performance will inevitably lead to an increase in cost, sometimes even exponentially. And if a drone with more powerful perception performance is used but fails to successfully perceive the target, it will lead to a huge waste of resources. Therefore, it has become difficult to consider using multiple drones to implement target tracking and build a collaborative tracking framework.

[0003] Many existing UAV tracking works focus on the state estimation of targets or the control of multiple UAVs based on the known positions of targets. The state estimation of non-cooperative targets or the research results of cooperative tracking of known states are often difficult to apply to cooperative tracking of non-cooperative targets. Forcible application is likely to lead to tracking loss and unstable tracking effect. Given the current research status of cooperative tracking of multiple UAVs, designing a cooperative tracking strategy for non-cooperative targets is a very challenging problem. Summary of the invention

[0004] The purpose of this application is to provide a method, product, medium and equipment for multi-UAV collaborative tracking of non-cooperative targets, which can effectively enhance the stability and robustness of multi-UAV collaborative tracking of non-cooperative targets.

[0005] To achieve the above objectives, this application provides the following solutions:

[0006] In a first aspect, the present application provides a method for cooperative tracking of multiple UAVs of a non-cooperative target, comprising:

[0007] Get the status of each drone at the future moment;

[0008] Model the control of each UAV based on its state at the future moment, and obtain the control amount of each UAV;

[0009] Inputting the non-cooperative target images acquired by each UAV into a target position detection model to obtain the coordinates of the non-cooperative target; the target position detection model includes YOLOv5 and Siamese-RPN connected in sequence;

[0010] Perform dynamic modeling based on the coordinates of the non-cooperative target to determine the final state estimation result of the non-cooperative target;

[0011] Model the pod control of each UAV according to the pod status of each UAV at the current moment, and obtain the control quantity of the pod of each UAV;

[0012] Multi-UAV cooperative tracking of non-cooperative targets is performed based on the final state estimation results of the non-cooperative targets, the control amount of each UAV and the control amount of each UAV pod.

[0013] Optionally, obtaining the state of each drone at a future time specifically includes:

[0014] Building a dynamic model of the drone Get the state x(k+1) of the drone at time k+1; where x(k) is the state of the drone at time k; t is the time step; u p (k) is the input control variable of the UAV at time k.

[0015] Optionally, the control of each drone is modeled based on the state of each drone at a future moment to obtain the control amount of each drone, specifically including:

[0016] Model the control of the drone based on its state at future moments: ,

[0017] Get the control amount J of the drone c ; where x(k+l|k) is the predicted state of the drone at time k; N is the number of prediction steps, 0≤l≤N-1; u p (k+l|k) is the input control quantity of the UAV at the future time; u p (k+l|k) T for u p (k+l|k) is the transpose; x(k+N) is the state of the drone at the predicted final moment; Q, R and V are the state adjustment matrix, control adjustment matrix and final state adjustment matrix respectively.

[0018] Optionally, the step of inputting the non-cooperative target images acquired by each drone into a target position detection model to obtain the coordinates of the non-cooperative target specifically includes:

[0019] The non-cooperative target image is acquired through the camera installed on the drone pod and input into YOLOv5 to obtain the visual detection result;

[0020] The visual detection results are input into Siamese-RPN to obtain the pixel coordinates of the non-cooperative target in the image plane;

[0021] Through the current state of the UAV and the internal parameters of the camera, the pixel coordinates are converted to the UAV coordinate system and the world coordinate system in turn to obtain the coordinates of the non-cooperative target in the world coordinate system.

[0022] Optionally, the performing of dynamic modeling based on the coordinates of the non-cooperative target to determine the final state estimation result of the non-cooperative target specifically includes:

[0023] Coordinates based on non-cooperative targets Get the actual state of the non-cooperative target in, and are the coordinate values ​​of the non-cooperative target in the X-axis, Y-axis and Z-axis directions respectively; and are the speed values ​​of the non-cooperative target in the X-axis, Y-axis and Z-axis directions respectively;

[0024] The non-cooperative target is dynamically modeled according to its actual state X(k) at time k. Get the state of the non-cooperative target predicted by the drone at time k+1 Among them, t is the time step; w(k) is the time difference between the actual moments;

[0025] Using formula Calculate the final state estimation result of the non-cooperative target Among them, Q, D and M are the communication topology matrix, detection threshold matrix and prediction threshold matrix respectively.

[0026] Optionally, the pod control of each drone is modeled according to the pod state of each drone at the current moment to obtain the control amount of the pod of each drone, specifically including:

[0027] Model the pod control of the drone based on the current pod state of the drone: Get the control amount u of the drone pod c (k); where e(k) is the state of the pod at time k+1 and the pod status X at time k c e(k-1) is the state of the pod at time k. c (k) and the pod status at time k-1 The difference between is the accumulation of all differences before time k; K p , K mand K d are the corresponding PID parameters respectively.

[0028] Optionally, collision avoidance control rules are set when a collision between drones is about to occur. Among them, J b is the collision cost function; d b is the preset collision threshold; δd is a very small value; p is the position of the drones other than the specified drone;

[0029] The collision avoidance control rules are optimized using Newton iteration. During the iteration process, the nearest UAV within the range is searched. If there are multiple UAVs, the two nearest ones are selected as collision avoidance targets. Then, the normal vector of the direction vector connecting the designated UAV with the two collision avoidance targets and the direction vector connecting with the non-cooperative target are fused to obtain the optimized direction of the gradient, and the optimal position of the designated UAV is found along this direction.

[0030] In a second aspect, the present application provides a computer program product, including a computer program, which, when executed by a processor, implements the multi-UAV collaborative tracking method for the non-cooperative target.

[0031] In a third aspect, the present application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the multi-UAV collaborative tracking method for the non-cooperative target.

[0032] In a fourth aspect, the present application provides a computer device, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the multi-UAV collaborative tracking method for the non-cooperative target.

[0033] According to the specific embodiments provided in this application, this application has the following technical effects:

[0034] The present application provides a method, product, medium and equipment for cooperative tracking of multiple UAVs of non-cooperative targets, which mainly relates to a vision-based cooperative tracking method of multiple UAVs, which inputs the non-cooperative target images acquired by each UAV into the target position detection model constructed by the present application to achieve stable visual tracking of the non-cooperative target, and then performs dynamic modeling on the coordinates of the non-cooperative target obtained by the target position detection model, so as to stably obtain the final state estimation result of the non-cooperative target. The present application also models the pod control of each UAV and the control of each UAV, respectively, obtains the control amount of each UAV pod and the control amount of each UAV, and finally performs cooperative tracking of multiple UAVs of non-cooperative targets based on the final state estimation result of the non-cooperative target, the control amount of each UAV and the control amount of each UAV pod, which can effectively enhance the stability and robustness of cooperative tracking of multiple UAVs for non-cooperative targets. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] In order to more clearly illustrate the technical solutions in the embodiments of the present application or related technologies, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0036] Figure 1 A schematic diagram of the process flow of the multi-UAV cooperative tracking method for non-cooperative targets provided in this application;

[0037] Figure 2 A schematic diagram of a scenario in which multiple UAVs track non-cooperative targets in a three-dimensional plane provided for this application;

[0038] Figure 3 Schematic diagram of the gradient optimization strategy provided for this application;

[0039] Figure 4 Schematic diagram of tracking of linear motion of non-cooperative targets provided for this application;

[0040] Figure 5 A schematic diagram of tracking the rotational motion of a non-cooperative target provided for this application;

[0041] Figure 6 Schematic diagram of tracking of random motion of non-cooperative targets provided for this application;

[0042] Figure 7 Schematic diagram of abnormal perception recovery of the drone provided for this application;

[0043] Figure 8 Schematic diagram of the mutual collision avoidance of UAVs provided for this application 1;

[0044] Fig. 9 Schematic diagram of situation 2 in which drones avoid collision with each other provided for this application. DETAILED DESCRIPTION

[0045] In order to solve the state estimation problem of non-cooperative targets, Shafiei et al. used Kalman filter or extended Kalman filter to realize motion modeling of non-cooperative targets, and realized real-time prediction of the state of non-cooperative targets by obtaining ranging and angle measurement information. In the case of known target state, Abdelmawgoud et al. proposed a semi-centralized or distributed method for the collaborative method, which often needs to transfer data to the central node for calculation. Once the central node is lost, the entire system will be paralyzed. The method transfers effective data through the topological structure, which effectively improves the robustness of the algorithm. Liu et al. proposed a trajectory generation method for the control problem in the collaborative process, which effectively realized the mutual collision in the collaborative tracking process. In order to make the collision avoidance process a real-time process, Rodríguez et al. designed a real-time collision avoidance method, calculated feasible control inputs in each control, and effectively improved the robustness of collaborative tracking in dynamic scenes. However, the research results of the above-mentioned studies on state estimation of non-cooperative targets or collaborative tracking of known states are difficult to apply to the collaborative tracking of non-cooperative targets. The forced combination of the above-mentioned state estimation methods and collaborative methods will lead to the phenomenon of tracking loss, making the effect of collaborative tracking poor. Therefore, the present application provides a method, product, medium and device for collaborative tracking of multiple UAVs of non-cooperative targets, which can effectively enhance the stability and robustness of collaborative tracking of multiple UAVs of non-cooperative targets.

[0046] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0047] In order to make the above-mentioned objects, features and advantages of the present application more obvious and easy to understand, the present application is further described in detail below in conjunction with the accompanying drawings and specific implementation methods.

[0048] like Figure 1 As shown, the present application discloses a method for cooperative tracking of multiple UAVs of a non-cooperative target, comprising:

[0049] Step 1: Get the status of each drone at the future moment.

[0050] For the case of multiple UAVs tracking non-cooperative targets (referred to as targets), to obtain the state of each UAV at the future moment, it is necessary to first construct a scene of multiple UAVs tracking targets in a three-dimensional plane, such as Figure 2 shown.

[0051] Assume that the distribution of drones and targets is as follows Figure 2 As shown in the figure, each drone has its own coordinate system, namely the drone coordinate system, which is represented by the subscript B. At the same time, each drone is equipped with a visual sensor (camera), and the camera coordinate system is represented by the subscript C. T represents a non-cooperative target, T C represents the position of the non-cooperative target in the image coordinate system taken by the camera, so (X B1 , Y B1 , Z B1 ) represents the position of the first drone on the X-axis, Y-axis, and Z-axis in its own coordinate system. C1 , Y C1 , Z C1 ) represents the position of the first drone on the X-axis, Y-axis, and Z-axis in the camera coordinate system, T C1 It represents the position of the non-cooperative target in the coordinate system of the image captured by the camera of the first UAV. The second and third UAVs are represented in the same way. The world coordinate system is represented by the subscript W.

[0052] The state of the drone is represented by x, and the dynamic model of the drone is constructed as follows:

[0053]

[0054] Among them, x(k) is the state of the UAV at time k; x(k+1) is the state of the UAV at time k+1; t is the time step; u p (k) is the input control variable of the UAV at time k.

[0055] Step 2: Model the control of each UAV based on its state at the future moment to obtain the control value of each UAV.

[0056] Based on step 1, the control of the UAV is modeled based on the state of the UAV at the future moment, so as to design a dynamic programming algorithm to make the UAV fly around the non-cooperative target.

[0057]

[0058] Among them, J c is the control quantity of the UAV; x(k+l|k) is the predicted state of the UAV at the kth moment; N is the number of prediction steps, 0≤l≤N-1; u p (k+l|k) is the input control quantity of the UAV at the future time; u p (k+l|k) T for u p(k+l|k) is the transpose; x(k+N) is the state of the drone at the predicted final moment; U, R and V are the state adjustment matrix, control adjustment matrix and final state adjustment matrix respectively.

[0059] Step 3: Input the non-cooperative target images acquired by each UAV into the target position detection model to obtain the coordinates of the non-cooperative target.

[0060] After building the scenario of multiple drones tracking non-cooperative targets in a three-dimensional plane in step 1, this application established a visual detection front end, as follows.

[0061] On the basis of the above, we first need to obtain the position coordinates of the non-cooperative target. For non-cooperative targets, this application hopes to build an autonomous detection model, so YOLOv5 and Siamese-RPN are used as the front-end visual tracking system of this application, that is, the target position detection model. Specifically, this application uses the visual detection results of YOLOv5 as the input of the single target tracker Siamese-RPN, thereby achieving stable visual tracking and outputting the pixel coordinates of the non-cooperative target in the image plane.

[0062]

[0063] Tack=f(Det) (4)

[0064] Among them, Det is the YOLOv5 detection network The output of the pod input image I is received; Tack is the tracking result of the single target tracking network f(·) of Siamese-RPN, that is, the pixel coordinates of the output non-cooperative target in the image plane.

[0065] For the obtained visual detection information, that is, the pixel coordinates T of the non-cooperative target outputted above on the image plane C =(u i ,v i ,1), where u i 、v i and 1 are the values ​​of the position of the non-cooperative target in the image coordinate system taken by the camera on the i-th UAV in the X-axis, Y-axis and Z-axis directions respectively. The state of the non-cooperative target is estimated by the position of the UAV itself. Specifically, it is necessary to first convert the position of the non-cooperative target to the UAV coordinate system (machine system) through pixel coordinates through the current state x of the UAV and the intrinsic parameters of the camera, and then further convert it to the world system:

[0066]

[0067] Among them, R WB is the rotation and translation matrix of the world system rotation system; RBC is the rotation and translation matrix of the camera system; K is the camera internal parameter, including the physical size of the horizontal and vertical axis focus and the center point of the pixel horizontal axis coordinate system; H is the height of the drone; is the translation from the drone system to the world system; and are the coordinate values ​​of the non-cooperative target in the X-axis, Y-axis and Z-axis directions respectively.

[0068] Specifically, the target position detection model described in this application includes YOLOv5 and Siamese-RPN connected in sequence. The non-cooperative target image is obtained through the camera installed in the drone pod and input into YOLOv5 to obtain the visual detection result; the visual detection result is input into Siamese-RPN to obtain the pixel coordinates of the non-cooperative target in the image plane; through the current state of the drone and the internal parameters of the camera, the pixel coordinates are converted to the drone coordinate system and the world coordinate system in turn to obtain the coordinates of the non-cooperative target in the world coordinate system

[0069] Step 4: Perform dynamic modeling based on the coordinates of the non-cooperative target to determine the final state estimation result of the non-cooperative target.

[0070] In order to further estimate the state of the target, the target is dynamically modeled. and are the speed values ​​of the non-cooperative target in the X-axis, Y-axis and Z-axis directions respectively. Considering the high real-time performance of the algorithm, the target is modeled as a uniform motion model to estimate the target state at the next moment. That is, the non-cooperative target is dynamically modeled according to its actual state X(k) at time k, and the state of the non-cooperative target predicted by the UAV at time k+1 is obtained.

[0071]

[0072] Among them, t is the time step; w(k) is the time difference between the actual moments.

[0073] Considering that the transmission of messages in the collaborative process is directed, a directed graph model is constructed so that drones can transmit the target state information through existing communication conditions, and Q is constructed as the communication topology, which represents the communication topology matrix. Considering the stability of detection, the detection value λ of the i-th drone is i Set the detection threshold λ t As a criterion for determining whether the test is effective.

[0074]

[0075] Among them, D(i) is the probability of effective detection of the i-th drone, abbreviated as D, which represents the detection threshold matrix.

[0076] In view of the fact that false detection and missed detection will affect the stability of the entire system, this application constructs an optimization estimation method.

[0077]

[0078] Among them, F(i,j) is the estimated error between the i-th UAV and the j-th UAV, is the rotation and translation matrix from the ith drone system to the world system, t b is the distance threshold, M(i,j) means that if the threshold between the i-th UAV and the j-th UAV exceeds a certain value, it will be considered an invalid value, abbreviated as M, which represents the prediction threshold matrix. The variance of invalid values ​​is calculated to filter out outliers.

[0079] Therefore, the final effective estimation expression is shown as follows, which satisfies the communication topology matrix Q, the detection threshold matrix D and the prediction threshold matrix M respectively:

[0080]

[0081] in, The final state estimate for the non-cooperative target.

[0082] Step 5: Model the pod control of each UAV according to the pod status of each UAV at the current moment to obtain the control quantity of the pod of each UAV.

[0083] In order to realize that the pod can aim at the target in real time when the camera detects the target, a pod control model is designed:

[0084]

[0085] Among them, e(k) is the state of the pod at time k+1 That is, the expected pod state and the pod state X at time k c e(k-1) is the state of the pod at time k. c (k) and the pod status at time k-1 The difference between is the accumulation of all differences before time k; u c (k) is the control quantity of the UAV pod; K p , K m and K dare the corresponding PID parameters respectively. By calculating the expected and actual errors, a control model is constructed so that the target is always aligned with the center of the pod to achieve stable tracking.

[0086] Although the present application can realize the tracking control of the drone through the above steps, there is still the possibility of collision between drones. In order to solve this problem, the present application designs the collision avoidance control rules:

[0087]

[0088] Among them, J b is the collision cost function; d b is the preset collision threshold; δd is a very small value; p is the position of the drones other than the specified drone. The purpose of this formula is to find each point with the smallest distance to other drones for the predicted state and control the point to fly to another direction.

[0089] Therefore, the overall cost function J can be expressed as:

[0090]

[0091] Where v represents the speed of the drone, v min ,v max is the velocity constraint of the dynamics, a represents the acceleration of the drone, and a min ,a max is the acceleration constraint.

[0092] Based on the above, this application uses Newton iteration to optimize the algorithm. Specifically, L-BFGS-B is used as the optimizer. For an optimization problem: minimize the overall cost function J, the current state of the drone x k The update will be iteratively updated using the following formula:

[0093]

[0094] Among them, x k+1 is the state of the drone at (k+1 time) after iterative update, α k is the step length, J k is the overall cost function at the current moment (time k), For J k The gradient of The parameter meaning of E is the unit matrix s k =x k+1 -x k , Z k and is the intermediate variable generated by the above formula, Zk Used to store and update the inverse of the approximate Hessian matrix, Used to calculate Z in the last iteration update k The weight of Z is updated k+1 .

[0095] In order to ensure the efficiency of optimization and avoid local optimum, this application sets a gradient optimization strategy, such as Figure 3 As shown in , during the iteration process, the algorithm will search for the nearest drone within the range. If there are multiple drones, the two nearest drones will be selected as collision avoidance targets. Then, the normal vector of the direction vector connecting the designated drone with the two collision avoidance targets and the direction vector connecting with the non-cooperative target will be fused to obtain the optimized direction of the gradient, and the optimal target position will be found along this direction, as shown in Figure 3 This is shown in the process from (a) to (b).

[0096] Step 6: Based on the final state estimation result of the non-cooperative target, the control amount of each UAV and the control amount of each UAV pod, multi-UAV cooperative tracking of the non-cooperative target is performed.

[0097] Specifically, the control amount of the pod is used to control the pod to turn to the target area, thereby avoiding the loss of visual tracking due to the target maneuvering too fast. The control amount of the drone is used to maintain the desired distance to the target and the desired distance between drones based on the provided visual estimation. Joint control is performed based on the final state estimation result of the non-cooperative target, the control amount of each drone, and the control amount of each drone pod, thereby achieving multi-drone cooperative tracking of non-cooperative targets.

[0098] As a specific embodiment, this application verifies the effectiveness of the proposed multi-UAV cooperative tracking method for non-cooperative targets through a specific example of three UAVs cooperatively tracking a non-cooperative target. The specific implementation steps of this example are as follows:

[0099] (1) Position and motion settings of non-cooperative targets

[0100] Consider a scenario where three drones are tracking a non-cooperative target, such as Figure 2 As shown, let the non-cooperative target be located at the origin of the coordinate system (2,2). The motion of the non-cooperative target is randomly generated, which can be composed of motion along a straight line, in a circle, and at a random angle. The initial positions of the three drones are (0,2), (0,0), and (0,-2).

[0101] (2) Parameter settings

[0102] Setting the detection threshold λ in the multi-UAV cooperative tracking method for non-cooperative targets t =0.6, outlier threshold t b =1, collision threshold d between drones b=2, the entire non-cooperative target multi-UAV collaborative tracking method of this application is based on ROS operation, with a visual frequency of 30hz and a control frequency of 10hz.

[0103] (3) Results analysis

[0104] The simulation results based on the above data are as follows Figures 4 to 9 The horizontal and vertical coordinates in the figure are the positions of the non-cooperative target (referred to as the target) and the X-axis and Y-axis of the drone in the two-dimensional plane.

[0105] As can be seen from the figure, Figure 4 The tracking of the non-cooperative target in linear motion is shown. It can be seen that at the set position, the drone can achieve stable tracking along with the movement of the non-cooperative target. In addition, the application also tests the tracking of the drone in the case of rotational motion and random motion of the non-cooperative target. Figure 5 and Figure 6 As shown in , it can be seen that the drone moves with the non-cooperative target. At the same time, this application observes the detection of the drone during the tracking process, such as Figure 7 As shown, it can be seen that the detection of the drone has shifted, but due to the multi-drone cooperative tracking method of the non-cooperative target of the present application, the drone switches back to tracking the non-cooperative target, such as Figure 7 As shown in the process (a)-(f) in the figure, the robustness of the multi-UAV collaborative tracking method for non-cooperative targets in this application is effectively demonstrated.

[0106] For collision avoidance, this application sets the positions of the two drones to be 1 meter to the left and right of the non-cooperative target. Figure 8 As shown in the figure, another drone is then used to track the non-cooperative target. It can be seen that the designated drone flies over the right side of both drones. When the distance is set to 2 meters, as shown in the figure, Fig. 9 As shown, the designated UAV flies through the middle area between the two, which effectively proves the effectiveness of the collision avoidance control rules in the multi-UAV cooperative tracking method for non-cooperative targets of this application.

[0107] In summary, the multi-UAV cooperative tracking method for non-cooperative targets in this application establishes a multi-UAV tracking model based on vision, and gives a three-dimensional plane multi-UAV tracking model; establishes a visual detection front end, designs a pod control model based on the pinhole model, i.e., the camera's internal parameters, designs a control cost function based on the collision avoidance rule, and designs an optimization strategy. The designed method is applied to actual UAVs and moving targets, realizing multi-UAV cooperative tracking in the case of errors in the detection front end, and solving the problem of stable multi-UAV tracking in the case of errors in the detection.

[0108] In some embodiments, the present application also provides a computer program product, including a computer program, which, when executed by a processor, implements the multi-UAV collaborative tracking method for non-cooperative targets.

[0109] In some embodiments, the present application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the multi-UAV collaborative tracking method for non-cooperative targets.

[0110] In some embodiments, the present application also provides a computer device, including a processor, a memory, an input / output interface (I / O for short), a communication interface, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the multi-UAV collaborative tracking method for non-cooperative targets.

[0111] Wherein, the processor, memory and input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. Wherein, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store pending transactions. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, the multi-UAV cooperative tracking method of the non-cooperative target is realized.

[0112] It should be noted that the object information (including but not limited to object device information, object personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the object or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant laws, regulations and standards.

[0113] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ReadOnlyMemory, ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but are not limited to this.

[0114] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0115] This article uses specific examples to illustrate the principles and implementation methods of this application. The description of the above embodiments is only used to help understand the method and core ideas of this application. At the same time, for those skilled in the art, according to the ideas of this application, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as limiting this application.

Claims

1. A method for cooperative tracking of multiple UAVs of non-cooperative targets, characterized in that: include: Get the status of each drone at the future moment; Model the control of each UAV based on its state at the future moment, and obtain the control amount of each UAV; Inputting the non-cooperative target images acquired by each UAV into a target position detection model to obtain the coordinates of the non-cooperative target; the target position detection model includes YOLOv5 and Siamese-RPN connected in sequence; Perform dynamic modeling based on the coordinates of the non-cooperative target to determine the final state estimation result of the non-cooperative target; Model the pod control of each UAV according to the pod status of each UAV at the current moment, and obtain the control quantity of the pod of each UAV; Multi-UAV cooperative tracking of non-cooperative targets is performed based on the final state estimation results of the non-cooperative targets, the control amount of each UAV and the control amount of each UAV pod.

2. The method for cooperative tracking of non-cooperative targets by multiple UAVs according to claim 1, characterized in that: The obtaining of the state of each drone at a future time specifically includes: Building a dynamic model of the drone Get the state x(k+1) of the drone at time k+1; where x(k) is the state of the drone at time k; t is the time step; u p (k) is the input control variable of the UAV at time k.

3. The method for cooperative tracking of non-cooperative targets by multiple UAVs according to claim 2, characterized in that: The control of each drone is modeled based on the state of each drone at a future moment to obtain the control amount of each drone, specifically including: Model the control of the drone based on its state at future moments: , Get the control amount J of the drone c ; where x(k+l|k) is the predicted state of the drone at time k; N is the number of prediction steps, 0≤l≤N-1; u p (k+l|k) is the input control quantity of the UAV at the future time; u p (k+l|k) T for u p (k+l|k) is the transpose; x(k+N) is the state of the drone at the predicted final moment; Q, R and V are the state adjustment matrix, control adjustment matrix and final state adjustment matrix respectively.

4. The method for cooperative tracking of non-cooperative targets by multiple UAVs according to claim 1, characterized in that: The non-cooperative target images acquired by each UAV are input into the target position detection model to obtain the coordinates of the non-cooperative target, specifically including: The non-cooperative target image is acquired through the camera installed on the drone pod and input into YOLOv5 to obtain the visual detection result; The visual detection results are input into Siamese-RPN to obtain the pixel coordinates of the non-cooperative target in the image plane; Through the current state of the UAV and the internal parameters of the camera, the pixel coordinates are converted to the UAV coordinate system and the world coordinate system in turn to obtain the coordinates of the non-cooperative target in the world coordinate system.

5. The method for cooperative tracking of non-cooperative targets by multiple UAVs according to claim 1, characterized in that: The performing of dynamic modeling based on the coordinates of the non-cooperative target to determine the final state estimation result of the non-cooperative target specifically includes: Based on the coordinates of non-cooperative targets Get the actual state of the non-cooperative target in, and are the coordinate values ​​of the non-cooperative target in the X-axis, Y-axis and Z-axis directions respectively; and are the speed values ​​of the non-cooperative target in the X-axis, Y-axis and Z-axis directions respectively; The non-cooperative target is dynamically modeled according to its actual state X(k) at time k. Get the state of the non-cooperative target predicted by the drone at time k+1 Among them, t is the time step; w(k) is the time difference between the actual moments; Using formula Calculate the final state estimation result of the non-cooperative target Among them, Q, D and M are the communication topology matrix, detection threshold matrix and prediction threshold matrix respectively.

6. The method for cooperative tracking of non-cooperative targets by multiple UAVs according to claim 1, characterized in that: The pod control of each drone is modeled according to the pod state of each drone at the current moment to obtain the control amount of the pod of each drone, which specifically includes: Model the pod control of the drone based on the current pod state of the drone: Get the control amount u of the drone pod c (k); where e(k) is the state of the pod at time k+1 and the pod status X at time k c e(k-1) is the state of the pod at time k. c (k) and the pod status at time k-1 The difference between is the accumulation of all differences before time k; K p , K m and K d are the corresponding PID parameters respectively.

7. The method for cooperative tracking of multiple UAVs of a non-cooperative target according to claim 3, characterized in that: When a collision between drones is about to occur, collision avoidance control rules are also set. Among them, J b is the collision cost function; d b is the preset collision threshold; δd is a very small value; p is the position of the drones other than the specified drone; The collision avoidance control rules are optimized using Newton iteration. During the iteration process, the nearest UAV within the range is searched. If there are multiple UAVs, the two nearest ones are selected as collision avoidance targets. Then, the normal vector of the direction vector connecting the designated UAV with the two collision avoidance targets and the direction vector connecting with the non-cooperative target are fused to obtain the optimized direction of the gradient, and the optimal position of the designated UAV is found along this direction.

8. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the multi-UAV cooperative tracking method for a non-cooperative target described in any one of claims 1 to 7 is implemented.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the multi-UAV cooperative tracking method for a non-cooperative target described in any one of claims 1 to 7 is implemented.

10. A computer device comprising: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the multi-UAV cooperative tracking method for non-cooperative targets according to any one of claims 1 to 7.

Images