Multi-unmanned aerial vehicle cooperative tracking method, product, medium and equipment of non-cooperative target

Abstract

The application discloses a multi-unmanned aerial vehicle cooperative tracking method, product, medium and equipment of non-cooperative target, relates to the unmanned aerial vehicle technical field, and the method comprises the following steps: acquiring the state of each unmanned aerial vehicle at a future time and modeling the control of each unmanned aerial vehicle to obtain the control amount of each unmanned aerial vehicle; inputting the non-cooperative target image acquired by each unmanned aerial vehicle into a target position detection model to obtain the coordinates of the non-cooperative target; performing dynamic modeling based on the coordinates of the non-cooperative target to determine the final state estimation result of the non-cooperative target; modeling the nacelle control of each unmanned aerial vehicle according to the nacelle state of each unmanned aerial vehicle at the current time to obtain the control amount of the nacelle of each unmanned aerial vehicle; and performing multi-unmanned aerial vehicle cooperative tracking of the non-cooperative target based on the final state estimation result of the non-cooperative target, the control amount of each unmanned aerial vehicle and the control amount of the nacelle of each unmanned aerial vehicle. The application can effectively enhance the stability and robustness of the multi-unmanned aerial vehicle cooperative tracking of the non-cooperative target.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of unmanned aerial vehicles, in particular to a multi-unmanned aerial vehicle cooperative tracking method for non-cooperative targets, a product, a medium and equipment. BACKGROUND

[0002] With the rapid development of unmanned aerial vehicle technology, systems based on single unmanned aerial vehicles have been widely applied in agriculture, monitoring and map construction, among which, tracking systems play an important role as the core. However, with the expansion of the scene, a single unmanned aerial vehicle often has interference, occlusion and false negative and false positive situations, which brings the problem of target tracking loss, and it is difficult to complete the continuous tracking of the target. In order to solve this problem, it is usually required that the unmanned aerial vehicle has stronger perception performance or multiple unmanned aerial vehicles are used to cooperatively observe the scene. The enhancement of the perception performance will inevitably lead to an increase in cost, and sometimes even a multiple increase. If an unmanned aerial vehicle with stronger perception performance is used but fails to successfully perceive the target, it will cause great waste of resources. Therefore, it is difficult to consider using multiple unmanned aerial vehicles to implement target tracking and build a cooperative tracking framework.

[0003] Many existing unmanned aerial vehicle tracking works are mostly focused on state estimation of targets or control of multiple unmanned aerial vehicles according to the positions of known targets, and the research results of state estimation of non-cooperative targets or cooperative tracking of known states are often difficult to apply to cooperative tracking of non-cooperative targets, and forced application may lead to tracking loss and unstable tracking effect. In view of the current research status of multi-unmanned aerial vehicle cooperative tracking, it is a quite challenging problem to design a cooperative tracking strategy for non-cooperative targets. SUMMARY

[0004] The purpose of the present application is to provide a multi-unmanned aerial vehicle cooperative tracking method for non-cooperative targets, a product, a medium and equipment, which can effectively enhance the stability and robustness of multi-unmanned aerial vehicle cooperative tracking for non-cooperative targets.

[0005] To achieve the above purpose, the present application provides the following solutions:

[0006] In a first aspect, the present application provides a multi-unmanned aerial vehicle cooperative tracking method for non-cooperative targets, comprising:

[0007] obtaining the state of each unmanned aerial vehicle at a future time;

[0008] modeling the control of each unmanned aerial vehicle based on the state of each unmanned aerial vehicle at the future time, to obtain the control amount of each unmanned aerial vehicle;

[0009] input the non-cooperative target image acquired by each unmanned aerial vehicle into a target position detection model to obtain coordinates of the non-cooperative target; the target position detection model comprises YOLOv5 and Siamese-RPN connected in sequence;

[0010] perform dynamic modeling based on the coordinates of the non-cooperative target to determine a final state estimation result of the non-cooperative target;

[0011] model the nacelle control of each unmanned aerial vehicle according to the nacelle state of each unmanned aerial vehicle at the current time to obtain a control amount of the nacelle of each unmanned aerial vehicle;

[0012] perform multi-unmanned aerial vehicle cooperative tracking of the non-cooperative target based on the final state estimation result of the non-cooperative target, the control amount of each unmanned aerial vehicle, and the control amount of the nacelle of each unmanned aerial vehicle.

[0013] Optionally, the state of each unmanned aerial vehicle at a future time is obtained, and specifically includes:

[0014] a dynamic model of the unmanned aerial vehicle is constructed to obtain the state x(k+1) of the unmanned aerial vehicle at the k+1 time; wherein x(k) is the state of the unmanned aerial vehicle at the k time; t is a time step; u p (k) is the input control amount of the unmanned aerial vehicle at the k time.

[0015] Optionally, the control of each unmanned aerial vehicle is modeled based on the state of each unmanned aerial vehicle at a future time to obtain a control amount of each unmanned aerial vehicle, and specifically includes:

[0016] the control of the unmanned aerial vehicle is modeled based on the state of the unmanned aerial vehicle at a future time:

[0017] to obtain the control amount J c of the unmanned aerial vehicle; wherein x(k+l|k) is the predicted state of the unmanned aerial vehicle at a future time at the k time; N is a prediction step number, 0≤l≤N-1; u p (k+l|k) is the input control amount of the unmanned aerial vehicle at a future time; u p (k+l|k) T is the transpose of u p (k+l|k); x(k+N) is the predicted state of the unmanned aerial vehicle at a final time; Q, R, and V are respectively a state adjustment matrix, a control adjustment matrix, and a final state adjustment matrix.

[0018] Optionally, the non-cooperative target image acquired by each unmanned aerial vehicle is input into a target position detection model to obtain coordinates of the non-cooperative target, and specifically includes:

[0019] the non-cooperative target image is acquired by a camera installed on the nacelle of the unmanned aerial vehicle and is input into YOLOv5 to obtain a visual detection result;

[0020] The visual detection results are input into Siamese-RPN to obtain the pixel coordinates of the non-cooperative target in the image plane;

[0021] By using the current state of the UAV and the camera's intrinsic parameters, the pixel coordinates are sequentially transformed to the UAV coordinate system and the world coordinate system to obtain the coordinates of the non-cooperative target in the world coordinate system.

[0022] Optionally, the step of performing dynamic modeling based on the coordinates of the non-cooperative target to determine the final state estimation result of the non-cooperative target specifically includes:

[0023] Coordinates based on non-cooperative objectives Obtain the actual state of the non-cooperative goal in, and These are the coordinates of the non-cooperative target along the X, Y, and Z axes, respectively. and These represent the velocity values ​​of the non-cooperative target in the X, Y, and Z axes, respectively.

[0024] Dynamic modeling of the non-cooperative objective is performed based on its actual state X(k) at time k. Obtain the state of the non-cooperative target predicted by the drone at time k+1. Where t is the time step; w(k) is the actual time difference between moments;

[0025] Using formula The final state estimation results for the non-cooperative objective were calculated. Where Q, D, and M are the communication topology matrix, detection threshold matrix, and prediction threshold matrix, respectively.

[0026] Optionally, the step of modeling the pod control of each UAV based on the current pod status of each UAV to obtain the control variables for each UAV pod specifically includes:

[0027] Model the drone's pod control based on the drone's current pod status: Obtain the control quantity u of the UAV pod c (k); where e(k) is the pod state at time k+1. The pod state X at time k c The difference between (k); e(k-1) is the pod state X at time k. c (k) and the pod status at time k-1 The difference between them; K is the sum of all differences up to time k; p K mand K d are corresponding PID parameters, respectively.

[0028] Optionally, when a collision is about to occur between each UAV, a collision avoidance control rule is further provided wherein, J b is a collision cost function; d b is a preset collision threshold; δd is a very small value; p is the position of the UAVs other than the specified UAV;

[0029] The collision avoidance control rule is optimized using Newton iteration, in the iteration process, the nearest UAV in the search range is searched, if there are multiple UAVs, the nearest two UAVs are selected as the collision avoidance targets, then the normal vector of the direction vector connecting the specified UAV and the two collision avoidance targets is fused with the direction vector connecting the non-cooperative target to obtain the optimization direction of the gradient, and the optimal position of the specified UAV is searched along the direction.

[0030] In a second aspect, the present application provides a computer program product, comprising a computer program which, when executed by a processor, implements the multi-UAV cooperative tracking method of the non-cooperative target.

[0031] In a third aspect, the present application provides a computer readable storage medium, which stores a computer program, and the computer program, when executed by a processor, implements the multi-UAV cooperative tracking method of the non-cooperative target.

[0032] In a fourth aspect, the present application provides a computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, and the processor executes the computer program to implement the multi-UAV cooperative tracking method of the non-cooperative target.

[0033] According to the specific embodiments provided by the present application, the present application has the following technical effects:

[0034] The application provides a multi-unmanned aerial vehicle cooperative tracking method for a non-cooperative target, a product, a medium and equipment, and mainly relates to a multi-unmanned aerial vehicle cooperative tracking method based on vision. The non-cooperative target images obtained by each unmanned aerial vehicle are input into a target position detection model constructed in the application to realize stable visual tracking of the non-cooperative target. Then, the coordinates of the non-cooperative target obtained by the target position detection model are used for dynamic modeling, so that the final state estimation result of the non-cooperative target is stably obtained. The control of the nacelle of each unmanned aerial vehicle and the control of each unmanned aerial vehicle are modeled respectively to obtain the control amount of the nacelle of each unmanned aerial vehicle and the control amount of each unmanned aerial vehicle. Finally, the multi-unmanned aerial vehicle cooperative tracking of the non-cooperative target is performed based on the final state estimation result of the non-cooperative target, the control amount of each unmanned aerial vehicle and the control amount of the nacelle of each unmanned aerial vehicle, which can effectively enhance the stability and robustness of the multi-unmanned aerial vehicle cooperative tracking of the non-cooperative target. BRIEF DESCRIPTION OF DRAWINGS

[0035] In order to more clearly illustrate the technical solutions in the embodiments of the application or the related art, the drawings needed in the embodiments will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the application, and other drawings can be obtained by those skilled in the art without creative labor.

[0036] Figure 1 A flowchart of the multi-unmanned aerial vehicle cooperative tracking method for a non-cooperative target provided by the application is shown.

[0037] Figure 2 A scene diagram of the multi-unmanned aerial vehicle tracking a non-cooperative target in a three-dimensional plane provided by the application is shown.

[0038] Figure 3 A gradient optimization strategy diagram provided by the application is shown.

[0039] Figure 4 A tracking diagram of the linear motion of a non-cooperative target provided by the application is shown.

[0040] Figure 5 A tracking diagram of the rotational motion of a non-cooperative target provided by the application is shown.

[0041] Figure 6 A tracking diagram of the random motion of a non-cooperative target provided by the application is shown.

[0042] Figure 7 A diagram of the abnormal perception recovery of an unmanned aerial vehicle provided by the application is shown.

[0043] Figure 8 A diagram of the mutual collision avoidance of unmanned aerial vehicles provided by the application is shown.

[0044] Figure 9 The UAV mutual collision avoidance case 2 schematic diagram provided by the present application is shown. DETAILED DESCRIPTION

[0045] For the state estimation problem of non-cooperative targets, Shafiei et al. use Kalman filter or extended Kalman filter to realize the motion modeling of non-cooperative targets, and realize the real-time prediction of the state of non-cooperative targets through the obtained ranging and angle measurement information. In the case of known target state, Abdelmawgoud et al. often need to pass data to the center node for calculation in the cooperative method, and once the center node is lost, the whole system will be paralyzed, and a semi-central or distributed method is proposed, which effectively improves the robustness of the algorithm by passing effective data through the topological structure. Liu et al. proposed a trajectory generation method for the control problem in the cooperative process, which effectively realizes the mutual collision in the cooperative tracking process. In order to realize the real-time process of collision avoidance, Rodríguez et al. designed a real-time collision avoidance method, which can calculate the feasible control input at each control, effectively improving the robustness of cooperative tracking in dynamic scenes. However, the above research results on state estimation of non-cooperative targets or cooperative tracking of known states are difficult to apply to cooperative tracking of non-cooperative targets. Forced combination of the above state estimation method and cooperative method will lead to the phenomenon of tracking loss, and the effect of cooperative tracking is poor. Therefore, the present application provides a multi-UAV cooperative tracking method for non-cooperative targets, products, media and equipment, which can effectively enhance the stability and robustness of multi-UAV cooperative tracking for non-cooperative targets.

[0046] The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative labor are within the scope of protection of the present application.

[0047] In order to make the above-mentioned purposes, features and advantages of the present application more obvious and easy to understand, the present application will be further described in detail below with reference to the drawings and specific embodiments.

[0048] As shown in Figure 1 The multi-UAV cooperative tracking method for non-cooperative targets disclosed by the present application comprises:

[0049] Step 1: Obtain the state of each UAV at the future time.

[0050] For the case of multiple UAVs tracking a moving non-cooperative target (referred to as target), the state of each UAV at the future time needs to be obtained first. The scene of multiple UAVs tracking the target in a three-dimensional plane is constructed, as shown inFigure 2 as shown.

[0051] Suppose the distribution of UAVs and targets is as shown in Figure 2 Each UAV has its own coordinate system, namely the UAV coordinate system, denoted by subscript B, and each UAV is equipped with a visual sensor (camera), and the camera coordinate system is denoted by subscript C. T represents a non-cooperative target, and T C represents the position of the non-cooperative target in the image plane coordinate system of the camera, therefore, (X B1 , Y B1 , Z B1 ) represents the position of the first UAV at the X-axis, Y-axis and Z-axis in its own coordinate system, (X C1 , Y C1 , Z C1 ) represents the position of the first UAV at the X-axis, Y-axis and Z-axis in the camera coordinate system, T C1 represents the position of the non-cooperative target in the image plane coordinate system of the camera of the first UAV, the second UAV and the third UAV are denoted as above, and the world coordinate system is denoted by subscript W.

[0052] The state of the UAV is denoted by x, and the dynamic model of the UAV is constructed as follows:

[0053]

[0054] wherein x(k) is the state of the UAV at time k; x(k+1) is the state of the UAV at time k+1; t is the time step; u p (k) is the input control quantity of the UAV at time k.

[0055] Step 2: Model the control of each UAV based on the state of each UAV at the future time, and obtain the control quantity of each UAV.

[0056] Based on step 1, the control of the UAV is modeled based on the state of the UAV at the future time, so as to design a dynamic programming algorithm to make the UAV fly to the vicinity of the non-cooperative target.

[0057]

[0058] wherein J c is the control quantity of the UAV; x(k+l|k) is the predicted state of the UAV at the future time under the condition that the UAV is at time k; N is the prediction step number, 0≤l≤N-1; u p (k+l|k) is the input control quantity of the UAV at the future time; u p (k+l|k) T is u ptranspose of (k+l|k); x(k+N) is the predicted final time state of the UAV; U, R and V are the adjustment matrix of state, the adjustment matrix of control and the adjustment matrix of final state respectively.

[0059] Step 3: input the non-cooperative target image obtained by each UAV into the target position detection model to obtain the coordinates of the non-cooperative target.

[0060] After the scene of tracking the non-cooperative target by the multiple UAVs in the three-dimensional plane is constructed in step 1, the visual detection front end is established, which is as follows.

[0061] On the basis of the above, firstly, the position coordinates of the non-cooperative target need to be obtained. For the non-cooperative target, the application hopes to construct an autonomous detection model, so YOLOv5 and Siamese-RPN are used as the front-end visual tracking system of the application, that is, the target position detection model. Specifically, the visual detection result of YOLOv5 is used as the input of the single target tracker Siamese-RPN, so as to realize stable visual tracking, and the pixel coordinates of the non-cooperative target in the image plane are output.

[0062]

[0063] Tack=f(Det) (4)

[0064] wherein Det is the YOLOv5 detection network The output of receiving the pod input image I; Tack is the tracking result of the single target tracking network f(·) of Siamese-RPN, that is, the pixel coordinates of the non-cooperative target in the image plane.

[0065] For the obtained visual detection information, that is, the pixel coordinates of the non-cooperative target in the image plane T C =(u i ,v i ,1), wherein u i , v i and 1 are respectively the values of the position of the non-cooperative target in the image plane coordinate system in the X-axis, Y-axis and Z-axis directions photographed by the camera of the i-th UAV, the state of the non-cooperative target is estimated through the position of the UAV itself, specifically, the position of the non-cooperative target needs to be converted from the pixel coordinates to the UAV coordinate system (machine system) through the state x of the UAV at the current time and the camera internal parameter, and then further converted to the world system:

[0066]

[0067] wherein R WB is the rotation and translation matrix of the world system to the machine system; RBC is the rotation and translation matrix of the camera system; K is the camera intrinsic parameter, which specifically includes the physical size of the horizontal and vertical focal points and the center point of the pixel horizontal axis coordinate system; H is the height of the UAV; is the translation from the UAV system to the world system; wherein, and are the coordinate values of the non-cooperative target in the X-axis, Y-axis and Z-axis directions, respectively.

[0068] Specifically, the target position detection model described in the present application comprises YOLOv5 and Siamese-RPN connected in sequence. The non-cooperative target image is obtained by the camera installed on the UAV pod, and is input into YOLOv5 to obtain the visual detection result; the visual detection result is input into Siamese-RPN to obtain the pixel coordinates of the non-cooperative target in the image plane; the pixel coordinates are sequentially converted to the UAV coordinate system and the world coordinate system through the state of the UAV at the current moment and the intrinsic parameter of the camera, to obtain the coordinates of the non-cooperative target in the world coordinate system

[0069] Step 4: based on the coordinates of the non-cooperative target, a dynamic model is established to determine the final state estimation result of the non-cooperative target.

[0070] In order to further realize the state estimation of the target, a dynamic model is established for the target, and the actual state of the target and are the velocity values of the non-cooperative target in the X-axis, Y-axis and Z-axis directions, respectively. Considering the high real-time performance of the algorithm, the target is modeled as a uniform motion model, and the state of the target at the next moment is estimated that is, the non-cooperative target is dynamically modeled according to the actual state X(k) of the non-cooperative target at the k moment, to obtain the state of the non-cooperative target predicted by the UAV at the k+1 moment

[0071]

[0072] wherein t is the time step; w(k) is the actual time difference between moments.

[0073] Considering that the message transmission is directional in the cooperative process, a directed graph model is constructed, so that the UAVs can transmit the state information of the target through the existing communication conditions, and Q is constructed as the communication topology, representing the communication topology matrix. Considering the stability of detection, the detection value λ i of the i-th UAV is set as the detection threshold λ t as the judgment standard of whether it is a valid detection.

[0074]

[0075] where D(i) is the probability of the i-th UAV's effective detection, abbreviated as D, represents the detection threshold matrix.

[0076] In view of the fact that false detection and missed detection will affect the stability of the whole system, the application constructs an optimal estimation method.

[0077]

[0078] where F(i,j) is the estimation error between the i-th UAV and the j-th UAV, is the rotation and translation matrix of the i-th UAV system to the world system, t b is the distance threshold, M(i,j) represents that if the threshold between the i-th UAV and the j-th UAV exceeds a certain specific value, it will be considered as an invalid value, abbreviated as M, represents the prediction threshold matrix, and the invalid value is filtered out by calculating the variance.

[0079] Therefore, the final effective estimation expression is as shown in the following formula, which respectively satisfies the communication topology matrix Q, the detection threshold matrix D and the prediction threshold matrix M:

[0080]

[0081] where, is the final state estimation result of the non-cooperative target.

[0082] Step 5: Model the nacelle control of each UAV according to the current time nacelle state of each UAV, and obtain the control amount of the nacelle of each UAV.

[0083] In order to realize that the nacelle can be aligned with the target in real time when the camera detects the target, a nacelle control model is designed:

[0084]

[0085] where e(k) is the nacelle state at k+1 time , that is, the difference between the expected nacelle state and the nacelle state X c (k) at k time; e(k-1) is the difference between the nacelle state X c (k) at k time and the nacelle state X at k-1 time; is the cumulative sum of all differences before k time; u c (k) is the control amount of the UAV nacelle; K p , K m and K dPID parameters corresponding respectively. By calculating the error between the expected and actual, a control model is constructed so that the target is always aligned with the center of the pod, achieving stable tracking.

[0086] Although the present application can achieve tracking control of the unmanned aerial vehicle through the above steps, there is still a possibility of collision between unmanned aerial vehicles, in order to solve this problem, the present application designs a collision avoidance control rule:

[0087]

[0088] Wherein, J b is the cost function of collision; d b is a preset collision threshold; δd is a very small value; p is the position of the unmanned aerial vehicle except the designated unmanned aerial vehicle. The purpose of the formula is to find the point with the minimum distance to other unmanned aerial vehicles for the predicted state, and control the point to fly to another direction.

[0089] Therefore, the overall cost function J can be expressed as:

[0090]

[0091] Wherein, v represents the speed of the unmanned aerial vehicle, v min ,v max is the dynamic speed constraint, a represents the acceleration of the unmanned aerial vehicle, a min ,a max is the acceleration constraint.

[0092] On the basis of the above, the present application uses Newton iteration to optimize the algorithm, specifically using L-BFGS-B as an optimizer, for an optimization problem: minimizing the overall cost function J, the state x k of the unmanned aerial vehicle at the current time is updated using the following formula for iterative update:

[0093]

[0094] Wherein, x k+1 is the state of the unmanned aerial vehicle after iterative update (k+1 time), α k is the step size, J k is the overall cost function at the current time (k time), is the gradient of J k , The parameter meaning of E is a unit matrix s k =x k+1 -x k , Z k and are intermediate variables generated by the above formula, Zk for storing and updating an approximate inverse of the hessian matrix, for calculating the weight of Z k in the last iteration update, and updating Z k+1 .

[0095] In order to ensure the efficiency of optimization and avoid local optimum, the gradient optimization strategy is set in the application, as shown in Figure 3 , in the iteration process, the algorithm will search for the nearest unmanned aerial vehicle in the range, if there are multiple unmanned aerial vehicles, the nearest two are selected as the collision avoidance target, then the normal vector of the direction vector connecting the specified unmanned aerial vehicle and the two collision avoidance targets is fused with the direction vector connected with the non-cooperative target to obtain the optimization direction of the gradient, and the optimal target position is found along this direction, as shown in the process of Figure 3 (a) to (b).

[0096] Step 6: Based on the final state estimation result of the non-cooperative target, the control amount of each unmanned aerial vehicle, and the control amount of each unmanned aerial vehicle pod, the multi-unmanned aerial vehicle cooperative tracking of the non-cooperative target is carried out.

[0097] Specifically, the control amount of the pod is used to control the pod to turn to the target area, so as to avoid the loss of visual tracking due to the too fast target maneuver. The unmanned aerial vehicle control amount is used to maintain the desired distance from the target and the desired distance between the unmanned aerial vehicles according to the provided visual estimation. Based on the final state estimation result of the non-cooperative target, the control amount of each unmanned aerial vehicle, and the control amount of each unmanned aerial vehicle pod, joint control is carried out, so as to realize the multi-unmanned aerial vehicle cooperative tracking of the non-cooperative target.

[0098] As a specific embodiment, the effectiveness of the proposed multi-unmanned aerial vehicle cooperative tracking method of non-cooperative targets is verified through a specific example of three unmanned aerial vehicles cooperatively tracking a non-cooperative target. The specific implementation steps of this example are as follows:

[0099] (1) Position and motion setting of non-cooperative target

[0100] Considering the scenario of three unmanned aerial vehicles tracking a non-cooperative target, as shown in Figure 2 , the non-cooperative target is located at the coordinate origin (2, 2), and the motion of the non-cooperative target is randomly generated and can be composed of straight line, turning and random angle motion. The initial positions of the three unmanned aerial vehicles are (0, 2), (0, 0) and (0, -2) respectively.

[0101] (2) Parameter setting

[0102] The detection threshold λ t = 0.6, the outlier threshold t b = 1, and the collision threshold d b= 2, the multi-UAV cooperative tracking method of the non-cooperative target of the application is based on ROS, the visual frequency is 30 hz, and the control frequency is 10 hz.

[0103] (3) Result analysis

[0104] The simulation results based on the above data are shown in Figures 4 to 9 . The horizontal and vertical coordinates in the figure are the positions of the non-cooperative target (referred to as the target) and the UAV in the X-axis and Y-axis of the two-dimensional plane, respectively.

[0105] As can be seen from the figure, Figure 4 the tracking of the non-cooperative target under linear motion is shown, and it can be seen that the UAV realizes stable tracking with the movement of the non-cooperative target at the set position. In addition, the application also tests the tracking of the UAV under the conditions of rotational motion and random motion of the non-cooperative target, as shown in Figure 5 and Figure 6 , it can be seen that the UAV moves with the non-cooperative target. At the same time, the application observes the detection of the UAV in the tracking process, as shown in Figure 7 , it can be seen that the detection of the UAV has deviated, but due to the multi-UAV cooperative tracking method of the non-cooperative target of the application, the UAV switches back to the tracking of the non-cooperative target, as shown in the processes of (a)-(f) in Figure 7 , effectively proving the robustness of the multi-UAV cooperative tracking method of the non-cooperative target of the application.

[0106] For collision avoidance, the application sets the positions of the two UAVs to be 1 meter on the left and right sides of the non-cooperative target, as shown in Figure 8 , and then makes another UAV track the non-cooperative target, it can be seen that the specified UAV flies from the right side of the two. When set to 2 meters, as shown in Figure 9 , the specified UAV flies from the middle area of the two, effectively proving the effectiveness of the collision avoidance control rule in the multi-UAV cooperative tracking method of the non-cooperative target of the application.

[0107] In summary, the multi-UAV cooperative tracking method of the non-cooperative target of the application establishes a multi-UAV tracking model based on vision, gives a multi-UAV tracking model in a three-dimensional plane, establishes a vision detection front end, designs a pod control model based on the pinhole model, that is, the internal parameters of the camera, designs a control cost function based on the collision avoidance rule, and designs an optimization strategy. The method designed is applied to actual UAVs and moving targets, realizing multi-UAV cooperative tracking under the condition of error in the detection front end, and solving the stable tracking of multiple machines under the condition of detection error.

[0108] In some embodiments, the present application also provides a computer program product comprising a computer program which, when executed by a processor, implements the method for cooperative tracking of a non-cooperative target by multiple unmanned aerial vehicles.

[0109] In some embodiments, the present application also provides a computer readable storage medium having stored thereon a computer program which, when executed by a processor, implements the method for cooperative tracking of a non-cooperative target by multiple unmanned aerial vehicles.

[0110] In some embodiments, the present application also provides a computer device comprising a processor, a memory, an input / output interface, a communication interface, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the method for cooperative tracking of a non-cooperative target by multiple unmanned aerial vehicles.

[0111] The processor, the memory and the input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. The processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device comprises a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for running the operating system and the computer program in the non-volatile storage medium. The database of the computer device is configured to store to-be-processed transactions. The input / output interface of the computer device is configured to exchange information between the processor and external devices. The communication interface of the computer device is configured to communicate with external terminals through network connection. The computer program is executed by the processor to implement the method for cooperative tracking of a non-cooperative target by multiple unmanned aerial vehicles.

[0112] It should be noted that the object information (including but not limited to object device information, object personal information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) involved in the present application are all information and data authorized by the object or fully authorized by all parties, and the collection, use and processing of related data need to comply with relevant laws, regulations and standards.

[0113] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a non-volatile computer readable storage medium, and when the computer program is executed, the processes of the above-mentioned embodiments of the methods can be included. Any reference to memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ReadOnly Memory, ROM), magnetic tape, floppy disk, flash memory, optical storage, high-density embedded non-volatile memory, resistive random access memory (Resistive Random Access Memory, ReRAM), magnetoresistive random access memory (Magnetoresistive Random Access Memory, MRAM), ferroelectric memory (Ferroelectric Random Access Memory, FRAM), phase change memory (Phase Change Memory, PCM), graphene memory, etc. Volatile memory can include random access memory (Random Access Memory, RAM) or external cache memory, etc. As an illustration but not limitation, RAM can be in various forms, such as static random access memory (Static Random Access Memory, SRAM) or dynamic random access memory (Dynamic Random Access Memory, DRAM), etc. The database involved in the embodiments provided in the present application can include at least one of a relational database and a non-relational database. The non-relational database can include a distributed database based on a block chain, etc., without being limited thereto. The processor involved in the embodiments provided in the present application can be a general-purpose processor, a central processing unit, a graphics processing unit, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., without being limited thereto.

[0114] Any combination of the technical features of the above embodiments can be made. In order to make the description simple, all possible combinations of the technical features in the above embodiments are not described, however, as long as the combination of the technical features does not exist contradictory, it should be considered as the scope of the present application.

[0115] The principles and implementation modes of the present application are described by using specific examples in the present application. The above embodiments are only used to help understand the method and its core idea of the present application; at the same time, for those skilled in the art, according to the idea of the present application, the specific implementation mode and application range will be changed. In conclusion, the content of the present application should not be understood as a limitation.

Claims

1. A multi-UAV cooperative tracking method for non-cooperative targets, characterized in that, include: Obtain the status of each drone at a future moment; The control of each UAV is modeled based on its future state, and the control variables of each UAV are obtained. The non-cooperative target images acquired by each UAV are input into the target location detection model to obtain the coordinates of the non-cooperative targets; the target location detection model includes YOLOv5 and Siamese-RPN connected in sequence; Dynamic modeling is performed based on the coordinates of the non-cooperative objective to determine the final state estimation result of the non-cooperative objective; Model the pod control of each UAV based on the current pod status of each UAV, and obtain the control variables of each UAV pod. Multi-UAV cooperative tracking of non-cooperative targets is performed based on the final state estimation results of the non-cooperative target, the control variables of each UAV, and the control variables of each UAV pod. The dynamic modeling based on the coordinates of the non-cooperative target, and the determination of the final state estimation result of the non-cooperative target, specifically includes: Coordinates based on non-cooperative objectives Obtain the actual state of the non-cooperative goal in, and These are the coordinates of the non-cooperative target along the X, Y, and Z axes, respectively. and These represent the velocity values ​​of the non-cooperative target in the X, Y, and Z axes, respectively. Dynamic modeling of the non-cooperative objective is performed based on its actual state X(k) at time k. Obtain the state of the non-cooperative target predicted by the drone at time k+1. Where t is the time step; w(k) is the actual time difference between moments; Using formula The final state estimation results for the non-cooperative objective were calculated. Where Q, D, and M are the communication topology matrix, detection threshold matrix, and prediction threshold matrix, respectively; Where D(i) is the probability of effective detection of the i-th UAV, abbreviated as D, and represents the detection threshold matrix; λ i Let λ be the detection value of the i-th drone. t The detection threshold; Where F(i,j) is the estimation error between the i-th drone and the j-th drone. Let t be the rotation and translation matrix from the i-th unmanned aerial vehicle system to the world system. b M(i,j) represents the distance threshold. If the threshold between the i-th drone and the j-th drone exceeds a certain value, it will be considered an invalid value. It is abbreviated as M. It represents the prediction threshold matrix. For invalid values, the variance is calculated to filter out outliers.

2. The multi-UAV cooperative tracking method for non-cooperative targets according to claim 1, characterized in that, The acquisition of the state of each drone at a future time specifically includes: Constructing a dynamic model of a drone Obtain the state x(k+1) of the UAV at time k+1; where x(k) is the state of the UAV at time k; t is the time step; u p (k) represents the input control quantity of the UAV at time k.

3. The multi-UAV cooperative tracking method for non-cooperative targets according to claim 2, characterized in that, The control of each UAV is modeled based on its future state to obtain the control variables for each UAV, specifically including: Modeling the control of the drone based on its future state: Obtain the control quantity J of the UAV c Where x(k+l|k) is the predicted future state of the UAV at time k; N is the number of prediction steps, 0≤l≤N-1; u p (k+l|k) represents the input control quantity of the UAV at a future time; u p (k+l|k) T For u p The transpose of (k+l|k); x(k+N) is the predicted state of the UAV at the final moment; Q, R, and V are the state adjustment matrix, control adjustment matrix, and final state adjustment matrix, respectively.

4. The multi-UAV cooperative tracking method for non-cooperative targets according to claim 1, characterized in that, The step of inputting the non-cooperative target images acquired by each UAV into the target location detection model to obtain the coordinates of the non-cooperative targets specifically includes: Images of non-cooperative targets are acquired by cameras mounted on drone pods and input into YOLOv5 to obtain visual detection results; The visual detection results are input into Siamese-RPN to obtain the pixel coordinates of the non-cooperative target in the image plane; By using the current state of the UAV and the camera's intrinsic parameters, the pixel coordinates are sequentially transformed to the UAV coordinate system and the world coordinate system to obtain the coordinates of the non-cooperative target in the world coordinate system.

5. The multi-UAV cooperative tracking method for non-cooperative targets according to claim 1, characterized in that, The process of modeling the pod control of each UAV based on its current pod status to obtain the control variables for each UAV pod specifically includes: Model the drone's pod control based on the drone's current pod status: Obtain the control quantity u of the UAV pod c (k); where e(k) is the pod state at time k+1. The pod state X at time k c The difference between (k); e(k-1) is the pod state X at time k. c (k) and the pod status at time k-1 The difference between them; K is the sum of all differences up to time k; p K m and K d These are the corresponding PID parameters.

6. The multi-UAV cooperative tracking method for non-cooperative targets according to claim 3, characterized in that, Collision avoidance control rules are also set up to prevent collisions between drones. Among them, J b The cost function for collisions; d b δd is a preset collision threshold; δd is a very small value; p is the position of drones other than the specified drone. The collision avoidance control rule is optimized using Newton's iteration. During the iteration process, the nearest drone within the range is searched. If there are multiple drones, the two closest ones are selected as collision avoidance targets. Then, the normal vector of the direction vector connecting the specified drone to the two collision avoidance targets and the direction vector connecting to the non-cooperative target are fused to obtain the gradient optimization direction. The optimal position of the specified drone is found along this direction.

7. A computer program product, comprising a computer program, characterized in that, When executed by a processor, the computer program implements the multi-UAV cooperative tracking method for non-cooperative targets as described in any one of claims 1-6.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by a processor, the computer program implements the multi-UAV cooperative tracking method for non-cooperative targets as described in any one of claims 1-6.

9. A computer device, comprising: A memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that the processor executes the computer program to implement the multi-UAV cooperative tracking method for a non-cooperative target as described in any one of claims 1-6.

Images