Target tracking method for fixed-wing unmanned aerial vehicle

Abstract

The invention discloses a target tracking method for a fixed-wing unmanned aerial vehicle, and the method comprises the steps: converting the unmanned aerial vehicle into a fixed-wing flight mode, carrying out the recognition of a plurality of targets according to the image information collected by a camera in real time, and outputting a recognition result; tracking the identified target according to a Sort multi-target tracking algorithm, obtaining a predicted flight trajectory of a predicted target, calculating a crossing-to-union ratio between the predicted target and the identified target, and constructing a crossing-to-union ratio matrix based on the crossing-to-union ratio to obtain a tracked target; calculating sight angle information of the tracking target relative to the horizontal direction and the vertical direction of the unmanned aerial vehicle; the unmanned aerial vehicle is controlled based on the sight angle information in the horizontal direction and the altitude information obtained by the unmanned aerial vehicle height sensor, and tracking flight of the unmanned aerial vehicle on the target is achieved. According to the invention, the target is identified and tracked by adopting an identification algorithm and a Sort multi-target tracking algorithm, so that the fixed-wing unmanned aerial vehicle can intelligently track the specific target while the reliability is ensured; and the tracking target of the unmanned aerial vehicle can be automatically switched.

Description

Technical Field

[0001] The present invention relates to the field of unmanned aerial vehicle (UAV) flight control, and more specifically, to a target tracking method for a fixed-wing UAV. Background Art

[0002] With the increasing popularity of low-cost commercial drones, their ease of operation and high flexibility have led to their application in various specialized missions, such as disaster relief and urban security. Furthermore, with the gradual maturity of computer vision and artificial intelligence technologies, advanced image processing algorithms are being deployed to ensure that drones can perform their missions more efficiently and accurately.

[0003] Currently, most drone target tracking missions use a human-in-the-loop (HIL) mode. This requires a human to select a tracking area on the image returned by the drone's ground station. The onboard image processor then locks onto the selected area using a single-target tracking algorithm, outputting this information as input for controlling the drone's tracking flight. However, due to the instability of the drone's platform during high-altitude flight, the scale and brightness of the target in the image fluctuate, resulting in significant pixel variations in the tracking area, which can easily lead to target loss and pose a significant challenge to drone tracking. Furthermore, common drone target tracking methods are limited by the need for human intervention and are only designed for single-target tracking, which hinders the intelligent performance requirements of drone missions. Summary of the Invention

[0004] The present invention provides a target tracking method for a fixed-wing unmanned aerial vehicle (UAV) to solve at least one of the problems existing in the prior art.

[0005] In order to achieve the above object, the present invention adopts the following technical solutions:

[0006] The present invention provides a target tracking method for a fixed-wing UAV, the method comprising:

[0007] Switch the drone to fixed-wing flight mode, perform multiple target recognition based on the image information collected by the camera in real time, and output the recognition results corresponding to the multiple targets;

[0008] Track the identified target according to the Sort multi-target tracking algorithm, obtain the predicted flight trajectory of the predicted target, calculate the intersection-over-union ratio between the predicted target and the identified target, and construct an intersection-over-union ratio matrix between the predicted target and the identified target based on the intersection-over-union ratio to obtain the tracked target;

[0009] Calculate the horizontal and vertical sight angle information of the tracking target relative to the drone;

[0010] The drone is controlled based on the horizontal line of sight angle information and the altitude information obtained by the drone's altitude sensor to achieve target tracking flight.

[0011] Optionally, the method of performing multiple target recognition based on the image information collected in real time by the camera and outputting the recognition results corresponding to the multiple targets includes:

[0012] If the target identified by the recognition algorithm meets the minimum number of recognition frames of the Sort multi-target tracking algorithm, the target is considered to be the identified target, and the recognition result is output through the recognition algorithm; if the target identified by the recognition algorithm does not meet the minimum number of recognition frames of the Sort multi-target tracking algorithm, the target is considered invalid; if the recognition algorithm does not identify the target, the UAV continues to fly according to the preset route.

[0013] Optionally, obtaining the predicted flight trajectory of the predicted target includes:

[0014] Based on the recognition result output by the recognition algorithm, determine whether the number of trackers in the Sort multi-target tracking algorithm is zero: if the number of trackers in the Sort multi-target tracking algorithm is zero, initialize the tracker; if the number of trackers in the Sort multi-target tracking algorithm is not zero, obtain the predicted flight trajectory of the predicted target, and output the prediction result of at least one tracker in the Sort multi-target tracking algorithm.

[0015] Optionally, the calculation of the intersection-over-union ratio between the predicted target and the identified target includes

[0016] Based on the recognition target frame output by the recognition result and the predicted target frame obtained by the Sort multi-target tracking algorithm, the intersection-over-union ratio between the predicted target frame and the recognition target frame is calculated;

[0017] Determine whether the categories of the predicted target and the recognized target are the same: if the categories of the predicted target and the recognized target are the same, the intersection-over-union ratio between the predicted target box and the recognized target box remains unchanged, and the intersection-over-union ratio matrix between the predicted target and the recognized target is constructed; if the categories of the predicted target and the recognized target are different, the intersection-over-union ratio between the predicted target box and the recognized target box is set to zero.

[0018] Optionally, the calculation of the intersection-over-union ratio between the predicted target and the identified target also includes

[0019] Compare the intersection-and-union ratio between the predicted target frame and the recognized target frame with a preset threshold: if the intersection-and-union ratio between the predicted target frame and the recognized target frame is greater than or equal to the preset threshold, keep the intersection-and-union ratio between the predicted target frame and the recognized target frame unchanged; if the intersection-and-union ratio between the predicted target frame and the recognized target frame is less than the first preset threshold, set the intersection-and-union ratio between the predicted target frame and the recognized target frame to zero.

[0020] Optionally, based on the intersection-over-union ratio between the predicted target frame and the recognized target frame, construct an intersection-over-union ratio matrix between the predicted target and the recognized target;

[0021] The Hungarian algorithm linear allocation is used to perform association matching between the predicted target and the identified target: if the predicted target matches the identified target, the tracker of the target is updated to obtain the tracked target; if the predicted target does not match the identified target, the tracker of the predicted target is initialized and the tracker of the disappeared target is eliminated.

[0022] Optionally, the calculation of the sight angle information of the tracking target relative to the drone in the horizontal and vertical directions includes:

[0023]

[0024] The horizontal sight angle information of the tracking target relative to the drone is A+D, and the vertical sight angle information of the tracking target relative to the drone is B+C.

[0025] Where (x, y) represents the pixel coordinates of the target; A represents the horizontal line of sight angle from the target to the camera; B represents the vertical line of sight angle from the target to the camera; θ represents the horizontal field of view angle of the camera; H represents the pixel height of the image taken by the camera; W represents the pixel width of the image taken by the camera; C represents the pitch angle of the camera gimbal; and D represents the yaw angle of the camera gimbal.

[0026] Optionally, the controlling of the drone based on the horizontal line of sight angle information and the altitude information obtained by the drone altitude sensor includes:

[0027] The yaw loop of the UAV is controlled based on the horizontal line of sight angle information;

[0028] The altitude information of the UAV is obtained from the altitude sensor, and the pitch loop control of the UAV is performed based on the altitude information.

[0029] Optionally, the method further includes

[0030] If the UAV changes the tracking target during the tracking flight, the horizontal and vertical sight angle information of the tracking target relative to the UAV will be recalculated;

[0031] If the drone loses the tracking target during the tracking flight, it will continue to fly according to the preset route.

[0032] Optionally, the method further includes

[0033] If a tracker in the Sort multi-target tracking algorithm fails to identify the target within a preset number of times, it will be eliminated as an invalid tracker.

[0034] The beneficial effects of the present invention are as follows:

[0035] The present invention adopts the recognition algorithm and the Sort multi-target tracking algorithm to identify and track targets, realizing the intelligent tracking task of a fixed-wing UAV for a specific target while ensuring reliability; the present invention can also automatically track the switching target when the fixed-wing UAV automatically switches targets without manual operation, thus solving the problem of only single target tracking in the prior art. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] The specific embodiments of the present invention will be further described in detail below with reference to the accompanying drawings.

[0037] Figure 1 A flowchart of the target tracking method for a fixed-wing UAV of the present invention is shown;

[0038] Figure 2 A flow chart showing the Sort multi-target tracking algorithm of the present invention is shown;

[0039] Figure 3 A graph showing the relationship between the predicted target frame and the recognized target frame of the present invention;

[0040] Figure 4 The figure shows the positional relationship between the fixed-wing UAV of the present invention and the tracking target. DETAILED DESCRIPTION

[0041] In order to more clearly illustrate the present invention, the present invention is further described below in conjunction with preferred embodiments and accompanying drawings. Similar components in the accompanying drawings are represented by the same reference numerals. It should be understood by those skilled in the art that the following detailed description is illustrative rather than restrictive and should not be used to limit the scope of protection of the present invention.

[0042] The present invention proposes a target tracking method for fixed-wing UAVs, which uses a recognition algorithm and a Sort multi-target recognition algorithm to identify and track multiple specific targets, thereby achieving the intelligent tracking task of the fixed-wing UAV for the specified target.

[0043] Convert the drone's flight mode to fixed-wing flight mode, perform multiple target recognition based on the image information collected by the camera in real time, and output the recognition results corresponding to the multiple targets;

[0044] Control the drone to take off. The tilt-rotor drone takes off in quad-rotor flight mode. When the drone reaches the preset altitude and the accelerator meets the minimum requirements for fixed-wing drones, the flight mode is switched to fixed-wing flight mode.

[0045] Based on the fixed-wing flight mode of the UAV, it performs track flight on the preset route according to GPS information. The camera gimbal of the fixed-wing UAV obtains video stream information in real time and performs multiple target recognition according to the recognition algorithm. If the minimum recognition frame number of the Sort multi-target tracking algorithm is met, the target is considered to be the recognition target, and the recognition result is output through the recognition algorithm. The image position of the next target at the next moment is estimated through the Sort multi-target tracking algorithm; if the minimum recognition frame number of the Sort multi-target tracking algorithm is not met, the target is considered invalid; if the recognition algorithm does not recognize the target, the UAV continues to fly according to the preset route.

[0046] Track the identified target according to the Sort multi-target tracking algorithm, obtain the predicted flight trajectory of the predicted target, calculate the intersection-over-union ratio between the predicted target and the identified target, and construct an intersection-over-union ratio matrix between the predicted target and the identified target based on the intersection-over-union ratio to obtain the tracked target;

[0047] Based on the recognition result output by the recognition algorithm, determine whether the number of trackers in the Sort multi-target tracking algorithm is zero: if the number of trackers in the Sort multi-target tracking algorithm is zero, initialize the tracker; if the number of trackers in the Sort multi-target tracking algorithm is not zero, obtain a predicted flight trajectory of the predicted target and output a prediction result of at least one tracker in the Sort multi-target tracking algorithm;

[0048] Based on the recognition target frame output by the recognition result and the predicted target frame obtained by the Sort multi-target tracking algorithm, the intersection-over-union ratio between the predicted target frame and the recognition target frame is calculated;

[0049] Determine whether the categories of the predicted target and the recognized target are the same: if the categories of the predicted target and the recognized target are the same, the intersection-and-union ratio between the predicted target box and the recognized target box remains unchanged, and the intersection-and-union ratio matrix between the predicted target and the recognized target is constructed; if the categories of the predicted target and the recognized target are different, the intersection-and-union ratio between the predicted target box and the recognized target box is set to zero;

[0050] Compare the intersection-and-union (IoU) between the predicted target frame and the recognized target frame with a preset threshold: if the IoU is greater than or equal to the preset threshold, keep the IoU unchanged; if the IoU is less than the preset threshold, set the IoU to zero.

[0051] Based on the intersection-over-union ratio between the predicted target frame and the recognized target frame, an intersection-over-union ratio matrix between the predicted target and the recognized target is constructed;

[0052] The Hungarian algorithm linear allocation is used to perform association matching between the predicted target and the identified target: if the predicted target matches the identified target, the tracker of the target is updated; if the predicted target does not match the identified target, the tracker of the predicted target is initialized and the tracker of the disappeared target is eliminated.

[0053] If a tracker in the Sort multi-target tracking algorithm fails to identify the target within a preset number of times, it will be eliminated as an invalid tracker.

[0054] Calculate the horizontal and vertical sight angle information of the tracking target relative to the drone;

[0055]

[0056] The horizontal sight angle information of the tracking target relative to the drone is A+D, and the vertical sight angle information of the tracking target relative to the drone is B+C.

[0057] Where (x, y) represents the pixel coordinates of the target; A represents the horizontal line of sight angle from the target to the camera; B represents the vertical line of sight angle from the target to the camera; θ represents the horizontal field of view angle of the camera; H represents the pixel height of the image taken by the camera; W represents the pixel width of the image taken by the camera; C represents the pitch angle of the camera gimbal; and D represents the yaw angle of the camera gimbal.

[0058] The drone is controlled based on the horizontal line of sight angle information and the altitude information obtained by the drone's altitude sensor, and the yaw loop of the drone is controlled based on the horizontal line of sight angle information; the altitude information of the drone is obtained from the altitude sensor, and the pitch loop of the drone is controlled based on the altitude information to achieve the drone's tracking flight of the target.

[0059] If the UAV changes the tracking target during the tracking flight, the line of sight angle information of the tracking target relative to the UAV in the horizontal and vertical directions will be recalculated; if the UAV loses the tracking target during the tracking flight, it will continue to fly according to the preset route.

[0060] In a specific embodiment, after the UAV power and avionics are connected, the UAV is automatically started through the onboard computer and the UAV flight control. When the onboard computer is started, the target recognition and tracking program is run. The flow chart of the entire UAV recognition and tracking system is as follows: Figure 1As shown. First, the drone is controlled to take off. The tilt-rotor drone takes off in quadrotor flight mode. After reaching a preset altitude and the airspeed meets the minimum fixed-wing flight limit, it switches to fixed-wing flight mode. After switching to fixed-wing flight mode, it performs track flight on the preset route based on GPS information. If the track flight ends, the fixed-wing drone automatically returns to land or performs the next mission. If the track flight has not ended, the fixed-wing drone's camera gimbal obtains video stream information on the preset route to identify multiple targets. If a target is found and meets the minimum recognition frame number of the Sort multi-target tracking algorithm, that is, the Sort multi-target tracking algorithm has locked onto the target for more than 3 frames, the recognition algorithm outputs the recognition result. This recognition result is the target's position in the image and the circumscribed rectangular box of the identified target. The recognition result is input into the Sort multi-target tracking algorithm, and the Sort multi-target tracking algorithm is run to estimate the target's image position at the next moment. If no target is found, the track flight continues.

[0061] Figure 2 This is a flowchart of the Sort multi-target tracking algorithm in this embodiment. After obtaining the recognition result, it is determined whether the number of trackers in the Sort multi-target tracking algorithm is zero: if the number of trackers in the Sort multi-target tracking algorithm is zero, the tracker is initialized, and the output result of the Sort multi-target tracking algorithm is the output result of the recognition algorithm; if the number of trackers in the Sort multi-target tracking algorithm is not zero, the prediction result of the predicted target tracked by each tracker in the current multi-target tracking algorithm is obtained.

[0062] Figure 3 The intersection over union (IOU) relationship diagram of the predicted target frame and the identified target frame is obtained by using the Sort multi-target tracking algorithm to obtain the bounding rectangle of the predicted target. The intersection over union (IOU) between the predicted target frame and the identified target frame is calculated based on the predicted target frame and the identified target frame. Among them, M is the predicted target box and N is the recognized target box. Get the overlap rate of the "predicted bounding box" and the "real bounding box". The ideal situation is that the predicted target box and the recognized target box completely overlap, that is, the intersection-over-union ratio is 1. Determine whether the categories of the predicted target and the recognized target are the same: if the categories of the predicted target and the recognized target are different, the IOU value is set to zero; if the categories of the predicted target and the recognized target are the same, the IOU value remains unchanged. Compare the value of the intersection-over-union ratio with the preset threshold: if the intersection-over-union ratio between the predicted target box and the recognized target box is greater than or equal to the preset threshold, keep the intersection-over-union ratio between the predicted target box and the recognized target box unchanged; if the intersection-over-union ratio between the predicted target box and the recognized target box is less than the first preset threshold, set the intersection-over-union ratio between the predicted target box and the recognized target box to zero. In this embodiment, the preset threshold is set to 0.3, that is, the intersection-over-union ratio between the predicted target frame and the recognized target frame is greater than or equal to 0.3, then the intersection-over-union ratio between the predicted target frame and the recognized target frame is kept unchanged; if the intersection-over-union ratio between the predicted target frame and the recognized target frame is less than 0.3, then the intersection-over-union ratio between the predicted target frame and the recognized target frame is set to zero. Construct the intersection-over-union ratio matrix between the predicted target and the recognized target. For example, at a certain frame moment, the number of predicted targets is 3 and the number of recognized targets is 4. The IOU matrix between the predicted target and the recognized target is shown in Table 1. The IOU matrix between the predicted target and the recognized target is used as the input of the Hungarian algorithm, and the Hungarian algorithm linear allocation is used to perform association matching between the predicted target and the recognized target: if the predicted target matches the recognized target, the predicted target and the recognized target are fused into a tracking target, and the tracker of the target is updated; if the predicted target does not match the recognized target, the predicted target and the recognized target are not fused, the tracker of the predicted target is initialized, and the tracker of the disappeared target is eliminated. If the Sort multi-target tracking algorithm fails to identify a target for three consecutive frames, the tracker of the target is judged as an invalid tracker and is eliminated.

[0063] Table 1 IOU matrix between predicted targets and recognized targets

[0064]

[0065] Figure 4 The position relationship between the fixed-wing UAV and the tracking target is calculated based on the tracking target's horizontal and vertical line of sight angle information relative to the UAV.

[0066]

[0067] The horizontal sight angle information of the tracking target relative to the drone is A+D, and the vertical sight angle information of the tracking target relative to the drone is B+C.

[0068] Where (x, y) represents the pixel coordinates of the target; A represents the horizontal line of sight angle from the target to the camera; B represents the vertical line of sight angle from the target to the camera; θ represents the horizontal field of view angle of the camera; H represents the pixel height of the image taken by the camera; W represents the pixel width of the image taken by the camera; C represents the pitch angle of the camera gimbal; and D represents the yaw angle of the camera gimbal.

[0069] according to Figure 4 The yaw loop of the UAV is controlled based on the line of sight angle information in the horizontal direction; the pitch loop of the UAV is controlled based on the altitude information obtained by the UAV's built-in altitude sensor, thereby realizing the UAV's tracking flight of the target.

[0070] It should be noted that if the UAV changes the tracking target during the tracking flight, the horizontal and vertical line of sight angle information and altitude information relative to the UAV will be repeatedly calculated to enable the UAV to track the tracking target after the target is changed.

[0071] If the drone loses the tracking target during tracking flight, it will continue to fly according to the preset route.

[0072] The mission can be ended during the entire drone's target tracking flight phase. After the mission is completed, the drone will automatically return to land or perform the next mission.

[0073] The present invention adopts the recognition algorithm and the Sort multi-target tracking algorithm to identify and track targets, realizing the intelligent tracking task of a fixed-wing UAV for a specific target while ensuring reliability; the present invention can also automatically track the switching target when the fixed-wing UAV automatically switches targets without manual operation, thus solving the problem of only single target tracking in the prior art.

[0074] In the description of the present invention, it should be noted that the orientation or positional relationship indicated by the terms "upper" and "lower" is based on the orientation or positional relationship shown in the accompanying drawings, and is only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation on the present invention. Unless otherwise clearly specified and limited, the terms "installed", "connected", and "connected" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection, or it can be indirectly connected through an intermediate medium, or it can be internal communication between two elements. For those of ordinary skill in the art, the specific meanings of the above terms in the present invention can be understood according to the specific circumstances.

[0075] It should also be noted that, in the description of the present invention, relational terms such as first and second, etc., are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article, or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or device. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of other identical elements in the process, method, article, or device comprising the element.

[0076] Obviously, the above embodiments of the present invention are merely examples for clearly illustrating the present invention, and are not limitations on the implementation methods of the present invention. For ordinary technicians in the relevant field, other different forms of changes or modifications can be made based on the above description. It is impossible to list all the implementation methods here. All obvious changes or modifications derived from the technical solution of the present invention are still within the scope of protection of the present invention.

Claims

1. A target tracking method for a fixed-wing UAV, characterized in that: The method comprises: Switch the drone to fixed-wing flight mode, perform multiple target recognition based on the image information collected by the camera in real time, and output the recognition results corresponding to the multiple targets; Track the identified target according to the Sort multi-target tracking algorithm, obtain the predicted flight trajectory of the predicted target, calculate the intersection-over-union ratio between the predicted target and the identified target, and construct an intersection-over-union ratio matrix between the predicted target and the identified target based on the intersection-over-union ratio to obtain the tracked target; Calculate the horizontal and vertical sight angle information of the tracking target relative to the UAV; The drone is controlled based on the horizontal line of sight angle information and the altitude information obtained by the drone's altitude sensor to achieve target tracking flight.

2. The method according to claim 1, characterized in that The method of performing multiple target recognition based on the image information collected by the camera in real time and outputting the recognition results corresponding to the multiple targets includes: If the target identified by the recognition algorithm meets the minimum recognition frame number of the Sort multi-target tracking algorithm, the target is considered to be the recognized target, and the recognition result is output through the recognition algorithm; If the target identified by the recognition algorithm does not meet the minimum number of recognition frames of the Sort multi-target tracking algorithm, the target is considered invalid; if the recognition algorithm does not identify the target, the drone continues to fly according to the preset route.

3. The method according to claim 2, characterized in that The predicted flight trajectory of the target is obtained by: Based on the recognition result output by the recognition algorithm, determine whether the number of trackers in the Sort multi-target tracking algorithm is zero: if the number of trackers in the Sort multi-target tracking algorithm is zero, initialize the tracker; if the number of trackers in the Sort multi-target tracking algorithm is not zero, obtain the predicted flight trajectory of the predicted target, and output the prediction result of at least one tracker in the Sort multi-target tracking algorithm.

4. The method according to claim 3, characterized in that The calculation of the intersection-over-union ratio between the predicted target and the identified target includes Based on the recognition target frame output by the recognition result and the predicted target frame obtained by the Sort multi-target tracking algorithm, the intersection-over-union ratio between the predicted target frame and the recognition target frame is calculated; Determine whether the categories of the predicted target and the recognized target are the same: if the categories of the predicted target and the recognized target are the same, the intersection-over-union ratio between the predicted target box and the recognized target box remains unchanged, and the intersection-over-union ratio matrix between the predicted target and the recognized target is constructed; if the categories of the predicted target and the recognized target are different, the intersection-over-union ratio between the predicted target box and the recognized target box is set to zero.

5. The method according to claim 4, characterized in that The calculation of the intersection-over-union ratio between the predicted target and the identified target also includes Comparing the intersection-and-union ratio between the predicted target frame and the recognized target frame with a preset threshold: if the intersection-and-union ratio between the predicted target frame and the recognized target frame is greater than or equal to the preset threshold, keeping the intersection-and-union ratio between the predicted target frame and the recognized target frame unchanged; If the intersection-and-union (IoU) ratio between the predicted target frame and the recognized target frame is less than a first preset threshold, the IoU ratio between the predicted target frame and the recognized target frame is set to zero.

6. The method according to claim 5, characterized in that Based on the intersection-over-union ratio between the predicted target frame and the recognized target frame, an intersection-over-union ratio matrix between the predicted target and the recognized target is constructed; The Hungarian algorithm linear allocation is used to perform association matching between the predicted target and the identified target: if the predicted target matches the identified target, the tracker of the target is updated to obtain the tracked target; if the predicted target does not match the identified target, the tracker of the predicted target is initialized and the tracker of the disappeared target is eliminated.

7. The method according to claim 1, characterized in that The calculated tracking target's horizontal and vertical sight angle information relative to the drone includes The horizontal sight angle information of the tracking target relative to the drone is A+D, and the vertical sight angle information of the tracking target relative to the drone is B+C. Where (x, y) represents the pixel coordinates of the target; A represents the horizontal line of sight angle from the target to the camera; B represents the vertical line of sight angle from the target to the camera; θ represents the horizontal field of view angle of the camera; H represents the pixel height of the image taken by the camera; W represents the pixel width of the image taken by the camera; C represents the pitch angle of the camera gimbal; and D represents the yaw angle of the camera gimbal.

8. The method according to claim 1, characterized in that The control of the UAV based on the horizontal line of sight angle information and the altitude information obtained by the UAV altitude sensor includes: The yaw loop of the UAV is controlled based on the horizontal line of sight angle information; The altitude information of the UAV is obtained from the altitude sensor, and the pitch loop control of the UAV is performed based on the altitude information.

9. The method according to claim 1, characterized in that The method further includes If the UAV changes the tracking target during the tracking flight, the horizontal and vertical sight angle information of the tracking target relative to the UAV will be recalculated; If the drone loses the tracking target during the tracking flight, it will continue to fly according to the preset route.

10. The method according to claim 3, characterized in that The method further includes If a tracker in the Sort multi-target tracking algorithm fails to identify the target within a preset number of times, it will be eliminated as an invalid tracker.

Images