A reliable target tracking method for fast convergence process

Abstract

The present invention discloses a reliable target tracking method and system in a rapid convergence process. The method constructs a tracking model of the rapid convergence process between a drone and a target, uses the tracking model to obtain a rough estimate of the target scale and position, and fine-tunes the rough estimate of the target scale to obtain the final target scale and position, thereby realizing reliable tracking of the target in the rapid convergence process, and enabling the drone to accurately perform tasks. The method provided by the present invention solves the problem that when a drone quickly approaches a target, the target scale rapidly expands in the drone's field of view, causing tracking failure, and can accurately track the target and perform landing or strike tasks. The method provided by the present invention is flexible to use, has strong versatility, and is fast to calculate.

Description

Technical Field

[0001] The invention relates to the field of target tracking, and in particular to a method for reliable tracking of a target in a rapid intersection process. Background Art

[0002] Drones have the advantages of small size, light weight, few restrictions on take-off and landing conditions, flexibility, and simple operation. As drones can perform many tasks that human-controlled drones cannot, they have gradually become an important focus of various countries and even the world in the aviation field. Today, they have been widely used in many fields, such as military operations, daily life, and civilian fields.

[0003] In recent years, the intelligence of drones has developed rapidly, and one of the important aspects is the autonomous tracking of targets by drones. Based on target tracking, drones can autonomously land on moving vehicles or autonomously attack targets in a suicide attack. However, when a drone quickly approaches a target, the scale of the target will expand rapidly in the drone's field of view. For visual technology, scale is a very important issue. If the target scale is not predicted accurately, the tracking task will fail.

[0004] Most correlation filter trackers use the method in the literature [A Scale Adaptive Kernel Correlation Filter Tracker with Feature Integration [C] / / European Conference on Computer Vision. Springer, Cham, 2014.] to filter multiple predicted target scales and finally obtain the scale with the highest score as the target scale.

[0005] SiamRPN [Li B, Yan J, Wu W, et al. High performance visual tracking with siamese region proposal network [C] / / Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2018: 8971-8980.] uses a dual-branch twin network method, in which one branch is used to roughly locate the target, and the other branch uses a regression method to fine-tune the target position and estimate the change in the target scale.

[0006] The above methods all make fine adjustments based on the target scale in the previous frame, and can only be used when the target scale changes slightly and the scale change process is slow. When the drone approaches the target quickly, the target scale expands rapidly in the drone's field of view and finally fills the entire field of view. The above methods cannot solve the problem of rapid scale expansion well. Summary of the invention

[0007] In order to overcome the above problems, the inventors have conducted intensive research and designed a reliable target tracking method and system for a rapid convergence process. The method constructs a tracking model for the rapid convergence process between the drone and the target, uses the tracking model to obtain a rough estimate of the target scale and position, and fine-tunes the rough estimate of the target scale to obtain the final target scale and position, thereby achieving reliable tracking of the target during the rapid convergence process, so that the drone can accurately perform tasks. The method provided by the present invention solves the problem that when the drone quickly approaches the target, the target scale expands rapidly in the drone's field of view, causing tracking failure, and can accurately track the target and perform landing or strike tasks. The method provided by the present invention is flexible to use, has strong versatility, and is fast to calculate, thereby completing the present invention.

[0008] The object of the present invention is to provide a method for reliable tracking of a target in a fast convergence process, the method comprising:

[0009] Construct a tracking model suitable for the rapid convergence process between the UAV and the target;

[0010] The tracking model is used to realize the tracking of the target by the UAV.

[0011] The construction of the tracking model includes:

[0012] Initialization phase;

[0013] Determine the target position and scale according to the tracking stage of the i-th frame;

[0014] Final tracking stage.

[0015] The initialization stage includes: determining the parameter information during initialization, the parameter information includes the height H of the drone, 0 , the yaw angle of the target in the UAV's field of view ψ 0 and pitch angle

[0016] The parameter information also includes the target scale l 0 , w 0 and h 0 Respectively represent the width and height of the target in the pixel coordinate system;

[0017] The initialization stage also includes: determining the distance between the drone and the target H 0 represents the height of the drone at initialization, θ 0 Indicates the pitch angle of the pod.

[0018] The i-th frame tracking stage includes: determining the roughly estimated target position and scale; fine-tuning the roughly estimated target position and scale to obtain the final target position and scale,

[0019] Preferably, the roughly estimated target position and scale are determined by a tracking algorithm, and more preferably, the tracking algorithm is a SiamRPN tracking algorithm.

[0020] Determining a rough estimate of the target location and scale involves:

[0021] Get the height H of the drone at the i-th frame i 、The pitch angle of the target in the field of view θ i ,

[0022] Get the distance d between the drone and the target i ,

[0023] Make a rough estimate of the target scale to obtain a rough estimated target scale.

[0024] l e = l i-1 ·d i-1 / d i

[0025] Among them, l e represents the rough estimated target scale, l i-1 Indicates the target scale obtained in the previous frame.

[0026] According to the rough estimated target scale of the i-th frame, the search area is obtained.

[0027] The center of the search area is the position of the target in the previous frame.

[0028] Preferably, the size of the search area is the roughly estimated target scale l e 2 to 3 times, such as 2.5 times.

[0029] The method according to claim 2 is characterized in that the final target position and scale are obtained according to formula (1.1),

[0030] (x i ,y i ,w i ,h i )=f(x i-1 ,y i-1 ,l e ) (1.2)

[0031] In formula (1.1), (x i-1 ,y i-1 ) represents the target position of the i-1th frame, l e represents the rough estimated target scale of the i-th frame, f() is the SiamRPN algorithm; (x i ,y i ,w i ,h i ) represents the abscissa, ordinate, width, and height of the target in the i-th frame output by the algorithm in the image coordinate system.

[0032] The final tracking stage includes: the UAV performs the task according to the final target position and scale obtained;

[0033] Tracking the target using the tracking model includes: determining parameter information, obtaining a rough estimate of the target scale and position, obtaining a regression output result, determining the final target position and scale, and when the target fills the field of view, entering the final tracking stage, and the UAV performs the task.

[0034] The present invention also provides a reliable target tracking system for a rapid convergence process, the system comprising:

[0035] An acquisition module, used to acquire video of the target in the field of view;

[0036] A solution module is used to obtain a rough estimate of the target position and scale, and to fine-tune the rough estimate of the target position and scale to obtain the final target position and scale;

[0037] The control module is used to control the drone to perform tasks.

[0038] The solving module performs solving according to the tracking model, wherein the tracking model includes a tracking model positioning branch unit and a tracking model regression branch unit, wherein the tracking model positioning branch unit obtains a roughly estimated target position and scale, and the tracking model regression branch unit obtains a regression output result.

[0039] The beneficial effects of the present invention are:

[0040] (1) The method provided by the present invention solves the problem that when a UAV approaches a target, the target scale expands rapidly, causing tracking failure;

[0041] (2) The method provided by the present invention can be used in combination with any tracking algorithm, and is flexible, versatile, and widely applicable;

[0042] (3) The method provided by the present invention is fast in calculation, does not add too much computational burden to the tracking algorithm, and can be run in real time on an onboard computer. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] Figure 1 An image diagram of a drone in an initial state in an embodiment of the present invention is shown;

[0044] Figure 2 , Figure 3 , Figure 4 and Figure 5 The drone images obtained in the process of the drone gradually approaching the target in the embodiments of the present invention are respectively shown. DETAILED DESCRIPTION

[0045] The present invention is further described in detail below through the accompanying drawings and preferred embodiments. Through these descriptions, the characteristics and advantages of the present invention will become more clear and distinct.

[0046] According to the present invention, a method for reliable tracking of a target in a fast convergence process is provided, the method comprising:

[0047] Construct a tracking model suitable for the rapid convergence process between the UAV and the target;

[0048] The tracking model is used to realize the tracking of the target by the UAV.

[0049] In the present invention, the rapid intersection process between the UAV and the target refers to the process in which the scale of the target rapidly expands in the field of view of the UAV when the UAV is rapidly approaching the target. In this process, if the scale of the target changes greatly and the size changes rapidly, it is easy to cause tracking failure. The present invention overcomes the above problems by fine-tuning the search area and the scale of the target to obtain the final target position and scale.

[0050] According to the present invention, the construction of a tracking model suitable for the rapid convergence process between the drone and the target includes:

[0051] Initialization phase;

[0052] The i-th frame tracking stage;

[0053] Final tracking stage.

[0054] According to the present invention, the initialization stage includes: determining the parameter information during initialization, wherein the parameter information includes the altitude coordinate H of the drone during initialization. 0 , the yaw angle of the target in the UAV's field of view ψ 0 and pitch angle

[0055] It can be obtained through an airborne optoelectronic pod, and the optoelectronic pod is preferably a Guanzhong EOT-90A3 optoelectronic pod.

[0056] The parameter information also includes the target scale l 0 , w0 and h 0 are the width and height of the target in pixel coordinates, respectively.

[0057] The initialization stage also includes: determining the distance between the drone and the target H 0 represents the height of the drone at initialization, θ 0 The pitch angle of the pod is 0 degrees vertically downward and 180 degrees vertically upward. When the UAV observes and strikes ground targets, θ 0 Usually 10-80 degrees.

[0058] According to the present invention, the tracking stage comprises determining the target position and scale according to the i-th frame tracking stage, wherein i is a positive integer greater than or equal to 1.

[0059] According to the present invention, the i-th frame tracking stage includes: determining a roughly estimated target position and scale; fine-tuning the roughly estimated target position and scale to obtain a final target position and scale,

[0060] Preferably, the roughly estimated target position and scale are determined by a tracking algorithm, and more preferably, the tracking algorithm is a SiamRPN tracking algorithm.

[0061] According to a preferred embodiment of the present invention, determining a roughly estimated target position and scale includes:

[0062] Determining a rough estimate of the target location and scale involves:

[0063] Get the height H of the drone at the i-th frame i 、The pitch angle of the target in the field of view θ i ,

[0064] Get the distance d between the drone and the target i ,

[0065] Make a rough estimate of the target scale to obtain a rough estimated target scale.

[0066] l e = l i-1 ·d i-1 / d i

[0067] Among them, l e represents the rough estimated target scale, l i-1 represents the target scale obtained in the previous frame, i.e., the i-1th frame, d i-1 Indicates the distance between the drone and the target obtained at the i-1th frame.

[0068] According to the present invention, the rough estimated target scale of the i-th frame is cropped on the current frame (i.e., the i-th frame) image to obtain a search area, the center of which is the position of the target in the previous frame (i.e., the i-1-th frame), 2.5 times l e Then, according to formula (1.1), we can obtain the final target position and scale.

[0069] (x i ,y i ,w i ,h i )=f(x i-1 ,y i-1 ,l e ) (1.3)

[0070] In formula (1.1), (x i-1 ,y i-1 ) represents the target position of the i-1th frame, l e represents the rough estimated target scale of the i-th frame, and f() represents the SiamRPN algorithm. (x i ,y i ,w i ,h i )x i Represents the horizontal coordinate of the target in the i-th frame output by the algorithm in the image coordinate system; y i represents the ordinate of the target in the i-th frame output by the algorithm in the image coordinate system; w i Indicates the width of the target in the i-th frame output by the algorithm in the image coordinate system; h i Indicates the height of the target in the i-th frame output by the algorithm in the image coordinate system.

[0071] In the present invention, in the tracking phase, after determining the final target position and scale, the UAV quickly approaches the target and enters the final tracking phase. The initiative of the UAV is utilized to perform rapid maneuvers in the final tracking phase to execute the task.

[0072] According to the present invention, in the final tracking stage, when the target fills the field of view (i.e., l 0 >720, 720 is the vertical resolution of the image), it is determined to enter the final tracking stage,

[0073] If the UAV mission is a landing mission, a pulse command for horizontal forward maneuver at full throttle is output, and then the engine is quickly shut down, and the UAV lands smoothly on the target;

[0074] If the drone's mission is a strike mission, it will crash into the target at full throttle to carry out the mission.

[0075] According to the present invention, in the process of rapid intersection between the UAV and the target, tracking the target using the tracking model includes: determining parameter information, obtaining a rough estimate of the target position and scale, obtaining a regression output result, determining the final target position and scale, and when the target fills the field of view, entering the final tracking stage, and the UAV performs the task.

[0076] According to the present invention, the determining parameter information includes: determining the height of the drone, the yaw angle of view and the pitch angle of view of the target in the field of view; and obtaining the distance between the drone and the target.

[0077] According to the present invention, based on the parameter information, a roughly estimated target position and scale are determined, a regression output result is obtained, and the final target position and scale are obtained according to formula (1.1).

[0078] According to the present invention, in the final tracking stage, when the target fills the field of view, it is determined to enter the final tracking stage.

[0079] If the UAV mission is a landing mission, a pulse command for horizontal forward maneuver at full throttle is output, and then the engine is quickly shut down, and the UAV lands smoothly on the target;

[0080] If the drone's mission is a strike mission, it will crash into the target at full throttle to carry out the mission.

[0081] The reliable target tracking method in the rapid intersection process provided by the present invention roughly estimates the target scale and position, determines the search area according to the roughly estimated target scale, locates the target, and fine-tunes the target scale to obtain the final target position and scale. When the target fills the field of view, the UAV performs the task. The method of the present invention is suitable for accurately tracking the target when the UAV quickly approaches the target, and solves the problem of rapid expansion of the target scale and causing tracking failure. The method adopted by the method can be used in combination with any tracking algorithm, is flexible and universal, and does not increase excessive calculation burden.

[0082] The present invention also provides a reliable target tracking system for a rapid convergence process, the system comprising:

[0083] The acquisition module is used to collect real-time video and parameter information of the drone and the target;

[0084] A solution module is used to obtain a rough estimate of the target position and scale, and to fine-tune the rough estimate of the target position and scale to obtain the final target position and scale;

[0085] The control module is used to control the drone to perform tasks.

[0086] According to a preferred embodiment of the present invention, the acquisition module is used to acquire real-time video and images of the drone in the field of view, such as determining an image of a target in the i-th frame of the target.

[0087] According to the present invention, the parameter information of the drone and the target includes the altitude of the drone, the yaw angle and the pitch angle of the target in the field of view, and the distance between the drone and the target.

[0088] According to the present invention, the solution module performs solution according to the tracking model, the tracking model includes a tracking model positioning branch unit and a tracking model regression branch unit, the solution module includes an onboard computer, and preferably the onboard computer includes an NVIDIA TX2 chip.

[0089] Preferably, the tracking model includes a SiamRPN tracking algorithm.

[0090] According to the present invention, the tracking model positioning branch unit is used to obtain a rough estimate of the target scale and position. Specifically, based on the parameters of the drone in the i-th frame and the distance between the drone and the target, a rough estimate of the target scale and a rough estimate of the target position are obtained.

[0091] According to the present invention, the tracking model obtains the fine-tuned target position and scale.

[0092] Example

[0093] The drone was equipped with an EOT-90A3 optoelectronic pod and conducted a simulated attack on a car on the ground. The onboard computer used an NVIDIA TX2 chip and the tracking model used the SiamRPN tracking algorithm. The drone's mission was to land on top of the van.

[0094] First, determine the tracking target and make sure the target is in the UAV's field of view.

[0095] In the initialization phase, the target video or photo is collected in real time to obtain the parameter information of the drone, including the altitude of the drone, the pitch angle of the video pod, and the distance between the drone and the target. Figure 1 As shown in , the altitude of the drone when it takes off is 75m. At this time, the drone is 122 meters above sea level, the height of the drone from the ground is 47 meters, and the pitch angle of the pod is 64 degrees.

[0096] In the tracking stage, the rough estimated target position and scale are obtained based on the parameter information of the UAV in the i-th frame and the distance between the UAV and the target;

[0097] Specifically, it includes obtaining the height H of the drone at the i-th frame i 、The pitch angle of the target in the field of view θ i ,

[0098] Get the distance d between the drone and the target i ,

[0099] Make a rough estimate of the target scale to obtain a rough estimated target scale.

[0100] l e = l i-1 ·d i-1 / d i

[0101] Among them, l e represents the rough estimated target scale, l i-1 represents the target scale obtained at the i-1th frame, d i-1 Indicates the distance between the drone and the target obtained at the i-1th frame;

[0102] Crop the current frame image according to the roughly estimated target scale, and set the search area to 2.5 times the roughly estimated target scale.

[0103] The target is located and the roughly estimated target scale is fine-tuned to obtain the final target position and scale.

[0104] When the target fills the field of view, the drone is very close to the target, and after approaching the target, the pilot controls the drone to land on the target.

[0105] During the tracking process, as the UAV gradually approaches the target, the image information obtained is gradually updated, that is, the following Figure 2 , Figure 3 , Figure 4 and Figure 5 The images are arranged in chronological order as shown in the figure. The tracking box is a green box. Figures 2 to 5 It can be seen that the tracking frame in this tracking scheme gradually increases as the UAV approaches the target, which can effectively solve the scale expansion problem when the UAV approaches the target.

[0106] Then use the traditional original SiamRPN algorithm to track the same target, and the tracking frame obtained is as follows Figure 2 , Figure 3 , Figure 4 and Figure 5 As shown in the yellow box in ; the original SiamRPN algorithm fails to track because it cannot adapt to scale expansion.

[0107] The present invention is described in detail above in combination with preferred embodiments and exemplary examples. However, it should be noted that these specific embodiments are only illustrative explanations of the present invention and do not constitute any limitation on the protection scope of the present invention. Without exceeding the spirit and protection scope of the present invention, various improvements, equivalent substitutions or modifications may be made to the technical content of the present invention and its embodiments, which all fall within the protection scope of the present invention. The protection scope of the present invention shall be subject to the attached claims.

Claims

1. A reliable target tracking method for fast convergence process, It is characterized in that The method comprises: Construct a tracking model suitable for the rapid convergence process between the UAV and the target; Using the tracking model to realize the tracking of the target by the UAV; The tracking model for the rapid convergence process between the UAV and the target includes: Initialization phase; A tracking phase, wherein the tracking phase includes determining the target position and scale according to the i-th frame tracking phase; Final follow-up phase; The initialization stage includes: determining the parameter information during initialization, the parameter information includes the height H of the drone, 0 , the yaw angle of the target in the UAV's field of view ψ 0 and pitch angle The parameter information also includes the target scale l 0 , w 0 and h 0 Respectively represent the width and height of the target in the pixel coordinate system, The initialization stage also includes: determining the distance between the drone and the target H 0 represents the height of the drone at initialization, θ 0 Indicates the pitch angle of the pod; The i-th frame tracking stage includes: determining the roughly estimated target position and scale; fine-tuning the roughly estimated target position and scale to obtain the final target position and scale, Determine the roughly estimated target position and scale by a tracking algorithm, wherein the tracking algorithm is a SiamRPN tracking algorithm; Determining a rough estimate of the target location and scale involves: Get the height H of the drone at the i-th frame i 、The pitch angle θ of the target in the field of view i , Get the distance d between the drone and the target i , Make a rough estimate of the target scale to obtain a rough estimated target scale. L e l i-1 ·d i-1 / d i Among them, l e represents the rough estimated target scale, l i-1 Indicates the target scale obtained in the previous frame; According to the rough estimated target scale of the i-th frame, the search area is obtained. The center of the search area is the position of the target in the previous frame. The size of the search area is 2.5 times the roughly estimated target scale.

2. The method according to claim 1, It is characterized in that According to formula (1.1), the final target position and scale are obtained. (x i ,y i ,w i ,h i )=f(x i-1 ,y i-1 ,l e )(1.1) In formula (1.1), (x i-1 ,y i-1 ) represents the target position of the i-1th frame, l e represents the rough estimated target scale of the i-th frame, f() is the SiamRPN algorithm; (x i ,y i ,w i ,h i ) represents the abscissa, ordinate, width, and height of the target in the i-th frame output by the algorithm in the image coordinate system.

3. The method according to claim 1, It is characterized in that The final tracking stage includes: the UAV performs the task according to the final target position and scale obtained; Tracking the target using the tracking model includes: determining parameter information, obtaining a rough estimate of the target scale and position, obtaining a regression output result, determining the final target position and scale, and when the target fills the field of view, entering the final tracking stage, and the UAV performs the task.

Images