A method and system for following a drone based on scale and anti-occlusion optimization

Abstract

The application provides a UAV following method and system based on scale and anti-occlusion optimization, and belongs to the technical field of target tracking; the problems of poor robustness and low applicability of the existing UAV target following system based on machine vision are solved; the method comprises the following steps: a UAV obtains a video of a tracking target, a ground station takes a target of interest in an initial frame as initial target position information by manually framing the target, and initializes a tracker, wherein the tracker is a computer program of the UAV following algorithm based on scale and anti-occlusion optimization which has been optimized; the ground station converts the position and scale information of the target output by the tracker into a coordinate system, converts the motion information of the target into flight motion information in the world coordinate system of the UAV, and realizes the flight motion of the UAV following the target; the next frame of image is processed, and the above steps are repeated until the UAV following ends; and the application is applied to UAV following.

Description

TECHNICAL FIELD

[0001] The application provides a UAV following method and system based on scale and anti-occlusion optimization, and belongs to the technical field of target tracking and UAV target following. BACKGROUND

[0002] With the development of computer vision, target detection and target tracking technology have made great progress, and applying machine vision technology to realize autonomous target following of the unmanned platform has become a hot research.

[0003] Due to the small computing capacity of the UAV-mounted chip and the inability to carry GPU and other acceleration computing devices, the target tracking algorithm designed to be carried has a low complexity requirement. During the target tracking process, there are problems of target occlusion, deformation and scale change, which affect the performance of target tracking.

[0004] Liu et al. used TLD combined with KCF as a target tracking algorithm to build a UAV target following system, which has remarkable effect, but poor adaptability to target scale change. In 2022, Rao Liuzhong et al. built a target following system for an unmanned ship by using a monocular camera, obtained the target color space by using the camera, used HSV color change and binary processing to obtain the position information of the target, and used the deviation angle and deviation distance to control the movement of the unmanned ship to realize tracking. However, the tracking method used has poor robustness, and when an interfering object with the same color space as the tracking target appears, the target is easily lost. SUMMARY

[0005] The application proposes a UAV following method and system based on scale and anti-occlusion optimization to solve the problems of poor robustness and low applicability of existing unmanned platform target following systems based on machine vision.

[0006] To solve the above technical problems, the application adopts the technical scheme of a UAV following method based on scale and anti-occlusion optimization, comprising the following steps:

[0007] S1: obtaining a video of a tracking target by a UAV, a ground workstation receiving the video and manually framing an initial target of interest in an initial frame as initial target position information to initialize a tracker, wherein the tracker is a computer program of an optimized UAV following algorithm based on scale and anti-occlusion optimization;

[0008] The optimization process of the UAV following algorithm based on scale and anti-occlusion optimization is as follows:

[0009] S1.1: Estimate target position by using KCF filter: set target search area, extract hog features of initial target, and obtain target template by multiplying hog features with Hanning window, calculate Gaussian matrix graph, and determine possible position of target in next frame by using kernel correlation filtering algorithm;

[0010] S1.2: Estimate target scale: estimate target scale in different scales based on target search area in previous frame;

[0011] S1.3: Calculate output response of target according to estimated target scale and target position;

[0012] S1.4: Solve confidence of target output response, update target template when target is in high confidence, and stop target template update when target is in low confidence, and find back tracking target by using anti-occlusion mechanism;

[0013] S2: Ground station converts position and scale information of target output by tracker from image coordinate system to world coordinate system, converts motion information of target in image into flight motion information in world coordinate system of unmanned aerial vehicle, and transmits control information to unmanned aerial vehicle by using local area network to realize flight of unmanned aerial vehicle following target;

[0014] S3: Process next frame of image, and repeat steps S1-S2 until unmanned aerial vehicle following ends.

[0015] In step S1.2, image scale pyramid method is used to estimate target in multiple scales, and the following formula is used to obtain optimal scale:

[0016] F_S max =maxf(z s )=max(response_S t );

[0017] In the formula, F_S max represents maximum response value in scale transformation, response_S t is feature response estimated in different scales in t-th frame, and the formula is used to solve each scale to obtain optimal target scale estimation.

[0018] In step S1.4, the following formula is used to calculate confidence of target output response:

[0019]

[0020] In the formula, F max and F min respectively represent maximum value and minimum value of target response, and F x,y represents value of target in x row and y column in response matrix.

[0021] The confidence evaluation threshold is:

[0022]

[0023] In the above formula: APCE mean represents the mean of response fluctuation history, F mean represents the mean of response peak value history, when the target confidence value of the tth frame is greater than τ2, it is considered that the target is in a high confidence state, and when the target confidence value of the tth frame is less than τ1, it is considered that the target is in a low confidence state.

[0024] The anti-occlusion mechanism in the step S1.4 refers to enabling Kalman filtering to optimally estimate the target position information based on the target information when the target is in a low confidence state, so as to make the tracker recover and track the target again. The specific steps are as follows:

[0025] Suppose the centroid position information of the target in the tth frame is X t (x, y), corresponding to the pos of the kernel correlation filter, then the estimation of the target position in the t+1th frame is:

[0026]

[0027] In the above formula: X t is the target state information in the tth frame, A is the state transition matrix, Q t is the current frame system noise covariance, P t+1 / t is the predicted estimation covariance matrix, B is the control matrix, u is the current frame input matrix, P t is the estimation covariance matrix of the current frame, X t+1 / t is the predicted target motion information;

[0028] By default, there is no input influence in the tracking process, so the value of B is zero. In the update stage, the optimal solution of the target is obtained by using the following formula based on the current system prediction result and the measurement value of the current state:

[0029]

[0030] In the above formula: Z t+1 is the observation value of the target position, K is the Kalman gain coefficient, H is the observation matrix, and I is the unit matrix;

[0031] K=P t1 / t H T (HP t+1 / t H T +R) -1 ;

[0032] In the above formula: R is the measurement noise covariance matrix;

[0033] By iterating the above formula, the optimal estimation of the target position is completed, and the target template of the tracker is updated by using the following formula:

[0034]

[0035] In the above formula: α t is the current frame tracker parameter, x t is the current frame tracker sample template, and θ is a linear interpolation factor.

[0036] The step S2 specifically comprises:

[0037] An image coordinate system is established, and an image coordinate system origin (x, y) is set, x = 0.5Wide, y = 0.5Hight, wherein Wide is the width of the image window, Hight is the height of the image window, the area of the initial frame target frame is S0, S t represents the area of the current frame target frame, and the current frame target centroid is (x t ,y t );

[0038] The obtained target image area information is converted into a forward or backward motion control of the unmanned aerial vehicle in the world coordinate system, and the calculation formula is as follows:

[0039]

[0040] Wherein c represents the throttle channel lever amount of the unmanned aerial vehicle;

[0041] The above formula indicates that when S t >1.1S0, the throttle channel lever amount of the unmanned aerial vehicle is controlled to be -20, and the unmanned aerial vehicle is realized to make a backward motion at 20 cm / s in the world coordinate system; when S t <0.9S0, the throttle channel lever amount of the unmanned aerial vehicle is controlled to be +20, and the unmanned aerial vehicle is realized to make a forward motion at 20 cm / s in the world coordinate system.

[0042] The obtained target image position information is converted into a left or right rotation control of the unmanned aerial vehicle in the world coordinate system, and the calculation formula is as follows:

[0043]

[0044] Wherein d represents the rotation channel lever amount of the unmanned aerial vehicle, and 96 represents an image pixel threshold, which is in pixels;

[0045] The above formula indicates that when x t -x>96, the rotation channel lever amount of the unmanned aerial vehicle is controlled to be +20, and the unmanned aerial vehicle is realized to rotate to the right at 20 rad / s in the world coordinate system; and when x-x tWhen y>72, the throttle channel stick amount of the UAV is controlled as +20, and the UAV is made to ascend at a speed of 20 cm / s in the world coordinate system.

[0046] The acquired target image position information is converted into UAV ascending or descending motion control in the world coordinate system, and the calculation formula is as follows:

[0047]

[0048] Where b represents the pitch channel stick amount of the UAV, and 72 represents an image pixel threshold value, which is in pixels.

[0049] The above formula indicates that when y t When y>72, the throttle channel stick amount of the UAV is controlled as +20, and the UAV is made to ascend at a speed of 20 cm / s in the world coordinate system. t When y>72, the throttle channel stick amount of the UAV is controlled as +20, and the UAV is made to ascend at a speed of 20 cm / s in the world coordinate system.

[0050] Finally, the channel stick amount [b, c, d] is transmitted from the ground station to the UAV.

[0051] A UAV following system based on scale and anti-occlusion optimization comprises a UAV and a ground station, the UAV is provided with a wireless Wifi module and a high-definition visual image sensor, the ground station and the UAV are connected through the Wifi module for data transmission, the ground station receives video returned by the UAV in real time, the ground station is provided with a memory and a processor, the memory stores a computer program of a UAV following method based on scale and anti-occlusion optimization, and the processor is used for executing the computer program and transmitting a control signal to the UAV to realize target following.

[0052] The UAV following method and system based on scale and anti-occlusion optimization provided by the application take a target tracking method based on scale and anti-occlusion optimization as a technical core, overcome the low target tracking performance in a shielding scene and a scale change scene, and complete real-time following flight movement of the UAV on an interesting target by constructing a mapping relationship between a world coordinate system and an image coordinate system, so that long-term stable real-time following movement of the interesting target can be effectively realized.

[0053] This invention utilizes the Kalman filtering method and confidence evaluation mechanism to construct an anti-occlusion optimization strategy, and uses a scale pyramid to construct a scale adaptive strategy to optimize the kernel correlation filtering algorithm, thereby improving the target tracking performance. Combined with the image-world coordinate system transformation mechanism, it constructs a UAV following system that can still have excellent tracking performance in occlusion scenarios and scale change scenarios during target movement, providing a new and effective solution for target following systems. Attached Figure Description

[0054] The present invention will be further described below with reference to the accompanying drawings:

[0055] Figure 1 This is a schematic diagram of the system structure of the present invention;

[0056] Figure 2 This is a flowchart of the method of the present invention;

[0057] Figure 3 This is a schematic diagram illustrating the objective of this invention: achieving UAV control through coordinate system transformation. Figure 3 (a) is a schematic diagram of the UAV rotation and pitch control achieved by the present invention through target coordinate system transformation. Figure 3 (b) is a schematic diagram of the present invention's objective of achieving drone throttle control through coordinate system transformation;

[0058] Figure 4 This is a graph showing the confidence level and peak value variation of the method of the present invention when tracking a target in an occluded scene;

[0059] Figure 5 The graph shows the tracking accuracy curves of the method of this invention and other algorithms under all disturbances.

[0060] Figure 6 The graph shows the tracking success rate curves of the method of this invention and other algorithms under all interferences.

[0061] Figure 7 The following experimental results are shown to verify the method of the present invention. Detailed Implementation

[0062] like Figures 1 to 7 As shown, this invention provides a scale- and anti-occlusion optimized UAV following system, including a UAV and a ground workstation. The ground workstation and the UAV are connected via a local area network (LAN). The ground workstation receives video transmitted back from the UAV in real time and uses a scale- and anti-occlusion optimized target tracking method on the ground workstation to accurately and stably track the target. The ground workstation performs the conversion from the image coordinate system to the world coordinate system and transmits the control information to the UAV via the LAN to follow the target's flight motion.

[0063] Specifically, a wireless Wifi module and a high-definition visual image sensor are loaded on the unmanned aerial vehicle, the ground station is connected with the unmanned aerial vehicle through the Wifi module, and data transmission is performed.

[0064] The application further provides an unmanned aerial vehicle following method based on scale and anti-occlusion optimization.

[0065] Step 1: data connection between the ground station and the unmanned aerial vehicle is performed through a local area network;

[0066] A wireless Wifi module and a high-definition visual image sensor are loaded on the unmanned aerial vehicle, the ground station is connected with the unmanned aerial vehicle through the Wifi module, and data transmission is performed.

[0067] Step 2: the ground station receives the video returned by the unmanned aerial vehicle end in real time, and a target tracking method based on scale and anti-occlusion optimization is run on the ground station, as shown in a tracking algorithm flow chart Figure 2 .

[0068] On the ground station, the initial frame of the received video is manually framed to select a target of interest as initial target position information, and the initial target position information is initialized on the tracker of the target tracking algorithm based on scale and anti-occlusion optimization.

[0069] The target candidate region is set to 1.8 times the initial target frame, the hog feature of the initial target is extracted, the target template is obtained by multiplying the hog feature with a Hanning window, and the Gaussian matrix graph is calculated accordingly. The kernel correlation filtering algorithm is used to determine the possible position of the target in the next frame. The target search region (patch) is calculated by the formula The target is extracted by the formula The target feature of the next frame is calculated by the formula F max The output response distribution graph is obtained by the formula F

[0070] Based on the target search region of the current frame, different scale estimates of the target in the next frame are performed. The output response of each scale estimate is solved, and the maximum output response is the optimal scale estimate of the target. The specific process is as follows:

[0071] Let the scale estimate be S, and the expression is as follows:

[0072]

[0073] In the above formula, N represents the number of layers of scale estimates, that is, N layers of different scale estimates are performed on the original sample; a is a scale increment factor; denotes down-rounding; in the above formula, negative value denotes scale reduction, and positive value denotes scale enlargement;

[0074] Assume that the target scale of the t-1th frame is S t-1 = w t-1 × h t-1 , wherein w denotes the image window width, h denotes the image window width, and the target of the t-1th frame is scaled by a scale S t = sw t-1 × sh t-1 , s∈S, and the output response is calculated by solving all scale estimations, and the calculation formula is as follows:

[0075] F_S max = max(response_S t );

[0076] In the formula, F_S max denotes the maximum response value in the scale transformation, and response_S t is the feature response of the tth frame in different scale estimations. The maximum scale output response is the optimal scale of the tth frame, and the optimal target scale estimation is obtained, thereby realizing the scale self-adaptation.

[0077] The confidence of the target response is solved. When the target is in a high confidence, it is indicated that the target motion state is good, and the target template can be updated; when the target is in a low confidence, it is judged that the target motion state is disturbed, and the target template update is stopped at this time, and the anti-occlusion mechanism is used to find back the tracking target.

[0078] The solving formula of the target response confidence is as follows:

[0079]

[0080] In the formula, F max and F min denote the maximum value and the minimum value of the target response respectively, and F x,y denotes the value of the target in the x row and y column of the response matrix. Thresholds τ1 and τ2 are set. When the current confidence value of the target of the tth frame is greater than τ2, it is considered that the target is in a high confidence state, and when the current confidence value of the target of the tth frame is less than τ1, it is considered that the target is in a low confidence state. The values of the thresholds τ1 and τ2 are as follows:

[0081]

[0082] In the formula, APCE mean denotes the average value of the response fluctuation history, and F mean denotes the average value of the response peak history.

[0083] The anti-occlusion mechanism of the present application is that when the target is in a low confidence state, the Kalman filter is enabled to make optimal estimation of the target position information based on the un-occluded target information, so that the tracker can recover and track the target again. t (x,y), which corresponds to the pos of the kernel correlation filter tracker. Then the estimation of the target position in the t+1 frame is:

[0084]

[0085] In the above formula, X t is the target state information in the t frame, A is the state transition matrix, Q t is the current frame system noise covariance, P t+1 / t is the predicted estimation covariance matrix, B is the control matrix, u is the input matrix in the current frame, P t is the estimation covariance matrix in the current frame, X t+1 / t is the predicted target motion information; by default, there is no input influence in the tracking process, so B is zero. In the update stage, the optimal solution of the target is obtained by using the following formula based on the current system prediction result and the measurement value of the current state:

[0086]

[0087] In the above formula, Z t+1 is the observation value of the target position, K is the Kalman gain coefficient, H is the observation matrix, and I is the unit matrix.

[0088] K=P t+1 / t H T (HP t+1 / t H T +R) -1 ;

[0089] By iterating the above formula, the optimal estimation of the target position is completed, and the target template of the tracker is updated by using the following formula:

[0090]

[0091] In the above formula, a t is the tracker parameter in the current frame, x t is the tracker sample template in the current frame, and theta is the linear interpolation factor.

[0092] Step 3: The ground station converts the position and scale information of the target from the image coordinate system to the world coordinate system, converts the image motion information of the target of interest to the flight motion information in the world coordinate system of the unmanned aerial vehicle, as shown in Figure 3 . The control information is transmitted to the unmanned aerial vehicle through the local area network, and the unmanned aerial vehicle follows the target flight motion in the world coordinate system.

[0093] Establish an image coordinate system with its origin at (x, y), where x = 0.5Wide and y = 0.5Hight, where Wide is the width of the image window and Height is its height. The area of ​​the initial frame's target bounding box is S0, S... t This represents the area of ​​the bounding box in the current frame, where the centroid of the target in the current frame is (x). t ,y t ).

[0094] The acquired target image area information is converted into UAV forward or backward motion control in the world coordinate system, and the calculation formula is as follows:

[0095]

[0096] Where c represents the throttle position of the drone, and ± represents the forward or backward movement of the drone. Through experiments, the present invention selects the most ideal judgment thresholds as 1.1S0 and 0.9S0, and the most ideal control parameter as 20cm / s;

[0097] The above formula means: when S t When S > 1.1S0, the throttle channel lever of the control drone is set to -20, enabling the drone to move backward at 20cm / s in the world coordinate system. t When <0.9S0, control the drone's throttle channel lever to +20, so that the drone can move forward at 20cm / s in the world coordinate system;

[0098] like Figure 3 As shown in (b), the target size has become smaller, indicating that the target has moved forward, and the drone also flies forward.

[0099] The acquired target image position information is converted into the UAV's left and right rotation control in the world coordinate system, and the calculation formula is as follows:

[0100]

[0101] Where d represents the rotation channel lever position of the UAV, 96 represents the image pixel threshold (in pixels), and ± indicates whether the UAV rotates to the right or left. Through experiments, this invention selects the most ideal judgment threshold as 96 pixels and the most ideal control parameter as 20 rad / s.

[0102] The above formula means: when x t When -x>96, the control lever position for the UAV rotation channel is +20, achieving a rightward rotation of 20 rad / s in the UAV's world coordinate system; xx t When the value is >96, the throttle channel lever of the UAV is set to -20, so that the UAV can rotate to the left at 20 rad / s in the world coordinate system.

[0103] likeFigure 3 (a) shows that the target escapes to the right, at this time the control unmanned aerial vehicle also rotates to the right, repositioning the target at the origin of the image coordinate system.

[0104] The acquired target image position information is converted into unmanned aerial vehicle ascending or descending motion control in the world coordinate system, and the calculation formula is as follows:

[0105]

[0106] Where b represents the pitch channel rod amount of the unmanned aerial vehicle, 72 represents the image pixel threshold value, the unit is pixel, and ± represents the unmanned aerial vehicle ascending or descending motion. The most ideal judgment threshold value of the present application is 72 pixel values, and the most ideal control parameter is 20 cm / s through experiments;

[0107] The above formula shows that when y t -y>72, the control unmanned aerial vehicle rotation channel rod amount is -20, realizing the unmanned aerial vehicle descending at 20 cm / s in the world coordinate system; y-y t >72, the control unmanned aerial vehicle throttle channel rod amount is +20, realizing the unmanned aerial vehicle ascending at 20 cm / s in the world coordinate system;

[0108] As shown in Figure 3 (a), the target escapes downward, at this time the unmanned aerial vehicle also descends, repositioning the target at the origin of the image coordinate system.

[0109] Finally, the channel rod amount [b, c, d] is transmitted from the ground work station to the unmanned aerial vehicle.

[0110] Step 4: Process the next frame of image, repeat step 2 and step 3 until the unmanned aerial vehicle following ends.

[0111] As shown in Figure 4 , when the target moves normally, as shown in Figure 4 (a), the target response peak distribution at this time is a single peak, indicating that the tracker tracks accurately. When the target is in a half-shielded state, as shown in Figure 4 (b), the single peak in the target response peak distribution begins to decline, and fluctuation peaks begin to appear around the single peak. When the target is in a shielded state, as shown in Figure 4 (c), the target response peak distribution at this time not only increases the number of interference fluctuation peaks, but also decreases the response peak value. The response peak value of the whole tracking process is shown in Figure 4 (d), it can be seen that as the disturbance of the target motion state deepens, the response peak value and the confidence are sharply declining.

[0112] In order to verify the effectiveness of the target tracking method based on scale and anti-occlusion optimization, a plurality of simulation experiments are respectively performed on the method of the application, the KCF method, the DSST method, the Staple method, the MIL method, the CSK method and the TLD method in the Matlab software.

[0113] The simulation experiment is explained as follows:

[0114] Simulation data set and evaluation index

[0115] The OTB (Object Tracking Benchmark) data set contains 11 interference attributes such as illumination variation (IV), scale variation (SV), deformation (DEF) and occlusion (OCC). 50 video sequences are selected for simulation test experiment.

[0116] The OTB data set has two evaluation indexes of precision plot and success plot. The precision is calculated by calculating the center error of the tracking algorithm estimation frame and the real frame of the target motion, and the center error calculation formula is where (x p ,y p ) is the center position of the target predicted by the tracker, and (x t ,y t ) is the real position of the target; the success rate is calculated by calculating the overlap rate of the tracking algorithm estimation frame and the real frame of the target motion, and the overlap rate calculation formula is R p is the predicted target frame, and R t is the real frame of the target.

[0117] 2. The simulation results are as follows:

[0118] As Figure 5 and Figure 6As shown, using the OTB species test method, using the commonly used OPE (One-Pass Evaluation) benchmark, the tracking precision plot and the tracking success plot of the seven methods under all interference are obtained. The tracking precision of the method of the application is 0.844, and the tracking success rate is 0.806. The target tracking precision is improved by 5.3% compared with the KCF algorithm, and improved by 4.4% compared with the DSST algorithm. The target tracking success rate is improved by 15.9% compared with the KCF algorithm, and improved by 19.2% compared with the DSST algorithm. As can be seen from Table 1 below, the tracking success rate of the method of the application is the highest in most scenes, especially in the occlusion scene, the light scene and the deformation scene.

[0119]

[0120] Table 1 Comparison of simulation results.

[0121] In order to verify the effectiveness of the unmanned aerial vehicle following method based on scale and anti-occlusion optimization proposed in the application, an unmanned aerial vehicle following experiment is performed, as shown in Figure 7 .

[0122] The experiment is performed in an unstructured forest. As can be seen, when the target moves to the right and passes through the tree occlusion, and the target exceeds the field of view during the experiment. In Figure 7 , the target moves horizontally, and the ground station controls the unmanned aerial vehicle to realize horizontal movement through information processing. When the target is occluded by the tree, the unmanned aerial vehicle does not lose the target and still accurately tracks and follows the target horizontal movement in real time. When the target exceeds the field of view, the unmanned aerial vehicle can adjust the body displacement to reposition the target.

[0123] Regarding the specific structure of the application, the connection relationship between each component module adopted by the application is determined and can be realized. Except for the special description in the examples, the specific connection relationship can bring corresponding technical effects, and based on the premise of not relying on the execution of the corresponding software program, the technical problems proposed in the application are solved. The model of the components, modules, specific components, the connection mode between them, and the conventional use method and the expected technical effects brought by the above technical features, except for the specific description, belong to the public content disclosed in the patent, journal paper, technical manual, technical dictionary, textbook, etc. before the application date, or belong to the existing technology such as conventional technology and common knowledge, which need not be repeated. The technical solution provided in the case is clear, complete, and can be realized, and the corresponding physical product can be reproduced or obtained according to the technical means.

[0124] The unmanned aerial vehicle following method and system based on scale and anti-occlusion optimization can realize real-time stable and effective target following.

[0125] Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present application, but not limited to them; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that: it can still modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacement to part or all of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the scope of the technical solutions of the embodiments of the present application.

Claims

1. A drone following method based on scale and anti-occlusion optimization, characterized in that: Includes the following steps: S1: The drone acquires video of the target to be tracked. The ground workstation uses the received video to manually select the target of interest in the initial frame as the initial target position information and initializes the tracker. The tracker is a computer program of a drone following algorithm that has been optimized based on scale and anti-occlusion optimization. The optimization process of the drone following algorithm based on scale and anti-occlusion optimization is as follows: S1.1: Using KCF filter to estimate target position: Set target search area, extract HOG features of initial target, and obtain target template by multiplying HOG features with Hanning window. Calculate Gaussian matrix diagram accordingly, and use kernel correlation filtering algorithm to determine the possible position of target in the next frame. S1.2: Target scale estimation: Based on the target search area in the current frame, estimate the target scale in the next frame at different scales; In step S1.2, the target is estimated at multiple scales using the image scale pyramid method, and the optimal scale is calculated using the following formula: ； In the above formula: F_S max The response_S represents the maximum response value during scaling. t The feature responses estimated at different scales for frame t are obtained by solving the above formula for each scale to obtain the optimal target scale estimate. S1.3: Calculate the target's output response based on the estimated target scale and target location; S1.4: Solve for the confidence level of the target output response. When the target has a high confidence level, update the target template. When the target is at a low confidence level, stop updating the target template and use the anti-occlusion mechanism to find and track the target again. The formula for calculating the confidence level of the target output response in step S1.4 is as follows: ； In the above formula: F max F min F represents the maximum and minimum values ​​of the target response, respectively. x,y This represents the value of the target in the x-row and y-column of the response matrix; The confidence assessment threshold is: ； In the above formula: APCE mean F represents the historical mean of the response fluctuation. mean This represents the historical average of the peak response. When the target confidence value in frame t is greater than τ2, the target is considered to be in a high confidence state. When the target confidence value in frame t is less than τ1, the target is considered to be in a low confidence state. The anti-occlusion mechanism in step S1.4 refers to the process of using Kalman filtering to estimate the target's position based on its unoccluded location information when the target is in a low-confidence state, thereby enabling the tracker to re-find and track the target. The specific steps are as follows: Let X be the centroid position information of the target in frame t. t (x, y) corresponds to pos of the kernel correlation filter tracker. Therefore, the estimate of the target's position in frame t+1 is: ； In the above formula: X t Let A be the target state information for frame t, and Q be the state transition matrix. t Let P be the system noise covariance of the current frame. t+1∕t To predict and estimate the covariance matrix, B is the control matrix, u t P is the input matrix for the current frame. t Let X be the estimated covariance matrix of the current frame. t+1 / t To predict target motion information; By default, there is no input influence during the tracking process, so B is set to zero. During the update phase, the optimal solution for the target is obtained using the following formula based on the current system prediction results and the measured values ​​of the current state: ； In the above formula: Z t+1 Here, K represents the observed value at the target location, H is the Kalman gain coefficient, H is the observation matrix, and I is the identity matrix. ； In the above formula: R is the measurement noise covariance matrix; By iterating through the above formula, the optimal estimate of the target position is obtained, and the target template of the tracker is updated using the following formula: ； In the above formula: α is the optimal estimate of the tracker parameters for the current frame. t-1 These are the tracker parameters from the previous frame. x is the optimal estimate of the current frame tracker sample template. t-1 θ is the tracker sample template from the previous frame, and θ is the linear interpolation factor. S2: The ground workstation converts the target's position and scale information output by the tracker from the image coordinate system to the world coordinate system, converts the target's motion information in the image into the UAV's flight motion information in the UAV's world coordinate system, and uses the local area network to transmit control information to the UAV, enabling the UAV to follow the target's flight motion. Step S2 specifically includes: Establish an image coordinate system with its origin at (x, y), x = 0.5Wide, y = 0.5Hight, where Wide is the width of the image window and Hight is the height of the image window. The area of ​​the initial frame target bounding box is S0, S... t This represents the area of ​​the bounding box in the current frame, where the centroid of the target in the current frame is (x...). t’ ,y t’ ); The acquired target image area information is converted into UAV forward or backward motion control in the world coordinate system, and the calculation formula is as follows: ； Where c represents the throttle channel lever position of the drone; The above formula means: when S t When S > 1.1S0, the throttle channel lever of the control drone is set to -20, enabling the drone to move backward at 20cm / s in the world coordinate system. t When <0.9S0, control the drone's throttle channel lever to +20, so that the drone can move forward at 20cm / s in the world coordinate system; The acquired target image position information is converted into the UAV's left and right rotation control in the world coordinate system, and the calculation formula is as follows: ； Where d represents the rotation channel lever of the drone, and 96 represents the image pixel threshold, in pixels; The above formula means: when x t’ When -x>96, the control lever position for the UAV rotation channel is +20, achieving a rightward rotation of 20 rad / s in the UAV's world coordinate system; xx t’ When the value is >96, the throttle channel lever of the UAV is set to -20, so that the UAV can rotate to the left at 20 rad / s in the world coordinate system. The acquired target image location information is converted into UAV ascent or descent motion control in the world coordinate system, and the calculation formula is as follows: ； Where b represents the pitch channel lever value of the UAV, and 72 represents the image pixel threshold, in pixels; The above formula means: when y t’ When -y>72, the control lever for rotating the drone is set to -20, enabling the drone to descend at 20cm / s in the world coordinate system; yy t’ When the value is >72, the throttle channel lever of the drone is set to +20, so that the drone can ascend at a speed of 20cm / s in the world coordinate system. Finally, the channel rod measurements [b, c, d] are transmitted from the ground workstation to the UAV; S3: Process the next frame image and repeat steps S1-S2 until the drone finishes following.

2. A drone following system based on scale and anti-occlusion optimization, characterized in that: The system includes a drone and a ground workstation. The drone is equipped with a wireless Wi-Fi module and a high-definition visual image sensor. The ground workstation is connected to the drone via the Wi-Fi module for data transmission. The ground workstation receives video transmitted back from the drone in real time. The ground workstation is equipped with a memory and a processor. The memory stores a computer program for the drone following method based on scale and anti-occlusion optimization as described in claim 1. The processor is used to execute the computer program and transmit control signals to the drone to achieve target following.

Images