Quick target tracking control system and method based on nano four-rotor unmanned aerial vehicle

Abstract

The application discloses a visual target tracking system and method based on a nano unmanned aerial vehicle, aiming at the problem that a traditional kernel correlation filter tracking algorithm cannot adapt to target size change, introduces a scale filter, so that the tracking algorithm can adaptively adjust the size and position of a target frame according to the change of the target scale; a method based on time context information is adopted to realize template updating, so that the tracking template has time correlation and can more accurately reflect the change of the target in appearance and motion state; a long-short-term tracking strategy is adopted to combine the information of short-term tracking and long-term tracking, so that the target tracker can still maintain high precision and robustness in a long-time tracking task, and the real-time requirement is met; the application is experimented on a Crazyflie2.1 nano quadrotor platform, and realizes stable and rapid target tracking control on the nano quadrotor unmanned aerial vehicle.

Description

Technical Field

[0001] This invention relates to the fields of unmanned aerial vehicle (UAV) target tracking and computer vision, and in particular to a rapid target tracking control system and method based on a nano-quadrotor UAV. Background Technology

[0002] Quadrotor drones are gradually becoming a hot topic in emerging technology fields, with widespread applications in civilian, commercial, and military sectors, and various new application scenarios have been developed, such as delivery and express delivery, miniature reconnaissance, and drone photography. Target tracking is an essential component in the various tasks undertaken by drones. Current applications mainly focus on medium and large-sized unmanned aerial vehicles. However, for covert missions such as military reconnaissance or strikes, or detection missions in confined spaces, medium and large-sized drones are no longer suitable, necessitating the research of nano-sized quadrotor drones (weighing less than 50g and with a characteristic dimension of less than 15cm).

[0003] To achieve a balance between accuracy and speed, correlation filtering methods based on KCF (Knowledge, Computation, and Cross-Referencing) are the most effective. However, existing research shows that deploying the KCF algorithm on UAVs still cannot handle changes in target size. Secondly, simply applying traditional correlation filtering algorithms cannot address target drift that may occur during long-term tracking. Furthermore, few studies focus on visual tracking of nanoscale quadcopter UAVs. Due to the small size, limited payload capacity, and susceptibility to external interference of nanoscale UAVs, researching visual tracking of nanoscale UAVs has become a challenging task. Summary of the Invention

[0004] In view of this, the purpose of the present invention is to provide a visual target tracking system and method based on a nano-quadrotor unmanned aerial vehicle.

[0005] A visual target tracking system based on a nano-quadrotor UAV includes a target tracker and a quadruple cascade PID controller; the target tracker includes a short-term tracking module and a long-term tracking module.

[0006] The short-term tracking module uses a correlation filter method to track the target; where k is set as the number of frames in the current video stream, and N frames are used as a loop. The first N-1 consecutive frames are directly processed by the short-term tracking module to output the target's position and scale information.

[0007] The long-term tracking module employs a learning-based target detection network; the Nth frame in each loop of the video stream serves as the input to the long-term tracking module, and the optimal target is selected for template updates in the short-term tracking module.

[0008] The quadruple cascaded PID controller includes a visual servo controller, a speed controller, an attitude controller, and an angular velocity controller.

[0009] The visual servo controller uses the center point of the target box output by the target tracking algorithm as the actual position and the center of the camera's field of view as the desired position. The resulting pixel error is input into the visual servo controller, and the desired speed is output.

[0010] The speed controller outputs the desired attitude and the desired total thrust of the UAV to the thrust distributor based on the speed error. Specifically, the actual speed of the UAV is measured using sensors, IMU, optical flow plates, etc., and compared with the desired speed to obtain the speed error.

[0011] The attitude controller outputs the desired angular rate based on the attitude error; where the attitude error is obtained by comparing the actual attitude of the UAV measured by the IMU with the desired attitude.

[0012] The angular rate controller measures the actual attitude of the UAV based on the IMU and the desired angular rate output by the attitude controller. It then outputs the desired three-axis torque of the UAV to the thrust distributor for controlling the thrust of the motors.

[0013] A visual target tracking method based on a nano quadrotor UAV includes:

[0014] Step 1: Receive the image sequence transmitted from the onboard camera of the nano quadcopter UAV, set the current image frame number to k, and take N frames as a loop. The first N-1 consecutive frames are directly processed by the short-term tracking module, and the Nth frame is used as the input of the long-term tracking module; acquire the first frame image, and determine the target to be tracked in the first frame through the target detection algorithm;

[0015] Step 2: In the short-term tracking module, extract the HOG features and color features of the target tracking area as a fusion feature vector, and initialize the position filter and scale filter according to the target position in the first frame;

[0016] Step 3: Obtain the next frame image:

[0017] If it is the first N-1 frames in the loop, the image is fused and features are extracted. The target position is output by the position filter and the target scale is output by the scale filter.

[0018] If it is the Nth frame image in a loop, the image is input into the long-term tracking module, which outputs several detected bounding boxes with target labels; these bounding boxes are cropped into image patches, the optimal target is selected, and this result is used for template update in the short-term tracking module;

[0019] Step 4: Using the center point of the target bounding box tracked by the short-term or long-term tracking module as the actual position and the center of the camera's field of view as the desired position, the target pixel error is input into the visual servo controller, and the desired speed is output.

[0020] Step 5: Measure the drone's current actual speed using sensors, IMU, optical flow board, etc., and compare it with the desired speed. Input the speed error into the drone's speed PID controller, which outputs the desired attitude and the desired total thrust.

[0021] Step 6: Measure the actual attitude of the UAV using the IMU and compare it with the desired attitude. Output the desired angular rate through the attitude controller; finally, the UAV's angular rate controller outputs the desired triaxial torque to obtain the thrust distribution to the motors.

[0022] Preferably, in step 3, the filter template is trained based on the output target position and scale, and the template is obtained by weighted summation of the nearest few frames before the current frame using a sliding window; wherein, the weighting is performed in chronological order, the frame closer to the current frame has a larger weight, and the frame farther away from the current frame has a smaller weight.

[0023] Preferably, in step 3, the method for obtaining the filter template includes:

[0024] Find the function f(x) = w T x makes the objective function of the position filter:

[0025]

[0026] Where, x i f(x) represents the i-th training sample, which is obtained by performing an affine transformation on the current frame image; i ) represents the training sample x i The actual response value; the response value is obtained by convolving image features with the current filter template; y i Indicates training sample x i The expected response value, w represents the filter template, λ is the regularization coefficient; m0 represents the index of the first frame in the sliding window, k represents the index of the current frame, M is the total number of frames in the sliding window, β m It is the weight of the m-th frame, where η∈(0,1);

[0027] The starting frame of the sliding window is designed as follows:

[0028] m0 = max{k-M+1, 1}

[0029] The frame weights are designed as follows:

[0030]

[0031] Extending to the nonlinear space using the Gaussian kernel function, the solution corresponding to equation (1) is obtained as follows:

[0032]

[0033] Where y represents the Fourier transform of the variable. m This represents the expected response value of the m-th frame. Let represent the Gaussian kernel function; α is the dual variable of w, from which the filter template w is obtained.

[0034] Ideally, during target tracking, HOG features and CN color features are linearly fused to obtain image features.

[0035] Preferably, in the short-term tracking module, an aspect ratio pyramid is constructed around the center point of the target bounding box output by the position filter, and a corresponding image patch is cropped. The size of the image patch is:

[0036] a n A×b n B

[0037] Where A×B is the current target box size, and a and b are scaling factors. S represents the size of the proportional filter;

[0038] Iterate through the values ​​of n to obtain image patches of different sizes, and use the aspect ratio of the image with the largest filtering response as the current tracking scale.

[0039] Preferably, in the long-term tracking module, the method for selecting the optimal target includes:

[0040] Perform SURF matching between each detected object and the template image, calculate the matching points of SURF matching for different detected objects, and finally express the scores as follows:

[0041]

[0042] Where i represents the index of the top three candidate image patches in the similarity test, ScoreHi represents the similarity value based on the image histogram, S represents the area size of the image patch, and ScoreKi represents the number of matched SURF points; the patch with the highest score is the output of the long-term tracking module.

[0043] Preferably, the image-based visual servo controller is designed as a PD controller:

[0044]

[0045] Where u(t) represents the output of the controller, K p It is the proportionality coefficient, K d These are the differential coefficients, and e(t) represents the error.

[0046] The target tracking algorithm can obtain the center point coordinates (y, z) of the target in the image, the size of the target bounding box w×h, and the image resolution W×H. Therefore, the relative pixel distance between the target and the target is designed as follows:

[0047]

[0048] Where x represents the desired target pixel height;

[0049] Input the relative pixel distance output control command in the controller. The visual servo controller can then be written as:

[0050]

[0051] Where Δx t Δy t Δz t Δx represents the relative pixel distance between the target and the drone in the current image. t-1 Δy t-1 Δz t-1 Δt represents the relative pixel distance between the target and the drone in the previous frame, and Δt is the time interval between the two frames. For the corresponding proportional and differential coefficients;

[0052] Using the incremental PD algorithm, it can be written as:

[0053]

[0054] Where k represents the k-th sampling time; q0 = K p +K d q1 = -K p -2K d q2=K d .

[0055] The present invention has the following beneficial effects:

[0056] This invention provides a visual target tracking system and method based on a nano-sized UAV. The method includes: a long-short-term tracking strategy that combines short-term and long-term tracking information; a position filter based on temporal context template updates and feature fusion; a scale filter based on aspect ratio changes; a long-term tracking module based on deep learning and template matching algorithms; and an image-based visual servoing PD controller. To address the problem that traditional kernel correlation filter tracking algorithms cannot adapt to changes in target size, a scale filter is introduced, enabling the tracking algorithm to adaptively adjust the size and position of the target bounding box according to changes in target scale. The template is updated using a method based on temporal context information, making the tracking template temporally relevant and more accurately reflecting changes in the target's appearance and motion state. The long-short-term tracking strategy combines short-term and long-term tracking information, ensuring that the target tracker maintains high accuracy and robustness during long-term tracking tasks while meeting real-time requirements. This invention was experimentally tested on a Crazyflie2.1 nano-sized quadcopter platform, achieving stable and fast target tracking control on a nano-sized quadcopter UAV. Attached Figure Description

[0057] Figure 1 This is a flowchart of a rapid target tracking process for a nano-quadrotor UAV in an embodiment of the present invention.

[0058] Figure 2 This is a control system framework diagram of the present invention.

[0059] Figure 3 This is a comparison chart of the accuracy curves of the target tracking algorithm in an embodiment of the present invention.

[0060] Figure 4 This is a comparison chart of the accuracy curves of the target tracking algorithm in the embodiments of the present invention.

[0061] Figure 5 This is a partial tracking result diagram on the OTB50 dataset in an embodiment of the present invention.

[0062] Figure 6 This is a schematic diagram illustrating the target tracking algorithm in a real-world environment, as described in an embodiment of the present invention.

[0063] Figure 7 This is a schematic diagram of the relative pixel distance output in a pedestrian tracking experiment according to an embodiment of the present invention.

[0064] Figure 8 This is a schematic diagram illustrating the speed error between the expected speed and the actual measured speed in two axial directions on the flight plane of a nano-quadcopter UAV when it flies at a certain altitude, according to an embodiment of the present invention. Detailed Implementation

[0065] The present invention will now be described in detail with reference to the accompanying drawings and embodiments.

[0066] A fast target tracking and control system based on a nano-quadrotor UAV includes a target tracker and a quadruple cascade PID controller; the target tracker consists of a short-term tracking module and a long-term tracking module;

[0067] The short-term tracking module uses a correlation filter method to quickly track the target. Here, k is set as the number of frames in the current video stream, and N frames are used as a loop. The first N-1 consecutive frames are directly processed by the short-term tracking module to output the position and scale information of the target. The value of N depends on the specific experimental conditions. In this invention, N is set to 30.

[0068] Specifically, in the short-term tracking module, a position filter based on temporal context template update and feature fusion is used, and a sliding window and frame weight parameters are designed to make the tracking template temporally relevant.

[0069] In the short-term tracking module, a scaling filter based on aspect ratio variation is designed to adaptively adjust the size and position of the target box;

[0070] The long-term tracking module employs a learning-based object detection network (YOLO) to track and detect targets, compensating for the limitations of the short-term tracking module in adapting to rapid target movement, deformation, or occlusion. The Nth frame in each loop of the video stream serves as the input to the long-term tracking module, and the output of the long-term processing is used to modify the template in the short-term tracking module, making the template more robust to changes in object appearance.

[0071] Specifically, in the long-term tracking module, in order to achieve fast tracking on a nanometer-scale platform, the lightweight deep learning-based network YOLO Fastest is used to output several detected bounding boxes with pedestrian labels; in the long-term tracking module, a target matching algorithm is designed to select the optimal target for template updates in the short-term tracking module.

[0072] The quadruple cascaded PID controller includes a visual servo controller, a speed controller, an attitude controller, and an angular velocity controller.

[0073] The visual servo controller uses the center point of the target box output by the target tracking algorithm as the actual position and the center of the camera's field of view as the desired position. The resulting pixel error is input into the visual servo controller, and the desired speed is output.

[0074] The speed controller outputs the desired attitude and the desired total thrust of the UAV to the thrust distributor based on the speed error. Specifically, the actual speed of the UAV is measured using sensors, IMU, optical flow plates, etc., and compared with the desired speed to obtain the speed error.

[0075] The attitude controller outputs the desired angular rate based on the attitude error; where the attitude error is obtained by comparing the actual attitude of the UAV measured by the IMU with the desired attitude.

[0076] The angular rate controller measures the actual attitude of the UAV based on the IMU and the desired angular rate output by the attitude controller. It then outputs the desired three-axis torque of the UAV to the thrust distributor for controlling the thrust of the motors.

[0077] A visual target tracking method based on a nano quadcopter UAV includes the following steps:

[0078] Step 1: Receive the image sequence transmitted from the onboard camera of the nano quadcopter UAV. Set the current image frame number to k, and use N frames as a loop. The first N-1 consecutive frames are directly processed by the short-term tracking module, and the Nth frame is used as the input to the long-term tracking module. Acquire the first frame image and determine the target to be tracked in the first frame using a target detection algorithm;

[0079] Step 2: In the short-term tracking module, extract the HOG features and color features of the target tracking area as a fusion feature vector, and initialize the position filter and scale filter according to the target position in the first frame;

[0080] Step 3: Obtain the next frame image:

[0081] If it is the first N-1 frames in the loop, the image is fused and features are extracted. The target position is output by the position filter and the target scale is output by the scale filter. The output target position and scale are used as the basis to train the filter template. The template is obtained by weighting and summing the nearest few frames before the current frame using a sliding window. The weighting is performed in time order, with frames closer to the current frame having a larger weight and frames farther away from the current frame having a smaller weight, so as to better describe the changes of the target.

[0082] If the image is the Nth frame in a loop, it is input into the YOLO Fastest network of the long-term tracking module. This network outputs several bounding boxes with target labels. These bounding boxes are cropped into image patches and input into the target matching algorithm to select the optimal target. This result is then used for template updates in the short-term tracking module.

[0083] Step 4: Using the center point of the target bounding box tracked by the short-term or long-term tracking module as the actual position and the center of the camera's field of view as the desired position, the target pixel error is input into the visual servo controller, and the desired speed is output.

[0084] Step 5: Measure the drone's current actual speed using sensors, IMU, optical flow board, etc., and compare it with the desired speed. Input the speed error into the drone's speed PID controller, which outputs the desired attitude and the desired total thrust.

[0085] Step 6: Measure the actual attitude of the UAV using an IMU and compare it with the desired attitude. Output the desired angular rate through the attitude controller. Finally, the UAV's angular rate controller outputs the desired triaxial torque to obtain the thrust distribution to the motors.

[0086] This invention considers the template time correlation of the correlation filter and finds the function f(x) = w T x makes the objective function of the position filter:

[0087]

[0088] Where, x i f(x) represents the i-th training sample, which is obtained by performing an affine transformation on the current frame image; i ) represents the training sample x i The actual response value; the response value is obtained by convolving image features with the current filter template; y i Indicates training sample x i The expected response value, w represents the filter template, and λ is the regularization coefficient. Adding a regularization term can prevent overfitting and increase the model's generalization ability. m0 represents the index of the first frame in the sliding window, k represents the index of the current frame, M is the total number of frames in the sliding window, and β... m Let β be the weight of the m-th frame, where η∈(0,1). It can be seen that β... m As m increases, the filter increases, giving it the ability to reflect the time-varying nature of the target.

[0089] The starting frame of the sliding window is designed as follows:

[0090] m0 = max{k-M+1, 1} (2)

[0091] The frame weights are designed as follows:

[0092]

[0093] Extending to the nonlinear space using the Gaussian kernel function, the solution corresponding to equation (1) can be obtained as follows:

[0094]

[0095] Where ^ denotes the Fourier transform of the variable, y m This represents the expected response value of the m-th frame. Let α represent the Gaussian kernel function; α is the dual variable of ω, from which the filter template ω is obtained.

[0096] The Gaussian kernel function is defined as:

[0097]

[0098] Where x represents the feature extracted from the training sample, and c represents the number of feature channels. and Let F represent the Fourier transforms of the values ​​of any two vectors x and x' in channel c, respectively. -1 This represents the inverse Fourier transform.

[0099] In this invention, feature fusion technology is used to linearly fuse HOG features and CN color features. The features can be represented as:

[0100]

[0101] Where, x 0 x represents the HOG eigenvector. 1 This represents the CN color feature vector.

[0102] HOG features are constructed by calculating the gradients of pixels in a specific region to form a direction vector.

[0103] G x (x,y)=I(x+1,y)-I(x-1,y) (7)

[0104] G y (x,y)=I(x,y+1)-I(x,y-1) (8)

[0105]

[0106]

[0107] Among them, G x (x, y) and G y (x, y) represent the horizontal and vertical gradients at point (x, y), respectively; G(x, y) represents the gradient at point (x, y); I(x, y) represents the pixel value at point (x, y); and θ(x, y) represents the gradient direction. The gradient direction is divided into several intervals, and the gradient of each interval forms a vector x. 0 CN color features map pixel values ​​in an image to a set of predefined color names, and obtain the CN feature vector by statistically analyzing the color histogram of the target template.

[0108] x 1 =b(RR(x,y),GG(x,y),BB(x,y)) (11)

[0109] Where b() is the mapping function, and RR(x,y), GG(x,y), and BB(x,y) represent the RGB channel values ​​of the image, respectively.

[0110] During detection, given a candidate image patch, the input z is determined by the tracking result of the previous frame. All loop patches are evaluated using z. The response result can be obtained according to the following formula:

[0111]

[0112] right Performing an inverse Fourier transform yields the region of maximum value in the image, which is the region where the target is located.

[0113] Furthermore, this invention designs an adaptive scaling filter to better describe the deformation of the target while scaling changes.

[0114] A pyramid with aspect ratios is constructed around the center point of the target bounding box output by the position filter, and the corresponding image patch is cropped. The size of the image patch is:

[0115] a n A×b n B (13)

[0116] Where A×B is the current target box size, and a and b are scaling factors. S represents the size of the proportional filter.

[0117] By iterating through the values ​​of n, image patches of different sizes are obtained. The aspect ratio of the image with the largest filtering response is used as the current tracking scale, which better reflects the active or passive appearance changes of the target. Specifically, active appearance changes include a person standing or crouching, while passive appearance changes include changes in target size caused by changes in the distance of the camera's viewpoint, and changes in the target's perspective in the field of view caused by changes in the translation of the camera's viewpoint, such as a person's front or side view, or the front or body of a car.

[0118] Furthermore, the long-term filtering module is designed to include the lightweight YOLO Fastest network and a template matching algorithm.

[0119] Correlation filtering methods sample around the target position in the previous frame. On low-computing-power devices such as nano-quadcopter drones, the stuttering caused by deep learning methods requiring high hardware performance can significantly impact tracking performance, leading to target drift or tracking failure, and making it difficult to guarantee real-time performance. Therefore, this invention employs the lightweight target detection network YOLO Fastest, with the backbone network using ShuffleNet V2. Channel rearrangement technology reduces computation and parameter count, making the overall network more lightweight while achieving fast real-time detection while maintaining detection accuracy. Among the multiple detected objects output by YOLO Fastest, a matching algorithm is still needed to find the unique optimal tracking target. In the matching algorithm, firstly, each detected object is subjected to a histogram similarity test with the template image, and a score is calculated. Then, each detected object is subjected to SURF matching with the template image, and the number of matching points for different detected objects in SURF matching is calculated. Finally, the scores are visualized as follows:

[0120]

[0121] Where i represents the index of the top three candidate image patches in the similarity test, ScoreHi represents the similarity value based on the image histogram, S represents the area of ​​the image patch, and ScoreKi represents the number of matched SURF points. The patch with the highest score is used as the output of the long-term tracking module. This is used to modify the template for short-term tracking.

[0122] Furthermore, such as Figure 2 As shown, a quadruple cascaded PID controller is designed, in which the image-based visual servo controller is designed as a PD controller:

[0123]

[0124] Where u(t) represents the output of the controller, K p It is the proportionality coefficient, K d These are the differential coefficients, and e(t) represents the error.

[0125] The target tracking algorithm can obtain the center point coordinates (y, z) of the target in the image, the size of the target bounding box w×h, and the image resolution W×H. Therefore, the relative pixel distance between the target and the target is designed as follows:

[0126]

[0127] Where x represents the desired target pixel height.

[0128] Input the relative pixel distance output control command in the controller. The visual servo controller can then be written as:

[0129]

[0130] Where Δx t Δy t Δz t Δx represents the relative pixel distance between the target and the drone in the current image. t-1 Δy t-1 Δz t-1 Δt represents the relative pixel distance between the target and the drone in the previous frame, and Δt is the time interval between the two frames. The corresponding proportional and differential coefficients can be adjusted according to the actual flight conditions.

[0131] To reduce the computational load, an incremental PD algorithm is used, which can be written as:

[0132]

[0133] Where k represents the kth sampling time;

[0134] q0 = K p +K d

[0135] q1 = -K p -2K d

[0136] q2=K d .

[0137] Example 1: First, the effectiveness of the target tracking algorithm was verified. MATLAB R2019a software was used to conduct experiments on a computer with an i5-10210U CPU (1.60GHz) and 12GB RAM. This invention only utilizes the CPU. The experimental environment is shown in Table 1:

[0138] Table 1 Experimental Environment

[0139]

[0140] The test dataset used was OTB50, and the evaluation criteria were center location error (CLE) and overlap. To compare performance, several common correlation filter trackers were selected, including KCF, CSK, TLD, and Struck. Among these methods, the target tracking algorithm proposed in this invention performed best on the OTB50 dataset and met real-time requirements. The results are as follows: Figure 3 As shown in 4 and Table 2.

[0141] Table 2 FPS Results

[0142]

[0143] And from Figure 5 As can be seen, when the target undergoes deformation or changes in size, the algorithm proposed in this invention can still track the target relatively stably, while other trackers experience tracking drift or loss.

[0144] Example 2:

[0145] In this embodiment, a spacious indoor space is selected, and human subjects are used to conduct the target tracking experiment. For example... Figure 6 As shown, the algorithm can completely frame and track the target. Each frame of the video stream is manually labeled to obtain the ground truth. The center location error and overlap are calculated using formulas with the data tracked during flight. The final results show a center location error of 6.75 pixels, an overlap of 99.4%, and a tracking speed of 40fps, meeting real-time requirements and achieving fast target tracking control on a nano-sized UAV platform. Figure 7 As shown, du and dv represent the relative pixel distance between the center point of the tracking box and the center of the field of view, dh represents the relative pixel distance between the height of the tracking box and the desired pixel, and the horizontal axis represents the frame number. It can be seen that the relative pixel distance remains stable during tracking.

[0146] like Figure 8 As shown in the figure, in our experiment, the nano-sized UAV can maintain relatively stable and fast target tracking, with the speed error remaining within ±0.1m / s.

[0147] In summary, the above are merely preferred embodiments of the present invention and are not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A visual target tracking system based on a nano quadcopter UAV, characterized in that, It includes a target tracker and a quadruple cascaded PID controller; the target tracker includes a short-term tracking module and a long-term tracking module; The short-term tracking module uses a correlation filter method to track the target; where k is set as the number of frames in the current video stream, and N frames are used as a loop. The first N-1 consecutive frames are directly processed by the short-term tracking module to output the target's position and scale information. The long-term tracking module employs a learning-based target detection network; the Nth frame in each loop of the video stream serves as the input to the long-term tracking module, and the optimal target is selected for template updates in the short-term tracking module. The quadruple cascaded PID controller includes a visual servo controller, a speed controller, an attitude controller, and an angular velocity controller. The visual servo controller uses the center point of the target box output by the target tracking algorithm as the actual position and the center of the camera's field of view as the desired position. The resulting pixel error is input into the visual servo controller, and the desired speed is output. The visual servo controller is designed as a PD controller: in Indicates the output of the controller. It is a proportionality coefficient. These are differential coefficients. Indicates error; The center point coordinates of a target in an image can be obtained through target tracking algorithms. The size of the target box and image resolution Then the relative pixel distance of the design target is: in, Indicates the desired target pixel height; Input the relative pixel distance output control command in the controller. , , The visual servo controller can then be written as: in , , This represents the relative pixel distance between the target and the drone in the current image. , , This represents the relative pixel distance between the target and the drone in the previous frame. The interval between two frames. , , , , , For the corresponding proportional and differential coefficients; Using the incremental PD algorithm, it can be written as: Where k represents the kth sampling time; , , ; The speed controller outputs the desired attitude and the desired total thrust of the UAV to the thrust distributor based on the speed error; in this process, the actual speed of the UAV is measured by sensors, IMU, and optical flow board, and compared with the desired speed to obtain the speed error; The attitude controller outputs the desired angular rate based on the attitude error; where the attitude error is obtained by comparing the actual attitude of the UAV measured by the IMU with the desired attitude. The angular rate controller measures the actual attitude of the UAV based on the IMU and the desired angular rate output by the attitude controller. It then outputs the desired three-axis torque of the UAV to the thrust distributor for controlling the thrust of the motors.

2. A visual target tracking method based on a nano quadcopter UAV, characterized in that, include: Step 1: Receive the image sequence transmitted from the onboard camera of the nano quadcopter UAV, set the current image frame number to k, and take N frames as a loop. The first N-1 consecutive frames are directly processed by the short-term tracking module, and the Nth frame is used as the input of the long-term tracking module; acquire the first frame image, and determine the target to be tracked in the first frame through the target detection algorithm; Step 2: In the short-term tracking module, extract the HOG features and color features of the target tracking area as a fusion feature vector, and initialize the position filter and scale filter according to the target position in the first frame; Step 3: Obtain the next frame image: If it is the first N-1 frames in the loop, the image is fused and features are extracted. The target position is output by the position filter and the target scale is output by the scale filter. If it is the Nth frame image in a loop, the image is input into the long-term tracking module, which outputs several detected bounding boxes with target labels; these bounding boxes are cropped into image patches, the optimal target is selected, and this result is used for template update in the short-term tracking module; Step 4: Using the center point of the target bounding box tracked by the short-term or long-term tracking module as the actual position and the center of the camera's field of view as the desired position, the target pixel error is input into the visual servo controller, and the desired speed is output. The visual servo controller is designed as a PD controller: in Indicates the output of the controller. It is a proportionality coefficient. These are differential coefficients. Indicates error; The center point coordinates of a target in an image can be obtained through target tracking algorithms. The size of the target box and image resolution Then the relative pixel distance of the design target is: in, Indicates the desired target pixel height; Input the relative pixel distance output control command in the controller. , , The visual servo controller can then be written as: in , , This represents the relative pixel distance between the target and the drone in the current image. , , This represents the relative pixel distance between the target and the drone in the previous frame. The interval between two frames. , , , , , For the corresponding proportional and differential coefficients; Using the incremental PD algorithm, it can be written as: Where k represents the kth sampling time; , , ; Step 5: Measure the current actual speed of the drone using sensors, IMU, and optical flow board, compare it with the expected speed, input the speed error into the drone speed PID controller, and output the expected attitude and the expected total thrust. Step 6: Measure the actual attitude of the UAV using the IMU and compare it with the desired attitude; output the desired angular rate through the attitude controller; finally, the UAV's angular rate controller outputs the desired triaxial torque to obtain the thrust distribution to the motors.

3. The visual target tracking method based on a nano quadcopter UAV as described in claim 2, characterized in that, In step 3, the filter template is trained based on the output target position and scale. The template is obtained by weighted summation of the nearest frames before the current frame using a sliding window. The weighting is performed in chronological order, with frames closer to the current frame having a larger weight and frames farther away having a smaller weight.

4. The visual target tracking method based on a nano quadcopter UAV as described in claim 3, characterized in that, In step 3, the method for obtaining the filter template includes: Find the function Make the objective function of the position filter: in, This represents the i-th training sample, which is obtained by performing an affine transformation on the current frame image. Indicates training samples The actual response value; the response value is obtained by convolving the image features with the current filter template. Indicates training samples The expected response value, Indicates the filter template. The regularization coefficient is used. This indicates the index of the first frame in the sliding window. Indicates the current frame index. It is the total number of frames in the sliding window. It is the first The weights of each frame, where ; The starting frame of the sliding window is designed as follows: The frame weights are designed as follows: Extending to the nonlinear space using the Gaussian kernel function, the solution corresponding to equation (1) is obtained as follows: in, Represents the Fourier transform of the variable. This represents the expected response value of the m-th frame. Represents the Gaussian kernel function; yes The dual variables are obtained from this, thus yielding the filter template. .

5. The visual target tracking method based on a nano-quadrotor UAV as described in claim 2, characterized in that, During target tracking, HOG features and CN color features are linearly fused to obtain image features.

6. The visual target tracking method based on a nano quadcopter UAV as described in claim 2, characterized in that, In the short-term tracking module, a pyramid with aspect ratios is constructed around the center point of the target bounding box output by the position filter, and the corresponding image patch is cropped. The size of the image patch is: in This is the current target bounding box size. and It is a scaling factor. S represents the size of the proportional filter; Iterate through the values ​​of n to obtain image patches of different sizes, and use the aspect ratio of the image with the largest filtering response as the current tracking scale.

7. The visual target tracking method based on a nano-quadrotor UAV as described in claim 2, characterized in that, In the long-term tracking module, the method for selecting the optimal target includes: Perform SURF matching between each detected object and the template image, calculate the matching points of SURF matching for different detected objects, and finally express the scores as follows: in, This represents the index of the top three candidate image patches in the similarity test. Represents the similarity value based on the image histogram. Indicates the size of the image patch. This indicates the number of matched SURF points; the patch with the highest score is the output of the long-term tracking module.

Images