Unmanned aerial vehicle tracking method based on adaptive search area

Abstract

The invention provides an unmanned aerial vehicle tracking method based on an adaptive search area, and aims to solve the problem that an unmanned aerial vehicle performs high-precision and high-efficiency real-time tracking on a target in a complex dynamic environment. According to the method, a Mama attention mechanism and the efficient feature extraction capability of ViT are combined, an adaptive search region mechanism is introduced, and a search region is dynamically adjusted to optimize the tracking precision and the calculation efficiency. The backbone network extracts features of a template image and a search image based on Mamba-ViT; the prediction head is composed of a plurality of convolution-batch normalization-ReLU layers and is used for generating a final tracking result. Experimental results show that the method is excellent in performance when data sets such as DTB70 and VisDrone2018 are used, compared with different trackers, the precision and the success rate are remarkably improved in the overall performance evaluation result, and particularly remarkable robustness is shown in the aspect of processing the rapid movement problem.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of unmanned aerial vehicle target tracking, and particularly to an unmanned aerial vehicle tracking method based on an adaptive search region. BACKGROUND

[0002] With the rapid development of unmanned aerial vehicle (UAV) technology, it plays an increasingly important role in various application scenarios, including but not limited to surveillance, search and rescue, environmental monitoring, and package delivery, etc. In these applications, achieving fast and reliable real-time target tracking is one of the core challenges for UAV systems. UAVs usually operate in dynamic and unpredictable environments, so their tracking systems must be able to cope with rapid target motion, scale changes, occlusions, and environmental changes, while maintaining high computational efficiency to meet real-time requirements.

[0003] Traditional UAV tracking algorithms mostly use fixed search regions to locate targets. These methods define a preset region around the previously detected target in consecutive video frames to predict the target's location in the next frame. Although this approach is computationally efficient and easy to implement, it has significant limitations in dynamic environments. Fixed search regions lack adaptability to target motion or appearance changes, and when the target motion is unpredictable, such as sudden acceleration, sharp turns, or direction changes, the static search region may not cover the target's new position, leading to tracking drift. In addition, fixed search regions are susceptible to environmental changes such as occlusions, lighting changes, or complex backgrounds, resulting in decreased tracking accuracy.

[0004] In recent years, deep learning-based trackers have improved tracking accuracy by introducing powerful feature extraction models such as convolutional neural networks and transformers. Among them, Vision Transformer (ViT) has attracted attention due to its ability to capture both local and global contextual information. However, the quadratic complexity attention mechanism of ViT results in high computational overhead, making it difficult to meet the strict computational resource constraints of UAV real-time applications.

[0005] To overcome these challenges, the present application proposes an unmanned aerial vehicle tracking method based on an adaptive search region, which combines the Mamba attention mechanism and the efficient feature extraction capabilities of Vision Transformer (ViT), and introduces an adaptive search region mechanism to dynamically adjust the search region to optimize tracking accuracy and computational efficiency. This method significantly reduces computational complexity while maintaining high accuracy, making it suitable for real-time tracking tasks on UAVs. SUMMARY

[0006] The purpose of the present application is to provide a UAV tracking method capable of achieving high-precision and high-efficiency real-time tracking in a complex dynamic environment. The method effectively deals with problems such as rapid movement, scale change and occlusion of the target by adaptively adjusting the search area and optimizing the feature extraction network, while fully utilizing the limited computing resources of the UAV.

[0007] The technical solution of the present application is to design and train the proposed model, and test the model on a UAV dataset.

[0008] The framework is composed of a backbone network and a prediction head, wherein the backbone network includes a search area adaptive network and a feature extraction network, and the prediction head is a tracking head network.

[0009] The feature extraction network in the backbone network is based on Mamba attention mechanism and Vision Transformer (ViT), and is used for extracting features of the template image and the search image.

[0010] The search area adaptive network is activated in the inference stage, calculates the relative speed of the target according to the position change of the target in the consecutive two frames, and dynamically adjusts the scaling factor of the search area based on the relative speed. The calculation formula of the scaling factor is:

[0011] (1)

[0012] wherein, is an initial scaling factor, is a scaling factor change amount adjusted according to the relative speed of the target, and its expression is:

[0013] (2)

[0014] is the relative speed of the target at t-1 moment, and a, b and c are pre-defined constants for adjusting the change range and sensitivity of the scaling factor;

[0015] A full convolution network is selected as the prediction head, which is composed of multiple convolution-batch normalization-ReLU layers; the prediction head has three branches, respectively used for predicting classification scores, predicting down-sampling offset values and predicting normalized bounding box sizes.

[0016] The overall loss function during model training is composed of a classification loss function, an IoU regression loss function and an L1 regression loss function, and is defined as follows:

[0017] (3)

[0018] wherein, is a classification loss function, and a weighted focal loss function is adopted; IoU regression loss function for bounding box regression. L1 regression loss function for bounding box regression. Weight coefficient λ , Used to balance the contribution of different loss functions. BRIEF DESCRIPTION OF DRAWINGS

[0019] Figure 1 Unmanned aerial vehicle tracking framework based on adaptive search region

[0020] Figure 2 Adaptive search region model diagram

[0021] Figure 3 Comparison of properties of different algorithms on the fast motion subset of VisDrone2018

[0022] Figure 4 Comparison of effects with and without adaptive search region DETAILED DESCRIPTION

[0023] The present application proposes an unmanned aerial vehicle tracking method based on adaptive search region, the specific implementation steps are as follows:

[0024] (1) Prepare four public datasets required for training the framework, including GOT-10k, LaSOT, COCO and TrackingNet. These datasets cover a variety of scenarios and challenges such as fast motion, scale change, occlusion, etc., which can effectively improve the robustness and generalization ability of the model.

[0025] (2) Construct the unmanned aerial vehicle tracking framework. Take the network model based on Mamba-ViT as the backbone network, extract the features of the template image with a size of 128x128 and the search image with a size of 256x256 input into the framework, and input them into the prediction head. The prediction head is composed of 4 convolution-batch normalization-ReLU layers stacked together, which is used to generate the final tracking result.

[0026] (3) In the training stage, set the batch size to 32, use the AdamW optimizer to train the model, and set the weight decay to , and the initial learning rate to . A total of 300 rounds of training are performed, 60000 image pairs are input per round, and the learning rate is reduced by 10 times after the 240th round. Through this training method, the model can learn robust feature representation, so as to realize high-precision tracking in complex environments.

[0027] In the inference stage, a Hanning window penalty is applied to introduce a position prior, specifically, the classification map is multiplied by a Hanning window of the same size, and the bounding box with the highest score after multiplication is selected as the final tracking result. This processing method can effectively improve the stability of tracking and reduce misjudgment.

[0028] (4) Prepare test data sets, including DTB70, UAVDT, VisDrone2018, UAV123 and UAV123@10fps. These test sets cover video sequences of unmanned aerial vehicles under different flight conditions, which can comprehensively evaluate the performance of the tracking algorithm. The first frame of each video in the test set is taken as a template, and each subsequent frame is taken as a search image, which is sequentially sent into the framework, and the output result is the predicted frame of each frame. The experimental results show that the method of the application performs well on the VisDrone2018 dataset, and compared with different trackers, the precision and success rate are significantly improved in the overall performance evaluation result and the performance evaluation result of the fast motion subset, especially in handling fast motion problems, which shows significant robustness.

Claims

1. A UAV tracking method based on adaptive search region, characterized in that, Specifically, it includes the following steps: S1: A single-stream architecture is used as the framework for the model, which takes the target template image and the search image as input; S2: The framework consists of a backbone network and a prediction head, where the backbone network includes a search region adaptation network and a feature extraction network, and the prediction head is a tracking head network; S3: The feature extraction network in the backbone network is based on the Mamba attention mechanism and Vision Transformer (ViT) to extract features from the template image and the search image; S4: The search region adaptive network is activated during the inference phase. It calculates the relative velocity of the target based on the positional changes of the target in two consecutive frames, and dynamically adjusts the scaling factor of the search region based on this relative velocity. The formula for calculating the scaling factor is: (1) in, The initial scaling factor. The scaling factor change adjusted according to the target relative velocity is expressed as follows: (2) Let a be the relative velocity of the target at time t-1, and let a, b, and c be predefined constants used to adjust the range and sensitivity of the scaling factor. S5: A fully convolutional network is selected as the prediction head, which consists of multiple convolution-batch normalization-ReLU layers; the prediction head has three branches, which are used to predict the classification score, predict the downsampling offset value, and predict the normalized bounding box size, respectively. S6: The overall loss function during model training consists of the classification loss function, the IoU regression loss function, and the L1 regression loss function, defined as shown in formula (3), where , All are constants: (3) S6.1: , , Defined by formulas (4), (5), and (6) respectively; (4) (5) (6) In formula (4) For the target predicted value, For the adjustment factor; in formula (5) This represents the area of ​​the intersection of the predicted bounding box and the truth bounding box. The area represents the union of the predicted bounding box and the truth bounding box; in formula (6) The target value is x. is the estimated value, and n is the sample size.

Images