Lightweight airborne visual tracking method and device based on random projection, and medium
By combining Locality Sensitive Hash Random Projection (LSH) technology with a lightweight Siamese architecture, low-cost, high-efficiency, and high-precision target tracking for airborne visual tracking was achieved, solving the tracking problem under the resource constraints of airborne platforms.
Patent Information
- Application Number
- CN202511839428.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-12-08
- Publication Date
- 2026-03-13
AI Technical Summary
Existing airborne visual tracking technologies struggle to achieve efficient and real-time target localization in resource-constrained scenarios, exhibiting issues such as an imbalance between feature dimensions and storage costs, low efficiency in similarity calculations, and insufficient adaptability of dimensionality reduction solutions.
By combining Locality Sensitive Hash (LSH) random projection technology with a lightweight Siamese architecture, high-dimensional features are mapped to low-dimensional binary codes through a random projection matrix, and feature correlation calculation is performed in a dual-branch structure to reduce computation and storage costs while maintaining tracking accuracy.
The system achieves low computational cost, low storage consumption, and high real-time target tracking on the airborne platform, improving feature matching calculation efficiency, maintaining tracking accuracy, and adapting to the resource requirements of complex airborne scenarios.
Smart Images
Figure CN121661097A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to a lightweight airborne visual tracking method, device, and medium based on random projection, belonging to the field of airborne computer vision and target tracking technology. Background Technology
[0002] Airborne visual tracking, as a core supporting technology for low-altitude operations such as UAV reconnaissance, power line inspection, and geographic mapping, needs to achieve real-time and accurate positioning of dynamic targets under complex conditions such as high-altitude overhead views, strong background interference, and drastic changes in target scale. With the development of deep learning technology, the accuracy of tracking algorithms has continued to improve, but the challenge of balancing "accuracy and efficiency" under the resource constraints of airborne platforms has not yet been effectively solved.
[0003] Current mainstream tracking solutions can be divided into two categories: one is the Transformer-based target tracking architecture, which models the global contextual relationship between the target and the background through a query-key attention mechanism. It achieves state-of-the-art performance on target tracking datasets such as UAV123 and DTB70. However, the inherent multi-head attention computing mechanism of Transformer has high hardware computing power requirements. Even after lightweight improvements such as pruning and quantization, it is still difficult to adapt to airborne embedded devices with limited computing power due to the high complexity of global feature interaction. The other category is lightweight trackers represented by Siamese-like network structures. They adopt CNN backbones and achieve feature matching between templates and search regions. Although the number of parameters is greatly compressed, it still faces the efficiency bottleneck of high-dimensional feature processing.
[0004] Specifically, existing technologies suffer from three major limitations: First, there is an imbalance between feature dimensionality and storage cost. The template and search features extracted by the backbone network of Siamese-like models are both high-dimensional real-valued features, stored in 32-bit floating-point format. In template update scenarios involving continuous multi-frame tracking, storage overhead increases linearly with tracking duration, easily exceeding the limited memory capacity of airborne equipment. Second, similarity calculation is inefficient. The core matching step of traditional Siamese architecture, namely matching template and search features, relies on cosine similarity or Euclidean distance calculations, requiring traversal of all feature dimensions to perform floating-point operations. In scenarios involving large search areas due to high-speed drone flight, the computational latency of feature matching increases significantly, making it difficult to meet the real-time requirements of airborne tasks exceeding 30 FPS. Third, existing dimensionality reduction schemes lack adaptability. Some studies use model compression or knowledge distillation techniques to reduce the number of backbone network parameters, but this leads to a loss of tracking accuracy. Traditional dimensionality reduction methods such as PCA rely on offline statistical characteristics, which are poorly adaptable to sudden target occlusion and lighting changes in airborne scenarios. Moreover, the dimensionality reduction results in real-valued features, which do not fundamentally reduce the computational complexity of feature matching and subsequent processing.
[0005] Locality Sensitive Hash (LSH) is a random projection technique that maps high-dimensional features to low-dimensional binary codes through random vector projection and sign function quantization. Its core advantages lie in: significantly reducing the storage cost of binary features while maintaining the feature similarity structure (similar features have a higher collision probability after hashing), and significantly improving the speed of Hamming distance calculation during similarity matching compared to real-valued feature similarity calculation. However, current technologies have not yet deeply integrated LSH random projection with lightweight Siamese architectures, especially lacking engineering adaptation solutions for airborne vision scenarios, thus failing to fully unleash the deployment potential of lightweight models on airborne platforms. Summary of the Invention
[0006] In view of this, the present invention provides a lightweight airborne visual tracking method, device and medium based on random projection, which can reduce the computation and storage costs of feature processing on airborne platforms, ensure target tracking accuracy and adapt to resource-constrained scenarios.
[0007] To achieve the above objectives, the technical solution of the present invention is as follows.
[0008] A lightweight airborne visual tracking method based on random projection includes the following steps:
[0009] A template image frame and a search image frame containing the tracked target are obtained; wherein the template image frame is obtained by cropping the expanded region of the target bounding box in the first frame of the video based on the initial target bounding box, and the search image frame is obtained by cropping the expanded region of the target bounding box predicted in the previous frame in subsequent frames of the video.
[0010] The template image frame and the search image frame are respectively input into the feature extraction network to extract template features and search features;
[0011] The template features and the search features are binarized and encoded using a random projection matrix to generate template binary features and search binary features;
[0012] The template binary features and the search binary features are converted into bipolar features;
[0013] The bipolar features are correlated using a dual-branch structure, which includes a global association branch for capturing global context information and a local matching branch for capturing local texture information.
[0014] The outputs of the global correlation branch and the local matching branch are combined to generate a classification response map and a regression response map;
[0015] Based on the classification response map and the regression response map, output the predicted bounding box of the tracking target.
[0016] The feature extraction network is a lightweight convolutional neural network with shared weights.
[0017] The lightweight convolutional neural network is the MobileNetV3 network.
[0018] The step of using a random projection matrix to binarize and encode the template features and the search features includes: expanding the features in the channel dimension and performing matrix multiplication with the random projection matrix; applying a sign function to the result of the matrix multiplication to quantize it into binary code.
[0019] The random projection matrix is pre-generated and fixed before model training, or optimized as a learnable parameter during model training.
[0020] The global association branch calculates global similarity through matrix multiplication, while the local matching branch calculates local similarity between the cropped bipolar template features and the search features by using the inner product of matrices as a sliding window.
[0021] During model training, a joint loss function is used for optimization. The joint loss function includes at least the cross-entropy loss for classification and the IoU loss for bounding box regression.
[0022] The present invention also provides a lightweight airborne visual tracking device based on random projection, comprising:
[0023] Memory, used to store computer programs;
[0024] A processor is used to execute the computer program to implement the lightweight airborne visual tracking method based on random projection described in this invention.
[0025] The present invention also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the lightweight airborne visual tracking method based on random projection described in the present invention.
[0026] Beneficial effects
[0027] 1. The method of this invention is based on Local Sensitive Hash (LSH) random projection to achieve lightweight airborne visual tracking. It is suitable for resource-constrained scenarios of airborne platforms such as UAVs and helicopters. It can efficiently handle typical challenges such as drastic changes in target scale, complex background interference, and high-altitude overhead viewpoint shift under airborne vision, and meet the core requirements of airborne systems for tracking algorithms of "low computing power consumption, low storage consumption, and high real-time performance".
[0028] 2. This invention organically combines LSH random projection technology with the existing lightweight tracking architecture, further reducing the computation and storage costs of feature processing without significantly sacrificing tracking accuracy, thus breaking through the "resource-performance" constraints of airborne visual tracking.
[0029] 3. The present invention proposes a feature binarization scheme based on random projection for airborne visual single-target tracking. By performing binary encoding conversion before feature correlation calculation, the computational complexity of feature matching is fundamentally reduced. The random projection process preserves the feature similarity structure, and through an optimized training strategy, it effectively maintains tracking accuracy while significantly reducing feature storage costs.
[0030] 4. The method of this invention incorporates random projection into a two-branch Siamese tracking architecture. After feature extraction, random projection is used for binarization, enabling efficient computation of subsequent feature matching in binary space. The correlation calculation of binary features adopts a similarity metric based on Hamming distance, replacing the cosine similarity calculation of traditional floating-point features, significantly improving computational efficiency. A carefully designed combination of loss functions ensures that the model maintains superior tracking performance even in resource-constrained environments. Attached Figure Description
[0031] Figure 1 This is a flowchart illustrating the implementation of the method of the present invention.
[0032] Figure 2 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention. Detailed Implementation
[0033] The present invention will be further described in detail below with reference to specific embodiments.
[0034] This invention provides a lightweight airborne visual tracking method based on random projection, the specific process of which is as follows: Figure 1 As shown, it includes the following steps:
[0035] Step 1: Collect target tracking video sequences in airborne scenarios and label the bounding boxes of targets in the videos as ground truth.
[0036] Specifically, airborne scene videos are collected using drone-borne cameras. These airborne scenes include typical scenarios such as high-altitude overhead shots, dramatic changes in target scale, and target occlusion, ensuring that the dataset can meet the actual needs of airborne single-target tracking. Annotation tools are used to mark the bounding boxes of targets in the video frame by frame, clarifying the position of the target in each frame, generating annotation files, and storing them in association with the corresponding video frames.
[0037] Step 2: Crop the video frames into template frames and search frames, convert them into tensors, and normalize them;
[0038] Specifically, for the video sequences in the dataset, the first frame is cropped into a template frame by expanding an appropriate area outward from the target bounding box, and subsequent frames are cropped into search frames by expanding the corresponding area outward from the target bounding box of the previous frame; the image resolution is uniformly adjusted to a fixed size that adapts to the model input, converted into tensor format, and standardized.
[0039] Step 3: Load the tensor dataset into the training data loader and the validation data loader;
[0040] Specifically, the tensor dataset is divided into training and validation sets according to a reasonable ratio; a custom dataset object is created to realize the association reading of tensor data and bounding box truth values; the training and validation sets are loaded separately through a data loader; the data loader supports batch iteration and random shuffling.
[0041] Step 4: Set hyperparameters, load the network and data onto the GPU, initialize model weights and set it to training mode;
[0042] The hyperparameters include learning rate, training epochs, batch size, and weight decay; a learning rate decay strategy is adopted; the weights of the network model are initialized, the network is set to training mode, and the input data and corresponding labels are moved to the CUDA device (the network model is moved to the CUDA device for computation).
[0043] Step 5: Input the template frame and search frame tensors into the feature extraction network to extract template features and search features;
[0044] Specifically, the network can be a Siamese network with shared weights; a lightweight MobileNetV3 network is used as the backbone. The template frame tensor and the search frame tensor are input into the weighted MobileNetV3 network. The template branch outputs a feature map focusing on the target region, and the search branch outputs a feature map containing candidate regions. The feature maps of both branches maintain the same channel dimension (e.g., 96 channels) to provide input for subsequent random projection encoding.
[0045] Step 6: Perform Local Sensitive Hash (LSH) random projection encoding on the template features and search features respectively to generate low-dimensional binary features;
[0046] Specifically, a pre-generated random projection matrix is used to map high-dimensional floating-point features to low-dimensional binary features. First, the feature map is unfolded along the channel dimension and multiplied with the projection matrix. Then, the result is quantized into binary encoding of 0s and 1s using a sign function (by mapping 0 to -1 and 1 to 1, it is converted into bipolar features, making the matrix multiplication operation equivalent to a Hamming distance metric). The random projection matrix used for random projection is generated during initialization and is either set as a fixed parameter that does not participate in gradient updates during training, or as a learnable parameter that is optimized along with the network.
[0047] In this embodiment, a dual-branch processing module containing a global association branch and a local matching branch is constructed. Locally sensitive hashing (LSH) random projection encoding is performed on the template features and search features respectively, mapping high-dimensional real-valued features to low-dimensional binary features. The LSH random projection projects the feature map in the channel dimension through a preset projection matrix and performs binarization using a sign function. This process keeps the spatial structure of the feature map unchanged.
[0048] Step 7: Calculate the global similarity in the global association branch and the local similarity in the local matching branch by calculating the correlation of binary features, and then fuse them to obtain the classification and regression response maps.
[0049] Specifically, the correlation calculation includes: in the global correlation branch, converting binary features into bipolar features and performing matrix multiplication to capture global context information; in the local matching branch, centering and cropping the template binary features, and calculating the Hamming distance between the cropped template features and the search features to capture local texture information; finally, concatenating the calculation results of the two branches and fusing them through a convolutional layer to generate a response map.
[0050] Specifically, to balance global contextual information with local texture details, this step employs a dual-branch parallel processing architecture for correlation calculation. First, the template generated in step 6 and the search binary features (0 / 1) are converted into bipolar features (-1 / 1) to simulate Hamming distance through multiplication. Next, the data flow splits into two paths: one enters the global association branch, calculating global similarity through batch matrix multiplication of the bipolar features; the other enters the local matching branch, first center-clipping the bipolar template features, then using the clipped bipolar template features as a sliding window to calculate the Hamming distance between the clipped template features and the search features through matrix inner product, obtaining the local similarity. Finally, the feature maps output from the two branches are concatenated along the channel dimension and linearly fused through a 1×1 convolutional layer to generate the final response map used for classification and regression tasks.
[0051] Step 8: Input the classification branch and regression branch respectively into the response map, and output the probability distribution of the target and background, as well as the precise offset of the bounding box;
[0052] Specifically, the classification branch receives the response map, processes it through a multi-layer convolutional network, and outputs a target background probability distribution map, with each location containing a confidence score for the presence of the target; the regression branch also receives the response map, processes it through a multi-layer convolutional network, and outputs bounding box offsets, including center point coordinate offsets and width and height adjustment parameters.
[0053] Step 9: Calculate the error using the joint loss function and update the parameters through backpropagation of the optimizer;
[0054] In this embodiment, a multi-task joint loss function is used, including cross-entropy loss for classification tasks and IoU loss for regression tasks; the Adam optimizer is used in conjunction with a learning rate decay strategy to update parameters.
[0055] Furthermore, the gradient of the overall loss with respect to the model parameters is calculated by automatic differentiation. The chain rule is used to pass and calculate the overall loss along each parameter of the model to obtain the contribution of each parameter to the overall loss. The optimizer adjusts the model parameters according to the set learning rate and gradient value.
[0056] Step 10: Based on the training iterations and learning rate strategy, repeat steps 5 to 9 until the model converges;
[0057] Step 11: Input the test data into the trained model and output the tracking results to achieve airborne target tracking.
[0058] This involves collecting test videos of airborne scenarios that were not used in the training process, and processing these videos according to the method in step 2: The first frame is cropped based on the target's true bounding box to obtain a template frame; subsequent frames are cropped based on the prediction results of the previous frame to obtain search frames; all images are converted to tensors and standardized, including:
[0059] Template feature processing: Input the template tensor of the first frame into the trained network to extract template features, use the projection matrix to generate binary template features and store them; Search feature processing: For each subsequent search frame tensor, input it into the network to extract search features, and use the projection matrix to generate binary search features;
[0060] Binary feature matching: Using the stored binary template features and the binary search features of the current frame, global matrix multiplication and matrix inner product are performed respectively, and the results are fused to generate classification and regression response maps;
[0061] Two-branch prediction: The response map is input into the classification branch and the regression branch respectively. The classification branch outputs the target confidence score, and the regression branch outputs the bounding box refinement parameters; Tracking result generation: The optimal target region is selected based on the classification confidence score, and the precise bounding box coordinates are calculated by combining the regression offset.
[0062] Specifically, during the model inference phase, the process is as follows:
[0063] (1) Template feature processing: Input the template tensor of the first frame into the trained dual-branch convolutional network to extract template features, use the trained random projection matrix to encode the template features in binary form, generate binary template features, and store them as tracking templates;
[0064] (2) Search feature processing: For each subsequent search frame tensor, the search features are extracted by a dual-branch convolutional network and then binary encoded by a random projection matrix to generate binary search features.
[0065] (3) Binary feature matching: Convert the stored binary template features and the binary search features of the current frame into bipolar format, and send them to the global association branch to perform matrix multiplication to calculate global similarity and to the local matching branch to perform matrix inner product to calculate local similarity. Then, fuse the calculation results of the two to generate classification and regression response maps.
[0066] (4) Two-branch prediction: The response map is input into the classification branch and the regression branch respectively. The classification branch outputs the target confidence score, and the regression branch outputs the bounding box refinement parameters.
[0067] (5) Tracking result generation: Optimal target region is selected based on classification confidence, and precise bounding box coordinates are calculated by combining regression offset;
[0068] (6) Result visualization output: Map the final predicted bounding box coordinates back to the original test video frame, draw the bounding box and label the target information, synthesize the output video according to the frame sequence, and complete the real-time target tracking in the airborne scene.
[0069] This application also provides an electronic device. Figure 2 The structure of an electronic device provided in an embodiment of the present invention is illustrated. For example, the electronic device 20 may include a processor 21, a memory 22, and a transmission device 23. The processor 21 is used to execute the lightweight airborne visual tracking method based on random projection mentioned in the above embodiments. The processor and the memory can be connected via a bus or other means, taking a bus connection as an example. The transmission device can be connected to the processor and the memory via wired or wireless means. The memory, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs, non-transitory computer-executable programs, and modules, such as the program instructions / modules corresponding to the lightweight airborne visual tracking method based on random projection in the embodiments of this application. The processor executes various functional applications and data processing by running the non-transitory software programs, instructions, and modules stored in the memory, thereby realizing the lightweight airborne visual tracking method based on random projection in the above method embodiments. The memory may include a program storage area and a data storage area. The program storage area may store the operating system and at least one application program required for a function; the data storage area may store data created by the processor, etc. Furthermore, the memory may include high-speed random access memory and non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some embodiments, the memory may optionally include memory remotely located relative to the processor, which can be connected to the processor via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof. The one or more modules stored in the memory, when executed by the processor, perform the lightweight airborne visual tracking method based on random projection in the embodiments.
[0070] In another aspect, this application also provides a computer-readable storage medium, which may be the computer-readable storage medium included in the apparatus described in the above embodiments; or it may be a standalone computer-readable storage medium not assembled into the device. The computer-readable storage medium may be a tangible storage medium, such as random access memory (RAM), main memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, register, floppy disk, hard disk, removable storage disk, CD-ROM, or any other form of storage medium known in the art. The computer-readable storage medium stores one or more programs, which are used by one or more processors to execute the lightweight airborne vision tracking method based on random projection described in this application.
[0071] In summary, the above are merely preferred embodiments of the present invention and are not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.
Claims
1. A lightweight airborne visual tracking method based on random projection, characterized in that, Includes the following steps: Acquire template image frames and search image frames containing the target being tracked; The template image frame is obtained by cropping the expanded region of the initial target bounding box in the first frame of the video, and the search image frame is obtained by cropping the expanded region of the target bounding box predicted in the previous frame in subsequent frames of the video. The template image frame and the search image frame are respectively input into the feature extraction network to extract template features and search features; The template features and the search features are binarized and encoded using a random projection matrix to generate template binary features and search binary features; The template binary features and the search binary features are converted into bipolar features; The bipolar features are correlated using a dual-branch structure, which includes a global association branch for capturing global context information and a local matching branch for capturing local texture information. The outputs of the global correlation branch and the local matching branch are combined to generate a classification response map and a regression response map; Based on the classification response map and the regression response map, output the predicted bounding box of the tracking target.
2. The lightweight airborne visual tracking method based on random projection according to claim 1, characterized in that, The feature extraction network is a lightweight convolutional neural network with shared weights.
3. The lightweight airborne visual tracking method based on random projection according to claim 2, characterized in that, The lightweight convolutional neural network is the MobileNetV3 network.
4. The lightweight airborne visual tracking method based on random projection according to any one of claims 1-3, characterized in that, The step of using a random projection matrix to binarize and encode the template features and the search features includes: expanding the features in the channel dimension and performing matrix multiplication with the random projection matrix; applying a sign function to the result of the matrix multiplication to quantize it into binary code.
5. The lightweight airborne visual tracking method based on random projection according to claim 4, characterized in that, The random projection matrix is pre-generated and fixed before model training, or optimized as a learnable parameter during model training.
6. The lightweight airborne visual tracking method based on random projection according to any one of claims 1-3 or 5, characterized in that, The global association branch calculates global similarity through matrix multiplication, while the local matching branch calculates local similarity between the cropped bipolar template features and the search features through matrix inner product by using the cropped bipolar template features as a sliding window.
7. The lightweight airborne visual tracking method based on random projection according to any one of claims 1-3 or 5, characterized in that, During model training, a joint loss function is used for optimization. The joint loss function includes at least the cross-entropy loss for classification and the IoU loss for bounding box regression.
8. A lightweight airborne visual tracking device based on random projection, characterized in that, include: Memory, used to store computer programs; A processor for executing the computer program to implement the lightweight airborne visual tracking method based on random projection as described in any one of claims 1 to 7.
9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the lightweight airborne visual tracking method based on random projection as described in any one of claims 1 to 7.