Camouflage target tracking method based on multilevel feature perception
By introducing multi-level feature-aware encoder and prediction heads of full convolutional networks in target tracking, the accuracy and stability problems of traditional algorithms in camouflage target tracking are solved, and high-precision tracking effect is achieved in complex environments.
Patent Information
- Application Number
- CN202411130758.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-16
- Publication Date
- 2024-12-13
AI Technical Summary
Traditional target tracking algorithms are difficult to maintain high tracking accuracy when processing camouflage targets, especially in complex scenarios. Existing CNN-based methods are limited to local feature capture and insufficient global context information, resulting in unstable tracking.
Using a multi-level feature-aware encoder based on Vision Transformer (ViT), the stacked CEBlocks and multi-layer perceptron (MLP) enhancement feature representation is achieved by combining the prediction heads of the full convolutional network to achieve high-precision tracking of the camouflage target.
It improves the tracking accuracy and robustness of camouflage targets, and can achieve stable and accurate tracking in complex environments. It is suitable for application scenarios such as military reconnaissance, security monitoring and wildlife monitoring.
Smart Images
Figure CN119131345A_ABST
Abstract
Description
Technical Field
[0001] The invention relates to the field of target tracking, and in particular to a camouflaged target tracking method based on multi-level feature perception. Background Art
[0002] Object tracking technology is crucial for many key fields such as video surveillance, military reconnaissance, environmental monitoring, and wildlife monitoring. Camouflaged targets, due to their high similarity to the surrounding environment, pose significant challenges to tracking algorithms. Traditional tracking algorithms that rely on hand-crafted features and simple models often have difficulty maintaining high tracking accuracy when faced with complex scenes such as fast movement, lighting changes, or occlusion.
[0003] Although deep learning methods based on convolutional neural networks (CNNs) have made breakthroughs in automatic feature extraction and improving target tracking accuracy, they are still limited by the capture of local features and insufficient global context information when dealing with camouflaged targets. These limitations are particularly evident when the camouflaged target is highly integrated with the background, making it difficult for existing technologies to achieve stable and accurate tracking.
[0004] To overcome these limitations, researchers began to explore models based on Vision Transformer (ViT). ViT can effectively capture long-range dependencies and global contextual information in images through the self-attention mechanism, providing a new perspective for camouflaged target tracking. However, ViT still faces challenges when dealing with camouflaged targets, especially in achieving high tracking accuracy while maintaining the real-time performance of the algorithm.
[0005] The present invention aims to propose a camouflaged target tracking method based on multi-level feature perception. This method improves the tracking accuracy and robustness of camouflaged targets by combining an improved ViT architecture with a multi-level feature perception encoder. This method particularly optimizes the perception capability of camouflaged target features, enabling stable and accurate tracking even in complex environments, and is suitable for a variety of application scenarios such as military reconnaissance, security monitoring, and wildlife monitoring. Summary of the invention
[0006] The purpose of the present invention is to provide a camouflaged target tracking method based on multi-level feature perception, aiming to improve the tracking accuracy and robustness of the camouflaged target by extracting and fusing multi-level features.
[0007] The technical solution of the present invention is to design and train the proposed model, and use the model to test on a disguised data set.
[0008] The ViT-based camouflage target tracking framework is as follows Figure 1 As shown in Figure 2, the framework consists of a backbone network based on multi-level feature perception ViT and a prediction head.
[0009] (1) Backbone network
[0010] The backbone network adopts the ViT architecture and integrates the early candidate elimination module to form a multi-level feature-aware encoder. The input target template Z and search image X are cut into small blocks of the same size (16×16), and then tokenized through a trainable linear layer to produce K vectors, which are represented as:
[0011] (1)
[0012] Where d represents the embedding dimension of each vector, and the vector sequence and denote the template and the search image respectively, where .
[0013] The core idea of this framework is the multi-level feature-aware encoder. This structure consists of stacked CEBlocks, and we introduce a multi-level feature-aware structure to process the multi-level features output by the ViT block. Features from all levels are directly combined through splicing, allowing the model to use detailed information from lower levels and abstract information from higher levels at the same time. A multi-layer perceptron (MLP) effectively reduces the dimensionality of these multi-level features, using three linear layers and ReLU activation functions to compress and optimize feature representations. Through reshaping and permutation, this module adapts the data to the linear layer, successively reducing the dimensionality from 12 to 6, to 3, and finally to 1. This series of reductions is supplemented by ReLU activation, which introduces nonlinearity and enhances the model's ability to capture complex feature patterns. Considering that the early candidate elimination module in each CEBlock may change the shape of the input features, we divide CEBlock into two parts: one part represents the ViT block with early candidate elimination ( ), and the other part represents ViT blocks without early candidate elimination ( ). The corresponding output features are respectively used and Represents. The multi-level feature perception encoder is defined as:
[0014] (2)
[0015] in , , Indicates that and Aligned padding and reshaping, ∈ is the equilibrium aggregation characteristic and The weighting constant of importance. MLP stands for Multi-Layer Perceptron Network. Therefore, our multi-level feature-aware encoder enhances feature representation by capturing information at multiple levels.
[0016] (2) Prediction head
[0017] Use a prediction head based on a fully convolutional network , contains three branches, each branch consists of 4 stacked convolution-batch normalization-ReLU layers to estimate the bounding box of the target. Cut the part of the search image from the vector output by the backbone network And reinterpreted as a 2D spatial feature map and input into the prediction head. The result is the target classification score , local offset , and the normalized bounding box size (in Respectively represent the height and width of the search image, represents the side length of the small blocks into which the image is cut). The initial estimate of the position is determined by the maximum classification score, expressed as . Then based on this rough position, the predicted target bounding box is calculated as:
[0018] (3) BRIEF DESCRIPTION OF THE DRAWINGS
[0019] Figure 1 Camouflaged target tracking framework based on multi-level feature perception
[0020] Figure 2 Multi-level feature-aware encoder architecture
[0021] Figure 3 Prediction box visualization
[0022] Figure 4 Camouflaged target tracking test DETAILED DESCRIPTION
[0023] The present invention is a camouflaged target tracking method based on multi-level feature perception, and the specific steps are as follows:
[0024] (1) First, prepare the training datasets, which include GOT-10k, LaSOT, COCO, and TrackingNet. These four datasets are very famous datasets in the field of target tracking.
[0025] (2) Create a camouflaged target tracking framework. The backbone network in this framework uses DeiT-Tiny, which is a network model based on ViT. The prediction head consists of 4 stacked convolution-batch normalization-ReLU layers.
[0026] (3) This framework includes two inputs, namely the template and the search image, whose sizes are 128×128 and 256×256 respectively. The input image is scaled to the specified size. The batch size is set to 32. The AdamW optimizer is used to train the model, and the weight decay is set to The initial learning rate is A total of 300 rounds of training were conducted, with 60,000 image pairs input in each round, and the learning rate was reduced by a factor of 10 after 240 rounds. After training with our multi-level feature perception module, the model can recognize objects more accurately.
[0027] (4) Prepare a test dataset, the test set is COTD. We carefully prepared a test dataset called COTD as a benchmark for camouflaged target tracking tests. This dataset contains videos shot in various complex environments and is specifically designed to evaluate the performance of camouflaged target tracking algorithms. The video materials cover the violent movements of camouflaged targets in different natural and artificial environments, including diverse environments such as the wild, urban, and ocean. The scenes are full of cluttered interference objects, such as vegetation, buildings, and other objects, which increase the difficulty of the tracking task. In addition, scenes simulating military reconnaissance and ecological research are specially added to improve the actual combat simulation and application potential of the test. The COTD dataset provides researchers with a comprehensive platform for testing and comparing the performance of camouflaged target tracking algorithms under diverse and challenging conditions. The first frame of each video in the test set is used as a template, and each subsequent frame is used as a search image. They are sequentially fed into the framework, and the output result is the predicted box of each frame. The predicted box can be displayed on the image through the visualization output, such as Figure 3 shown.
[0028] (5) A target tracking method with a multi-level feature-aware encoder to track camouflaged targets, Figure 4 The test results of 3 sequences are given.
Claims
1. A camouflaged target tracking method based on multi-level feature perception, Features: S1: Collect camouflaged target video sequences, classify all sequences according to target categories, and annotate each frame; S2: A single-stream single-stage architecture is used as the model framework. The input of this framework includes the target template to be tracked and the search image. S3: The framework includes the backbone network and the prediction head; S4: The backbone network uses a ViT-based network model DeiT-Tiny, and the input is segmented and flattened by the target template. vectors and segmented and flattened by the search image vector, the output is a 8×8 feature map; S5: The backbone network introduces a multi-level feature-aware encoder to process the multi-level features output by the ViT block; the multi-level feature-aware encoder is defined as: (1) in , , Indicates that and Aligned padding and reshaping, ∈ is the equilibrium aggregation characteristic and The weighted constant of importance; MLP stands for multi-layer perceptron network; S6: CEBlock is divided into two parts: one part represents the ViT block with early candidate elimination ( ), and the other part represents ViT blocks without early candidate elimination ( ) ; S7: Multilayer Perceptron (MLP) effectively reduces the dimensionality of multi-level features, using three linear layers and ReLU activation function to compress and optimize feature representation; S8: The prediction head has three branches, which are used to predict the classification score, predict the downsampling offset value, and predict the normalized bounding box size. Each branch consists of four stacked convolution-batch normalization-ReLU.