Object tracking method combining hierarchical encoder and parallel attention mechanism

Abstract

The present invention relates to the field of computer vision technology, and particularly to a target tracking method combining a hierarchical encoder and a parallel attention mechanism. The method comprises the following steps: preprocessing a video sequence to obtain an initial template image and an initial search image; extracting features from the initial template image and the initial search image using an improved feature extraction backbone network VGG16 to obtain template features and search features; processing the template features and the search features through a parallel attention mechanism to obtain two features; fusing the obtained two features and inputting them into the last two convolution blocks of the improved feature extraction backbone network VGG16 and a DWConv‑Transformer encoder for processing to obtain final template features and final search features; performing a cross-correlation convolution on the final template features after processing through the parallel attention mechanism and the final search features to obtain a similarity score map of the two, and mapping the coordinates of the maximum point in the similarity score map back to the original image to obtain a tracking result.

Description

Technical Field

[0001] This invention relates to the field of computer vision technology, and in particular to a single-target tracking method using a Siamese network that combines a Transformer encoder and a parallel attention mechanism. Background Technology

[0002] Target tracking technology, a popular and challenging research area in computer vision, is characterized by its focus on predicting target trajectories through modeling their appearance and motion. This technology allows for the acquisition of future target location information, leading to its widespread application in fields such as intelligent traffic monitoring, intelligent human-computer interaction, and military reconnaissance.

[0003] Some works apply Siamese networks to tracking tasks, transforming the tracking problem into a similarity-based problem, greatly simplifying the solution process. Others introduce region proposal networks into the tracking framework, using them for foreground and background classification and bounding box regression, effectively improving the accuracy of bounding box prediction. Still others improve model training by utilizing effective sampling strategies to control the distribution of the training dataset. Finally, some works enhance tracker performance by improving the feature extraction backbone network. However, most Siamese network-based tracking methods use a single feature processing approach, leading to tracking drift when facing complex tracking environments such as occlusion, lighting changes, and rapid movement. Summary of the Invention

[0004] The purpose of this invention is to provide a target tracking method that combines a hierarchical encoder and a parallel attention mechanism, aiming to solve the tracking drift problem of existing trackers in complex tracking environments, including the following steps:

[0005] The video sequence is preprocessed to obtain the initial template image and the initial search image;

[0006] The improved feature extraction backbone network VGG16 is used to extract features from the initial template image and the initial search image to obtain template features and search features.

[0007] Based on the template features and the search features, a parallel attention mechanism is used to focus on the weight differences of the target in different feature channels and the dependencies between feature channels, to obtain channel features with channel information weights and spatial features with location information weights.

[0008] The channel features containing channel information weights and the spatial features containing location information weights are fused and then input into the last two convolutional blocks of the improved feature extraction backbone network VGG16 and the DWConv-Transformer encoder for processing to obtain the final template features and the final search features.

[0009] The final template features are processed by the parallel attention mechanism and then cross-correlatedly convolved with the final search features to obtain a similarity score map. The coordinates of the maximum point in the similarity score map are then used to map back to the original image to obtain the tracking result.

[0010] The preprocessing includes cropping and RGB mean fill.

[0011] The step of using the improved feature extraction backbone network VGG16 to extract features from the initial template image and the initial search image to obtain template features and search features includes:

[0012] The original feature extraction backbone network VGG16 was optimized to obtain the improved feature extraction backbone network VGG16.

[0013] The initial template image and the initial search image are input into the improved feature extraction backbone network VGG16 to obtain template features and search features.

[0014] The optimization of the original feature extraction backbone network VGG16 to obtain the improved feature extraction backbone network VGG16 includes:

[0015] The original 5-layer max pooling of the original feature extraction backbone network VGG16 is reduced to 3 layers. A pruning module is introduced to remove the outermost layer features. Dilated convolutions are introduced in the fourth and fifth layers of the network to obtain the improved feature extraction backbone network VGG16.

[0016] Specifically, the step of focusing on the weight differences of the target in different feature channels and the dependencies between feature channels through a parallel attention mechanism based on the template features and the search features to obtain channel features containing channel information weights and spatial features containing location information weights includes:

[0017] The template features and the search features are input into the channel attention mechanism. Based on the response degree of different channels of the feature map to different target descriptors, different weights are adaptively assigned to different feature channels to adjust the importance of channels to different targets, thereby obtaining channel features containing channel information weights.

[0018] The template features and the search features are input into a spatial attention mechanism to describe the dependencies between information within the feature channels, obtain information weights at different spatial locations in the feature map, and obtain spatial features containing location information weights.

[0019] The step of inputting the template features and the search features into a channel attention mechanism, and adaptively assigning different weights to different feature channels based on the response of different channels of the feature map to different target descriptors, adjusts the importance of channels to different targets, resulting in channel features containing channel information weights, including:

[0020] The input template features and search features are subjected to a two-dimensional adaptive average pooling operation to obtain a description matrix for the channel dimension;

[0021] The description matrix is ​​transformed and processed by one-dimensional convolution to obtain the processing matrix;

[0022] The channel information weights obtained by transforming the processing matrix into a matrix reshape and applying the sigmoid activation function are assigned to the input template features and the search features to obtain channel features containing channel information weights.

[0023] The step of inputting the template features and the search features into a spatial attention mechanism to describe the dependencies between information within the feature channels, obtaining information weights at different spatial locations in the feature map, and obtaining spatial features containing location information weights includes:

[0024] The input template features and the search features are subjected to average pooling and max pooling operations, respectively, to obtain the first pooling features and the second pooling features.

[0025] The first pooling feature and the second pooling feature are stacked to obtain a stacked feature;

[0026] The template features and the search features are multiplied by the stacked features after passing them through a standard convolution to obtain the product features;

[0027] The product features are passed through a convolutional layer with a kernel size of 7×7 and a sigmoid activation function to obtain spatial features with location information weights.

[0028] The improved feature extraction backbone network VGG16 consists of three convolutional layers with a kernel size of 3×3 and two convolutional layers with a kernel size of 1×1 in its last two convolutional blocks.

[0029] Specifically, the process of fusing the channel features with channel information weights and the spatial features with location information weights, and then inputting them into the last two convolutional layers of the improved feature extraction backbone network VGG16 and the DWConv-Transformer encoder for processing, yields the final template features and the final search features, including:

[0030] The channel features containing channel information weights and the spatial features containing location information weights are fused and downsampled by a standard convolution before being input into the DWConv-Transformer encoder. In the DWConv-Transformer encoder, a depthwise separable convolution with a preset stride and a kernel size of 3×3 is used to perform convolutional projection mapping and obtain the query, key, and value through matrix transformation.

[0031] The query, the key, and the value are used as inputs to the multi-head attention module. The similarity between the query and the key is calculated using the Einstein summation convention. Then, after passing through the Softmax function, the attention weight matrix is ​​obtained. Finally, the value is multiplied by the attention weight matrix to obtain a vector with attention weights.

[0032] The vector with attention weights is processed by Layer Normalization and Multilayer Perceptron (MLP), then added to its own residual, and finally processed by matrix transformation to obtain the processed features.

[0033] The channel features containing channel information weights and the spatial features containing location information weights are fused and then input into the last two convolutional blocks of the improved feature extraction backbone network VGG16 to obtain the output features.

[0034] The processed features and the output features are added element-wise to obtain the final template features and the final search features.

[0035] This invention discloses a target tracking method combining a hierarchical encoder and a parallel attention mechanism. The method preprocesses a video sequence to obtain an initial template image and an initial search image. An improved feature extraction backbone network (VGG16) is used to extract features from the initial template image and the initial search image, resulting in template features and search features. Based on the template features and the search features, a parallel attention mechanism focuses on the weight differences of the target in different feature channels and the dependencies between feature channels, resulting in channel features with channel information weights and spatial features with location information weights. The channel features with channel information weights and the spatial features with location information weights are fused and then input into the last two convolutional layers of the improved feature extraction backbone network (VGG16) and the DWConv-Transformer encoder for processing, resulting in final template features and final search features. The final template features are processed by the parallel attention mechanism and then cross-correlatedly convolved with the final search features to obtain a similarity score map. The coordinates of the maximum point in the similarity score map are then used to map back to the original image to obtain the tracking result. This invention provides a target tracking method that combines a hierarchical encoder and a parallel attention mechanism, enabling accurate tracking in complex scenarios such as changes in lighting and interference from similar objects. Attached Figure Description

[0036] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0037] Figure 1 This is a flowchart illustrating the steps of a target tracking method combining a hierarchical encoder and a parallel attention mechanism provided by the present invention.

[0038] Figure 2 This is a schematic diagram of a target tracking method that combines a hierarchical encoder and a parallel attention mechanism, provided by the present invention.

[0039] Figure 3 yes Figure 2 The principle block diagram of the DWConv-TransformerBlock encoder.

[0040] Figure 4 yes Figure 2 Block diagram of SAM principle.

[0041] Figure 5 yes Figure 2 Block diagram of CAM principle.

[0042] Figure 6 This is a flowchart of a target tracking method that combines a hierarchical encoder and a parallel attention mechanism, provided by the present invention.

[0043] Figure 7 This is a performance comparison of the method provided by this invention with other methods on the OTB100 dataset. Detailed Implementation

[0044] Embodiments of the present invention are described in detail below, examples of which are illustrated in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain the present invention, and should not be construed as limiting the present invention.

[0045] Please see Figures 1 to 7 This invention provides a target tracking method combining a hierarchical encoder and a parallel attention mechanism, comprising the following steps:

[0046] S1 preprocesses the video sequence to obtain the initial template image and the initial search image;

[0047] Specifically, the preprocessing includes cropping and RGB mean fill.

[0048] The initial template image and search image are obtained through preprocessing operations such as cropping and RGB mean fill in the input video sequence. Specifically, the first frame of the input video sequence is taken as the initial target template, and cropping is performed with the target as the center. When the cropping range exceeds the image range, the excess part is filled with a constant by taking the RGB mean. The preprocessed template image has a resolution of 127×127 and a channel dimension of 3. The search image is also obtained from other frames of the video sequence in the same way, and its final resolution is 255×255 with a channel dimension of 3.

[0049] S2 uses the improved feature extraction backbone network VGG16 to extract features from the initial template image and the initial search image to obtain template features and search features;

[0050] VGG16 is a deep neural network used for feature processing of input images;

[0051] The specific method is as follows:

[0052] S21 optimizes the original feature extraction backbone network VGG16 to obtain the improved feature extraction backbone network VGG16.

[0053] Specifically, the original 5-layer max pooling of the original feature extraction backbone network VGG16 is reduced to 3 layers, a pruning module is introduced to remove the outermost features, and dilated convolutions are introduced in the fourth and fifth layers of the network to obtain the improved feature extraction backbone network VGG16.

[0054] Optimizations for VGG16: 1) The original 5-layer max pooling was reduced to 3 layers to avoid excessively long network strides leading to low resolution of output features, which would affect tracker performance; 2) A pruning module was introduced to remove the outermost features affected by padding operations, reducing the impact of potential positional bias caused by padding operations; 3) Dilated convolutions were introduced in the fourth and fifth layers of the network to alleviate the problem of insufficient receptive field caused by shortened network strides.

[0055] Dilated convolution supports exponential expansion of the receptive field without loss of resolution. The receptive field of each element is calculated using the following formula:

[0056] F i+1 =F i *2 i k i i = 0, 1, ..., n-2

[0057] Where k is a 3×3 sliding filter, and F represents the features output after convolution. Analysis of the above formula shows that the receptive field size of each element in the feature layer is based on the convolution result of the previous layer. When using a dilatancy ratio of 2... i The filter (i = 0, 1, 2…) in F i After the layer undergoes a convolution operation, F i+1 The size of the receptive field of each element in the layer is expressed by the following formula:

[0058]

[0059] Therefore, introducing dilated convolutions into the backbone network VGG16 can effectively alleviate the problem of insufficient receptive field caused by shortening the network stride.

[0060] S22 inputs the initial template image and the initial search image into the improved feature extraction backbone network VGG16 to obtain template features and search features.

[0061] Specifically, the template image and search image, which have been preprocessed by cropping and mean RGB padding, are input into the VGG16 network. After processing through the first three convolutional blocks, the template feature resolution is 11×11 and the channel dimension is 256. The search feature resolution is 27×27 and the channel dimension is 25. The first three convolutional blocks include 7 convolutional layers with a kernel size of 3×3 and 3 max pooling layers.

[0062] S3 focuses on the weight differences of the target in different feature channels and the dependency relationship between feature channels through a parallel attention mechanism based on the template features and the search features, to obtain channel features with channel information weights and spatial features with location information weights.

[0063] The specific method is as follows:

[0064] S31 inputs the template features and the search features into the channel attention mechanism, and adaptively assigns different weights to different feature channels according to the response degree of different channels of the feature map to different target descriptors, so as to adjust the importance of the channels to different targets, and obtains channel features containing channel information weights;

[0065] Specifically, a two-dimensional adaptive average pooling operation is used to obtain a channel-dimensional description matrix from the input template features and search features; the description matrix is ​​then processed by matrix transformation and one-dimensional convolution to obtain a processing matrix; the channel information weights obtained by reshaping the processing matrix and applying a sigmoid activation function are assigned to the input template features and search features to obtain channel features containing channel information weights.

[0066] The Channel Attention Mechanism (CAM) adjusts the importance of different channels for different targets by adaptively assigning different weights to different feature channels based on their response to different target descriptors. This applies to the input features. First, a two-dimensional adaptive average pooling operation is used to obtain the description matrix of the channel dimension.

[0067]

[0068] Where h and w are the height and width of the feature map, and (i,j) is the input feature map F. c The pixel position on the a. c Obtained through matrix transformation and one-dimensional convolution. One-dimensional convolution is used to exchange information between different channels, and the number of channels N that can be exchanged is also adaptively adjusted according to the number of input channels.

[0069]

[0070] Where C is the number of input feature channels. Let α represent the odd number closest to x, and β = 1.

[0071] b c The channel information weights obtained after matrix transformation reshape and sigmoid activation function are assigned to the input feature F. cObtain the final channel self-attention output

[0072]

[0073] S32 inputs the template features and the search features into the spatial attention mechanism to describe the dependencies between information within the feature channels, obtains the information weights of different spatial locations in the feature map, and obtains spatial features containing location information weights.

[0074] Specifically, the input template features and search features are subjected to average pooling and max pooling operations, respectively, to obtain first pooling features and second pooling features; the first pooling features and the second pooling features are stacked to obtain stacked features; the template features and the search features are multiplied by the stacked features through a standard convolution to obtain product features; the product features are passed through a convolutional layer with a kernel size of 7×7 and a sigmoid activation function to obtain spatial features containing positional information weights.

[0075] Spatial Attention Mechanism (SAM) describes the dependencies between information within feature channels, extracting information weights from different spatial locations in the feature map. For input features... Where C, H, and W are the number of channels, height, and width of the input feature, respectively. First, consider the input feature F... s Performing average pooling and max pooling operations respectively yields the results. and Then F A and F M Stacking Where r is the channel reduction factor. F s Then pass through a standard convolutional layer with a kernel size of 1×1 and then with a s Element-wise multiplication yields

[0076]

[0077] b s Spatial features containing location-information weights are obtained by passing through a 7×7 convolutional layer and a sigmoid activation function.

[0078]

[0079] Where ω ij The information weights at position (i,j) on the spatial attention feature represent the information weights.

[0080] As attached Figure 4As shown, the input feature F in The dimensions are C×H×W, and F is obtained after processing by the Spatial Attention Mechanism (SAM). out The dimensions are 1×H×W, and the specific calculation process can be described as follows:

[0081]

[0082] Where conv 7×7 and conv 1×1 These represent convolutional layers with kernel sizes of 7×7 and 1×1, respectively, and Cat represents feature stacking.

[0083] S4 fuses the channel features with channel information weights and the spatial features with location information weights, and then inputs them into the last two convolutional blocks of the improved feature extraction backbone network VGG16 and the DWConv-Transformer encoder for processing to obtain the final template features and the final search features.

[0084] The specific method is as follows:

[0085] S41 fuses the channel features containing channel information weights and the spatial features containing position information weights, then downsamples them through a standard convolution and inputs them into the DWConv-Transformer encoder. In the DWConv-Transformer encoder, a depthwise separable convolution with a preset stride and a kernel size of 3×3 is used to perform convolutional projection mapping and obtain the query, key, and value through matrix transformation.

[0086] The improved Transformer structure was named DWConv-Transformer.

[0087] For details, see attached. Figure 3 The specific processing steps of the features processed by the first parallel attention module, which are then input into the DWConv-Transformer encoder through a convolutional layer, are as follows:

[0088] The features processed by parallel attention are each used with a step size S. q =1,S v =S k =2, the convolution kernel size is 3×3, the depthwise separable convolution is performed, convolution projection mapping is carried out, and the query Q, key K and value V are obtained through matrix transformation. The calculation process is as follows:

[0089]

[0090] Where DWConv represents depthwise separable convolution, conv3×3 denoted as standard convolution, and F represents the input feature.

[0091] S42 takes the query, the key, and the value as input to the multi-head attention module, calculates the similarity between the query and the key using the Einstein summation convention, then obtains the attention weight matrix after passing through Softmax, and finally performs a dot product between the value and the attention weight matrix to obtain a vector with attention weights.

[0092] Specifically, Q, K, and V are used as inputs to the multi-head attention module. The similarity between Q and K is calculated using the Einstein summation convention. Then, after passing through Softmax, the attention weight matrix is ​​obtained. Finally, the value V is multiplied by the attention weight matrix to obtain a vector with attention weights. The calculation process is as follows:

[0093]

[0094] Where, d k It is the scaling factor of the attention weight matrix.

[0095] S43 processes the attention-weighted vector through Layer Normalization and Multilayer Perceptron (MLP), adds it to its own residual, and finally obtains the processed features through matrix transformation.

[0096] Specifically, the vectors obtained by the multi-head attention module are processed by LayerNormalization and Multilayer Perceptron (MLP), and then the residuals are added to themselves. Finally, through matrix transformation, processed features with the same resolution and channel dimension as the original features input to the DWConv-Transformer encoder are obtained.

[0097] S44 fuses the channel features containing channel information weights and the spatial features containing location information weights, and then inputs them into the last two convolutional blocks of the improved feature extraction backbone network VGG16 to obtain the output features;

[0098] Specifically, the features processed by parallel attention are input into the last two convolutional blocks of the backbone network VGG16, resulting in template features with a resolution of 5×5 and a channel dimension of 256, and search features with a resolution of 21×21 and a channel dimension of 256. The last two convolutional blocks consist of three convolutional layers with a kernel size of 3×3 and two convolutional layers with a kernel size of 1×1.

[0099] S45 adds the processed features and the output features element by element to obtain the final template features and the final search features.

[0100] Specifically, the features obtained by the multi-layer DWConv-Transformer encoder are added element-wise with the features output by the last two convolutional blocks of VGG16 to obtain the final template features and search features.

[0101] S5 processes the final template features through the parallel attention mechanism and performs a cross-correlation convolution with the final search features to obtain a similarity score map between the two. The coordinates of the maximum point in the similarity score map are then used to map back to the original image to obtain the tracking result.

[0102] Specifically, the final template features obtained in step S3 are processed by the parallel attention mechanism to increase the template's representational ability; and a cross-correlation convolution is performed with the final search features to obtain a similarity score map of the two. The coordinates of the maximum point in the similarity score map are then used to map back to the original image to obtain the tracking result.

[0103] This invention designs a target tracking method that combines a hierarchical encoder and a parallel attention mechanism. The key technical means are as follows:

[0104] 1. This invention improves VGG16 as the feature extraction backbone network, and designs pruning blocks to eliminate potential positional biases caused by padding operations in the network. It also combines dilated convolution to alleviate the problem of insufficient receptive field caused by shortening the network stride. The improved feature extraction backbone network of this invention can fully mine the effective target information in the features and enhance the feature learning ability.

[0105] 2. The parallel attention mechanism designed in this invention focuses on the weight differences of the target in different feature channels and the dependencies between feature channels, thereby adjusting the tracker's focus and enhancing the expressive power of the features.

[0106] 3. This invention utilizes the ability of hierarchical Transformer structures to capture long-term temporal context information, combined with the ability of convolutional networks to capture local context information, to improve the robustness of the tracker in complex environments. It replaces the linear mapping in the Transformer encoder with convolutional mapping and replaces the standard convolution with depthwise separable convolution, thereby reducing the introduction of additional parameters.

[0107] 4. This invention designs a target tracking method that combines a hierarchical encoder and a parallel attention mechanism. Tracking performance is evaluated on the public dataset OTB100, showing significant improvements in both tracking precision and tracking success rate (AUC), outperforming mainstream algorithms such as SiamFC, SiamRPN, and DaSiamRPN. The results are shown in the appendix. Figure 7 Ours represents the performance results obtained by the method provided in this invention.

[0108] The above-disclosed embodiments are merely preferred embodiments of a target tracking method combining a hierarchical encoder and a parallel attention mechanism according to the present invention. Of course, they should not be construed as limiting the scope of the present invention. Those skilled in the art can understand that implementing all or part of the above embodiments and making equivalent changes in accordance with the claims of the present invention are still within the scope of the invention.

Claims

1. A target tracking method combining a hierarchical encoder and a parallel attention mechanism, characterized in that, Includes the following steps: The video sequence is preprocessed to obtain the initial template image and the initial search image; The original 5-layer max pooling of the feature extraction backbone network VGG16 is retained as 3 layers. A pruning module is introduced to prune the outermost layer features. Dilated convolutions are introduced in the fourth and fifth layers of the network to obtain the improved feature extraction backbone network VGG16. The initial template image and the initial search image are input into the improved feature extraction backbone network VGG16 to obtain template features and search features. Based on the template features and the search features, a parallel attention mechanism is used to focus on the weight differences of the target in different feature channels and the dependencies between feature channels, to obtain channel features with channel information weights and spatial features with location information weights. The channel features containing channel information weights and the spatial features containing location information weights are fused and then input into the last two convolutional blocks of the improved feature extraction backbone network VGG16 and the DWConv-Transformer encoder for processing to obtain the final template features and the final search features. The final template features are processed by the parallel attention mechanism and then cross-correlated convolution is performed with the final search features to obtain a similarity score map between the two. The coordinates of the maximum point in the similarity score map are then used to map back to the original image to obtain the tracking result. The process involves fusing the channel features with channel information weights and the spatial features with location information weights, and then inputting them into the last two convolutional layers of the improved feature extraction backbone network VGG16 and the DWConv-Transformer encoder for processing to obtain the final template features and the final search features, including: The channel features containing channel information weights and the spatial features containing location information weights are fused and downsampled by a standard convolution before being input into the DWConv-Transformer encoder. In the DWConv-Transformer encoder, a depthwise separable convolution with a preset stride and a kernel size of 3×3 is used to perform convolutional projection mapping and obtain the query, key, and value through matrix transformation. The query, the key, and the value are used as inputs to the multi-head attention module. The similarity between the query and the key is calculated using the Einstein summation convention. Then, after passing through Softmax, the attention weight matrix is ​​obtained. Finally, the value is multiplied by the attention weight matrix to obtain a vector with attention weights. The vector with attention weights is processed by Layer Normalization and a multilayer perceptron, and then added to itself as a residual. Finally, the processed features are obtained through matrix transformation. The channel features containing channel information weights and the spatial features containing location information weights are fused and then input into the last two convolutional blocks of the improved feature extraction backbone network VGG16 to obtain the output features. The processed features and the output features are added element-wise to obtain the final template features and the final search features.

2. The target tracking method combining hierarchical encoder and parallel attention mechanism as described in claim 1, characterized in that, The preprocessing includes cropping and RGB mean fill.

3. The target tracking method combining hierarchical encoder and parallel attention mechanism as described in claim 2, characterized in that, The process, based on the template features and the search features, uses a parallel attention mechanism to focus on the weight differences of the target in different feature channels and the dependencies between feature channels, obtaining channel features with channel information weights and spatial features with location information weights, including: The template features and the search features are input into the channel attention mechanism. Based on the response degree of different channels of the feature map to different target descriptors, different weights are adaptively assigned to different feature channels to adjust the importance of channels to different targets, thereby obtaining channel features containing channel information weights. The template features and the search features are input into a spatial attention mechanism to describe the dependencies between information within the feature channels, obtain information weights at different spatial locations in the feature map, and obtain spatial features containing location information weights.

4. The target tracking method combining hierarchical encoder and parallel attention mechanism as described in claim 3, characterized in that, The process involves inputting the template features and the search features into a channel attention mechanism. Based on the response levels of different channels in the feature map to different target descriptors, different weights are adaptively assigned to different feature channels to adjust the importance of each channel to different targets, resulting in channel features containing channel information weights. This includes: The input template features and search features are subjected to a two-dimensional adaptive average pooling operation to obtain a description matrix for the channel dimension; The description matrix is ​​transformed and processed by one-dimensional convolution to obtain the processing matrix; The channel information weights obtained by transforming the processing matrix into a matrix reshape and applying the sigmoid activation function are assigned to the input template features and the search features to obtain channel features containing channel information weights.

5. The target tracking method combining hierarchical encoder and parallel attention mechanism as described in claim 4, characterized in that, The step of inputting the template features and the search features into a spatial attention mechanism to describe the dependencies between information within the feature channels, obtaining information weights at different spatial locations in the feature map, and obtaining spatial features containing location information weights includes: The input template features and the search features are subjected to average pooling and max pooling operations, respectively, to obtain the first pooling features and the second pooling features. The first pooling feature and the second pooling feature are stacked to obtain a stacked feature; The template features and the search features are multiplied by the stacked features after passing them through a standard convolution to obtain the product features; The product features are passed through a convolutional layer with a kernel size of 7×7 and a sigmoid activation function to obtain spatial features with location information weights.

6. The target tracking method combining hierarchical encoder and parallel attention mechanism as described in claim 5, characterized in that, The last two convolutional blocks of the improved feature extraction backbone network VGG16 include three convolutional layers with a kernel size of 3×3 and two convolutional layers with a kernel size of 1×1.

Images