Adaptive region selection feature fusion method, target tracking method and system

Abstract

The present disclosure relates to the technical field of target tracking, and proposes an adaptive region selection feature fusion method, a target tracking method and a system. Two sets of region predictors are designed for template features and search features respectively, the template foreground region features and the search target potential region features are extracted, and a feature extraction and adaptive region selection feature fusion network is proposed. In the feature extraction process, the complex attention mechanism is limited within the target region range of the two sets of feature maps, the tracking efficiency of the Transformer target tracking network is improved, and excellent tracking performance is maintained.

Description

Technical Field

[0001] This disclosure relates to the field of target tracking technology, specifically to an adaptive region selection feature fusion method, target tracking method and system. Background Technology

[0002] The statements in this section are merely background information relating to this disclosure and do not necessarily constitute prior art.

[0003] In early applications of Transformer networks in computer vision tasks, they were primarily used as auxiliary modules, combined with convolutional neural networks, to supplement global information about image features. However, with continuous research into visual Transformer architectures, more and more works have begun to explore the possibility of Transformers replacing convolutional neural networks as a general backbone network for computer vision tasks, demonstrating good results, particularly in fundamental tasks such as image classification. In the field of object tracking, a method has emerged that rebuilds the backbone network based on Transformers for feature extraction. This method introduces correlation calculations between template features and search features during the feature extraction stage. This approach can extract feature maps specific to the target being tracked, capturing more discriminative information belonging to the target, thus achieving excellent tracking results.

[0004] The inventors discovered that in the field of target tracking, the use of Transformer for feature extraction generates multi-level and multi-scale feature maps. When introducing correlation calculation based on attention mechanism between multi-level feature maps, the computational overhead of the attention module is large, which leads to an increase in the number of network parameters and a slowdown in running speed, greatly reducing the real-time performance of target tracking. Summary of the Invention

[0005] To address the aforementioned issues, this disclosure proposes an adaptive region selection feature fusion method, target tracking method, and system. By recombining template feature sequences and search feature sequences through depthwise separable convolution, different attention calculation methods are designed for different features, and target-corresponding feature sub-maps are extracted from the two sets of features, achieving accurate target information integration and efficient and accurate target tracking.

[0006] To achieve the above objectives, the present disclosure adopts the following technical solution:

[0007] One or more embodiments provide a feature fusion method based on adaptive region selection, including the following steps:

[0008] Obtain the template feature map to be processed, and use an adaptive foreground region selection and pooling method to extract the foreground target region features of the template;

[0009] Obtain the search feature map to be processed, and use the search feature adaptive target potential region selection method to obtain the search target potential region features;

[0010] In the feature extraction process, which involves extracting features from template foreground target regions and search target latent regions, a dual attention operation is performed within the target regions of both the template feature map and the search feature map to obtain fused features.

[0011] One or more embodiments provide an adaptive region selection feature fusion system, including: a template feature foreground region predictor, a search feature target latent region predictor, and an adaptive region selection feature fusion module;

[0012] The template feature foreground region predictor is configured to acquire the template feature map to be processed and use an adaptive foreground region selection and pooling method to extract the features of the template foreground target region.

[0013] The search feature target latent region predictor is configured to acquire the search feature map to be processed and adopt the search feature adaptive target latent region selection method to obtain the search target latent region features.

[0014] The adaptive region selection feature fusion module is configured to perform a dual attention operation within the target region range of both the template feature map and the search target feature map during the feature extraction process, targeting the foreground target region features of the template and the potential region features of the search target, to obtain the fused features.

[0015] One or more embodiments provide a target tracking method based on adaptive region selection feature fusion, which employs a multi-stage architecture cascaded network for progressive feature extraction and feature fusion, including the following steps:

[0016] In the first feature processing stage, convolutional embedding is performed on the obtained template image and the search image, and feature fusion is performed by predicting the target region to fuse the two sets of features.

[0017] In the feature processing stage following the first feature processing stage, the fused features output from the previous feature processing stage are received, and after convolutional embedding, the feature fusion method based on adaptive region selection described above is used to perform feature fusion.

[0018] The fused features output from the final feature processing stage are subjected to a full convolution operation to locate the corner points and obtain the bounding box for target tracking.

[0019] One or more embodiments provide a target tracking system based on adaptive region selection feature fusion, constructing a Transformer target tracking network with adaptive region selection feature fusion, including:

[0020] The first feature processing stage network is used to perform convolutional embedding on the obtained template image and the search image, and to fuse the two sets of features by using feature fusion based on target region prediction.

[0021] The high-level feature processing stage network is used to receive the fused features output by the previous-level feature processing stage network, perform convolutional embedding, and then perform feature fusion using the adaptive region selection feature fusion method described in Example 1.

[0022] The fully convolutional corner localization head network is used to perform a fully convolutional operation on the fused features output from the last stage of feature processing to locate corner points and obtain the bounding box for target tracking.

[0023] Compared with the prior art, the beneficial effects of this disclosure are as follows:

[0024] In this disclosure, target region prediction is performed separately for template features and search features. Foreground region features of the template and potential region features of the search target are extracted. A feature extraction and adaptive region selection feature fusion network is proposed. During the feature extraction process, the complex attention mechanism is restricted to the target region of the two sets of feature maps. Target tracking is performed through the fused features, which can improve the tracking efficiency of the target tracking network and maintain excellent tracking performance. It is especially suitable for improving the tracking efficiency of Transformer target tracking networks.

[0025] The advantages of this disclosure, as well as its additional advantages, will be described in detail in the following specific embodiments. Attached Figure Description

[0026] The accompanying drawings, which form part of this disclosure, are used to provide a further understanding of this disclosure. The illustrative embodiments of this disclosure and their descriptions are used to explain this disclosure and do not constitute a limitation thereof.

[0027] Figure 1 This is a schematic diagram of the template feature foreground region predictor of Embodiment 1 of this disclosure;

[0028] Figure 2 This is a comparison chart of the adaptive pooling ratio effect of Embodiment 1 of this disclosure;

[0029] Figure 3 This is a schematic diagram of the search feature target latent region predictor structure of Embodiment 1 of this disclosure;

[0030] Figure 4 This is a schematic diagram of the Adaptive Region Selection Feature Fusion (ARSFM) module structure of Embodiment 1 of this disclosure;

[0031] Figure 5This is a diagram of the Transformer target tracking network framework based on adaptive region selection feature fusion in Embodiment 3 of this disclosure;

[0032] Figure 6 This is a structural diagram of the feature fusion module without region adaptive feature selection in the first feature processing stage network of Embodiment 3 of this disclosure;

[0033] Figure 7(a) Comparison results of the ARSformer network of Embodiment 4 of this disclosure and existing tracking algorithms in the case of low resolution video attributes;

[0034] Figure 7(b) Comparison results of the ARSformer network of Embodiment 4 of this disclosure and existing tracking algorithms when the video attribute is out of the field of view;

[0035] Figure 7(c) Comparison results of the ARSformer network of Embodiment 4 of this disclosure and existing tracking algorithms under the condition of motion blur in video;

[0036] Figure 7(d) Comparison results of the ARSformer network of Embodiment 4 of this disclosure and existing tracking algorithms in the case of fast motion in video;

[0037] Figure 7(e) shows the comparison between the ARSformer network of Embodiment 4 of this disclosure and existing tracking algorithms when the video attribute is deformed;

[0038] Figure 7(f) Comparison results of the ARSformer network of Embodiment 4 of this disclosure and existing tracking algorithms in the case of video attributes with changing illumination;

[0039] Figure 7(g) Comparison results of the ARSformer network of Embodiment 4 of this disclosure and existing tracking algorithms when the video attribute is scale-changing;

[0040] Figure 7(h) shows the comparison results of the ARSformer network of Embodiment 4 of this disclosure and existing tracking algorithms when the video attribute is occlusion;

[0041] Figure 8 Visualization results of the ARSformer network attention mechanism in Embodiment 4 of this disclosure. Detailed Implementation

[0042] The present disclosure will be further described below with reference to the accompanying drawings and embodiments.

[0043] It should be noted that the following detailed descriptions are exemplary and intended to provide further illustration of this disclosure. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this disclosure pertains.

[0044] It should be noted that the terminology used herein is for descriptive purposes only and is not intended to limit the exemplary embodiments according to this disclosure. As used herein, the singular form is intended to include the plural form as well, unless the context clearly indicates otherwise. Furthermore, it should be understood that when the terms "comprising" and / or "including" are used in this specification, they indicate the presence of features, steps, operations, devices, components, and / or combinations thereof. It should be noted that, without conflict, the various embodiments and features within those embodiments can be combined with each other. The embodiments will now be described in detail with reference to the accompanying drawings.

[0045] Explanation of technical terms

[0046] Transformer is a neural network model used to process sequential data. It was proposed in 2017 by Vaswani et al. in the paper "Attention is All You Need". The core idea of ​​Transformer is to use an attention mechanism to calculate the influence of each position in the sequence on other positions. Traditional recurrent neural networks (such as RNN or LSTM) usually combine the input of each position in the sequence with the hidden state of the previous position, while Transformer can directly consider the input of all positions.

[0047] Example 1

[0048] In one or more of the technical solutions disclosed in the embodiments, such as Figures 1 to 4 As shown, the adaptive region selection feature fusion method includes the following steps:

[0049] Step 1: Obtain the template feature map to be processed, and use an adaptive foreground region selection and pooling method to extract the foreground target region features of the template;

[0050] Step 2: Obtain the search feature map to be processed, and use the search feature adaptive target potential region selection method to obtain the search target potential region features;

[0051] Step 3: During the feature extraction process for the template foreground target region features and the search target latent region features, a dual attention operation is performed within the target region range of the template feature map and the search feature map to obtain the fused features.

[0052] In this embodiment, target region prediction is performed on template features and search features respectively. Foreground region features of the template and potential region features of the search target are extracted. A feature extraction and adaptive region selection feature fusion network is proposed. During the feature extraction process, the complex attention mechanism is restricted to the target region of the two sets of feature maps. Target tracking is performed through the fused features, which can improve the tracking efficiency of the target tracking network and maintain excellent tracking performance. It is especially suitable for improving the tracking efficiency of the Transformer target tracking network.

[0053] In this embodiment, the template feature map is an image after feature extraction based on the template image, and the search feature map is an image after feature extraction based on the search image; the template image contains the target object, and the search image is used to identify whether the target object exists and its location in the template image.

[0054] Optionally, in step 1, the template feature adaptive foreground region selection and pooling includes the following steps:

[0055] Step 11: By setting the corner prediction network, predict the top left and bottom right corners of the target bounding box in the template area to obtain the target foreground region range;

[0056] Step 12: Adaptively adjust the pooling ratio by predicting the aspect ratio of the bounding box, and use integral mean pooling to extract a set number and size of foreground region template features.

[0057] Among them, the template foreground region feature is the feature sub-map that belongs to the target range of the template region in the template features.

[0058] In this embodiment, to address the pooling imbalance problem caused by changes in the aspect ratio of the target appearance, an aspect ratio adaptive pooling method is proposed. By adjusting the pooling ratio based on the predicted aspect ratio of the bounding box, the feature of the target region can be extracted more effectively.

[0059] In some embodiments, the template feature adaptive foreground region selection and pooling method in step 1 can be implemented through a constructed template feature foreground region predictor. The specific network structure is as follows: Figure 1 As shown, it includes a corner predictor and a pooling module; the corner predictor includes a fully convolutional network (conv), a spatial dimension normalization module, and a corner coordinate calculation module;

[0060] Optionally, spatial information of the template feature map can be extracted through a fully convolutional network (conv), and the dimension of the feature map can be reduced layer by layer through multiple convolutional layers to obtain a single-channel one-dimensional template feature map.

[0061] To predict the top-left and bottom-right corners of the target, and to enable the fully convolutional network (conv) to extract spatial information from the template feature map, two sets of 5-layer convolutional modules were designed to progressively reduce the dimensionality of the feature map, ultimately resulting in two single-channel one-dimensional template feature maps. Predict the top left and bottom right corner points respectively.

[0062] The spatial dimension normalization module is used to normalize the obtained one-dimensional feature map. Spatial dimension normalization is performed to generate a heatmap of the corner points of the target bounding box, P(x,y). The normalization calculation formula is as follows:

[0063]

[0064] The corner coordinate calculation module predicts corner coordinates by calculating the expected value of the probability distribution of the heatmap. x corner y The calculation formula is expressed as follows:

[0065]

[0066] Here, H and W represent the width and height of the feature map, and the two feature maps each yield a set of corner coordinates. This corner predictor is a fully convolutional network with a simple structure and no post-processing.

[0067] The pooling module is used to extract template features by pooling based on the target region obtained from the predicted corner points.

[0068] Optionally, the number of foreground region features obtained by pooling each feature map layer can be determined based on the relationship between the template region size and the target range size.

[0069] For example, if the template region is four times the size of the target region, the number of foreground region features obtained by pooling each feature map layer is also one-quarter of the number of template features.

[0070] In this embodiment, the integral mean pooling method is used to pool the template features. This pooling method can avoid the quantization of coordinates and the coordinates on the bounding box have continuous gradients.

[0071] Specifically, given a template feature map, let w i,j Let f(x,y) be the feature at a discrete position (i,j) on the feature map. Using bilinear interpolation, the feature value f(x,y) at any continuous coordinate (x,y) on the discrete feature map F is calculated, and its formula is expressed as:

[0072]

[0073] Where IC(x,y,i,j)=max(0,1-|xi|)×max(0,1-|yi|) are the coefficients calculated by bilinear interpolation.

[0074] The predicted foreground region is represented as FR = {(x1,y1),(x2,y2)}, where (x1,y1) and (x2,y2) are the continuous coordinates of the top-left and bottom-right points, respectively. The feature F is then subjected to average pooling by calculating the second-order integral. The formula for integral average pooling is:

[0075]

[0076] The Region of Interest Pooling (ROL Pooling) method requires discretization and weighted calculation of continuous coordinates, which can easily lead to quantization errors. However, the integral mean pooling method used in this embodiment can directly calculate the second integral of the continuous feature map based on the size of the foreground region, thus avoiding quantization errors.

[0077] Furthermore, this embodiment proposes an adaptive pooling ratio selection method for the target aspect ratio. The adaptive selection method is as follows: multiple pooling ratios are preset, and the pooling ratio that is closest to the aspect ratio of the foreground region is selected based on the aspect ratio of the foreground region.

[0078] Specifically, there are three preset pooling ratios, such as 1:1, 1:4, and 4:1; which respectively include length and width similar, width being a multiple of length, and length being a multiple of width.

[0079] When the pooling ratio is simply set to 1:1, meaning the output feature map is square, the pooling results of 1:1 pooling are unevenly distributed because the aspect ratio of the target to be tracked is variable; some targets are longer or wider. For example... Figure 2 As shown, the bounding box of the shark is relatively wide. When a 1:1 pooling ratio is used, the sampling points are mainly concentrated in the dense central region, resulting in an uneven distribution of features in the acquired foreground region. Figure 2 The black dots in the middle represent sampling points; at a fixed ratio, the sampling points are arranged in a four-row, four-column matrix. This embodiment presets three pooling ratios based on the template feature map size: 1:1, 1:4, and 4:1. The corresponding pooling ratio is adaptively selected for pooling mapping based on the predicted aspect ratio of the foreground region. Figure 2 As shown, when the pooling ratio becomes an aspect ratio of 4:1, the sampling points are evenly distributed, ensuring that the number of pooling points remains unchanged while obtaining a more uniform pooling mapping feature.

[0080] When building a backbone network for target tracking feature extraction and fusion based on Transformer, introducing correlation calculation between template features and search features while extracting features can yield feature maps that focus more on the fine information of the target to be tracked. However, the search region is usually large because it needs to include as much of the range that the target may move to in the next frame as possible, resulting in a large size of the search features. The global computation paradigm of the attention mechanism makes it necessary for the input feature map size to significantly increase the computational burden and reduce the running speed.

[0081] To reduce the computational overhead of the attention mechanism and improve the accuracy of the Transformer target tracking network while maintaining its operating speed, this embodiment proposes a search feature-adaptive target latent region selection method. Specifically, this can be implemented by constructing a search feature-based target latent region predictor, as shown in the schematic diagram below. Figure 3 As shown.

[0082] Optionally, in step 2, the search feature adaptive target potential region selection method includes the following steps:

[0083] Step 21: Predict the possible locations of the target on the search feature map using a fully convolutional network;

[0084] The location prediction network in this step has the same structure as the corner predictor in step 1. By constructing a fully convolutional network to compress the search feature dimension, a one-dimensional feature map containing search feature space information is obtained. Then, the expected probability distribution of the normalized feature map is calculated, and the position of the maximum response on the feature map is used as the possible position of the predicted target.

[0085] Step 22: Based on the predicted target location, round the target location coordinates, adaptively set the potential range of the target according to the position of the target location on the search feature map, and select the search feature sub-map.

[0086] In this embodiment, the predicted position is rounded, and a feature sub-map of a preset target potential range size is selected with the predicted target potential position as the center.

[0087] The predicted potential location of the target is usually a continuous value, not a discrete point on the feature map. Rounding the location can ensure the integrity of the feature submap and avoid errors in the feature mapping process.

[0088] Optionally, the method for adaptively setting the potential range of the target is as follows: when the target is located at the center of the image, the feature sub-image can be directly selected; when the target is located at the edge of the image, the surrounding area does not meet the size of the preset potential range of the target, so the potential range of the target is selected from the image boundary that the target is close to, according to the preset range size.

[0089] A further technical solution employs a search feature adaptive target latent region selection method to extract the search target latent region and set it in the high-level feature extraction layer of a multi-layer cascaded feature extraction network.

[0090] Specifically, in the Transformer target tracking network, during the feature extraction process, the shallow layers of the cascaded network exhibit larger prediction errors when predicting the target location within the search region. As the network deepens and merges, the predicted target location becomes increasingly accurate. In the second and third layers of the three-layer cascaded feature extraction and fusion network, two potential target location prediction processes are inserted. Furthermore, considering the prediction capabilities of networks at different depths, different feature sub-map selection sizes are designed, centered on the predicted location. The second layer has a larger range, while the third layer has a slightly smaller range. The size of the potential target range significantly impacts the final target tracking accuracy. Multiple ablation experiments can be conducted to compare the different potential range sizes, ultimately achieving a balance between accuracy and speed.

[0091] The Transformer network comprises multiple feature extraction layers. In this embodiment, during the layer-by-layer feature extraction process, the potential range of the target in the search features is predicted. Then, a search feature sub-graph is selected based on the predicted range, and the correlation between the search feature sub-graph and the template features is calculated. As the cascaded feature extraction network deepens, the predicted potential range of the target continuously shrinks, and the size of the search feature sub-graph also continuously shrinks. This method significantly reduces the computational complexity of attention while maintaining network tracking accuracy, thus improving network operating speed.

[0092] Optionally, in step 3, the dual attention operation is as follows: for the two sets of feature maps, namely the foreground target region features of the template and the potential region features of the search target, while focusing on their own target information based on self-attention, the embedded feature sequences of the two sets of feature maps are subjected to cross-attention operation.

[0093] In step 3, this can be achieved by constructing an Adaptive Region Selection Fusion Module (ARSFM). The input of ARSFM is the mapping and embedding feature sequence of the template region and the search region. It can simultaneously perform feature mapping on them in different dimensions, extract different target region features from the template feature map and the search feature map respectively, and fuse the information between them.

[0094] Compared with existing target tracking feature fusion networks based on attention mechanisms, ARSFM extracts feature sub-maps of two specific regions by using the range of the target region in the template region and the range of the potential target region in the search region for cross-attention operations. This concentrates complex attention calculations within the target range, achieving accurate feature fusion.

[0095] Optional, such as Figure 4 As shown, ARSFM includes an input feature grouping module, a convolutional mapping module, a feature pooling and flattening module, a concatenation module, an attention module, and a linear mapping module connected in sequence. The template feature foreground region predictor and the template feature foreground region predictor are connected to the feature flattening and segmentation module. The template feature foreground region predictor is connected to the input feature grouping module and the feature pooling and flattening module, and the search feature target latent region predictor is connected to the input feature grouping module and the feature pooling and flattening module.

[0096] In step 3, a dual attention operation is performed on the target regions of the template foreground target region and the search target latent region features within the target regions of the two sets of feature maps to obtain the fused features. The feature fusion method, implemented in ARSFM, includes the following steps:

[0097] Step 31: Input feature grouping module: This module is used to convolve and embed the template feature map and the search feature map to obtain an embedding sequence. The obtained embedding sequence features are then divided into two groups of features and reshaped into 2D feature maps.

[0098] Specifically, ARSFM first divides the acquired sequence features into two groups of features and then reshapes them into 2D feature maps.

[0099] Step 32, Convolutional Mapping Module: In order to introduce additional modeling of local spatial context information, depthwise separable convolutions are performed on each group of reshaped 2D feature maps in different dimensional spaces to obtain feature mapping matrices in different dimensional spaces.

[0100] Specifically, the different dimensional spaces correspond to the query Q, key K, and value V of the attention mechanism, respectively.

[0101] Downsampling is performed on the key matrix and value matrix to reduce the dimensionality of the key K(key) and value V(value), resulting in updated key K(key) and value V(value) matrices. Reducing the dimensionality of the matrices can reduce the amount of computation.

[0102] Step 33, Feature Pooling and Flattening Module: The feature mapping matrices of query Q, key K and value V corresponding to different dimensions obtained by the convolutional mapping module are pooled and flattened within the target region of the feature sub-map of the template foreground target region and the feature sub-map of the search target potential region.

[0103] Pooling selection is performed within the target range. Specifically, for the template feature map, during the pooling process, within the template foreground target region feature sub-map obtained by the template feature foreground region predictor, the key K (key) feature mapping matrix and value V (value) feature mapping matrix corresponding to the template feature map are pooled and selected to extract the key K (key) matrix and value V (value) matrix of the template feature foreground template region.

[0104] For the search feature map, within the range of the search target potential region feature sub-map obtained by the search feature target potential region predictor, the key K (key) feature mapping matrix and value V (value) feature mapping matrix corresponding to the search feature map are selected to extract the key K (key) matrix and value V (value) matrix of the search feature target potential region.

[0105] To be specific, use q t k t and v t k represents the projection of the template feature sequence. t-sub and v t-sub q represents the key-value projection of the feature submap of the target region of the template. s k s and v s k represents the projection of the search feature sequence. s-sub and v s-sub The key-value projection of the feature submap represents the potential range of the target in the search area. First, two sets of key-value projections are extracted from the feature submaps based on the predicted target range. The calculation formula is as follows:

[0106] k t-sub =PrPool(TR,k t ), v t-sub =PrPool(TR,v t )

[0107] k s-sub =select(SR,k s ), v s-sub =select(SR,v s (1.5)

[0108] Here, select(·) represents the subgraph selection module for the potential region of the search feature target.

[0109] Step 34, Stitching Module: Stitch the key-value projections (i.e., key projection and value projection) of the feature sub-image of the template foreground target region and the feature sub-image of the search target potential region respectively.

[0110] The formula for splicing calculation is expressed as follows:

[0111] k m =Concat(k t-sub ,k s-sub )

[0112] v m =Concat(v t-sub ,v s-sub (1.6)

[0113] Concat(·) is used to concatenate features along the channel dimension.

[0114] Step 35, Attention Module: Calculate all self-attention for the features after flattening the template feature map; for the search features, calculate the cross-attention between the features obtained by concatenating the feature sub-map of the template foreground target region and the feature sub-map of the search target potential region and the features after flattening the search features.

[0115] Step 36, Linear Mapping Module: The template attention result and the search attention result are residually connected through linear projection to obtain the final fused features, which is the output feature sequence.

[0116] The formula for calculating attention is as follows:

[0117]

[0118]

[0119] Where d is the dimension of the value matrix, and Attention t For template feature attention maps, Attention s This is an attention map for the search features.

[0120] In this embodiment, full self-attention is calculated for the template features to extract spatial information, better extract the target foreground region range, and ensure that the template information is not interfered with by the search feature information during tracking. For the search features, cross-attention is calculated between the two sets of feature sub-images and the search features to more accurately integrate target information.

[0121] The template attention results and the search attention results are residually connected by linear projection.

[0122] In this embodiment, the proposed regional feature fusion method recombines the template feature sequence and the search feature sequence through depthwise separable convolution, designs different attention calculation methods for different features, and extracts the target corresponding feature sub-map from the two sets of features, thereby achieving accurate target information integration.

[0123] Example 2

[0124] Based on Embodiment 1, this embodiment provides an adaptive region selection feature fusion system, including: a template feature foreground region predictor, a search feature target latent region predictor, and an adaptive region selection feature fusion module.

[0125] The template feature foreground region predictor is configured to acquire the template feature map to be processed and use an adaptive foreground region selection and pooling method to extract the features of the template foreground target region.

[0126] The search feature target latent region predictor is configured to acquire the search feature map to be processed and adopt the search feature adaptive target latent region selection method to obtain the search target latent region features.

[0127] The adaptive region selection feature fusion module is configured to perform a dual attention operation within the target region range of both the template feature map and the search target feature map during the feature extraction process, targeting the foreground target region features of the template and the potential region features of the search target, to obtain the fused features.

[0128] Optional, template feature foreground region predictor, network structure as follows: Figure 1 As shown, it includes a corner predictor and a pooling module; the corner predictor includes a fully convolutional network (conv), a spatial dimension normalization module, and a corner coordinate calculation module;

[0129] The fully convolutional network (conv) is used to extract spatial information of the template feature map. By setting multiple convolutional layers, the dimension of the feature map is reduced layer by layer to obtain a single-channel one-dimensional template feature map.

[0130] The spatial dimension normalization module is used to normalize the obtained one-dimensional feature map. Perform spatial dimension normalization to generate a heatmap of the target bounding box corners;

[0131] The corner coordinate calculation module predicts corner coordinates by calculating the expected value of the probability distribution of the heatmap.

[0132] The pooling module is used to extract template features by pooling based on the target region obtained from the predicted corner points.

[0133] Optionally, a search feature target latent region predictor, the structure of which is shown in the diagram below. Figure 4As shown, it includes a location prediction network and an adaptive target setting module;

[0134] Location prediction network, used to predict the possible locations of targets on the search feature map using a fully convolutional network;

[0135] The adaptive target setting module is used to round the target position coordinates according to the predicted target position, adaptively set the potential range of the target according to the position of the target position on the search feature map, and select the search feature submap.

[0136] Optionally, the adaptive region selection feature fusion module includes, in sequence, an input feature grouping module, a convolutional mapping module, a feature pooling and flattening module, a concatenation module, an attention module, and a linear mapping module; wherein, the template feature foreground region predictor and the template feature foreground region predictor are connected to the feature flattening and segmentation module. The template feature foreground region predictor is connected to the input feature grouping module and the feature pooling and flattening module, and the search feature target latent region predictor is connected to the input feature grouping module and the feature pooling and flattening module.

[0137] It should be noted that each module in this embodiment corresponds one-to-one with each step or module in embodiment 1, and their specific implementation process is the same, so it will not be repeated here.

[0138] Example 3

[0139] Based on Example 1, this example provides a target tracking method based on adaptive region selection feature fusion, such as... Figure 5 As shown, a Transformer target tracking network framework based on adaptive region selection and feature fusion is constructed. A multi-stage cascaded network architecture is used for step-by-step feature extraction and fusion, including the following steps:

[0140] Step 1: In the first feature processing stage, convolutional embedding is performed on the obtained template image and the search image, and feature fusion is performed using target region prediction to fuse the two sets of features.

[0141] Since the first layer of a cascaded network is the first information integration in the network, the target representation information in the two sets of features is relatively small. Therefore, a feature fusion module without region selection is used to facilitate information exchange between the two sets of features. This module is as follows: Figure 6 As shown, compared with the adaptive region feature fusion module, the target region prediction module is removed, and the two sets of features are directly fused.

[0142] At the beginning of the cascaded network, a template image (size H) is given. t ×W t ×3) and search images (size H) s ×W s×3), the convolutional embedding module with a stride of 4 and a kernel size of 7 maps the two sets of images into overlapping image embedding maps f. t ∈H t / 4×W t / 4×C and f s ∈H s / 4×W s / 4×C, where C is 64 in this embodiment. Then, the image embedding map is expanded in the spatial dimension and concatenated in the channel dimension to form a shape of size (H). t / 4×W t / 4+H s / 4×W s The fusion feature sequence of (4)×C is fed into the attention-based feature fusion network to collect target information.

[0143] Step 2: In the feature processing stage following the first feature processing stage, the fused features output from the previous feature processing stage are received, and after convolutional embedding, feature fusion is performed using the adaptive region selection feature fusion method described in Example 1.

[0144] Step 3: Perform a full convolution operation on the fused features output from the last feature processing stage to locate the corner points and obtain the bounding box for target tracking.

[0145] In this embodiment, as Figure 5 As shown, a three-stage processing network is employed. An adaptive region feature fusion module is used in the second and third layers of the cascaded network for fusion. Before fusion, a convolutional embedding layer with a stride of 2 is used, so that as the network deepens, the spatial size of the feature map decreases while the depth of the feature map increases. The feature map dimension of the second layer is 3C, and that of the third layer is 6C. Finally, a feature map of size (H) is obtained. t / 16×W t / 16+H s / 16×W s The search feature map is (16)×6C. It is then fed into a fully convolutional corner prediction network for target tracking bounding box regression.

[0146] In step 3, the target bounding box regression can be performed on the final output feature map of the feature extraction and adaptive region selection fusion network by constructing a fully convolutional corner localization head network.

[0147] The fully convolutional corner localization head network has a structure basically the same as the template feature foreground region predictor. It processes the template feature map through five convolutional network layers to obtain a probability map, and then calculates the expectation of the probability distribution of the two corner points to obtain the predicted corner point position, thus obtaining the bounding box. This head network is a fully convolutional head network.

[0148] Example 4

[0149] Based on Embodiments 1 and 3, this embodiment provides a target tracking system based on adaptive region selection feature fusion, namely, a Transformer target tracking network (ARSformer network) based on adaptive region selection feature fusion, such as... Figure 5 As shown, it includes:

[0150] The first feature processing stage network is used to perform convolutional embedding on the obtained template image and the search image, and to fuse the two sets of features by using feature fusion based on target region prediction.

[0151] The high-level feature processing stage network is used to receive the fused features output by the previous-level feature processing stage network, perform convolutional embedding, and then perform feature fusion using the adaptive region selection feature fusion method described in Example 1.

[0152] The fully convolutional corner localization head network is used to perform a fully convolutional operation on the fused features output from the last stage of feature processing to locate corner points and obtain the bounding box for target tracking.

[0153] ARSformer network framework, such as Figure 5 As shown, ARSFormer can progressively extract features from the template region and the search region, and perform deep integration of the information between the two for adaptive region selection.

[0154] This embodiment is based on the framework of a currently popular tracker based on a Transformer backbone network. This tracker selects appropriate template region features and search region features through an adaptive region prediction module and fuses them without any post-processing modules, forming a more compact and accurate tracking framework.

[0155] Specifically, the high-level feature processing stage network has two layers, forming a three-level cascaded network with the first feature processing stage network, including stage1, stage2, and stage3. Each stage network consists of a feature extraction module and an N-layer region selection and fusion module. Figure 5 In this model, N1 is set to 1, N2 and N3 are 4 and 16 respectively. The architecture of each stage is roughly the same, but the number of layers in the region selection fusion module differs in each stage.

[0156] The training process of ARSFormer is the same as the standard training method for Transformer trackers. The loss function mainly consists of three regression tasks: template feature foreground region prediction, search for potential range region prediction, and search for bounding box prediction. This embodiment uses the l1 norm loss function L1(.,.) and the generalized IoU loss function L... GIoUThe linear combination of (.,.) forms the loss function for training ARSFormer, and its calculation formula is expressed as:

[0157]

[0158] Among them, b t The bounding box of the target area in the predicted template region. b is the actual bounding box of the template region target. s The target bounding box for the predicted search region. For the actual target bounding box of the search region, c s The coordinates of the center point of the target in the predicted search area. Let λ1 and λ2 be the coordinates of the center point of the target in the actual search area. G The weights for the linear combination of the two losses are set to 5 and 2, respectively.

[0159] The effectiveness of ARSformer is verified and analyzed through experiments. First, ablation experiments are conducted on each module of ARSformer. Then, the performance of ARSformer is compared with existing state-of-the-art algorithms on four short-term test sets (GOT-10K, TrackingNet, OTB100, and UAV123) and one long-term test set (LaSOT) to verify the advanced performance of ARSformer.

[0160] 1.1 Experimental Setup

[0161] ARSformer was implemented using PyTorch toolkit version 1.7.1 and Python version 3.6, and trained on a single GeForce RTX 3090 GPU.

[0162] (1) Model

[0163] The ARSformer feature extraction and adaptive region selection feature fusion network is initialized using a pre-selected 21-layer CVT network from the ImageNet dataset, while other network parameters are initialized using the Xavier initialization method. The feature extraction and adaptive region selection feature fusion network consists of three stages, with the number of layers in each stage set to 1, 4, and 16 respectively. The overall dropout value is set to 0.1. In the adaptive latent region selection module for the search feature, the region size in the second stage of the cascaded network is set to 32×32, and the region size in the third stage is set to 12×12.

[0164] (2) Training phase

[0165] The training datasets included LaSOT, GOT-10K, COCO2017, and TrackingNet. The model used for testing on the GOT-10K dataset was trained solely on the GOT-10K training set. The entire ARSformer training process consisted of 500 epochs, using the AdamW optimizer, which decayed by 10% after 400 epochs. The initial learning rate for the backbone network parameters was 1e-5, and for other parameters, it was 1e-4. The template and search images were 320×320 pixels and 128×128 pixels, respectively. During training on a single GeForce RTX 3090 GPU, the batch size was set to 32, and data augmentation techniques including horizontal flipping and brightness jitter were used.

[0166] (3) Testing phase

[0167] During the testing phase, the input to ARSformer is the template image determined by the first image and the search image determined by the currently tracked image. The search image range is selected based on the tracking results of the previous frame.

[0168] 1.2 Analysis of ablation comparison experiment results.

[0169] To verify the effectiveness of the proposed ARSformer network, detailed ablation experiments were conducted to verify and analyze the role of each module. The dataset used for verification was the long-term tracking dataset LaSOT.

[0170] (1) A unified feature extraction and feature fusion network framework.

[0171] ARSformer, based on Transformer, unifies the framework of feature extraction and feature fusion, incorporating the information integration process into the feature extraction process. Cvt stands for pre-trained feature extraction network, Ex is short for feature extraction, and Fu is short for feature fusion. The framework of the algorithm proposed in this paper integrates feature extraction and feature fusion into a whole, with feature extraction and feature fusion occurring alternately in the network.

[0172] To demonstrate the effectiveness of the unified feature extraction and feature fusion network, three network architectures were designed: a Cvt feature extraction network plus a cross-attention mechanism feature fusion network (Cvt+Cross-Attention, Cvt+CA), a Cvt feature extraction network plus the feature fusion method proposed in step 3 of Example 1 (Cvt+Fu, Cvt+Fu), and the unified feature extraction and feature fusion network proposed in this disclosure (Features Extraction+Features Fusion, Ex+Fu). All three networks use all template features and search features for information interaction. The results are shown in Table 1, which reports the number of parameters, computational cost (FLOPs), and accuracy (AUC) of different network combinations. The results show that the unified feature extraction and feature fusion network achieves an accuracy of 67.6% on the LaSOT dataset, exceeding the other two networks by 7.1% and 1.4%, respectively, while having lower parameter and computational costs. This demonstrates the effectiveness of the unified feature extraction and feature fusion; the backbone network of the unified feature extraction and feature fusion proposed in this paper can extract features that are more relevant to the target information, thereby achieving better tracking performance.

[0173] Table 1

[0174]

[0175] (2) Search for the target region size of the potential region predictor.

[0176] In the adaptive target latent region selection module, the size of the predefined latent region range also significantly impacts tracking performance. The original search feature map has sizes of 40×40 and 20×20 in the second and third stages, respectively. A larger predicted target latent region range results in better tracking accuracy but slower network speed; conversely, a smaller range reduces computational cost and increases speed but decreases tracking accuracy. To balance accuracy and speed, this section designs three different combinations of target latent region ranges: 36×36 and 16×16, 32×32 and 12×12, and 28×28 and 10×10 in the second and third stages, respectively. These combinations are then compared with networks where all search features are used for feature fusion.

[0177] The final tracking results are shown in Table 2. The table reports the number of network parameters, computational cost, network speed, and tracking accuracy for different combinations of potential target region ranges. The speed refers to the speed achieved on a GTX 1080Ti device. The results show that adding a target potential region predictor as a search feature increases the number of network parameters. However, the reduction in the range of search features involved in the fusion process significantly reduces the computational cost and greatly improves the speed. Of course, the tracking accuracy decreases. Compared to all search features, networks selecting target potential region ranges of 36×36 and 16×16 show a 0.8% decrease in accuracy, while 32×32 and 12×12 show a 1.9% decrease. The 28×28 and 10×10 networks show the largest decrease, at 5.4%. Therefore, the final target potential region range sizes selected are 32×32 and 12×12, which can maintain tracking accuracy while significantly reducing the computational cost of the network.

[0178] Table 2 Results of different combinations of potential target region extents on the LaSOT dataset

[0179]

[0180] (3) Template feature foreground region predictor: template feature adaptive foreground region selection and pooling.

[0181] Ablation experiments were conducted to analyze the adaptive foreground region selection and pooling based on template features. To verify the superiority of the proposed adaptive foreground region selection and pooling module based on template features, three different networks were designed in this section. The difference lies in the template features involved in the feature extraction and feature fusion networks: Full TemplateFeatures (Full-TF), Template Features corresponding to the Foreground Region based on Adaptive Pooling (TFFR-Adapool), and Template Features corresponding to the Foreground Region based on Adaptive Pooling with Aspect Ratio (TFFR-Adapool-AR). The search features involved in the fusion ranged from 32×32 to 12×12. The number of parameters, computational cost, speed, and tracking accuracy of the three networks are shown in Table 3. When the adaptive foreground region selection and pooling module based on template features was added, the number of network parameters also increased, but the computational cost did not increase significantly. This is because the number of template features involved in the feature fusion network decreased, and the overhead of the attention mechanism was also reduced. When the foreground region features extracted by this module are fused, the tracking accuracy of the network without aspect ratio adaptation is improved by 0.9%, while the accuracy of the network with aspect ratio adaptation pooling is improved by 0.4%. This verifies the effectiveness of foreground region features in improving network tracking accuracy, and also verifies that the proposed aspect ratio adaptation template foreground region feature pooling can extract template foreground region features with a more uniform distribution, fully containing template target information.

[0182] Table 3 shows the network results of different template features participating in the fusion on the LaSOT dataset.

[0183]

[0184] 1.3 Experimental Results on Different Datasets

[0185] To verify the advanced performance of ARSformer, this section compares the experimental results with those of various mainstream target tracking algorithms on five large public target tracking datasets.

[0186] (1) TrackingNet dataset

[0187] Table 4 shows the performance of ARSformer on the TrackingNet test set, where "-" indicates that relevant results were not provided in the original paper. The results show that ARSformer achieves a tracking success rate, normalized tracking accuracy, and precision of 83.2%, 87.9%, and 81.9%, respectively, all exceeding SwinTrack-B and reaching the current state-of-the-art level, and outperforming other Transformer-based tracking algorithms. These excellent test results on this large-scale dataset demonstrate the powerful tracking performance of ARSformer.

[0188] Table 4 shows the comparison results of ARSformer and the best tracker on the TrackingNet dataset.

[0189]

[0190] Among them, SwinTrack-B is a shift-window Transformer tracking network proposed in a 2022 NIPS conference paper; STARK is a spatiotemporal Transformer network proposed in a 2021 CVPR conference paper; DTT is a Transformer-based discriminative tracking network proposed in a 2021 ICCV conference paper; TREG is a target transformation regression network proposed in a 2021 CoRR conference paper; DualTFR is a two-stream Transformer network proposed in a 2021 ICCV conference paper; TransT is a Transformer tracking network proposed in a 2021 CVPR conference paper; TrDIMP and TrSiam are a Transformer-based correlation filter network and twin tracking network proposed in a 2021 CVPR conference paper, respectively. AutoMatch is an automatic matching tracking network proposed in a 2021 ICCV conference paper; STMtracker is a template-free tracking network based on spatiotemporal memory networks proposed in a 2021 CVPR conference paper; SiamR-CNN is a tracking network based on re-detection proposed in a 2020 CVPR conference paper; KYS is a target tracking network based on environmental information proposed in a 2020 ECCV conference paper; PrDiMP is a target tracking network based on probabilistic regression proposed in a 2020 CVPR conference paper; SiamAtnn is a deformable conjoined attention network proposed in a 2020 CVPR conference paper; SiamRPN++ is an improved tracking network based on region of interest prediction networks proposed in a 2019 CVPR conference paper; and SiamFC is a fully convolutional Siamese neural network proposed in a 2016 ECCV conference paper.

[0191] (2) GOT-10k dataset

[0192] The ARSformer model was tested on the GOT-10k dataset. The results of ARSformer on the test set were submitted to the official website for validation. The results show that ARSformer's average bounding box overlap reached 70.2%, surpassing the current best tracker SBT-base's 0.3%, achieving state-of-the-art performance. Furthermore, ARSformer achieved a tracking accuracy of 67.2% at a threshold of 0.75, significantly outperforming other target tracking algorithms. This demonstrates that ARSformer's bounding boxes are highly accurate, reflecting that the proposed adaptive region selection feature fusion network still maintains deep-level feature information interaction.

[0193] (3) UAV123 dataset

[0194] This dataset consists of ground scenes captured by drones at high altitudes, exhibiting significant changes in perspective and posing new challenges to trackers. A comparison of AUC accuracy between ARSformer and the current best tracker on the UAV123 dataset demonstrates that ARSformer achieves the best performance to date, with a success rate exceeding 70% and an accuracy of 91.5%.

[0195] (4) OTB100 dataset

[0196] The tracking success rate comparison results of ARSformer on the OTB100 dataset with existing trackers show that ARSformer achieves a tracking success rate of 69.8%, reaching the level of current advanced target tracking algorithms. Furthermore, to further analyze the performance of ARSformer in different scenarios, comparative experiments were conducted on eight video attributes of the OTB100 dataset. The success rate curves are shown below. Figures 7(a) to 7(h) As shown, the eight video attributes tested were low resolution, out of view, motion blur, fast motion, deformation, illumination variation, scale variation, and occlusion. The results demonstrate that ARSformer maintains good robustness across various challenging scenarios. ARSformer performs well even with motion-induced target blur, reflecting the excellent discriminative ability of the feature extraction and adaptive region selection feature fusion network for target appearance.

[0197] 1.4 Visualization of Tracking Results

[0198] To observe the attention information between different layers in the proposed feature extraction and adaptive region selection feature fusion network, two video sequences on the GOT-10k test set were selected for attention weight visualization. The visualization results are as follows: Figure 8 As shown in the figure, the template-search cross-attention and search region self-attention weights of frame 40 in two sets of GOT-10k test set video sequences are visualized. In the template-search cross-attention, the key and value are the template image, and the query is the search image; while in the search region self-attention, both the query, key, and value are the search image. S2-B0 represents the attention visualization result of the Block 0 adaptive feature fusion layer in Stage 2. Figure 8 The effects of the attention mechanism during network deepening are displayed sequentially from bottom to top. Specifically, the attention at layers 0, 2, 4, 6, 8, 10, 12, 14, and 15 are visualized. It can be seen that the template-search cross-attention gradually eliminates the background as the network deepens, ultimately determining the target foreground location at layers 14 and 15. Meanwhile, the search region self-attention gradually focuses on the target's boundary information as the network deepens, which is more beneficial for the subsequent localization head network's regression of the target bounding box.

[0199] The above description is merely a preferred embodiment of this disclosure and is not intended to limit this disclosure. Various modifications and variations can be made to this disclosure by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this disclosure should be included within the scope of protection of this disclosure.

[0200] While the specific embodiments of this disclosure have been described above in conjunction with the accompanying drawings, this is not intended to limit the scope of protection of this disclosure. Those skilled in the art should understand that various modifications or variations that can be made by those skilled in the art without creative effort based on the technical solutions of this disclosure are still within the scope of protection of this disclosure.

Claims

1. A feature fusion method based on adaptive region selection, characterized in that, Includes the following steps: Obtain the template feature map to be processed, and use an adaptive foreground region selection and pooling method to extract the foreground target region features of the template; Template feature adaptive foreground region selection and pooling includes the following steps: The target foreground region range is obtained by setting the top left and bottom right corners of the target bounding box in the template area using a corner prediction network. By adaptively adjusting the pooling ratio based on the predicted aspect ratio of the bounding box, and using integral mean pooling, a set number and size of foreground region template features are extracted. Obtain the search feature map to be processed, and use the search feature adaptive target potential region selection method to obtain the search target potential region features; In the process of feature extraction of template features and search target latent region features, a dual attention operation is performed within the target region of both the template feature map and the search feature map to obtain the fused features. The dual attention operation is as follows: for two sets of feature maps, namely the foreground target region features of the template and the potential region features of the search target, while focusing on their own target information based on self-attention, the embedded feature sequences of the two sets of feature maps are subjected to cross-attention operation.

2. The adaptive region selection feature fusion method as described in claim 1, characterized in that: The method for adaptive foreground region selection and pooling based on template features includes the following steps: Spatial information of the template feature map is extracted by a fully convolutional network (conv), and the dimension of the feature map is reduced layer by layer by setting multiple convolutional layers to obtain a single-channel one-dimensional template feature map. The obtained one-dimensional feature map is spatially normalized to generate a heatmap of the corner points of the target bounding box. The corner coordinates are predicted by calculating the expected value of the probability distribution of the heat map; Based on the target region obtained from the predicted corner points, the template features are extracted using integral mean pooling. Alternatively, an adaptive pooling ratio selection method can be used: multiple pooling ratios are preset, and the pooling ratio that is closest to the aspect ratio of the foreground region is selected based on the aspect ratio of the foreground region.

3. The adaptive region selection feature fusion method as described in claim 1, characterized in that: The search feature-adaptive target latent region selection method includes the following steps: The search feature map is predicted using a fully convolutional network; Based on the predicted target location, the target location coordinates are rounded, and the potential range of the target is adaptively set according to the position of the target location on the search feature map, and a search feature submap is selected.

4. The adaptive region selection feature fusion method as described in claim 1, characterized in that, Based on the foreground target region features of the template and the latent target region features of the search, a dual attention operation is performed within the target region of the two sets of feature maps to obtain the fused features. The steps of the feature fusion method are as follows: Obtain the embedding sequence after convolutional embedding of template feature map and search feature map to extract features, divide the obtained sequence features into two groups of features, and reshape them into 2D feature maps; Depth-separable convolutions are performed on each reshaped 2D feature map in different dimensional spaces to obtain feature mapping matrices in different dimensional spaces: query matrix, key matrix, and value matrix, where downsampling is performed on the key matrix and value matrix. The query matrix, key matrix, and value matrix corresponding to different dimensions obtained by the convolutional mapping module are pooled and flattened within the target region of the template foreground target region feature sub-map and the search target potential region feature sub-map. The key-value projections of the template foreground target region feature sub-map and the search target potential region feature sub-map are stitched together respectively; Calculate full self-attention for the flattened features of the template feature map; for the search features, calculate the cross attention between the features obtained by concatenating the template foreground target region feature map and the search target potential region feature map and the flattened features of the search features. The template attention results and search attention results are residually connected by linear projection to obtain the final fused features.

5. A feature fusion system based on adaptive region selection, characterized in that, include: Template feature foreground region predictor, search feature target latent region predictor, and adaptive region selection feature fusion module; The template feature foreground region predictor is configured to acquire the template feature map to be processed and use an adaptive foreground region selection and pooling method to extract the features of the template foreground target region. Template feature adaptive foreground region selection and pooling includes the following steps: The target foreground region range is obtained by setting the top left and bottom right corners of the target bounding box in the template area using a corner prediction network. By adaptively adjusting the pooling ratio based on the predicted aspect ratio of the bounding box, and using integral mean pooling, a set number and size of foreground region template features are extracted. The search feature target latent region predictor is configured to acquire the search feature map to be processed and adopt the search feature adaptive target latent region selection method to obtain the search target latent region features. The adaptive region selection feature fusion module is configured to perform a dual attention operation within the target region range of the template feature map and the search target latent region feature during the feature extraction process of the template feature map and the search feature map to obtain the fused features. The dual attention operation is as follows: for two sets of feature maps, namely the foreground target region features of the template and the potential region features of the search target, while focusing on their own target information based on self-attention, the embedded feature sequences of the two sets of feature maps are subjected to cross-attention operation.

6. A target tracking method based on adaptive region selection feature fusion, which employs a multi-stage architecture cascaded network for step-by-step feature extraction and feature fusion, is characterized by: Includes the following steps: In the first feature processing stage, convolutional embedding is performed on the obtained template image and the search image, and feature fusion is performed by predicting the target region to fuse the two sets of features. In the feature processing stage following the first feature processing stage, the fused features output from the previous feature processing stage are received, and after convolutional embedding, feature fusion is performed using the adaptive region selection feature fusion method described in any one of claims 1-4. The fused features output from the final feature processing stage are subjected to a full convolution operation to locate the corner points and obtain the bounding box for target tracking.

7. A target tracking system based on adaptive region selection feature fusion, characterized in that, Constructing an adaptive region selection feature fusion Transformer target tracking network, including: The first feature processing stage network is used to perform convolutional embedding on the obtained template image and the search image, and to fuse the two sets of features by using feature fusion based on target region prediction. The high-level feature processing stage network is used to receive the fused features output by the previous-level feature processing stage network, perform convolutional embedding, and then perform feature fusion using the adaptive region selection feature fusion method described in any one of claims 1-4. The fully convolutional corner localization head network is used to perform a fully convolutional operation on the fused features output from the last stage of feature processing to locate corner points and obtain the bounding box for target tracking.

8. The target tracking system based on adaptive region selection feature fusion as described in claim 7, characterized in that, The loss function used for training the Transformer object tracking network with adaptive region selection feature fusion is... A linear combination of the norm loss function and the generalized IoU loss function.

Images