A Target Tracking Method and System Based on Global-Local Dual-Path Extraction and Asynchronous Enhancement

Abstract

This invention proposes a target tracking method and system based on global-local dual-path extraction and asynchronous enhancement. The method includes: initializing a template image and a search image, and performing sequence partitioning through block embedding; simultaneously extracting global and local features from both the template and search images, and fusing these features; inputting the output of the global-local joint module into the asynchronous enhancement module for asynchronous interaction and feature enhancement; and finally inputting the enhanced output features into a prediction head to obtain the tracking result. This invention improves the robustness and feature representation capability of the model by simultaneously extracting global and local features through the feature joint module and using asynchronous enhancement operations in the feature fusion stage to enhance the interacted features.

Description

Technical Field

[0001] This invention relates to the field of computer vision and image processing technology, and in particular to a target tracking method and system based on global and local dual-path extraction and asynchronous enhancement. Background Technology

[0002] Target tracking is a crucial component of computer vision. It aims to continuously and accurately locate the position and state of a target within a video sequence. With the rapid development of signal processing and computer technologies, target tracking has been widely applied in various fields such as intelligent security, autonomous driving, and human-computer interaction. Although target tracking technology has been applied in multiple fields, its performance in complex dynamic scenes is still constrained by various factors, such as drastic changes in appearance, complex background interference, partial occlusion, and motion blur. Therefore, solving these problems remains a current challenge.

[0003] With the rapid development of deep learning technology in computer vision, network frameworks such as Siamese networks, Transformer, and Mamba have emerged one after another. Transformer-based trackers have become the mainstream framework in object tracking due to their powerful global modeling capabilities. Transformer's unique self-attention mechanism and global modeling ability effectively address the problem of insufficient long-range information mining in traditional trackers. However, while Transformer is powerful in capturing global information, it neglects the mining of local information. Insufficient local information capture affects the tracker's accurate perception and dynamic adaptation of targets, and significantly reduces its robustness to occluded scenes. Therefore, how to effectively capture both global and local information simultaneously remains a problem to be solved. Summary of the Invention

[0004] In view of the above situation, the main objective of this invention is to propose a target tracking method and system based on global and local dual-path extraction and asynchronous enhancement to solve the above-mentioned technical problems.

[0005] This invention proposes a target tracking method based on global-local dual-path extraction and asynchronous enhancement, the method comprising the following steps:

[0006] Step 1: Construct a tracking model based on the global-local joint module, prediction head, and asynchronous enhancement module;

[0007] Step 2: Pre-train the global-local joint module and the asynchronous enhancement module using a large-scale dataset, and adjust the model parameters to obtain the parameter-adjusted global-local joint module and the parameter-adjusted asynchronous enhancement module;

[0008] Step 3: Initialize the template image and the search image, and perform sequence partitioning through block embedding to obtain the template image sequence and the search image sequence; input the template image sequence and the search image sequence into the global-local joint module after parameter adjustment, and extract global features and local features at the same time, and then perform feature fusion of global features and local features to obtain the output of the global-local joint module;

[0009] Step 4: Input the output of the global-local joint module into the asynchronous enhancement module after parameter adjustment for asynchronous interaction and feature enhancement to obtain the enhanced output features;

[0010] Step 5: Input the enhanced output features into the prediction head to obtain the tracking results. Calculate the center classification loss and boundary regression loss based on the tracking results. Optimize the tracking model using the center classification loss and boundary regression loss to obtain the optimized tracking model.

[0011] Step 6: Use the optimized tracking model to track the target.

[0012] This invention also proposes a target tracking system based on global-local dual-path extraction and asynchronous enhancement, the system comprising:

[0013] Build modules are used for:

[0014] The tracking model is composed of a global-local joint module, a prediction head, and an asynchronous enhancement module.

[0015] Pre-trained modules are used for:

[0016] The global-local joint module and the asynchronous enhancement module are pre-trained using a large-scale dataset, and the model parameters are adjusted to obtain the parameter-adjusted global-local joint module and the parameter-adjusted asynchronous enhancement module.

[0017] Extraction module, used for:

[0018] The template image and the search image are initialized and the sequence is divided by block embedding to obtain the template image sequence and the search image sequence. The template image sequence and the search image sequence are input into the global-local joint module after parameter adjustment. At the same time, global features and local features are extracted. Then, the global features and local features are fused to obtain the output of the global-local joint module.

[0019] The calculation module is used for:

[0020] The output of the global-local joint module is input into the asynchronous enhancement module after parameter adjustment for asynchronous interaction and feature enhancement, resulting in enhanced output features.

[0021] The learning module is used for:

[0022] The enhanced output features are input into the prediction head to obtain the tracking results. The center classification loss and boundary regression loss are calculated based on the tracking results. The tracking model is then optimized using the center classification loss and boundary regression loss to obtain the optimized tracking model.

[0023] The tracking module is used for:

[0024] Target tracking is performed using the optimized tracking model.

[0025] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0026] 1. This invention extracts global and local features simultaneously through a feature union module, and uses asynchronous enhancement operations in the feature fusion stage to enhance the interactive features, thereby improving the robustness and feature representation ability of the model.

[0027] 2. This invention designs a global-local feature joint module that uses a parallel approach to simultaneously extract global and local features. The global branch employs a self-attention mechanism, performing a linear transformation on the input feature map to obtain a query vector, key vector, and value vector. The attention score map obtained by performing attention calculations on the query vector and key vector is then weighted and summed to extract the global features. The local branch first processes the input feature map through a 1×1 convolutional layer to adjust the number of channels, then groups the features at the channel level, performs spatial attention calculations on each group, concatenates the processed feature maps along the channel dimension, performs channel shuffle for cross-group mixing, and then processes the mixed feature maps through a 1×1 convolutional layer to restore the channel dimension. Finally, residual connections are used to extract the local features. The global and local features are added together to obtain the output of the global-local feature joint module.

[0028] 3. This invention employs asynchronous enhanced attention. First, it performs 3×3, 5×5, and 7×7 convolution operations on the search region features. Then, it performs asynchronous interactive calculations on the search region and template image features. Finally, the output feature map is input into the enhancement layer for feature enhancement. In the feature extraction stage, this invention uses a global-local feature joint module to extract global and local features, improving the completeness of feature representation. In the feature fusion stage, asynchronous enhanced attention is used on the template and search region, not only acquiring multi-scale information and fused features but also performing feature enhancement, improving the accuracy of target localization to achieve robust tracking performance.

[0029] Additional aspects and advantages of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by means of embodiments of the invention. Attached Figure Description

[0030] Figure 1This is a flowchart illustrating the steps of the target tracking method based on global and local dual-path extraction and asynchronous enhancement proposed in this invention.

[0031] Figure 2 This is a target tracking framework diagram of the target tracking method based on global and local dual-path extraction and asynchronous enhancement proposed in this invention.

[0032] Figure 3 This is a structural diagram of the global-local feature joint module of the target tracking method based on global-local dual-path extraction and asynchronous enhancement proposed in this invention.

[0033] Figure 4 This is a structural diagram of the asynchronous enhancement module in the target tracking method based on global and local dual-path extraction and asynchronous enhancement proposed in this invention.

[0034] Figure 5 This is a structural diagram of the target tracking system based on global and local dual-path extraction and asynchronous enhancement proposed in this invention. Detailed Implementation

[0035] Embodiments of the present invention are described in detail below. Examples of these embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and should not be construed as limiting the present invention.

[0036] These and other aspects of the embodiments of the present invention will become clear from the following description and accompanying drawings. In these descriptions and drawings, some specific embodiments of the present invention are specifically disclosed to provide some ways of implementing the principles of the embodiments of the present invention; however, it should be understood that the scope of the embodiments of the present invention is not limited thereto.

[0037] Please see Figure 1 This invention proposes a target tracking method based on global-local dual-path extraction and asynchronous enhancement, which includes the following steps:

[0038] Step 1: Construct a tracking model based on the global-local joint module, the prediction head, and the asynchronous enhancement module.

[0039] In step 1, a tracking model is constructed based on the global-local joint module, the prediction head, and the asynchronous enhancement module. The global-local joint module consists of a first global-local joint module and a second global-local joint module. The construction of the first global-local joint module is as follows:

[0040] Based on the self-attention mechanism, a self-attention module is constructed, and a group space shuffling module is constructed according to the group space attention mechanism and the channel shuffling mechanism. The self-attention module and the group space shuffling module constitute the first global-local joint module. The first global-local joint module and the second global-local joint module share weights and have the same structure.

[0041] The asynchronous enhancement module is constructed as follows:

[0042] An asynchronous enhancement module is constructed based on a feature fusion mechanism and an asynchronous enhanced attention mechanism.

[0043] Step 2: Pre-train the global-local joint module and the asynchronous enhancement module using a large-scale dataset, and adjust the model parameters to obtain the parameter-adjusted global-local joint module and the parameter-adjusted asynchronous enhancement module.

[0044] Furthermore, in step 2, this invention is performed on an RTX 3060 Ti GPU, using Python version 3.12, PyTorch version 2.2.1, and CUDA version 11.8. The network is trained for 300 epochs using the GOT-10k, LaSOT, TrackingNet, and COCO datasets. The AdamW optimizer is used for network training, with a learning rate set to 1e-5 during the feature extraction stage and 1e-4 during the feature fusion stage. At 240 epochs, the learning rate is reduced to 10% to achieve rapid convergence. The number of self-attention heads is set to 8, and the number of groups for grouped spatial attention is set to 8.

[0045] Step 3: Initialize the template image and the search image, and perform sequence partitioning through block embedding to obtain the template image sequence and the search image sequence; input the template image sequence and the search image sequence into the global-local joint module after parameter adjustment, and extract global features and local features at the same time, and then perform feature fusion of global features and local features to obtain the output of the global-local joint module.

[0046] For further details, please refer to Figure 2 Within the Transformer tracking framework, the initial frame of the template image and the search region image in subsequent frames are initialized. The template image and the search region image are divided into a sequence of patches through patch embedding and input into the global-local feature joint module.

[0047] It should be noted that, in Figure 2In this embodiment, the template image is processed by the first global-local joint module, and the search image is processed by the second global-local joint module. In this embodiment, the number of the first global-local joint modules is set to 9, and the number of the second global-local joint modules is set to 6. The structural topology of the first global-local joint module and the second global-local joint module is the same, and the only difference is in the number of modules and the input data. Therefore, the processing of the search image in the second global-local joint module will not be described in detail.

[0048] For further details, please refer to Figure 2 and Figure 3 In step 3, the template image sequence and the search image sequence are input into the global-local joint module after parameter adjustment. At the same time, global features and local features are extracted, and then the global features and local features are fused to obtain the output of the global-local joint module. The specific steps include the following:

[0049] The template image sequence is input into the first global-local joint module in the parameter-adjusted global-local joint module. Feature mapping is performed through the fully connected layer of the self-attention module to obtain the query vector, key vector, and value vector of the template image. Attention is calculated on the query vector, key vector, and value vector of the template image to obtain the attention score of the template image. The attention score of the template image is normalized, and then the value vector is weighted to obtain the output feature of the template image. The output feature of the template image is used as the global branch result of the template region.

[0050] The search image sequence is input into the second global-local joint module in the parameter-adjusted global-local joint module. Feature mapping is performed through the fully connected layer of the self-attention module to obtain the query vector, key vector, and value vector of the search image. Attention is calculated on the query vector, key vector, and value vector of the search image to obtain the attention score of the search image. Then, the attention score of the search image is normalized and weighted to obtain the output feature of the search image. The output feature of the search image is used as the global branch result of the search region.

[0051] The template image sequence is convolved using a grouped spatial shuffling module to obtain the convolved features of the template image.

[0052] The features of the template image after convolution are grouped along the channel dimension to obtain the features of each group of template images.

[0053] Spatial attention operation is performed on the features of each group of template images after grouping to obtain the features of the template images after spatial attention calculation;

[0054] The features of each group of template images are concatenated along the channel dimension to obtain the features of the concatenated template images.

[0055] The features of the template image after spatial attention calculation and the features of the template image after stitching are used to perform cross-group information interaction through channel shuffling operation to obtain the features of the template image after channel shuffling.

[0056] A residual connection operation is performed on the features of the template image after the channels are shuffled and the template image sequence to obtain the features after residual processing of the template image. The features after residual processing of the template image are used as the local branch results of the template region.

[0057] The search image sequence is convolved using a grouped spatial shuffling module to obtain the convolved features of the search image.

[0058] The features of the search images after convolution are grouped along the channel dimension to obtain the features of each group of search images.

[0059] Spatial attention operation is performed on the features of each group of search images after grouping to obtain the features of the search images after spatial attention calculation;

[0060] The features of each group of search images are concatenated along the channel dimension to obtain the features of the concatenated search images.

[0061] The features of the search image after spatial attention calculation and the features of the search image after stitching are used to perform cross-group information interaction through channel shuffling operation to obtain the features of the search image after channel shuffling.

[0062] A residual connection operation is performed on the features of the search image after the channels are shuffled and the search image sequence to obtain the features of the search image after residual processing. The features of the search image after residual processing are used as the local branch results of the search region.

[0063] The global branch results and local branch results of the template region are fused by feature addition to obtain the output of the first global-local joint module;

[0064] The global branch results and local branch results of the search region are fused by feature addition to obtain the output of the second global-local joint module.

[0065] The template image sequence is input into the first global-local joint module in the parameter-adjusted global-local joint module. Feature mapping is performed through the fully connected layer of the self-attention module to obtain the query vector, key vector, and value vector of the template image. Attention is calculated on the query vector, key vector, and value vector of the template image to obtain the attention score. The attention score of the template image is normalized, and then the value vector is weighted to obtain the output features of the template image. The corresponding relationship in this process is as follows:

[0066] ;

[0067] in, Represents the query vector. Represents the key vector. Represents a value vector. Indicates a fully connected layer. This represents the input to the global-local feature joint module. This represents the result of the global branch. This indicates normalization processing. Represents the key vector transpose , Represents the key vector Dimensions Perform a scaled dot product;

[0068] In the step of performing convolution processing on the template image sequence through a grouped spatial shuffling module to obtain the features of the convolutionally processed template image, the corresponding relationship in the process is as follows:

[0069] ;

[0070] in, This represents the features after convolution of the input features. Indicates convolution processing;

[0071] In the step of grouping the features of the template image after convolution along the channel dimension to obtain the features of each group of template images, the corresponding relationship in the process is as follows:

[0072] ;

[0073] in, This represents the characteristics of each group after grouping. Indicates grouping operation. Indicates the number of groups;

[0074] In the step of performing spatial attention operations on the features of each group of template images to obtain the spatial attention-calculated features of the template images, the corresponding relationship in the process is as follows:

[0075] ;

[0076] in, This represents the processed features obtained after spatial attention computation. This represents spatial attention operations.

[0077] The features of each group of template images are concatenated along the channel dimension to obtain the features of the concatenated template images. The corresponding relationship in this process is as follows:

[0078] ;

[0079] in, This represents the feature obtained by concatenating the features from each group. This indicates a splicing operation. This represents the characteristics after grouping into the first group. This represents the characteristics after grouping into the second group. This indicates the characteristics after grouping into the third group;

[0080] In the step of obtaining the channel-scrambled features of the template image by performing cross-group information interaction through channel scrambling operations on the features after spatial attention calculation of the template image and the features after template image stitching, the corresponding relationship in the process is as follows:

[0081] ;

[0082] in, This indicates the characteristics after the passage has been scrambled. This indicates a channel shuffling operation;

[0083] It should be noted that a channel shuffling mechanism is used for cross-group information exchange.

[0084] In the step of performing a residual connection operation on the features of the template image after channel scrambling and the template image sequence to obtain the features after residual processing of the template image, the corresponding relationship in the process is as follows:

[0085] ;

[0086] in, Indicates the result of a local branch;

[0087] It should be noted that residual connections are used to preserve the original features and avoid information loss.

[0088] In the step of fusing the global branch results and local branch results of the template region through feature addition to obtain the output of the first global-local joint module, the corresponding relationship in the process is as follows:

[0089] ;

[0090] in, This represents the output of the global-local joint module, which includes the output of the first global-local joint module and the output of the second global-local joint module.

[0091] It should be noted that in the global branch, the global-local feature joint module utilizes the attention mechanism's ability to dynamically compute global dependencies to extract global features, playing a crucial role in the model's extraction of global information. In the local branch, grouped spatial attention is used to extract local features from the input features, and then a channel shuffling mechanism is used for cross-group information interaction.

[0092] Step 4: Input the output of the global-local joint module into the asynchronous enhancement module after parameter adjustment for asynchronous interaction and feature enhancement to obtain the enhanced output features.

[0093] Please see Figure 2 and Figure 4 In step 4, the output of the global-local joint module is input into the asynchronous enhancement module after parameter adjustment for asynchronous interaction and feature enhancement to obtain the enhanced output features. The specific steps include the following:

[0094] The outputs of the first global-local joint module and the second global-local joint module are input into the asynchronous enhancement module after parameter adjustment to perform multi-scale operations, thereby obtaining the multi-scale features of the template region and the multi-scale features of the search region.

[0095] The multi-scale features of the template region and the multi-scale features of the search region are linearly transformed through a fully connected layer to obtain the key vector of the template region, the value vector of the template region, the query vector of the search region, the key vector of the search region, and the value vector of the search region.

[0096] The key vector of the template region is concatenated with the key vector of the search region to obtain the concatenated key vector; attention is calculated between the concatenated key vector and the query vector of the search region to obtain the attention calculation result; the value vector of the template region is concatenated with the value vector of the search region to obtain the value vector concatenation result; and the attention calculation result and the value vector concatenation result are then weighted and summed to obtain the features after asynchronous interaction.

[0097] The features after asynchronous interaction are input into the fully connected layer and segmented along the channel dimension to obtain two segmented features;

[0098] Convolution and activation operations are performed on the two segmented features respectively to obtain convolutional and activation features. Then, the convolutional and activation features are concatenated according to the channel dimension to obtain the enhanced output features.

[0099] The outputs of the first global-local joint module and the second global-local joint module are input into the parameter-adjusted asynchronous enhancement module to perform multi-scale operations, respectively, to obtain the multi-scale features of the template region and the multi-scale features of the search region. The corresponding relationship in the process is as follows:

[0100] ;

[0101] in, This represents the multi-scale features of the template region. This represents a 7×7 convolution operation. This represents a 5×5 convolution operation. This represents a 3×3 convolution operation. The input features represent the template region. This represents the multi-scale features of the search region. Indicates the input features of the search region;

[0102] In the steps of linearly transforming the multi-scale features of the template region and the multi-scale features of the search region through fully connected layers to obtain the key vector, value vector, query vector, key vector, and value vector of the template region, the corresponding relationships in the process are as follows:

[0103] ;

[0104] in, The query vector representing the template region. The key vector representing the template region. A value vector representing the template region. The query vector represents the search region. The key vector representing the search region. Represents the value vector of the search region;

[0105] The relationships between the steps of concatenating the key vector of the template region with the key vector of the search region to obtain the concatenated key vector, performing attention calculation on the concatenated key vector and the query vector of the search region to obtain the attention calculation result, concatenating the value vector of the template region with the value vector of the search region to obtain the value vector concatenation result, and finally performing a weighted summation of the attention calculation result and the value vector concatenation result to obtain the features after asynchronous interaction are as follows:

[0106] ;

[0107] in, This indicates the features after asynchronous interaction. This represents the transpose of the concatenation of the key vector of the template region and the key vector of the search region. This indicates that the value vector of the template region is concatenated with the value vector of the search region.

[0108] The features obtained after asynchronous interaction are input into the fully connected layer and segmented along the channel dimension to obtain two segmented features. The relationship between the corresponding processes is as follows:

[0109] ;

[0110] in, This represents the features after the first segmentation. This represents another feature after segmentation. Indicates a splitting operation;

[0111] In the process of performing convolution and activation operations on the two segmented features respectively to obtain convolutional and activation-enhanced features, and then concatenating the convolutional and activation-enhanced features along the channel dimension to obtain the enhanced output features, the corresponding relationships in the process are as follows:

[0112] ;

[0113] in, This represents the enhanced output features. This indicates an activation operation.

[0114] Furthermore, 3×3, 5×5, and 7×7 convolution operations are performed on the search region features, followed by asynchronous interactive computation between the search region and template image features. The output features are then input into the enhancement layer for feature enhancement. This not only achieves feature interaction and enhancement between the template image and the search region image but also enriches multi-scale information. In the feature fusion stage, asynchronous enhanced attention is used to fuse and enhance the template features and search region features.

[0115] It should be noted that the enhancement layer is the layer that incorporates features from asynchronous interactions. go through Fully connected layer, then use Divided along the channel dimension and Then... and Perform 3×3 convolution and The activation function is used, and finally, the two enhanced features are concatenated to obtain the enhanced output features. The asynchronous enhancement module first acquires multi-scale information through multi-scale operations, improving the richness of feature representation and enhancing robustness to scale changes. Then, it asynchronously interacts with the template features and search region features after acquiring multi-scale information to obtain fused features. The asynchronous enhancement module performs segmentation operations in the feature enhancement part to reduce the complexity of network learning, and then applies convolution operations and activation functions to the features separately, enhancing spatial features and mitigating information loss issues.

[0116] Step 5: Input the enhanced output features into the prediction head to obtain the tracking results. Calculate the center classification loss and boundary regression loss based on the tracking results. Optimize the tracking model using the center classification loss and boundary regression loss to obtain the optimized tracking model.

[0117] In step 5, the center classification loss and boundary regression loss are calculated based on the enhanced output features. The relationship between the center classification loss and the process is as follows:

[0118] ;

[0119] in, Indicates the central classification loss. Indicates the number of positive samples. Indicates the true label, Represents an exponential function. This indicates the probability of predicting the center. This indicates a balance between positive and negative sample parameters. Indicates the focus parameter;

[0120] The relationship between the boundary regression loss and the process is as follows:

[0121] ;

[0122] in, This represents the calculated intersection-union ratio. Represents the bounding box. This represents another bounding box. This represents the area of ​​the intersection of the two frames. This represents the area of ​​the union of the two frames. This represents the generalized intersection-union ratio. Indicates that it also includes a bounding box. and another bounding box Minimum area of ​​the circumscribed rectangle This represents the boundary regression loss.

[0123] Furthermore, in addition to the center classification loss and the boundary regression loss, there is also a total loss, and the corresponding relationship in the process is as follows:

[0124] ;

[0125] in, Indicates the total loss. This represents the weighting coefficient.

[0126] It should be noted that the central classification loss Used to confirm whether the classification target center is at the corresponding location, boundary regression loss. Used to optimize the overlap between predicted bounding boxes and ground truth bounding boxes. Set to 2.

[0127] Step 6: Use the optimized tracking model to track the target.

[0128] Furthermore, within the Transformer network architecture, a Transformer-based target tracking framework is constructed using a global-local feature joint module and asynchronous enhanced attention. The global-local feature joint module extracts global and local features simultaneously in parallel. The global branch employs a self-attention mechanism, performing a linear transformation on the input feature map to obtain query vectors, key vectors, and value vectors. The attention score map obtained by performing attention calculations on the query vectors and key vectors is then weighted and summed to extract global features. The local branch first processes the input feature map through a 1×1 convolutional layer to adjust the number of channels, then groups the features at the channel level. Spatial attention is calculated for each group of features, and the processed feature maps are concatenated along the channel dimension. A channel shuffling operation is then used for cross-group mixing. The mixed feature map is then passed through a 1×1 convolutional layer to restore the channel dimension, and finally, residual connections are used to extract local features.

[0129] The global and local features are summed to obtain the output of the global-local feature joint module. Asynchronous enhanced attention is used in the feature fusion stage. First, 3×3, 5×5, and 7×7 convolution operations are performed on the search region features. Then, asynchronous interactive calculations are performed on the search region and template image features. Finally, the output feature map is input into the enhancement layer for feature enhancement. This invention uses a global-local feature joint module to extract global and local features in the feature extraction stage, improving the completeness of feature representation. The asynchronous enhanced attention used on the template and search region in the feature fusion stage not only acquires multi-scale information and fused features but also performs feature enhancement, improving the accuracy of target localization and achieving robust tracking performance.

[0130] Please see Figure 5 The present invention also provides a target tracking system based on global-local dual-path extraction and asynchronous enhancement, the system comprising:

[0131] Build modules are used for:

[0132] The tracking model is composed of a global-local joint module, a prediction head, and an asynchronous enhancement module.

[0133] Pre-trained modules are used for:

[0134] The global-local joint module and the asynchronous enhancement module are pre-trained using a large-scale dataset, and the model parameters are adjusted to obtain the parameter-adjusted global-local joint module and the parameter-adjusted asynchronous enhancement module.

[0135] Extraction module, used for:

[0136] The template image and the search image are initialized and the sequence is divided by block embedding to obtain the template image sequence and the search image sequence. The template image sequence and the search image sequence are input into the global-local joint module after parameter adjustment. At the same time, global features and local features are extracted. Then, the global features and local features are fused to obtain the output of the global-local joint module.

[0137] The calculation module is used for:

[0138] The output of the global-local joint module is input into the asynchronous enhancement module after parameter adjustment for asynchronous interaction and feature enhancement, resulting in enhanced output features.

[0139] The learning module is used for:

[0140] The enhanced output features are input into the prediction head to obtain the tracking results. The center classification loss and boundary regression loss are calculated based on the tracking results. The tracking model is then optimized using the center classification loss and boundary regression loss to obtain the optimized tracking model.

[0141] The tracking module is used for:

[0142] Target tracking is performed using the optimized tracking model.

[0143] It should be understood that various parts of the present invention can be implemented in hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented in software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0144] In the description of this specification, references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0145] The embodiments described above are merely illustrative of several implementations of the present invention, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of the present invention. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these modifications and improvements all fall within the scope of protection of the present invention. Therefore, the scope of protection of this patent should be determined by the appended claims.

Claims

1. A target tracking method based on global-local dual-path extraction and asynchronous enhancement, characterized in that, The method comprises the following steps: Step 1, constructing a tracking model based on a global-local joint module, a prediction head and an asynchronous enhancement module; Step 2, pre-training the global-local joint module and the asynchronous enhancement module using a large-scale data set, and adjusting the model parameters to obtain the global-local joint module after parameter adjustment and the asynchronous enhancement module after parameter adjustment; Step 3, initializing the template image and the search image, and performing sequence division through block embedding to obtain a template image sequence and a search image sequence; inputting the template image sequence and the search image sequence into the global-local joint module after parameter adjustment, simultaneously extracting global features and local features, and then performing feature fusion on the global features and the local features to obtain the output of the global-local joint module; Step 4, inputting the output of the global-local joint module into the asynchronous enhancement module after parameter adjustment for asynchronous interaction and feature enhancement to obtain enhanced output features; Step 5, inputting the enhanced output features into the prediction head to obtain a tracking result, calculating a center classification loss and a boundary regression loss according to the tracking result, and optimizing the tracking model using the center classification loss and the boundary regression loss to obtain an optimized tracking model; Step 6, using the optimized tracking model for target tracking; In step 1, the tracking model is constructed based on the global-local joint module, the prediction head and the asynchronous enhancement module, wherein the global-local joint module is composed of a first global-local joint module and a second global-local joint module, and the first global-local joint module is constructed as follows: Based on the self-attention mechanism, a self-attention module is constructed, a grouping space scrambling module is constructed according to the grouping space attention mechanism and the channel scrambling mechanism, and the self-attention module and the grouping space scrambling module constitute the first global-local joint module; wherein the first global-local joint module and the second global-local joint module share weights and have the same structure; The asynchronous enhancement module is constructed as follows: Based on the feature fusion mechanism and the asynchronous enhancement attention mechanism, the asynchronous enhancement module is constructed.

2. The target tracking method based on global-local dual-path extraction and asynchronous enhancement according to claim 1, characterized in that, In step 3, the template image sequence and the search image sequence are input into the global-local joint module after parameter adjustment, global features and local features are extracted, and then the global features and the local features are fused to obtain the output of the global-local joint module, which comprises the following steps: The template image sequence is input into the first global-local joint module of the global-local joint module after parameter adjustment, feature mapping is performed through the fully connected layer of the self-attention module to obtain the query vector, the key vector and the value vector of the template image; attention calculation is performed on the query vector, the key vector and the value vector of the template image to obtain the attention score of the template image; The attention score of the template image is normalized, and the value vector is weighted to obtain the output feature of the template image, which is taken as the global branch result of the template region; The search image sequence is input into a second global-local joint module in the parameter-adjusted global-local joint module, feature mapping is performed through a full connection layer of a self-attention module to obtain a query vector, a key vector and a value vector of the search image; attention calculation is performed on the query vector, the key vector and the value vector of the search image to obtain an attention score of the search image; The attention score of the search image is normalized and weighted summed to obtain an output feature of the search image, and the output feature of the search image is taken as a global branch result of the search region; The template image sequence is subjected to convolution processing through the grouping spatial scrambling module to obtain a feature after convolution processing of the template image; the feature after convolution processing of the template image is grouped along a channel dimension to obtain a feature after grouping of each group of template images; Spatial attention operation is performed on the feature after grouping of each group of template images to obtain a feature after spatial attention calculation of the template image; Concatenation operation is performed on the feature after grouping of each group of template images along the channel dimension to obtain a feature after concatenation of the template image; The feature after spatial attention calculation of the template image and the feature after concatenation of the template image are subjected to cross-group information interaction through channel scrambling operation to obtain a feature after channel scrambling of the template image; Residual connection operation is performed on the feature after channel scrambling of the template image and the template image sequence to obtain a feature after residual processing of the template image, and the feature after residual processing of the template image is taken as a local branch result of the template region; The search image sequence is subjected to convolution processing through the grouping spatial scrambling module to obtain a feature after convolution processing of the search image; the feature after convolution processing of the search image is grouped along a channel dimension to obtain a feature after grouping of each group of search images; Spatial attention operation is performed on the feature after grouping of each group of search images to obtain a feature after spatial attention calculation of the search image; Concatenation operation is performed on the feature after grouping of each group of search images along the channel dimension to obtain a feature after concatenation of the search image; The feature after spatial attention calculation of the search image and the feature after concatenation of the search image are subjected to cross-group information interaction through channel scrambling operation to obtain a feature after channel scrambling of the search image; Residual connection operation is performed on the feature after channel scrambling of the search image and the search image sequence to obtain a feature after residual processing of the search image, and the feature after residual processing of the search image is taken as a local branch result of the search region; The global branch result of the template region and the local branch result of the template region are subjected to feature fusion through feature addition operation to obtain an output of the first global-local joint module; The global branch result of the search region and the local branch result of the search region are subjected to feature fusion through feature addition operation to obtain an output of the second global-local joint module.

3. The target tracking method based on global-local dual-path extraction and asynchronous boosting according to claim 2, characterized in that, The first global-local joint module in the parameter-adjusted global-local joint module inputs the template image sequence, maps the features through the full connection layer of the self-attention module to obtain the query vector, key vector and value vector of the template image, performs attention calculation on the query vector, key vector and value vector of the template image to obtain the attention score of the template image, normalizes the attention score of the template image, weights the value vector again, and obtains the output feature of the template image. The relationship formula existing in the corresponding process is as follows: ； in, Represents the query vector. Represents the key vector. Represents a value vector. Indicates a fully connected layer. This represents the input to the global-local feature joint module. This represents the result of the global branch. This indicates normalization processing. Represents the key vector transpose , Represents the key vector Dimensions Perform a scaled dot product; In the step of performing convolution processing on the template image sequence through the grouping space disorder module to obtain the feature of the template image after convolution processing, the relationship formula existing in the corresponding process is as follows: ； wherein, represents a feature after convolution processing on an input feature, represents convolution processing; In the step of grouping the feature of the template image after convolution processing along the channel dimension to obtain the feature of each group of template images after grouping, the relationship formula existing in the corresponding process is as follows: ； wherein, denotes the features after grouping of each component, denotes the grouping operation, denotes the number of groups; In the step of performing spatial attention operation on the feature of each group of template images after grouping to obtain the feature of the template image after spatial attention calculation, the relationship formula existing in the corresponding process is as follows: ； wherein, denotes performing spatial attention computation to obtain processed features, denotes a spatial attention operation.

4. The target tracking method based on global-local dual-path extraction and asynchronous boosting according to claim 3, characterized in that, In the step of performing spatial attention operation on the feature of each group of template images after grouping to obtain the feature of the template image after spatial attention calculation, the relationship formula existing in the corresponding process is as follows: ； wherein, represents the features after the concatenation of each group of features, represents the concatenation operation, represents the features after the grouping of the first group, represents the features after the grouping of the second group, represents the features after the grouping of the third group; In the step of performing spatial attention operation on the feature of each group of template images after grouping to obtain the feature of the template image after spatial attention calculation, the relationship formula existing in the corresponding process is as follows: ； wherein, represents the features after shuffling through the channel, represents a channel shuffling operation; In the step of performing spatial attention operation on the feature of each group of template images after grouping to obtain the feature of the template image after spatial attention calculation, the relationship formula existing in the corresponding process is as follows: ； wherein represents a local branch result; In the step of performing residual connection operation on the feature of the template image after channel disorder and the template image sequence to obtain the feature of the template image after residual processing, the relationship formula existing in the corresponding process is as follows: ； wherein, represents the output of the global-local joint module.

5. The target tracking method based on global-local dual-path extraction and asynchronous boosting according to claim 4, characterized in that, In the step of performing residual connection operation on the feature of the template image after channel disorder and the template image sequence to obtain the feature of the template image after residual processing, the relationship formula existing in the corresponding process is as follows: In the step of performing residual connection operation on the feature of the template image after channel disorder and the template image sequence to obtain the feature of the template image after residual processing, the relationship formula existing in the corresponding process is as follows: In the step 4, the output of the global-local joint module is input into the parameter-adjusted asynchronous enhancement module for asynchronous interaction and feature enhancement to obtain the enhanced output feature, which specifically includes the following steps: The output of the first global-local joint module and the output of the second global-local joint module are input into the parameter-adjusted asynchronous enhancement module for multi-scale operation respectively to obtain the multi-scale feature of the template region and the multi-scale feature of the search region. The multi-scale feature of the template region and the multi-scale feature of the search region are linearly transformed through the full connection layer respectively to obtain the key vector of the template region, the value vector of the template region, the query vector of the search region, the key vector of the search region and the value vector of the search region. The key vector of the template region and the key vector of the search region are spliced to obtain the spliced key vector, the spliced key vector and the query vector of the search region are calculated by attention to obtain the attention calculation result, the value vector of the template region and the value vector of the search region are spliced to obtain the value vector splicing result, and the attention calculation result and the value vector splicing result are weighted and summed to obtain the asynchronous interaction feature. The asynchronous interaction feature is input into a full connection layer, and a splitting operation is performed along the channel dimension to obtain two split features; The two split features are respectively subjected to convolution operation and activation operation to obtain a convolution operation feature and an activation operation feature, and the convolution operation feature and the activation operation feature are spliced along the channel dimension to obtain an enhanced output feature.

6. The target tracking method based on global-local dual-path extraction and asynchronous boosting according to claim 5, characterized in that, The outputs of the first global-local joint module and the second global-local joint module are input into the parameter-adjusted asynchronous enhancement module to perform multi-scale operations to obtain multi-scale features of the template region and multi-scale features of the search region, and the relationship formula existing in the corresponding process is as follows: ； wherein, denotes a multi-scale feature of a template region, denotes a 7x7 convolution operation, denotes a 5x5 convolution operation, denotes a 3x3 convolution operation, denotes an input feature of a template region, denotes a multi-scale feature of a search region, denotes an input feature of a search region; In the steps of performing linear transformation processing on the multi-scale features of the template region and the multi-scale features of the search region through the full connection layer to obtain the key vector of the template region, the value vector of the template region, the query vector of the search region, the key vector of the search region, and the value vector of the search region, the relationship formula existing in the corresponding process is as follows: ； wherein, a query vector representing a template region, a key vector representing a template region, a value vector representing a template region, a query vector representing a search region, a key vector representing a search region, a search region value vector; In the steps of splicing the key vector of the template region and the key vector of the search region to obtain a spliced key vector, performing attention calculation on the spliced key vector and the query vector of the search region to obtain an attention calculation result, splicing the value vector of the template region and the value vector of the search region to obtain a value vector splicing result, and performing weighted summation on the attention calculation result and the value vector splicing result to obtain an asynchronous interaction feature, the relationship formula existing in the corresponding process is as follows: ； wherein, represents the features after asynchronous interaction, represents the transpose of the concatenation of the key vector of the template region and the key vector of the search region, represents the concatenation of the value vector of the template region and the value vector of the search region.

7. The target tracking method based on global-local dual-path extraction and asynchronous boosting according to claim 6, characterized in that, The asynchronous interaction feature is input into a full connection layer, and a splitting operation is performed along the channel dimension to obtain two split features, and the relationship formula existing in the corresponding process is as follows: ； wherein, represents a first segmented feature, represents another segmented feature, represents a segmentation operation; In the steps of performing convolution operation and activation operation on the two split features to obtain a convolution operation feature and an activation operation feature, and splicing the convolution operation feature and the activation operation feature along the channel dimension to obtain an enhanced output feature, the relationship formula existing in the corresponding process is as follows: ； wherein, represents the enhanced output feature, represents the activation operation.

8. The target tracking method based on global-local dual-path extraction and asynchronous boosting according to claim 7, characterized in that, In the step 5, the center classification loss and the boundary regression loss are calculated according to the tracking result, and the relationship formula existing in the corresponding process of the center classification loss is as follows: ； wherein, denotes a center classification loss, denotes a number of positive samples, denotes a true label, denotes an exponential function, denotes a probability of being centered, denotes a balancing positive-negative sample parameter, denotes a focus parameter; The relationship formula existing in the corresponding process of the boundary regression loss is as follows: ； in, This represents the calculated intersection-union ratio. Represents the bounding box. This represents another bounding box. This represents the area of ​​the intersection of the two frames. This represents the area of ​​the union of the two frames. This represents the generalized intersection-union ratio. Indicates that it also includes a bounding box. and another bounding box Minimum area of ​​the circumscribed rectangle This represents the boundary regression loss.

9. A target tracking system based on global-local dual-path extraction and asynchronous augmentation, characterized in that, The system applies the target tracking method based on global-local dual-path extraction and asynchronous enhancement according to any one of claims 1 to 8, and the system comprises: A construction module is configured to: construct a tracking model based on the global-local joint module, the prediction head, and the asynchronous enhancement module; A pre-training module is configured to: pre-train the global-local joint module and the asynchronous enhancement module using a large-scale data set, and adjust the model parameters to obtain the parameter-adjusted global-local joint module and the parameter-adjusted asynchronous enhancement module; An extraction module is configured to: initialize the template image and the search image, perform sequence division through block embedding to obtain a template image sequence and a search image sequence, input the template image sequence and the search image sequence into the parameter-adjusted global-local joint module, simultaneously extract global features and local features, and perform feature fusion on the global features and the local features to obtain the output of the global-local joint module; The computing module is configured to: input the output of the global-local joint module into the parameter-adjusted asynchronous enhancement module for asynchronous interaction and feature enhancement, to obtain enhanced output features; The learning module is configured to: input the enhanced output features into a prediction head to obtain a tracking result, calculate a center classification loss and a boundary regression loss according to the tracking result, optimize the tracking model by using the center classification loss and the boundary regression loss, and obtain an optimized tracking model; The tracking module is configured to: perform target tracking by using the optimized tracking model.

Images