Target tracking method and system based on dual-domain attention and wavelet enhancement module

Abstract

The application provides a target tracking method and system based on a dual-domain attention and a wavelet enhancement module, the method comprising: pre-processing a template image and a search image, and inputting parameters into a feature extraction network, extracting features of the template image and the search image in a space domain and a channel domain through dual-domain attention of the feature extraction network; splicing the template image sequence features and the search image sequence features along a channel dimension, and using a self-attention module and a wavelet enhancement module of a parameter adjustment feature fusion network to perform interactive fusion and feature enhancement after splicing; and feeding the fused features into a prediction head to obtain a tracking result. The dual-domain attention extracts the space domain features and the channel domain features of the input image sequence, so that the model can pay attention to the spatial position of the target and model the importance of the channel semantics, realize dual-domain complementary modeling, more accurately locate the target region, and effectively distinguish the target features from the noise channels.

Description

Technical Field

[0001] This invention relates to the field of computer vision and image processing technology, and in particular to a target tracking method and system based on dual-domain attention and wavelet enhancement modules. Background Technology

[0002] Visual object tracking, an important branch of computer vision, aims to accurately predict the location of a target in each subsequent frame of a given video sequence, given the target's position in the initial frame. It has been widely applied in various practical fields such as human-computer interaction, autonomous driving, and video surveillance. In recent years, with the rapid development of Transformers, visual object tracking has reached new heights, its applications in daily life have become more widespread, and its research value has become increasingly prominent. However, current visual object tracking still faces a series of serious challenges. When the target is similar to the background, attention can easily divert to non-target areas; when the target's appearance changes significantly, tracking drift can easily occur. These interfering factors prevent the tracker from accurately tracking the target in subsequent frames.

[0003] In recent years, researchers have tried various methods to improve the accuracy of trackers. Currently, the mainstream methods, such as CNN, Transformer and Mamba, extract features from spatial location when used alone, which can easily lead to loss of channel semantics, resulting in blurred edge details and the tendency to focus on background information in complex texture scenes. In addition, motion blur makes the extracted edge features less obvious. Low frequencies in the wavelet domain are rarely affected by blur, and high frequencies can still extract directional information from the blur. However, the filters of traditional wavelets are fixed and cannot be learned, so they cannot cope with complex scene changes and also lose spatial information. Summary of the Invention

[0004] In view of the above, the main objective of this invention is to propose a target tracking method and system based on dual-domain attention and wavelet enhancement modules to solve the aforementioned technical problems.

[0005] This invention proposes a target tracking method based on dual-domain attention and wavelet enhancement modules, the method comprising the following steps:

[0006] Step 1: Construct a tracking model based on a feature extraction network, a feature fusion network, and a prediction head;

[0007] Step 2: Pre-train the feature extraction network and the feature fusion network using a large-scale dataset, and adjust the parameters to obtain the parameter-adjusted feature extraction network and the parameter-adjusted feature fusion network;

[0008] Step 3: Preprocess the template image and the search image, and input the parameters into the feature extraction network. By adjusting the dual-domain attention of the feature extraction network through parameter adjustment, the features of the template image and the search image in the spatial domain and the channel domain are extracted respectively to obtain the template image sequence features and the search image sequence features.

[0009] Step 4: Concatenate the template image sequence features and the search image sequence features along the channel dimension. After concatenation, use the parameters to adjust the self-attention module and wavelet enhancement module of the feature fusion network to perform interactive fusion and feature enhancement to obtain the enhanced search image sequence features.

[0010] Step 5: Replace the search image sequence features with the enhanced search image sequence features, and then concatenate them with the template image sequence features again. Repeat step 4 a preset number of times in an iterative manner to obtain the fused features.

[0011] Step 6: Feed the fused features into the prediction head to obtain the tracking results, calculate the classification loss and regression loss based on the tracking results, and optimize the tracking model to obtain the optimized tracking model;

[0012] Target tracking is performed using the optimized tracking model.

[0013] This invention also proposes a target tracking system based on dual-domain attention and wavelet enhancement modules, the system comprising:

[0014] Build modules are used for:

[0015] The tracking model is composed of a feature extraction network, a feature fusion network, and a prediction head.

[0016] Pre-trained modules are used for:

[0017] The feature extraction network and the feature fusion network are pre-trained using a large-scale dataset, and the parameters are adjusted to obtain the parameter-adjusted feature extraction network and the parameter-adjusted feature fusion network.

[0018] Extraction module, used for:

[0019] The template image and the search image are preprocessed and input into the parameter adjustment feature extraction network. The dual-domain attention of the feature extraction network is adjusted by the parameter adjustment to extract the features of the template image and the search image in the spatial domain and the channel domain respectively, so as to obtain the template image sequence features and the search image sequence features.

[0020] The learning module is used for:

[0021] The template image sequence features and the search image sequence features are concatenated along the channel dimension. After concatenation, the self-attention module and wavelet enhancement module of the feature fusion network are adjusted by parameters to perform interactive fusion and feature enhancement to obtain the enhanced search image sequence features.

[0022] The calculation module is used for:

[0023] The enhanced search image sequence features are replaced with the search image sequence features, and then concatenated with the template image sequence features again. The extraction module is repeatedly extracted a preset number of times in an iterative manner to obtain the fused features.

[0024] The tracking module is used for:

[0025] The fused features are fed into the prediction head to obtain the tracking results. The classification loss and regression loss are calculated based on the tracking results, and the tracking model is optimized to obtain the optimized tracking model.

[0026] Target tracking is performed using the optimized tracking model.

[0027] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0028] 1. This invention extracts spatial and channel features of the input image sequence through dual-domain attention, enabling the model to focus on both the spatial location of the target and the importance of channel semantics, achieving complementary dual-domain modeling, more accurately locating the target region, and effectively distinguishing target features from noise channels.

[0029] 2. By inventing the channel domain branch in the dual-domain attention, this invention reconstructs the channels of the input image sequence by grouping them, models the channel dependencies between different sub-sequences, reduces redundant computation between channels, and improves the utilization rate of channel features.

[0030] 3. The present invention provides rich scale features for the model by using a series of convolution operations before wavelet transform through the wavelet enhancement module described in the invention, suppresses noise through a simple gating mechanism, enhances nonlinear representation, and finally decomposes the target direction structure into different frequency bands through wavelet transform, providing a more structured feature expression.

[0031] Additional aspects and advantages of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by means of embodiments of the invention. Attached Figure Description

[0032] Figure 1 This is a flowchart illustrating the steps of the target tracking method based on dual-domain attention and wavelet enhancement module proposed in this invention.

[0033] Figure 2 This is a structural diagram of the target tracking framework proposed in this invention, which is based on dual-domain attention and wavelet enhancement modules.

[0034] Figure 3This is a schematic diagram of the dual-domain attention principle of the target tracking method based on dual-domain attention and wavelet enhancement module proposed in this invention.

[0035] Figure 4 This is a schematic diagram of the wavelet enhancement module in the target tracking method based on dual-domain attention and wavelet enhancement module proposed in this invention.

[0036] Figure 5 This is a schematic diagram of the target tracking system based on dual-domain attention and wavelet enhancement module proposed in this invention. Detailed Implementation

[0037] Embodiments of the present invention are described in detail below. Examples of these embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and should not be construed as limiting the present invention.

[0038] These and other aspects of the embodiments of the present invention will become clear from the following description and accompanying drawings. In these descriptions and drawings, some specific embodiments of the present invention are specifically disclosed to provide some ways of implementing the principles of the embodiments of the present invention; however, it should be understood that the scope of the embodiments of the present invention is not limited thereto.

[0039] Please see Figure 1 This invention proposes a target tracking method based on dual-domain attention and wavelet enhancement modules, which includes the following steps:

[0040] Step 1: Construct a tracking model based on a feature extraction network, a feature fusion network, and a prediction head.

[0041] Step 2: Pre-train the feature extraction network and the feature fusion network using a large-scale dataset, and adjust the parameters to obtain the parameter-adjusted feature extraction network and the parameter-adjusted feature fusion network.

[0042] Step 3: Preprocess the template image and the search image, and input the parameters into the feature extraction network. By adjusting the dual-domain attention of the feature extraction network through parameter adjustment, the features of the template image and the search image in the spatial domain and the channel domain are extracted respectively to obtain the template image sequence features and the search image sequence features.

[0043] Please see Figure 2 and Figure 3 In step 3, the template image and the search image are preprocessed and input into the parameter adjustment feature extraction network. The dual-domain attention of the feature extraction network is adjusted by the parameter adjustment to extract the features of the template image and the search image in the spatial domain and the channel domain, respectively, so as to obtain the template image sequence features and the search image sequence features. Specifically, the steps are as follows:

[0044] The template image and the search image are each divided into image blocks of the same size that do not overlap, and the image blocks of the same size that do not overlap are converted into a template image sequence and a search image sequence.

[0045] The template image sequence and the search image sequence are input into the parameter adjustment feature extraction network. The self-attention of the dual-domain attention in the feature extraction network is adjusted by the parameter adjustment to perform self-attention processing on the template image sequence and the search image sequence in the spatial domain, so as to obtain the attention weight matrix. The attention weight matrix is ​​normalized and multiplied with the value matrix to obtain the output in the spatial domain.

[0046] In the channel domain, spatial average pooling is performed on the template image sequence and the search image sequence to obtain features after spatial average pooling. The features after spatial average pooling are then grouped by channel to obtain the channel arrangement of the feature group. The channel arrangement of the feature group is then subjected to channel swapping to perform cross-channel information interaction and obtain the features after channel swapping.

[0047] The features after channel swapping are subjected to global average pooling to obtain the global average pooled features. The global average pooled features are then subjected to convolution operations to learn the dependencies between channels and obtain feature weights. The feature weights are then weighted by an activation function to obtain the mapped channel weights. The mapped channel weights are then reassembled back into the original channel order to obtain the channel weights after activation function mapping. Finally, the channel weights after activation function mapping are weighted and fused with the template image sequence and the search image sequence through multiplication operations to obtain the channel domain output.

[0048] The outputs of the spatial domain and the channel domain are added together and processed through a linear layer to unify the dimensions, resulting in the output of the dual-domain attention. The output of the dual-domain attention includes template image sequence features and search image sequence features.

[0049] The template image sequence and the search image sequence are input into the feature extraction network. The self-attention mechanism of the dual-domain attention within the feature extraction network is adjusted by these parameters. Self-attention processing is then performed on the template image sequence and the search image sequence in the spatial domain to obtain the attention weight matrix. This attention weight matrix is ​​then normalized and multiplied by the value matrix to obtain the spatial domain output. The corresponding relationship in this process is as follows:

[0050] ;

[0051] in, Represents the attention weight matrix. Represents the query matrix. Represents the transpose of the key matrix , This represents the dimension of the key matrix. Represents the output of the spatial domain. This indicates normalization processing. Represents a value matrix;

[0052] The steps of performing spatial average pooling on the template image sequence and the search image sequence to obtain spatially average pooled features; grouping the spatially average pooled features by channel to obtain a channel arrangement of the feature groupings; and exchanging the channel arrangement of the feature groupings to perform cross-channel information exchange to obtain the features after channel exchange, are related as follows:

[0053] ;

[0054] in, This represents the features after spatial average pooling. This indicates spatial average pooling. This represents the sequence of the input image after preprocessing. This indicates the channel arrangement after feature grouping. This represents the first group of channels in the first subsequence after grouping all channels in the sequence. This represents the first subsequence after grouping all channels in the sequence. Group channel, This represents the last group of channels in the first subsequence after all channels in the sequence have been grouped. This indicates the sequence number after grouping all channels. The first group of channels of each subsequence, This indicates the sequence number after grouping all channels. The first subsequence Group channel, This indicates the sequence number after grouping all channels. The last group of channels in each subsequence This represents the first group of channels in the last subsequence after grouping all channels in the sequence. This represents the last subsequence after grouping all channels in the sequence. Group channel, This represents the last group of channels in the last subsequence after all channels in the sequence have been grouped. Indicates the characteristics after the switching channel. This indicates channel exchange processing between subsequences.

[0055] It should be noted that, This represents the quantity in each subsequence, i.e. This represents the last group of channels in the first subsequence after all channels in the sequence have been grouped. Then it represents the number of subsequences, i.e. This represents the first group of channels in the last subsequence after all channels in the sequence have been grouped.

[0056] The features after channel swapping are subjected to global average pooling to obtain globally average pooled features. These globally average pooled features are then processed by convolution to learn the channel dependencies and obtain feature weights. The feature weights are then weighted using an activation function to obtain mapped channel weights. The mapped channel weights are then reassembled back into their original channel order to obtain channel weights mapped by the activation function. Finally, these channel weights are weighted and fused with the template image sequence and the search image sequence through multiplication to obtain the channel domain output. The relationships in this process are as follows:

[0057] ;

[0058] in, This represents the channel weights after being mapped by the activation function. This means mapping the channel weights back to the original channel order. This represents the activation function. express Convolution operation, This indicates global average pooling. Indicates the output of the channel domain. Indicates the multiplication operation;

[0059] The outputs of the spatial domain and the channel domain are added together and processed through a linear layer to unify the dimensions, resulting in the output of the dual-domain attention. The relationship between the steps in which the output of the dual-domain attention includes template image sequence features and search image sequence features is as follows:

[0060] ;

[0061] in, This represents the output of dual-domain attention. This indicates that the data has been processed through a linear layer.

[0062] Furthermore, by employing dual-domain attention, the global dependency between any two subsequences is calculated through self-attention, directly capturing long-distance relationships in the spatial domain. This concentrates attention on the target region and suppresses irrelevant locations. In the channel domain, the channels of the input image subsequence are first divided into several equal parts, and the channels from different subsequences are concatenated. Channel dependencies and weights between subsequences are calculated through global average pooling, convolution operations, and the Sigmoid activation function. Finally, the obtained channel weights are reassembled back into the original channel order and fused with the features of the original input image sequence to enhance the feature representation capabilities of different channels. The outputs from the spatial domain and the channel domain are added together, and the dimension is unified through a linear layer to obtain the output of dual-domain attention.

[0063] Step 4: Concatenate the template image sequence features and the search image sequence features along the channel dimension. After concatenation, use the parameter adjustment of the self-attention module and wavelet enhancement module of the feature fusion network to perform interactive fusion and feature enhancement to obtain the enhanced search image sequence features.

[0064] Please see Figure 4 In step 4, the template image sequence features and the search image sequence features are concatenated along the channel dimension. After concatenation, the self-attention module and wavelet enhancement module of the feature fusion network are adjusted using parameters to perform interactive fusion and feature enhancement to obtain the enhanced search image sequence features. Specifically, the steps are as follows:

[0065] The template image sequence features and the search image sequence features are concatenated along the channel dimension to obtain the concatenated image sequence;

[0066] The stitched image sequence is processed by the self-attention module of the feature fusion network to obtain the self-attention processed image sequence.

[0067] The self-attention processed image sequence is segmented to obtain the segmented search image sequence;

[0068] The segmented search image sequence is input into the wavelet enhancement module, and two types of convolution are used to extract basic local features. Three parallel depthwise separable convolutions are used to extract three multi-scale spatial features from the basic local features. The three multi-scale spatial features are then concatenated to obtain the output result after a series of convolutions.

[0069] The number of channels in the output after a series of convolutions is expanded to obtain the channel-expanded features. The channel-expanded features are then averaged along the channels and input into the simple gating of the wavelet enhancement module for simple gating mechanism processing to obtain the features processed by the simple gating mechanism. The features processed by the simple gating mechanism are then residually fused with the search image sequence features to obtain the residually fused sequence features.

[0070] The residual-fused sequence features are processed using a normalization layer to obtain normalized features. The normalized features are then decomposed into four frequency bands in the wavelet domain using wavelet transform. These four frequency bands are then concatenated and fused, and convolution is used to further enhance the structure, resulting in a structure-enhanced product. This structure-enhanced product is then processed using a nonlinear activation function to obtain a result after activation function processing. The residual-fused sequence features are then convolved to obtain a second convolution-processed feature. Finally, the result after activation function processing and the second convolution-processed feature are subjected to residual fusion to obtain the enhanced search image sequence features from the wavelet enhancement module.

[0071] The template image sequence features and the search image sequence features are concatenated along the channel dimension to obtain the concatenated image sequence. The corresponding relationship in this process is as follows:

[0072] ;

[0073] in, This represents the stitched image sequence. This indicates a splicing operation. Represents a template image sequence. Indicates a search for a sequence of images;

[0074] In the step of performing self-attention processing on the stitched image sequence through the self-attention module of the feature fusion network, the corresponding relationship in the process is as follows:

[0075] ;

[0076] in, This represents the output of self-attention during the feature fusion stage. This represents the query matrix derived from the search image sequence. Represents the transpose of the key matrix from the template image sequence , Represents the transpose of the key matrix from the search image sequence , This represents a value matrix derived from a template image sequence. This represents a value matrix derived from the search image sequence;

[0077] In the step of segmenting the self-attention processed image sequence to obtain the segmented search image sequence, the corresponding relationship in the process is as follows:

[0078] ;

[0079] in, This represents the segmented search image sequence. This indicates a splitting operation. This represents the image sequence after self-attention processing.

[0080] The segmented search image sequence is input into the wavelet enhancement module, where two types of convolution are used to extract basic local features. Three parallel depthwise separable convolutions are then used to extract three multi-scale spatial features from the basic local features. These three multi-scale spatial features are then concatenated to obtain the output result after a series of convolutions. The relationships between these processes are as follows:

[0081] ;

[0082] in, This represents the features extracted through 1x1 convolution and 3x3 convolution. This represents a 3x3 convolution operation. This represents the output of a depthwise separable convolution with a kernel of 1. This indicates a depthwise separable convolution with a kernel of 1. This represents the output of a depthwise separable convolution with a kernel of 3. This indicates a depthwise separable convolution with a kernel of 3. This represents the output of a depthwise separable convolution with a kernel of 5. This indicates a depthwise separable convolution with a kernel of 5. This represents the output result after a series of convolutions;

[0083] The steps of expanding the number of channels in the output after a series of convolutions to obtain expanded features; dividing the expanded features along the channels by average and inputting them into a simple gating mechanism of the wavelet enhancement module for simple gating processing to obtain features processed by the simple gating mechanism; and performing residual fusion between the features processed by the simple gating mechanism and the search image sequence features to obtain the residual fused sequence features, have the following relationships:

[0084] ;

[0085] in, This represents the first feature after segmentation along the channel dimension. This represents the second feature after segmentation along the channel dimension. This represents the characteristics of a simple gated output. This represents the sequence features after residual fusion.

[0086] The residual-fused sequence features are processed using a normalization layer to obtain normalized features. The normalized features are then decomposed into four frequency bands in the wavelet domain using wavelet transform. These four frequency bands are then concatenated and fused, and convolution is used to further enhance the structure, resulting in an enhanced structure. This enhanced structure is then processed using a nonlinear activation function, resulting in an activated structure. The residual-fused sequence features are then convolved to obtain a second convolutional feature. Finally, the activated structure feature and the second convolutional feature are subjected to residual fusion to obtain the enhanced search image sequence features from the wavelet enhancement module. The relationships between these processes are as follows:

[0087] ;

[0088] in, , , and These represent the four frequency bands generated in the wavelet domain. This represents the wavelet transform operation. express, This represents the features of the search image sequence enhanced by the wavelet enhancement module. This represents the activation function.

[0089] Furthermore, the template image sequence and the search image sequence are concatenated along the channel dimension. In the feature fusion network, the concatenated image sequence is first interactively fused using self-attention. Then, the search image sequence is segmented and sent to the wavelet enhancement module for processing. In this module, a series of convolutions and simple gating operations are first used to capture spatial feature information. Then, wavelet transform is used to decompose the target orientation structure into different frequency bands, and the decomposed frequency bands are used to enhance the target orientation texture and structure. The search image sequence processed by the wavelet enhancement module replaces the initial search image sequence sent to the feature fusion stage. The new search image sequence is then concatenated with the original template image sequence sent to the feature fusion stage. The concatenated image sequence is then sent back to the feature fusion stage to perform the same operation for further interactive fusion. The number of iterations is consistent with the number of layers in the feature fusion network, which is set to 3 in this system.

[0090] Step 5: Replace the search image sequence features with the enhanced search image sequence features, and then concatenate them with the template image sequence features again. Repeat step 4 a preset number of times in an iterative manner to obtain the fused features.

[0091] It should be noted that, utilizing the trained feature extraction network model and feature fusion network model, feature extraction in the spatial and channel domains of the input image sequence is performed through dual-domain attention during the feature extraction stage. Self-attention captures long-distance relationships in the spatial domain, calculating the dependency between any two sub-sequences. In the channel domain, the channels of the sub-sequences are divided into several equal parts, and the channels of different sub-sequences are concatenated. Global average pooling, convolution operations, and the sigmoid activation function are used to calculate the channel dependencies and weights between the sub-sequences. These weights are then reassembled back to the original channel order and fused with the features of the original input image sequence to enhance the feature representation capabilities of different channels. The outputs from the spatial and channel domains are summed and then passed through a linear layer to unify the dimensions, obtaining the output of the dual-domain attention. The template image sequence and the search image sequence after the feature extraction stage are concatenated along the channel dimension as input for the feature fusion stage. In the feature fusion stage, the concatenated image sequence first undergoes interactive fusion through self-attention, then the search image sequence is segmented. The segmented sequence uses a wavelet enhancement module to decompose the target orientation structure into different frequency bands, and the decomposed frequency bands are used to enhance the target orientation texture and structure. The wavelet-enhanced search image sequence replaces the initial search image sequence sent to the feature fusion stage. The new search image sequence is then concatenated with the original template image sequence sent to the feature fusion stage. The concatenated image sequence is then sent back to the feature fusion stage to perform the same operation for further interactive fusion. The number of iterations is consistent with the number of feature fusion network layers, which is set to 3 in this system.

[0092] Step 6: Feed the fused features into the prediction head to obtain the tracking results, calculate the classification loss and regression loss based on the tracking results, and optimize the tracking model to obtain the optimized tracking model;

[0093] Target tracking is performed using the optimized tracking model.

[0094] The prediction head consists of three convolutional branches: center classification, offset regression, and size regression. The center classification branch outputs a center score map, where each score represents the confidence level of the target center at the corresponding location. The offset regression branch calculates the dispersion error of the center, and the size regression branch predicts the height and width of the target. The location with the highest confidence in the center score map is taken as the target location. The target bounding box is predicted using the corresponding regression coordinates. Classification is performed using cross-entropy loss, and the loss is used to calculate the bounding box regression. The total loss function can be expressed as:

[0095] ;

[0096] in, Indicates the total loss. Represents cross-entropy loss, and These represent the regularization parameters, This represents the regression loss based on the generalized intersection-union loss. This represents the regression loss based on L1 loss.

[0097] This invention also proposes a target tracking system based on dual-domain attention and wavelet enhancement modules, the system comprising:

[0098] Build modules are used for:

[0099] The tracking model is composed of a feature extraction network, a feature fusion network, and a prediction head.

[0100] Pre-trained modules are used for:

[0101] The feature extraction network and the feature fusion network are pre-trained using a large-scale dataset, and the parameters are adjusted to obtain the parameter-adjusted feature extraction network and the parameter-adjusted feature fusion network.

[0102] Extraction module, used for:

[0103] The template image and the search image are preprocessed and input into the parameter adjustment feature extraction network. The dual-domain attention of the feature extraction network is adjusted by the parameter adjustment to extract the features of the template image and the search image in the spatial domain and the channel domain respectively, so as to obtain the template image sequence features and the search image sequence features.

[0104] The learning module is used for:

[0105] The template image sequence features and the search image sequence features are concatenated along the channel dimension. After concatenation, the self-attention module and wavelet enhancement module of the feature fusion network are adjusted by parameters to perform interactive fusion and feature enhancement to obtain the enhanced search image sequence features.

[0106] The calculation module is used for:

[0107] The enhanced search image sequence features are replaced with the search image sequence features, and then concatenated with the template image sequence features again. The extraction module is repeatedly extracted a preset number of times in an iterative manner to obtain the fused features.

[0108] The tracking module is used for:

[0109] The fused features are fed into the prediction head to obtain the tracking results. The classification loss and regression loss are calculated based on the tracking results, and the tracking model is optimized to obtain the optimized tracking model.

[0110] Target tracking is performed using the optimized tracking model.

[0111] It should be understood that various parts of the present invention can be implemented in hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented in software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0112] In the description of this specification, references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0113] The embodiments described above are merely illustrative of several implementations of the present invention, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of the present invention. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these modifications and improvements all fall within the scope of protection of the present invention. Therefore, the scope of protection of this patent should be determined by the appended claims.

Claims

1. A target tracking method based on dual-domain attention and wavelet enhancement module, characterized in that, The method includes the following steps: Step 1: Construct a tracking model based on a feature extraction network, a feature fusion network, and a prediction head; Step 2: Pre-train the feature extraction network and the feature fusion network using a large-scale dataset, and adjust the parameters to obtain the parameter-adjusted feature extraction network and the parameter-adjusted feature fusion network; Step 3: Preprocess the template image and search image, and input the parameters into the feature extraction network. By adjusting the dual-domain attention of the feature extraction network, the features of the template image and search image in the spatial domain and channel domain are extracted respectively to obtain the template image sequence features and search image sequence features. The specific steps include the following: The template image and the search image are each divided into image blocks of the same size that do not overlap, and the image blocks of the same size that do not overlap are converted into a template image sequence and a search image sequence. The template image sequence and the search image sequence are input into the feature extraction network. The self-attention of the dual-domain attention in the feature extraction network is adjusted by adjusting the parameters. The self-attention processing of the template image sequence and the search image sequence is performed in the spatial domain to obtain the attention weight matrix. The attention weight matrix is ​​normalized and multiplied with the value matrix to obtain the output in the spatial domain. In the channel domain, spatial average pooling is performed on the template image sequence and the search image sequence to obtain features after spatial average pooling. The features after spatial average pooling are then grouped by channel to obtain the channel arrangement of the feature group. The channel arrangement of the feature group is then subjected to channel swapping to perform cross-channel information interaction and obtain the features after channel swapping. The features after channel swapping are subjected to global average pooling to obtain the features after global average pooling. After global average pooling, the features are processed by convolutional operations to learn the dependencies between channels and obtain feature weights. The feature weights are mapped using an activation function to obtain the mapped channel weights; The mapped channel weights are reassembled back to the original channel order to obtain the channel weights mapped by the activation function. The channel weights mapped by the activation function are then multiplied and fused with the template image sequence and the search image sequence to obtain the output of the channel domain. The outputs of the spatial domain and the channel domain are added together and processed through a linear layer to unify the dimensions, resulting in the output of the dual-domain attention. The output of the dual-domain attention includes template image sequence features and search image sequence features. Step 4: Concatenate the template image sequence features and the search image sequence features along the channel dimension. After concatenation, use the parameters to adjust the self-attention module and wavelet enhancement module of the feature fusion network to perform interactive fusion and feature enhancement to obtain the enhanced search image sequence features. Step 5: Replace the search image sequence features with the enhanced search image sequence features, and then concatenate them with the template image sequence features again. Repeat step 4 a preset number of times in an iterative manner to obtain the fused features. Step 6: Feed the fused features into the prediction head to obtain the tracking results, calculate the classification loss and regression loss based on the tracking results, and optimize the tracking model to obtain the optimized tracking model; Target tracking is performed using the optimized tracking model.

2. The target tracking method based on dual-domain attention and wavelet enhancement module according to claim 1, characterized in that, The template image sequence and the search image sequence are input into the feature extraction network. The self-attention mechanism of the dual-domain attention within the feature extraction network is then used to perform self-attention processing on the template image sequence and the search image sequence in the spatial domain, resulting in an attention weight matrix. This attention weight matrix is ​​then normalized and multiplied by the value matrix to obtain the output in the spatial domain. The corresponding relationship in this process is as follows: ； in, Represents the attention weight matrix. Represents the query matrix. This represents the transpose of the key matrix. This represents the dimension of the key matrix. Represents the output of the spatial domain. This indicates normalization processing. Represents a value matrix; The steps of performing spatial average pooling on the template image sequence and the search image sequence to obtain spatially average pooled features; grouping the spatially average pooled features by channel to obtain a channel arrangement of the feature groupings; and exchanging the channel arrangement of the feature groupings to perform cross-channel information exchange to obtain the features after channel exchange, are related as follows: ； in, This represents the features after spatial average pooling. This indicates spatial average pooling. This represents the sequence of the input image after preprocessing. This indicates the channel arrangement after feature grouping. This represents the first group of channels in the first subsequence after grouping all channels in the sequence. This represents the first subsequence after grouping all channels in the sequence. Group channels, This represents the last group of channels in the first subsequence after all channels in the sequence have been grouped. This indicates the sequence number after grouping all channels. The first group of channels in the subsequences This indicates the sequence number after grouping all channels. The first subsequence Group channels, This indicates the sequence number after grouping all channels. The last group of channels in each subsequence This represents the first group of channels in the last subsequence after all channels in the sequence have been grouped. This represents the last subsequence after grouping all channels in the sequence. Group channel, This represents the last group of channels in the last subsequence after all channels in the sequence have been grouped. Indicates the characteristics after the switching channel. This indicates channel exchange processing between subsequences.

3. The target tracking method based on dual-domain attention and wavelet enhancement module according to claim 2, characterized in that, The features after channel swapping are subjected to global average pooling to obtain the features after global average pooling. After global average pooling, the features are processed by convolutional operations to learn the dependencies between channels and obtain feature weights. The feature weights are mapped using an activation function to obtain the mapped channel weights; The mapped channel weights are reassembled back to their original channel order to obtain the channel weights mapped by the activation function. These mapped channel weights are then multiplied and weighted to fuse the template image sequence and the search image sequence, yielding the channel domain output. The corresponding relationship in this process is as follows: ； in, This represents the channel weights after being mapped by the activation function. This means mapping the channel weights back to the original channel order. This represents the activation function. express Convolution operation, This indicates global average pooling. Indicates the output of the channel domain. Indicates the multiplication operation; The outputs of the spatial domain and the channel domain are added together and processed through a linear layer to unify the dimensions, resulting in the output of the dual-domain attention. The relationship between the steps in which the output of the dual-domain attention includes template image sequence features and search image sequence features is as follows: ； in, This represents the output of dual-domain attention. This indicates that the data has been processed through a linear layer.

4. The target tracking method based on dual-domain attention and wavelet enhancement module according to claim 3, characterized in that, In step 4, the template image sequence features and the search image sequence features are concatenated along the channel dimension. After concatenation, the self-attention module and wavelet enhancement module of the feature fusion network are adjusted using parameters to perform interactive fusion and feature enhancement to obtain the enhanced search image sequence features. Specifically, this includes the following steps: The template image sequence features and the search image sequence features are concatenated along the channel dimension to obtain the concatenated image sequence; The stitched image sequence is processed by the self-attention module of the parameter-adjusted feature fusion network to obtain the self-attention processed image sequence. The self-attention processed image sequence is segmented to obtain the segmented search image sequence; The segmented search image sequence is input into the wavelet enhancement module, and two types of convolution are used to extract basic local features. Three parallel depthwise separable convolutions are used to extract three multi-scale spatial features from the basic local features. The three multi-scale spatial features are then concatenated to obtain the output result after a series of convolutions. Expand the number of channels in the output after a series of convolutions to obtain the channel-expanded features; The features after channel expansion are averaged along the channel and then input into the simple gating of the wavelet enhancement module. The simple gating mechanism is used to process the features, and the features processed by the simple gating mechanism are then residually fused with the features of the search image sequence to obtain the residually fused sequence features. The sequence features after residual fusion are processed using a normalization layer to obtain the features processed by the normalization layer. The features processed by the normalization layer are decomposed into four frequency bands in the wavelet domain by wavelet transform operation. The four frequency bands generated in the wavelet domain are spliced ​​and fused, and the structure is enhanced by convolution operation to obtain the enhanced structure result. The result after structural enhancement is processed using a nonlinear activation function to obtain the result after activation function processing; the sequence features after residual fusion are convolved to obtain the features after the second convolution; the result after activation function processing and the features after the second convolution are residual fused to obtain the search image sequence features enhanced by the wavelet enhancement module.

5. The target tracking method based on dual-domain attention and wavelet enhancement module according to claim 4, characterized in that, The template image sequence features and the search image sequence features are concatenated along the channel dimension to obtain the concatenated image sequence. The corresponding relationship in this process is as follows: ； in, This represents the stitched image sequence. This indicates a splicing operation. Represents a template image sequence. Indicates a search for a sequence of images; In the step of performing self-attention processing on the stitched image sequence through the self-attention module of the feature fusion network, the corresponding relationship in the process is as follows: ； in, This represents the output of self-attention during the feature fusion stage. This represents the query matrix derived from the search image sequence. This represents the transpose of the key matrix from the template image sequence. This represents the transpose of the key matrix from the search image sequence. This represents a value matrix derived from a template image sequence. This represents a value matrix derived from the search image sequence; In the step of segmenting the self-attention processed image sequence to obtain the segmented search image sequence, the corresponding relationship in the process is as follows: ； in, This represents the segmented search image sequence. This indicates a splitting operation. This represents the image sequence after self-attention processing.

6. The target tracking method based on dual-domain attention and wavelet enhancement module according to claim 5, characterized in that, The segmented search image sequence is input into the wavelet enhancement module, where two types of convolution are used to extract basic local features. Three parallel depthwise separable convolutions are then used to extract three multi-scale spatial features from the basic local features. These three multi-scale spatial features are then concatenated to obtain the output result after a series of convolutions. The relationships between these processes are as follows: ； in, This represents the features extracted through 1x1 convolution and 3x3 convolution. This represents a 3x3 convolution operation. This represents the output of a depthwise separable convolution with a kernel of 1. This indicates a depthwise separable convolution with a kernel of 1. This represents the output of a depthwise separable convolution with a kernel of 3. This indicates a depthwise separable convolution with a kernel of 3. This represents the output of a depthwise separable convolution with a kernel of 5. This indicates a depthwise separable convolution with a kernel size of 5. This represents the output result after a series of convolutions; The steps of expanding the number of channels in the output after a series of convolutions to obtain expanded features; dividing the expanded features along the channels by average and inputting them into a simple gating mechanism of the wavelet enhancement module for simple gating processing to obtain features processed by the simple gating mechanism; and performing residual fusion between the features processed by the simple gating mechanism and the search image sequence features to obtain the residual fused sequence features, have the following relationships: ； in, This represents the first feature after segmentation along the channel dimension. This represents the second feature after segmentation along the channel dimension. This represents the characteristics of a simple gated output. This represents the sequence features after residual fusion.

7. The target tracking method based on dual-domain attention and wavelet enhancement module according to claim 6, characterized in that, The sequence features after residual fusion are processed using a normalization layer to obtain the features processed by the normalization layer. The features processed by the normalization layer are decomposed into four frequency bands in the wavelet domain by wavelet transform operation. The four frequency bands generated in the wavelet domain are spliced ​​and fused, and the structure is enhanced by convolution operation to obtain the enhanced structure result. The result after structural enhancement is processed using a nonlinear activation function to obtain the result after activation function processing. The sequence features after residual fusion are then convolved to obtain the features after a second convolution. The result after activation function processing and the features after the second convolution are then fused using residuals to obtain the search image sequence features enhanced by the wavelet enhancement module. The corresponding relationship in this process is as follows: ； in, , , and These represent the four frequency bands generated in the wavelet domain. This indicates the wavelet transform operation. express, This represents the features of the search image sequence enhanced by the wavelet enhancement module. This represents the activation function.

8. A target tracking system based on dual-domain attention and wavelet enhancement modules, characterized in that, The system employs the target tracking method based on dual-domain attention and wavelet enhancement module as described in any one of claims 1-7, and the system comprises: Build modules are used for: The tracking model is composed of a feature extraction network, a feature fusion network, and a prediction head. Pre-trained modules are used for: The feature extraction network and the feature fusion network are pre-trained using a large-scale dataset, and the parameters are adjusted to obtain the parameter-adjusted feature extraction network and the parameter-adjusted feature fusion network. Extraction module, used for: The template image and the search image are preprocessed and input into the parameter adjustment feature extraction network. The dual-domain attention of the feature extraction network is adjusted by the parameter adjustment to extract the features of the template image and the search image in the spatial domain and the channel domain respectively, so as to obtain the template image sequence features and the search image sequence features. The learning module is used for: The template image sequence features and the search image sequence features are concatenated along the channel dimension. After concatenation, the self-attention module and wavelet enhancement module of the feature fusion network are adjusted by parameters to perform interactive fusion and feature enhancement to obtain the enhanced search image sequence features. The calculation module is used for: The enhanced search image sequence features are replaced with the search image sequence features, and then concatenated with the template image sequence features again. The extraction module is repeatedly extracted a preset number of times in an iterative manner to obtain the fused features. The tracking module is used for: The fused features are fed into the prediction head to obtain the tracking results. The classification loss and regression loss are calculated based on the tracking results, and the tracking model is optimized to obtain the optimized tracking model. Target tracking is performed using the optimized tracking model.

Images