Target tracking method and system based on double-domain attention and wavelet enhancement module

Abstract

The invention provides a target tracking method and system based on a double-domain attention and wavelet enhancement module, and the method comprises the steps: carrying out the preprocessing of a template image and a search image, and inputting a parameter adjustment feature extraction network, respectively extracting features of the template image and the search image in a spatial domain and a channel domain through double-domain attention of a feature extraction network; splicing the template image sequence features and the search image sequence features along channel dimensions, adjusting a self-attention module and a wavelet enhancement module of a feature fusion network by using parameters after splicing, and performing interactive fusion and feature enhancement; and sending the fusion feature into a prediction head to obtain a tracking result. According to the method, the spatial domain features and the channel domain features of the input image sequence are extracted through the double-domain attention, so that the model not only can pay attention to the spatial position of a target, but also can model the importance of channel semantics, double-domain complementary modeling is realized, a target region is positioned more accurately, and target features and noise channels are effectively distinguished.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer vision and image processing, in particular to a target tracking method and system based on dual-domain attention and wavelet enhancement module. BACKGROUND

[0002] As an important branch of computer vision, the core idea of visual object tracking is to give the position of the target in the initial frame of the video sequence, and accurately predict the position of the target in each subsequent frame. It is widely used in human-computer interaction, automatic driving and video monitoring and other practical fields. In recent years, with the rapid development of Transformer, visual object tracking has reached a new height, and its application in daily life is more widespread, and its research value is more prominent. However, the current visual object tracking still faces a series of severe challenges. When the target and the background are similar, the attention is easy to spread to the non-target area, and when the target appearance changes greatly, the tracking drift is easy to occur. The existence of these interference factors makes the tracker unable to accurately track the target in the subsequent frames.

[0003] In recent years, researchers have tried various methods to improve the accuracy of the tracker. The current mainstream methods CNN, Transformer and Mamba all extract features from spatial positions when used alone, which can easily cause the loss of channel semantics, resulting in blurred edge details and easy to focus on background information in complex texture scenes. In addition, motion blur can make the extracted edge features not obvious enough, while the low frequency in the wavelet domain is rarely affected by blur, and the high frequency can still extract the direction information in the blur. However, the filter of the traditional wavelet is fixed and cannot be learned, which cannot cope with complex scene changes and also causes the loss of spatial information. SUMMARY

[0004] In view of the above situation, the main purpose of the present application is to propose a target tracking method and system based on dual-domain attention and wavelet enhancement module to solve the above technical problems.

[0005] The present application proposes a target tracking method based on dual-domain attention and wavelet enhancement module, which comprises the following steps: Step 1, a tracking model is constructed based on a feature extraction network, a feature fusion network and a prediction head; Step 2, pre-train the feature extraction network and the feature fusion network using a large-scale dataset, and adjust the parameters to obtain a parameter-adjusted feature extraction network and a parameter-adjusted feature fusion network; Step 3, pre-process the template image and the search image, and input parameters into the feature extraction network, and extract the features of the template image and the search image in the spatial domain and the channel domain through the dual-domain attention of the parameter adjustment feature extraction network, to obtain the template image sequence feature and the search image sequence feature; Step 4, the template image sequence feature and the search image sequence feature are spliced along the channel dimension, and after splicing, the self-attention module and the wavelet enhancement module of the parameter adjustment feature fusion network are used for interactive fusion and feature enhancement, to obtain the enhanced search image sequence feature; Step 5, the enhanced search image sequence feature is replaced with the search image sequence feature, and is spliced with the template image sequence feature again, and step 4 is repeated in an iterative manner for a preset number of times to obtain the fusion feature; Step 6, the fusion feature is sent into the prediction head to obtain the tracking result, the classification loss and the regression loss are calculated according to the tracking result, and the tracking model is optimized to obtain the optimized tracking model; The optimized tracking model is used for target tracking.

[0006] The application also provides a target tracking system based on the dual-domain attention and the wavelet enhancement module, and the system comprises: The construction module is used for: The tracking model is composed of the feature extraction network, the feature fusion network and the prediction head; The pre-training module is used for: The feature extraction network and the feature fusion network are pre-trained by using a large-scale data set, and the parameters are adjusted to obtain the parameter adjustment feature extraction network and the parameter adjustment feature fusion network; The extraction module is used for: The template image and the search image are pre-processed, and are input into the parameter adjustment feature extraction network, and the features of the template image and the search image in the spatial domain and the channel domain are extracted through the dual-domain attention of the parameter adjustment feature extraction network, to obtain the template image sequence feature and the search image sequence feature; The learning module is used for: The template image sequence feature and the search image sequence feature are spliced along the channel dimension, and after splicing, the self-attention module and the wavelet enhancement module of the parameter adjustment feature fusion network are used for interactive fusion and feature enhancement, to obtain the enhanced search image sequence feature; The calculation module is used for: The enhanced search image sequence feature is replaced with the search image sequence feature, and is spliced with the template image sequence feature again, and the extraction module is repeated in an iterative manner for a preset number of times to obtain the fusion feature; The tracking module is used for: The fused features are fed into the prediction head to obtain the tracking results. The classification loss and regression loss are calculated based on the tracking results, and the tracking model is optimized to obtain the optimized tracking model. Target tracking is performed using the optimized tracking model.

[0007] Compared with the prior art, the beneficial effects of the present invention are as follows: 1. This invention extracts spatial and channel features of the input image sequence through dual-domain attention, enabling the model to focus on both the spatial location of the target and the importance of channel semantics, achieving complementary dual-domain modeling, more accurately locating the target region, and effectively distinguishing target features from noise channels.

[0008] 2. By inventing the channel domain branch in the dual-domain attention, this invention reconstructs the channels of the input image sequence by grouping them, models the channel dependencies between different sub-sequences, reduces redundant computation between channels, and improves the utilization rate of channel features.

[0009] 3. The present invention provides rich scale features for the model by using a series of convolution operations before wavelet transform through the wavelet enhancement module described in the invention, suppresses noise through a simple gating mechanism, enhances nonlinear representation, and finally decomposes the target direction structure into different frequency bands through wavelet transform, providing a more structured feature expression.

[0010] Additional aspects and advantages of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by means of embodiments of the invention. Attached Figure Description

[0011] Figure 1 This is a flowchart illustrating the steps of the target tracking method based on dual-domain attention and wavelet enhancement module proposed in this invention.

[0012] Figure 2 This is a structural diagram of the target tracking framework proposed in this invention, which is based on dual-domain attention and wavelet enhancement modules.

[0013] Figure 3 This is a schematic diagram of the dual-domain attention principle of the target tracking method based on dual-domain attention and wavelet enhancement module proposed in this invention.

[0014] Figure 4 This is a schematic diagram of the wavelet enhancement module in the target tracking method based on dual-domain attention and wavelet enhancement module proposed in this invention.

[0015] Figure 5 This is a schematic diagram of the target tracking system based on dual-domain attention and wavelet enhancement module proposed in this invention. Detailed Implementation

[0016] Embodiments of the present invention are described in detail below. Examples of these embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and should not be construed as limiting the present invention.

[0017] These and other aspects of the embodiments of the present invention will become clear from the following description and accompanying drawings. In these descriptions and drawings, some specific embodiments of the present invention are specifically disclosed to provide some ways of implementing the principles of the embodiments of the present invention; however, it should be understood that the scope of the embodiments of the present invention is not limited thereto.

[0018] Please see Figure 1 This invention proposes a target tracking method based on dual-domain attention and wavelet enhancement modules, which includes the following steps: Step 1: Construct a tracking model based on a feature extraction network, a feature fusion network, and a prediction head.

[0019] Step 2: Pre-train the feature extraction network and the feature fusion network using a large-scale dataset, and adjust the parameters to obtain the parameter-adjusted feature extraction network and the parameter-adjusted feature fusion network.

[0020] Step 3: Preprocess the template image and the search image, and input the parameters into the feature extraction network. By adjusting the dual-domain attention of the feature extraction network through parameter adjustment, the features of the template image and the search image in the spatial domain and the channel domain are extracted respectively to obtain the template image sequence features and the search image sequence features.

[0021] Please see Figure 2 and Figure 3 In step 3, the template image and the search image are preprocessed and input into the parameter adjustment feature extraction network. The dual-domain attention of the feature extraction network is adjusted by the parameter adjustment to extract the features of the template image and the search image in the spatial domain and the channel domain, respectively, so as to obtain the template image sequence features and the search image sequence features. Specifically, the steps are as follows: The template image and the search image are each divided into image blocks of the same size that do not overlap, and the image blocks of the same size that do not overlap are converted into a template image sequence and a search image sequence. The template image sequence and the search image sequence are input into the parameter adjustment feature extraction network. The self-attention of the dual-domain attention in the feature extraction network is adjusted by the parameter adjustment to perform self-attention processing on the template image sequence and the search image sequence in the spatial domain, so as to obtain the attention weight matrix. The attention weight matrix is ​​normalized and multiplied with the value matrix to obtain the output in the spatial domain. In the channel domain, spatial average pooling is performed on the template image sequence and the search image sequence to obtain features after spatial average pooling. The features after spatial average pooling are then grouped by channel to obtain the channel arrangement of the feature group. The channel arrangement of the feature group is then subjected to channel swapping to perform cross-channel information interaction and obtain the features after channel swapping. The features after channel swapping are subjected to global average pooling to obtain the global average pooled features. The global average pooled features are then subjected to convolution operations to learn the dependencies between channels and obtain feature weights. The feature weights are then weighted by an activation function to obtain the mapped channel weights. The mapped channel weights are then reassembled back into the original channel order to obtain the channel weights after activation function mapping. Finally, the channel weights after activation function mapping are weighted and fused with the template image sequence and the search image sequence through multiplication operations to obtain the channel domain output. The outputs of the spatial domain and the channel domain are added together and processed through a linear layer to unify the dimensions, resulting in the output of the dual-domain attention. The output of the dual-domain attention includes template image sequence features and search image sequence features.

[0022] The template image sequence and the search image sequence are input into the feature extraction network. The self-attention mechanism of the dual-domain attention within the feature extraction network is adjusted by these parameters. Self-attention processing is then performed on the template image sequence and the search image sequence in the spatial domain to obtain the attention weight matrix. This attention weight matrix is ​​then normalized and multiplied by the value matrix to obtain the spatial domain output. The corresponding relationship in this process is as follows: ; in, Represents the attention weight matrix. Represents the query matrix. Represents the transpose of the key matrix , This represents the dimension of the key matrix. Represents the output of the spatial domain. This indicates normalization processing. Represents a value matrix; The steps of performing spatial average pooling on the template image sequence and the search image sequence to obtain spatially average pooled features; grouping the spatially average pooled features by channel to obtain a channel arrangement of the feature groupings; and exchanging the channel arrangement of the feature groupings to perform cross-channel information exchange to obtain the features after channel exchange, are related as follows: ; in, This represents the features after spatial average pooling. This indicates spatial average pooling. This represents the sequence of the input image after preprocessing. This indicates the channel arrangement after feature grouping. This represents the first group of channels in the first subsequence after grouping all channels in the sequence. This represents the first subsequence after grouping all channels in the sequence. Group channels, This represents the last group of channels in the first subsequence after all channels in the sequence have been grouped. This indicates the sequence number after grouping all channels. The first group of channels in the subsequences This indicates the sequence number after grouping all channels. The first subsequence Group channels, This indicates the sequence number after grouping all channels. The last group of channels in each subsequence This represents the first group of channels in the last subsequence after all channels in the sequence have been grouped. This represents the last subsequence after grouping all channels in the sequence. Group channels, This represents the last group of channels in the last subsequence after all channels in the sequence have been grouped. Indicates the characteristics after the switching channel, This indicates channel exchange processing between subsequences.

[0023] It should be noted that, This represents the quantity in each subsequence, i.e. This represents the last group of channels in the first subsequence after all channels in the sequence have been grouped. Then it represents the number of subsequences, i.e. This represents the first group of channels in the last subsequence after all channels in the sequence have been grouped.

[0024] The features after channel swapping are subjected to global average pooling to obtain globally average pooled features. These globally average pooled features are then processed by convolution to learn the channel dependencies and obtain feature weights. The feature weights are then weighted using an activation function to obtain mapped channel weights. The mapped channel weights are then reassembled back into their original channel order to obtain channel weights mapped by the activation function. Finally, these channel weights are weighted and fused with the template image sequence and the search image sequence through multiplication to obtain the channel domain output. The relationships in this process are as follows: ; in, This represents the channel weights after being mapped by the activation function. This means mapping the channel weights back to the original channel order. This represents the activation function. express Convolution operation, This indicates global average pooling. Indicates the output of the channel domain. Indicates the multiplication operation; The outputs of the spatial domain and the channel domain are added together and processed through a linear layer to unify the dimensions, resulting in the output of the dual-domain attention. The relationship between the steps in which the output of the dual-domain attention includes template image sequence features and search image sequence features is as follows: ; in, This represents the output of dual-domain attention. This indicates that the data has been processed through a linear layer.

[0025] Furthermore, by employing dual-domain attention, the global dependency between any two subsequences is calculated through self-attention, directly capturing long-distance relationships in the spatial domain. This concentrates attention on the target region and suppresses irrelevant locations. In the channel domain, the channels of the input image subsequence are first divided into several equal parts, and the channels from different subsequences are concatenated. Channel dependencies and weights between subsequences are calculated through global average pooling, convolution operations, and the Sigmoid activation function. Finally, the obtained channel weights are reassembled back into the original channel order and fused with the features of the original input image sequence to enhance the feature representation capabilities of different channels. The outputs from the spatial domain and the channel domain are added together, and the dimension is unified through a linear layer to obtain the output of dual-domain attention.

[0026] Step 4: Concatenate the template image sequence features and the search image sequence features along the channel dimension. After concatenation, use the parameter adjustment of the self-attention module and wavelet enhancement module of the feature fusion network to perform interactive fusion and feature enhancement to obtain the enhanced search image sequence features.

[0027] Please see Figure 4 In step 4, the template image sequence features and the search image sequence features are concatenated along the channel dimension. After concatenation, the self-attention module and wavelet enhancement module of the feature fusion network are adjusted using parameters to perform interactive fusion and feature enhancement to obtain the enhanced search image sequence features. Specifically, the steps are as follows: The template image sequence features and the search image sequence features are concatenated along the channel dimension to obtain the concatenated image sequence; The stitched image sequence is processed by the self-attention module of the feature fusion network to obtain the self-attention processed image sequence. The self-attention processed image sequence is segmented to obtain the segmented search image sequence; The segmented search image sequence is input into the wavelet enhancement module, and two types of convolution are used to extract basic local features. Three parallel depthwise separable convolutions are used to extract three multi-scale spatial features from the basic local features. The three multi-scale spatial features are then concatenated to obtain the output result after a series of convolutions. The number of channels in the output after a series of convolutions is expanded to obtain the channel-expanded features. The channel-expanded features are then averaged along the channels and input into the simple gating of the wavelet enhancement module for simple gating mechanism processing to obtain the features processed by the simple gating mechanism. The features processed by the simple gating mechanism are then residually fused with the search image sequence features to obtain the residually fused sequence features. The residual-fused sequence features are processed using a normalization layer to obtain normalized features. The normalized features are then decomposed into four frequency bands in the wavelet domain using wavelet transform. These four frequency bands are then concatenated and fused, and convolution is used to further enhance the structure, resulting in a structure-enhanced product. This structure-enhanced product is then processed using a nonlinear activation function to obtain a result after activation function processing. The residual-fused sequence features are then convolved to obtain a second convolution-processed feature. Finally, the result after activation function processing and the second convolution-processed feature are subjected to residual fusion to obtain the enhanced search image sequence features from the wavelet enhancement module.

[0028] The template image sequence features and the search image sequence features are concatenated along the channel dimension to obtain the concatenated image sequence. The corresponding relationship in this process is as follows: ; in, This represents the stitched image sequence. This indicates a splicing operation. Represents a template image sequence. Indicates a search for a sequence of images; In the step of performing self-attention processing on the stitched image sequence through the self-attention module of the feature fusion network, the corresponding relationship in the process is as follows: ; in, This represents the output of self-attention during the feature fusion stage. This represents the query matrix derived from the search image sequence. Represents the transpose of the key matrix from the template image sequence , Represents the transpose of the key matrix from the search image sequence , This represents a value matrix derived from a template image sequence. This represents a value matrix derived from the search image sequence; In the step of segmenting the self-attention processed image sequence to obtain the segmented search image sequence, the corresponding relationship in the process is as follows: ; in, This represents the segmented search image sequence. This indicates a splitting operation. This represents the image sequence after self-attention processing.

[0029] The segmented search image sequence is input into the wavelet enhancement module, where two types of convolution are used to extract basic local features. Three parallel depthwise separable convolutions are then used to extract three multi-scale spatial features from the basic local features. These three multi-scale spatial features are then concatenated to obtain the output result after a series of convolutions. The relationships between these processes are as follows: ; in, This represents the features extracted through 1x1 convolution and 3x3 convolution. This represents a 3x3 convolution operation. This represents the output of a depthwise separable convolution with a kernel of 1. This indicates a depthwise separable convolution with a kernel of 1. This represents the output of a depthwise separable convolution with a kernel of 3. This indicates a depthwise separable convolution with a kernel of 3. This represents the output of a depthwise separable convolution with a kernel of 5. This indicates a depthwise separable convolution with a kernel size of 5. This represents the output result after a series of convolutions; The steps of expanding the number of channels in the output after a series of convolutions to obtain expanded features; dividing the expanded features along the channels by average and inputting them into a simple gating mechanism of the wavelet enhancement module for simple gating processing to obtain features processed by the simple gating mechanism; and performing residual fusion between the features processed by the simple gating mechanism and the search image sequence features to obtain the residual fused sequence features, have the following relationships: ; in, This represents the first feature after segmentation along the channel dimension. This represents the second feature after segmentation along the channel dimension. This represents the characteristics of a simple gated output. This represents the sequence features after residual fusion.

[0030] The residual-fused sequence features are processed using a normalization layer to obtain normalized features. The normalized features are then decomposed into four frequency bands in the wavelet domain using wavelet transform. These four frequency bands are then concatenated and fused, and convolution is used to further enhance the structure, resulting in an enhanced structure. This enhanced structure is then processed using a nonlinear activation function, resulting in an activated structure. The residual-fused sequence features are then convolved to obtain a second convolutional feature. Finally, the activated structure feature and the second convolutional feature are subjected to residual fusion to obtain the enhanced search image sequence features from the wavelet enhancement module. The relationships between these processes are as follows: ; in, , , and These represent the four frequency bands generated in the wavelet domain. This represents the wavelet transform operation. express, This represents the features of the search image sequence enhanced by the wavelet enhancement module. This represents the activation function.

[0031] Furthermore, the template image sequence and the search image sequence are concatenated along the channel dimension. In the feature fusion network, the concatenated image sequence is first interactively fused using self-attention. Then, the search image sequence is segmented and sent to the wavelet enhancement module for processing. In this module, a series of convolutions and simple gating operations are first used to capture spatial feature information. Then, wavelet transform is used to decompose the target orientation structure into different frequency bands, and the decomposed frequency bands are used to enhance the target orientation texture and structure. The search image sequence processed by the wavelet enhancement module replaces the initial search image sequence sent to the feature fusion stage. The new search image sequence is then concatenated with the original template image sequence sent to the feature fusion stage. The concatenated image sequence is then sent back to the feature fusion stage to perform the same operation for further interactive fusion. The number of iterations is consistent with the number of layers in the feature fusion network, which is set to 3 in this system.

[0032] Step 5: Replace the search image sequence features with the enhanced search image sequence features, and then concatenate them with the template image sequence features again. Repeat step 4 a preset number of times in an iterative manner to obtain the fused features.

[0033] It should be noted that, utilizing the trained feature extraction network model and feature fusion network model, feature extraction in the spatial and channel domains of the input image sequence is performed through dual-domain attention during the feature extraction stage. Self-attention captures long-distance relationships in the spatial domain, calculating the dependency between any two sub-sequences. In the channel domain, the channels of the sub-sequences are divided into several equal parts, and the channels of different sub-sequences are concatenated. Global average pooling, convolution operations, and the sigmoid activation function are used to calculate the channel dependencies and weights between the sub-sequences. These weights are then reassembled back to the original channel order and fused with the features of the original input image sequence to enhance the feature representation capabilities of different channels. The outputs from the spatial and channel domains are summed and then passed through a linear layer to unify the dimensions, obtaining the output of the dual-domain attention. The template image sequence and the search image sequence after the feature extraction stage are concatenated along the channel dimension as input for the feature fusion stage. In the feature fusion stage, the concatenated image sequence first undergoes interactive fusion through self-attention, then the search image sequence is segmented. The segmented sequence uses a wavelet enhancement module to decompose the target orientation structure into different frequency bands, and the decomposed frequency bands are used to enhance the target orientation texture and structure. The wavelet-enhanced search image sequence replaces the initial search image sequence sent to the feature fusion stage. The new search image sequence is then concatenated with the original template image sequence sent to the feature fusion stage. The concatenated image sequence is then sent back to the feature fusion stage to perform the same operation for further interactive fusion. The number of iterations is consistent with the number of feature fusion network layers, which is set to 3 in this system.

[0034] Step 6: Feed the fused features into the prediction head to obtain the tracking results, calculate the classification loss and regression loss based on the tracking results, and optimize the tracking model to obtain the optimized tracking model; Target tracking is performed using the optimized tracking model.

[0035] The prediction head consists of three convolutional branches: center classification, offset regression, and size regression. The center classification branch outputs a center score map, where each score represents the confidence level of the target center at the corresponding location. The offset regression branch calculates the dispersion error of the center, and the size regression branch predicts the height and width of the target. The location with the highest confidence in the center score map is taken as the target location. The target bounding box is predicted using the corresponding regression coordinates. Classification is performed using cross-entropy loss, and the loss is used to calculate the bounding box regression. The total loss function can be expressed as: ; in, Indicates the total loss. Represents cross-entropy loss, and These represent the regularization parameters, This represents the regression loss based on the generalized intersection-union loss. This represents the regression loss based on L1 loss.

[0036] This invention also proposes a target tracking system based on dual-domain attention and wavelet enhancement modules, the system comprising: Build modules are used for: The tracking model is composed of a feature extraction network, a feature fusion network, and a prediction head. Pre-trained modules are used for: The feature extraction network and the feature fusion network are pre-trained using a large-scale dataset, and the parameters are adjusted to obtain the parameter-adjusted feature extraction network and the parameter-adjusted feature fusion network. Extraction module, used for: The template image and the search image are preprocessed and input into the parameter adjustment feature extraction network. The dual-domain attention of the feature extraction network is adjusted by the parameter adjustment to extract the features of the template image and the search image in the spatial domain and the channel domain respectively, so as to obtain the template image sequence features and the search image sequence features. The learning module is used for: The template image sequence features and the search image sequence features are concatenated along the channel dimension. After concatenation, the self-attention module and wavelet enhancement module of the feature fusion network are adjusted by parameters to perform interactive fusion and feature enhancement to obtain the enhanced search image sequence features. The calculation module is used for: The enhanced search image sequence features are replaced with the search image sequence features, and then concatenated with the template image sequence features again. The extraction module is repeatedly extracted a preset number of times in an iterative manner to obtain the fused features. The tracking module is used for: The fused features are fed into the prediction head to obtain the tracking results. The classification loss and regression loss are calculated based on the tracking results, and the tracking model is optimized to obtain the optimized tracking model. Target tracking is performed using the optimized tracking model.

[0037] It should be understood that various parts of the present invention can be implemented in hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented in software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0038] In the description of this specification, references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0039] The embodiments described above are merely illustrative of several implementations of the present invention, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of the present invention. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these modifications and improvements all fall within the scope of protection of the present invention. Therefore, the scope of protection of this patent should be determined by the appended claims.

Claims

1. A target tracking method based on dual-domain attention and wavelet enhancement module, characterized in that, The method comprises the following steps: Step 1, a tracking model is constituted based on a feature extraction network, a feature fusion network and a prediction head; Step 2, the feature extraction network and the feature fusion network are pre-trained using a large-scale data set, and parameters are adjusted to obtain a parameter-adjusted feature extraction network and a parameter-adjusted feature fusion network; Step 3, the template image and the search image are preprocessed and input into the parameter-adjusted feature extraction network, and the spatial domain and channel domain features of the template image and the search image are extracted through the dual-domain attention of the parameter-adjusted feature extraction network to obtain template image sequence features and search image sequence features; Step 4, the template image sequence features and the search image sequence features are spliced along the channel dimension, and after splicing, the self-attention module and the wavelet enhancement module of the parameter-adjusted feature fusion network are used for interactive fusion and feature enhancement to obtain enhanced search image sequence features; Step 5, the enhanced search image sequence features are replaced with the search image sequence features, and the template image sequence features are spliced again, and step 4 is repeated in an iterative manner for a predetermined number of times to obtain fusion features; Step 6, the fusion features are input into the prediction head to obtain tracking results, classification loss and regression loss are calculated according to the tracking results, and the tracking model is optimized to obtain an optimized tracking model; Target tracking is performed using the optimized tracking model.

2. The target tracking method based on dual domain attention and wavelet enhancement module according to claim 1, characterized in that, In the step 3, the template image and the search image are preprocessed and input into the parameter-adjusted feature extraction network, and the spatial domain and channel domain features of the template image and the search image are extracted through the dual-domain attention of the parameter-adjusted feature extraction network to obtain template image sequence features and search image sequence features, which specifically comprises the following steps: The template image and the search image are divided into image blocks of the same size and non-overlapping, and the image blocks of the same size and non-overlapping are converted into template image sequences and search image sequences; The template image sequences and the search image sequences are input into the parameter-adjusted feature extraction network, and the self-attention of the dual-domain attention in the parameter-adjusted feature extraction network is used to perform self-attention processing on the template image sequences and the search image sequences in the spatial domain to obtain an attention weight matrix, the attention weight matrix is normalized and multiplied by a value matrix to obtain the output in the spatial domain; In the channel domain, spatial average pooling operation is performed on the template image sequences and the search image sequences to obtain features after spatial average pooling processing; the features after spatial average pooling processing are grouped in channels to obtain channel arrangement after feature grouping, and the channel arrangement after feature grouping is processed by channel exchange to interact across channels to obtain features after channel exchange; The features after channel exchange are processed by global average pooling to obtain features after global average pooling processing; The features after global average pooling processing are processed by convolution operation to learn the dependence between channels to obtain feature weights; The feature weights are mapped by an activation function to obtain mapped channel weights; The mapped channel weight is reordered to the original channel order to obtain the channel weight mapped by the activation function, and the channel weight mapped by the activation function is weighted and fused with the template image sequence and the search image sequence through a multiplication operation to obtain the output in the channel domain. The output in the spatial domain and the output in the channel domain are added and calculated, and are processed by a linear layer to unify the dimensions to obtain the output of the dual-domain attention; wherein the output of the dual-domain attention includes the template image sequence feature and the search image sequence feature.

3. The target tracking method based on dual domain attention and wavelet enhancement module according to claim 2, characterized in that, The template image sequence and the search image sequence are input into the parameter adjustment feature extraction network, and the self-attention of the dual-domain attention in the parameter adjustment feature extraction network is used to perform self-attention processing on the template image sequence and the search image sequence in the spatial domain to obtain an attention weight matrix, the attention weight matrix is normalized and multiplied with a value matrix to obtain the output in the spatial domain, and the relationship formula existing in the corresponding process is as follows: ； wherein, denotes an attention weight matrix, denotes a query matrix, denotes a transpose of a key matrix , denotes a dimension of a key matrix, denotes an output of a spatial domain, denotes a normalization process, denotes a value matrix; The template image sequence and the search image sequence are subjected to spatial average pooling to obtain features subjected to spatial average pooling; the features subjected to spatial average pooling are grouped by channels to obtain channel arrangement after feature grouping, and the channel arrangement after feature grouping is subjected to channel exchange processing for cross-channel information interaction to obtain features after channel exchange; in the step of processing the features after channel exchange by global average pooling to obtain features after global average pooling, the relationship formula existing in the corresponding process is as follows: ； wherein, represents the feature after spatial average pooling processing, represents spatial average pooling processing, represents the sequence after pre-processing of the input image, represents the channel arrangement after grouping of the feature, represents the first group of channels of the first sub-sequence after grouping of all channels in the sequence, represents the first group of channels of the first sub-sequence after grouping of all channels in the sequence, represents the last group of channels of the first sub-sequence after grouping of all channels in the sequence, represents the first group of channels of the first sub-sequence after grouping of all channels in the sequence, represents the first group of channels of the first sub-sequence after grouping of all channels in the sequence, represents the first group of channels of the first sub-sequence after grouping of all channels in the sequence, represents the first group of channels of the first sub-sequence after grouping of all channels in the sequence, represents the first group of channels of the first sub-sequence after grouping of all channels in the sequence, represents the first group of channels of the first sub-sequence after grouping of all channels in the sequence, represents the first group of channels of the first sub-sequence after grouping of all channels in the sequence, represents the first group of channels of the first sub-sequence after grouping of all channels in the sequence, represents the first group of channels of the first sub-sequence after grouping of all channels in the sequence, represents the first group of channels of the first sub-sequence after grouping of all channels in the sequence, represents the first group of channels of the first sub-sequence after grouping of all channels in the sequence, represents the first group of channels of the first sub-sequence after grouping of all channels in the sequence, represents the feature after channel exchange, represents the channel exchange processing between sub-sequences.

4. The target tracking method based on dual-domain attention and wavelet enhancement module according to claim 3, characterized in that, The features after global average pooling are processed by global average pooling to obtain features after global average pooling; The features after global average pooling are processed by convolution to learn the dependence between channels to obtain feature weights; The feature weights are mapped by an activation function to obtain mapped channel weights; The mapped channel weight is reordered to the original channel order to obtain the channel weight mapped by the activation function, and the channel weight mapped by the activation function is weighted and fused with the template image sequence and the search image sequence through a multiplication operation to obtain the output in the channel domain, and the relationship formula existing in the corresponding process is as follows: ； wherein, represents the channel weights mapped through an activation function, represents mapping the channel weights back to the original channel order, represents an activation function, represents a convolution operation, represents a global average pooling process, represents the output in the channel domain, represents a multiplication operation; In the step of adding and calculating the output in the spatial domain and the output in the channel domain, and processing by a linear layer to unify the dimensions to obtain the output of the dual-domain attention; wherein the output of the dual-domain attention includes the template image sequence feature and the search image sequence feature, the relationship formula existing in the corresponding process is as follows: ； wherein, represents the output of the dual-attention, represents the processing by the linear layer.

5. The target tracking method based on dual domain attention and wavelet enhancement module according to claim 4, characterized in that, In the step 4, the template image sequence feature and the search image sequence feature are spliced along the channel dimension, and the self-attention module and the wavelet enhancement module of the parameter adjustment feature fusion network are used for interactive fusion and feature enhancement to obtain the enhanced search image sequence feature, which specifically includes the following steps: The template image sequence feature and the search image sequence feature are spliced along the channel dimension to obtain a spliced image sequence; The spliced image sequence is subjected to self-attention processing by the self-attention module of the parameter adjustment feature fusion network to obtain an image sequence after self-attention processing; The image sequence after self-attention processing is segmented to obtain a segmented search image sequence; The segmented search image sequence is input into a wavelet enhancement module, basic local features are extracted using two kinds of convolution; three kinds of multi-scale spatial features are respectively extracted using three parallel depth separable convolutions on the basic local features; the three kinds of multi-scale spatial features are spliced to obtain an output result after a series of convolutions; The channel number of the output result after a series of convolutions is expanded to obtain a channel-expanded feature; The channel-expanded feature is input into a simple gating mechanism of the wavelet enhancement module after being evenly divided along the channel, and is subjected to simple gating mechanism processing to obtain a feature after simple gating mechanism processing, and the feature after simple gating mechanism processing is subjected to residual fusion with the search image sequence feature to obtain a sequence feature after residual fusion; The sequence feature after residual fusion is processed using a normalization layer to obtain a feature after normalization layer processing; The feature target direction structure is decomposed using a wavelet transform operation to obtain four frequency bands generated by a wavelet domain, the four frequency bands generated by the wavelet domain are spliced and fused, and a convolution operation is used to further enhance the structure to obtain a result after structure enhancement; The result after structure enhancement is processed using a nonlinear activation function to obtain a result after activation function processing; the sequence feature after residual fusion is subjected to convolution processing to obtain a second convolution-processed feature; the result after activation function processing and the second convolution-processed feature are subjected to residual fusion to obtain a search image sequence feature enhanced by the wavelet enhancement module.

6. The target tracking method based on dual domain attention and wavelet enhancement module according to claim 5, characterized in that, The template image sequence feature and the search image sequence feature are spliced along the channel dimension to obtain a spliced image sequence, and the relationship formula corresponding to the process is as follows: ； wherein, denotes the sequence of stitched images, denotes the stitching operation, denotes the sequence of template images, denotes the sequence of search images; In the step of performing self-attention processing on the spliced image sequence through the self-attention module of the feature fusion network, the relationship formula corresponding to the process is as follows: ； wherein, denotes the output of self-attention in the feature fusion stage, denotes the query matrix from the search image sequence, denotes the transpose of the key matrix from the template image sequence , denotes the transpose of the key matrix from the search image sequence , denotes the value matrix from the template image sequence, denotes the value matrix from the search image sequence; In the step of dividing the image sequence after self-attention processing to obtain the segmented search image sequence, the relationship formula corresponding to the process is as follows: ； wherein, denotes the segmented search image sequence, denotes the segmentation operation, denotes the image sequence processed by self-attention.

7. The target tracking method based on dual domain attention and wavelet enhancement module according to claim 6, characterized in that, The segmented search image sequence is input into a wavelet enhancement module, basic local features are extracted using two kinds of convolution; three kinds of multi-scale spatial features are respectively extracted using three parallel depth separable convolutions on the basic local features; the three kinds of multi-scale spatial features are spliced to obtain an output result after a series of convolutions, and the relationship formula corresponding to the process is as follows: ； wherein, represents the feature extracted by 1x1 convolution and 3x3 convolution, represents the 3x3 convolution operation, represents the output result after the depth separable convolution with the convolution kernel of 1, represents the depth separable convolution with the convolution kernel of 1, represents the output result after the depth separable convolution with the convolution kernel of 3, represents the depth separable convolution with the convolution kernel of 3, represents the output result after the depth separable convolution with the convolution kernel of 5, represents the depth separable convolution with the convolution kernel of 5, represents the output result after a series of convolutions; In the step of expanding the channel number of the output result after a series of convolutions to obtain a channel-expanded feature, and inputting the channel-expanded feature into a simple gating mechanism of the wavelet enhancement module after being evenly divided along the channel, and performing simple gating mechanism processing to obtain a feature after simple gating mechanism processing, and performing residual fusion on the feature after simple gating mechanism processing with the search image sequence feature to obtain a sequence feature after residual fusion, the relationship formula corresponding to the process is as follows: ； wherein, represents the first feature after splitting along the channel dimension, represents the second feature after splitting along the channel dimension, represents the feature of simple gating output, represents the sequence feature after residual fusion.

8. The target tracking method based on dual-domain attention and wavelet enhancement module according to claim 7, characterized in that, The sequence feature after residual fusion is processed using a normalization layer to obtain a feature after normalization layer processing; The normalized layer processed features are decomposed into four frequency bands generated by a wavelet domain using a wavelet transform operation, the four frequency bands generated by the wavelet domain are spliced and fused, and a convolution operation is used to further enhance the structure to obtain a result after structure enhancement; The result after structure enhancement is processed using a nonlinear activation function to obtain a result after activation function processing; the residual fused sequence features are convolved to obtain a second convolution processed feature; the result after activation function processing and the second convolution processed feature are residual fused to obtain a search image sequence feature enhanced by a wavelet enhancement module, and the relationship of the corresponding process is as follows: ； wherein, , , and represent four frequency bands generated in the wavelet domain, represents a wavelet transform operation, represents, represents the enhanced search image sequence features by the wavelet enhancement module, represents an activation function.

9. A target tracking system based on dual domain attention and wavelet enhancement module, characterized in that, The system adopts the target tracking method based on the dual-domain attention and wavelet enhancement module of any one of claims 1-8, and the system comprises: The construction module is configured to: construct a tracking model based on a feature extraction network, a feature fusion network, and a prediction head; The pre-training module is configured to: pre-train the feature extraction network and the feature fusion network using a large-scale data set, and adjust the parameters to obtain a parameter-adjusted feature extraction network and a parameter-adjusted feature fusion network; The extraction module is configured to: preprocess the template image and the search image, and input them into the parameter-adjusted feature extraction network to extract the features of the template image and the search image in the spatial domain and the channel domain through the dual-domain attention of the parameter-adjusted feature extraction network, so as to obtain a template image sequence feature and a search image sequence feature; The learning module is configured to: splice the template image sequence feature and the search image sequence feature along the channel dimension, and then use the self-attention module and the wavelet enhancement module of the parameter-adjusted feature fusion network to interact and fuse and enhance the features to obtain an enhanced search image sequence feature; The calculation module is configured to: replace the search image sequence feature with the enhanced search image sequence feature, and splice it with the template image sequence feature again, and repeat the extraction module for a predetermined number of times in an iterative manner to obtain a fused feature; The tracking module is configured to: feed the fused feature into the prediction head to obtain a tracking result, calculate a classification loss and a regression loss according to the tracking result, and optimize the tracking model to obtain an optimized tracking model; use the optimized tracking model for target tracking.

Images