Target tracking methods, electronic devices, storage media, and software products

Abstract

This disclosure provides a target tracking method, electronic device, storage medium, and program product, relating to the field of computer vision. The disclosure includes: tracking a target using a target tracking model based on a search region image, a static template image, and a dynamic template image, to obtain a target tracking result. The target tracking model performs the following steps: extracting features from the search region image, the static template image, and the dynamic template image respectively to obtain corresponding feature maps; splitting the static template feature map into multi-scale sub-feature maps, performing correlation operations, superposition, and interpolation processing with the search region feature map to obtain an input mask; using the input mask to filter the background and enhance target features to obtain a fused feature map; adding spatial location encoding information to the fused feature map, processing it through an encoder and decoder to obtain a decoded feature map; predicting the target position and size based on the decoded feature map to obtain the target tracking result.

Description

Technical Field

[0001] This disclosure relates to the field of computer vision technology, specifically to a target tracking method, electronic device, storage medium, and program product. Background Technology

[0002] Video object tracking (VOT) is a key research area in computer vision. Based on a given initial target position, VOT continuously identifies and locates the target in subsequent video sequences. VOT technology has been widely applied in important fields such as autonomous driving, drone applications, video surveillance, human-computer interaction, and augmented reality. However, the complexity of real-world environments and the dynamic changes of targets pose significant challenges to this task. Furthermore, camera shake or rapid movement can cause significant changes in the target's appearance and scale, further increasing the difficulty of tracking. In summary, video object tracking is affected by multiple factors, including external environmental factors and changes in the target itself, making robust VOT tracking a challenging task.

[0003] The mainstream tracking frameworks used in existing visual object tracking methods can be broadly divided into two categories: generative and discriminative tracking frameworks. In generative models, the tracking algorithm generates a large number of candidate samples based on the state of the target in the previous frame, and uses the sample most similar to the target template as the tracking result for the current frame. In discriminative models, a classifier is trained to distinguish between the target and the background, and the candidate sample with the highest response is used as the prediction result.

[0004] Since the minimum output sum of squares filter algorithm was first proposed, discriminant correlation filter target tracking algorithms have seen extensive development. This type of method utilizes the Fast Fourier Transform to transform the matrix multiplication operation between cyclically shifted samples and the filter in the spatial domain into a point-by-point multiplication operation in the frequency domain, effectively improving tracking speed.

[0005] While correlation filtering algorithms excel in real-time performance, their tracking performance suffers in complex scenarios involving target deformation, out-of-plane rotation, scale changes, and occlusion. With the development of deep learning technology, Siamese network frameworks, represented by Fully-Convolutional Siamese Networks (SiamFC), have attracted widespread attention due to their superior tracking performance. These trackers employ simple cross-correlation operations to achieve target feature similarity matching. However, limited by their finite receptive field and feature interaction capabilities, their tracking performance has gradually been surpassed by Transformer-based methods.

[0006] In recent years, thanks to the global attention modeling capabilities of the Transformer architecture, this type of method has gradually become the mainstream research framework in the field of target tracking. However, existing Transformer trackers still have considerable room for improvement in tracking accuracy and robustness when dealing with practical problems such as scale changes, complex background transformations, and target occlusion. Specifically, existing Transformer trackers have the following limitations: First, the single-scale, pixel-level attention mechanism of most Transformer trackers limits the tracker's multi-scale perception capability; second, the single-scale, pixel-level attention mechanism destroys the integrity of the positional and semantic information of the target appearance in the template, which is not conducive to the accurate representation of target features; third, single-scale samples without shift operations will cause the Transformer network to fall into the "data hunger" dilemma, that is, the tracking network needs to be trained with massive amounts of data to obtain saturated tracking performance; fourth, most Transformer networks use a single template as the only standard for tracking matching, resulting in the failure to perceive changes in target appearance information in a timely manner. Summary of the Invention

[0007] This disclosure provides a target tracking method, electronic device, storage medium, and program product.

[0008] According to one aspect of this disclosure, a target tracking method is provided, comprising: tracking a target using a target tracking model based on a search region image, a static template image, and a dynamic template image, to obtain a target tracking result, wherein the target tracking model performs the following steps: extracting features from the search region image, the static template image, and the dynamic template image respectively to obtain corresponding search region feature maps, static template feature maps, and dynamic template feature maps; splitting the static template feature map into multiple scales to obtain sub-feature maps of the static template feature map at different scales; performing correlation operations between each sub-feature map at each scale and the search region feature map to obtain block-related feature maps corresponding to each sub-feature map; and superimposing the block-related feature maps corresponding to each sub-feature map at each scale to obtain a block-related feature map for each scale. The process involves: generating scale-related feature maps; mapping scale-related feature maps at different scales to the same resolution using interpolation to obtain target-related feature maps for each scale; summing the target-related feature maps element-wise to obtain an input mask; filtering background redundancy information in the search region feature map based on the input mask and enhancing target features in the search region feature map similar to the static template feature map to obtain a fused feature map; adding spatial location encoding information to the fused feature map to obtain an input feature map; performing multi-scale feature extraction and fusion processing on the input feature map using an encoder to obtain an output feature map; performing feature decoding on the output feature map using a decoder to obtain a decoded feature map; and predicting the target position and size based on the decoded feature map to obtain the target tracking result.

[0009] According to at least one embodiment of the target tracking method of this disclosure, background redundancy information in the search region feature map is filtered based on the input mask, and target features similar to the static template feature map in the search region feature map are enhanced to obtain a fused feature map. The method includes: reshaping both the static template feature map and the dynamic template feature map into matrix form to obtain a target static template feature map and a target dynamic template feature map; performing background suppression on the input mask and multiplying it element-wise with the search region feature map to obtain an output mask; performing a dot product between the output mask and the search region feature map to obtain a first target feature map; reshaping the first target feature map into matrix form to obtain a second target feature map; and concatenating the target static template feature map, the target dynamic template feature map, and the second target feature map along a first dimension to obtain a fused feature map.

[0010] According to at least one embodiment of the target tracking method of this disclosure, the input feature map is subjected to multi-scale feature extraction and fusion processing by an encoder to obtain an output feature map, including: each encoder head, according to the size of the window, uses a frequency domain attention-based encoding method or a spatial domain attention-based encoding method to obtain a single-head attention matrix corresponding to each encoder head; the single-head attention matrices corresponding to each encoder head are concatenated along the dimension of the channel to obtain a second attention matrix; the second attention matrix is ​​then combined with the first... k After the input feature maps of the first layer are fused, they are normalized by the second layer to obtain intermediate feature maps. These intermediate feature maps are then input into a feedforward neural network for processing, followed by residual connections and layer normalization to obtain the first layer. k Layer output feature map, k Indicates the number of floors. , K Indicates the total number of layers in the encoder; and the number of layers in the encoder. K The layer output feature map is used as the output feature map of the encoder.

[0011] According to at least one embodiment of the target tracking method of this disclosure, the encoding head uses a frequency domain attention-based encoding method to obtain a corresponding single-head attention matrix, including: performing a linear transformation on each encoding head to obtain a query tensor corresponding to each encoding head. Key tensors Sum tensor ; For query tensors Key tensors Sum tensor Perform two-dimensional fast Fourier transforms to obtain the query transform tensor. Key Transformation Tensor Sum Transformation Tensor ; Transform tensor for query Expanding the query transform tensor yields the expanded query transform tensor. ; Calculate the key transformation tensor The conjugate tensor of the bond transformation is obtained. ; Expand tensor based on query transformation Bond Transformation Conjugate Tensor The first feature attention matrix is ​​obtained through the following formula. :

[0012] in, Represents the first feature attention matrix Bank of China code n All elements; Represents the column vectorization operator; Represents the operator for taking the real part; This represents the two-dimensional inverse fast Fourier transform. This indicates element-wise multiplication; Represents the query transformation extended tensor The fourth dimension index is c All elements; Represents the key transformation conjugate tensor The fourth dimension index is c All elements; c Indicates the index of the channel; Indicating the focus of bullish attention i The number of channels in each encoder header; Indicates the window size; the superscript T indicates the matrix transpose; logarithmic transformation tensor By expanding the dimensions, we obtain the value-dimensional tensor. , making ,in, Representation of value extended tensor The first dimension index is n The second dimension index is h The fifth dimension index is c All elements; Representation value transformation tensor The first dimension index is h The fourth dimension index is c All elements; N Indicates the total number of windows; n Indicates the window number; the first feature attention matrix Reconstructing it into tensor form yields the second feature attention-reconstructed tensor. Attention reshaping tensor for the second feature Perform a two-dimensional fast Fourier transform to obtain the second feature attention transform tensor. Attention Transform Tensor for Second Feature By expanding the dimension, we obtain the second feature attention dimension-expanded tensor. , making ; Represents the second feature attention dimension expansion tensor The first dimension index is n The second dimension index is h The fifth dimension index is c All elements; Represents the second feature attention transformation tensor The first dimension index is n The second dimension index is h All elements; expand the tensor based on values Second feature attention extended dimension tensor The attention score matrix corresponding to each encoder head is obtained by the following formula. :

[0013] in, Represents the attention score matrix Bank of China code All elements; Represents the second feature attention dimension expansion tensor The first dimension index is n The second dimension index is h All elements; Representation of value extended tensor The first dimension index is n The second dimension index is h All elements; h Indicates the index number; for the attention score matrix Reshape the matrix to obtain a single-head attention matrix. ,in, This represents the single-head attention matrix corresponding to the i-th encoder head.

[0014] According to at least one embodiment of the target tracking method of this disclosure, after obtaining the decoded feature map, the method further includes: inputting the decoded feature map into a multilayer perceptron, activating the output of the multilayer perceptron through an activation function to obtain a confidence score; and updating the dynamic template image if the confidence score is greater than a score threshold.

[0015] According to at least one embodiment of the target tracking method of this disclosure, updating the dynamic template image includes: taking the center of the target bounding box corresponding to the target tracking result as a reference, expanding outward to a region twice the area of ​​the dynamic template image, and using the image corresponding to the expanded region as the updated dynamic template image.

[0016] According to another aspect of this disclosure, an electronic device is provided, comprising: a memory storing execution instructions; and a processor executing the execution instructions stored in the memory, causing the processor to perform a target tracking method according to any embodiment of this disclosure.

[0017] According to another aspect of this disclosure, a readable storage medium is provided, wherein executable instructions are stored therein, which, when executed by a processor, are used to implement the target tracking method of any embodiment of this disclosure.

[0018] According to another aspect of this disclosure, a computer program product is provided, including a computer program that, when executed by a processor, implements a target tracking method according to any embodiment of this disclosure.

[0019] This disclosure can enhance the perception of multi-scale appearance information of targets, achieve accurate multi-scale capture and enhancement of target features, and ensure the integrity of the target's semantic structure and spatial position relationship; it can accurately strengthen the representation of the target's core features and filter background redundancy interference, providing highly discriminative features with multi-scale target perception capabilities for tracking tasks, thereby achieving a substantial breakthrough in target tracking performance in complex scenarios. Attached Figure Description

[0020] The accompanying drawings illustrate exemplary embodiments of the present disclosure and, together with the description thereof, serve to explain the principles of the present disclosure. These drawings are included to provide a further understanding of the present disclosure and are incorporated in and constitute a part of this specification.

[0021] Figure 1 This is a schematic flowchart of a target tracking method according to one embodiment of the present disclosure.

[0022] Figure 2 This is a flowchart illustrating the method corresponding to step S107 of one embodiment of the present disclosure.

[0023] Figure 3 This is a flowchart illustrating the method corresponding to step S109 of one embodiment of the present disclosure.

[0024] Figure 4 This is a schematic diagram of the overall structure of a target tracking model according to one embodiment of the present disclosure.

[0025] Figure 5 This is a schematic diagram of the structure of a multi-scale correlation module according to one embodiment of the present disclosure.

[0026] Figure 6 This is a schematic diagram of the structure of a target perception module according to one embodiment of the present disclosure.

[0027] Figure 7 This is a processing flow of a spatial encoder according to one embodiment of the present disclosure.

[0028] Figure 8 This is a schematic diagram of a standard self-attention calculation process according to one embodiment of the present disclosure.

[0029] Figure 9 This is a schematic diagram of the self-attention process of a fusion selective elimination module according to one embodiment of the present disclosure.

[0030] Figure 10 This is a schematic diagram of a standard softmax calculation method according to one embodiment of the present disclosure.

[0031] Figure 11This is a schematic diagram of the softmax calculation method of a fusion selective elimination module according to one embodiment of the present disclosure.

[0032] Figure 12 This is a schematic diagram of the processing flow of a frequency domain encoder according to one embodiment of the present disclosure.

[0033] Figure 13 This is a schematic diagram of the structure of a decoder according to one embodiment of the present disclosure.

[0034] Figure 14 This is a schematic diagram of the structure of a prediction head according to one embodiment of the present disclosure.

[0035] Figure 15 This is a schematic structural block diagram of an electronic device employing a processor-based hardware implementation according to one embodiment of the present disclosure. Detailed Implementation

[0036] The present disclosure will now be described in further detail with reference to the accompanying drawings and examples. It should be understood that the specific examples described herein are for illustrative purposes only and are not intended to limit the scope of the disclosure. Furthermore, it should be noted that, for ease of description, only the parts relevant to the present disclosure are shown in the accompanying drawings.

[0037] It should be noted that, where there is no conflict, the embodiments and features described in this disclosure can be combined with each other. The technical solutions of this disclosure will now be described in detail with reference to the accompanying drawings and embodiments.

[0038] This disclosure applies to scenarios such as security and intelligent monitoring.

[0039] To address the technical bottlenecks commonly found in traditional Transformer tracking frameworks, this disclosure proposes a target tracking method, which includes: performing target tracking using a target tracking model based on a search region image, a static template image, and a dynamic template image, to obtain the target tracking result.

[0040] The search region image can be a video frame or image region where the target to be detected is located, and it is used to locate and track the target in the search region image.

[0041] A static template image can be a pre-selected or initialized target template image used to characterize the fixed appearance features of the target, which remain unchanged during tracking.

[0042] Dynamic template images can be adaptively updated target template images based on historical tracking results, used to adapt to changes in target pose, lighting, and appearance, thereby improving tracking robustness. As one possible implementation, dynamic template images can employ an online adaptive update strategy, which can promptly memorize the target's dynamic changes in the video time sequence, effectively addressing feature shift issues caused by target pose, scale, and environmental interference.

[0043] As one possible implementation, before performing target tracking using a target tracking model based on the search region image, static template image, and dynamic template image to obtain the target tracking result, the method may further include: acquiring the search region image, static template image, and dynamic template image.

[0044] Figure 1 A flowchart illustrating the execution process of a target tracking model according to one embodiment of this disclosure is shown. Figure 1 The method shown includes steps S101 to S111. This method can be executed by electronic devices such as mobile phones and tablets.

[0045] In step S101, feature extraction is performed on the search region image, the static template image, and the dynamic template image to obtain the corresponding search region feature map, static template feature map, and dynamic template feature map.

[0046] In step S102, the static template feature map is split into multiple scales to obtain sub-feature maps of the static template feature map at different scales.

[0047] In step S103, each sub-feature map at each scale is correlated with the search region feature map to obtain the block-related feature map corresponding to each sub-feature map.

[0048] In step S104, the block-related feature maps corresponding to each sub-feature map at each scale are superimposed to obtain the scale-related feature map corresponding to each scale.

[0049] In step S105, through interpolation, the scale-related feature maps corresponding to different scales are mapped to the same resolution to obtain the target-related feature maps corresponding to each scale.

[0050] In step S106, the target-related feature maps corresponding to each scale are added element by element to obtain the input mask.

[0051] In step S107, background redundancy information in the search region feature map is filtered based on the input mask, and target features in the search region feature map that are similar to the static template feature map are enhanced to obtain a fused feature map.

[0052] In step S108, spatial location encoding information is added to the fused feature map to obtain the input feature map.

[0053] In step S109, the input feature map is subjected to multi-scale feature extraction and fusion processing by the encoder to obtain the output feature map.

[0054] In step S110, the output feature map is decoded by the decoder to obtain the decoded feature map.

[0055] In step S111, the target position and size are predicted based on the decoded feature map to obtain the target tracking result.

[0056] This disclosure enhances the perception of multi-scale appearance information of targets through multi-scale correlation calculations, achieving accurate multi-scale capture and enhancement of target features while ensuring the integrity of the target's semantic structure and spatial positional relationships. By performing multi-scale target similarity assessments on the search region and then designing an asymmetric mask for the search region based on these assessment results, the "asymmetric" masking mechanism effectively addresses the shortcomings of traditional trackers that treat targets and backgrounds equally. It fully exploits local block information of the template, highlighting effective features related to the target while eliminating invalid features related to the background. This disclosure can accurately enhance the representation of core target features and filter redundant background interference, providing highly discriminative features with multi-scale target perception capabilities for tracking tasks, thereby achieving a substantial breakthrough in target tracking performance in complex scenarios.

[0057] Regarding step S107, in some embodiments of this disclosure, it may include, for example... Figure 2 Steps S1071 to S1075 are shown.

[0058] In step S1071, both the static template feature map and the dynamic template feature map are reshaped into matrix form to obtain the target static template feature map and the target dynamic template feature map.

[0059] In step S1072, after background suppression is performed on the input mask, it is multiplied element-wise with the feature map of the search region to obtain the output mask.

[0060] In step S1073, the output mask is multiplied by the search region feature map to obtain the first target feature map.

[0061] In step S1074, the first target feature map is reshaped into a matrix form to obtain the second target feature map.

[0062] In step S1075, the target static template feature map, the target dynamic template feature map, and the second target feature map are spliced ​​along the first dimension to obtain a fused feature map.

[0063] Regarding step S109, in some embodiments of this disclosure, it may include, for example... Figure 3 Steps S1091 to S1093 are shown.

[0064] In step S1091, an encoder containing multiple encoding heads is constructed, with the multiple encoding heads corresponding to at least two window sizes.

[0065] In step S1092, each encoder head obtains its own single-head attention matrix by using either a frequency-domain attention-based encoding method or a spatial-domain attention-based encoding method, depending on the window size.

[0066] In step S1093, the single-head attention matrices corresponding to each encoding head are concatenated along the dimension of the channel to obtain the second attention matrix.

[0067] In step S1094, the second attention matrix is ​​combined with the first... k After the input feature maps of the layer are fused, the intermediate feature maps are obtained by layer normalization.

[0068] In step S1095, the intermediate feature map is input into the feedforward neural network for processing, and then processed through residual connections and layer normalization to obtain the first... k Layer output feature map, k Indicates the number of floors. , K This indicates the total number of layers in the encoder.

[0069] In step S1096, the first K The layer output feature map is used as the output feature map of the encoder.

[0070] This implementation introduces a multi-scale cyclic shift window attention mechanism, replacing pixel-level attention with window-level attention. This segment the target into several regions with a certain degree of integrity, avoiding the destruction of the target's semantic information. The cyclic shift mechanism generates target samples with different positions, thus providing the Transformer neural network with samples rich in positional information and effectively solving the "data hunger" problem of the Transformer neural network. Furthermore, by adaptively selecting a computationally efficient attention inference mechanism based on the window size, this implementation effectively alleviates the heavy computational and storage burden caused by cyclic shift operations.

[0071] As a further implementation, after obtaining the decoded feature map, the method further includes: inputting the decoded feature map into a multilayer perceptron, activating the output of the multilayer perceptron through an activation function to obtain a confidence score. If the confidence score is greater than a score threshold, the dynamic template image is updated.

[0072] As one possible implementation, updating the dynamic template image includes: taking the center of the target bounding box corresponding to the target tracking result as a reference, expanding outward to a region twice the area of ​​the dynamic template image, and using the image corresponding to the expanded region as the updated dynamic template image.

[0073] In one specific implementation, such as Figure 4 As shown, the target tracking model consists of six core functional modules: feature extraction module, multi-scale correlation module, target perception module, Transformer-based feature fusion module, prediction head, and score head. Figure 1 The method shown can be implemented using these six core functional modules. Each module will be introduced below.

[0074] 1. Feature extraction module.

[0075] The input to the feature extraction module includes the search region image. Static template images and dynamic template images Three types of image inputs. Among them, Represents the set of real numbers; Indicates the height of the search area image; Indicates the width of the search area image; Indicates the height of static template images and dynamic template images; This indicates the width of static and dynamic template images.

[0076] Image of the search area Static template images and dynamic template images The three types of images are input together into the feature extraction module to obtain the search region image. Corresponding search region feature map Static template images Corresponding static template feature map and dynamic template images Corresponding dynamic template feature map As shown in the following formula.

[0077] (1) in, Indicates the number of channels. This represents the feature extraction backbone network in the feature extraction module. For example, the feature extraction backbone network can be a ResNet-101 network.

[0078] 2. Multiscale correlation module (MSCB).

[0079] The main function of the multi-scale correlation module is to enhance the multi-scale target perception capability of the target tracking module. Figure 5 A schematic diagram of the structure of a multi-scale correlation module according to one embodiment of this disclosure is shown. Figure 5 The multi-scale correlation module shown includes a multi-scale segmentation (MSS) submodule and a multi-scale correlation (MSC) submodule. The multi-scale segmentation submodule can be used to apply different scales to the static template feature map, while the multi-scale correlation submodule can use the multi-scale sub-feature maps segmented by the multi-scale segmentation submodule to apply correlation operations to the search region feature map, thereby obtaining a multi-scale similarity measure related to the target in the search region feature map.

[0080] (1) Multi-scale segmentation submodule.

[0081] static template feature map Decomposed into multiple different scales (e.g., scale) This process yields sub-feature maps of the static template feature map at different scales, covering feature information from fine to coarse. Mathematically, this can be represented as: (2) in, This indicates that the static template feature map will be used. By scale The split Each block feature; This represents a multi-scale splitting operator along spatial directions.

[0082] (2) Multiscale correlation submodule.

[0083] First, each sub-feature at each scale is correlated with the feature map of the search region to obtain the block-related feature map corresponding to each sub-feature map. For example, the correlation operation can employ a similarity metric. Furthermore, the block-based correlation feature maps corresponding to each sub-feature map at each scale are superimposed to obtain the scale-related feature map for each scale. Mathematically, this can be represented as: (3) in, , , These represent scale-related feature maps at scales of 1, 2, and 4, respectively. This represents the relevant operational operators.

[0084] Then, to fuse scale-related feature maps of different scales, this embodiment uses interpolation to map the scale-related feature maps of different scales to the same resolution, obtaining target-related feature maps corresponding to each scale. Furthermore, the target-related feature maps corresponding to each scale are added element-wise to obtain the input mask. Mathematically, this can be represented as: (4) in, This represents an interpolation operation used to map scale-dependent feature maps of different scales to the same resolution.

[0085] 3. Target perception module (OA module).

[0086] The target perception module uses the input mask to determine the region from the target in the search area image. Then, the background suppression (BS) module suppresses background information, enhancing target features while suppressing background information. The structure of the target perception module is as follows: Figure 6 As shown.

[0087] First, the static template feature map and dynamic template feature map Reconstruct them into matrix form to obtain the target static template feature map. and target dynamic template feature map Mathematically, this can be represented as: (5) in, This represents the reshaping operator, used to reshape the tensor to be processed. (Can be a static template feature map) or dynamic template feature map Reshape to a pre-defined fixed size (such as the first size). This indicates the first dimension, with a size of .

[0088] Next, the input mask After background suppression, it is compared with the feature map of the search region. Element-wise multiplication is performed to filter out redundant background and highlight effective features, resulting in the output mask. ,Right now: (6) in, , Indicates the input mask The difference between the maximum and minimum values, This represents a pre-set threshold, and the corresponding formula means: when inputting a mask... The elements in the mask are greater than the input mask. The difference between the maximum and minimum values With threshold When multiplying, the value of the corresponding element is set to 1, otherwise it is set to 0; This indicates the background suppression module (i.e., the input mask is modified by the background suppression module). (Perform background suppression).

[0089] Based on this, the output mask will be... With search region feature map Perform dot multiplication to suppress background interference and obtain the first target feature map. ,Right now: (7) in, This indicates element-wise multiplication.

[0090] The first target feature map Reconstructing the data into a matrix form yields the second target feature map. ,Right now: (8) in, This indicates the second size, corresponding to the second target feature map. The size.

[0091] Finally, the reconstructed target static template feature map, target dynamic template feature map, and second target feature map are concatenated along the first dimension to obtain the fused feature map. , , This represents the total number of pixels in the fused feature map, i.e.: (9) in, This represents the concatenation operator.

[0092] 4. Transformer-based feature fusion module.

[0093] The fused feature map output by the target perception module Add spatial location encoding information After that, the input feature map required by the encoder can be obtained. ( initial value, k The sequence number indicating the layer number of the encoder and decoder. , (This represents the total number of layers in the encoder and decoder), i.e.: The feature fusion module includes an encoder and a decoder.

[0094] (1) Encoder.

[0095] An encoder comprising multiple encoding heads is constructed in the feature fusion module. These multiple encoding heads correspond to at least two window sizes. As one possible implementation, the multiple encoding heads correspond to four window sizes, including 1, 2, 4, and 8. For example, the encoder may include eight encoding heads, each with a window size of: By constructing an encoder containing multiple encoder heads, multi-scale shift window feature extraction and fusion processing can be performed, thereby protecting the semantic integrity of the target and mining rich positional information.

[0096] Considering the inherent data redundancy in cyclic matrices, performing related operations directly in the spatial domain would significantly increase the computational complexity of the model. Therefore, this implementation adopts the following strategy: when the window size is large (greater than the size threshold) (e.g., equal to 8), a frequency domain attention-based encoding method (frequency domain encoder) is used; conversely, when the window size is small (less than or equal to the size threshold) (e.g., equal to 1, 2, or 4), a spatial domain attention-based encoding method (spatial domain encoder) is used.

[0097] (a) Spatial encoder.

[0098] Figure 7 The processing flow of a spatial encoder according to one embodiment of this disclosure is shown.

[0099] First, the input feature map The data is preprocessed into blocks and then input into an 8-head, 6-layer encoder. The block processing is shown in the following formula: (10) in, Indicates the first i Block feature maps corresponding to each encoded header; i Indicates the sequence number of the encoding header; Operators that represent partitioning along a channel; Indicates the third dimension; Indicates the number of windows (tokens). Indicates the size of the window.

[0100] Next, the block feature maps are transformed using a linear transformation. A linear transformation is performed along the channel dimension, resulting in an output dimension three times that of the original feature dimension. This output is then divided into blocks along the channels to obtain the query tensors for each block. Key tensors Sum tensor , The specific formula is as follows: (11) in, This represents a linear mapping transformation operator.

[0101] Then, by adjusting the key tensor Sum tensor By performing a cyclic shift operation, the key cyclic tensor can be obtained respectively. Sum Circular Tensor ,Right now: (12) in, , , Indicating the focus of bullish attention i The number of channels in each encoder header. This indicates a cyclic shift operation on the second and third dimensions of the tensor.

[0102] Then, for the query tensor , key cyclic tensor Sum Circular Tensor Dimensional reshaping is performed separately to obtain the query matrix. Key cyclic matrix Sum cyclic matrix Right now: (13) in, Indicates the fourth dimension; This indicates the fifth dimension.

[0103] Subsequently, the key cyclic matrix Transpose to obtain the key cyclic transpose matrix The first feature attention matrix is ​​obtained by calculating the following formula. ,Right now: (14) in, This represents the scaling factor for attention calculation.

[0104] Considering that cyclic shift operations can cause boundary effects (discontinuities in boundary stitching due to shifting), this implementation constructs a penalty matrix to address this issue. To obtain the first attention matrix with spatial penalty properties The specific formula is shown below.

[0105] (15) Among them, the penalty matrix ; Indicates the size is A matrix of all 1s; Represents the Kronecker product operator; ; This represents the mask matrix; the superscript T indicates the transpose of the matrix. Represents a matrix whose elements are all 1s; This represents the vector used to construct the mask matrix. , Indicates the use of constructing vectors The vector, , Indicates the use of constructing vectors Intermediate parameters.

[0106] The self-attention mechanism in the Transformer architecture is prone to involving irrelevant windows in the feature fusion process, causing interference between target and background features. To address this issue, this implementation introduces a Selective Elimination (SE) module into the Transformer self-attention mechanism to enhance the model's ability to filter and focus on key visual features, effectively improving the accuracy of the tracking task while ensuring computational efficiency.

[0107] like Figure 8 As shown, in the original self-attention activation process, the generated attention matrix is ​​input without preprocessing. The activation function causes the target and background information to be treated indiscriminately, which can lead to background and interference misleading tracking and localization. To solve this problem, this embodiment designs an "asymmetric" information processing module, namely a selective elimination module.

[0108] Figure 9 The signal processing flow of the fusion selective cancellation module is shown for precise filtering of attention scores. Figure 10 and Figure 11 The self-attention activation of the original self-attention activation and the self-attention activation of the fused selective elimination module are shown respectively. As shown in the following equation: (16) in, This represents the second feature attention matrix; Indicates the selection elimination operator; Indicates the activation function; Representing the scatter operator, such as Figure 11 As shown.

[0109] Attention matrix of the second target feature after processing by the selective elimination module AND-valued cyclic matrix Perform matrix multiplication to obtain the attention score matrix. ,Right now: (17) Then, the attention score matrix Reshape to generate a single-head attention matrix. This is done to unify the spatial characteristics of each encoding header and facilitate subsequent channel splicing processing, as shown in the following formula: (18) in, This represents the single-head attention matrix of the i-th encoder head; This indicates the sixth dimension.

[0110] (b) Frequency domain encoder.

[0111] Figure 12 The processing flow of a frequency domain encoder according to one embodiment of this disclosure is illustrated. When the window size is set to 8, the network structure of the frequency domain encoder can improve the training efficiency and inference speed of the network. The design principles of the frequency domain encoder are described in detail below.

[0112] To obtain the first feature attention matrix The frequency domain encoder needs to execute formulas (10) and (11) in the same way to obtain the input feature map. The corresponding first feature attention matrix .

[0113] According to equations (14) and (9), the first feature attention matrix The spatial domain calculation results can be represented as follows: (19) in, Represents the first feature attention matrix The n line, number m Column elements; Representing the query matrix The OK; Represents the key cyclic transpose matrix The first in Columns; symbols It represents all elements of a matrix or tensor in the corresponding dimension.

[0114] Through reasoning, it can be concluded that... m It can be determined by parameters and parameters Calculated, i.e. ,and, , .

[0115] Therefore, first fix the parameters Then, in the traversal formula (19) The cases are combined into row vectors, and it can be observed that equation (20) holds.

[0116] (20) in, Indicates the relevant operations; Represents the column vectorization operator; Indicates the channel index number; Represents the query tensor The first dimension is The fourth dimension is All elements; Represents the key cyclic tensor The first dimension is The second dimension is 1, and the fifth dimension is... All elements; Key tensor The first dimension is The fourth dimension is All elements.

[0117] Considering the conversion relationship between correlation operations and convolution operations, i.e. Then equation (20) can be transformed into equation (21), that is: (twenty one) in, This represents the convolution operation; Tensor The reverse fold signal.

[0118] Based on this, the parameters in equation (20) Perform traversal and concatenation into a new row vector, while simultaneously adjusting the parameters. It is also traversed, therefore the first feature attention matrix Satisfying equation (22), that is: (twenty two) According to the convolution theorem, convolution operations in the spatial domain can be converted into element-wise operations in the frequency domain, and the corresponding frequency domain calculation form is as follows: (twenty three) in, This indicates element-wise multiplication; Represents the query tensor Perform a two-dimensional Fourier transform in the second and third dimensions (i.e. Figure 10 In , This represents the tensor formed by expanding the result after a 2-bit Fast Fourier Transform (FFT), specifically... , Indicates tensor repeat The tensor generated by the second expansion; This indicates that the index of the fourth dimension in the query transformation extended tensor is... c All elements; Represents the key transformation conjugate tensor The fourth dimension index is c All elements; The key transformation conjugate tensor is a key tensor. The bond transform tensor obtained by performing a two-dimensional fast Fourier transform The conjugate tensor; This represents the two-dimensional inverse fast Fourier transform. This represents the operator for taking the real part; Represents the set of complex numbers.

[0119] Thus far, formulas (19) to (23) have detailed how the frequency domain encoder uses the query tensor... Key tensors How to obtain the first feature attention matrix The process.

[0120] The frequency domain encoder uses the first feature attention matrix Obtain the attention score matrix The method is the same as that of the spatial encoder, and will not be repeated here.

[0121] according to Attention score matrix The line, number A column can be represented as: (twenty four) in, Represents the attention score matrix Bank of China code , column number is Element; Indicates that the parameter o Represented by parameters f ,parameter g and parameters c Constructed scalar, , , ; Represents the attention score matrix Bank of China code , column number is Element; Indicates that the parameter Represented by parameters h ,parameter u A scalar constructed with parameter v, , , ; Represents the second feature attention matrix Bank of China code , column number is Element; Represents a value cyclic matrix Bank of China code , column number is Element; Represents a value circular matrix Bank of China code , column number is Element. Represents a value-cyclic matrix, which is a value-cyclic tensor. In matrix form.

[0122] Noting that the value tensor is formed by cyclic shifting, we have: (25) Through reasoning, we can conclude that: , Therefore, the second feature attention matrix Reshaping the Tensor for Secondary Features: Attention Reshaping Tensor Then, combine the value cyclic tensor AND-value tensor The relationship between them can be rewritten as (25) as follows: (26) in, The second feature attention reshaping tensor The first dimension index is n The second dimension index is h The third-dimensional index is u The fourth dimension index is v Element; Value tensor The first dimension index is h The second dimension index is The third-dimensional index is The fourth dimension index is c Element.

[0123] According to the definition of discrete convolution, equation (26) can be further modified as follows: (27) in, The second feature attention reshaping tensor The first dimension index is n The second dimension index is h All elements; Value tensor The first dimension index ish The fourth dimension index is c All elements.

[0124] Obviously, when the window When the value is large, equation (27) can be computed quickly by introducing the Fast Fourier Transform, i.e.: (28) in, The second feature attention reshaping tensor The second feature attention transformation tensor obtained by two-dimensional fast Fourier transform; Represents the second feature attention transformation tensor The first dimension index is n The second dimension index is h All elements; Value tensor The value transformation tensor obtained by the two-dimensional fast Fourier transform; Representation value transformation tensor The first dimension index is h The fourth dimension index is c All elements.

[0125] To facilitate element-wise multiplication of the frequency domain tensor, the tensor is reshaped by attention to the second feature. Sum Transformation Tensor Dimensional expansion is achieved through a broadcast mechanism, enabling... ,make Then we have: (29) in, Represents the attention score matrix Bank of China code All elements; Represents the second feature attention transformation tensor The second feature attention dimension-expanded tensor obtained after performing a fifth-dimensional expansion; Represents the second feature attention dimension expansion tensor The first dimension index is n The second dimension index is h All elements; Representation value transformation tensor The value-dimensional tensor obtained after performing the first-dimensional expansion; Representation of value extended tensor The first dimension index is n The second dimension index is h All elements.

[0126] Therefore, the attention score matrix can be obtained according to formula (29). Obtain the attention score matrix. The subsequent steps are the same as those for the spatial encoder, and will not be repeated here.

[0127] Existing traditional methods in attention score matrix During the calculation process, it is necessary to convert the value tensor (Fourth-order tensor) First, perform a circular shift in space to obtain a value circular tensor. Then loop the value tensor Further reshaping into a value loop matrix in matrix form Finally, it is combined with the second feature attention matrix after selective elimination. Multiplying yields the attention score matrix. While this approach can increase the diversity of the sample, it also leads to a significant increase in sample storage and computational load.

[0128] After research, the above-described embodiments of this disclosure perform reasoning using frequency domain computation, eliminating the need for further analysis of the value tensor. (For a fourth-order tensor) instead of performing a spatial circular shift, the value tensor is directly transformed using a two-dimensional fast Fourier transform. The algorithm is transformed to the frequency domain, and then uses a broadcast mechanism and a two-dimensional fast Fourier transform algorithm to achieve the same computational effect as the traditional method. This innovative approach adopted in this disclosure avoids the operation of generating cyclically shifted samples in the spatial domain, significantly improving network inference efficiency and thus greatly reducing sample storage and computational load.

[0129] Based on the above frequency domain encoder, the process of obtaining the corresponding single-head attention matrix using a frequency domain attention-based encoding method may include steps S201 to S212.

[0130] S201: Perform a linear transformation on each encoder header to obtain the query tensor corresponding to each encoder header. Key tensors Sum tensor .

[0131] S202: For the query tensor Key tensors Sum tensor Perform two-dimensional fast Fourier transforms to obtain the query transform tensor. Key Transformation Tensor Sum Transformation Tensor .

[0132] S203: Transform the query tensor Expanding the query transform tensor yields the expanded query transform tensor. .

[0133] S204: Calculate the key transformation tensor The conjugate tensor of the bond transformation is obtained. .

[0134] S205: Expand tensor based on query transformation Bond Transformation Conjugate Tensor The first feature attention matrix is ​​obtained through the following formula. :

[0135] in, Represents the first feature attention matrix Bank of China code n All elements; Represents the column vectorization operator; Represents the operator for taking the real part; This represents the two-dimensional inverse fast Fourier transform. This indicates element-wise multiplication; Represents the query transformation extended tensor The fourth dimension index is c All elements; Represents the key transformation conjugate tensor The fourth dimension index is c All elements; c Indicates the index of the channel; Indicating the focus of bullish attention i The number of channels in each encoder header; Indicates the size of the window; the superscript T indicates the transpose of the matrix.

[0136] S206: Log-value Transformation Tensor By expanding the dimensions, we obtain the value-dimensional tensor. , making ,in, Representation of value extended tensor The first dimension index is n The second dimension index is h The fifth dimension index is c All elements; Representation value transformation tensor The first dimension index is h The fourth dimension index is c All elements; N Indicates the total number of windows; n Indicates the window number.

[0137] S207: Apply the first feature attention matrix Reconstructing it into tensor form yields the second feature attention-reconstructed tensor. .

[0138] S208: Attention to the second feature reshapes the tensor Perform a two-dimensional fast Fourier transform to obtain the second feature attention transform tensor. .

[0139] S209: Attention Transform Tensor for Second Feature By expanding the dimension, we obtain the second feature attention dimension-expanded tensor. , making ; Represents the second feature attention dimension expansion tensor The first dimension index is n The second dimension index is h The fifth dimension index is c All elements; Represents the second feature attention transformation tensor The first dimension index is n The second dimension index is h All elements.

[0140] S210: Dimensional expansion of tensors based on values Second feature attention extended dimension tensor The attention score matrix corresponding to each encoder head is obtained by the following formula. :

[0141] in, Represents the attention score matrix Bank of China code All elements; Represents the second feature attention dimension expansion tensor The first dimension index is n The second dimension index is h All elements; Representation of value extended tensor The first dimension index is n The second dimension index is h All elements; h Indicates the index number.

[0142] S211: Attention score matrix Reshape the matrix to obtain a single-head attention matrix. ,in, Indicates the first i The single-head attention matrix corresponding to each encoding head.

[0143] Thus, the single-head attention matrix corresponding to each encoder (including spatial and frequency domains) is obtained. Subsequently, the size is adjusted by dimensionality reshaping to obtain the single-head attention matrix corresponding to each encoder. Along the dimension where the channel is located (i.e., the last dimension) The two attention matrices are concatenated to obtain the second attention matrix. ,Right now: (30) in, .

[0144] The second attention matrix With the k Layer input feature map The data is then fused and fed into a Layer Normalization (LN) layer to obtain intermediate feature maps. .

[0145] Then the intermediate feature map The data is fed into a feedforward network (FFN) for information processing, and then passes through a residual connection layer and a normalization layer to obtain the first... k +1 layer input feature map (i.e., k-th layer output feature map): (31) in, , and Both represent linear projection weight matrices. and Each represents the corresponding bias matrix. Indicates the activation function; Presentation layer normalization operation, Representation of features The mean, Representation of features standard deviation It is a very small number to ensure that the denominator is not zero. This represents the learnable scaling parameter. This represents the learnable translation parameters.

[0146] At this point, the encoder block... The complete signal flow has been described. This process is executed cyclically. K Each layer's output is directly used as the input to the next layer, and the output of the last layer is used as the encoder's output feature map. Specifically, it is shown in equation (16): (32) (2) Decoder Figure 13 A schematic diagram of the decoder structure according to one embodiment of this disclosure is shown. Figure 13 As shown. The initial input to the decoder is Iterate using equation (33). Figure 13 First, the decoder passes through a multi-head self-attention (MSA) module, followed by skip connections and layer normalization. Then, the normalized output and encoder output are fed together into a multi-head cross-attention (MCA) module. The output from the MCA module then passes through residual connections and a layer normalization layer, before being fed into an open-ended nonlinear generation (FFN) layer. Finally, the output of the FFN layer passes through residual connections and a layer normalization layer again to obtain the decoder output. This process is repeated iteratively. K =6 times, the mathematical model is as follows: (33) The decoder finally (the first) K Decoded feature map output by layer Feed to the prediction head and the score head.

[0147] 4. Predicting the head.

[0148] The prediction head predicts the target location based on the decoded feature map, thus obtaining the target tracking result.

[0149] As one possible implementation, to optimize bounding box estimation performance, a corner prediction head can be used. The corner prediction head obtains the final corner coordinates by outputting the coordinates of the two corner points (top left and bottom right corners) of the target bounding box and performing probability distribution expectation estimation on the biprobability map.

[0150] like Figure 14 As shown, based on the feature map of the search region From the encoder's output feature map Extract target feature map and utilize Calculate target feature map and The similarity is used to obtain the modulation vector. As shown in the following formula.

[0151] (34) in, , , This indicates a cross-correlation operation.

[0152] Then, the modulation vector repeat Next, compare with the target feature map Element-wise multiplication yields the enhanced features. As shown in the following formula.

[0153] (35) in, This indicates that the modulation vector (column vector) will be used. repeat Second-rate.

[0154] Enhanced features The feature map is obtained after reshaping. Then the feature map The input is fed into a Fully Convolutional Network (FCN) to obtain a first probability matrix corresponding to the top-left corner and a second probability matrix corresponding to the bottom-right corner of the target bounding box. Then, the coordinates of the top-left corner are calculated using the first probability matrix, and the coordinates of the bottom-right corner are calculated using the second probability matrix. The optimal candidate sample can be obtained from the top-left and bottom-right corners.

[0155] 5. Fraction header.

[0156] During target tracking, the target's appearance deforms over time, making it crucial to capture its latest state in real time. To address the update failure issue caused by complete target occlusion or removal from the field of view, a fractional head is designed in one possible implementation to evaluate the reliability of the target tracking results corresponding to the search region image. The decoded feature map output by the decoder... The data is input into a multilayer perceptron, and then... The function activates the output, yielding a confidence score. The specific expression is shown in the following formula: (36) in, This represents a three-layer perceptron, where all hidden layers are activated using the ReLU function. This represents the activation function. The dynamic template image is updated when the confidence score of the target tracking result is greater than a score threshold.

[0157] The target tracking model also needs to be pre-trained. One possible implementation is to use an end-to-end training approach, utilizing... Optimize the Generalized Intersection Over Union (GIOU) ratio between the predicted bounding boxes (predicted target bounding boxes) and the ground truth bounding boxes (ground truth target bounding boxes), while introducing... The loss function improves the prediction accuracy of the target spatial geometry information. The loss function is designed as shown in the following equation.

[0158] (37) in, This represents the loss function during the first phase of training. Indicates the first A vector consisting of the x and y coordinates of the top-left corner of each training sample's ground truth bounding box, and the width and height of the target bounding box; subscript Indicates the sample sequence number; This represents the vector of the same dimension corresponding to the prediction box (i.e., the vector composed of the x-coordinate and y-coordinate of the top left corner of the prediction box, as well as the width and height of the target box). and All of these represent non-negative hyperparameters; Indicates the batch sample size; Indicates the crossover and union ratio loss; This represents L1 loss.

[0159] As a possible implementation, to avoid the joint training of localization and classification tasks from easily getting trapped in local optima, a two-stage decoupled training strategy is adopted. In the two-stage decoupled training strategy, the first stage removes the candidate sample score headers and performs end-to-end training only using the localization loss in equation (37) to ensure that the search area can effectively cover the target and enhance the model's localization ability. The second stage of training focuses on the classification task (determining whether the current frame contains the target), and its loss function is defined as follows: (38) in, This represents the loss function during the second phase of training. Indicates the first Binarized labels for each training sample; Indicates the first The score head confidence score of each training sample.

[0160] According to further embodiments of this disclosure, an electronic device is also provided. Figure 15This diagram illustrates a schematic block diagram of an electronic device employing a processor-based hardware implementation according to an embodiment of the present disclosure. The hardware structure of the electronic device of the present disclosure can be implemented using a bus architecture. The bus architecture can include any number of interconnect buses and bridges, depending on the specific application and overall design constraints of the hardware. Bus 1100 connects various circuits including one or more processors 1200, memory 1300, and / or hardware modules. Bus 1100 can also connect various other circuits 1400 such as peripheral devices, voltage regulators, power management circuits, external antennas, etc. Bus 1100 can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Component (EISA) bus, etc. Buses can be categorized as address buses, data buses, control buses, etc. For ease of illustration, only one connecting line is used in this figure, but this does not indicate that there is only one bus or one type of bus.

[0161] This disclosure also provides a readable storage medium storing a computer program that, when executed by a processor, is used to implement the methods described above. A "readable storage medium" can be any means capable of containing, storing, communicating, propagating, or transmitting a program for use by or in conjunction with an instruction execution system, apparatus, or device. More specific examples of a readable storage medium include: an electrical connection with one or more wires (electronic device), a portable computer disk drive (magnetic device), random access memory (RAM), read-only memory (ROM), erasable and programmable read-only memory (EPROM or flash memory), fiber optic devices, and portable read-only memory (CDROM), etc.

[0162] This disclosure also provides a computer program product, the methods of which can be implemented wholly or partially through software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented wholly or partially as a computer program product. The computer program product includes one or more computer programs or instructions. When the computer program or instructions are loaded and executed, all or part of the processes or functions of this disclosure are performed.

[0163] Computer programs or instructions can be stored in a readable storage medium or transferred from one readable storage medium to another. For example, the computer program or instructions can be transferred from one website, computer, server, or data center to another website, computer, server, or data center via wired or wireless means. The readable storage medium can be any available medium capable of access, or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium, such as a floppy disk, hard disk, or magnetic tape; an optical medium, such as a digital video optical disc; or a semiconductor medium, such as a solid-state drive. The computer-readable storage medium can be a volatile or non-volatile storage medium, or it can include both volatile and non-volatile types of storage media.

[0164] Those skilled in the art will understand that embodiments of this disclosure can be provided as methods, systems, or computer program products. Therefore, this disclosure can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this disclosure can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0165] This disclosure is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus, and computer program products according to this disclosure. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0166] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0167] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0168] In the description of this specification, the references to terms such as "one embodiment / mode," "some embodiments / modes," "example," "specific example," or "some examples," etc., refer to specific features, structures, or characteristics described in connection with that embodiment / mode or example, which are included in at least one embodiment / mode or example of this disclosure. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment / mode or example. Moreover, the specific features, structures, or characteristics described may be combined in any suitable manner in one or more embodiments / modes or examples. Furthermore, without contradiction, those skilled in the art can combine and integrate the different embodiments / modes or examples described in this specification, as well as the features of different embodiments / modes or examples.

[0169] Those skilled in the art should understand that the above embodiments are merely for illustrating the present disclosure and are not intended to limit the scope of the disclosure. Those skilled in the art can make other changes or modifications based on the above disclosure, and these changes or modifications still fall within the scope of the present disclosure.

Claims

1. A target tracking method characterized by, include: Based on the search region image, static template image, and dynamic template image, a target tracking model is used to track the target and obtain the target tracking result. The target tracking model performs the following steps: Feature extraction is performed on the search region image, static template image, and dynamic template image respectively to obtain the corresponding search region feature map, static template feature map, and dynamic template feature map; The static template feature map is split into multiple scales to obtain sub-feature maps of the static template feature map at different scales; Each sub-feature map at each scale is correlated with the feature map of the search region to obtain the block-related feature map corresponding to each sub-feature map; The block-related feature maps corresponding to each sub-feature map at each scale are superimposed to obtain the scale-related feature map corresponding to each scale. By interpolating, scale-related feature maps corresponding to different scales are mapped to the same resolution to obtain target-related feature maps corresponding to each scale. The target-related feature maps at each scale are summed element by element to obtain the input mask; The process involves filtering background redundancy in the search region feature map based on the input mask and enhancing target features similar to the static template feature map in the search region feature map to obtain a fused feature map. This includes: reshaping both the static template feature map and the dynamic template feature map into matrix form to obtain a target static template feature map and a target dynamic template feature map; performing background suppression on the input mask and then multiplying it element-wise with the search region feature map to obtain an output mask; performing a dot product between the output mask and the search region feature map to obtain a first target feature map; reshaping the first target feature map into matrix form to obtain a second target feature map; and concatenating the target static template feature map, the target dynamic template feature map, and the second target feature map along a first dimension to obtain a fused feature map. Spatial location encoding information is added to the fused feature map to obtain the input feature map; The input feature map is subjected to multi-scale feature extraction and fusion processing by an encoder to obtain an output feature map; The output feature map is decoded using a decoder to obtain a decoded feature map; and Based on the decoded feature map, the target position and size are predicted to obtain the target tracking result.

2. The object tracking method of claim 1, wherein, The input feature map is subjected to multi-scale feature extraction and fusion processing by an encoder to obtain an output feature map, including: Construct an encoder containing multiple encoding heads, the multiple encoding heads corresponding to at least two window sizes; Each encoder head uses either a frequency-domain attention-based coding method or a spatial-domain attention-based coding method, depending on the size of the window, to obtain a single-head attention matrix corresponding to each encoder head; The single-head attention matrices corresponding to each encoder are concatenated along the dimension of the channel to obtain the second attention matrix; fusing the second attention matrix with the input feature map of the layer k After the input feature map of the layer is fused, an intermediate feature map is obtained through layer normalization. The intermediate feature map is input into a feedforward neural network for processing, and then is connected with a residual connection and layer normalization to obtain a first output feature map k layer output feature map, k representing the number of layers, , K representing the total number of layers of the encoder; and The first K The layer output feature map is used as the output feature map of the encoder.

3. The target tracking method as described in claim 2, characterized in that, The encoding head uses a frequency-domain attention-based encoding method to obtain the corresponding single-head attention matrix, including: Perform a linear transformation on each encoded header to obtain the query tensor corresponding to each encoded header. Key tensors Sum tensor ; For query tensors Key tensors Sum tensor Perform two-dimensional fast Fourier transforms to obtain the query transform tensor. Key Transformation Tensor Sum Transformation Tensor ; Transform tensor for query Expanding the query transform tensor yields the expanded query transform tensor. ; Calculate the key transformation tensor The conjugate tensor of the bond transformation is obtained. ; Transform the tensor based on the query Bond Transformation Conjugate Tensor The first feature attention matrix is ​​obtained through the following formula. : in, Represents the first feature attention matrix Bank of China code n All elements; Represents the column vectorization operator; Represents the operator for taking the real part; This represents the two-dimensional inverse fast Fourier transform. This indicates element-wise multiplication; Represents the query transformation extended tensor The fourth dimension index is c All elements; Represents the key transformation conjugate tensor The fourth dimension index is c All elements; c Indicates the index of the channel; Indicating the focus of bullish attention i The number of channels in each encoder header; Indicates the window size; the superscript T indicates the transpose of the matrix; Log-value transformation tensor By expanding the dimensions, we obtain the value-dimensional tensor. , making ,in, Representation of value extended tensor The first dimension index is n The second dimension index is h The fifth dimension index is c All elements; Representation value transformation tensor The first dimension index is h The fourth dimension index is c All elements; N Indicates the total number of windows; n Indicates the window's sequence number; The first feature attention matrix Reconstructing it into tensor form yields the second feature attention-reconstructed tensor. ; Attention Reshaping Tensor for Second Feature Perform a two-dimensional fast Fourier transform to obtain the second feature attention transform tensor. ; Attention Transform Tensor for Second Feature By expanding the dimension, we obtain the second feature attention dimension-expanded tensor. , making ; Represents the second feature attention dimension expansion tensor The first dimension index is n The second dimension index is h The fifth dimension index is c All elements; Represents the second feature attention transformation tensor The first dimension index is n The second dimension index is h All elements; Expanding the dimensionality of the tensor based on value Second feature attention extended dimension tensor The attention score matrix corresponding to each encoder head is obtained by the following formula. : in, Represents the attention score matrix Bank of China code All elements; Represents the second feature attention dimension expansion tensor The first dimension index is n The second dimension index is h All elements; Representation of value extended tensor The first dimension index is n The second dimension index is h All elements; h Indicates the index number; Attention score matrix Reshape the matrix to obtain a single-head attention matrix. ,in, This represents the single-head attention matrix corresponding to the i-th encoder head.

4. The target tracking method as described in claim 1, characterized in that, After obtaining the decoded feature map, the following steps are also included: After inputting the decoded feature map into a multilayer perceptron, the output of the multilayer perceptron is activated by an activation function to obtain a confidence score; and If the confidence score is greater than the score threshold, the dynamic template image is updated.

5. The target tracking method as described in claim 4, characterized in that, Updating the dynamic template image includes: Using the center of the target bounding box corresponding to the target tracking result as a reference, expand outward to a region with an area twice the area of ​​the dynamic template image, and use the image corresponding to the expanded region as the updated dynamic template image.

6. An electronic device, characterized in that, include: The memory stores execution instructions; as well as A processor that executes the execution instructions stored in the memory, causing the processor to perform the target tracking method according to any one of claims 1 to 5.

7. A readable storage medium, characterized in that, The readable storage medium stores execution instructions, which, when executed by a processor, are used to implement the target tracking method according to any one of claims 1 to 5.

8. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the target tracking method according to any one of claims 1 to 5.

Images