A Target Tracking Method and System Based on a Hybrid Network of Mamba and Attention Mechanism

Abstract

This invention proposes a target tracking method and system based on a hybrid network of Mamba and attention mechanisms. The method includes: concatenating a template image and a search region image in length; inputting the concatenated result into a Mamba block in a parameter-adjusted hybrid network of Mamba and attention mechanisms, enhancing the concatenated features through weighted depthwise convolution, multi-directional state space scanning, and residual MLP layers; performing self-attention calculation on the Mamba block output using a multi-head self-attention mechanism via an attention module to achieve feature interaction and fusion between the template image and the search region image, obtaining fused global features; and inputting the fused global features into a prediction head for target tracking. This invention effectively solves the order dependency problem of the Mamba model when processing two-dimensional visual data through a multi-directional state space scanning and recovery mechanism.

Description

Technical Field

[0001] This invention relates to the field of computer vision and image processing technology, and in particular to a target tracking method and system based on a hybrid network of Mamba and attention mechanisms. Background Technology

[0002] Visual tracking is one of the fundamental tasks in the field of computer vision, and it is widely used in mobile robots, video surveillance, and autonomous driving, among other areas. However, there are many challenges that affect the robustness of trackers during the tracking process, such as occlusion, drastic changes in appearance, and deformation.

[0003] In recent years, to characterize complex patterns in visual data, two main backbone networks have been proposed: Convolutional Neural Networks (CNNs) and Transform Networks (VTs), which have been widely applied in various visual tasks. Compared to CNNs, ViTs, by integrating self-attention mechanisms, have demonstrated stronger learning capabilities on large-scale datasets. However, the quadratic complexity of self-attention mechanisms leads to significant computational overhead in downstream tasks involving high spatial resolution. To address this challenge, researchers have devoted considerable effort to improving the computational efficiency of attention. However, existing methods are either limited by the size of the effective receptive field or show a significant decline in performance across various tasks. Summary of the Invention

[0004] In view of the above, the main objective of this invention is to propose a target tracking method and system based on a hybrid network of Mamba and attention mechanisms to solve the aforementioned technical problems.

[0005] This invention proposes a target tracking method based on a hybrid network of Mamba and attention mechanisms, the method comprising the following steps:

[0006] Step 1: Construct a tracking model based on a hybrid network of Mamba and attention mechanisms and a prediction head;

[0007] Step 2: Pre-train and tune the parameters of the hybrid Mamba and attention mechanism network using a large-scale dataset to obtain a parameter-tuned hybrid Mamba and attention mechanism network;

[0008] Step 3: Concatenate the template image and the search region image along their length to obtain the concatenated result. Input the concatenated result into the Mamba block of the parameter-adjusted Mamba and attention mechanism hybrid network. After linear transformation and partitioning the template and search region, calculate and reconstruct the scanning sequence in different directions through equal-weight depthwise convolution to perform multi-directional state space scanning using the state space model to obtain the scanning result. After restoring the original spatial relationship of the scanning result, perform aggregation and normalization processing to fuse multi-directional features and enhance network stability. Then process it through linear layers and residual MLP layers to obtain the Mamba block output.

[0009] The attention module in the hybrid network of Mamba and attention mechanism, which uses parameter-adjusted Mamba and attention mechanisms, performs self-attention calculation through a multi-head self-attention mechanism to realize feature interaction and fusion between the template image and the search region image, and obtain the fused global features.

[0010] Step 4: Input the fused global features into the prediction head to obtain the target's classification score and predicted bounding box in the search area; combine the target's classification score and predicted bounding box in the search area with the real labels given in the dataset to construct a loss function to optimize the tracking model, resulting in an optimized tracking model; use the optimized tracking model to track the target.

[0011] This invention also proposes a target tracking system based on a hybrid network of Mamba and attention mechanisms, the system comprising:

[0012] Build modules are used for:

[0013] The tracking model is based on a hybrid network of Mamba and attention mechanisms and a prediction head.

[0014] Pre-trained modules are used for:

[0015] We pre-trained and tuned the parameters of a hybrid Mamba and attention mechanism network using a large-scale dataset to obtain a parameter-tuned hybrid Mamba and attention mechanism network.

[0016] The calculation module is used for:

[0017] The template image and the search region image are concatenated along their length to obtain the concatenated result. The concatenated result is then input into a Mamba block of a hybrid network of parameter-adjusted Mamba and attention mechanisms. After linear transformation and partitioning into template and search regions, the network is calculated and reconstructed into scanning sequences in different directions using equal-weight depthwise convolutions to perform multi-directional state space scanning using a state space model, thus obtaining the scanning result. The scanning result is then processed to restore the original spatial relationships, followed by aggregation and normalization to fuse multi-directional features and enhance network stability. Finally, it is processed through linear layers and residual MLP layers to obtain the output of the Mamba block.

[0018] The output of the Mamba block utilizes the attention module in the hybrid network of parameter-adjusted Mamba and attention mechanisms to perform self-attention calculation through a multi-head self-attention mechanism, thereby realizing feature interaction and fusion between the template image and the search region image to obtain the fused global features.

[0019] The tracking module is used for:

[0020] The fused global features are input into the prediction head to obtain the target's classification score and predicted bounding box in the search region. The target's classification score and predicted bounding box in the search region are combined with the real labels given in the dataset to construct a loss function to optimize the tracking model, resulting in an optimized tracking model. The optimized tracking model is then used for target tracking.

[0021] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0022] 1. This invention achieves efficient interaction and fusion of template and search region features through the collaborative design of a single-stream network structure and a Mamba-attention hybrid module. Specifically, by combining shallow Mamba blocks with deep attention modules, the secondary computational complexity brought by traditional attention mechanisms is significantly reduced while retaining the ability to model global context. The same-weighted deep convolution and multi-directional state space scanning mechanism introduced in the Mamba block enhance the diversity and robustness of features by performing feature concatenation and state space modeling of the template and search region in four directions, and optimizes the computational complexity to a linear level, thereby improving tracking efficiency while maintaining high accuracy.

[0023] 2. This invention effectively solves the order dependency problem of the Mamba model when processing two-dimensional visual data through a multi-directional state space scanning and recovery mechanism. Through transpose, flatten, and invert operations, it realizes the feature combination and scanning of the template and search region in four directions. Then, through inverse operations, it restores the original order and performs normalization processing, ensuring spatial consistency and stability in the feature extraction process. This not only enhances the model's adaptability to changes in the appearance of the target, but also improves the tracking robustness of complex scenes such as occlusion and deformation.

[0024] 3. This invention further enhances the model's ability to perceive and interact with local details by deeply integrating the attention mechanism with Mamba features; it introduces a multi-head self-attention module into the deep network, using the features output by the Mamba block as query, key, and value vectors for cross-regional interaction, thereby achieving fine-grained differentiation between the target and the background; combined with the residual MLP structure and the multi-loss joint training strategy, the model exhibits excellent performance in classification, localization, and bounding box regression tasks, ultimately achieving high-precision and robust real-time target tracking.

[0025] Additional aspects and advantages of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by means of embodiments of the invention. Attached Figure Description

[0026] Figure 1 This is a flowchart illustrating the steps of the target tracking method based on a hybrid network of Mamba and attention mechanisms proposed in this invention.

[0027] Figure 2 This is a schematic diagram illustrating the principle of multi-directional state space scanning for a target tracking method based on a hybrid network of Mamba and attention mechanisms.

[0028] Figure 3 This is a structural diagram of the target tracking framework of the target tracking method based on a hybrid network of Mamba and attention mechanism proposed in this invention.

[0029] Figure 4 This is a schematic diagram of the target tracking system based on a hybrid network of Mamba and attention mechanisms proposed in this invention. Detailed Implementation

[0030] Embodiments of the present invention are described in detail below. Examples of these embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and should not be construed as limiting the present invention.

[0031] These and other aspects of the embodiments of the present invention will become clear from the following description and accompanying drawings. In these descriptions and drawings, some specific embodiments of the present invention are specifically disclosed to provide some ways of implementing the principles of the embodiments of the present invention; however, it should be understood that the scope of the embodiments of the present invention is not limited thereto.

[0032] Please see Figure 1 This invention proposes a target tracking method based on a hybrid network of Mamba and attention mechanisms, which includes the following steps:

[0033] Step 1: Construct a tracking model based on a hybrid network of Mamba and attention mechanisms and a prediction head.

[0034] The hybrid network of Mamba and attention mechanisms includes Mamba blocks and attention modules. The Mamba block consists of linear layers, equally weighted depthwise convolutions, a state-space model, and residual MLP layers.

[0035] Step 2: Pre-train and tune the parameters of the hybrid Mamba and attention mechanism network using a large-scale dataset to obtain a parameter-tuned hybrid Mamba and attention mechanism network.

[0036] Furthermore, this invention is performed on an RTX 5080 GPU, with PyTorch version 2.7.1. The network is trained simultaneously for 400 epochs using the TrackingNet, GOT-10k, LaSOT, and COCO datasets. Image pairs from video sequences are randomly selected, and the target bounding boxes are expanded by 4x and 2x respectively to obtain the search region image and template region image. The network is trained using the AdamW optimizer with a weight decay of 1e-4. After 240 epochs, the learning rate is reduced to 10% of the initial value to facilitate rapid model convergence.

[0037] Step 3: Concatenate the template image and the search region image along their length to obtain the concatenated result. Input the concatenated result into the Mamba block of the parameter-adjusted Mamba and attention mechanism hybrid network. After linear transformation and partitioning the template and search region, calculate and reconstruct the scanning sequence in different directions through equal-weight depthwise convolution to perform multi-directional state space scanning using the state space model to obtain the scanning result. After restoring the original spatial relationship of the scanning result, perform aggregation and normalization processing to fuse multi-directional features and enhance network stability. Then process it through linear layers and residual MLP layers to obtain the Mamba block output.

[0038] The attention module in the hybrid network of Mamba and attention mechanism, which uses parameter adjustment, performs self-attention calculation through a multi-head self-attention mechanism on the output of the Mamba block, realizing feature interaction and fusion between the template image and the search region image, and obtaining the fused global features.

[0039] Please see Figure 2 and Figure 3In step 3, the splicing result is input into the Mamba block of the hybrid network of parameter-adjusted Mamba and attention mechanism. After linear transformation and partitioning of template and search region, it is calculated and reconstructed into scanning sequences in different directions through equal-weight depth convolution to perform multi-directional state space scanning using the state space model, thus obtaining the scanning result. After restoring the original spatial relationship of the scanning result, it is aggregated and normalized to fuse multi-directional features and enhance network stability. Then, it is processed by linear layers and residual MLP layers to obtain the Mamba block output. Specifically, it includes the following steps:

[0040] The splicing result is fed into the linear layer of the Mamba block in the parameter-adjusted Mamba and attention mechanism hybrid network to obtain the result of the first linear layer;

[0041] The results of the first linear layer are divided according to the length of the template region and the search region to obtain the template region and the search region after length division. The template region and the search region after length division are calculated by depthwise convolution with the same weight to obtain the convolution calculation results.

[0042] The convolution calculation results are spliced ​​in four different directions through transpose, flatten, and invert operations to obtain the spliced ​​results of the template region and the search region in four different directions.

[0043] The splicing results of the template region and the search region in four different directions are respectively fed into the state space model of the Mamba block for multi-directional state space scanning processing to obtain the results after four state space scans.

[0044] The results of the four state space scans are restored to their original order through transpose, flatten and invert operations to obtain four scan results after restoration. The four scan results after restoration are added together and normalized to obtain the normalized result.

[0045] The normalized result is input into the linear layer for processing to obtain the result of the second linear layer.

[0046] The results from the second linear layer are input into the residual MLP layer for processing, resulting in the Mamba block output.

[0047] The template region and the search region after length division are calculated using depthwise convolution with the same weights to obtain the convolution calculation results. The corresponding relationship in the process is as follows:

[0048] ;

[0049] in, This represents the result of the convolution calculation. This represents depthwise convolution calculation. This indicates a splicing operation. This represents the template region after length division. This indicates the search area after length division.

[0050] The convolution calculation results are concatenated in four different directions through transpose, flattening, and inversion operations to obtain the concatenated results of the template region and the search region in four different directions. The corresponding relationship in the process is as follows:

[0051] ;

[0052] in, , , , These represent the results of concatenating the template region with four search regions in different directions. , , , These represent the search areas in four different directions.

[0053] It should be noted that, to better adapt to the needs of tracking tasks, unlike other vision tasks, this invention proposes to preserve the order of the template region and combine the order of the search region in different directions, stitching the two together to achieve multi-directional scanning. This scanning method effectively preserves the original features of the template region and proposes a new approach for fusing template region features with features from search regions in different directions.

[0054] The concatenation results of the template region and the search region in four different directions are fed into the state space model of the Mamba block for multi-directional state space scanning, resulting in the four state space scan results. The corresponding relationship in the process is as follows:

[0055] ;

[0056] in, , , , These represent the results after scanning four different state spaces. This represents a state-space model.

[0057] It should be noted that in this invention, by using different combinations of template regions and search regions in different orders, the state space model can enhance feature representation by fusing features of template regions and search regions in different directions.

[0058] The results of the four state space scans are restored to their original order through transpose, flatten, and inversion operations, resulting in four restored scan results. These four restored scan results are then added together and normalized to obtain a normalized result. The corresponding relationships in this process are as follows:

[0059] ;

[0060] in, , , , These represent the scan results after four recovery sequences. This means taking the second half of the sequence with a length equal to the template region length and restoring its original order for processing;

[0061] It should be noted that in this invention, in order to better integrate the scanning results from different directions, the order of the latter half of the search area is restored so that the features of the search areas at corresponding positions can be correctly integrated.

[0062] In the step of adding the scan results after the four recovery sequences and performing a normalization operation to obtain the normalized result, the corresponding relationship in the process is as follows:

[0063] ;

[0064] in, This represents the normalization result. This indicates a normalization operation.

[0065] It should be noted that the purpose of using multi-directional state space scanning in this embodiment is to achieve the fusion of features of the template region and the search region from multiple directions by scanning the search template in multiple directions, and then to achieve the fusion of features from multiple directions by adding the results of scanning in multiple directions. Finally, a layer is used to avoid gradient explosion and enhance network stability.

[0066] The attention module in the hybrid network of Mamba and attention mechanisms, which utilizes parameter-tuned Mamba inputs for Mamba block outputs, performs self-attention computation through a multi-head self-attention mechanism, specifically including:

[0067] The template image and search region image in the Mamba block output are flattened in one dimension and then concatenated to obtain a one-dimensional concatenated vector of the template region and search region. This concatenated vector is then used as the query vector, key vector, and value vector for self-attention calculation to obtain the fused global features. The corresponding relationship in this process is as follows:

[0068] ;

[0069] in, This represents the result of attention calculation. Represents the normalized exponential activation function. Represents the query vector. Represents the transpose of the key vector , Indicates the scaling factor. The dimension of the key vector. Represents a value vector.

[0070] Furthermore, the formula for calculating the state-space model is as follows:

[0071] ;

[0072] in, Represents the learnable hidden state The changing trend Represents a timestamp. This represents the state transition matrix, which describes how the state changes over time. An input matrix that represents how control inputs affect state changes. Indicates input, Indicates the output. This represents the output matrix that generates the output based on the current state.

[0073] Furthermore, to adapt to the needs of real-world scenarios, the scanning process is discretized, assuming a time scale of [missing information]. Using the zero-order preservation rule, we can obtain three discretized matrices, as shown in the following formula:

[0074] ;

[0075] in, This represents the state transition matrix, which describes how the state changes over time after discretization. Represents the matrix index. , These represent the state transition matrices describing how the state changes over time. How do control inputs affect the input matrix of state changes? In interval Upsampling, The input matrix represents how the control input affects the state change after discretization. Represents the identity matrix. This represents the output matrix generated based on the current state after discretization.

[0076] It should be noted that, when dealing with matrices , , After discretization, the calculation formula for the state-space model is updated as follows:

[0077] ;

[0078] in, Indicates the discretized value at time step The hidden state matrix, Indicates the discretized value at time step The hidden state matrix, Indicates the discretized value at time step The output, Indicates the discretized value at time step Input.

[0079] It should be noted that, in Figure 3 In this study, the Mamba and attention mechanism hybrid network is built within the tracking framework of a single-stream network. Mamba blocks are used in the shallow layers of the first to sixth layers of the Mamba and attention mechanism hybrid network, while self-attention is used in the deep layers of the sixth to twelfth layers to enhance local features.

[0080] Step 4: Input the fused global features into the prediction head to obtain the target's classification score and predicted bounding box in the search area; combine the target's classification score and predicted bounding box in the search area with the real labels given in the dataset to construct a loss function to optimize the tracking model, resulting in an optimized tracking model; use the optimized tracking model to track the target.

[0081] In step 4, a loss function is constructed to optimize the tracking model by combining the target's classification score in the search region with the predicted bounding box and the given true label in the dataset, including:

[0082] The first loss function is constructed by combining the target's classification score in the search region with the true label at the given target location in the dataset. The corresponding relationship in this process is as follows:

[0083] ;

[0084] in, Denotes the first loss function. The actual label representing the target location. Represents the logarithmic function. This represents the probability value of the target location. This represents the total number of samples. Indicates the sample index;

[0085] The second and third loss functions are constructed using the predicted bounding boxes and the ground truth labels, respectively. The corresponding relationship in the process is as follows:

[0086] ;

[0087] in, This represents the second loss function. This represents the bounding box reference window in the actual label. This indicates the coordinates of the center position of the prediction window within the prediction bounding box. This represents the third loss function. These represent the x and y coordinates of the center position of the reference window in the actual label, respectively. These represent the x-coordinate and y-coordinate of the center position of the prediction window within the prediction bounding box, respectively.

[0088] The tracking model is jointly optimized using the first loss function, the second loss function, and the third loss function.

[0089] Furthermore, by jointly optimizing the tracking model using the first, second, and third loss functions, a total loss function also exists, with the following relationship:

[0090] ;

[0091] in, Represents the total loss function. and These represent the weight hyperparameters corresponding to the second loss function and the weight hyperparameters corresponding to the third loss function, respectively.

[0092] Please see Figure 4 The present invention also proposes a target tracking system based on a hybrid network of Mamba and attention mechanisms, the system comprising:

[0093] Build modules are used for:

[0094] The tracking model is based on a hybrid network of Mamba and attention mechanisms and a prediction head.

[0095] Pre-trained modules are used for:

[0096] We pre-trained and tuned the parameters of a hybrid Mamba and attention mechanism network using a large-scale dataset to obtain a parameter-tuned hybrid Mamba and attention mechanism network.

[0097] The calculation module is used for:

[0098] The template image and the search region image are concatenated along their length to obtain the concatenated result. The concatenated result is then input into a Mamba block of a hybrid network of parameter-adjusted Mamba and attention mechanisms. After linear transformation and partitioning into template and search regions, the network is calculated and reconstructed into scanning sequences in different directions using equal-weight depthwise convolutions to perform multi-directional state space scanning using a state space model, thus obtaining the scanning result. The scanning result is then processed to restore the original spatial relationships, followed by aggregation and normalization to fuse multi-directional features and enhance network stability. Finally, it is processed through linear layers and residual MLP layers to obtain the output of the Mamba block.

[0099] The output of the Mamba block utilizes the attention module in the hybrid network of parameter-adjusted Mamba and attention mechanisms to perform self-attention calculation through a multi-head self-attention mechanism, thereby realizing feature interaction and fusion between the template image and the search region image to obtain the fused global features.

[0100] The tracking module is used for:

[0101] The fused global features are input into the prediction head to obtain the target's classification score and predicted bounding box in the search region. The target's classification score and predicted bounding box in the search region are combined with the real labels given in the dataset to construct a loss function to optimize the tracking model, resulting in an optimized tracking model. The optimized tracking model is then used for target tracking.

[0102] It should be understood that various parts of the present invention can be implemented in hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented in software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0103] In the description of this specification, references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0104] The embodiments described above are merely illustrative of several implementations of the present invention, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of the present invention. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these modifications and improvements all fall within the scope of protection of the present invention. Therefore, the scope of protection of this patent should be determined by the appended claims.

Claims

1. A target tracking method based on a hybrid network of Mamba and attention mechanisms, characterized in that, The method includes the following steps: Step 1: Construct a tracking model based on a hybrid network using Mamba and attention mechanisms and a prediction head; Step 2: Pre-train and tune the parameters of the hybrid Mamba and attention mechanism network using a large-scale dataset to obtain a parameter-tuned hybrid Mamba and attention mechanism network; Step 3: Concatenate the template image and the search region image along their length to obtain the concatenated result. Input the concatenated result into the Mamba block of the hybrid network of parameter-adjusted Mamba and attention mechanism. After linear transformation and partitioning into template and search regions, calculate and reconstruct scanning sequences in different directions using equal-weight depthwise convolutions to perform multi-directional state space scanning using the state space model to obtain the scanning result. After restoring the original spatial relationship of the scanning result, perform aggregation and normalization processing to fuse multi-directional features and enhance network stability. Then process through linear layers and residual MLP layers to obtain the Mamba block output. The specific steps include the following: The splicing result is fed into the linear layer of the Mamba block in the parameter-adjusted Mamba and attention mechanism hybrid network to obtain the result of the first linear layer; The results of the first linear layer are divided according to the length of the template region and the search region to obtain the template region and the search region after length division. The template region and the search region after length division are calculated by depthwise convolution with the same weight to obtain the convolution calculation results. The convolution calculation results are spliced ​​in four different directions through transpose, flatten, and invert operations to obtain the spliced ​​results of the template region and the search region in four different directions. The splicing results of the template region and the search region in four different directions are respectively fed into the state space model of the Mamba block for multi-directional state space scanning processing to obtain the results after four state space scans. The results of the four state space scans are restored to their original order through transpose, flatten and invert operations to obtain four scan results after restoration. The four scan results after restoration are added together and normalized to obtain the normalized result. The normalized result is input into the linear layer for processing to obtain the result of the second linear layer. The results from the second linear layer are input into the residual MLP layer for processing to obtain the Mamba block output. The attention module in the hybrid network of Mamba and attention mechanism, which uses parameter-adjusted Mamba and attention mechanisms, performs self-attention calculation through a multi-head self-attention mechanism to realize feature interaction and fusion between the template image and the search region image, and obtain the fused global features. Step 4: Input the fused global features into the prediction head to obtain the target's classification score and predicted bounding box in the search area; combine the target's classification score and predicted bounding box in the search area with the real labels given in the dataset to construct a loss function to optimize the tracking model, resulting in an optimized tracking model; use the optimized tracking model to track the target.

2. The target tracking method based on a hybrid network of Mamba and attention mechanisms according to claim 1, characterized in that, The template region and the search region after length division are calculated using depthwise convolution with the same weight to obtain the convolution calculation result. The corresponding relationship in the process is as follows: ； in, This represents the result of the convolution calculation. This represents depthwise convolution calculation. This indicates a splicing operation. This represents the template region after length division. This indicates the search area after length division.

3. The target tracking method based on a hybrid network of Mamba and attention mechanisms according to claim 2, characterized in that, The convolution calculation results are then concatenated in four different directions through transpose, flattening, and inversion operations to obtain the concatenated template region and search region in four different directions. The corresponding relationships in this process are as follows: ； in, , , , These represent the results of concatenating the template region with four search regions in different directions. , , , These represent the search areas in four different directions.

4. The target tracking method based on a hybrid network of Mamba and attention mechanisms according to claim 3, characterized in that, The splicing results of the template region and the search region in four different directions are respectively fed into the state space model of the Mamba block for multi-directional state space scanning processing to obtain the results after four state space scans. The corresponding relationship in the process is as follows: ； in, , , , These represent the results after scanning four different state spaces. This represents a state-space model.

5. The target tracking method based on a hybrid network of Mamba and attention mechanisms according to claim 4, characterized in that, The results of scanning the four state spaces are restored to their original order through transpose, flatten, and inversion operations, resulting in four restored scan results. These four restored scan results are then added together and normalized to obtain a normalized result. The corresponding relationship in this process is as follows: ； in, , , , These represent the scan results after four recovery sequences. This means taking the second half of the sequence with a length equal to the template region length and restoring its original order for processing; In the step of adding the scan results after the four recovery sequences and performing a normalization operation to obtain the normalized result, the corresponding relationship in the process is as follows: ； in, This represents the normalization result. This indicates a normalization operation.

6. The target tracking method based on a hybrid network of Mamba and attention mechanisms according to claim 5, characterized in that, The attention module in the hybrid Mamba and attention mechanism network that utilizes parameter-adjusted Mamba block outputs performs self-attention calculation through a multi-head self-attention mechanism, specifically including: The template image and search region image in the Mamba block output are flattened in one dimension and then concatenated to obtain a one-dimensional concatenated vector of the template region and search region. This concatenated vector is then used as the query vector, key vector, and value vector for self-attention calculation to obtain the fused global features. The corresponding relationship in this process is as follows: ； in, This represents the result of attention calculation. Represents the normalized exponential activation function. Represents the query vector. This represents the transpose of the key vector. Indicates the scaling factor. The dimension of the key vector. Represents a value vector.

7. The target tracking method based on a hybrid network of Mamba and attention mechanisms according to claim 6, characterized in that, In step 4, a loss function is constructed to optimize the tracking model by combining the target's classification score in the search region with the predicted bounding box and the given true label in the dataset, including: The first loss function is constructed by combining the target's classification score in the search region with the true label at the given target location in the dataset. The corresponding relationship in this process is as follows: ； in, Denotes the first loss function. The actual label representing the target location. Represents the logarithmic function. This represents the probability value of the target location. This represents the total number of samples. Indicates the sample index; The second and third loss functions are constructed using the predicted bounding boxes and the ground truth labels, respectively. The corresponding relationship in the process is as follows: ； in, This represents the second loss function. This represents the bounding box reference window in the actual label. This indicates the coordinates of the center position of the prediction window within the prediction bounding box. This represents the third loss function. , These represent the x and y coordinates of the center position of the reference window in the actual label, respectively. , These represent the x-coordinate and y-coordinate of the center position of the prediction window within the prediction bounding box, respectively. The tracking model is jointly optimized using the first loss function, the second loss function, and the third loss function.

8. A target tracking system based on a hybrid network of Mamba and attention mechanisms, characterized in that, The system employs the target tracking method based on a hybrid network of Mamba and attention mechanisms as described in any one of claims 1-7, and the system comprises: Build modules are used for: The tracking model is based on a hybrid network of Mamba and attention mechanisms and a prediction head. Pre-trained modules are used for: We pre-trained and tuned the parameters of a hybrid Mamba and attention mechanism network using a large-scale dataset to obtain a parameter-tuned hybrid Mamba and attention mechanism network. The calculation module is used for: The template image and the search region image are concatenated along their length to obtain the concatenated result. The concatenated result is then input into a Mamba block of a hybrid network of parameter-adjusted Mamba and attention mechanisms. After linear transformation and partitioning into template and search regions, the network is calculated and reconstructed into scanning sequences in different directions using equal-weight depthwise convolutions to perform multi-directional state space scanning using a state space model, thus obtaining the scanning result. The scanning result is then processed to restore the original spatial relationships, followed by aggregation and normalization to fuse multi-directional features and enhance network stability. Finally, it is processed through linear layers and residual MLP layers to obtain the output of the Mamba block. The output of the Mamba block utilizes the attention module in the hybrid network of parameter-adjusted Mamba and attention mechanisms to perform self-attention calculation through a multi-head self-attention mechanism, thereby realizing feature interaction and fusion between the template image and the search region image to obtain the fused global features. The tracking module is used for: The fused global features are input into the prediction head to obtain the target's classification score and predicted bounding box in the search region. The target's classification score and predicted bounding box in the search region are combined with the real labels given in the dataset to construct a loss function to optimize the tracking model, resulting in an optimized tracking model. The optimized tracking model is then used for target tracking.

Images