A Neural Network Construction Method for Object Tracking Based on Mamba

Abstract

The present invention discloses a method for constructing a neural network for object tracking based on Mamba, belonging to the technical field of object tracking. For object tracking, it includes preparing an object dataset, constructing a neural network based on Mamba, performing neural network training, verifying the error of the neural network after training. If the error is less than the expected value, the test set data is processed and the object tracking result is output. If the error is greater than or equal to the expected value, return to perform neural network training; The neural network based on Mamba includes a backbone layer Backbone, a neck layer Neck, and a head layer Head; The present invention improves the adaptability and accuracy for objects with complex shapes and pose changes, better serializes the processing of features, and captures long-range dependencies; It realizes the simultaneous capture of local details (deformable convolution) and global (Mamba) information, and is suitable for object tracking tasks in complex scenarios.

Description

Technical Field

[0001] The present invention discloses a method for constructing a neural network based on Mamba for target tracking, belonging to the technical field of target tracking. Background Art

[0002] Target tracking in remote sensing images faces three core problems: First, the low resolution and complex background interference lead to a weak target-environment contrast; second, the unstable feature representation caused by multi-scale targets and cloud occlusion; third, the significant intra-class differences caused by sensor noise and illumination variation. Traditional target tracking methods usually rely on pre-defined feature models (such as classification models based on edge, texture, or spectral features), and need to adjust parameters according to specific datasets. However, such models have poor adaptability to complex scenes, and the parameter optimization process requires a large amount of manual intervention, with low automation, especially limited efficiency when dealing with massive remote sensing data. In addition, traditional methods are difficult to simultaneously consider the overall morphological features of the target (such as building outlines, road directions) and local detail features (such as texture, shadow changes) during target tracking, resulting in limited accuracy and integrity of target recognition. Summary of the Invention

[0003] The purpose of the present invention is to provide a method for constructing a neural network based on Mamba for target tracking to solve the problem of insufficient accuracy of the neural network for target tracking in the prior art.

[0004] A method for constructing a neural network based on Mamba for target tracking, preparing a target dataset, constructing a neural network based on Mamba, training the neural network, verifying the neural network error after training, if the error is less than the expected value, then processing the test set data and outputting the target tracking result, if the error is greater than or equal to the expected value, then returning to train the neural network;

[0005] The neural network based on Mamba includes a backbone layer Backbone, a neck layer Neck, and a head layer Head; the Backbone sequentially includes a deformable convolution parallel multi-scale global local scanning Mamba module CM and 4 stage layers, the first 3 stage layers include CM and a deformable cross-stage partial layer Deform_CSPLayer with add=Ture, and the 4th stage layer includes CM, a Deform_CSPLayer with add=Ture, and a spatial pyramid pooling layer SPPF;

[0006] The Neck includes upsampling the output of the SPPF, performing feature fusion with the output of the 3rd stage layer, inputting it into the Deform_CSPLayer with add=False, then performing upsampling again, performing feature fusion with the output of the 2nd stage layer, inputting it into the Deform_CSPLayer with add=False again, then inputting it into the CM, performing feature fusion with the output of the first Deform_CSPLayer with add=False, inputting it into the Deform_CSPLayer with add=False again, then inputting it into the CM, performing feature fusion with the output of the SPPF, and finally inputting it into the Deform_CSPLayer with add=False;

[0007] The Head includes taking the outputs of the 2nd, 3rd, and 4th Deform_CSPLayers with add=False in the Neck as three input branches. Each input branch feeds the input features into two branches simultaneously. The first branch sequentially includes the CM, a 2D convolutional layer Conv2d, and a bounding box loss Bbox.Loss. The second branch sequentially includes the CM, a 2D convolutional layer Conv2d, and a classification loss Cls.Loss.

[0008] The Deform_CSPLayer sequentially includes a 1×1 convolutional layer, a split layer, multiple Bottleneck layers, a concatenation layer, and a 1×1 convolutional layer. The output feature of the 1×1 convolutional layer and the split layer and the output feature of the first Bottleneck layer have a shortcut connection to the concatenation layer. A convolutional layer that halves the number of channels of the feature is provided in the shortcut connection;

[0009] When add=Ture, the Bottleneck includes two CM in series. The input feature of the Bottleneck is fused with the output feature of the last CM. When add=False, the Bottleneck includes two CM in series.

[0010] The input of the CM is X_0. The CM includes a 1×1 convolutional layer. The output feature of the 1×1 convolutional layer is X. X is divided into two branches. One branch is input into a deformable convolution module and then outputs Y_1. One branch is input into a ME-Mamba module and then outputs Y_2. Y_1 and Y_2 are input into an adaptive parameter adjustment module together and then output through a 1×1 convolutional layer.

[0011] The adaptive parameter adjustment module includes inputting \(Y_1\) and \(Y_2\) into the shared feature encoder and the private feature encoder simultaneously. The output of \(Y_1\) in the shared feature encoder is \(S_1\), the output of \(Y_2\) in the shared feature encoder is \(S_2\), the output of \(Y_1\) in the private feature encoder is \(O_1\), and the output of \(Y_2\) in the private feature encoder is \(O_2\). Input \(S_1\) and \(S_2\) into the shared feature aggregation module, input \(O_1\) and \(O_2\) into the private feature retention module, and finally send the output values of the shared feature aggregation module and the private feature retention module into the feature enhancement interaction module, and then output.

[0012] Use the cross-channel contrast loss function in the shared feature encoder and the private feature encoder :

[0013] ;

[0014] ;

[0015] In the formula, is the similarity metric function, and take values of 1 or 2, and are two independent variables in the similarity metric function, is the number of channels;

[0016] Use the weighted mean squared error loss to measure the feature reconstruction quality:

[0017] ;

[0018] In the formula, and are the reconstructed and , , is the channel dimension;

[0019] In the feature enhancement interaction module, take the mean of the encoder outputs:

[0020] ;

[0021] ;

[0022] In the formula, is the shared feature mean, is the private feature mean;

[0023] Perform feature fusion:

[0024] ;

[0025] In the formula, is the Sigmoid activation function, is the fused feature, represents and to perform cross-modal feature fusion.

[0026] The deformable convolution module sequentially includes a 1×1 convolutional layer, two cascaded deformable convolutional layers, a feature fusion layer, and a 1×1 convolutional layer. The output of the 1×1 convolutional layer has three branches. One branch is input into the first deformable convolutional layer, one branch is input into a 3×3 convolutional layer, and after DDP pooling, it is input into the first deformable convolutional layer. One branch is input into the feature fusion layer. The output of the first deformable convolutional layer has two branches. One branch is input into the second deformable convolutional layer, and one branch is input into a 3×3 convolutional layer, and after DDP pooling, it is input into the second deformable convolutional layer.

[0027] The ME-Mamba module sequentially includes a Layer Normalization layer, a LinearProjection layer, a Discrete Wavelet Transform 2D Convolution module (DWT2DConv), a Mish activation function layer, a Multi-Scale Global Local 2D Selective Scanning module (MGLS2D), a Layer Normalization layer, a first feature fusion layer, a Linear Projection layer, and a second feature fusion layer;

[0028] The output of the first Layer Normalization has a branch that connects to the Linear Projection and Mish and then is input into the first feature fusion layer. The input of the ME-Mamba module has a shortcut connection that connects to the second feature fusion layer.

[0029] The DWT2Dconv sequentially includes a combined layer of batch normalization-convolution-ReLU activation function (Conv-BN-ReLU), a Discrete Wavelet Transform 2D layer (DWT2D), Conv-BN-ReLU, a first feature fusion layer, an Inverse Discrete Wavelet Transform 2D layer (IDWT2D), a second feature fusion layer, and a 1×1 convolutional layer;

[0030] The output of the DWT2D has a branch that sequentially connects to the DWT2D, Conv-BN-ReLU, and IDWT2D, and then is input into the first feature fusion layer. The output of the first Conv-BN-ReLU has a branch that connects to the second feature fusion layer.

[0031] MGLS2D includes two branches. The first branch inputs the input features into the multi-scale global-local scanning strategy layer MGLS to generate features T1 and T2, and then inputs T1 and T2 into the selective scanning spatial state sequence module S6. The output of S6 is sent to the merging layer Merge. The first branch inputs the input features into the depthwise separable convolutional layer DWconv, and then inputs them into the multi-scale global-local scanning strategy layer MGLS to generate features T3 and T4. Then, T3 and T4 are input into the selective scanning spatial state sequence module S6, and then input into the upsampling interpolation layer Interprolate, and finally sent to Merge;

[0032] In MGLS, the input feature image is scanned in a zigzag pattern globally to obtain continuous global image information. MGLS includes two scanning paths. The first scanning path scans the local image patches in an S-shaped pattern, and the second scanning path scans the local image patches in a zigzag pattern; two different scanning path processing information of large-scale remote sensing images are obtained, and then the input feature map is passed through another depthwise separable convolution layer, and the zigzag scan is used in different directions to obtain small-scale feature images.

[0033] Neural network training includes initializing model parameters, inputting the original remote sensing image and the corresponding remote sensing image class label data training set into the neural network for iteration, and saving the model parameters;

[0034] When performing target tracking, the remote sensing images in the test set are put into the trained neural network, and the target tracking results of the remote sensing images are output.

[0035] Compared with the prior art, the present invention has the following beneficial effects: The present invention improves the adaptability and accuracy for targets with complex shapes and pose changes, better serializes the processing of features, and captures long-range dependencies; it realizes the simultaneous capture of local details (deformable convolution) and global (mamba) information, and is suitable for target tracking tasks in complex scenarios. Description of the Drawings

[0036] Figure 1 The overall structure diagram of the present invention is shown;

[0037] Figure 2 The structure diagram of the Deform CSP Layer module of the present invention is shown;

[0038] Figure 3 The basic structure diagram of the CM module of the present invention is shown;

[0039] Figure 4 The structure diagram of the adaptive parameter adjustment module of the present invention is shown;

[0040] Figure 5The figure shows the structural diagram of the deformable convolution module according to the present invention;

[0041] Figure 6 The figure shows the structural diagram of the multi-scale global-local scanning mamba module according to the present invention;

[0042] Figure 7 The figure shows the structural diagram of the discrete wavelet convolution module according to the present invention;

[0043] Figure 8 The figure shows the structural diagram of MGLS2D according to the present invention;

[0044] Figure 9 The figure shows the overall flowchart of the remote sensing image target tracking method according to the present invention;

[0045] Figure 10 The figure shows the flowchart of the training steps of the present invention;

[0046] Figure 11 The figure shows the flowchart of the remote sensing image target tracking steps according to the present invention. Detailed implementation manners

[0047] To make the objectives, technical solutions, and advantages of the present invention clearer, the technical solutions in the present invention will be clearly and completely described below. Obviously, the described embodiments are some, but not all, of the embodiments of the present invention. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without making creative efforts shall fall within the protection scope of the present invention.

[0048] A method for constructing a neural network for target tracking based on Mamba, as Figure 9 shown, prepare a target data set, construct a neural network based on Mamba, perform neural network training, verify the neural network error after training is completed. If the error is less than the expected value, process the test set data and output the target tracking result. If the error is greater than or equal to the expected value, return to perform neural network training;

[0049] The neural network based on Mamba is as Figure 1 shown, and includes a backbone layer Backbone, a neck layer Neck, and a head layer Head; the Backbone sequentially includes a deformable convolution parallel multi-scale global-local scanning mamba module CM and 4 stage layers. The first 3 stage layers include CM and a deformable cross-stage partial layer Deform_CSPLayer with add = Ture. The 4th stage layer includes CM, a Deform_CSPLayer with add = Ture, and a spatial pyramid pooling layer SPPF;

[0050] The Neck includes upsampling the output of the SPPF through Upsample, fusing features with the output of the 3rd stage layer, inputting to the Deform_CSPLayer with add=False, then performing Upsample again, fusing features with the output of the 2nd stage layer, inputting to the Deform_CSPLayer with add=False again, then inputting to the CM, fusing features with the output of the first Deform_CSPLayer with add=False, inputting to the Deform_CSPLayer with add=False again, then inputting to the CM, fusing features with the output of the SPPF, and finally inputting to the Deform_CSPLayer with add=False;

[0051] The Head includes taking the outputs of the 2nd, 3rd, and 4th Deform_CSPLayers with add=False in the Neck as three input branches. Each input branch feeds the input features into two branches simultaneously. The first branch sequentially includes the CM, a 2D convolutional layer Conv2d, and a bounding box loss Bbox.Loss. The second branch sequentially includes the CM, a 2D convolutional layer Conv2d, and a classification loss Cls.Loss.

[0052] The Deform_CSPLayer is as follows Figure 2 shown, and sequentially includes a 1×1 convolutional layer, a split layer, multiple Bottleneck layers, a concatenation layer, a 1×1 convolutional layer. For the 1×1 convolutional layer and the split layer, the output features of the first Bottleneck have a shortcut connection to the concatenation layer, and there is a convolutional layer in the shortcut connection that halves the number of channels of the features;

[0053] When add=Ture, the Bottleneck includes two CM in series, and the input features of the Bottleneck are fused with the output features of the last CM. When add=False, the Bottleneck includes two CM in series.

[0054] The CM is as follows Figure 3 shown. The input of the CM is X_0. The CM includes a 1×1 convolutional layer. The output features of the 1×1 convolutional layer are X. X is divided into two branches. One branch is input to the deformable convolution module and then outputs Y_1. One branch is input to the ME-Mamba module and then outputs Y_2. Y_1 and Y_2 are input to the adaptive parameter adjustment module together, and then output through a 1×1 convolutional layer.

[0055] The said adaptive parameter adjustment module is as follows Figure 4As shown in the figure, it includes simultaneously inputting Y_1 and Y_2 into the shared feature encoder and the private feature encoder. The output of Y_1 in the shared feature encoder is S_1, the output of Y_2 in the shared feature encoder is S_2, the output of Y_1 in the private feature encoder is O_1, and the output of Y_2 in the private feature encoder is O_2. Input S_1 and S_2 into the shared feature aggregation module, input O_1 and O_2 into the private feature retention module, and finally send the output values of the shared feature aggregation module and the private feature retention module to the feature enhancement interaction module, and then output.

[0056] Use the cross-channel contrast loss function in the shared feature encoder and the private feature encoder :

[0057] ;

[0058] ;

[0059] In the formula, is the similarity measurement function, and take values of 1 or 2, and are two independent variables in the similarity measurement function, is the number of channels;

[0060] Use the weighted mean squared error loss to measure the feature reconstruction quality:

[0061] ;

[0062] In the formula, and are the reconstructed and , , is the channel dimension;

[0063] In the feature enhancement interaction module, calculate the mean of the encoder outputs:

[0064] ;

[0065] ;

[0066] In the formula, is the mean of the shared features, is the mean of the private features;

[0067] Perform feature fusion:

[0068] ;

[0069] In the formula, is the Sigmoid activation function, is the fused feature, represents and for cross-modal feature fusion.

[0070] The deformable convolution module is as shown in Figure 5 and successively includes a 1×1 convolution layer, two cascaded deformable convolution layers, a feature fusion layer, and a 1×1 convolution layer. The output of the 1×1 convolution layer has three branches, one branch is input into the first deformable convolution layer, one branch is input into a 3×3 convolution layer and after DDP pooling is input into the first deformable convolution layer, and one branch is input into the feature fusion layer. The output of the first deformable convolution layer has two branches, one branch is input into the second deformable convolution layer, and one branch is input into a 3×3 convolution layer and after DDP pooling is input into the second deformable convolution layer.

[0071] The ME-Mamba module is as shown in Figure 6 and successively includes a layer normalization layer Layer Normalization, a linear projection layer Linear Projection, a discrete wavelet transform convolution module DWT2DConv, an activation function layer Mish, a multi-scale global local two-dimensional selective scanning module MGLS2D, Layer Normalization, a first feature fusion layer, LinearProjection, and a second feature fusion layer;

[0072] The output of the first Layer Normalization has a branch that connects to Linear Projection and Mish and then is input into the first feature fusion layer. The input of the ME-Mamba module has a shortcut connection that is connected to the second feature fusion layer.

[0073] DWT2Dconv is as shown in Figure 7 and successively includes a combined layer of batch normalization-convolution-ReLU activation function Conv-BN-ReLU, a discrete wavelet transform layer DWT2D, Conv-BN-ReLU, a first feature fusion layer, an inverse discrete wavelet transform layer IDWT2D, a second feature fusion layer, and a 1×1 convolution layer;

[0074] The output of DWT2D has a branch that successively connects to DWT2D, Conv-BN-ReLU, and IDWT2D, and then is input into the first feature fusion layer. The output of the first Conv-BN-ReLU has a branch that is connected to the second feature fusion layer.

[0075] MGLS2D is as shown in Figure 8As shown in the figure, it includes two branches. The first branch inputs the input features into the multi-scale global-local scanning strategy layer MGLS, generating features T1 and T2. Then, T1 and T2 are input into the selective scanning spatial state sequence module S6, and the output of S6 is sent to the merging layer Merge. The first branch inputs the input features into the depthwise separable convolution layer DWconv, and then into the multi-scale global-local scanning strategy layer MGLS, generating features T3 and T4. Then, T3 and T4 are input into the selective scanning spatial state sequence module S6, and then into the upsampling interpolation layer Interprolate, and finally sent to Merge;

[0076] In MGLS, the input feature image is scanned in a zigzag pattern globally to obtain continuous global image information. MGLS includes two scanning paths. The first scanning path scans the local image patches in an S-shaped pattern, and the second scanning path scans the local image patches in a zigzag pattern; to obtain the processing information of two different scanning paths for large-scale remote sensing images. Then, the input feature map is passed through another depthwise separable convolution layer, and the small-scale feature images are obtained by using the zigzag scan in different directions.

[0077] As Figure 10 shown, the neural network training includes initializing the model parameters, inputting the original remote sensing images and the corresponding remote sensing image class label data training set into the neural network for iteration, and saving the model parameters;

[0078] As Figure 11 shown, when performing target tracking, the remote sensing images in the test set are put into the trained neural network, and the target tracking results of the remote sensing images are output.

[0079] MWORKS is a new-generation scientific computing and system modeling and simulation platform. Through the unified modeling language and high-performance solving engine, it provides full-life-cycle support for the research, development, simulation, verification, and optimization of complex systems. In the field of industrial software, traditional tools (such as MATLAB / Simulink, ANSYS, etc.) have long been monopolized by foreign countries, suffering from problems such as insufficient multi-domain collaborative modeling capabilities, low simulation efficiency, complex code generation and hardware adaptation, low model reuse rate, and limited intelligence level, making it difficult to meet the requirements for efficient, independent, and controllable complex system design in digital transformation. MWORKS addresses these pain points by building a multi-disciplinary unified modeling environment based on the Modelica language and its own kernel, supporting the deep integration of fields such as machinery, electricity, and control, achieving large-scale simulation acceleration through a high-performance solving engine, and combining AI technology to enhance parameter optimization and automated modeling capabilities. At the same time, its scientific computing environment (MWORKS.Syslab) is compatible with languages such as Python and Julia, providing algorithm development and data analysis support, while the industrial knowledge model platform (MoHub) promotes the cloud sharing and collaborative innovation of models. This platform not only realizes the domestic substitution of foreign industrial software but also significantly improves the efficiency and reliability of complex system design through its intelligent, open, and cloud collaborative capabilities, especially promoting the digital and intelligent transformation of equipment R & D in fields such as aerospace, automotive, and energy.

[0080] The single-stage architecture of YOLOv8 mainly focuses on local feature extraction and is limited in tasks that require global context understanding (such as long-tail categories and complex scene reasoning). For example, when there are highly similar textures or colors between the target and the background, false detections or missed detections are likely to occur. In night-time monitoring or bad weather, due to its reliance on local features, YOLOv8's robustness decreases in scenarios where the target contour is blurred or partially occluded.

[0081] Mamba is an efficient sequence modeling architecture based on the state space model (SSM). By dynamically screening key information and recursively transmitting hidden states, it can effectively capture long-range dependencies and global information. To meet the requirements of vision tasks, the researchers proposed Visual Mamba. The main improvements include: introducing a cross-path scanning mechanism to unfold the two-dimensional image into a four-way sequence, solving the defect of traditional Mamba in spatial continuity modeling; designing a direction-sensitive state transition matrix to enhance the perception ability of local geometric features such as edge directions. However, compared with models such as YOLOv8 that are specifically optimized for local features, VMamba still has deficiencies in detailed modeling. For example, in the object tracking task, although the cross-path scanning mechanism alleviates the problem of pixel-level information loss, due to the linear time-invariant characteristics of the state space model, its selective calculation mechanism may over-smooth high-frequency textures, resulting in lower edge segmentation accuracy of small targets than convolutional models.

[0082] Specifically, YOLOv8 can precisely capture sub-pixel offsets of object boundaries through dense coverage of multi-scale convolutional kernels and feature pyramid fusion. However, under the advantage of long-range modeling, VMamba still has room for improvement in adapting to local deformations (such as broken contours and serrated edges).

[0083] Based on the MWORKS platform, the present invention proposes an innovative fusion architecture to process remote sensing image information by integrating the advantages of YOLOv8 and the Mamba model. This method innovatively proposes to replace the convolutional module in the traditional YOLOv8 backbone network with a parallel fusion structure of a deformable convolutional module and a multi-scale global-local scanning Mamba module to improve the adaptability and accuracy of the model for targets with complex shapes and pose changes. At the same time, it better serializes the processing of features and captures long-range dependencies. Finally, an adaptive parameter adjustment module is introduced to timely adjust the channel weight ratio of the deformable convolutional module and the multi-scale global-local scanning Mamba module to achieve simultaneous capture of local details and global information, suitable for target tracking tasks in complex scenarios, thus solving the problems in the prior art that remote sensing target tracking cannot balance local features and global features, resulting in poor small target tracking effect and weak generalization ability in complex scenarios.

[0084] The present invention needs to prepare three datasets: a training set, a validation set, and a test set, and the ratio of the three is approximately 3:1:1. Among them, the training set is responsible for training the target tracking model, the validation set is responsible for adjusting the target tracking model, and the test set is responsible for testing the target tracking effect of the model. To better demonstrate the good characteristics of the present invention in balancing local features and global features, the DOTA dataset, a relatively feasible target tracking dataset in the current remote sensing field, is used. This dataset has 15 category numbers and large-scale, multi-category, and high-complexity annotation data. In this dataset, more than 200 images are selected as the training set, and each remote sensing image corresponds to a piece of data after target tracking, and the latter is the label of the former. The construction methods of the validation set and the test set are the same as those of the dataset, but the remote sensing images and corresponding data in the validation set should be different from those in the dataset.

[0085] Input the remote sensing image data in the training set into the constructed network model. The output of the network is the data for target tracking using the network described in the present invention. Compare the target tracking results output by this network with the data that has been processed in the original dataset, and calculate the error between the two at the same time. If the error is greater than the set threshold (the size of the threshold is set according to the actual situation), the parameters are updated by backpropagation. Iterate repeatedly until the error is less than the set threshold, stop updating the parameters, and save the network parameters. The present invention uses the Adam optimization algorithm to solve the parameters, and updates the parameters according to the error gradient information during error backpropagation. The loss function is :

[0086] ;

[0087] Wherein, is the quantity of batch-processed data, represents the target tracking network model of the remote sensing image of the present invention, represents the target tracking label - original remote sensing image data training pair in the training set, are the parameters of the remote sensing image target tracking model of the present invention.

[0088] Input the remote sensing image data in the validation set into the constructed network model. The output of the network is the data after target tracking using the network of the present invention. Compare the data after target tracking with the original remote sensing image target category label, and calculate the error between the two at the same time. If the error is less than the set threshold (the size of the threshold is set according to the actual situation), it proves that the trained network parameters have reached the optimal, and the target tracking of the test set can be carried out. If the error is greater than the set threshold, it proves that the trained network parameters have not reached the optimal, and return to retrain the network until a good target tracking effect can be achieved in the validation stage.

[0089] Input the original remote sensing data in the test set into the network of the present invention that has been trained and verified. The output of the network is the data after target tracking using the network of the present invention.

[0090] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that: they can still modify the technical solutions recorded in the foregoing embodiments, or perform equivalent replacements on some or all of the technical features, and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for constructing a neural network for object tracking based on Mamba, characterized in that, Prepare the target dataset, construct a neural network based on Mamba, perform neural network training, verify the neural network error after training. If the error is less than the expected value, process the test set data and output the target tracking result. If the error is greater than or equal to the expected value, return to perform neural network training; The neural network based on Mamba includes a backbone layer Backbone, a neck layer Neck, and a head layer Head; Backbone sequentially includes a deformable convolutional parallel multi-scale global local scanning mamba module CM and 4 stage layers. The first 3 stage layers include CM and a deformable cross-stage partial layer Deform_CSPLayer with add=Ture. The 4th stage layer includes CM, a Deform_CSPLayer with add=Ture, and a spatial pyramid pooling layer SPPF; Neck includes upsampling the output of SPPF, performing feature fusion with the output of the 3rd stage layer, inputting it into a Deform_CSPLayer with add=False, then performing upsampling, performing feature fusion with the output of the 2nd stage layer, inputting it into a Deform_CSPLayer with add=False, then inputting it into CM, performing feature fusion with the output of the first Deform_CSPLayer with add=False, then inputting it into a Deform_CSPLayer with add=False, then inputting it into CM, performing feature fusion with the output of SPPF, and then inputting it into a Deform_CSPLayer with add=False; Head includes taking the outputs of the 2nd, 3rd, and 4th Deform_CSPLayers with add=False in Neck as three input branches. Each input branch feeds the input features into two branches simultaneously. The first branch sequentially includes CM, a two-dimensional convolutional layer Conv2d, and a bounding box loss Bbox.Loss. The second branch sequentially includes CM, a two-dimensional convolutional layer Conv2d, and a classification loss Cls.Loss.

2. The neural network construction method for target tracking based on Mamba according to claim 1, wherein The Deform_CSPLayer sequentially includes a 1×1 convolutional layer, a split layer, multiple bottleneck layers Bottleneck, a concatenation layer, a 1×1 convolutional layer. The output features of the 1×1 convolutional layer, the split layer, and the first Bottleneck are provided with a shortcut connection to the concatenation layer. A convolutional layer that halves the number of channels of the features is provided in the shortcut connection; When add=Ture, Bottleneck includes two serially connected CMs, and the input features of Bottleneck are fused with the output features of the last CM. When add=False, Bottleneck includes two serially connected CMs.

3. A neural network construction method for target tracking based on Mamba according to claim 2, characterized in that, The input of the CM is \(X_0\). The CM includes a 1×1 convolutional layer, and the output feature of the 1×1 convolutional layer is \(X\). \(X\) is divided into two branches. One branch is input into the deformable convolution module and then outputs \(Y_1\), and the other branch is input into the ME-Mamba module and then outputs \(Y_2\). \(Y_1\) and \(Y_2\) are input into the adaptive parameter adjustment module together, and then output through a 1×1 convolutional layer.

4. A method for constructing a neural network for object tracking based on Mamba according to claim 3, characterized in that, The adaptive parameter adjustment module includes inputting \(Y_1\) and \(Y_2\) into the shared feature encoder and the private feature encoder simultaneously. The output of \(Y_1\) in the shared feature encoder is \(S_1\), the output of \(Y_2\) in the shared feature encoder is \(S_2\), the output of \(Y_1\) in the private feature encoder is \(O_1\), and the output of \(Y_2\) in the private feature encoder is \(O_2\). \(S_1\) and \(S_2\) are input into the shared feature aggregation module, \(O_1\) and \(O_2\) are input into the private feature retention module. Finally, the output values of the shared feature aggregation module and the private feature retention module are sent to the feature enhancement interaction module, and then output.

5. A method for constructing a neural network for object tracking based on Mamba according to claim 4, characterized in that, Using a cross-channel contrastive loss function in the shared feature encoder and the private feature encoder : ； ； In the formula, is a similarity metric function, and take values of 1 or 2, and are two independent variables in the similarity metric function, is the number of channels; Use weighted mean squared error loss Measure the quality of feature reconstruction: ； In the formula, and are the and after reconstruction, , is the channel dimension; In the feature enhancement interaction module, the mean value of the output of the encoder is calculated: ； ； In the formula, is the shared feature mean value, is the private feature mean value; Feature fusion is performed: ； In the formula, is the Sigmoid activation function, is the fused feature, represents and to perform cross-modal feature fusion.

6. A method for constructing a neural network for target tracking based on Mamba according to claim 5, characterized in that, The deformable convolution module successively includes a 1×1 convolutional layer, two cascaded deformable convolutional layers, a feature fusion layer, and a 1×1 convolutional layer. The output of the 1×1 convolutional layer has three branches. One branch is input into the first deformable convolutional layer, one branch is input into a 3×3 convolutional layer and after DDP pooling is input into the first deformable convolutional layer, and one branch is input into the feature fusion layer. The output of the first deformable convolutional layer has two branches. One branch is input into the second deformable convolutional layer, and one branch is input into a 3×3 convolutional layer and after DDP pooling is input into the second deformable convolutional layer.

7. A method for constructing a neural network for object tracking based on Mamba according to claim 6, characterized in that, The ME-Mamba module successively includes a Layer Normalization layer, a LinearProjection layer, a Discrete Wavelet Transform 2D Convolution module (DWT2DConv), a Mish activation function layer, a Multi-scale Global Local 2D Selective Scanning module (MGLS2D), a Layer Normalization layer, a first feature fusion layer, a Linear Projection layer, and a second feature fusion layer; The output of the first Layer Normalization has a branch that connects to the Linear Projection, Mish and then inputs to the first feature fusion layer. The input of the ME-Mamba module has a shortcut connection that connects to the second feature fusion layer.

8. A method for constructing a neural network for object tracking based on Mamba according to claim 7, characterized in that, DWT2Dconv successively includes a combination layer of batch normalization-convolution-ReLU activation function (Conv-BN-ReLU), a Discrete Wavelet Transform 2D layer (DWT2D), Conv-BN-ReLU, a first feature fusion layer, an Inverse Discrete Wavelet Transform 2D layer (IDWT2D), a second feature fusion layer, and a 1×1 convolutional layer; The output of DWT2D has a branch that successively connects to DWT2D, Conv-BN-ReLU, IDWT2D, and then inputs to the first feature fusion layer. The output of the first Conv-BN-ReLU has a branch that connects to the second feature fusion layer.

9. A neural network construction method for object tracking based on Mamba according to claim 8, characterized in that, MGLS2D includes two branches. The first branch inputs the input features into the multi-scale global-local scanning strategy layer MGLS to generate features T1 and T2, and then inputs T1 and T2 into the selective scanning spatial state sequence module S6. The output of S6 is sent to the merging layer Merge. The first branch inputs the input features into the depthwise separable convolutional layer DWconv, and then inputs them into the multi-scale global-local scanning strategy layer MGLS to generate features T3 and T4. Then, T3 and T4 are input into the selective scanning spatial state sequence module S6, and then input into the upsampling interpolation layer Interprolate, and finally sent to Merge; In MGLS, the input feature image is scanned in a zigzag pattern globally to obtain continuous global image information; MGLS includes two scanning paths. The first scanning path scans the local image patches in an S-shaped pattern, and the second scanning path scans the local image patches in a zigzag pattern; two different scanning path processing information of large-scale remote sensing images are obtained, and then the input feature map is further passed through a depthwise separable convolution layer, and the zigzag scan is used from different directions to obtain small-scale feature images.

10. A method for constructing a neural network for object tracking based on Mamba according to claim 9, characterized in that, Neural network training includes initializing model parameters, inputting the original remote sensing images and the corresponding remote sensing image class label data training set into the neural network for iteration, and saving the model parameters; When performing target tracking, the remote sensing images in the test set are put into the trained neural network, and the target tracking results of the remote sensing images are output.

Images