Neural network construction method for target tracking based on Mama

Abstract

The invention discloses a Mama-based neural network construction method for target tracking, which belongs to the technical field of target tracking, is used for target tracking and comprises the following steps: preparing a target data set, constructing a Mama-based neural network, training the neural network, verifying the error of the neural network after the training is finished, and processing test set data if the error is smaller than an expected value. Outputting a target tracking result, and if an error is greater than or equal to an expected value, returning to perform neural network training; the Mama-based neural network comprises a backbone layer (Backbone), a neck layer (Neck) and a head layer (Head); according to the method, the adaptability and the accuracy of a complex shape and attitude change target are improved, the characteristics are better serialized and processed, and long-range dependence is captured; local detail (deformable convolution) and global (Mangbar) information are captured at the same time, and the method is suitable for a target tracking task of a complex scene.

Description

Technical Field

[0001] The present invention discloses a neural network construction method based on Mamba for target tracking, belonging to the technical field of target tracking. Background Art

[0002] Target tracking in remote sensing images faces three core problems: First, the weak target-environment contrast caused by low resolution and complex background interference; second, the unstable feature representation caused by multi-scale targets and cloud occlusion; third, the significant intra-class differences caused by sensor noise and illumination variation. Traditional target tracking methods usually rely on pre-defined feature models (such as classification models based on edge, texture, or spectral features), and parameters need to be adjusted according to specific datasets. However, such models have poor adaptability to complex scenes, and the parameter optimization process requires a large amount of manual intervention, with low automation, especially limited efficiency when dealing with massive remote sensing data. In addition, traditional methods are difficult to simultaneously consider the overall morphological features (such as building outlines, road directions) and local detail features (such as textures, shadow changes) of the target during target tracking, resulting in limited accuracy and integrity of target recognition. Summary of the Invention

[0003] The purpose of the present invention is to provide a neural network construction method based on Mamba for target tracking to solve the problem of insufficient accuracy of the neural network for target tracking in the prior art.

[0004] A neural network construction method based on Mamba for target tracking, prepare a target dataset, construct a neural network based on Mamba, perform neural network training, verify the neural network error after training is completed. If the error is less than the expected value, process the test set data and output the target tracking result. If the error is greater than or equal to the expected value, return to perform neural network training;

[0005] The neural network based on Mamba includes a backbone layer Backbone, a neck layer Neck, and a head layer Head; the Backbone sequentially includes a deformable convolution parallel multi-scale global local scanning Mamba module CM and 4 stage layers. The first 3 stage layers include CM and a deformable cross-stage partial layer Deform_CSPLayer with add=Ture. The 4th stage layer includes CM, a Deform_CSPLayer with add=Ture, and a spatial pyramid pooling layer SPPF;

[0006] The Neck includes upsampling the output of the SPPF, performing feature fusion with the output of the 3rd stage layer, inputting it into the Deform_CSPLayer with add=False, then performing upsampling again, performing feature fusion with the output of the 2nd stage layer, inputting it into the Deform_CSPLayer with add=False again, then inputting it into the CM, performing feature fusion with the output of the first Deform_CSPLayer with add=False, inputting it into the Deform_CSPLayer with add=False again, then inputting it into the CM, performing feature fusion with the output of the SPPF, and finally inputting it into the Deform_CSPLayer with add=False;

[0007] The Head includes using the outputs of the 2nd, 3rd, and 4th Deform_CSPLayers with add=False in the Neck as three input branches. Each input branch feeds the input features into two branches simultaneously. The first branch sequentially includes the CM, a 2D convolutional layer Conv2d, and a bounding box loss Bbox.Loss. The second branch sequentially includes the CM, a 2D convolutional layer Conv2d, and a classification loss Cls.Loss.

[0008] The Deform_CSPLayer sequentially includes a 1×1 convolutional layer, a split layer, multiple bottleneck layers Bottleneck, a concatenation layer, and a 1×1 convolutional layer. The output feature of the 1×1 convolutional layer and the split layer, and the output feature of the first Bottleneck have a shortcut connection to the concatenation layer. A convolutional layer that halves the number of channels of the feature is provided in the shortcut connection;

[0009] When add=True, the Bottleneck includes two CM connected in series. The input feature of the Bottleneck is fused with the output feature of the last CM. When add=False, the Bottleneck includes two CM connected in series.

[0010] The input of the CM is X_0. The CM includes a 1×1 convolutional layer. The output feature of the 1×1 convolutional layer is X. X is divided into two branches. One branch is input into a deformable convolution module and then outputs Y_1. The other branch is input into an ME-Mamba module and then outputs Y_2. Y_1 and Y_2 are input into an adaptive parameter adjustment module together, and then output through a 1×1 convolutional layer.

[0011] The adaptive parameter adjustment module includes inputting Y_1 and Y_2 into the shared feature encoder and the private feature encoder simultaneously. The output of Y_1 in the shared feature encoder is S_1, the output of Y_2 in the shared feature encoder is S_2, the output of Y_1 in the private feature encoder is O_1, and the output of Y_2 in the private feature encoder is O_2. Input S_1 and S_2 into the shared feature aggregation module, input O_1 and O_2 into the private feature retention module, and finally send the output values of the shared feature aggregation module and the private feature retention module into the feature enhancement interaction module, and then output.

[0012] Use the cross-channel contrast loss function in the shared feature encoder and the private feature encoder :

[0013] ;

[0014] ;

[0015] In the formula, is the similarity metric function, and take values of 1 or 2, and are two independent variables in the similarity metric function, is the number of channels;

[0016] Use the weighted mean square error loss to measure the feature reconstruction quality:

[0017] ;

[0018] In the formula, and are the reconstructed and , , is the channel dimension;

[0019] In the feature enhancement interaction module, calculate the mean of the encoder outputs:

[0020] ;

[0021] ;

[0022] In the formula, is the shared feature mean, is the private feature mean;

[0023] Perform feature fusion:

[0024] ;

[0025] In the formula, is the Sigmoid activation function, is the fused feature, represents and for cross-modal feature fusion.

[0026] The deformable convolution module sequentially includes a 1×1 convolutional layer, two cascaded deformable convolutional layers, a feature fusion layer, and a 1×1 convolutional layer. The output of the 1×1 convolutional layer has three branches. One branch is input into the first deformable convolutional layer, one branch is input into a 3×3 convolutional layer, and after DDP pooling, it is input into the first deformable convolutional layer. One branch is input into the feature fusion layer. The output of the first deformable convolutional layer has two branches. One branch is input into the second deformable convolutional layer, and one branch is input into a 3×3 convolutional layer, and after DDP pooling, it is input into the second deformable convolutional layer.

[0027] The ME-Mamba module sequentially includes a Layer Normalization layer, a LinearProjection layer, a Discrete Wavelet Transform 2D Convolution module (DWT2DConv), a Mish activation function layer, a Multi-scale Global Local 2D Selective Scanning module (MGLS2D), a Layer Normalization layer, a first feature fusion layer, a Linear Projection layer, and a second feature fusion layer;

[0028] The output of the first Layer Normalization has a branch that connects to the Linear Projection and Mish and then is input into the first feature fusion layer. The input of the ME-Mamba module has a shortcut connection that is connected to the second feature fusion layer.

[0029] The DWT2Dconv sequentially includes a combined layer of batch normalization-convolution-ReLU activation function (Conv-BN-ReLU), a discrete wavelet transform layer (DWT2D), a Conv-BN-ReLU, a first feature fusion layer, an inverse discrete wavelet transform layer (IDWT2D), a second feature fusion layer, and a 1×1 convolutional layer;

[0030] The output of the DWT2D has a branch that sequentially connects to the DWT2D, Conv-BN-ReLU, and IDWT2D, and then is input into the first feature fusion layer. The output of the first Conv-BN-ReLU has a branch that is connected to the second feature fusion layer.

[0031] MGLS2D includes two branches. The first branch inputs the input features into the multi-scale global-local scanning strategy layer MGLS, generating features T1 and T2. Then, T1 and T2 are input into the selective scanning spatial state sequence module S6, and the output of S6 is sent to the merging layer Merge. The first branch inputs the input features into the depthwise separable convolutional layer DWconv, and then into the multi-scale global-local scanning strategy layer MGLS, generating features T3 and T4. Then, T3 and T4 are input into the selective scanning spatial state sequence module S6, and then into the upsampling interpolation layer Interprolate, and finally sent to Merge;

[0032] In MGLS, the input feature image is scanned in a zigzag pattern globally to obtain continuous global image information. MGLS includes two scanning paths. The first scanning path scans the local image patches in an S-shaped pattern, and the second scanning path scans the local image patches in a zigzag pattern; two different scanning path processing information of large-scale remote sensing images are obtained, and then the input feature map is passed through another depthwise separable convolution layer, and the small-scale feature images are obtained by using the zigzag scan in different directions.

[0033] Neural network training includes initializing model parameters, inputting the original remote sensing image and the corresponding remote sensing image class label data training set into the neural network for iteration, and saving the model parameters;

[0034] When performing target tracking, the remote sensing images in the test set are put into the trained neural network, and the target tracking results of the remote sensing images are output.

[0035] Compared with the prior art, the present invention has the following beneficial effects: The present invention improves the adaptability and accuracy for targets with complex shapes and pose changes, better serializes the processing of features, and captures long-range dependencies; it realizes the simultaneous capture of local details (deformable convolution) and global (mamba) information, and is suitable for target tracking tasks in complex scenarios. Description of the Drawings

[0036] Figure 1 The overall structure diagram of the present invention is shown;

[0037] Figure 2 The structure diagram of the Deform CSP Layer module of the present invention is shown;

[0038] Figure 3 The basic structure diagram of the CM module of the present invention is shown;

[0039] Figure 4 The structure diagram of the adaptive parameter adjustment module of the present invention is shown;

[0040] Figure 5The structure diagram of the deformable convolution module according to the present invention is shown;

[0041] Figure 6 The structure diagram of the multi-scale global-local scanning Mamba module according to the present invention is shown;

[0042] Figure 7 The structure diagram of the discrete wavelet convolution module according to the present invention is shown;

[0043] Figure 8 The structure diagram of the MGLS2D according to the present invention is shown;

[0044] Figure 9 The overall flowchart of the remote sensing image target tracking method according to the present invention is shown;

[0045] Figure 10 The flowchart of the training steps of the present invention is shown;

[0046] Figure 11 The flowchart of the remote sensing image target tracking steps according to the present invention is shown. Detailed implementation manners

[0047] To make the objectives, technical solutions, and advantages of the present invention clearer, the technical solutions in the present invention will be clearly and completely described below. Obviously, the described embodiments are some, but not all, of the embodiments of the present invention. All other embodiments obtained by those of ordinary skill in the art based on the embodiments in the present invention without creative efforts shall fall within the protection scope of the present invention.

[0048] A method for constructing a neural network for target tracking based on Mamba is as Figure 9 shown. Prepare a target data set, construct a neural network based on Mamba, perform neural network training, verify the neural network error after training is completed. If the error is less than the expected value, process the test set data and output the target tracking result. If the error is greater than or equal to the expected value, return to perform neural network training;

[0049] The neural network based on Mamba is as Figure 1 shown, and includes a backbone layer Backbone, a neck layer Neck, and a head layer Head; the Backbone sequentially includes a deformable convolution parallel multi-scale global-local scanning Mamba module CM and 4 stage layers. The first 3 stage layers include CM and a deformable cross-stage partial layer Deform_CSPLayer with add=Ture. The 4th stage layer includes CM, a Deform_CSPLayer with add=Ture, and a spatial pyramid pooling layer SPPF;

[0050] The Neck includes upsampling the output of the SPPF, fusing features with the output of the 3rd stage layer, inputting to the Deform_CSPLayer with add=False, then performing upsampling, fusing features with the output of the 2nd stage layer, inputting to the Deform_CSPLayer with add=False again, then inputting to the CM, fusing features with the output of the first Deform_CSPLayer with add=False, inputting to the Deform_CSPLayer with add=False again, then inputting to the CM, fusing features with the output of the SPPF, and finally inputting to the Deform_CSPLayer with add=False;

[0051] The Head includes taking the outputs of the 2nd, 3rd, and 4th Deform_CSPLayers with add=False in the Neck as three input branches. Each input branch feeds the input features into two branches simultaneously. The first branch sequentially includes the CM, a 2D convolutional layer Conv2d, and a bounding box loss Bbox.Loss. The second branch sequentially includes the CM, a 2D convolutional layer Conv2d, and a classification loss Cls.Loss.

[0052] The Deform_CSPLayer is as Figure 2 shown, and sequentially includes a 1×1 convolutional layer, a split layer, multiple Bottleneck layers, a concatenation layer, and a 1×1 convolutional layer. For the 1×1 convolutional layer and the split layer, the output features of the first Bottleneck have a shortcut connection to the concatenation layer, and there is a convolutional layer in the shortcut connection that halves the number of channels of the features;

[0053] When add=True, the Bottleneck includes two CM in series, and the input features of the Bottleneck are fused with the output features of the last CM. When add=False, the Bottleneck includes two CM in series.

[0054] The CM is as Figure 3 shown. The input of the CM is X_0. The CM includes a 1×1 convolutional layer. The output features of the 1×1 convolutional layer are X. X is divided into two branches. One branch is input to the deformable convolution module and then outputs Y_1. The other branch is input to the ME-Mamba module and then outputs Y_2. Y_1 and Y_2 are input to the adaptive parameter adjustment module together, and then output through a 1×1 convolutional layer.

[0055] The adaptive parameter adjustment module is as Figure 4As shown in the figure, it includes simultaneously inputting Y_1 and Y_2 into a shared feature encoder and a private feature encoder. The output of Y_1 in the shared feature encoder is S_1, the output of Y_2 in the shared feature encoder is S_2, the output of Y_1 in the private feature encoder is O_1, and the output of Y_2 in the private feature encoder is O_2. Input S_1 and S_2 into the shared feature aggregation module, input O_1 and O_2 into the private feature retention module, and finally send the output values of the shared feature aggregation module and the private feature retention module to the feature enhancement interaction module, and then output.

[0056] Use a cross-channel contrast loss function in the shared feature encoder and the private feature encoder :

[0057] ;

[0058] ;

[0059] In the formula, is a similarity metric function, and take values of 1 or 2, and are two independent variables in the similarity metric function, is the number of channels;

[0060] Use a weighted mean squared error loss to measure the feature reconstruction quality:

[0061] ;

[0062] In the formula, and are the reconstructed and , , is the channel dimension;

[0063] In the feature enhancement interaction module, take the mean of the outputs of the encoders:

[0064] ;

[0065] ;

[0066] In the formula, is the shared feature mean, is the private feature mean;

[0067] Perform feature fusion:

[0068] ;

[0069] Wherein, is the Sigmoid activation function, is the fused feature, represents the cross-modal feature fusion of and .

[0070] The deformable convolution module is shown in Figure 5 and successively includes a 1×1 convolution layer, two cascaded deformable convolution layers, a feature fusion layer, and a 1×1 convolution layer. The output of the 1×1 convolution layer has three branches. One branch is input into the first deformable convolution layer, one branch is input into a 3×3 convolution layer, and after DDP pooling, it is input into the first deformable convolution layer, and one branch is input into the feature fusion layer. The output of the first deformable convolution layer has two branches. One branch is input into the second deformable convolution layer, and one branch is input into a 3×3 convolution layer, and after DDP pooling, it is input into the second deformable convolution layer.

[0071] The ME-Mamba module is shown in Figure 6 and successively includes a layer normalization layer Layer Normalization, a linear projection layer Linear Projection, a discrete wavelet transform convolution module DWT2DConv, an activation function layer Mish, a multi-scale global local two-dimensional selective scanning module MGLS2D, Layer Normalization, a first feature fusion layer, LinearProjection, and a second feature fusion layer;

[0072] The output of the first Layer Normalization has a branch that connects to Linear Projection and Mish and then is input into the first feature fusion layer. The input of the ME-Mamba module has a shortcut connection that is connected to the second feature fusion layer.

[0073] The DWT2Dconv is shown in Figure 7 and successively includes a combined layer of batch normalization-convolution-ReLU activation function Conv-BN-ReLU, a discrete wavelet transform layer DWT2D, Conv-BN-ReLU, a first feature fusion layer, an inverse discrete wavelet transform layer IDWT2D, a second feature fusion layer, and a 1×1 convolution layer;

[0074] The output of DWT2D has a branch that successively connects to DWT2D, Conv-BN-ReLU, and IDWT2D, and then is input into the first feature fusion layer. The output of the first Conv-BN-ReLU has a branch that is connected to the second feature fusion layer.

[0075] The MGLS2D is shown in Figure 8As shown in the figure, it includes two branches. The first branch inputs the input features into the multi-scale global-local scanning strategy layer MGLS, generating features T1 and T2, and then inputs T1 and T2 into the selective scanning spatial state sequence module S6. The output of S6 is sent to the merging layer Merge. The first branch inputs the input features into the depthwise separable convolutional layer DWconv, and then inputs them into the multi-scale global-local scanning strategy layer MGLS, generating features T3 and T4. Then, T3 and T4 are input into the selective scanning spatial state sequence module S6, and then input into the upsampling interpolation layer Interprolate, and finally sent to Merge;

[0076] In MGLS, the input feature image is scanned in a zigzag pattern globally to obtain continuous global image information. MGLS includes two scanning paths. The first scanning path scans the local image patches in an S-shaped pattern, and the second scanning path scans the local image patches in a zigzag pattern; two different scanning path processing information of large-scale remote sensing images are obtained, and then the input feature map is passed through another depthwise separable convolution, and the zigzag scan is used from different directions to obtain small-scale feature images.

[0077] As Figure 10 shown, the neural network training includes initializing the model parameters, inputting the original remote sensing image and the corresponding remote sensing image class label data training set into the neural network for iteration, and saving the model parameters;

[0078] As Figure 11 shown, when performing target tracking, the remote sensing image in the test set is put into the trained neural network, and the target tracking result of the remote sensing image is output.

[0079] MWORKS is a new-generation scientific computing and system modeling and simulation platform. Through the unified modeling language and high-performance solving engine, it provides full-life-cycle support for the research and development, simulation, verification, and optimization of complex systems. In the field of industrial software, traditional tools (such as MATLAB / Simulink, ANSYS, etc.) have been monopolized by foreign countries for a long time, suffering from problems such as insufficient multi-domain collaborative modeling capabilities, low simulation efficiency, complex code generation and hardware adaptation, low model reuse rate, and limited intelligence level, making it difficult to meet the requirements for efficient, independent, and controllable complex system design in the digital transformation. MWORKS addresses these pain points by building a multi-disciplinary unified modeling environment based on the Modelica language and its own kernel, supporting the deep integration of fields such as machinery, electricity, and control, achieving large-scale simulation acceleration through a high-performance solving engine, and combining AI technology to improve parameter optimization and automated modeling capabilities. At the same time, its scientific computing environment (MWORKS.Syslab) is compatible with languages such as Python and Julia, providing algorithm development and data analysis support, while the industrial knowledge model platform (MoHub) promotes the cloud sharing and collaborative innovation of models. This platform not only realizes the domestic substitution of foreign industrial software but also significantly improves the efficiency and reliability of complex system design through its intelligent, open, and cloud collaborative capabilities, especially promoting the digital and intelligent transformation of equipment research and development in fields such as aerospace, automotive, and energy.

[0080] The single-stage architecture of YOLOv8 mainly focuses on local feature extraction and is limited in tasks that require global context understanding (such as long-tail categories and complex scene reasoning). For example, when there are highly similar textures or colors between the target and the background, false detections or missed detections are likely to occur. In night-time monitoring or adverse weather conditions, due to its reliance on local features, YOLOv8's robustness decreases in scenarios where the target outline is blurred or partially occluded.

[0081] Mamba is an efficient sequence modeling architecture based on the state space model (SSM). By dynamically screening key information and recursively passing hidden states, it can effectively capture long-range dependencies and global information. To meet the requirements of visual tasks, the researchers proposed Visual Mamba, and the main improvements include: introducing a cross-path scanning mechanism to unfold the two-dimensional image into a four-way sequence to address the defect of traditional Mamba in spatial continuity modeling; designing a direction-sensitive state transition matrix to enhance the perception ability of local geometric features such as edge directions. However, compared with models specifically optimized for local features such as YOLOv8, VMamba still has deficiencies in detailed modeling. For example, in the object tracking task, although the cross-path scanning mechanism alleviates the problem of pixel-level information loss, due to the linear time-invariant characteristics of the state space model, its selective calculation mechanism may over-smooth high-frequency textures, resulting in lower edge segmentation accuracy of small targets than convolutional models.

[0082] Specifically, YOLOv8 can accurately capture sub-pixel offsets of object boundaries through dense coverage of multi-scale convolutional kernels and feature pyramid fusion. However, under the advantage of long-range modeling, VMamba still has room for improvement in adapting to local deformations (such as broken contours and serrated edges).

[0083] Based on the MWORKS platform, the present invention proposes an innovative fusion architecture to process remote sensing image information by integrating the advantages of YOLOv8 and the Mamba model. The method innovatively proposes to replace the convolutional module in the traditional YOLOv8 backbone network with a parallel fusion structure of a deformable convolutional module and a multi-scale global-local scanning Mamba module to improve the adaptability and accuracy of the model for targets with complex shapes and pose changes. At the same time, it better serializes the processing of features and captures long-range dependencies. Finally, an adaptive parameter adjustment module is introduced to timely adjust the channel weight ratio of the deformable convolutional module and the multi-scale global-local scanning Mamba module to achieve the simultaneous capture of local details and global information, suitable for target tracking tasks in complex scenarios, thus solving the problems in the prior art that remote sensing target tracking cannot balance local features and global features, resulting in poor small target tracking effect and weak generalization ability in complex scenarios.

[0084] The present invention needs to prepare three datasets: a training set, a validation set, and a test set, and the ratio of the three is approximately 3:1:1. Among them, the training set is responsible for training the target tracking model, the validation set is responsible for adjusting the target tracking model, and the test set is responsible for testing the target tracking effect of the model. In order to better demonstrate the good characteristics of the present invention in balancing local features and global features, the DOTA dataset, a relatively feasible target tracking dataset in the current remote sensing field, is used. This dataset has 15 category numbers and large-scale, multi-category, and high-complexity annotation data. In this dataset, more than 200 images are selected as the training set, and each remote sensing image corresponds to a piece of data after target tracking, and the latter is the label of the former. The construction methods of the validation set and the test set are the same as those of the dataset, but the remote sensing images and corresponding data in the validation set should be different from those in the dataset.

[0085] Input the remote sensing image data in the training set into the constructed network model. The output of the network is the data for target tracking using the network described in the present invention. Compare the target tracking results output by this network with the data that has been processed in the original dataset, and at the same time calculate the error between the two. If the error is greater than the set threshold (the threshold size is set according to the actual situation), the parameters are updated by backpropagation. Iterate repeatedly until the error is less than the set threshold, stop updating the parameters, and save the network parameters. The present invention uses the Adam optimization algorithm to solve the parameters. During error backpropagation, the parameters are updated according to the error gradient information. The loss function is :

[0086] ;

[0087] In the formula, is the quantity of batch-processed data, represents the target tracking network model of the remote sensing image of the present invention, represents the target tracking label - original remote sensing image data training pair in the training set, are the parameters of the remote sensing image target tracking model of the present invention.

[0088] Input the remote sensing image data in the validation set into the constructed network model. The output of the network is the data after target tracking using the network of the present invention. Compare the data after target tracking with the original remote sensing image target category label, and at the same time calculate the error between the two. If the error is less than the set threshold (the size of the threshold is set according to the actual situation), it proves that the trained network parameters have reached the optimal, and the target tracking of the test set can be carried out. If the error is greater than the set threshold, it proves that the trained network parameters have not reached the optimal, and return to retrain the network until a good target tracking effect can be achieved in the validation stage.

[0089] Input the original remote sensing data in the test set into the network of the present invention that has been trained and verified. The output of the network is the data after target tracking using the network of the present invention.

[0090] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that: they can still modify the technical solutions recorded in the foregoing embodiments, or perform equivalent replacements on some or all of the technical features, and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for constructing a neural network for target tracking based on Mamba, characterized in that: Prepare the target data set, build a neural network based on Mamba, perform neural network training, verify the neural network error after training, process the test set data if the error is less than the expected value, output the target tracking result, and return to neural network training if the error is greater than or equal to the expected value; The neural network based on Mamba includes the backbone layer Backbone, the neck layer Neck and the head layer Head; Backbone includes the deformable convolutional parallel multi-scale global local scanning Mamba module CM and four stage layers stage layer in turn, the first three stage layers include CM and the deformable cross-stage partial layer Deform_CSPLayer of add=Ture, and the fourth stagelayer includes CM, Deform_CSPLayer of add=Ture and the spatial pyramid pooling layer SPPF; Neck includes upsampling the output of SPPF, fusing the features with the output of the third stage layer, inputting it to Deform_CSPLayer with add=False, then upsampling it again, fusing the features with the output of the second stage layer, then inputting it to Deform_CSPLayer with add=False, then inputting it to CM, fusing the features with the first Deform_CSPLayer with add=False, then inputting it to Deform_CSPLayer with add=False, then inputting it to CM, fusing the features with the output of SPPF, and then inputting it to Deform_CSPLayer with add=False; Head includes the outputs of the second, third, and fourth Deform_CSPLayer with add=False in Neck as three input branches. Each input branch sends the input features to two branches at the same time. The first branch includes CM, two-dimensional convolution layer Conv2d, and bounding box loss Bbox.Loss in sequence, and the second branch includes CM, two-dimensional convolution layer Conv2d, and classification loss Cls.Loss in sequence.

2. A method for constructing a neural network for target tracking based on Mamba according to claim 1, characterized in that: The Deform_CSPLayer includes a 1×1 convolution layer, a split layer, multiple bottleneck layers, a splicing layer, and a 1×1 convolution layer in sequence. The output features of the 1×1 convolution layer, the split layer, and the first Bottleneck are provided with a shortcut connection connected to the splicing layer, and the shortcut connection is provided with a convolution layer that reduces the number of channels of the feature by half; When add=Ture, Bottleneck includes two CMs connected in series, and the input features of Bottleneck are fused with the output features of the last CM. When add=False, Bottleneck includes two CMs connected in series.

3. A method for constructing a neural network for target tracking based on Mamba according to claim 2, characterized in that: The input of CM is X_0. CM includes a 1×1 convolution layer. The output feature of the 1×1 convolution layer is X. X is divided into two branches. One branch is input into the deformable convolution module and outputs Y_1. The other branch is input into the ME-Mamba module and outputs Y_2. Y_1 and Y_2 are input into the adaptive parameter adjustment module together and then output through a 1×1 convolution layer.

4. A method for constructing a neural network for target tracking based on Mamba according to claim 3, characterized in that: The adaptive parameter adjustment module includes inputting Y_1 and Y_2 into a shared feature encoder and a private feature encoder at the same time, the output of Y_1 in the shared feature encoder is S_1, the output of Y_2 in the shared feature encoder is S_2, the output of Y_1 in the private feature encoder is O_1, the output of Y_2 in the private feature encoder is O_2, S_1 and S_2 are input into a shared feature aggregation module, O_1 and O_2 are input into a private feature retention module, and finally the output values ​​of the shared feature aggregation module and the private feature retention module are sent to a feature enhancement interaction module and then output.

5. A method for constructing a neural network for target tracking based on Mamba according to claim 4, characterized in that: Using cross-channel contrastive loss function in shared feature encoder and private feature encoder : ； ； In the formula, is the similarity measure function, and The value of is 1 or 2. and are the two independent variables in the similarity measurement function, is the number of channels; Use weighted mean squared error loss Measuring feature reconstruction quality: ； In the formula, and It is rebuilt and , , is the channel dimension; In the feature enhancement interaction module, the output of the encoder is averaged: ； ； In the formula, is the shared feature mean, is the private feature mean; Perform feature fusion: ； In the formula, is the Sigmoid activation function, is the fused feature, Express and Perform cross-modal feature fusion.

6. A method for constructing a neural network for target tracking based on Mamba according to claim 5, characterized in that: The deformable convolution module includes a 1×1 convolution layer, two variable convolution layers connected in series, a feature fusion layer and a 1×1 convolution layer in sequence. The output of the 1×1 convolution layer has three branches, one is input into the first variable convolution layer, one is input into the 3×3 convolution layer and then input into the first variable convolution layer after DDP pooling, and one is input into the feature fusion layer. The output of the first variable convolution layer has two branches, one is input into the second variable convolution layer, and the other is input into the 3×3 convolution layer and then input into the second variable convolution layer after DDP pooling.

7. A method for constructing a neural network for target tracking based on Mamba according to claim 6, characterized in that: The ME-Mamba module includes layer normalization layer Layer Normalization, linear projection layer LinearProjection, discrete wavelet transform convolution module DWT2DConv, activation function layer Mish, multi-scale global local two-dimensional selective scanning module MGLS2D, Layer Normalization, first feature fusion layer, Linear Projection, and second feature fusion layer; The output of the first Layer Normalization is branched, connected to Linear Projection and Mish, and then input to the first feature fusion layer. The input of the ME-Mamba module is connected to a shortcut connection to the second feature fusion layer.

8. A method for constructing a neural network for target tracking based on Mamba according to claim 7, characterized in that: DWT2Dconv includes batch normalization-convolution-ReLU activation function combination layer Conv-BN-ReLU, discrete wavelet transform layer DWT2D, Conv-BN-ReLU, first feature fusion layer, inverse discrete wavelet transform layer IDWT2D, second feature fusion layer, and 1×1 convolution layer in sequence; The output of DWT2D is branched, connected to DWT2D, Conv-BN-ReLU, IDWT2D in sequence, and then input to the first feature fusion layer. The output of the first Conv-BN-ReLU is branched and connected to the second feature fusion layer.

9. A method for constructing a neural network for target tracking based on Mamba according to claim 8, characterized in that: MGLS2D includes two branches. The first branch inputs the input features into the multi-scale global local scanning strategy layer MGLS to generate features T1 and T2, and then inputs T1 and T2 into the selective scanning space state sequence module S6. The output of S6 is sent to the merging layer Merge. The first branch inputs the input features into the depth-separable convolution layer DWconv, and then inputs them into the multi-scale global local scanning strategy layer MGLS to generate features T3 and T4. Then T3 and T4 are input into the selective scanning space state sequence module S6, and then input into the upsampling interpolation layer Interprolate, and finally sent to Merge. In MGLS, the input feature image is scanned globally in a zigzag pattern to obtain continuous global image information. MGLS includes two scanning paths, the first scanning path uses an S-shaped scan on a local image block, and the second scanning path uses a zigzag scan on a local image block. The two different scanning path processing information of the large-scale remote sensing image is obtained, and then the input feature map is subjected to another layer of depth-separable convolution, and a small-scale feature image is obtained using a zigzag scan from different directions.

10. A method for constructing a neural network for target tracking based on Mamba according to claim 9, characterized in that: The neural network training includes initializing the model parameters, inputting the original remote sensing image and the corresponding remote sensing image class label data training set into the neural network for iteration, and saving the model parameters; When performing target tracking, the remote sensing images in the test set are placed into the trained neural network, and the target tracking results of the remote sensing images are output.

Images