A self-supervised SAR target recognition method and system based on the MSP-MAE model

Abstract

This invention relates to the field of automatic radar target recognition technology, and in particular to a self-supervised SAR target recognition method and system based on the MSP-MAE model. The method includes acquiring SAR image data for training and testing; performing data augmentation based on the acquired SAR image data to construct an MSP-MAE model; performing self-supervised training of the MSP-MAE model using the augmented image data; initializing some weight parameters of the MSP-CT model using encoder weight parameters; fine-tuning the parameters of the initialized MSP-CT model; and using the fine-tuned MSP-CT model for SAR target recognition. This invention transfers the weight parameters of the MSP-MAE encoder to the MSP-CT model, achieving effective knowledge transfer, reducing dependence on labeled data, and improving the model's performance on small sample data. This effectively solves the problem of SAR image target recognition and improves recognition accuracy.

Description

Technical Field

[0001] This invention relates to the field of radar automatic target recognition technology, and in particular to a self-supervised SAR target recognition method and system based on the MSP-MAE model. Background Technology

[0002] Synthetic Aperture Radar (SAR), as an important microwave remote sensing technology, plays a crucial role in numerous fields such as topographic mapping and target monitoring due to its all-weather, all-time operation capabilities. For example, in natural disaster monitoring, SAR can penetrate clouds and dark environments to acquire images of disaster-stricken areas, providing important information for rescue decisions. However, the inherent speckle noise problem in SAR images severely interferes with the semantic interpretation of the images, making accurate target identification extremely challenging. Currently, with the rapid development of computer vision technology, Automatic Target Recognition (ATR) technology is increasingly widely used in SAR image interpretation. Machine learning, especially deep learning methods, has gradually become the mainstream technology for SAR ATR due to its powerful feature learning capabilities. Deep learning models can automatically learn features from raw data, eliminating the need for tedious manual feature extraction and classifier design, greatly improving the efficiency and accuracy of SAR target identification.

[0003] However, the training of deep learning models is highly dependent on a large amount of labeled data. In the field of SAR images, the difficulty and cost of data labeling lead to a relative scarcity of labeled data. This makes deep learning models prone to overfitting, which severely restricts their performance in SAR target recognition. SAR images generate severe speckle noise due to electromagnetic backscattering, which greatly interferes with the semantic interpretation of images, making it difficult to accurately identify targets and hindering the effective application of SAR in practical scenarios such as terrain monitoring and target surveillance. At present, there is a need for a self-supervised SAR target recognition method and system based on the MSP-MAE model. Summary of the Invention

[0004] To address the issues of speckle noise and low target recognition accuracy in SAR images, this invention provides a self-supervised SAR target recognition method and system based on the MSP-MAE model. The MSP-MAE-based self-supervised SAR target recognition process encompasses multiple stages, including data processing, model construction and training, model fine-tuning, and performance evaluation, effectively solving the challenge of target recognition in SAR images and improving recognition accuracy.

[0005] In a first aspect, the present invention provides a self-supervised SAR target recognition method based on the MSP-MAE model, which adopts the following technical solution:

[0006] A self-supervised SAR target recognition method based on the MSP-MAE model includes:

[0007] Acquire SAR image data for training and testing, including image data from the OpenSAR-Ship, MSTAR, and FUSAR-Ship datasets;

[0008] Data augmentation is performed on the acquired SAR image data, including data augmentation using random cropping, random horizontal flipping, and the Mixup method;

[0009] An MSP-MAE model was constructed, and the MSP-MAE model was trained in a self-supervised manner using the enhanced image data to obtain the model weight parameters of MSP-MAE.

[0010] Obtain the encoder weight parameters in the trained MSP-MAE model, and use the encoder weight parameters to initialize some weight parameters of the MSP-CT model;

[0011] The parameters of the initialized MSP-CT model are fine-tuned, and the MSP-CT model is then trained in a supervised manner.

[0012] The finely tuned MSP-CT model is used for SAR target recognition, and the output is the recognition result for each test image.

[0013] Furthermore, the construction of the MSP-MAE model and the self-supervised training of the MSP-MAE model using the enhanced image data include constructing an MSP-MAE model composed of an encoder and a decoder. The MSP-MAE model combines a multi-scale convolution module and a multi-head self-attention module, and uses a window masking strategy to mask the input image. The encoder is an MSP-CT encoder, and a window masking layer is added between the linear embedding layer in the first stage and the MSP-CT module.

[0014] Furthermore, the construction of the MSP-MAE model and the self-supervised training of the MSP-MAE model using the enhanced image data also include the use of a lightweight decoder with three stages. In each stage, the input features are first expanded by a factor of two through a Patch Expanding layer consisting of a linear layer and a normalization layer. The label dimension is mapped using a prediction head to reconstruct the original image based on the latent vector output by the encoder. For the prediction error of the masked part in the image, the mean square error of the original image and the reconstructed image is calculated as a loss function. The completed MSP-MAE model is pre-trained using the loss function.

[0015] Furthermore, the initialization of some weight parameters of the MSP-CT model using encoder weight parameters includes discarding the decoder part after the MSP-MAE model is trained, extracting the encoder weight parameters from the trained MSP-MAE model, storing the weight parameters in the form of a numerical matrix in each layer of the encoder, determining the corresponding network layer based on the network structure of the MSP-MAE encoder and the MSP-CT model, and assigning the weight parameters of the corresponding layer extracted from the MSP-MAE encoder to the corresponding network layer in the MSP-CT model.

[0016] Furthermore, the parameter fine-tuning of the initialized MSP-CT model includes inputting complete, unmasked image data into the encoder of the assigned MSP-CT to obtain the output features of the MSP-CT encoder, performing global pooling on the output features, inputting the pooled feature vector into the classifier for classification, and fine-tuning the MSP-CT using the MSTAR and FUSAR-Ship datasets respectively.

[0017] Furthermore, the supervised training of the MSP-CT model includes a structure in which the four stages of MSP-CT adopt an alternating stacked structure of MSDC and MHSA layers, and linear embedding operation is performed through convolutional kernels. Each stage except the first stage is downsampled. The MSDC layer consists of an MSCF module, a channel attention module, and a spatial attention module. The MSCF module uses convolutional branches to improve the feature representation, the spatial attention module performs feature enhancement through pooling, convolution, and activation operations, and finally the attention module uses the SE module to obtain the channel weights.

[0018] Furthermore, the supervised training of the MSP-CT model also includes using a multi-scale inverted feedforward network composed of convolutional layers and feature fusion layers as the feedforward network, then inserting convolutional branches to form residual connections, and the output features sequentially passing through a global average pooling layer, layer normalization, Dropout layer and fully connected layer. The fully connected layer outputs the classification prediction value, and the predicted probability of each class and the corresponding true label are compared by the cross-entropy loss function. Finally, the backpropagation algorithm is used to train the MSP-CT model.

[0019] Secondly, a self-supervised SAR target recognition system based on the MSP-MAE model includes:

[0020] The data acquisition module is configured to acquire SAR image data for training and testing, including image data acquired through OpenSAR-Ship, MSTAR, and FUSAR-Ship datasets;

[0021] The preprocessing module is configured to perform data augmentation based on the acquired SAR image data, including data augmentation using random cropping, random horizontal flipping, and the Mixup method;

[0022] The model module is configured to: construct an MSP-MAE model and perform self-supervised training on the MSP-MAE model using enhanced image data to obtain the model weight parameters of MSP-MAE;

[0023] The conversion module is configured to: obtain the encoder weight parameters in the trained MSP-MAE model, and initialize some weight parameters of the MSP-CT model using the encoder weight parameters;

[0024] The training module is configured to fine-tune the parameters of the initialized MSP-CT model and perform supervised training on the MSP-CT model.

[0025] The output module is configured to perform SAR target recognition using the fine-tuned MSP-CT model, and the output is the recognition result for each test image.

[0026] Thirdly, the present invention provides a computer-readable storage medium storing a plurality of instructions adapted for loading and execution by a processor of a terminal device of the self-supervised SAR target recognition method based on the MSP-MAE model.

[0027] Fourthly, the present invention provides a terminal device, including a processor and a computer-readable storage medium, wherein the processor is used to implement various instructions; the computer-readable storage medium is used to store multiple instructions, which are adapted to be loaded and executed by the processor to provide a self-supervised SAR target recognition method based on the MSP-MAE model.

[0028] In summary, the present invention has the following beneficial technical effects:

[0029] 1. This invention employs the MSP-MAE model for self-supervised training, which can fully utilize a large amount of unlabeled SAR image data. Through window masking strategy and mean squared error loss function, the model can focus on learning the features of the masked parts of the image, mining the inherent structure and semantic information of the data, thereby learning high-quality feature representations without the need for extensive manual annotation, laying a solid foundation for subsequent target recognition tasks.

[0030] 2. This invention combines the MSP-MAE model with a multi-scale convolution module and a multi-head self-attention module. The multi-scale convolution module can capture detailed information about the target, improving the quality of the reconstructed image, while the multi-head self-attention module can acquire global information about the target, helping the model to better reconstruct the information of the masking blocks. This multi-module collaborative approach enables the model to more comprehensively and accurately understand the features of SAR images.

[0031] 3. This invention uses the MSTAR and FUSAR-Ship datasets to fine-tune the MSP-CT model, enabling the model to be optimized according to specific target recognition tasks. By processing and classifying complete and unmasked image data, the model can better adapt to real-world application scenarios and improve the accuracy and reliability of recognition.

[0032] 4. The MSP-CT model of this invention adopts an alternating stacked structure of MSDC and MHSA layers, as well as a multi-scale backfeedforward network, which enables the model to effectively fuse local and global features. The MSDC layer achieves the fusion of multi-scale information and attention to key information through the MSCF module, channel attention module and spatial attention module; the MSI-FFN module improves the feature extraction capability and training stability of the model by fusing multi-scale features and residual connections, thereby increasing the clarity of image recognition.

[0033] 5. This invention transfers the weight parameters of the MSP-MAE encoder to the MSP-CT model, achieving effective knowledge transfer. This transfer learning method allows the MSP-CT model to have good feature extraction capabilities in the initial stage, reduces the dependence on labeled data, accelerates the convergence speed of the model, and also improves the performance of the model under small sample data, effectively solving the problem of SAR image target recognition and improving recognition accuracy. Attached Figure Description

[0034] Figure 1 This is an overall framework diagram of a self-supervised SAR target recognition method based on the MSP-MAE model in Embodiment 1 of the present invention;

[0035] Figure 2 This is a detailed structural diagram of the MSP-CT in Embodiment 1 of the present invention. The leftmost part of the image is the overall structural diagram of the MSP-CT, and the right side is a detailed structural diagram of the modules in the network.

[0036] Figure 3 This is a diagram showing the effect of dilated convolution under different dilation rates in Embodiment 1 of the present invention;

[0037] Figure 4 This is a structural diagram of the SE module in Embodiment 1 of the present invention;

[0038] Figure 5 This is a structural diagram of the MSF-FFN of Embodiment 1 of the present invention;

[0039] Figure 6 This is the attention heatmap of each stage in the MSP-CT of Embodiment 1 of the present invention;

[0040] Figure 7 This is a flowchart of the Patch Merging module in Embodiment 1 of the present invention;

[0041] Figure 8 This is a diagram of the MSP-MAE network structure of Embodiment 1 of the present invention;

[0042] Figure 9 This is a flowchart of window masking in Embodiment 1 of the present invention;

[0043] Figure 10 These are effect diagrams of different masking strategies in Embodiment 1 of the present invention, wherein (a) is the original image, (b) is the image after random masking, and (c) is the image after window masking;

[0044] Figure 11 The five types of SAR vessel datasets in OpenSARShip of Embodiment 1 of the present invention are: (a) Cargo data map, (b) Fishing data map, (c) Passenger data map, (d) Tanker data map, and (e) Tug data map.

[0045] Figure 12 The ten types of SAR vehicle datasets in MSTAR of Embodiment 1 of the present invention are: (a) 2S1 data map, (b) BMP2 data map, (c) BRDM2 data map, (d) BTR60 data map, (e) BTR70 data map, (f) D7 data map, (g) T62 data map, (h) T72 data map, (i) ZIL131 data map, and (j) ZSU234 data map.

[0046] Figure 13 These are four types of SAR ship datasets in FUSAR-Ship of Embodiment 1 of the present invention: (a) Cargo data map, (b) Fishing data map, (c) Tanker data map, and (d) Other data map.

[0047] Figure 14 This is a diagram showing the effect of the random cropping method in Embodiment 1 of the present invention, where (a) is the original image, and (b), (c) and (d) are the images after random cropping;

[0048] Figure 15 This is a flowchart and effect diagram of the Mixup method in Embodiment 1 of the present invention;

[0049] Figure 16 These are reconstruction results of MSP-MAE under different masking rates according to Embodiment 1 of the present invention: (a) original image, (b) 75%, (c) 50%, and (d) 25%.

[0050] Figure 17 This is a schematic diagram illustrating the recognition accuracy of each method in Embodiment 1 of the present invention on the MSTAR dataset;

[0051] Figure 18 This is a schematic diagram illustrating the recognition accuracy of each method in Embodiment 1 of the present invention on the FUSAR-Ship dataset. Detailed Implementation

[0052] The present invention will be further described in detail below with reference to the accompanying drawings.

[0053] Example 1

[0054] Reference Figure 1 This embodiment of a self-supervised SAR target recognition method based on the MSP-MAE model includes:

[0055] Acquire SAR image data for training and testing, including image data from the OpenSAR-Ship, MSTAR, and FUSAR-Ship datasets;

[0056] Data augmentation is performed on the acquired SAR image data, including data augmentation using random cropping, random horizontal flipping, and the Mixup method;

[0057] An MSP-MAE model was constructed, and the MSP-MAE model was trained in a self-supervised manner using the enhanced image data to obtain the model weight parameters of MSP-MAE.

[0058] Obtain the encoder weight parameters in the trained MSP-MAE model, and use the encoder weight parameters to initialize some weight parameters of the MSP-CT model;

[0059] The parameters of the initialized MSP-CT model are fine-tuned, and the MSP-CT model is then trained in a supervised manner.

[0060] The finely tuned MSP-CT model is used for SAR target recognition, and the output is the recognition result for each test image.

[0061] Specifically, a self-supervised SAR target identification method based on the MSP-MAE model includes the following steps:

[0062] S1. Acquire SAR image data for training and testing, including image data from OpenSAR-Ship, MSTAR, and FUSAR-Ship datasets;

[0063] like Figure 11 , Figure 12 , Figure 13 As shown, three datasets were collected: OpenSAR-Ship, MSTAR, and FUSAR-Ship. The OpenSAR-Ship dataset is derived from Sentinel-1 images with a resolution of 2.5m × 1.0m, and five categories of ship target data were selected to provide rich samples for the pre-training of MSP-MAE. The MSTAR dataset was acquired using X-band spotlight SAR and HH polarization with a resolution of 0.3m × 0.3m, and contains 10 categories of vehicle targets, mainly used for fine-tuning and testing of MSP-CT. The FUSAR-Ship dataset was prepared by the Key Laboratory of Electromagnetic Wave Information Science of the Ministry of Education at Fudan University based on the Gaofen-3 satellite (GF-3), and four categories of ship target data were selected. The OpenSAR-Ship dataset was used for self-supervised training of MSP-MAE, while the MSTAR and FUSAR-Ship datasets were used for fine-tuning and testing of MSP-CT.

[0064] S2. Data augmentation based on the acquired SAR image data, including data augmentation using random cropping, random horizontal flipping, and Mixup methods;

[0065] like Figure 14 , Figure 15As shown, based on the acquired SAR image data, a portion of the original SAR image is randomly selected for cropping, and then the cropped image is resized to a specific size (e.g., 224×224). During cropping, the cropping area ratio is randomly selected between [0.6, 1.0] to simulate the presentation of targets of different sizes in the image. Simultaneously, the aspect ratio of the cropped image is limited to the range of [3 / 4, 4 / 3] to avoid excessively distorted images. After cropping, the image is resized to the specified size using bicubic interpolation, a method that can preserve image detail to some extent and reduce image distortion. This operation simulates different viewpoints and zoom levels, exposing the model to SAR images at different scales and viewpoints during training. This helps the model learn richer features. Data augmentation is then performed using random horizontal flipping and a mixup method. Random horizontal flipping is applied to the SAR image with a 50% probability. Since the orientation of targets in SAR images is random in real-world scenes, random horizontal flipping increases data diversity, allowing the model to learn the characteristics of targets in different orientations. Finally, the mixup method generates new samples by merging two images and their labels in a certain proportion. In SAR image data augmentation, the probability of performing the mixup operation on each image is set to 50%. Specifically, two different SAR images are linearly combined according to specific weights, and their corresponding labels are also mixed with the same weight proportion. This approach not only increases data diversity but also enables the model to learn transitional features between samples during training, improving the model's robustness. For example, by performing a mixup operation on SAR images of a cargo ship and a fishing boat, the generated new sample contains some features of both types of ships. During the learning process, the model can better understand the differences and relationships between different categories, thus enabling more accurate target recognition when faced with SAR images in complex scenes.

[0066] S3. Construct the MSP-MAE model and use the enhanced image data to perform self-supervised training on the MSP-MAE model to obtain the model weight parameters of MSP-MAE.

[0067] like Figure 1 As shown, the MSP-MAE of this invention is obtained by improving MAE, and the network structure is as follows: Figure 8As shown, MSP-MAE, as a self-supervised learning method, similar to other autoencoder methods, can be divided into two parts: an encoder and a decoder. The encoder in the network is responsible for encoding the image into corresponding tokens and mapping them to a high-dimensional semantic space, while the decoder references these latent representations and reconstructs the original image. In the method of this invention, the OpenSAR-Ship dataset is used as unlabeled data to pre-train the MSP-MAE network. The input image is randomly masked by 75% of its region and then input into the encoder. The decoder then reconstructs the original image based on the latent vectors output by the encoder.

[0068] MSP-MAE is pre-trained using an unlabeled dataset. In this process, 75% of the regions of each input image are randomly masked before being fed into the encoder. The masking process involves first dividing the input image into individual windows; then, obtaining the first sub-window of each window as its index; next, shuffling all the sub-windows to shuffle their order; then, randomly selecting a certain percentage of sub-windows from the shuffled sub-windows as unmasked windows, and the remaining unselected windows as masked windows; finally, all unselected windows are masked, resulting in the masked input data for the model.

[0069] The masked input data is fed into the encoder of the MSP-MAE model. The encoder can infer the features of the masked parts based on the unmasked data. In this process, the encoder can establish contextual relationships by analyzing the information in the unmasked blocks. This allows it to better understand the potential features of the masked blocks, thus enabling accurate reconstruction during decoding. Furthermore, by randomly masking the input parts during training, the encoder learns how to effectively generate high-quality representations even with incomplete information, thereby improving the model's robustness and generalization ability.

[0070] In MSP-MAE, the decoder is responsible for converting the latent representation generated by the encoder back to the original data form. Its main task is to reconstruct the masked parts. It uses contextual information to recover missing details, ensuring that the generated output is as consistent as possible with the original input in terms of features and structure. By continuously optimizing the mapping relationship, the decoder can effectively handle cases of missing information and improve the accuracy of reconstruction.

[0071] MSP-MAE combines a multi-scale convolutional module and a multi-head self-attention module. The multi-scale convolutional module helps the model acquire detailed information about the target, which helps improve the quality of the reconstructed image. The multi-head self-attention module is used to acquire global information about the target, so that the model can better reconstruct the information of the masked block through the visible block. The loss function used for training calculates the mean squared error of the original image and the reconstructed image.

[0072] ,

[0073] in, It is the true value of the image data. The model's predicted value is N, which is the number of samples. The loss calculation only involves masking patches, so that the network training focuses on the prediction of masking patches. The mean squared error of the original image and the reconstructed image is calculated as the loss function, and the completed MSP-MAE model is pre-trained using the loss function.

[0074] like Figure 7 As shown, the encoder of this invention uses the MSP-CT proposed above. Unlike MAE, the window masking layer in MSP-MAE does not remove mask tokens; instead, it directly and uniformly replaces them with learnable vectors, thus maintaining the token count. Deleting these mask tokens would result in insufficient tokens to perform the next downsampling operation (Patch Merging). Under this strategy of not removing mask tokens, relative position encoding is independent in the MSP-CT module, and using both relative and absolute position encoding theoretically does not bring any performance improvement. If mask tokens are not removed, there is no need to add absolute position encoding before the window masking layer. Therefore, there is no need to add a position embedding layer. The only difference between the MSP-MAE encoder and MSP-CT is that in the first stage, this window masking layer is added between the linear embedding layer and the MSP-CT module, masking 75% of the image area.

[0075] Since only encoder weights are used in the transfer learning of downstream tasks, the decoder can be designed flexibly. A lightweight decoder design reduces computation and memory usage, thus reducing training time and allowing for larger batch sizes. Furthermore, it allows the network training to focus more on the encoder, resulting in better transfer learning outcomes in downstream tasks. As mentioned above, since the decoder's reconstruction target is the original pixels, it is sufficient that the final reconstructed size can be resized to the original image size. The decoder design of this invention is as follows: Figure 8 As shown, only three stages are needed to reconstruct the encoder's encoded features to the original image size. Each stage first performs downsampling (PatchExpanding) on ​​the input, doubling the feature size. PatchExpanding consists of a linear layer and a normalization layer. The decoder's prediction head directly uses the predictor prediction layer to map the labeled dimensions to 16.

[0076] The principles of the encoder and decoder for MSP-MAE are explained;

[0077] The MSP-MAE encoder uses the same encoder as MSP-CT. The window masking layer in MSP-MAE does not remove the mask tokens; instead, it uniformly replaces them with learnable vectors, thus maintaining the same number of tokens. MSP-MAE does not require an additional positional embedding layer. The only difference between the MSP-MAE encoder and the MSP-CT encoder is that a window masking layer is added between the Linear Embedding and the encoder in the first stage, masking 75% of the image area.

[0078] The MSP-MAE decoder is designed to be more lightweight. Since the decoder's reconstruction target is the original pixels, it only needs to reshape the final image size to match the original image size. The MSP-MAE decoder has only three stages, each containing an MSP-CT module. After these three MSP-CT modules, the encoder's encoded features can be reconstructed to the original image size. In each stage, the input is first patch expanded to double the feature size. Patch expanding consists of a linear layer and a layer normalization. The decoder's predictor head directly uses the Predictor Projection layer to map the dimension of the label to 16.

[0079] When using the random masking method in MAE, even with a masking rate of 0.75, the small area of ​​each block in MSP-MAE (4×4 size) allows the model to easily reconstruct the mask patch using interpolation. This easily leads to MSP-MAE learning a shortcut solution, as described in the following flowchart. Figure 9 As shown, in the selection of masking ratio, a smaller masking ratio makes the task too simple, thus affecting the learning of higher-dimensional semantic features; conversely, a larger masking ratio makes the task too complex, leading to underfitting during training, resulting in... Figure 10 The image shown is a screenshot of the result.

[0080] S4. Obtain the encoder weight parameters in the trained MSP-MAE model, and use the encoder weight parameters to initialize some weight parameters of the MSP-CT model.

[0081] like Figure 2As shown, after the MSP-MAE training is complete, the decoder part can be discarded since the downstream task mainly uses the encoder's feature extraction capability. The encoder weight parameters are extracted from the trained MSP-MAE model and stored in the model's layers as numerical matrices. Some weight parameters of the MSP-CT model are initialized using the encoder weight parameters. The network structures of the MSP-MAE encoder and the MSP-CT model are analyzed, and corresponding network layers are matched to identify the corresponding network layers. Since the MSP-MAE encoder is based on an improvement of MSP-CT, some network layer structures are similar, and the weight parameters of these similar network layers can be transferred.

[0082] Weight parameter assignment: The weight parameters of the corresponding layers extracted from the MSP-MAE encoder are directly assigned to the corresponding network layers in the MSP-CT model. For example, the weight parameters of a convolutional layer in the MSP-MAE encoder can be directly copied to a structurally similar convolutional layer in the MSP-CT model. In this way, before supervised training begins, some network layers of the MSP-CT model already possess the feature extraction capabilities learned by the MSP-MAE encoder.

[0083] Subsequent training fine-tuning: After weight initialization, the MSP-CT model is trained in a supervised manner using labeled datasets (such as the MSTAR and FUSAR-Ship datasets). During training, the model further adjusts the weight parameters based on the labeled data, enabling the model to better adapt to specific SAR target recognition tasks.

[0084] S5. Fine-tune the parameters of the initialized MSP-CT model and perform supervised training on the MSP-CT model.

[0085] Some weight parameters of the pre-trained MSP-MAE encoder are transferred to the MSP-CT encoder, and then labeled training samples are used to fine-tune the entire MSP-CT network. During this process, the input image data is not masked but directly fed into the MSP-CT encoder; then, the output features of the MSP-CT encoder are passed through a global average pooling layer, and the globally averaged features are input into the classifier for final SAR target classification.

[0086] like Figure 3 As shown, in each iteration, all parameters of the network are updated using the backpropagation algorithm with a loss function. This process minimizes the loss value and optimizes the network. Specifically, the model uses cross-entropy loss to compare the predicted probability of each class with the corresponding true label, as shown in the following formula:

[0087] ,

[0088] in, Indicates the first The true label of each sample The model represents the first The predicted probability of each sample belonging to each category. This indicates the output of the last layer of a given model. Under the given conditions, the predicted probability that a sample belongs to the i-th category is: This is the output result obtained at the last layer after a series of forward propagation calculations of the model. 'c' represents the total number of categories. The Multi-Scale Dilated Convolutional Module (MSDC) proposed in this embodiment consists of three parts: a Multi-Scale Convolutional Fusion Module (MSCF), a Channel Attention Module, and a Spatial Attention Module. The combination of these three structures enables the entire MSDC to effectively fuse and process multi-scale information; none of them can be omitted. The following sections will describe these three modules in detail.

[0089] MSCF: This module uses 3×3 kernel convolutions and, by controlling the dilation rate, constructs convolutions equivalent to 3×3, 5×5, and 7×7 kernels respectively. These three convolutional modules are complementary during feature fusion, thus better characterizing the local texture and global contour information of the target, thereby improving the completeness of the feature representation. A 1×1 kernel branch is also added to this module. The purpose of this branch is to further refine the feature representation with minimal parameters and computational cost. This improves the discontinuity of the convolution center points to some extent, creating conditions for subsequent feature fusion and acting as a booster. The features from the four branches are then concatenated as follows:

[0090] ,

[0091] ,

[0092] in, This is the input feature map, where H and W represent the height and width of the feature map, respectively. This indicates that a convolution with a kernel size of 1×1 is applied to the input feature map. Perform convolution operations. It is a 3×3 dilated convolution with a d-value as the dilation rate. Indicates feature splicing, This indicates a batch normalization operation. The Gaussian error linear unit activation function is used as the basis for this. Following this, a 1×1 convolution kernel is applied to effectively integrate the multi-scale features and compress them by a factor of four. After each convolution, a batch normalization layer and a Gelu activation function are added to accelerate the model's convergence and improve its non-linearity. The overall structure of this module is as follows:

[0093] ,

[0094] Spatial Attention: This spatial attention approach uses both mean and max pooling to converge spatial features, concatenating the two feature sets to obtain a 2×W×H feature map. Then, a 7×7 convolutional kernel is used to compress this feature map, resulting in a 1×W×H feature map. Finally, a sigmoid function is used to activate the feature map, and the activated feature map is multiplied by the initial fused feature. The spatial attention module can be represented as:

[0095] ,

[0096] ,

[0097] in, and These represent average pooling and max pooling operations, respectively. This represents the activation function. Representing a spatial attention map, This represents the spliced ​​feature map. For example... Figure 4 As shown, channel attention: The channel attention module used here is squeeze and activate (SE) attention. SE has two key operations: squeeze and activate. Squeezing is achieved through global average pooling, directly compressing the C×W×H feature map containing global information into a C×1×1 feature vector; activation is achieved through two fully connected layers. The first fully connected layer compresses C channels into C / r channels to reduce computation, and then introduces non-linearity through a ReLU layer. The second fully connected layer restores the number of channels to C channels, and then obtains weights through Sigmoid activation; finally, the attention weights obtained above are weighted onto the features of each channel. The SE module formula is as follows:

[0098] ,

[0099] ,

[0100] ,

[0101] in, This means directly compressing the feature map into a C×1×1 feature vector. This means that the feature vector obtained after the squeezing operation is input into the first fully connected layer. , Indicates The activated feature vectors are then input into the second fully connected layer. , The channel attention weight vector is represented by the following. Based on the effective combination of the above three modules, the overall architecture of the MSDC module is described as follows:

[0102] ,

[0103] in, This indicates that the MSCF module processes the input feature map. To process, This represents the output of the spatial attention module. With the weighting of spatial attention and channel attention, the MSDC module can effectively focus on key information regions of the target while acquiring multi-scale target information, and model the dependencies between these features. This further improves the model's ability to understand complex scenes, thereby enhancing overall performance.

[0104] like Figure 2 As shown, MSP-CT uses multi-head self-attention to model the global information of the target. MHSA allows the model to jointly pay attention to information from different representation subspaces at different locations. Therefore, the MHSA module plays a crucial role in the final performance of the model. In the four stages of MSP-CT, the MHSA module is connected after the MSDC module. The number of MHSA heads is set to 4, 8, 16, and 32 in the four stages, respectively, to further acquire global features, thereby realizing the interaction between local and global information and making the features learned by the model more comprehensive. Specifically, for the input features... First, the input features are enhanced through a fully connected layer to obtain the query q, keyword k, and value v. Then, k is reshaped and multiplied by q to generate an attention map. The dot product of A and v yields attention-weighted features, which are then integrated through a fully connected layer. In summary, attention is defined as:

[0105] ,

[0106] ,

[0107] ,

[0108] in, It outputs the feature map. It is a learnable scaling parameter used to control softmax. This is represented by using a fully connected layer ( ) Input features To process, This is represented as a reshaping operation. Represented as the softmax function, This is represented as calculating attention-weighted features. Represented as another fully connected layer, in order to achieve multi-head attention, q, k and v are split into h heads along the feature channel dimension, and the dimension of each head is P, so individual attention maps can be learned in parallel efficiently.

[0109] like Figure 5 , Figure 6 As shown, FFN consists of two linear layers separated by Gelu activations. The first layer expands the size by a factor of 4, and the second layer shrinks the size by the same ratio.

[0110] ,

[0111] in, and This represents the weights of two linear layers. and It is a bias term. The input feature map, such as Figure 5 The diagram shown is the structure of the MSI-FFN designed in this invention. Unlike the standard FFN, this module consists of two 1×1 convolutional layers and a feature fusion module. The overall structure resembles an inverted residual block. The proposed Multi-Scale Inverted Feedforward Network (MSF-FFN) replaces the FFN structure. This module fuses multi-scale features to obtain more refined features, achieving better performance. This module can be represented as:

[0112] ,

[0113] ,

[0114] ,

[0115] The activation layer and batch normalization are omitted here. This indicates the feature Figure X It integrates multi-scale feature information. Represented as multi-scale feature fusion and residual connection operations, this module helps the model to understand the input features more comprehensively. It inserts a 1×1 convolutional branch to expand the channels by a factor of 3, forming residual connections with the fused features to promote gradient propagation across layers. Finally, a 1×1 convolution compresses the overall features.

[0116] All four stages of MSP-CT employ an alternating stacked structure of MSDC and MHSA modules. MSP-CT uses a 4×4 convolutional kernel with a stride of 4 for Linear Embedding. Except for the first stage, each stage first undergoes Patch Merging for downsampling. Specifically, the overall architecture is described below:

[0117] ,

[0118] ,

[0119] in, It is the input image. This is the output of the previous stage. It is a linear embedding. It is a downsampling operation (Patch Merging). It is the output of each stage. The final output of the model is shown in MHSA, which stands for Multi-Head Attention Block. After four stages, the output features are fed into a global average pooling layer. This layer generates low-dimensional feature vectors by calculating the spatial average of each feature map. This step effectively reduces the number of parameters and computational complexity while preserving important global information. Next, layer normalization is applied to the global average pooled feature vectors to standardize the mean and variance of the input features. This helps alleviate internal covariate bias, accelerates model convergence, and improves training stability. Subsequently, the normalized feature vectors pass through a Dropout layer. Dropout is a regularization technique that reduces overfitting by randomly dropping neurons, enhancing the model's generalization ability. In this stage, a dropout ratio of 0.5 is used to ensure that the model can effectively learn robust feature representations during training. Finally, the dropout-processed feature vectors are fed into a fully connected layer, and the output is the model's classification prediction.

[0120] S6. Use the finely tuned MSP-CT model to perform SAR target recognition, and the output is the recognition result for each test image.

[0121] Test set images from the MSTAR and FUSAR-Ship datasets are used as input data. These test images should retain their original format and size and should not have undergone any special processing such as masking during the training phase, in order to simulate SAR images to be identified in real-world scenarios.

[0122] The prepared test images are input one by one into the fine-tuned MSP-CT model. The model processes the input images according to its predetermined network structure.

[0123] Encoder Feature Extraction: The input image first enters the encoder section of MSP-CT. The encoder extracts features from the image through a series of operations such as convolution and attention mechanisms, including multi-scale dilated convolution (MSDC) and multi-head self-attention (MHSA) modules. These modules can capture local and global feature information from the image, transforming the input image into a feature representation with rich semantics.

[0124] Global pooling: The features output by the encoder are passed through a global average pooling layer. This layer calculates the spatial average of each feature map, transforming the high-dimensional feature maps into low-dimensional feature vectors. This reduces the number of parameters and computational complexity while preserving important global information.

[0125] Normalization: The feature vectors after global pooling undergo layer normalization to standardize the mean and variance of the input features. This helps alleviate the internal covariate bias problem, accelerates model convergence, and improves training stability.

[0126] Dropout Regularization: The feature vectors, after being normalized by the first layer, pass through a Dropout layer. Dropout is a regularization technique that randomly discards neurons during training with a certain probability (e.g., 0.5) to reduce overfitting and enhance the model's generalization ability. During testing, the Dropout layer randomly perturbs the feature vectors, making the model's predictions more robust. After these feature processing steps, the feature vectors are fed into a fully connected layer. The fully connected layer performs a linear transformation and classification prediction based on the input feature vectors, outputting the probability that each test image belongs to each category. These probability values ​​represent the model's confidence in the image belonging to different target categories.

[0127] Based on the probability values ​​output by the fully connected layer, the category with the highest probability is selected as the recognition result for the test image. This yields the target category for each test image. These results can be used to evaluate the model's performance in SAR target recognition tasks, such as calculating accuracy and recall, and can also be directly applied to practical SAR image analysis scenarios, such as military reconnaissance and marine monitoring.

[0128] Experimental verification:

[0129] In the method proposed in this invention, MSP-CT serves as the backbone network of MSP-MAE, and the method is implemented in the PyTorch framework. All experiments use the Adam optimizer. During the pre-training of MSP-MAE, the masking ratio is set to 0.75, the batch size is 32, and the initial learning rate is 0.0001. A cosine annealing strategy is used to gradually reduce the learning rate to 0.000001, for a total of 400 epochs. During the classification training phase of MSP-CT, the batch size is 32, the initial learning rate is 0.0001, and a cosine annealing strategy is used to gradually reduce the learning rate to 0.00001, for a total of 200 epochs. In the experiments, each recognition result is obtained from five trials, and the average of these five trials is taken as the final recognition result. The experiments were tested and evaluated on a server configured with two RTX A4000 processors, running Windows 10 and with 64GB of memory.

[0130] Data augmentation: To make the experiments more convincing, all comparison methods underwent the same data augmentation, including random cropping, random horizontal flipping, and Mixup

[44] . In random cropping, the input image is randomly cropped and then adjusted to a specified target size of 224×224. The cropping area ratio is randomly selected between [0.6, 1.0], and the aspect ratio of the cropped image varies in the range of [3 / 4, 4 / 3]. Finally, the image is adjusted to the specified size using bicubic interpolation. The cropping effect is as follows: Figure 14 As shown, this method can simulate different viewpoints and zoom levels, allowing the model to perform better when faced with images of different scales and viewpoints. This prevents the model from relying solely on certain fixed parts of the image for prediction. Random horizontal flipping increases data diversity by randomly flipping the input image horizontally with a 50% probability. Mixup further increases data diversity and model robustness by mixing two images and their labels in a certain proportion to generate new samples, achieving the desired effect. Figure 15 As shown, the probability of performing a Mixup operation on each image is also 50%. These data augmentation methods are applied to each input sample during training, introducing more variability, which can effectively prevent overfitting and improve the model's performance on different datasets, enabling the model to generalize better when faced with unseen test data.

[0131] In the MSP-MAE reconstruction task, the ideal masking rate should strike a balance between simplicity and complexity, enabling the model to effectively extract complex semantic features without becoming too simplistic and lacking challenge. In this experiment, masking rates of 75%, 50%, and 25% were set and their performance on downstream tasks was tested to find the most suitable masking rate strategy. The experiments were validated on the MSTAR and FUSAR-Ship datasets. On the MSTAR dataset, the recognition performance was verified with 200, 100, 50, 20, and 10 training samples for each class. On the FUSAR-Ship dataset, only the recognition performance with a full dataset was verified.

[0132] like Figure 16 As shown in the figure, the model reconstructs the ship target structure in OpenSAR-Ship under different masking rates. It can be seen from the figure that as the masking rate decreases, the reconstruction effect becomes closer to the real image. At a masking rate of 25%, the model basically reconstructs the detailed structure of the original image. However, as can be seen from the experimental data in Tables 4 and 5, at this masking rate, the model performs worse in downstream recognition tasks than the other two masking strategies. This result indicates that a lower masking rate simplifies the reconstruction task, exposing the model to more raw information during training. This leads to relatively simple features learned by the model, which can only capture low-dimensional or superficial features and cannot deeply understand higher-dimensional semantic features, thus limiting the model's expressive power.

[0133] Table 1. Recognition rates (%) of MSP-MAE at different masking rates on the MSTAR dataset

[0134]

[0135] Results indicate: From Figure 16 As can be seen from the data, when the masking rate is 75%, the model can still reconstruct the general outline of the model, but it cannot reconstruct the detailed features. However, in downstream tasks, the recognition rate of the model under this masking rate condition is greatly improved. As shown in Table 4, as the training data becomes less and less, the recognition performance of the model under the 75% masking rate in downstream tasks is better than the other two masking rate strategies. This indicates that a larger masking rate complicates the task, and the model needs to recover the complete structure from the limited data. This allows the model to learn higher-dimensional semantic features, and the experiments show that these learned high-dimensional features are very beneficial for recognition tasks.

[0136] Experimental verification 2:

[0137] To verify the improvement of MSP-MAE pre-training in cases of low recognition rates due to a lack of training data, this experiment selected a certain number of samples from each of the ten vehicle classes in MSTAR and the four ship classes in FUSAR as training samples to test the recognition performance of each method under different training sample strategies. For MSTAR, the data was the same as in Table 4 above, with 200, 100, 50, 20, and 10 training samples selected for each class; for FUSAR, 150, 75, 20, and 10 training samples were selected for each class.

[0138] The experimental results are shown in Tables 2 and 3. It can be seen that when there are only 10 labeled samples per class, the proposed method achieves an accuracy of 74.33% on the MSTAR dataset and 80.28% on the FUSAR dataset, which is significantly higher than other methods. For example, the MA method achieves accuracies of 70.2% and 74.73% under the same conditions, while DenseNet121 achieves accuracies of 57.69% and 78.08%, respectively. Similarly, when there are only 20 labeled samples per class, the proposed method achieves even higher accuracies of 87.44% and 84.90% on both datasets, which are 8.61% and 3.78% higher than the MAE method's 78.83% and 81.12%, respectively. This fully demonstrates the significant advantage of the proposed method under small sample conditions. It is also worth noting that on the MSTAR dataset, when there are 200 and 100 labeled samples per class, the proposed method still performs excellently, achieving high recognition rates of 99.84% and 99.33%, respectively. In comparison, the untrained MSP-CT also achieves relatively satisfactory recognition results, reaching 99.63% with 200 samples per class, which is excellent but slightly lower than the proposed method. ResNet18 and DenseNet121 also perform well with increasing sample size, but are slightly inferior to the proposed method under small sample conditions. The above experimental results show that the proposed method outperforms other methods in all metrics, demonstrating its potential and superior performance. This also fully demonstrates the advantages of the self-supervised pre-training method in this paper, which can learn effective representations from multi-scale features and, through a masking mechanism, enables the model to learn latent feature representations, effectively improving the model's generalization ability.

[0139] Table 3 shows the recognition rates (%) of each method under different training sample conditions in the MSTAR SOC scenario.

[0140]

[0141] Table 2. Recognition rates (%) of each method under different training sample conditions on the FUSAR dataset.

[0142]

[0143] Results Explanation: For example Figure 17 , Figure 18 As shown, a line graph displays the classification accuracy of different methods on the MSTAR and FUSAR datasets. The horizontal axis represents the number of labeled samples for each class, and the vertical axis represents classification accuracy. The graph includes the performance curves of eight methods, with a magnified view of the upper right corner to show the accuracy differences of each method in more detail. The graph shows that the accuracy of all methods improves with the increase of the number of samples. In the MSTAR dataset, most methods show significant accuracy improvements when the number of samples increases from 10 to 50, and tend to stabilize when the number of samples increases from 100 to 200.

[0144] Example 2

[0145] The difference between this embodiment and Embodiment 1 is that this embodiment provides a self-supervised SAR target recognition system based on the MSP-MAE model, including:

[0146] The data acquisition module is configured to acquire SAR image data for training and testing, including image data acquired through OpenSAR-Ship, MSTAR, and FUSAR-Ship datasets;

[0147] The preprocessing module is configured to perform data augmentation based on the acquired SAR image data, including data augmentation using random cropping, random horizontal flipping, and the Mixup method;

[0148] The model module is configured to: construct an MSP-MAE model and perform self-supervised training on the MSP-MAE model using enhanced image data to obtain the model weight parameters of MSP-MAE;

[0149] The conversion module is configured to: obtain the encoder weight parameters in the trained MSP-MAE model, and initialize some weight parameters of the MSP-CT model using the encoder weight parameters;

[0150] The training module is configured to fine-tune the parameters of the initialized MSP-CT model and perform supervised training on the MSP-CT model.

[0151] The output module is configured to perform SAR target recognition using the fine-tuned MSP-CT model, and the output is the recognition result for each test image.

[0152] A computer-readable storage medium storing a plurality of instructions adapted for loading and execution by a processor of a terminal device of the self-supervised SAR target recognition method based on the MSP-MAE model.

[0153] A terminal device includes a processor and a computer-readable storage medium, the processor being used to implement various instructions; the computer-readable storage medium being used to store multiple instructions, the instructions being adapted to be loaded and executed by the processor to provide a self-supervised SAR target recognition method based on the MSP-MAE model.

[0154] The above are all preferred embodiments of the present invention and are not intended to limit the scope of protection of the present invention. Therefore, all equivalent changes made in accordance with the structure, shape and principle of the present invention should be covered within the scope of protection of the present invention.

Claims

1. A self-supervised SAR target recognition method based on an MSP-MAE model, characterized in that, The application relates to a method for SAR target recognition, and belongs to the field of SAR target recognition. The method comprises the following steps: SAR image data for training and testing is acquired, including acquiring image data through OpenSAR-Ship, MSTAR and FUSAR-Ship public data sets, and performing data enhancement on the acquired SAR image data, including performing data enhancement by means of random cropping, random horizontal flipping and a Mixup method; An MSP-MAE model is constructed, and the MSP-MAE model is self-supervised trained by using the enhanced image data, so as to obtain model weight parameters of the MSP-MAE, and the construction of the MSP-MAE model and the self-supervised training of the MSP-MAE model by using the enhanced image data comprise the following steps: an MSP-MAE model composed of an encoder and a decoder is constructed, the encoder adopts an MSP-CT model encoder, and a window mask layer is added between a linear embedding layer and an MSP-CT module in a first stage; The weight parameters of the encoder in the trained MSP-MAE model are acquired, part of the weight parameters of the MSP-CT model is initialized by using the weight parameters of the encoder, the initialized MSP-CT model is fine-tuned, the MSP-CT model is supervised trained, and finally the SAR target recognition is performed by using the fine-tuned MSP-CT model; The MSP-CT model and the MSP-MAE model both comprise an MSP-CT module, the supervised training of the MSP-CT model comprises the following steps: the four stages of the MSP-CT adopt an alternating stacking structure of an MSDC layer and an MHSA layer, a linear embedding operation is performed through a convolution kernel, each stage except the first stage is down-sampled, a multi-scale inverted feedforward network MSF-FFN composed of a convolution layer and a feature fusion layer is used as a feedforward network, and the MSF-FFN is specifically as follows: The MSDC layer is composed of the following three parts: a multi-scale convolution fusion module MSCF, a channel attention module and a spatial attention module; ， ， wherein, is an input feature map, H and W represent the height and width of the feature map respectively, represents a convolution operation on the input feature map with a convolution kernel size of 1x1, is a dilated convolution with a convolution kernel of 3x3, and d is the dilation rate, represents feature splicing, represents a batch normalization operation, represents a Gaussian error linear unit activation function; then the multi-scale features are effectively integrated by a convolution with a convolution kernel of 1x1, and compressed by four times, and a batch normalization layer and a Gelu activation function are added after each convolution, and the overall structure of the MSCF module is as follows: ； The MSCF module uses a convolution kernel with a size of 3*3, and three convolutions are constructed by controlling the expansion rate; in addition, a parallel branch with a convolution kernel size of 1*1 is additionally added in the module, and finally the features of the four branches are spliced, as follows: ， in, This indicates that the MSCF module processes the input feature map. To process, This represents the output of the spatial attention module; The overall architecture of the MSDC module is described as follows: ， ， ， wherein the activation layer and batch normalization are omitted, indicates that the feature map X integrates multi-scale feature information, indicates a multi-scale feature fusion and residual connection operation; the module inserts a 1x1 convolution branch to expand the channel by 3 times to form a residual connection with the fused features, and finally compresses the overall features through a 1x1 convolution; Specifically, the overall architecture of the MSP-CT model is described as follows: ， ， wherein, is the input image, is the output of the previous stage, is a linear embedding, is a down-sampling operation, MHSA stands for multi-head self-attention, is the output of each stage, is the final output of the model.

2. The self-supervised SAR target recognition method based on the MSP-MAE model according to claim 1, characterized in that, The MSF-FFN module is represented as follows: The decoder adopts a lightweight decoder composed of multiple stages, each stage first expands the input features by two times through a Patch Expanding layer composed of a linear layer and a normalization layer, maps the label dimension by using a prediction head, and is used for reconstructing the original image according to the latent vector output by the encoder, the mean square error between the original image and the reconstructed image is calculated as a loss function for the prediction error of the masked part in the image, and the pre-training of the constructed MSP-MAE model is performed by using a back propagation algorithm.

3. The self-supervised SAR target recognition method based on the MSP-MAE model according to claim 1, characterized in that, The parameter fine-tuning of the initialized MSP-CT model comprises inputting complete unmasked image data into the assigned MSP-CT model encoder to obtain MSP-CT encoder output features, performing global pooling on the output features, and inputting the pooled feature vectors into a classifier for classification.

4. The self-supervised SAR target recognition method based on the MSP-MAE model according to claim 1, characterized in that, The supervised training of the MSP-CT model further comprises using a multi-scale inverted feedforward network composed of a convolutional layer and a feature fusion layer as a feedforward network, then inserting a convolutional branch to form a residual connection, and outputting features in sequence through a global average pooling layer, a layer normalization layer, a Dropout layer and a fully connected layer, outputting a classification prediction value from the fully connected layer, comparing the prediction probability of each class and the corresponding true label through a cross-entropy loss function, and finally training the MSP-CT model through a back propagation algorithm.

5. A self-supervised SAR target recognition system based on MSP-MAE model, performing the method of claim 1, characterized in that, The method comprises: a data acquisition module configured to acquire SAR image data for training and testing, including acquiring image data through OpenSAR-Ship, MSTAR and FUSAR-Ship data sets; a preprocessing module configured to perform data augmentation based on the acquired SAR image data, including using random cropping, random horizontal flipping and Mixup method for data augmentation; a model module configured to construct a MSP-MAE model and perform self-supervised training on the MSP-MAE model using the enhanced image data to obtain the model weight parameters of the MSP-MAE; a conversion module configured to acquire the encoder weight parameters of the trained MSP-MAE model and use the encoder weight parameters to initialize part of the weight parameters of the MSP-CT model; a training module configured to fine-tune the initialized MSP-CT model and perform supervised training on the MSP-CT model; an output module configured to use the fine-tuned MSP-CT model for SAR target recognition, and the output is the recognition result of each test image.

6. A computer readable storage medium having stored therein a plurality of instructions, wherein the instructions, when executed by a processor, cause the processor to perform the method of any one of claims 1-5. The instructions are adapted to be loaded and executed by the processor of the terminal device to perform the self-supervised SAR target recognition method based on the MSP-MAE model. 7.A terminal device, comprising a processor and a computer readable storage medium, the processor is configured to implement instructions; the computer readable storage medium is configured to store a plurality of instructions, characterized in that, The instructions are adapted to be loaded and executed by the processor to perform the self-supervised SAR target recognition method based on the MSP-MAE model.

Images