SAR (Synthetic Aperture Radar) image target detection method and device based on combination of Mangbar and diffusion models

Abstract

The invention relates to an SAR (Synthetic Aperture Radar) image target detection method and device based on the combination of a Mangbar and a diffusion model, and provides a target detection model taking a Resnet50-FPN network as a framework, the Resnet50 network is utilized to extract a plurality of resolution features of an SAR sample image, the extracted features are processed by a Mangbar SAR unit to obtain complete semantic features, and the complete semantic features are extracted by a Mangbar SAR unit. Then, a feature pyramid is constructed from top to bottom through the processed features by adopting an FPN network, feature images of multiple scales are obtained, Gaussian noise controlled by variance scheduling is added into bounding box truth value data through a diffusion model, regions of interest are cut out from the feature images of the multiple scales through the obtained noise boxes, and the feature images of the multiple scales are obtained; and the dynamic detection head obtains prediction data of each target in the SAR sample image according to the feature image of the region of interest, then the true value data and the prediction data are adopted to calculate a loss function, and learnable parameters are adjusted according to a calculation result until convergence. When the method is used for target detection, the accuracy of a detection result can be effectively improved.

Description

Technical Field

[0001] The present application relates to the technical field of SAR image target detection, and in particular to a SAR image target detection method and device based on a combination of a Mamba and diffusion model. Background Art

[0002] Synthetic Aperture Radar (SAR), an active microwave remote sensing imaging radar capable of acquiring images in all weather and at all times, has been widely used in fields such as topographic mapping, disaster risk monitoring, and ecology. SAR target detection, a crucial foundation for SAR image interpretation, aims to quickly and efficiently extract the direction and position of targets and identify their types within complex scenes. This provides strong support for applications such as dynamic monitoring of critical areas, scenario analysis, and emergency rescue. Its detection accuracy also directly determines a range of subsequent projects.

[0003] However, due to its unique imaging mechanism, SAR images only contain echo information from a single band, compared to optical images. Furthermore, SAR imaging performance is closely related to the target medium, moisture content, and roughness, factors that can interfere with SAR image target detection tasks to varying degrees. Summary of the Invention

[0004] Based on this, it is necessary to provide a SAR image target detection method and device based on the combination of Mamba and diffusion models, which can improve the target detection accuracy, in order to address the above technical problems.

[0005] A SAR image target detection method based on a combination of a Mamba and a diffusion model, the method comprising:

[0006] Acquire a training data set, wherein the training data set includes multiple groups of training data, each group of training data includes a SAR sample image and category truth values ​​and bounding box truth value data of all targets in the sample image;

[0007] Inputting a set of SAR sample images in the training data into the target detection model, the target detection model uses the Resnet50-FPN network as a framework, extracting multiple resolution features from the SAR sample images using the Resnet50 network, processing the extracted features through the Mamba SAR unit to obtain complete semantic features, and then constructing a feature pyramid from top to bottom using the FPN network to obtain feature images at multiple scales;

[0008] The diffusion model is used to add variance-controlled Gaussian noise to the ground truth bounding box data in the training data to obtain a noise box.

[0009] Using the noise frame, cropping regions of interest from feature images at multiple scales to obtain feature images of the region of interest, and using a dynamic detection head to obtain predicted categories and bounding box prediction data of each target in the SAR sample image based on the feature images of the region of interest;

[0010] Calculating a loss function based on the predicted category, boundary prediction data, and corresponding category true value and bounding box true value data, adjusting the learnable parameters in the target detection model based on the calculation results, and training the target detection model using the training data set until convergence, thereby obtaining a trained target detection model;

[0011] Obtain a SAR image for target detection, input the SAR image into the trained target detection model, and obtain the categories and positions of all targets in the SAR image.

[0012] In one embodiment, it is characterized in that, in the target detection model:

[0013] The output of the Resnet50 network includes multiple layers of feature images with resolutions from low to high, and the Mamba SAR unit is connected between two adjacent layers of feature images in the middle layer;

[0014] The feature image with lower resolution in the two adjacent layers of feature images is processed by the Mamba SAR unit, fused with the feature image with higher resolution, and then output to the FPN network as a higher-layer feature image.

[0015] In one embodiment, the output of the Resnet50 network includes four layers of feature images with resolutions from low to high, wherein the third layer of feature images is processed by the Mamba SAR unit and fused with the fourth layer of feature images, and then output to the FPN network as the fourth layer of feature images.

[0016] In one embodiment, the Mamba SAR unit includes a plurality of Mamba blocks and a plurality of smart attention blocks connected in sequence, wherein each of the Mamba blocks and the smart attention block is further connected to a multilayer perceptron;

[0017] In the Mamba SAR unit, the input feature image is processed by the adaptive window scratch mechanism, and then processed by the Mamba block and the intelligent attention block in sequence and merged to obtain the output feature image.

[0018] In one embodiment, the Mamba block includes two asymmetric branches, one of which includes a linear layer, a one-dimensional convolution layer, and an activation function layer in sequence, and the other branch includes a linear layer, a one-dimensional convolution layer, an activation function layer, and a Mamba scan layer in sequence;

[0019] The image of the input mamba block is processed by the two branches respectively, spliced ​​through the splicing layer and then passed through the linear layer to obtain the image of the output mamba block.

[0020] In one embodiment, in the Mamba scanning layer:

[0021] Expanding the input Mamba scan layer image into a plurality of sub-images using a scan expansion operation, wherein each sub-image is obtained by scanning the input image from a different angle;

[0022] The S6 block operation is used to extract features from each sub-image, and the extracted features are merged by scanning to obtain an image of the output Mamba scan layer having the same size as the input image.

[0023] In one embodiment, the true value data of the bounding box includes the center coordinates, width and height of the bounding box.

[0024] In one embodiment, the diffusion model includes a forward diffusion part and a reverse diffusion part;

[0025] Using the forward diffusion part, adding variance-controlled Gaussian noise to the true value data of the bounding box to obtain the noise box;

[0026] The reverse diffusion part is used as the dynamic detection head to perform target detection.

[0027] The present application also provides a SAR image target detection device based on a combination of a Mamba and a diffusion model, the device comprising:

[0028] A training data set acquisition module is used to acquire a training data set, wherein the training data set includes multiple sets of training data, each set of training data includes a SAR sample image and the category truth value and bounding box truth value data of all targets in the sample image;

[0029] A multi-scale feature acquisition module is used to input a set of SAR sample images in the training data into the target detection model. The target detection model uses the Resnet50-FPN network as a framework, extracts multiple resolution features from the SAR sample images using the Resnet50 network, processes the extracted features using the Mamba SAR unit to obtain complete semantic features, and then constructs a feature pyramid from top to bottom using the FPN network to obtain feature images at multiple scales.

[0030] A noise adding module, configured to add variance-controlled Gaussian noise to the ground truth data of the bounding box in the set of training data using a diffusion model to obtain a noise box;

[0031] a dynamic detection head prediction module, configured to use the noise frame to crop regions of interest from feature images at multiple scales to obtain feature images of the region of interest, and use the dynamic detection head to obtain predicted categories and bounding box prediction data for each target in the SAR sample image based on the feature images of the region of interest;

[0032] An object detection model training module is configured to calculate a loss function based on the predicted category, boundary prediction data, and corresponding category true value and bounding box true value data, adjust the learnable parameters in the object detection model based on the calculation results, and train the object detection model using the training data set until convergence to obtain a trained object detection model;

[0033] The target detection module is used to obtain a SAR image to be subjected to target detection, input the SAR image into the trained target detection model, and obtain the categories and positions of all targets in the SAR image.

[0034] A computer device includes a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the following steps are implemented:

[0035] Acquire a training data set, wherein the training data set includes multiple groups of training data, each group of training data includes a SAR sample image and category truth values ​​and bounding box truth value data of all targets in the sample image;

[0036] Inputting a set of SAR sample images in the training data into the target detection model, the target detection model uses the Resnet50-FPN network as a framework, extracting multiple resolution features from the SAR sample images using the Resnet50 network, processing the extracted features through the Mamba SAR unit to obtain complete semantic features, and then constructing a feature pyramid from top to bottom using the FPN network to obtain feature images at multiple scales;

[0037] The diffusion model is used to add variance-controlled Gaussian noise to the ground truth bounding box data in the training data to obtain a noise box.

[0038] Using the noise frame, cropping regions of interest from feature images at multiple scales to obtain feature images of the region of interest, and using a dynamic detection head to obtain predicted categories and bounding box prediction data of each target in the SAR sample image based on the feature images of the region of interest;

[0039] Calculating a loss function based on the predicted category, boundary prediction data, and corresponding category true value and bounding box true value data, adjusting the learnable parameters in the target detection model based on the calculation results, and training the target detection model using the training data set until convergence, thereby obtaining a trained target detection model;

[0040] Obtain a SAR image for target detection, input the SAR image into the trained target detection model, and obtain the categories and positions of all targets in the SAR image.

[0041] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements the following steps:

[0042] Acquire a training data set, wherein the training data set includes multiple groups of training data, each group of training data includes a SAR sample image and category truth values ​​and bounding box truth value data of all targets in the sample image;

[0043] Inputting a set of SAR sample images in the training data into the target detection model, the target detection model uses the Resnet50-FPN network as a framework, extracting multiple resolution features from the SAR sample images using the Resnet50 network, processing the extracted features through the Mamba SAR unit to obtain complete semantic features, and then constructing a feature pyramid from top to bottom using the FPN network to obtain feature images at multiple scales;

[0044] The diffusion model is used to add variance-controlled Gaussian noise to the ground truth bounding box data in the training data to obtain a noise box.

[0045] Using the noise frame, cropping regions of interest from feature images at multiple scales to obtain feature images of the region of interest, and using a dynamic detection head to obtain predicted categories and bounding box prediction data of each target in the SAR sample image based on the feature images of the region of interest;

[0046] Calculating a loss function based on the predicted category, boundary prediction data, and corresponding category true value and bounding box true value data, adjusting the learnable parameters in the target detection model based on the calculation results, and training the target detection model using the training data set until convergence, thereby obtaining a trained target detection model;

[0047] Obtain a SAR image for target detection, input the SAR image into the trained target detection model, and obtain the categories and positions of all targets in the SAR image.

[0048] The above-mentioned SAR image target detection method and device based on the combination of Mamba and diffusion models trains the target detection model using a training dataset. The target detection model uses the Resnet50-FPN network as the framework, extracting features at multiple resolutions from the SAR sample image using the Resnet50 network. The extracted features are processed by the Mamba SAR unit to obtain complete semantic features. The processed features are then used to construct a feature pyramid from top to bottom using the FPN network to obtain feature images at multiple scales. The diffusion model then adds variance-controlled Gaussian noise to the ground truth bounding box data in the training data to obtain noise boxes. The noise boxes are then used to crop regions of interest from the feature images at multiple scales to obtain region-of-interest feature images. A dynamic detection head is used to obtain the predicted category and bounding box prediction data for each target in the SAR sample image based on the region-of-interest feature images. The loss function is then calculated using the ground truth data and the predicted data, and the learnable parameters in the model are adjusted based on the calculation results until convergence. This method can effectively improve the accuracy of target detection results. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] Figure 1 1 is a flow chart of a SAR image target detection method based on a combination of a Mamba and diffusion models in one embodiment;

[0050] Figure 2 Schematic diagram of the structure of an object detection model in one embodiment;

[0051] Figure 3 Schematic diagram of a denoising process of a diffusion model in one embodiment;

[0052] Figure 4 Schematic diagram of the structure of a Mamba SAR unit in one embodiment;

[0053] Figure 5 A schematic diagram of the structure of a Mamba block in one embodiment;

[0054] Figure 6 Schematic diagram of the structure of a Mamba scanning layer in one embodiment;

[0055] Figure 7 Schematic diagram of an attention mechanism in one embodiment, where: Figure 7 (a) is a schematic diagram of the softmax attention mechanism. Figure 7 (b) is a schematic diagram of the intelligent attention mechanism;

[0056] Figure 8 1 is a structural block diagram of a SAR image target detection device based on a combination of a Mamba and diffusion model in one embodiment;

[0057] Figure 9FIG. 1 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0058] In order to make the purpose, technical solutions and advantages of this application more clear, the following further describes this application in detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain this application and are not intended to limit this application.

[0059] Currently, SAR target detection faces challenges including diverse target scales and types, discrete and discontinuous features, and complex background interference.

[0060] The diversity of target scales and types refers to the fact that in SAR images, different types of ships and aircraft, as well as image resolutions, can lead to differences in target scale. When faced with multi-scale targets, feature extraction using fixed prior bounding boxes and fixed receptive fields cannot meet detection requirements. As the network becomes deeper, ships with weak scattering information or small pixels are more likely to be missed. To ensure effective target detection and recognition, a more sophisticated network structure design is required.

[0061] Discontinuous features refer to the fact that SAR images are composed of bright pixels caused by strong scattering from various components of targets such as ships and aircraft, and dark pixels caused by weak scatterers in smooth areas (such as ship hulls and airport lawns). Weak correlation between target structures leads to incomplete image features, blurred contour information, and difficulty distinguishing adjacent targets. Therefore, accurately representing the target's global feature information as well as local texture details helps the network learn more discriminative intra-class features, which is beneficial for improving the fine-grained recognition accuracy of the detection algorithm.

[0062] Among them, complex background interference refers to the fact that due to the uniqueness of SAR detection wavelength and imaging mechanism, near-shore buildings and metal facilities near airports will produce strong echoes in SAR images, and strong scattering areas such as pseudo-ships and pseudo-aircraft will appear in SAR images. These cluttered backgrounds pose a major challenge to the correct detection of targets.

[0063] In the existing technology, deep learning-based target detection algorithms for SAR images have made significant progress. Although existing algorithms have alleviated the performance degradation caused by the diversity of target sizes to a certain extent, they still require the use of FPN (Feature Pyramid Network) or prior assumptions to generate region proposal boxes, which have problems such as high complexity and reliance on fixed learnable queries. On the other hand, the commonly used convolutional neural networks are mainly based on convolution kernels to obtain local receptive fields, and the computational complexity of the self-attention mechanism increases linearly with the length of the context, resulting in very low computational efficiency.

[0064] In response to the above challenges and problems existing in the prior art, in this application, Figure 1 As shown, a SAR image target detection method based on the combination of Mamba and diffusion models is provided, which specifically includes the following steps:

[0065] Step S100 : obtaining a training data set, which includes multiple groups of training data. Each group of training data includes a SAR sample image and category truth values ​​and bounding box truth value data of all targets in the sample image.

[0066] In step S110, a SAR sample image in a set of training data is input into a target detection model. The target detection model uses the Resnet50-FPN network as a framework, and uses the Resnet50 network to extract multiple resolution features from the SAR sample image. The extracted features are processed by the Mamba SAR unit to obtain complete semantic features. The processed features are then constructed into a feature pyramid from top to bottom using the FPN network to obtain feature images at multiple scales.

[0067] Step S120 , using a diffusion model to add variance-controlled Gaussian noise to the true value data of the bounding box in the set of training data to obtain a noise box.

[0068] In step S130 , the noise frame is used to crop the regions of interest from the feature images at multiple scales to obtain feature images of the regions of interest. The dynamic detection head is used to obtain the predicted category and bounding box prediction data of each target in the SAR sample image based on the feature images of the regions of interest.

[0069] In step S140, a loss function is calculated based on the predicted category, boundary prediction data, and the corresponding category true value and bounding box true value data. The learnable parameters in the target detection model are adjusted according to the calculation results. The target detection model is trained using the training data set until convergence to obtain a trained target detection model.

[0070] Step S150 , obtaining a SAR image to be subjected to target detection, inputting the SAR image into a trained target detection model, and obtaining the categories and positions of all targets in the SAR image.

[0071] In this application, a diffusion detection network based on Mamba is proposed to capture complete and rich target features in a simpler and more flexible manner. This is the first algorithm to combine Mamba and diffusion models for target detection in SAR images. By combining the advantages of state-space models and diffusion-based detectors, the model can dynamically capture global contextual information and local features in complex backgrounds, enhancing the semantic information of the target while suppressing background interference, thereby improving the model's target detection accuracy.

[0072] Specifically, to address the multi-scale problem of multi-type objects, the object detection model in this paper adopts the main architecture of a diffusion-based detector, mapping the detection task into a denoising and refinement process from random noise boxes to the target ground truth boxes. Object detection is described as the task of generating the position (center coordinates) and size (width and height) of the object bounding box in the image in space, without the need for heuristic target priors or fixed learning boxes.

[0073] Furthermore, the above-mentioned steps S100 to S140 are steps for training the target detection model, wherein step 100 describes the input data and true value data of the model during training, steps S110 to S140 further illustrate the model structure by processing the input data, i.e., the SAR sample image in the model, and step S150 is the actual application step of the trained target detection model.

[0074] In step S100, a training dataset is obtained, including multiple training data sets. Each training data set includes a SAR sample image as model input, as well as target category truth and bounding box truth data as ground truth labels. In other words, the target detection model proposed in this application can detect the category and location of targets in SAR images. At the same time, the targets in each SAR sample image can be one or more different types of targets. The corresponding ground truth labels include the type and bounding box data of each target in the SAR sample image, where the bounding box truth data includes the center point coordinates, width, and height of the target bounding box.

[0075] Specifically, when constructing a training data set, the target type in the SAR sample image is selected according to the application scenario of the trained target detection model.

[0076] like Figure 2 As shown in the figure, the architecture of the target detection model, namely MaDiNet, is shown. The target detection model is divided into a denoising part based on a diffusion model and a backbone network including MambaSAR.

[0077] Specifically, in the backbone network, i.e., step S120, following the overall design of Resnet50-FPN, deformable convolution is used at the high level to extract features from SAR sample images, flexibly capturing target characteristics at any scale. At the core, the MambaSAR unit is used to capture richer contextual information. Features are first downsampled by the MambaSAR unit, global features are obtained through the Mamba block, and then local key features are focused on by the intelligent attention block. These features are then added to the intermediate layer features to form new high-dimensional image features. The neck portion adopts an FPN design, and a scattering feature enhancement module containing PDC (Pixel Difference Convolution) is used to further suppress background interference.

[0078] Furthermore, in the object detection model, the Resnet50 network outputs multiple layers of feature images, ranging from low to high resolution. The Mamba SAR unit is connected between two adjacent layers of feature images in the middle layer. The lower-resolution feature image in the two adjacent layers is processed by the Mamba SAR unit and fused with the higher-resolution feature image before being output as the higher-resolution feature image to the FPN network.

[0079] Preferably, the output of the Resnet50 network includes four layers of feature images with resolutions from low to high, wherein the third layer of feature images is processed by the Mamba SAR unit, fused with the fourth layer of feature images, and then output to the FPN network as the fourth layer of feature images.

[0080] Specifically, the diffusion-based box diffusion and denoising component is based on the denoising diffusion implicit model (DDIM). In step S130, object detection is described as a task of generating the position (center coordinates) and size (width and height) of the object bounding box in the image in space, without the need for heuristic object priors or fixed learned boxes. The extracted features are fed into the detection head along with the randomly generated boxes by diffusion to obtain detection results with fine-grained recognition.

[0081] In this embodiment, the learning target of the target detection model is the input-target pair, i.e., a set of training data, denoted as (x, b, c), where x is the input image, i.e., the SAR sample image, and b and c are a set of bounding boxes and category labels of the objects in image x, respectively. More specifically, the i-th box in the set is denoted as bi = (c i x ,c i y ,w i ,h i ), where (c ix ,c i y ) is the center coordinate of the bounding box, (w i ,h i ) are the width and height of the bounding box respectively.

[0082] Furthermore, considering the wide variety and large variance of target scales, we introduce diffusiodet, which requires a heuristic object prior, for SAR target detection. We formulate SAR target detection as the task of spatially generating the position (center coordinates) and size (width and height) of bounding boxes in an image. This framework does not require a candidate box prior, alleviating the problems of inaccurate positioning and missed detections caused by aircraft scale variations and manual configuration errors.

[0083] In this embodiment, the training data is a set of bounding boxes z0=b, where b∈R N×4 It is a set of N boxes (the four dimensions represent the center coordinates and width and height of the box respectively). θ (zt, t, x) from the noise box z t In the prediction z0 and the corresponding category label c are generated.

[0084] like Figure 3 As shown in Figure 1, during the training phase, variance-controlled Gaussian noise is added to the ground truth box to obtain a noise box. These noise boxes are then used to crop the RoI (region of interest) features from the shallow feature map and the enhanced high-level semantic features, respectively. Finally, these RoI features are sent to the detection decoder, which is trained to predict the noise-free ground truth box, i.e., step S130. In the test phase, MaDiNet generates bounding boxes through the diffusion process learned by the reverse process of the diffusion model, which adjusts the noise prior distribution to the distribution of the learned bounding box.

[0085] Specifically, when the diffusion model is used to add noise to the true value frame, the noise is actually added based on the coordinates of the four intersection points of the true value frame.

[0086] In this embodiment, the diffusion model includes a forward diffusion part and a reverse diffusion part. The forward diffusion part is used to add variance-controlled Gaussian noise to the true value data of the bounding box to obtain a noise box, and the reverse diffusion part is used as a dynamic detection head for target detection.

[0087] Specifically, when the trained target detection model performs target detection on SAR images, the forward diffusion part of the diffusion model adds multiple bounding boxes to the feature images at multiple scales, and then the reverse diffusion part obtains accurate bounding boxes during the denoising process.

[0088] In step S140, when training the target detection model, the loss function is calculated based on the predicted category, boundary prediction data, and the corresponding category true value and bounding box true value data.

[0089] In this embodiment, the data sample is a set of bounding boxes z o =b, where b∈R N×4 It is a set of N boxes (the four dimensions represent the center coordinates and width and height of the box respectively). θ (z t ,t,x) by minimizing the training loss From the noise box z t In the prediction z0 and the corresponding category label c are generated.

[0090]

[0091] Next, the Mamba SAR unit in the target detection model is explained.

[0092] In order to obtain a complete semantic structure representation of the target, a mambaSAR unit was constructed to improve the network's ability to extract local and global features of the target. Considering that the complex background in SAR images is distributed in any spatial direction of the target, an omnidirectional selective scanning mechanism is designed in the MambaSAR unit to globally model the feature distribution of images in multiple directions and capture rich spatial feature information from different directions. On the other hand, since Mamba compresses the filtered global context into a hidden state to provide linear complexity, the compression inevitably leads to the loss of fine-grained local dependency information between tokens. Therefore, an attention structure is constructed in the Mamba backend to better model global information by focusing on different positions in the image, so that the model can learn more refined representations, thereby improving the model's ability to distinguish targets and complex backgrounds.

[0093] like Figure 4 As shown, the Mamba SAR unit consists of multiple Mamba blocks and multiple smart attention blocks connected in sequence. Each Mamba block and smart attention block is also connected to a multi-layer perceptron. In the Mamba SAR unit, the input feature image is processed by the adaptive window scratch mechanism, then processed by the Mamba blocks and smart attention blocks, and then merged to produce the output feature image.

[0094] Specifically, the Mamba SAR unit is a hybrid architecture consisting of a Mamba component (i.e., multiple sequentially connected Mamba blocks) and a Transformer component (i.e., multiple sequentially connected intelligent attention blocks). Furthermore, an adaptive window scratching mechanism is employed within the unit to better adapt to SAR images of various sizes. Experimental results and analysis demonstrate that the use of several self-attention layers in the final stage significantly enhances the ability to capture global context and long-range spatial dependencies.

[0095] Preferably, if we assume that the input The sequence length is T, the embedding dimension is C, and 6 layers are designed in the Mamba SAR unit. The output of layer n can be calculated as:

[0096] x n =MambaBlock(Norm(x n-1 ))+x n-1 ,n=0,1,2,

[0097] x m =AttentionBlock(Norm(x m-1 ))+x m-1 ,n=3,4,5,

[0098] x i =MLP(Norm(x i ))+x i ,i=0,1...5.

[0099] In the above formula, MambaBlock represents Mamba block, AttentionBlock represents intelligent attention block, and MLP represents multi-layer perceptron.

[0100] Considering that the essence of SAR imaging is the mapping from the target scattering characteristic space to the target image space, in this embodiment, the original Mamba block is redesigned to make it more suitable for SAR target detection tasks, such as Figure 5 As shown, the Mamba block includes two asymmetric branches, one of which includes a linear layer, a one-dimensional convolution layer, and an activation function layer in sequence, and the other branch includes a linear layer, a one-dimensional convolution layer, an activation function layer, and a Mamba scanning layer in sequence. The image input to the Mamba block is processed by the two branches respectively, spliced ​​through the splicing layer, and then passed through the linear layer to obtain the output image of the Mamba block.

[0101] Specifically, in the Mamba block, a regularized convolution is proposed to replace the causal convolution, which restricts the influence to a single direction, which is unnecessary and restrictive for SAR target detection tasks. Secondly, considering the complex target structure in SAR images and the extreme sensitivity of the scattering center intensity to azimuth, a scanning mechanism with four different scanning directions is designed to globally model the features of images in multiple directions and capture rich spatial feature information from different directions.

[0102] Furthermore, targets in SAR images are discontinuous, consisting of multiple discrete, irregularly scattered bright spots. However, these scattering centers contain key semantic information for target recognition. Therefore, combining these discrete bright spots into a cohesive whole is crucial for target detection and recognition. To this end, a symmetric branch without the SAR-SSM (Mamba Scan layer) is added to the Mamba block. This branch consists of additional convolutions and SiLU activations to compensate for the content lost due to the sequential constraints of the SAR-SSM.

[0103] The outputs of the two branches are then concatenated and projected into the final linear layer. This combination ensures that the final feature representation contains both sequence and spatial information, leveraging both branches at the same time.

[0104] Furthermore, given input x in , Mamba Block x out The output will be calculated according to the following formula:

[0105]

[0106] In the above formula, Linear(C in , C out )(·) indicates C in and C out A linear layer with the same input and output embedding dimensions. SARScan represents a multi-directional selective scanning operation, also known as a Mamba scan layer. σ represents the activation function, using the Sigmoid Linear Unit (SiLU). Furthermore, Conv and Concat represent one-dimensional convolution and concatenation operations.

[0107] Due to the complex structure of the target, different parts of the target exhibit different scattering mechanisms, including cavity scattering and edge diffraction. During SAR imaging, the target's scattering varies to varying degrees with changes in scattering conditions such as the angle of incidence and azimuth, resulting in highly variable scattering results and difficulty in determining the target type. To address this issue, a SAR-SSM scan merging mechanism was designed within the Mamba block to capture the complete target's features from multiple directions and angles.

[0108] In this embodiment, in the Mamba scan layer: a scan expansion operation is first used to expand the input Mamba scan layer image into multiple sub-images, each sub-image is obtained by scanning the input image from different angles, and then an S6 blocking operation is used to extract features from each sub-image. The extracted features are merged through scanning to obtain an output Mamba scan layer image with a size consistent with the input image.

[0109] like Figure 6 As shown in FIG, the Mamba scanning layer includes three main steps: scan expansion, S6 blocking and scan merging.

[0110] Specifically, the scan-and-expand operation expands the input image into a series of sub-images, each representing a specific orientation. When viewed diagonally, the scan-and-expand operation is performed along four symmetrical directions: top-to-bottom, bottom-to-top, left-to-right, and right-to-left. This layout not only comprehensively covers all regions of the input image but also provides a rich multidimensional information base for subsequent feature extraction through systematic directional transformations. This improves the efficiency and comprehensiveness of multidimensional image feature capture and effectively alleviates the orientation sensitivity of targets in SAR images.

[0111] Specifically, in the S6 block operation, feature extraction is performed on the above sub-images, and finally the sub-images are reassembled together in the order of the sequence blocks through a scan-and-merge operation to form an output image of the same size as the input image.

[0112] In the Mamba SAR unit, the Mamba architecture compresses the filtered global context into hidden states to achieve linear complexity. This compression inevitably results in the loss of fine-grained local dependencies between tokens, affecting SAR target detection accuracy. To address this, an attention architecture is built into the backend of the Mamba architecture. This architecture enables the model to filter important information based on the input content, focusing on different locations in the image to better model global information. This allows the model to learn more refined representations, thereby improving its ability to distinguish targets from complex backgrounds.

[0113] Considering that commonly used Softmax and linear attention methods suffer from either excessive computational complexity or insufficient model expressiveness, this embodiment introduces agent attention, leveraging the redundancy between attention weights to achieve high model expressiveness and low computational complexity. Agent attention effectively forms an elegant integration of Softmax and linear attention, benefiting from both linear complexity and high expressiveness.

[0114] Among them, Softmax attention is expressed as:

[0115]

[0116] In the formula, Q, K, V represent query, key, and value, and d is the number of attention heads.

[0117] And intelligent attention is expressed as:

[0118] AgentAtten(Q,A,K,V)=Atten(Q,A,Atten(A,K,V))

[0119] In the above formula, Denote the newly defined smart token. First, the proxy token A is treated as the query, and attention computation is performed between A, K, and V to aggregate the proxy features A from all values. Then, in a second attention computation on the query matrix Q, using A as the key and V A as the value, the global information of the proxy features is broadcast to each query token, resulting in the final output o. This way, while avoiding the computation of similarity between Q and K, the information exchange between each query-key pair is preserved through the proxy token.

[0120] The difference between softmax attention and smart attention is that softmax attention calculates the similarity between all query key pairs, resulting in quadratic complexity. Smart attention utilizes a small group of agent tokens to aggregate and broadcast global information, thus elegantly integrating softmax and linear attention, and naturally enjoying the advantages of high expressiveness and low computational complexity. Where N is the initial baseline of computational complexity. Figure 7 shown.

[0121] Among the aforementioned SAR image target detection methods based on the combination of Mamba and diffusion models, a SAR target detection algorithm combining diffusion and Mamba was proposed for the first time. Applying the Mamba model to SAR image detection is expected to advance the development of artificial intelligence in the field of SAR image interpretation. Considering the challenges of discontinuous target features and complex backgrounds in SAR images, a MambaSAR unit was designed to capture the rich spatial structure of the target. This unit focuses on the target's position in the image to better model global information and improve the model's ability to distinguish between the target and the complex background. Furthermore, this method achieved state-of-the-art performance on three SAR target detection datasets with different target types and scales, including fine-grained recognition.

[0122] It should be understood that although Figure 1 The steps in the flowchart are shown in sequence as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified in this document, there is no strict order restriction for the execution of these steps, and these steps can be executed in other orders. In addition, Figure 1At least part of the steps may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily executed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be executed in turn or alternately with other steps or at least part of the sub-steps or stages of other steps.

[0123] In one embodiment, Figure 8 As shown, a SAR image target detection device based on the combination of Mamba and diffusion models is provided, including: a training data set acquisition module 200, a multiple scale feature acquisition module 210, a noise addition module 220, a dynamic detection head prediction module 230, a target detection model training module 240 and a target detection module 250, wherein:

[0124] A training data set acquisition module 200 is used to acquire a training data set, wherein the training data set includes multiple sets of training data, each set of training data includes a SAR sample image and the true value of the category and the true value data of the bounding box of all targets in the sample image;

[0125] A multi-scale feature acquisition module 210 is configured to input a set of SAR sample images from the training data into the target detection model. The target detection model uses a Resnet50-FPN network as a framework, extracts multi-resolution features from the SAR sample images using the Resnet50 network, processes the extracted features using a Mamba SAR unit to obtain complete semantic features, and then constructs a feature pyramid from top to bottom using an FPN network to obtain feature images at multiple scales.

[0126] A noise adding module 220 is configured to add variance-controlled Gaussian noise to the ground truth data of the bounding box in the set of training data using a diffusion model to obtain a noise box;

[0127] A dynamic detection head prediction module 230 is configured to use the noise frame to crop regions of interest from feature images at multiple scales to obtain region-of-interest feature images, and use the dynamic detection head to obtain predicted categories and bounding box prediction data for each target in the SAR sample image based on the region-of-interest feature images.

[0128] An object detection model training module 240 is configured to calculate a loss function based on the predicted category, boundary prediction data, and the corresponding category truth value and bounding box truth value data, adjust the learnable parameters in the object detection model based on the calculation results, and train the object detection model using the training data set until convergence to obtain a trained object detection model;

[0129] The target detection module 250 is used to obtain a SAR image to be subjected to target detection, input the SAR image into the trained target detection model, and obtain the categories and positions of all targets in the SAR image.

[0130] Regarding the specific limitations of the SAR image target detection device based on the combination of the Mamba and diffusion models, please refer to the limitations of the SAR image target detection method based on the combination of the Mamba and diffusion models above, and will not be repeated here. Each module in the above-mentioned SAR image target detection device based on the combination of the Mamba and diffusion models can be implemented in whole or in part through software, hardware, or a combination thereof. Each of the above-mentioned modules can be embedded in or independent of the processor in the computer device in hardware form, or can be stored in the memory of the computer device in software form, so that the processor can call and execute the corresponding operations of each of the above modules.

[0131] In one embodiment, a computer device is provided. The computer device may be a terminal, and its internal structure diagram may be as follows: Figure 9 As shown. The computer device includes a processor, a memory, a network interface, a display screen and an input device connected via a system bus. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with an external terminal via a network connection. When the computer program is executed by the processor, a SAR image target detection method based on the combination of the Mamba and diffusion models is implemented. The display screen of the computer device can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device can be a touch layer covering the display screen, or a key, trackball or touchpad provided on the computer device housing, or an external keyboard, touchpad or mouse.

[0132] Those skilled in the art will understand that Figure 9 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than shown in the figure, or combine certain components, or have a different component arrangement.

[0133] In one embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and when the processor executes the computer program, the following steps are implemented:

[0134] Acquire a training data set, wherein the training data set includes multiple groups of training data, each group of training data includes a SAR sample image and category truth values ​​and bounding box truth value data of all targets in the sample image;

[0135] Inputting a set of SAR sample images in the training data into the target detection model, the target detection model uses the Resnet50-FPN network as a framework, extracting multiple resolution features from the SAR sample images using the Resnet50 network, processing the extracted features through the Mamba SAR unit to obtain complete semantic features, and then constructing a feature pyramid from top to bottom using the FPN network to obtain feature images at multiple scales;

[0136] The diffusion model is used to add variance-controlled Gaussian noise to the ground truth bounding box data in the training data to obtain a noise box.

[0137] Using the noise frame, cropping regions of interest from feature images at multiple scales to obtain feature images of the region of interest, and using a dynamic detection head to obtain predicted categories and bounding box prediction data of each target in the SAR sample image based on the feature images of the region of interest;

[0138] Calculating a loss function based on the predicted category, boundary prediction data, and corresponding category true value and bounding box true value data, adjusting the learnable parameters in the target detection model based on the calculation results, and training the target detection model using the training data set until convergence, thereby obtaining a trained target detection model;

[0139] Obtain a SAR image for target detection, input the SAR image into the trained target detection model, and obtain the categories and positions of all targets in the SAR image.

[0140] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the following steps are implemented:

[0141] Acquire a training data set, wherein the training data set includes multiple groups of training data, each group of training data includes a SAR sample image and category truth values ​​and bounding box truth value data of all targets in the sample image;

[0142] Inputting a set of SAR sample images in the training data into the target detection model, the target detection model uses the Resnet50-FPN network as a framework, extracting multiple resolution features from the SAR sample images using the Resnet50 network, processing the extracted features through the Mamba SAR unit to obtain complete semantic features, and then constructing a feature pyramid from top to bottom using the FPN network to obtain feature images at multiple scales;

[0143] The diffusion model is used to add variance-controlled Gaussian noise to the ground truth bounding box data in the training data to obtain a noise box.

[0144] Using the noise frame, cropping regions of interest from feature images at multiple scales to obtain feature images of the region of interest, and using a dynamic detection head to obtain predicted categories and bounding box prediction data of each target in the SAR sample image based on the feature images of the region of interest;

[0145] Calculating a loss function based on the predicted category, boundary prediction data, and corresponding category true value and bounding box true value data, adjusting the learnable parameters in the target detection model based on the calculation results, and training the target detection model using the training data set until convergence, thereby obtaining a trained target detection model;

[0146] Obtain a SAR image for target detection, input the SAR image into the trained target detection model, and obtain the categories and positions of all targets in the SAR image.

[0147] Those skilled in the art will appreciate that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0148] The technical features of the above embodiments can be combined arbitrarily. In order to make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0149] The above-described embodiments merely represent several implementation methods of the present application. While the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the present invention. It should be noted that a person skilled in the art could make various modifications and improvements without departing from the spirit of the present application, all of which fall within the scope of protection of the present application. Therefore, the scope of protection of the present patent application shall be determined by the appended claims.

Claims

1. A SAR image target detection method based on the combination of Mamba and diffusion models, characterized in that: The method comprises: Acquire a training data set, wherein the training data set includes multiple groups of training data, each group of training data includes a SAR sample image and category truth values ​​and bounding box truth value data of all targets in the sample image; A set of SAR sample images in the training data is input into the target detection model. The target detection model uses a Resnet50-FPN network as a framework, uses the Resnet50 network to extract multiple resolution features from the SAR sample images, processes the extracted features through a Mamba SAR unit to obtain complete semantic features, and then uses the FPN network to construct a feature pyramid from top to bottom for the processed features to obtain feature images of multiple scales; The diffusion model is used to add Gaussian noise controlled by variance scheduling to the bounding box truth data in the set of training data to obtain a noise box; Using the noise frame, cropping regions of interest from feature images of multiple scales to obtain feature images of the region of interest, and using a dynamic detection head to obtain predicted categories and bounding box prediction data of each target in the SAR sample image based on the feature images of the region of interest; Calculate the loss function according to the predicted category, boundary prediction data, and the corresponding category true value and bounding box true value data, adjust the learnable parameters in the target detection model according to the calculation result, and train the target detection model using the training data set until convergence to obtain a trained target detection model; A SAR image to be subjected to target detection is obtained, and the SAR image is input into the trained target detection model to obtain the categories and positions of all targets in the SAR image.

2. The SAR image target detection method according to claim 1, characterized in that: In the target detection model: The output of the Resnet50 network includes multiple layers of feature images with resolutions from low to high, and the Mamba SAR unit is connected between two adjacent layers of feature images in the middle layer; The feature image with lower resolution in the two adjacent layers of feature images is processed by the Mamba SAR unit, fused with the feature image with higher resolution, and then output to the FPN network as a higher-layer feature image.

3. The SAR image target detection method according to claim 2, characterized in that: The output of the Resnet50 network includes four layers of feature images with resolutions from low to high, wherein the third layer of feature images is processed by the Mamba SAR unit, fused with the fourth layer of feature images, and then output to the FPN network as the fourth layer of feature images.

4. The SAR image target detection method according to claim 2, characterized in that: The Mamba SAR unit includes a plurality of Mamba blocks and a plurality of intelligent attention blocks connected in sequence, wherein each of the Mamba blocks and the intelligent attention block is further connected to a multi-layer perceptron; In the Mamba SAR unit, the input feature image is processed by the adaptive window scratch mechanism, and then processed by the Mamba block and the intelligent attention block in turn and merged to obtain the output feature image.

5. The SAR image target detection method according to claim 4, characterized in that: The Mamba block includes two asymmetric branches, one of which includes a linear layer, a one-dimensional convolution layer, and an activation function layer in sequence, and the other branch includes a linear layer, a one-dimensional convolution layer, an activation function layer, and a Mamba scan layer in sequence; The image of the input mamba block is processed by the two branches respectively, spliced ​​through the splicing layer and then passed through the linear layer to obtain the image of the output mamba block.

6. The SAR image target detection method according to claim 5, characterized in that: In the Mamba Scan layer: Expanding the input Mamba scan layer image into a plurality of sub-images by using a scan expansion operation, wherein each sub-image is obtained by scanning the input image from a different angle; The S6 block operation is used to extract features from each of the sub-images, and the extracted features are merged by scanning to obtain an output image of the Mamba scan layer having a size consistent with the input image.

7. The SAR image target detection method according to any one of claims 1 to 6, characterized in that: It is characterized in that The bounding box true value data includes the center coordinates, width and height of the bounding box.

8. The SAR image target detection method according to claim 7, characterized in that: The diffusion model includes a forward diffusion part and a reverse diffusion part; Using the forward diffusion part, adding Gaussian noise controlled by variance scheduling to the true value data of the boundary box to obtain the noise box; The reverse diffusion part is used as the dynamic detection head to perform target detection.

9. A SAR image target detection device based on the combination of Mamba and diffusion models, characterized in that: The device comprises: A training data set acquisition module is used to acquire a training data set, wherein the training data set includes multiple groups of training data, each group of training data includes a SAR sample image and the category truth value and bounding box truth value data of all targets in the sample image; A module for obtaining multiple scale features, used for inputting a set of SAR sample images in the training data into the target detection model, wherein the target detection model uses a Resnet50-FPN network as a framework, uses a Resnet50 network to extract multiple resolution features from the SAR sample images, processes the extracted features through a Mamba SAR unit to obtain complete semantic features, and then uses an FPN network to construct a feature pyramid from top to bottom for the processed features to obtain feature images of multiple scales; A noise adding module, used for adding Gaussian noise controlled by variance scheduling to the bounding box truth data in the set of training data by using a diffusion model to obtain a noise box; A dynamic detection head prediction module is used to use the noise frame to crop the region of interest from the feature images of multiple scales to obtain feature images of the region of interest, and use the dynamic detection head to obtain the predicted category and bounding box prediction data of each target in the SAR sample image according to the feature images of the region of interest; A target detection model training module is used to calculate a loss function according to the predicted category, boundary prediction data, and corresponding category true value and bounding box true value data, adjust the learnable parameters in the target detection model according to the calculation result, and train the target detection model using the training data set until convergence to obtain a trained target detection model; The target detection module is used to obtain a SAR image to be subjected to target detection, input the SAR image into the trained target detection model, and obtain the categories and positions of all targets in the SAR image.

Images