Rotating target detection method and system for SAR images based on scattering key point guided diffusion model

Abstract

The present invention discloses a method and system for detecting rotating targets in SAR images based on a scattering keypoint-guided diffusion model. The present invention designs a fine-grained target representation based on convex polygons to adaptively model the fuzzy outer contour of the target; constructs a target region diffusion module and a denoising decoder based on the diffusion model, and uses the diffusion model to learn the statistical characteristics of the spatial distribution of target regions in SAR images to directly predict the target region; designs a multi-scale scattering keypoint feature fusion module, adopts a method of dynamically predicting scattering keypoints at a multi-scale feature level, and enhances the network's ability to identify targets in complex scenes based on scattering keypoint features at different scales, thereby suppressing the generation of false alarms; finally, through a convex polygon prediction module, a target convex polygon prediction is obtained in each target region. Based on the Detection Transformer architecture, the present invention implements end-to-end rotating target detection in SAR images.

Description

Technical Field

[0001] The present invention belongs to the field of remote sensing image processing, and relates to a method for detecting rotating targets in SAR images, and in particular to a method and system for detecting rotating targets in SAR images based on a scattering key point guided diffusion model. Background Art

[0002] Synthetic aperture radar (SAR) is an all-day, all-weather imaging sensor widely used in various fields, including environmental monitoring, military reconnaissance, post-disaster assessment, and urban planning. With the rapid development of spaceborne SAR, automated target detection in SAR images has attracted widespread attention. Rotating target detection is a recently emerging and challenging image interpretation task, particularly well-suited for SAR targets with arbitrary orientations, wide aspect ratios, and irregular shapes. Due to the unique imaging mechanism, SAR images often contain sparsely distributed non-Gaussian and nonlinear coherent speckle noise, which causes discontinuities in the texture features of objects in SAR images. As a result, the outer contours of targets in SAR images are often blurred, with no clear boundary between the target and the background, posing a challenge for target recognition. Furthermore, complex backgrounds in SAR images can have visual attributes similar to those of the target, resulting in a high number of false alarms. Traditional SAR target detection methods, typically based on constant false alarm rate (CFAR), are no longer able to meet the requirements of target detection in these complex scenarios. The emergence of deep learning methods, with their enhanced representation capabilities, has made this possible. How to design a deep learning detection method suitable for the target characteristics of SAR images has become a major challenge for rotating target detection in SAR images.

[0003] Early research often directly used horizontal bounding boxes (AR boxes) from general object detection to represent SAR targets. However, this representation fails to account for the characteristics of SAR targets and cannot effectively meet the requirements for accurate SAR target detection. Therefore, the RoI-Transformer proposed using a rotated bounding box representation with additional angular information to replace the horizontal bounding box. Compared to the horizontal bounding box, the rotated bounding box provides precise orientation and scale information of the target, making it more suitable for detecting rotated targets in SAR images. It has achieved excellent results and has been widely used in subsequent SAR target detection methods. However, the rotated bounding box representation is still based on a regular rectangle, which cannot well fit the fuzzy boundaries of targets in SAR images or accurately separate the target area from the background. With the increasing requirements for rotating target detection in SAR images in recent years, some researchers have gradually recognized this problem. SPAN and SFP-Net, combining the characteristics of SAR targets, proposed directly representing the target using a set of strong scattering points. By directly predicting all strong scattering points in the SAR image and classifying them, the set of strong scattering points belonging to the target is directly found, thereby obtaining the target detection result. This approach addresses the problems of the rectangular bounding box representation to a certain extent and achieves good detection results.

[0004] In addition to challenges with SAR target representation methods, false alarms in SAR target detection are also a key research area. Post-processing algorithms are the simplest approach to suppress false alarms. They calculate the saliency of predictions relative to the background and eliminate less salient predictions. However, this approach suffers from very limited robustness and has been largely limited to early research efforts. With the rise of deep learning methods, a growing number of researchers are exploring methods for suppressing false alarms using deep learning networks. S-SSD directly trains deep neural networks to autonomously learn saliency information from SAR images, enabling target features to be enhanced in a variety of complex scenarios and suppressing false alarms. However, this approach requires high-quality saliency maps to train the network, which is generally difficult to obtain. Therefore, with the widespread use of attention mechanisms in recent years, researchers have begun considering applying them to adaptively enhance target features. Both DPAN and AFRAN employ attention mechanisms in their network architectures to identify and enhance target features from multi-scale features while suppressing background features, achieving promising results in suppressing false alarms. SKG-Net goes a step further and uses an attention mechanism to enhance the features of strong scattering points of the target, thereby more specifically finding feature information that can distinguish the target from the complex background, enhancing the network's ability to recognize the target and suppress false alarms.

[0005] Although deep learning-based methods for detecting rotating targets in SAR images have developed rapidly, current detection methods still have some problems. First, the representation based on rectangular boxes cannot fit the fuzzy outer contour of the target well, especially for irregular and densely arranged targets. Although some methods use a set of scattering points to represent the target, which alleviates this problem to a certain extent, they usually require dense predictions to obtain the scattering points on the entire image, which is computationally intensive and has great difficulty in distinguishing the scattering points of the target and the background. Secondly, most of the existing attention-based false alarm suppression methods do not combine the characteristics of SAR targets, but simply use the attention mechanism to fuse multi-scale features. Although SKG-Net specifically uses the attention mechanism to fuse the features of the target's strong scattering points, it is mainly designed for aircraft targets and requires additional auxiliary algorithms. It has poor generalization ability on other SAR targets. Summary of the Invention

[0006] To address the problem of blurred target-background boundaries and false alarms caused by coherent speckle noise in SAR images, a more fine-grained representation is designed to better model the target's outer contour. Furthermore, taking into account the fact that strong scattering areas in SAR images are mostly target areas, a diffusion model is introduced to learn the statistical characteristics of the spatial distribution of target areas, minimizing the interference of complex backgrounds on target recognition and positioning. Furthermore, based on the fact that SAR target scattering key points contain rich contextual information, false alarms are further suppressed, thereby improving target recognition accuracy. The present invention provides a method for detecting rotating targets in SAR images based on a diffusion model guided by scattering key points.

[0007] The technical solution of the present invention is a method for detecting rotating targets in SAR images based on a scattering key point guided diffusion model, comprising the following steps:

[0008] Construct and train a SAR image rotating target detection network based on a scattering keypoint guided diffusion model, wherein the SAR image rotating target detection network includes a feature extraction module, a multi-scale feature mapping module, an image feature encoder, a target region diffusion module, a target feature initialization module and a time step encoding module, a denoising decoder module and a multi-scale scattering keypoint feature fusion module, and a target convex polygon prediction module;

[0009] The feature extraction module is used to extract feature maps of different scales, input them into the multi-scale feature mapping module, obtain multi-scale feature vectors, input them into the image feature encoder, and fully fuse the features of different scales in the multi-scale feature vectors to obtain the feature vector after multi-scale feature fusion as output; the target area diffusion module is used to obtain the center point coordinates, width and height of the initial target area according to the rotation box annotation of the target in each image through a diffusion process based on Gaussian distribution, and extract the features of the corresponding position of each feature scale from the feature vector output by the image feature encoder according to the center point coordinates of the initialization area, and fuse the extracted features of different feature scales, so as to obtain a feature vector for each initial target area, which is used as the initial target feature vector; the time step encoding module is used to encode the sampling time step t in the target area diffusion module and output it The denoising decoder module first uses the initial target area and the initial target feature vector as a reference, and based on the multi-scale scattering key point feature fusion module, dynamically predicts scattering key points at different feature scales near the target area, and adaptively extracts features at the corresponding feature scales from the feature vector obtained from the image feature encoder based on the predicted scattering key points as the key features of the target. These extracted key features are then fused with the target feature vector to update the target feature vector, and then the target area parameters are updated based on the updated target feature vector to obtain a more accurate target area. The target convex polygon prediction module predicts the confidence score of each category for each target and a convex polygon point set consisting of 9 points based on the target feature vector and target area parameters finally updated by the denoising decoder module, thereby obtaining the target classification result and convex polygon prediction result.

[0010] The loss functions used in the training process are: the category prediction loss function is calculated by the predicted target category, confidence and the target true category label; the convex polygon prediction loss function is calculated by the predicted target convex polygon and the target true rotation box label parameters; the target area regression loss function is calculated by the final updated target area and the true target area, and then all the loss functions are added together to get the total loss function;

[0011] Finally, the trained target detection network is used to detect the data to be detected and obtain the required detection results.

[0012] Furthermore, the feature extraction module consists of five parts. The first part consists of a 7×7 convolutional layer with 64 channels, a stride of 2, and a padding of 3, a batch normalization layer, and a 3×3 maximum pooling layer with a stride of 2 and a padding of 1. The second part consists of one convolutional block and two residual blocks. The third part consists of one convolutional block and three residual blocks. The fourth part consists of one convolutional block and five residual blocks. The fifth part consists of one convolutional block and two residual blocks. The convolutional block consists of two The convolution block is composed of parallel branches, one of which consists of a 1×1 convolution layer, a ReLU activation function, a batch normalization layer, a 3×3 convolution layer, a ReLU activation function, a batch normalization layer, a 1×1 convolution layer and a batch normalization layer, and the other branch consists of a 1×1 convolution layer and a batch normalization layer. The outputs of these two branches are added together and then passed through the ReLU activation function to obtain the final convolution block output; the residual block is composed of a 1×1 convolution layer, a ReLU activation function, a batch normalization layer, a 3×3 convolution layer, a ReLU activation function, a batch normalization layer, a 1×1 convolution layer and a batch normalization layer. The output is added to the input of the residual block and then passed through the ReLU activation function to obtain the final residual block output; in each convolution block, the number of channels of the last 1×1 convolution layer of the two branches is twice the number of channels of the input feature map; the number of channels of the last convolution layer in the residual block is the number of channels of the input feature map, and the number of channels of the remaining convolution layers is 1 / 4 of the number of input feature map channels; except for the second part, all 3×3 convolution layers in the remaining convolution blocks and residual blocks have a step size of 2 and a padding of 1, a 1×1 convolution layer has a step size of 1 and a padding of 0, and all pooling layers have a step size of 2 and a padding of 1; while all 3×3 convolution layers in the second part of the convolution blocks and residual blocks have a step size of 1 and a padding of 1, a 1×1 convolution layer has a step size of 1 and a padding of 0, and all pooling layers have a step size of 2 and a padding of 1.

[0013] Furthermore, the specific processing process of the multi-scale feature mapping module is as follows:

[0014] The input feature map is the feature map obtained by the third, fourth and fifth parts of the feature extraction module in step 1;

[0015] The above three input feature maps are respectively subjected to a 3×3 convolution layer with a stride of 1 and a channel of 256 to obtain three output feature maps; then, the last output feature map will be subjected to another 3×3 convolution layer with a stride of 2 and a channel of 256 to obtain the fourth output feature map; the above four output feature maps will be transformed into one-dimensional vectors in the spatial dimension and spliced ​​in the dimension to finally obtain a multi-scale feature vector.

[0016] Furthermore, the specific processing process of the image feature encoder is as follows:

[0017] The image feature encoder consists of several encoder layers, each of which consists of a multi-head deformable self-attention layer, a batch normalization layer, a feedforward neural network, and a batch normalization layer. The feedforward neural network consists of a linear layer, a ReLU activation function, and a linear layer. Specifically, in the image feature encoder, the first encoder layer takes the multi-scale feature vector obtained by the multi-scale feature mapping module as input, and the subsequent encoder layers take the output feature vector of the previous encoder layer as input.

[0018] In each encoder layer, the multi-headed deformable self-attention layer uses the feature vector of the input encoder layer as the query, key, and value at the same time, and uses the position information of each element in the feature vector on the original two-dimensional feature map as the reference point to adaptively capture the relevant feature points that need to calculate the attention matrix, and fuses the features of the corresponding positions according to the calculated attention matrix to achieve the fusion of features at different positions of the input feature vector; the feedforward neural network will first increase the number of channels of the input feature vector and then reduce it to more fully fuse features of different channel dimensions; the image feature encoder fully fuses the features of different scales in the obtained multi-scale feature vector to obtain the feature vector after feature fusion as output.

[0019] Furthermore, the specific implementation process of the target area diffusion module is as follows:

[0020] First, mark the target's rotation box Get its minimum circumscribed horizontal rectangle, and then expand the width and height of the rectangle by b times to get the real target area ,Right now ,in A function to calculate the minimum horizontal circumscribed rectangle of a quadrilateral;

[0021] During the iterative process of training, the real target area is constructed To the noise area The diffusion process, at random sampling time steps , Afterwards, use To achieve this diffusion process, , , is the noise variance scheduling parameter, Schedule hyperparameters for noise variance; through the above formula, we can get The real target area is obtained Noisy target area In addition, due to the need to predict Results, in After the target noise area is obtained, it needs to be filled or deleted to obtain the initial target area; specifically, when When random sampling based on Gaussian distribution Regions are spliced ​​in After the noisy target area, we get Initial target area ; and when When The front of the noisy target area As the initial target area ;when When , directly The noisy target area is used as the initial target area; in the inference process, the target area diffusion module is directly generated based on the Gaussian distribution region as the initial target region , without the need to obtain the noisy target area based on the real target area.

[0022] Furthermore, the specific processing of the target feature initialization module and the time step encoding module is as follows:

[0023] When constructing the target feature initialization module, first obtain the initial target area according to the target area diffusion module. The center point position information is extracted and the feature vector obtained by the image feature encoder is extracted The features of the corresponding positions are extracted, and the extracted features are spliced ​​together in the channel dimension, and then the channel dimension is transformed to 256 through a linear layer, so that a feature vector is obtained for each initial target area, and these feature vectors together constitute the target feature vector; specifically, for the first Target initialization area , the extracted feature vector is ,in for The center point coordinates of Initial target area , we can get the length of The feature vector of , and then through 1 linear layer, the final initial target feature vector can be obtained;

[0024] When constructing the time step encoding module, first convert the sampled time steps , using the cosine position encoding method, we get a time code with 256 channels, and then after a linear layer, we get a time code with 4 times the number of channels, and then after a SiLU activation function and a linear layer, we get the final time code output .

[0025] Furthermore, the specific processing of the denoising decoder module and the multi-scale scattering key point feature fusion module is as follows:

[0026] The denoising decoder module consists of several denoising decoder layers and corresponding target region update modules. The denoising decoder layer consists of a multi-head self-attention layer, a batch normalization layer, a multi-scale scattering keypoint feature fusion module, a batch normalization layer, a feedforward neural network, and an adaptive batch normalization layer. The multi-scale scattering keypoint feature fusion module consists of four parallel linear layers, a linear layer, and a cross-attention layer. The feedforward neural network consists of a linear layer, a ReLU activation function, and a linear layer. The target region update module consists of a linear layer, a ReLU activation function, a linear layer, a ReLU activation function, and a linear layer.

[0027] Specifically, the first denoising decoder layer in the denoising decoder takes the initial target feature vector obtained by the target feature initialization module as input, and the subsequent denoising decoder layers all take the target feature vector updated by the previous denoising decoder layer as input; in each denoising decoder layer, the multi-head self-attention layer takes the target feature vector of the input denoising decoder layer as query, key and value at the same time to calculate The attention matrix between the target feature vectors is obtained, and the features at different positions are fused according to the calculated attention matrix to obtain a length of , the output feature vector with 256 channels;

[0028] The specific processing process of the multi-scale scattering key point feature fusion module is as follows:

[0029] The length of the multi-head self-attention layer is , the feature vector with a channel number of 256 passes through 4 parallel linear layers to obtain 4 lengths , the prediction results with 18 channels correspond to The coordinate offsets of 9 points in the target area at 4 characteristic scales are calculated based on these predicted offsets and The center point coordinates of the target area are obtained for each target area at 4 feature scales. These points are the scattering key points dynamically predicted in the target area. According to the position information of these points, features are extracted from the corresponding positions on the obtained multi-scale feature vector, and the features of the same point at different scales are spliced ​​together in the channel dimension. For each target area, a scattering key point feature vector is obtained. After a linear layer, a feature vector with a length of 36 and a number of channels of 256 is obtained. The feature vector is used as the key and value, and the input feature vector of the multi-scale scattering key point feature fusion module is sent as a query to the multi-head self-attention layer to obtain the target feature vector after the fusion of the scattering key point features. Its length is , the number of channels is 256.

[0030] Furthermore, the adaptive batch normalization layer is based on the ordinary batch normalization layer and introduces the time step Related scaling and offset parameters; specifically, the final time-encoded output obtained by the time-step encoding module After the linear layer and SiLU activation function, the output vector is obtained, which is divided into two vectors in the channel dimension and copied , and get the scale scaling parameters respectively and offset parameters , ,in is the channel averaging function, is the replication function, is a linear layer; in the adaptive batch normalization layer, the feature vector obtained by the feedforward neural network , first through the ordinary batch normalization layer, and then use the above-mentioned sampling time step The obtained scale scaling and offset parameters are processed to obtain the final output feature vector , ,in is a normal batch normalization layer, That is, the output target feature vector updated by each denoising decoder layer;

[0031] In addition, the target feature vector output by each denoising decoder layer is divided into two branches. One branch is directly input to the next decoder layer as the input target feature vector, and the other branch is input to the target region update module to update the target region according to the updated target feature vector and use the updated target region in the multi-scale scattering key point module in the next decoder layer, so that it can predict more accurate scattering key points. In the target region update module, the input target feature vector passes through the linear layer and obtains a length of , the prediction result with 4 channels corresponds to The horizontal and vertical coordinates, width and height offsets of the center point of the target area are calculated. The parameters of the target area can be updated based on these predicted offsets. The original target area is defined as , the predicted offset is , then the updated target area .

[0032] Furthermore, the target convex polygon prediction module includes a classification branch and a regression branch. The specific processing process is as follows:

[0033] In the classification branch, the denoising decoder finally updates the target feature vector, which has a length of , the number of channels is The prediction results, among which, The number of categories in the dataset is marked, and the prediction result on each channel is the confidence prediction score of a category;

[0034] In the regression branch, the denoising decoder finally updates the target feature vector, which is enhanced by the linear layer, ReLU activation function, linear layer and ReLU activation function, and finally passes through a linear layer to obtain a length of , the prediction result with 18 channels corresponds to The 18 horizontal and vertical coordinate offsets of 9 points in the target area are obtained. According to these predicted offsets and the horizontal and vertical coordinates of the center point of the corresponding target area, 9 predicted points can be obtained for each target area, forming the target convex polygon prediction of the target area. By calculating the minimum circumscribed horizontal rectangle of the predicted target convex polygon, the predicted target convex polygon is converted into a rotation box prediction result.

[0035] This invention addresses the issues of blurred target outlines and false alarms in complex backgrounds during rotating target detection in SAR images. By leveraging the diffusion model's ability to learn the statistical characteristics of the spatial distribution of strong scattering regions within an image, and the fact that most strong scattering regions in SAR images are targets, the invention minimizes the interference of complex backgrounds on target recognition and positioning by designing a target region diffusion module and a denoising decoder. Furthermore, a multi-scale scattering keypoint feature fusion module is incorporated into the denoising decoder to further enhance the distinguishability of the target relative to the background and reduce false alarms. Based on the Detection Transformer architecture, the network achieves end-to-end rotating target detection in SAR images, improving detection accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] Figure 1 The figure is a flow chart of a method for detecting rotating targets in SAR images according to an embodiment of the present invention.

[0037] Figure 2 2 is a schematic diagram of the structure of a feature extraction module according to an embodiment of the present invention.

[0038] Figure 3 2 is a schematic diagram of the structure of a multi-scale feature mapping module according to an embodiment of the present invention.

[0039] Figure 4 2 is a schematic diagram of the structure of an image feature encoder according to an embodiment of the present invention.

[0040] Figure 5 2 is a schematic diagram of the structure of a denoising decoder according to an embodiment of the present invention.

[0041] Figure 6 3 is a schematic diagram of the structure of a multi-scale scattering key point feature fusion module according to an embodiment of the present invention.

[0042] Figure 7 2 is a schematic diagram of the structure of a convex polygon prediction module according to an embodiment of the present invention.

[0043] Figure 8 4 is a flowchart of network training according to an embodiment of the present invention. DETAILED DESCRIPTION

[0044] This invention, based on a deep learning neural network, considers the blurred target outlines and false alarms that are common in SAR image rotating target detection, and proposes a method and system for detecting rotating targets in SAR images based on a scattering keypoint-guided diffusion model. This method fully addresses the blurred target outlines caused by speckle noise in SAR images and the false alarms that are common in complex scenes. By designing a fine-grained convex polygon representation to better fit the target's outline, the method can better distinguish target and background areas. A diffusion model is also introduced to learn the statistical characteristics of the spatial distribution of strong scattering areas in the image to directly predict the target area, effectively reducing the interference of complex backgrounds on target recognition and positioning. Furthermore, a multi-scale scattering keypoint feature fusion module is designed to dynamically capture the target's scattering keypoints. Based on these scattering keypoint features, the distinguishable features between the target and background are enhanced, improving the network's target recognition capability and reducing the occurrence of false alarms. The results obtained by this invention are more scientific and accurate.

[0045] The method provided by the present invention can be implemented using computer software technology. Figure 1 , the process of the present invention is specifically described below:

[0046] A method for detecting rotating targets in SAR images based on a scattering key point guided diffusion model comprises: constructing a SAR image rotating target detection network based on the scattering key point guided diffusion model, wherein the SAR image rotating target detection network comprises a feature extraction module, a multi-scale feature mapping module, an image feature encoder, a target region diffusion module, a target feature initialization module and a time step encoding module, a denoising decoder module and a multi-scale scattering key point feature fusion module therein, and a target convex polygon prediction module;

[0047] The feature extraction module is used to extract feature maps of different scales, input them into the multi-scale feature mapping module, obtain multi-scale feature vectors, input them into the image feature encoder, and fully fuse the features of different scales in the multi-scale feature vectors to obtain the feature vector after multi-scale feature fusion as output; the target area diffusion module is used to obtain the center point coordinates, width and height of the initial target area according to the rotation box annotation of the target in each image through a diffusion process based on Gaussian distribution, and extract the features of the corresponding position of each feature scale from the feature vector output by the image feature encoder according to the center point coordinates of the initialization area, and fuse the extracted features of different feature scales, so as to obtain a feature vector for each initial target area, which is used as the initial target feature vector; the time step encoding module is used to encode the sampling time step t in the target area diffusion module and output it The denoising decoder module first uses the initial target area and the initial target feature vector as a reference, and based on the multi-scale scattering key point feature fusion module, dynamically predicts scattering key points at different feature scales near the target area, and adaptively extracts features at the corresponding feature scales from the feature vector obtained from the image feature encoder based on the predicted scattering key points as the key features of the target. These extracted key features are then fused with the target feature vector to update the target feature vector, and then the target area parameters are updated based on the updated target feature vector to obtain a more accurate target area. The target convex polygon prediction module predicts the confidence score of each category for each target and a convex polygon point set consisting of 9 points based on the target feature vector and target area parameters finally updated by the denoising decoder module, thereby obtaining the target classification result and convex polygon prediction result.

[0048] The loss function used in the training process is: the category prediction loss function is calculated by the predicted target category, confidence and target true category label; the convex polygon prediction loss function is calculated by the predicted target convex polygon and the target true rotation box label parameters; the target area regression loss function is calculated by the final updated target area and the true target area, and then all loss functions are added together to obtain the total loss function; the trained network is used to test the data to obtain the required test results.

[0049] The following is a detailed description of each module:

[0050] Step 1: Construct a feature extraction module to process the input image and obtain input image feature maps of different scales. Figure 2 .

[0051] The feature extraction module consists of five parts. The first part consists of a 7×7 convolutional layer with 64 channels, a stride of 2, and a padding of 3, a batch normalization layer, and a 3×3 max pooling layer with a stride of 2 and a padding of 1. The second part consists of one convolutional block and two residual blocks. The third part consists of one convolutional block and three residual blocks. The fourth part consists of one convolutional block and five residual blocks. The fifth part consists of one convolutional block and two residual blocks. The convolutional block consists of two parallel branches. One branch consists of a 1×1 convolutional layer, ReLU activation, batch normalization, a 3×3 convolutional layer, ReLU activation, batch normalization, a 1×1 convolutional layer, and a batch normalization layer. The other branch consists of a 1×1 convolutional layer and a batch normalization layer. The outputs of these two branches are summed and then activated by the ReLU function to obtain the final convolutional block output. The residual block consists of a 1×1 convolutional layer, ReLU activation, batch normalization, a 3×3 convolutional layer, ReLU activation, batch normalization, and a 1×1 convolutional layer and a batch normalization layer. The output is summed with the input of the residual block and then activated by the ReLU function to obtain the final residual block output. In each convolutional block, the last 1×1 convolutional layer of both branches has twice the number of channels as the input feature map. The number of channels in the last convolutional layer in the residual block is the same as the number of channels in the input feature map, and the number of channels in the remaining convolutional layers is 1 / 4 of the number of channels in the input feature map. Except for the second part, all 3×3 convolutional layers in the remaining convolutional blocks and residual blocks have a stride of 2 and a padding of 1, all 1×1 convolutional layers have a stride of 1 and a padding of 0, and all pooling layers have a stride of 2 and a padding of 1. In the second part of the convolutional blocks and residual blocks, all 3×3 convolutional layers have a stride of 1 and a padding of 1, all 1×1 convolutional layers have a stride of 1 and a padding of 0, and all pooling layers have a stride of 2 and a padding of 1.

[0052] The specific implementation process is described as follows:

[0053] In the first part, the number of channels of the convolution layer is 64. After the input image of 800×800 size and 3 channels is convolved by 7×7, a 64-channel feature map of 400×400 size is obtained. After the 3×3 maximum pooling operation, a 64-channel feature map of 200×200 size is generated. In the second part, the last 1×1 convolution layer of the two branches of the convolution block has 256 channels, and the number of channels of the remaining convolution layers is 64. The last convolution in the residual block The number of channels of the convolution layer is 256, and the number of channels of the remaining convolution layers is 64. The 200×200 size and 64-channel feature map obtained in the first part is converted into a 200×200 size and 256-channel output feature map after the second part. In the third part, the last 1×1 convolution layer of the two branches of the convolution block has 512 channels, and the number of channels of the remaining convolution layers is 256. The number of channels of the last convolution layer in the residual block is 512, and the number of channels of the remaining convolution layers is 128. The 200×200 size, 256 channel feature map obtained in the second part is converted into a 100×100 size, 512 channel feature map after the third part; in the fourth part, the number of channels of the last 1×1 convolution layer of the two branches of the convolution block is 1024, and the number of channels of the remaining convolution layers is 512. The number of channels of the last convolution layer in the residual block is 1024, and the number of channels of the remaining convolution layers is 256. The 100×100 size, 512 channel feature map obtained in the third part is converted into a 1024 channel feature map. After the feature map passes through the fourth part, a 50×50, 1024 channel feature map is generated; in the fifth part, the number of channels of the last 1×1 convolution layer of the two branches of the convolution block is 2048, and the number of channels of the remaining convolution layers is 1024. The number of channels of the last convolution layer in the residual block is 2048, and the number of channels of the remaining convolution layers is 512. The 50×50, 1024 channel feature map obtained in the fourth part passes through the fifth part to generate a 25×25, 2048 channel feature map.

[0054] For the convolutional layer, define is the size of the input feature map, is the size of the output feature map, is the convolution kernel size, To fill the parameters, is the step size parameter, is the number of filters, is the number of convolution kernels in the filter (number of output channels).

[0055] The convolution layer performs a two-dimensional convolution operation on each channel dimension of the feature map, and obtains a single output feature value by sliding the convolution kernel on the two-dimensional feature and performing matrix multiplication on the features within the current convolution kernel range. The output feature is the weighted sum of the input features at the same position, and the weight is the parameter of the convolution kernel. In order to ensure that the output feature map is the same size as the input feature map, the present invention uses a padding parameter, that is, the edge of the input feature map is filled with zero-value features of the corresponding parameter size to ensure that when the convolution kernel slides on the input feature map, the original edge features can be located at the center of its kernel and extend to the padded features, thereby generating an output feature map of the same size as the input feature map. The number of padded features is determined by the padding parameter. Determine that the size of the input feature map becomes , and the output feature map size after the convolution layer is . When executing the technical solution of the present invention, the step size of the 1×1 convolution layer is 1 and the padding parameter is 0, while the step size of the 3×3 convolution layer is 2 and the padding parameter is 1. Each convolution layer has a number of filters equal to the number of channels of the output feature map, and each filter has a number of convolution kernels equal to the number of channels of the input feature map. Each convolution kernel in each filter performs a two-dimensional convolution operation on its corresponding channel. The obtained feature maps are added at the corresponding positions, and the bias is added to obtain the final output feature map.

[0056] In order to enhance the robustness of the features extracted by the feature extraction module, the present invention makes extensive use of the ReLU activation function to improve the feature expression ability. The calculation formula of the ReLU activation function is: ,in is the input eigenvalue of the function, To find the maximum value of the function. At the same time, in order to prevent the gradient vanishing and gradient exploding problems in the network during training, the feature extraction network also uses a large number of batch normalization layers. These batch normalization layers introduce learnable parameters , ,in is the output feature map of the convolutional layer, To find the variance function, As a function of finding the mean, the batch normalization output is The pooling layer performs a pooling operation on each channel of the feature map. Generally speaking, the feature map of each channel will be divided into multiple areas of pooling size without overlapping. The maximum pooling operation will select the largest eigenvalue in each area as the only output eigenvalue, while the average pooling operation will take the average of all eigenvalues ​​in each area as the only output eigenvalue. The feature map obtained after the above operation is the output feature map of the pooling layer.

[0057] Step 2: Construct a multi-scale feature mapping module to process the feature maps of different scales obtained by different parts of the feature extraction module in step 1 to obtain a multi-scale feature vector. Figure 3 .

[0058] When building this module, it is necessary to align feature maps with different numbers of channels. Specifically, the feature maps obtained from the third, fourth, and fifth parts of the feature extraction module in step 1 are first extracted and processed as follows: the feature map with 512 channels and a scale of 100×100 obtained from the third part is processed through a 3×3 convolution layer with a step size of 1, a padding of 1, and a channel number of 256 to obtain a feature map with 256 channels and a scale of 100×100; the feature map with 1024 channels and a scale of 50×50 obtained from the fourth part is processed through a 3×3 convolution layer with a step size of 1, a padding of 1, and a channel number of 256 to obtain a feature map with 256 channels and a scale of 100×100; After a 3×3 convolution with a stride of 1 and a padding of 1 and a channel number of 256, a feature map with 256 channels and a scale of 50×50 is obtained. The feature map with 2048 channels and a scale of 25×25 obtained from the fifth part is subjected to a 3×3 convolution layer with a stride of 1, a padding of 1 and a channel number of 256, and a 3×3 convolution layer with a stride of 2, a padding of 1 and a channel number of 256, respectively, resulting in a feature map with 256 channels and a scale of 25×25 and a feature map with 256 channels and a scale of 13×13. The four feature maps obtained above are transformed into one-dimensional vectors in the spatial dimension, resulting in feature vectors with 256 channels and lengths of 10,000, 2,500, 625, and 169, respectively. These feature vectors are then concatenated in the spatial dimension to obtain a multi-scale feature vector output with a length of 13,294 and a channel number of 256.

[0059] Step 3: Construct an image feature encoder and perform multi-scale feature fusion on the multi-scale feature vector obtained by the multi-scale feature mapping module in step 2 to obtain the output feature vector after feature fusion. Figure 4 .

[0060] The image feature encoder consists of six encoder layers, each of which consists of a multi-head deformable self-attention layer, a batch normalization layer, a feedforward neural network, and a batch normalization layer. The feedforward neural network consists of a linear layer, a ReLU activation function, and a linear layer. Specifically, in the image feature encoder, the first encoder layer takes as input the multi-scale feature vector of length 13294 and number of channels 256 obtained by the multi-scale feature mapping module in step 2, while subsequent encoder layers take as input the output feature vector of the previous encoder layer. In each encoder layer, the multi-headed deformable self-attention layer uses the feature vector of the input encoder layer as the query, key, and value at the same time, and uses the position information of each element in the feature vector on the original two-dimensional feature map as the reference point to adaptively capture the relevant feature points that need to calculate the attention matrix, and fuse the features of the corresponding positions according to the calculated attention matrix to achieve the fusion of features at different positions of the input feature vector; the feedforward neural network will first increase the number of channels of the input feature vector, and then reduce it to more fully fuse features of different channel dimensions. Specifically, the input parameter of the first linear layer in the feedforward neural network is 256, and the output parameter is 1024, while the input parameter of the second linear layer is 1024, and the output parameter is 256. The image feature encoder can fully fuse the features of different scales in the multi-scale feature vector obtained in step 2 to obtain a feature vector after feature fusion. as output.

[0061] Step 4: Construct a target region diffusion module to obtain the true target region based on the marked target rotation box, and then obtain the noisy target region through the diffusion process as the initial target region.

[0062] When constructing the target area diffusion module, firstly mark the target's rotation box Get its minimum circumscribed horizontal rectangle, and then expand the width and height of the rectangle by 1.2 times to get the real target area ,Right now ,in The function for calculating the minimum horizontal circumscribed rectangle of a quadrilateral. The diffusion model is a likelihood-based model inspired by non-equilibrium thermodynamics. It defines a Markov chain of the diffusion process by gradually adding noise to the sample data. Therefore, during the iterative training process, the present invention constructs a Markov chain from the real target area (strong scattering area) To the noise area The diffusion process, at random sampling time steps , Afterwards, you can use To achieve this diffusion process, , , is the noise variance scheduling parameter, The noise variance scheduling hyperparameter is adopted in this invention. Through the above formula, we can get The real target area is obtained Noisy target area Since the DetectionTransformer architecture used in the present invention is an end-to-end architecture, it requires the length of the input vector and the output vector to be the same. Therefore, considering that the present invention needs to predict A result (output), in the After the target noise area is obtained, it needs to be filled or deleted to obtain the initial target area. Specifically, when When random sampling based on Gaussian distribution Regions are spliced ​​in After the noisy target area, we get Initial target area ; and when When The front of the noisy target area As the initial target area ;when When , directly The noisy target area is used as the initial target area. In addition, the construction of this module in the inference process is slightly different from that in the training process. In the inference process, the module directly generates region as the initial target region , without the need to obtain the noisy target area based on the real target area.

[0063] Step 5: Construct the target feature initialization module and time step encoding module.

[0064] When constructing the target feature initialization module, first use the initial target area obtained by the target area diffusion module in step 4 The center point position information is extracted to extract the feature vector obtained by the image feature encoder in step 3 The features of the corresponding positions are extracted and concatenated in the channel dimension. Then, the channel dimension is transformed to 256 through a linear layer, so that a feature vector is obtained for each initial target area. These feature vectors together constitute the target feature vector. Specifically, for the first Target initialization area , the extracted feature vector is ,in for The center point coordinates, The number of channels is 1024 (the number of channels of each scale feature map is 256, and after extracting features from the corresponding positions of the four scale feature vectors and splicing them in the channel dimension, the number of channels becomes 1024). Initial target area , we can get the length of , a feature vector with 1024 channels, and then a linear layer with an input parameter of 1024 and an output parameter of 256, the final length is , the initial target feature vector with 256 channels.

[0065] When constructing the time step encoding module, first convert the time step sampled in step 4 into , using cosine position coding, we get a time code with 256 channels ,in It is a cosine position encoding function, and then after a linear layer with an input parameter of 256 and an output parameter of 1024, the time encoding after the number of channels is expanded by 4 times is obtained. , and then pass through a SiLU activation function and a linear layer with input and output parameters of 1024, to obtain the final time encoding output with a length of 1 and a channel number of 1024 .

[0066] Step 6: Construct the denoising decoder and multi-scale scattering key point feature fusion module, see Figure 5 and Figure 6 .

[0067] The denoising decoder consists of six denoising decoder layers and six corresponding target region update modules. The denoising decoder layer consists of a multi-head self-attention layer, a batch normalization layer, a multi-scale scattering keypoint feature fusion module, a batch normalization layer, a feedforward neural network, and an adaptive batch normalization layer; the multi-scale scattering keypoint feature fusion module consists of four parallel linear layers, a linear layer, and a cross-attention layer; the feedforward neural network consists of a linear layer, a ReLU activation function, and a linear layer; and the target region update module consists of a linear layer, a ReLU activation function, a linear layer, a ReLU activation function, and a linear layer.

[0068] Specifically, the first denoising decoder layer in the denoising decoder has a length of , the initial target feature vector with 256 channels is used as input, and the subsequent denoising decoder layers all use the target feature vector updated by the previous denoising decoder layer as input. In each denoising decoder layer, the multi-head self-attention layer uses the target feature vector of the input denoising decoder layer as query, key and value at the same time to calculate The attention matrix between the target feature vectors is obtained, and the features at different positions are fused according to the calculated attention matrix to obtain a length of , the output feature vector with 256 channels; in the multi-scale scattering key point feature fusion module, the length of the multi-head self-attention layer is , the feature vector with 256 channels will pass through 4 parallel linear layers with 256 input parameters and 18 output parameters, and get 4 lengths of , the prediction results with 18 channels correspond to The 18 horizontal and vertical coordinate offsets of 9 points in the target area on 4 feature scales are obtained. Based on the predicted offsets and the center point coordinates of the target area, 9 sampling points can be obtained for each target area on 4 feature scales. These sampling points are the scattering key points dynamically predicted in the target area. According to these predicted scattering key points, the corresponding positions of the feature vector obtained in step 3 can be extracted, and the features of the same point at different scales can be spliced ​​together in the channel dimension, so that a feature vector with a length of 36 and a number of channels of 1024 is obtained for each target area. After passing through a linear layer with an input parameter of 1024 and an output parameter of 256, a scattering key point feature vector with a length of 36 and a number of channels of 256 is obtained. The feature vector is used as the key and value, and the input target feature vector of this module is used as a query and sent to the cross attention layer to obtain the target feature vector after the scattering key point features are integrated, and its length is , the number of channels is 256; in the feedforward neural network, the length obtained by the multi-scale scattering key point feature fusion module is , the target feature vector with 256 channels is obtained after a linear layer with an input parameter of 256 and an output parameter of 1024, with a length of , the feature vector with 1024 channels, and then after the ReLU activation function to obtain nonlinear learning ability, through the linear layer with an input parameter of 1024 and an output parameter of 256, the length is obtained. , the target feature vector with 256 channels is output.

[0069] In addition, due to the diffusion process constructed by the target area diffusion module in step 4 of the present invention, it is necessary to The Gaussian noise generated by the guide is used to obtain the noisy target area. Therefore, the denoising decoder also needs to be based on the sampling time step. To guide the denoising process (update process) of the initial target area. To achieve this function, the present invention introduces an adaptive batch normalization layer after the feedforward neural network of the denoising decoder layer. The adaptive batch normalization layer introduces a time step-dependent Specifically, the time step encoding module in step 5 obtains a time step encoding with a length of 1 and a number of channels of 1024. After the linear layer with an input parameter of 1024 and an output parameter of 512 and the SiLU activation function, an output vector with a length of 1 and a number of channels of 512 is obtained. The vector is divided into two vectors with a length of 1 and a number of channels of 256 in the channel dimension and copied parts, we can get the length of , the scale scaling parameter with 256 channels and offset parameters , ,in is the channel averaging function, is the replication function, is a linear layer. In the adaptive batch normalization layer, the length of the feedforward neural network is , a feature vector with 256 channels , first through the ordinary batch normalization layer, and then use the above-mentioned sampling time step The obtained scale scaling and offset parameters are processed to obtain the final length , the number of channels is 256, output feature vector , ,in is a normal batch normalization layer. That is, the output target feature vector updated for each denoising decoder layer.

[0070] The target feature vector output by each denoising decoder layer is divided into two branches. One branch is directly input to the next decoder layer as the input target feature vector, and the other branch is input to the target region update module to update the target region according to the updated target feature vector and use the updated target region in the multi-scale scattered key point module in the next decoder layer, so that it can predict more accurate scattered key points. In the target region update module, the length is , the input target feature vector with a channel number of 256, after passing through a linear layer with an input parameter of 256 and an output parameter of 4, obtains a length of , the prediction result with 4 channels corresponds to The target area is defined as the horizontal and vertical coordinates, width and height offsets of the center point of the target area. The parameters of the target area can be updated based on these predicted offsets. The original target area is defined as , the predicted offset is , then the updated target area .

[0071] Step 7: Construct the target convex polygon prediction module, which consists of a classification branch and a regression branch. The target feature vector and target area finally updated by the denoising decoder in step 6 are used as input to obtain the target convex polygon prediction module. The classification and convex polygon prediction results of the target are shown in Figure 7 .

[0072] In the classification branch, the final updated length of the denoising decoder in step 6 is , the target feature vector with 256 channels, after the input parameter is 256, the output parameter is After the linear layer, we get a length of , the number of channels is The prediction results, among which, The number of categories in the dataset is marked, and the prediction result on each channel is the confidence prediction score of a category;

[0073] In the regression branch, the final updated length of the denoising decoder in step 6 is , the target feature vector with 256 channels is enhanced by a linear layer with 256 input and output parameters, a ReLU activation function, a linear layer with 256 input and output parameters, and a ReLU activation function. Finally, a linear layer with 256 input parameters and 18 output parameters is obtained to obtain a length of , the prediction result with 18 channels corresponds to The 18 horizontal and vertical coordinate offsets of 9 points in each target area are calculated. Based on these predicted offsets and the horizontal and vertical coordinates of the corresponding target area center point, 9 predicted points can be obtained for each target area, forming the target convex polygon prediction for that area. By calculating the minimum horizontal circumscribed rectangle of the predicted target convex polygon, the predicted target convex polygon can be converted into a rotation box prediction result.

[0074] Step 8: The SAR image rotation target detection network based on the scattering key point guided diffusion model constructed in the above steps is trained on the dataset. Figure 8 The network parameter optimization process is based on the total network loss value obtained during training. When the total loss value converges to a certain value as the number of network training iterations decreases and no longer fluctuates widely, it can be judged that the network has completed convergence. The network weight parameter value at this time is extracted and saved as the final network weight obtained by training. Figure 4 As shown, the total loss function of the present invention is divided into three parts, and the specific calculation method is as follows:

[0075] The first part is the category prediction loss , you can use Calculated, where For the The predicted category probability of the predicted results, For the The category probability of the annotation box corresponding to the prediction result, the loss function is essentially the The difference between the predicted category probability of a prediction result and the corresponding true category probability.

[0076] The second part is the convex polygon prediction loss ,use The formula is calculated, where For the The prediction target convex polygon of the prediction results, For the first The target real rotation box annotation corresponding to the prediction result, is and The minimum circumscribed polygon of The degree of fit between the predicted convex polygon of the prediction result and the corresponding target real rotation box;

[0077] The third part is the target area regression loss ,use The formula is calculated, where For the The regression target area corresponding to the prediction result is For the The target true area corresponding to the prediction result is is the minimum enclosing area between the regression target area and the target real area. The loss function is essentially The gap between the predicted target area and the true target area corresponding to the prediction result.

[0078] The parameters set during the training process of the embodiment of the present invention are as follows Table 1:

[0079] Table 1 Network training parameter settings

[0080]

[0081] Among them, the Batch parameter controls the number of images sent to the network for training each time; Img width and Img height are the sizes that need to be maintained for the input images during training. When the input image size is different from this size, it can be changed to the set size through proportional scaling and zero-filling strategies; Learning rate is the learning rate of training, which determines the update amplitude of the network parameters and the convergence speed of the network; Optimizer is the parameter optimizer of the network, which sets its parameters ; Param scheduler is the learning rate reduction strategy during network training. This embodiment selects the MultiStepLR strategy, that is, whenever training reaches a specified number of iterations, the learning rate drops to one-tenth of the original value, and the learning rate drops once when the iteration reaches the 175,000th and 215,000th times respectively; Warm up is the learning rate setting strategy for the initial stage of training, that is, at the beginning of training, the learning rate is changed to 1% of the set learning rate, and in the subsequent 2,000 steps of iteration, the learning rate is linearly increased to the set learning rate to stabilize the parameter optimization process of the network in the initial stage of training; Numsteps is the total number of iterations of training. This embodiment selects 225,000 as the total number of iterations. In actual operation, relevant technical personnel can change the above training parameters or add additional training parameters according to the specific implementation situation to achieve the best training effect for the network.

[0082] The specific embodiments described herein are merely illustrative of the spirit of the present invention. Persons skilled in the art may make various modifications, additions, or substitutions to the described specific embodiments without departing from the spirit of the present invention or exceeding the scope of the appended claims.

Claims

1. A method for detecting rotating targets in SAR images based on a scattering keypoint guided diffusion model, characterized in that: include: Construct and train a SAR image rotating target detection network based on a scattering keypoint guided diffusion model, wherein the SAR image rotating target detection network includes a feature extraction module, a multi-scale feature mapping module, an image feature encoder, a target region diffusion module, a target feature initialization module and a time step encoding module, a denoising decoder module and a multi-scale scattering keypoint feature fusion module, and a target convex polygon prediction module; The feature extraction module is used to extract feature maps of different scales, input them into the multi-scale feature mapping module, obtain multi-scale feature vectors, input them into the image feature encoder, and fully fuse the features of different scales in the multi-scale feature vectors to obtain the feature vector after multi-scale feature fusion as output; The target region diffusion module is used to annotate the rotating box of the target in each image, obtain the center point coordinates, width and height of the initial target region through a diffusion process based on Gaussian distribution, and extract the features corresponding to each feature scale from the feature vector output by the image feature encoder based on the center point coordinates of the initial region. The features of different feature scales are then fused to obtain a feature vector for each initial target region, which is used as the initial target feature vector. The time step encoding module is used to encode and output the sampling time step t in the target area diffusion module; the denoising decoder module first uses the initial target area and the initial target feature vector as a reference, and based on the multi-scale scattering key point feature fusion module, dynamically predicts scattering key points at different feature scales near the target area, and adaptively extracts features at the corresponding feature scales from the feature vector obtained from the image feature encoder based on the predicted scattering key points as the key features of the target, and then fuses these extracted key features with the target feature vector to update the target feature vector, and then updates the target area parameters based on the updated target feature vector to obtain a more accurate target area; the target convex polygon prediction module will predict the confidence score of each category for each target and a convex polygon point set consisting of 9 points based on the target feature vector and target area parameters finally updated by the denoising decoder module, thereby obtaining the target classification result and convex polygon prediction result; The loss functions used in the training process are: the category prediction loss function is calculated by the predicted target category, confidence and the target true category label; the convex polygon prediction loss function is calculated by the predicted target convex polygon and the target true rotation box label parameters; the target area regression loss function is calculated by the final updated target area and the true target area, and then all the loss functions are added together to get the total loss function; Finally, the trained target detection network is used to detect the data to be detected and obtain the required detection results.

2. The rotating target detection method in SAR images based on the scattering key point guided diffusion model according to claim 1, wherein: The feature extraction module consists of five parts. The first part consists of a 7×7 convolutional layer with 64 channels, a stride of 2, and a padding of 3, a batch normalization layer, and a 3×3 maximum pooling layer with a stride of 2 and a padding of 1. The second part consists of one convolutional block and two residual blocks. The third part consists of one convolutional block and three residual blocks. The fourth part consists of one convolutional block and five residual blocks. The fifth part consists of one convolutional block and two residual blocks. The convolutional block consists of two parallel The output of the two branches is added together and then passed through the ReLU activation function to obtain the final convolution block output; the residual block is composed of a 1×1 convolution layer, a ReLU activation function, a batch normalization layer, a 3×3 convolution layer, a ReLU activation function, a batch normalization layer, a 1×1 convolution layer and a batch normalization layer. The output is added together with the input of the residual block and then passed through the ReLU activation function to obtain the final residual block output; in each convolution block, the number of channels of the last 1×1 convolution layer of the two branches is twice the number of channels of the input feature map; the number of channels of the last convolution layer in the residual block is the number of channels of the input feature map, and the number of channels of the remaining convolution layers is 1 / 4 the number of input feature map channels; except for the second part, all 3×3 convolution layers in the remaining convolution blocks and residual blocks have a stride of 2 and a padding of 1, the 1×1 convolution layers have a stride of 1 and a padding of 0, and the stride of all pooling layers is 2 and the padding is 1; while all 3×3 convolution layers in the second part of the convolution blocks and residual blocks have a stride of 1 and a padding of 1, the 1×1 convolution layers have a stride of 1 and a padding of 0, and the stride of all pooling layers is 2 and the padding is 1.

3. The method for detecting rotating targets in SAR images based on a scattering key point guided diffusion model according to claim 2, wherein: The specific processing process of the multi-scale feature mapping module is as follows: The input feature map is the feature map obtained by the third, fourth and fifth parts of the feature extraction module in step 1; The above three input feature maps are respectively subjected to a 3×3 convolution layer with a stride of 1 and a channel of 256 to obtain three output feature maps; then, the last output feature map will be subjected to another 3×3 convolution layer with a stride of 2 and a channel of 256 to obtain the fourth output feature map; the above four output feature maps will be transformed into one-dimensional vectors in the spatial dimension and spliced ​​in the dimension to finally obtain a multi-scale feature vector.

4. The method for detecting rotating targets in SAR images based on a scattering key point guided diffusion model according to claim 1, wherein: The specific processing process of the image feature encoder is as follows: The image feature encoder consists of several encoder layers, each of which consists of a multi-head deformable self-attention layer, a batch normalization layer, a feedforward neural network, and a batch normalization layer. The feedforward neural network consists of a linear layer, a ReLU activation function, and a linear layer. Specifically, in the image feature encoder, the first encoder layer takes the multi-scale feature vector obtained by the multi-scale feature mapping module as input, and the subsequent encoder layers take the output feature vector of the previous encoder layer as input. In each encoder layer, the multi-headed deformable self-attention layer uses the feature vector of the input encoder layer as the query, key, and value at the same time, and uses the position information of each element in the feature vector on the original two-dimensional feature map as the reference point to adaptively capture the relevant feature points that need to calculate the attention matrix, and fuses the features of the corresponding positions according to the calculated attention matrix to achieve the fusion of features at different positions of the input feature vector; the feedforward neural network will first increase the number of channels of the input feature vector and then reduce it to more fully fuse features of different channel dimensions; the image feature encoder fully fuses the features of different scales in the obtained multi-scale feature vector to obtain the feature vector after feature fusion as output.

5. The method for detecting rotating targets in SAR images based on a scattering key point guided diffusion model according to claim 1, wherein: The specific implementation process of the target area diffusion module is as follows: First, mark the target's rotation box Get its minimum circumscribed horizontal rectangle, and then expand the width and height of the rectangle by b times to get the real target area ,Right now ,in A function to calculate the minimum horizontal circumscribed rectangle of a quadrilateral; During the iterative process of training, the real target area is constructed To the noise area The diffusion process, at random sampling time steps , Afterwards, use To achieve this diffusion process, , , is the noise variance scheduling parameter, Schedule hyperparameters for noise variance; through the above formula, we can get The real target area is obtained Noisy target area In addition, due to the need to predict Results, in After the target noise area is obtained, it needs to be filled or deleted to obtain the initial target area; specifically, when When random sampling based on Gaussian distribution Regions are spliced ​​in After the noisy target area, we get Initial target area ; and when When The front of the noisy target area As the initial target area ;when When , directly The noisy target area is used as the initial target area; in the inference process, the target area diffusion module is directly generated based on the Gaussian distribution region as the initial target region , without the need to obtain the noisy target area based on the real target area.

6. The method for detecting rotating targets in SAR images based on a scattering key point guided diffusion model according to claim 1, wherein: The specific processing procedures of the target feature initialization module and the time step encoding module are as follows: When constructing the target feature initialization module, first obtain the initial target area according to the target area diffusion module. The center point position information is extracted and the feature vector obtained by the image feature encoder is extracted The features of the corresponding positions are extracted, and the extracted features are spliced ​​together in the channel dimension, and then the channel dimension is transformed to 256 through a linear layer, so that a feature vector is obtained for each initial target area, and these feature vectors together constitute the target feature vector; specifically, for the first Target initialization area , the extracted feature vector is ,in for The center point coordinates of Initial target area , we can get the length of The feature vector of , and then through 1 linear layer, the final initial target feature vector can be obtained; When constructing the time step encoding module, first convert the sampled time steps , using the cosine position encoding method, we get a time code with 256 channels, and then after a linear layer, we get a time code with 4 times the number of channels, and then after a SiLU activation function and a linear layer, we get the final time code output .

7. The method for detecting rotating targets in SAR images based on a scattering key point guided diffusion model according to claim 1, wherein: The specific processing process of the denoising decoder module and the multi-scale scattering key point feature fusion module is as follows: The denoising decoder module consists of several denoising decoder layers and corresponding target region update modules. The denoising decoder layer consists of a multi-head self-attention layer, a batch normalization layer, a multi-scale scattering keypoint feature fusion module, a batch normalization layer, a feedforward neural network, and an adaptive batch normalization layer. The multi-scale scattering keypoint feature fusion module consists of four parallel linear layers, a linear layer, and a cross-attention layer. The feedforward neural network consists of a linear layer, a ReLU activation function, and a linear layer. The target region update module consists of a linear layer, a ReLU activation function, a linear layer, a ReLU activation function, and a linear layer. Specifically, the first denoising decoder layer in the denoising decoder takes the initial target feature vector obtained by the target feature initialization module as input, and the subsequent denoising decoder layers all take the target feature vector updated by the previous denoising decoder layer as input; in each denoising decoder layer, the multi-head self-attention layer takes the target feature vector of the input denoising decoder layer as query, key and value at the same time to calculate The attention matrix between the target feature vectors is obtained, and the features at different positions are fused according to the calculated attention matrix to obtain a length of , the output feature vector with 256 channels; The specific processing process of the multi-scale scattering key point feature fusion module is as follows: The length of the multi-head self-attention layer is , the feature vector with a channel number of 256 passes through 4 parallel linear layers to obtain 4 lengths , the prediction results with 18 channels correspond to The coordinate offsets of 9 points in the target area at 4 characteristic scales are calculated based on these predicted offsets and The center point coordinates of the target area are obtained for each target area at 4 feature scales. These points are the scattering key points dynamically predicted in the target area. According to the position information of these points, features are extracted from the corresponding positions on the obtained multi-scale feature vector, and the features of the same point at different scales are spliced ​​together in the channel dimension. For each target area, a scattering key point feature vector is obtained. After a linear layer, a feature vector with a length of 36 and a number of channels of 256 is obtained. The feature vector is used as the key and value, and the input feature vector of the multi-scale scattering key point feature fusion module is sent as a query to the multi-head self-attention layer to obtain the target feature vector after the fusion of the scattering key point features. Its length is , the number of channels is 256.

8. The method for detecting rotating targets in SAR images based on a scattering key point guided diffusion model according to claim 7, wherein: The adaptive batch normalization layer is based on the ordinary batch normalization layer and introduces the time step Related scaling and offset parameters; specifically, the final time-encoded output obtained by the time-step encoding module After the linear layer and SiLU activation function, the output vector is obtained, which is divided into two vectors in the channel dimension and copied , and get the scale scaling parameters respectively and offset parameters , ,in is the channel averaging function, is the replication function, is a linear layer; in the adaptive batch normalization layer, the feature vector obtained by the feedforward neural network , first through the ordinary batch normalization layer, and then use the above-mentioned sampling time step The obtained scale scaling and offset parameters are processed to obtain the final output feature vector , ,in is a normal batch normalization layer, That is, the output target feature vector updated by each denoising decoder layer; In addition, the target feature vector output by each denoising decoder layer is divided into two branches. One branch is directly input into the next decoder layer as the input target feature vector, and the other branch is input into the target region update module to update the target region according to the updated target feature vector. The updated target region is then used in the multi-scale scattering key point module in the next decoder layer, so that it can predict more accurate scattering key points. In the target region update module, the input target feature vector is passed through the linear layer to obtain a length of , the prediction result with 4 channels corresponds to The horizontal and vertical coordinates, width and height offsets of the center point of the target area are calculated. The parameters of the target area can be updated based on these predicted offsets. The original target area is defined as , the predicted offset is , then the updated target area .

9. The method for detecting rotating targets in SAR images based on a scattering key point guided diffusion model according to claim 1, wherein: The target convex polygon prediction module includes classification branch and regression branch. The specific processing process is as follows: In the classification branch, the denoising decoder finally updates the target feature vector, which has a length of , the number of channels is The prediction results, among which, The number of categories in the dataset is marked, and the prediction result on each channel is the confidence prediction score of a category; In the regression branch, the denoising decoder finally updates the target feature vector, which is enhanced by the linear layer, ReLU activation function, linear layer and ReLU activation function, and finally passes through a linear layer to obtain a length of , the prediction result with 18 channels corresponds to The 18 horizontal and vertical coordinate offsets of 9 points in the target area are obtained. According to these predicted offsets and the horizontal and vertical coordinates of the center point of the corresponding target area, 9 predicted points can be obtained for each target area, forming the target convex polygon prediction of the target area. By calculating the minimum circumscribed horizontal rectangle of the predicted target convex polygon, the predicted target convex polygon is converted into a rotation box prediction result.

10. A SAR image rotating target detection system based on a scattering keypoint guided diffusion model, characterized by: The method comprises a processor and a memory, wherein the memory is used to store program instructions, and the processor is used to call the stored instructions in the memory to execute the SAR image rotating target detection method based on the scattering key point guided diffusion model according to any one of claims 1 to 9.

Images