A SAR aircraft detection method, system, device and storage medium

Abstract

The present invention discloses a SAR aircraft detection method, system, device and storage medium. The method includes: obtaining an input SAR image; using an aircraft detection model to analyze the input SAR image to obtain an aircraft detection result; the aircraft detection model is generated by training a SAR image data set that has been labeled with aircraft categories and target bounding boxes; the aircraft detection model includes a classification branch and a regression branch, the classification branch has a deformable region association module, and the deformable region association module performs feature weighted integration through a deformable convolution branch and a conventional convolution branch. The present invention constructs a classification branch through a deformable association module, significantly improving the feature association capability, and performs model training through a SAR image data set that has been labeled with aircraft categories and target bounding boxes, making full use of prior knowledge of SAR aircraft scattering characteristic information. The present invention improves the performance of detecting aircraft in complex SAR images and achieves accurate aircraft detection and identification.

Description

Technical Field

[0001] The present invention relates to the technical field of aircraft detection, and in particular to a SAR aircraft detection method, system, device and storage medium. Background Art

[0002] Synthetic Aperture Radar (SAR) is an active microwave imaging sensor. Its all-day, all-weather imaging observation capabilities make it a crucial tool for automatic target recognition (ATR). Aircraft detection is a highly valuable application within this field. For example, in civilian applications, dynamic monitoring of aircraft movement facilitates efficient airport management. In the military, rapid and accurate aircraft detection is crucial for providing reconnaissance information. Therefore, accurately detecting aircraft in high-resolution SAR imagery is a highly valuable research topic.

[0003] SAR uses an antenna to transmit electromagnetic waves to an object, receives the reflected electromagnetic waves, and finally forms an image by recording the echo information. Due to its unique imaging mechanism, SAR images present complex appearances that are difficult to interpret. The original method of detecting objects in SAR images is constant false alarm detection (CFAR), which takes advantage of the fact that the echo of an object is often stronger than the background echo. Areas with strong echoes are extracted from the SAR image and regarded as targets. However, in complex scenarios, such as airports with various other equipment or buildings, the detection performance of CFAR will be greatly affected, and it will not be able to accurately identify the target. In addition, existing aircraft detection methods also require the ability to identify the category of the target, which is something CFAR cannot achieve.

[0004] Aircraft possess various classic reflective structures, such as dihedrals, trihedrals, and top hats. Under radar illumination, they are subject to a variety of scattering mechanisms, including direct scattering, multiple scattering, and diffraction scattering. Consequently, in later aircraft identification methods, experts in the SAR field proposed utilizing the scattering characteristics of aircraft in SAR images and employing template matching for detection. Feature extraction for matching is a key step. For example, one method uses the Harris-Laplace corner detector to extract salient points on an aircraft with stable scattering characteristics and describes them using salient point vectors. Other methods utilize Gaussian mixture models to extract scattering structure features, including strong scattering points on the aircraft and their corresponding distribution. However, due to the inadequacy of manual feature extraction and the inefficiency of matching measured images with multiple candidate templates, these methods still suffer from limitations in accuracy and speed.

[0005] With the rapid development of SAR technology, more high-resolution, high-quality, expertly annotated SAR images are becoming available, paving the way for the application of deep learning methods in SAR target automatic recognition. In recent years, deep learning methods based on convolutional neural networks (CNNs) have made significant progress in target detection. Many CNN methods, such as YOLOX, Cascade R-CNN, CenterNet, and RepPoints, have demonstrated superior performance compared to traditional methods in SAR target detection. However, most CNN methods are designed for optical target detection and cannot fully exploit their detection performance when directly applied to SAR aircraft detection without considering the aircraft's scattering characteristics. Aircraft scattering characteristics are specifically manifested in two aspects: 1) Discreteness. Due to the irregular distribution of various reflective structures on an aircraft, the radar cross section (RCS) varies across different parts of the aircraft. Therefore, an aircraft in a SAR image appears as a collection of discrete scattering points. 2) Variability. Due to the complex structure of an aircraft and the presence of multiple scattering mechanisms, the imaging results vary with the incident angle and sensor parameters. Therefore, even the scattering results for the same target can vary significantly under different imaging conditions. Under the SAR imaging mechanism, SAR aircraft images exhibit characteristics that are completely different from the corresponding optical images. Therefore, existing CNN methods cannot fully extract aircraft features through conventional convolution.

[0006] In view of this, how to use the prior knowledge of SAR aircraft scattering characteristic information to achieve high-precision aircraft detection performance is an urgent problem that needs to be solved. Summary of the Invention

[0007] In view of this, embodiments of the present invention provide a SAR aircraft detection method, system, device, and storage medium, which can effectively improve the detection and recognition performance of a network model for aircraft.

[0008] In one aspect, an embodiment of the present invention provides a SAR aircraft detection method, comprising:

[0009] Get the input SAR image;

[0010] Using the aircraft detection model, the input SAR image is analyzed to obtain the aircraft detection results;

[0011] The aircraft detection result includes at least one target bounding box regression result and the corresponding aircraft category confidence score;

[0012] The aircraft detection model is trained on a SAR image dataset that has been labeled with aircraft categories and target bounding boxes. The aircraft detection model consists of a classification branch and a regression branch. The classification branch has a deformable region association module, which performs weighted feature integration through a deformable convolution branch and a conventional convolution branch.

[0013] Optionally, it also includes:

[0014] Based on the classification branch and regression branch with the deformable region association module, a deformable scattering feature association network model is created;

[0015] Among them, the deformable scattering feature association network model includes a Swin Transformer backbone, a path aggregation feature pyramid network and a decoupling head; the decoupling head includes a classification branch and a regression branch with a deformable region association module.

[0016] Optionally, a deformable scattering feature association network model is created based on the classification branch and the regression branch with the deformable region association module, including:

[0017] Build a YOLOX model; the YOLOX model includes a Darknet-53 backbone, a path aggregation feature pyramid network, and an original decoupling head; the original decoupling head includes the original classification branch and the regression branch;

[0018] By introducing the Swin Transformer backbone to replace the Darknet-53 backbone of the YOLOX model, and by introducing the classification branch with a deformable region association module to replace the original classification branch of the YOLOX model, a deformable scattering feature association network model is created;

[0019] Among them, the Swin Transformer backbone includes 24 Swin Transformer layers.

[0020] Optionally, it also includes:

[0021] Determine training samples based on a SAR image dataset that has been labeled with aircraft categories and target bounding boxes, wherein a preset number of strong scattering regions are marked in the target bounding boxes;

[0022] According to the training samples and based on the preset training parameters, stochastic gradient descent (SGD) training is used to perform classification and regression training on the deformable scattering feature association network model. The network model parameters are optimized through the loss value back propagation method in combination with the model loss function to obtain the aircraft detection model.

[0023] Optionally, the aircraft detection model is used to analyze the input SAR image to obtain aircraft detection results, including:

[0024] The SAR image is segmented, spliced ​​and linearly mapped to obtain the first feature map;

[0025] Through the Swin Transformer backbone, continuous Swin Transformer layers are used to alternately perform regular window partitioning and transfer window partitioning to obtain a second feature map of preset specifications;

[0026] Performing semantic feature extraction on the second feature map through a path aggregation feature pyramid network to obtain a third feature map of preset specifications; wherein the semantic feature extraction includes upsampling, splicing, and convolution;

[0027] Through the regression branch, the position of the third feature map is predicted to obtain the target bounding box regression result; wherein the target bounding box regression result includes the center point, width and height of the bounding box;

[0028] Through the classification branch with a deformable region association module, the deformable convolution branch and the conventional convolution branch are used to perform feature weighted integration on the third feature map to obtain the category confidence score of the preset category of aircraft corresponding to the target bounding box regression result.

[0029] Optionally, the Swin Transformer layer includes a normalization function, a window-based multi-head self-attention mechanism, and a multi-layer perceptron. The steps of processing the feature map by the Swin Transformer layer include:

[0030] Normalize the target feature map through the normalization function to obtain the feature map X2;

[0031] According to the feature map X2, a window-based multi-head self-attention mechanism is used to obtain the feature map X3 through linear mapping and channel-based dimension splicing;

[0032] According to the addition of the target feature map and the feature map X3, the feature map X4 is obtained;

[0033] Normalize the feature map X4 through the normalization function to obtain the feature map X5;

[0034] Through the multi-layer perceptron, the GELU nonlinear activator is used to activate the feature map X5 to obtain the feature map X6; wherein the multi-layer perceptron includes two fully connected layers;

[0035] By adding the feature map X4 and the feature map X6, the feature map X7 is obtained.

[0036] Optionally, a classification branch having a deformable region association module is used to perform feature weighted integration on the third feature map using a deformable convolution branch and a conventional convolution branch to obtain a category confidence score of an aircraft of a preset category corresponding to the target bounding box regression result, including:

[0037] Through the deformable convolution branch, the first convolution is performed on the third feature map to obtain the feature map X9;

[0038] Perform a second convolution on the feature map X9 to obtain a sampling point offset map; perform a third convolution on the feature map X9 to obtain a score mask;

[0039] Perform deformable convolution based on the feature map X9, the sampling point offset map and the score mask to obtain the feature map Y;

[0040] Through the conventional convolution branch, the third feature map is convolved twice continuously to obtain the feature map Z;

[0041] Based on the preset learnable hyperparameters, the feature maps Y and Z are weighted added and the feature channel information is integrated to obtain the category confidence score of the preset category of aircraft corresponding to the target bounding box regression result.

[0042] In another aspect, an embodiment of the present invention provides a SAR aircraft detection system, comprising:

[0043] The first module is used to obtain the input SAR image;

[0044] The second module is used to analyze the input SAR image using the aircraft detection model to obtain the aircraft detection results;

[0045] The aircraft detection result includes at least one target bounding box regression result and the corresponding aircraft category confidence score;

[0046] The aircraft detection model is trained on a SAR image dataset that has been labeled with aircraft categories and target bounding boxes. The aircraft detection model consists of a classification branch and a regression branch. The classification branch has a deformable region association module, which performs weighted feature integration through a deformable convolution branch and a conventional convolution branch.

[0047] On the other hand, an embodiment of the present invention provides a SAR aircraft detection device, including a processor and a memory;

[0048] The memory is used to store programs;

[0049] The processor executes the program to implement the above method.

[0050] On the other hand, an embodiment of the present invention provides a computer-readable storage medium, wherein the storage medium stores a program, and the program is executed by a processor to implement the above method.

[0051] The present invention also discloses a computer program product or computer program, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device can read the computer instructions from the computer-readable storage medium and execute the computer instructions, causing the computer device to perform the above method.

[0052] The embodiment of the present invention first obtains an input SAR image; uses an aircraft detection model to analyze the input SAR image to obtain an aircraft detection result; wherein the aircraft detection result includes at least one target bounding box regression result and a corresponding aircraft category confidence score; the aircraft detection model is generated by training a SAR image dataset that has been labeled with aircraft categories and target bounding boxes; the aircraft detection model includes a classification branch and a regression branch, the classification branch has a deformable region association module, and the deformable region association module performs feature weighted integration through a deformable convolution branch and a conventional convolution branch. The present invention significantly improves the feature association capability by introducing a deformable association module to construct a classification branch, and further performs model training through a SAR image dataset that has been labeled with aircraft categories and target bounding boxes, making full use of the prior knowledge of SAR aircraft scattering feature information. The present invention improves the performance of detecting aircraft in complex SAR images and achieves accurate aircraft detection and recognition. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0054] Figure 1 A schematic flow chart of a SAR aircraft detection method provided in an embodiment of the present invention;

[0055] Figure 2 It is a structural diagram of the YOLOX model;

[0056] Figure 3 A schematic diagram of the structure of the Swin Transformer provided in an embodiment of the present invention;

[0057] Figure 4 A schematic diagram of the structure of the Swin Transformer layer provided in an embodiment of the present invention;

[0058] Figure 5 A schematic diagram of the structure of a DRCM provided in an embodiment of the present invention;

[0059] Figure 6A schematic diagram of the structure of the DSFCN provided in an embodiment of the present invention;

[0060] Figure 7 A schematic diagram of the process of constructing a DSFCN model provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0061] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0062] First of all, it should be noted that existing SAR aircraft detection technologies include:

[0063] 1. In their paper, “Aircraft Recognition in SAR Images Based on Scattering Structure Feature and Template Matching.” (IEEE Journal of Selected Topics in Applied Earth Observation and Remote Sensing, (2018) 4206-4217), Fu et al. proposed a template matching-based aircraft recognition method. This method analyzes the scattering characteristics of aircraft and uses a Gaussian mixture model to extract the target's scattering structure features. This method improves the efficiency of candidate template selection in the matching phase through a proposed sample decision optimization algorithm. In the detection phase, a coordinate translation Kullback-Leibler divergence method is proposed to achieve translation invariance in detection. However, this improved template matching method still suffers from low matching accuracy and requires handcrafted features based on specific data attributes, making it difficult to apply to SAR images of different resolutions.

[0064] 2. Zhao et al. proposed a pyramid attention dilated network to detect aircraft in their paper “Pyramid Attention Dilated Network for Aircraft Detection in SAR Images.” (IEEE GEOSCIENCE AND REMOTE SENSINGLETTERS, (2021) 662-666.). This method takes into account the discrete characteristics of SAR aircraft, uses a multi-branch dilated convolution module to enhance the relationship between the discrete backscatter features of the aircraft, and uses a convolutional fast attention module to refine redundant information and highlight the salient features of the aircraft. Although this method takes into account the discrete characteristics of the aircraft, the sampling flexibility of its convolution module for feature extraction is still limited. It can only sample on a regular grid, which is not enough to fully model the relationship between important discrete areas of the aircraft, thereby limiting the detection performance.

[0065] 3. Guo et al. proposed a scattering enhanced attention pyramid network to detect aircraft in their paper “Scattering Enhanced Attention Pyramid Network for Aircraft Detection in SAR Images.” (IEEE Transactions on Geoscience and Remote Sensing, (2021) 7570-7587.). This method extracts strong scattering points of aircraft through the Harris-Laplace corner detector, and then models them through density-based noise spatial clustering method and Gaussian mixture model. Finally, the correlation between the known target and the template is measured by Kullback-Leibler divergence. If they match, the scattering information of the SAR image is enhanced and then fed into the network. Although this method considers the idea of ​​using prior knowledge to model SAR aircraft and then assist the network in extracting features, it only performs γ-distributed CFAR enhancement on some images in the image preprocessing stage, which cannot give full play to the powerful modeling capabilities of the network.

[0066] However, the relevant existing technologies have the following shortcomings: 1. The existing detection methods based on template matching have low matching accuracy and require manual feature creation based on the attributes of specific data, which is not suitable for SAR data of different resolutions. 2. Most of the existing CNN-based detection methods were originally designed for optical image detection tasks. When introducing SAR aircraft detection, there is still a problem of insufficient consideration of the aircraft scattering characteristics in SAR images, resulting in limited detection effects. 3. The existing method of considering the aircraft scattering characteristics of SAR images to assist the network in extracting features has the problem that manual feature extraction and network feature extraction are two independent processes, and there is no organic combination, which is not conducive to significant improvement of network performance.

[0067] In view of this, on the one hand, referring to Figure 1 , an embodiment of the present invention provides a SAR aircraft detection method, comprising:

[0068] S100, obtaining an input SAR image;

[0069] The input SAR image for aircraft detection is obtained. Usually, a 1m resolution SAR image is used and the image size is unified to 640×640 pixels.

[0070] S200, using the aircraft detection model, analyzing the input SAR image to obtain an aircraft detection result;

[0071] It should be noted that the aircraft detection results include at least one target bounding box regression result and the corresponding aircraft category confidence score; the aircraft detection model is trained using a SAR image dataset that has been labeled with aircraft categories and target bounding boxes; the aircraft detection model includes a classification branch and a regression branch. The classification branch has a deformable region association module, which performs feature weighted integration through a deformable convolution branch and a conventional convolution branch.

[0072] In some embodiments, the method further includes: creating a deformable scattering feature association network model based on a classification branch and a regression branch having a deformable region association module; wherein the deformable scattering feature association network model includes a SwinTransformer backbone, a path aggregation feature pyramid network, and a decoupling head; and the decoupling head includes a classification branch and a regression branch having a deformable region association module. Creating a deformable scattering feature association network model based on the classification branch and the regression branch having the deformable region association module includes: constructing a YOLOX model; wherein the YOLOX model includes a Darknet-53 backbone, a path aggregation feature pyramid network, and an original decoupling head; and the original decoupling head includes an original classification branch and a regression branch; and creating a deformable scattering feature association network model by replacing the Darknet-53 backbone of the YOLOX model with a Swin Transformer backbone, and replacing the original classification branch of the YOLOX model with a classification branch having a deformable region association module; and wherein the Swin Transformer backbone includes 24 Swin Transformer layers.

[0073] Among them, some embodiments also include: determining training samples based on a SAR image dataset with labeled aircraft categories and target bounding boxes; wherein a preset number of strong scattering areas are marked in the target bounding box; according to the training samples, based on preset training parameters, using stochastic gradient descent SGD training to perform classification and regression training on the deformable scattering feature association network model, and combining the model loss function with the loss value back propagation method to optimize the network model parameters to obtain an aircraft detection model.

[0074] In some specific embodiments, the specific steps of obtaining a SAR image dataset with labeled aircraft categories and target bounding boxes are as follows:

[0075] Step (1): The 1m resolution SAR images of multiple airports around the world collected by the Gaofen-3 satellite were uniformly cropped to 640×640 pixels, resulting in a dataset of 2204 SAR images with 5429 aircraft in 7 categories. The category and target rectangular bounding box (i.e., target bounding box) of each aircraft were annotated by experts. The aircraft categories are divided into A220, A320-321, A330, ARJ21, Boeing 737-800, Boeing 787, and others according to the model. The target rectangular bounding box is the circumscribed rectangle of the aircraft.

[0076] Step (2): The SAR images are randomly divided into a training set (i.e., training samples) and a test set in a ratio of 3:1.

[0077] Step (3): Mark the Strong Scattering Region (SSR). For each aircraft in the SAR image, extract 25 SSRs within its target rectangular frame. Further steps are:

[0078] Step (31): Perform mean filtering on the pixels in the border, where the radius of the filter is one tenth of the shorter side of the border.

[0079] Step (32): Select the point with the largest value from the pixels in the filtered border as the center point of the first circular SSR with a radius.

[0080] Step (33): Exclude the pixels that fall within the first SSR and select the point with the largest value from the remaining pixels as the center point of the second SSR.

[0081] Step (34): Repeat the above steps and finally obtain 25 SSRs.

[0082] Specifically, an aircraft detection model is used to analyze the input SAR image to obtain an aircraft detection result, including: segmenting, splicing and linear mapping the SAR image to obtain a first feature map; using a Swin Transformer backbone, using continuous Swin Transformer layers to alternately perform regular window division and transfer window division to obtain a second feature map of preset specifications; using a path aggregation feature pyramid network to extract semantic features from the second feature map to obtain a third feature map of preset specifications; wherein the semantic feature extraction includes upsampling, splicing and convolution; using a regression branch, the position of the third feature map is predicted to obtain a target bounding box regression result; wherein the target bounding box regression result includes the center point, width and height of the bounding box; using a classification branch with a deformable region association module, the deformable convolution branch and the conventional convolution branch are used to perform feature weighted integration on the third feature map to obtain a category confidence score of an aircraft of a preset category corresponding to the target bounding box regression result. It should be noted that the target bounding box regression result includes the regression results of several target bounding boxes detected and identified in the SAR image, and the number of aircraft category confidence scores is consistent with the number of target bounding boxes, and corresponds one-to-one to the aircraft contained in each target bounding box.

[0083] In some embodiments, the Swin Transformer layer includes a normalization function, a window-based multi-head self-attention mechanism and a multi-layer perceptron, and the steps of processing the feature map by the Swin Transformer layer include: normalizing the target feature map through the normalization function to obtain a feature map X2; according to the feature map X2, using the window-based multi-head self-attention mechanism, through linear mapping and channel-based dimension splicing, obtain a feature map X3; according to the addition of the target feature map and the feature map X3, obtain a feature map X4; normalizing the feature map X4 through the normalization function to obtain a feature map X5; through the multi-layer perceptron, activating the feature map X5 using a GELU nonlinear activator to obtain a feature map X6; wherein the multi-layer perceptron includes two fully connected layers; according to the addition of the feature map X4 and the feature map X6, obtain a feature map X7.

[0084] Among them, in some embodiments, through a classification branch with a deformable region association module, a deformable convolution branch and a conventional convolution branch are used to perform feature weighted integration on the third feature map to obtain a category confidence score of an aircraft of a preset category corresponding to the target bounding box regression result, including: performing a first convolution on the third feature map through the deformable convolution branch to obtain a feature map X9; performing a second convolution on the feature map X9 to obtain a sampling point offset map; performing a third convolution on the feature map X9 to obtain a score mask; performing deformable convolution according to the feature map X9, the sampling point offset map and the score mask to obtain a feature map Y; performing two consecutive convolutions on the third feature map through the conventional convolution branch to obtain a feature map Z; based on preset learnable hyperparameters, weighted addition and feature channel information integration are performed on the feature map Y and the feature map Z to obtain a category confidence score of an aircraft of a preset category corresponding to the target bounding box regression result.

[0085] In some specific embodiments, creating a deformable scattering feature correlation network (DSFCN) model includes the following steps:

[0086] (1) Figure 2 As shown in the figure, the YOLOX model is first constructed. The YOLOX model includes the Darknet-53 backbone, the Path Aggregation Feature Pyramid Network (PAFPN), and the original decoupled head; the original decoupled head includes the original classification branch (Classification) and regression branch (Regression). The steps of the YOLOX model for aircraft detection include:

[0087] 1.1. For an input 640×640 SAR image, basic semantic features are extracted using the Darknet-53 backbone, and three feature maps of different sizes are output: 80×80, 40×40, and 20×20.

[0088] 1.2. The three feature maps are fed into the Path Aggregation Feature Pyramid Network (PAFPN) to fully generate semantic features through upsampling, concatenation, convolution and other operations, and output feature maps of three different sizes: 80×80, 40×40, and 20×20.

[0089] 1.3. The three feature maps are fed into the decoupled head (the original decoupled head) to output the final prediction results. The decoupled head consists of a 3×3 convolutional layer and two branches composed of stacked 3×3 convolutional layers: the classification branch and the regression branch. The regression branch outputs the target bounding box regression result for each location on the feature map, including the center point, width, and height of the bounding box, as well as the confidence score of whether the bounding box contains the target. The classification branch outputs the confidence score of each category that may be included in the bounding box predicted by the regression branch.

[0090] (2) Introduce the Swin Transformer backbone to replace the Darknet53 backbone (the structure of the Swin Transformer backbone is as follows Figure 3 ), which includes a total of 24 layers of Swin Transformer layers (STL) (the structure of the Swin Transformer layer is as follows Figure 4 ). For the Swin Transformer backbone, the steps for SAR image processing are:

[0091] 2.1. The input SAR image is evenly divided into 16 parts and spliced, and then linearly mapped to obtain the feature map X1.

[0092] 2.2. The feature map X1 is normalized by a layer normalization function (LN) to obtain the feature map X2.

[0093] 2.3. Use the window-based multi-head self-attention mechanism (W-MSA) to extract features. First, divide the feature map into non-overlapping local windows with 7×7 feature vectors. All feature vectors in each window are considered tokens. Then, use linear mapping to obtain the query vector matrix (Q), keyword vector matrix (K), and content vector matrix (V) of all tokens. The corresponding expressions are as follows:

[0094] Q=X2P Q ,K=X2P K ,V=X2P V

[0095] Among them, P Q 、P K and P V Matrices representing three learnable linear mappings.

[0096] The self-attention mechanism is performed on the obtained matrix, that is, the obtained Q, K, V vectors are divided into n groups along the channel dimension, and each subgroup Q in the n groups iss , K s , V s Execute the self-attention mechanism respectively. The corresponding expressions are as follows:

[0097]

[0098] Here, B represents the learnable relative position encoding matrix, and d represents the channel dimension of the vectors Q, K, and V. The results obtained from each subgroup are then concatenated along the channel dimension to obtain the values ​​of each feature vector on the output feature map X3.

[0099] 2.4. Feature map X3 is added to the original input feature map X1 to obtain feature map X4.

[0100] 2.5. Feature map X4 is normalized by LN to obtain feature map X5.

[0101] 2.6. Feature map X5 passes through a multilayer perceptron (MLP) with two fully connected layers and activated by a GELU nonlinear activator in the middle to obtain feature map X6.

[0102] 2.7. Add the feature map X6 to the feature map X4 obtained in step 2.4 to obtain the feature map X7.

[0103] 2.8. Steps (2.2) to (2.7) are a process of STL feature extraction, where the above X1 to X7 are only used to symbolize the feature map data of the corresponding stage and cannot be regarded as a restriction on the feature map data. In different STLs in the Swin Transformer backbone, X2 to X7 corresponding to each stage are different, where the target feature map includes the feature map X1 and X7 output by the previous layer of STL. There are two forms of STL window division. One is to divide the local window directly from the feature map point (0, 0), which is called conventional window division; the other is to divide the window from the point (3, 3), which is called transfer window division. The feature map repeats steps (2.2) to (2.7), and conventional window division and transfer window division are performed alternately in continuous STL. At the 4th, 8th, 20th, and 24th STLs, the feature maps are evenly divided into 4 parts and spliced ​​to obtain feature maps with half the size and double the dimension. Finally, three feature maps of different sizes of 80×80, 40×40, and 20×20 are output at the 8th, 20th, and 24th STLs (i.e., the second feature map of preset specifications).

[0104] (3) Introduce the deformable region association module DRCM to replace the classification branch of YOLOX (the structure of DRCM is as follows Figure 5 ), DRCM can be divided into a deformable convolution branch (top) and a conventional convolution branch (bottom). The steps of DRCM data processing include:

[0105] 3.1. After the feature map X8 (i.e., the third feature map) is extracted by PAFPN, a 3×3 convolution is performed on the upper branch to obtain the feature map X9.

[0106] 3.2 and X9 pass through two convolutional layers respectively to obtain a sampling point offset map Δp with a dimension of H×W×2 and a fractional mask Δm with a dimension of H×W×1.

[0107] 3.3. Embed Δp and Δm into a 5×5 deformable convolution, and X9 is convolved to obtain the output feature map Y. The expression is as follows:

[0108]

[0109] Among them, p represents the position on the feature map, k represents the kth sampling point, ω k Represents the weight of the kth offset sampling point, p+p k represents the kth sampling position of the 5×5 regular convolution, Δp k Represents the offset of the kth sampling position of the 5×5 regular convolution, Δm k Represents the fractional value of the k-th offset sampling point.

[0110] 3.4. In the next branch, two consecutive 3×3 convolutions are performed on feature map X8 to obtain feature map Z. Two learnable hyperparameters α and β are set, where α + β = 1. Y and Z are weighted summed using α and β, and feature channel information is integrated through a 1×1 convolution. The confidence scores for each position on the feature map for the seven aircraft categories are output.

[0111] In some specific embodiments, the specific steps of optimizing the network model parameters by the loss value back propagation method based on the loss function are as follows:

[0112] 4.1. In the classification branch, for the score vector of each position on the feature map, the cross entropy classification loss function with Sigmoid activation is calculated.

[0113]

[0114] Where N is the number of aircraft categories; y i Indicates whether it is the i-th type target, if so, it is 1, otherwise it is 0; p i Represents the confidence score of the network's prediction of the i-th category target. Sigmoid calculates the activation value for the input x.

[0115]

[0116] 4.2. Average all classification losses to get the total classification loss L cls .

[0117] L cls =mean(l cls )

[0118] 4.3. Calculate the chamfer distance loss for the offset sampling points obtained from the predicted target position on the feature map.

[0119]

[0120]

[0121] in, Represents the position of the nth predicted offset sampling point in the i-th target frame; Represents the center point position of the mth marked strong scattering area in the i-th target frame; r i Represents the radius of the strong scattering area marked in the bounding box of the i-th target.

[0122] 4.4. Average all SSR prediction losses to obtain the total SSR prediction loss.

[0123] L ssr =mean(l ssr )

[0124] 4.5. In the regression branch, the bounding box (bbox) regressed from the feature map responsible for predicting the target position is used to calculate the intersection over union (IoU) regression loss function.

[0125]

[0126] Among them, bbox pre Bbox is the bounding box predicted by the network model. gt is the true bounding box label of the target, Intersection is the function for calculating the intersection, and Union is the function for calculating the union.

[0127] 4.6. Take the average of all regression losses to get the total regression loss L reg .

[0128] L reg =mean(l reg )

[0129] 4.7. In the regression branch, the 1-norm loss is calculated for the score of whether the target exists at each position on the feature map.

[0130]

[0131] Where y is a true value, which is 1 if there is a target, and 0 otherwise. is the predicted value of the network model.

[0132] 4.8. Average the target presence loss of all positions to get the total target presence loss.

[0133] L obj =mean(l obj )

[0134] 4.9. Calculate the total loss value of training.

[0135] L=αL cls +βL reg +γL obj

[0136] Among them, the weighting coefficients α=3, β=3, γ=1, and λ=1.

[0137] Finally, the network model parameters are optimized through the loss value back propagation method.

[0138] In some specific embodiments, based on preset training parameters, the specific steps of using stochastic gradient descent (SGD) training to perform classification and regression training on the deformable scattering feature association network model are as follows:

[0139] 5.1. Stochastic gradient descent (SGD) training is used with 100 epochs, a batch size of 8, and the optimizer weight decay parameter and momentum are set to 0.0005 and 0.9, respectively.

[0140] 5.2. The learning rate is initialized to 0.002 and gradually decreased to 0.0001 with the number of cycles using the cosine annealing strategy.

[0141] 5.3. Given a batch of input SAR images, they first pass through the Swin Transformer backbone, then through the PAFPN neck to fully generate semantic features, and then through the DRCM to output classification results and the regression branch to output regression results. The model parameters are adjusted based on the output results.

[0142] In some specific embodiments, the method further includes testing the deformable scattering feature correlation network (DSFCN) model. Figure 6 As shown, the specific steps include:

[0143] 6.1. For the trained DSFCN model, the precision, recall, and mean average precision (mAP) are calculated using the test dataset. The score of the predicted bounding box is the product of the confidence score of whether the target is contained in the regression branch and the highest category confidence score in the classification branch. When the value is greater than 0.5, the target is considered to be present, and the category is the category with the highest confidence in the classification branch. A successful match is considered when the intersection-over-union ratio of the predicted bounding box to the true bounding box is greater than 0.5 and the category is consistent.

[0144] 6.2. Calculate the model parameters.

[0145] 6.3. Use the feature visualization technology ScoreCAM to visualize the activation degree of each image area for the detection results.

[0146] 6.4 Visualizing the offset sampling prediction results of deformable convolution in DRCM.

[0147] Among them, Figure 7 As shown in the figure, the overall construction steps of the DSFCN model are:

[0148] (1) Obtaining an input SAR image. It should be noted that the SAR image here refers to a SAR image dataset that has been annotated with aircraft categories and target bounding boxes.

[0149] (2) Constructing the DSFCN model; including: constructing the YOLOX model; introducing the Swin Transformer; introducing the DRCM; and constructing the model loss function;

[0150] (3) Train the DSFCN model;

[0151] (4) Test the DSFCN model.

[0152] Specifically, in some embodiments, the effects of the present invention are verified through ablation experiments. The ablation experiment results of steps 6.1 and 6.2 are shown in Table 1.

[0153] Table 1

[0154]

[0155] The performance comparison results with other existing algorithms are shown in Table 2.

[0156] Table 2

[0157]

[0158] As shown in Tables 1 and 2, the two improved modules proposed in this paper enable the network model to achieve higher scores across various metrics while reducing the number of parameters, significantly improving detection performance and outperforming existing algorithms. This superior network model performance demonstrates that the Swin Transformer module with its self-attention mechanism is capable of finely extracting scattering features from SAR images, while the DRCM, with its flexible sampling capabilities, is able to associate salient features of aircraft, thereby adapting to their variability in SAR images.

[0159] In summary, the purpose of the present invention is to utilize the discrete and variable scattering characteristics of aircraft in SAR images, improve the modules in the neural network model to fully extract aircraft features, and improve the network model's detection and recognition performance for aircraft. Convolution is based on the idea of ​​template matching to model the relationship between adjacent sampling points, and therefore has a local inductive bias, that is, adjacent sampling points have strong correlation. The self-attention mechanism used by the Swin Transformer can be regarded as an adaptive filter, whose weight is determined by the correlation between the query vector and the keyword vector between points, and is more suitable for the long-range dependency information of the sampling points. Considering that aircraft appear as a collection of discrete and sparsely distributed scattered points in SAR images, and the correlation between the pixels on the aircraft is relatively weak, the present invention abandons the backbone of the traditional convolution architecture and adopts the Swin Transformer backbone, so that the network model can more fully extract the scattering characteristics of the aircraft. In order to overcome the limitation that the original convolution can only sample on a regular grid, the present invention embeds deformable convolution into the classification branch. Furthermore, considering the variability of SAR aircraft, strong scattering regions on an aircraft are crucial for aircraft identification. This invention incorporates strong scattering region supervision information into the network model training process in the form of a loss function to guide the prediction of offset sampling points. Compared to previous methods that manually extract features and then preprocess and enhance the image, this invention fully leverages the powerful modeling capabilities of the network and proposes a DRCM to adaptively associate strong scattering regions with aircraft-significant features. This organically combines manually extracted and network-derived features, resulting in greater adaptability. Based on the advanced YOLOX detector, this embodiment employs the Swin Transformer with a self-attention mechanism as the backbone of the network to extract features from raw high-resolution SAR images. Furthermore, a Deformable Regional Correlation Module (DRCM) with the ability to automatically associate aircraft-significant regions is incorporated into the detection head. This results in a new SAR aircraft detector, named the Deformable Scattering Feature Correlation Network (DSFCN). Compared to existing solutions, the algorithm described in this invention exhibits superior feature extraction and correlation capabilities, improving aircraft detection performance in complex SAR images and achieving more accurate aircraft detection and identification than other methods.

[0160] On the other hand, an embodiment of the present invention provides a SAR aircraft detection system, comprising: a first module for acquiring an input SAR image; a second module for analyzing the input SAR image using an aircraft detection model to obtain an aircraft detection result; wherein the aircraft detection result includes at least one target bounding box regression result and a corresponding aircraft category confidence score; the aircraft detection model is generated by training a SAR image dataset that has been labeled with aircraft categories and target bounding boxes; the aircraft detection model includes a classification branch and a regression branch, the classification branch has a deformable region association module, and the deformable region association module performs feature weighted integration through a deformable convolution branch and a conventional convolution branch.

[0161] The contents of the method embodiments of the present invention are all applicable to the system embodiments. The functions specifically implemented by the system embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above methods.

[0162] Another aspect of an embodiment of the present invention further provides a SAR aircraft detection device, including a processor and a memory;

[0163] The memory is used to store programs;

[0164] The processor executes the program to implement the above method.

[0165] The contents of the method embodiments of the present invention are all applicable to the device embodiments. The functions specifically implemented by the device embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0166] Another aspect of the embodiments of the present invention further provides a computer-readable storage medium, wherein the storage medium stores a program, and the program is executed by a processor to implement the method described above.

[0167] The contents of the method embodiments of the present invention are all applicable to the computer-readable storage medium embodiments. The functions specifically implemented by the computer-readable storage medium embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0168] The present invention also discloses a computer program product or computer program, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device can read the computer instructions from the computer-readable storage medium and execute the computer instructions, causing the computer device to perform the above method.

[0169] In some optional embodiments, the function / operation mentioned in the block diagram may not occur in the order mentioned in the operation diagram. For example, depending on the function / operation involved, the two boxes shown in succession can actually be executed substantially simultaneously or the boxes can sometimes be executed in reverse order. In addition, the embodiment presented and described in the flow chart of the present invention is provided in an exemplary manner for the purpose of providing a more comprehensive understanding of the technology. The disclosed method is not limited to the operation and logic flow presented herein. Optional embodiments are contemplated in which the order of the various operations is changed and the sub-operations described as a part of a larger operation are performed independently.

[0170] Furthermore, although the present invention is described in the context of functional modules, it should be understood that, unless otherwise indicated, one or more of the functions and / or features described may be integrated into a single physical device and / or software module, or one or more functions and / or features may be implemented in separate physical devices or software modules. It will also be understood that a detailed discussion of the actual implementation of each module is not necessary for understanding the present invention. More specifically, given the properties, functions, and internal relationships of the various functional modules in the devices disclosed herein, the actual implementation of the module will be understood within the ordinary skill of an engineer. Therefore, a person skilled in the art using ordinary skill will be able to implement the present invention set forth in the claims without undue experimentation. It will also be understood that the specific concepts disclosed are merely illustrative and are not intended to limit the scope of the present invention, which is determined by the full scope of the appended claims and their equivalents.

[0171] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0172] The logic and / or steps represented in the flowcharts or otherwise described herein, for example, can be considered as an ordered list of executable instructions for implementing the logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution apparatus, device, or apparatus (e.g., a computer-based apparatus, a device including a processor, or other apparatus that can fetch instructions from and execute instructions on an instruction execution apparatus, device, or apparatus). For purposes of this specification, a "computer-readable medium" can be any device that can contain, store, communicate, propagate, or transport a program for use by, or in conjunction with, an instruction execution apparatus, device, or apparatus.

[0173] More specific examples (a non-exhaustive list) of computer-readable media include the following: an electrical connection with one or more wires (electronic devices), a portable computer disk cartridge (magnetic devices), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), a fiber optic device, and a portable compact disc read-only memory (CDROM). In addition, the computer-readable medium may even be paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, deciphering, or processing in another suitable manner as necessary, and then stored in a computer memory.

[0174] It should be understood that various parts of the present invention can be implemented using hardware, software, firmware, or a combination thereof. In the above-described embodiments, multiple steps or methods can be implemented using software or firmware stored in a memory and executed by a suitable instruction execution device. For example, if implemented using hardware, as in another embodiment, any one of the following technologies known in the art or a combination thereof can be used: a discrete logic circuit having a logic gate circuit for implementing a logic function on a data signal, an application-specific integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0175] Throughout this specification, reference to terms such as "one embodiment," "some embodiments," "examples," "specific examples," or "some examples" means that a specific feature, structure, material, or characteristic described in conjunction with that embodiment or example is included in at least one embodiment or example of the present invention. In this specification, schematic representations of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples.

[0176] While embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions, and variations may be made to the embodiments without departing from the principles and spirit of the invention, and that the scope of the invention is defined by the claims and their equivalents.

[0177] The above is a specific description of the preferred implementation of the present invention, but the present invention is not limited to the embodiments. Those skilled in the art can make various equivalent modifications or substitutions without violating the spirit of the present invention. These equivalent modifications or substitutions are all included in the scope defined by the claims of the present invention.

Claims

1. A SAR aircraft detection method, characterized in that: include: Get the input SAR image; Analyzing the input SAR image using an aircraft detection model to obtain an aircraft detection result; The aircraft detection result includes at least one target bounding box regression result and a corresponding aircraft category confidence score; The aircraft detection model is generated by training a SAR image dataset that has been labeled with aircraft categories and target bounding boxes. The aircraft detection model includes a classification branch and a regression branch. The classification branch has a deformable region association module, and the deformable region association module performs feature weighted integration through a deformable convolution branch and a conventional convolution branch. The method further comprises: Creating a deformable scattering feature association network model based on the classification branch and the regression branch with the deformable region association module; The deformable scattering feature association network model includes a Swin Transformer backbone, a path aggregation feature pyramid network, and a decoupling head; the decoupling head includes the classification branch and the regression branch with a deformable region association module; the aircraft detection model is obtained through the deformable scattering feature association network model; The aircraft detection model is used to analyze the input SAR image to obtain an aircraft detection result, including: Segmenting, splicing, and linear mapping the SAR image to obtain a first feature map; By using the Swin Transformer backbone, conventional window partitioning and transfer window partitioning are alternately performed using consecutive Swin Transformer layers to obtain a second feature map of a preset size; Performing semantic feature extraction on the second feature map through the path aggregation feature pyramid network to obtain a third feature map of the preset specifications; wherein the semantic feature extraction includes upsampling, splicing and convolution; The third feature map is predicted by the regression branch to obtain a target frame regression result; wherein the target frame regression result includes the center point, width and height of the frame; Performing feature weighted integration on the third feature map using the deformable convolution branch and the conventional convolution branch through the classification branch having the deformable region association module to obtain a category confidence score of the aircraft of the preset category corresponding to the target bounding box regression result; The classification branch having the deformable region association module uses the deformable convolution branch and the conventional convolution branch to perform feature weighted integration on the third feature map to obtain a category confidence score of the preset category of the aircraft corresponding to the target bounding box regression result, including: Performing a first convolution on the third feature map through the deformable convolution branch to obtain a feature map; Performing a second convolution on the feature map to obtain a sampling point offset map; performing a third convolution on the feature map to obtain a fractional mask; Performing deformable convolution according to the feature map, the sampling point offset map, and the score mask to obtain a feature map; Performing two consecutive convolutions on the third feature map through the conventional convolution branch to obtain a feature map; Based on preset learnable hyperparameters, the feature map and the feature map are weighted added and feature channel information is integrated to obtain a category confidence score of the preset category of the aircraft corresponding to the target bounding box regression result.

2. A SAR aircraft detection method according to claim 1, characterized in that: The method of creating a deformable scattering feature association network model based on the classification branch and the regression branch having the deformable region association module includes: Construct a YOLOX model; wherein the YOLOX model includes a Darknet-53 backbone, a path aggregation feature pyramid network, and an original decoupling head; the original decoupling head includes an original classification branch and the regression branch; A deformable scattering feature association network model is created by replacing the Darknet-53 backbone of the YOLOX model with the Swin Transformer backbone and replacing the original classification branch of the YOLOX model with the classification branch having the deformable region association module. The Swin Transformer backbone includes 24 Swin Transformer layers.

3. A SAR aircraft detection method according to claim 1, characterized in that: Also includes: Determining training samples based on the SAR image dataset that has been labeled with aircraft categories and target bounding boxes; wherein a preset number of strong scattering areas are marked in the target bounding box; According to the training samples and based on preset training parameters, stochastic gradient descent (SGD) training is used to perform classification and regression training on the deformable scattering feature association network model, and the network model parameters are optimized by the loss value back propagation method in combination with the model loss function to obtain the aircraft detection model.

4. A SAR aircraft detection method according to claim 1, characterized in that: The Swin Transformer layer includes a normalization function, a window-based multi-head self-attention mechanism, and a multi-layer perceptron. The step of processing the feature map by the Swin Transformer layer includes: The target feature map is normalized by the normalization function to obtain the feature map ; According to the feature map , using the window-based multi-head self-attention mechanism, the feature map is obtained through linear mapping and channel-based dimension splicing ; According to the target feature map and the feature map Add together to get the feature map ; The feature map is normalized by the normalization function Normalize and get the feature map ; Through the multi-layer perceptron, the feature map Use GELU nonlinear activator to activate and get the feature map ; Wherein, the multi-layer perceptron includes two fully connected layers; According to the feature map and feature maps Add together to get the feature map .

5. A SAR aircraft detection system, characterized in that: Applied to the SAR aircraft detection method according to claim 1, the system comprises: The first module is used to obtain the input SAR image; The second module is used to analyze the input SAR image using an aircraft detection model to obtain an aircraft detection result; The aircraft detection result includes at least one target bounding box regression result and a corresponding aircraft category confidence score; The aircraft detection model is generated by training a SAR image dataset with labeled aircraft categories and target bounding boxes. The aircraft detection model includes a classification branch and a regression branch. The classification branch has a deformable region association module, and the deformable region association module performs feature weighted integration through a deformable convolution branch and a conventional convolution branch.

6. A SAR aircraft detection device comprising a processor and a memory; The memory is used to store programs; The processor executes the program to implement the method according to any one of claims 1 to 4.

7. A computer-readable storage medium, characterized in that The storage medium stores a program, and the program is executed by a processor to implement the method according to any one of claims 1 to 4.

Images