An optimization method for target detection in SAR images based on edge enhancement network

Abstract

This application discloses an optimization method for SAR image target detection based on an edge enhancement network. The implementation steps of the method are as follows: constructing an edge enhancement network module; introducing the edge enhancement network module into a general convolutional neural network-based object detection model; dividing the object detection data set; performing predictive processing on the images in the data set; using the training data to collaboratively train the object detection model and the edge enhancement network module; feeding the verification data into the detection model for detection; and obtaining the detection results. The invention utilizes the constructed edge enhancement network module and a general deep convolutional neural network-based object detection model for collaborative training to enhance the detection model's feature extraction network for SAR image object detection, thereby improving object detection accuracy while maintaining comparable computational cost.

Description

Technical Field

[0001] The present invention relates to the field of remote sensing image building detection technology, and more particularly to a SAR image target detection optimization method and system based on an edge enhancement network. The present invention can be used to enhance the detection accuracy in SAR images. Background Art

[0002] Object detection is a key area of ​​intelligent interpretation of remote sensing images, providing fundamental data for many remote sensing applications. For example, buildings, ships, and other man-made objects in remote sensing images are crucial sources of information for intelligent analysis and decision-making. Currently, optical remote sensing images occupy a significant proportion of the market due to their high clarity and intuitive image quality. However, weather and lighting factors significantly impact optical remote sensing images, while images acquired using synthetic aperture radar (SAR) are unaffected by these factors. Synthetic aperture radar (SAR) imaging has become a key technology in remote sensing, offering numerous advantages over traditional optical sensors. SAR is capable of acquiring geographic images under all conditions, regardless of weather conditions, and can penetrate clouds, fog, rain, and snow. SAR has been widely used in various fields, including national defense, geographic surveying, and disaster risk monitoring.

[0003] In recent years, SAR images have garnered increasing attention in the field of target detection due to their advantages, resulting in the emergence of a large number of outstanding SAR image target detection algorithms. Currently, SAR image target detection primarily encompasses traditional detection algorithms and deep learning-based detection algorithms. Traditional SAR image target detection algorithms primarily include constant false-alarm rate (CFAR) detection methods and those based on manually extracted image texture features. Deep learning-based target detection methods, typically based on convolutional neural networks and transformers, have made significant progress in SAR image target detection due to their powerful feature learning capabilities. Deep learning methods can proactively extract deep features, eliminating manual selection and significantly improving detection efficiency. Furthermore, given a reliable dataset, deep learning exhibits robustness and generalization capabilities, making it adaptable to a wide range of complex scenarios. With the advancement of artificial intelligence (AI), many excellent deep learning target detection methods have been applied to SAR image detection. However, SAR images often exhibit characteristics such as coherent speckle noise and pronounced edge features, requiring further research in terms of detection accuracy and generalization.

[0004] Shaoqing Ren, Kaiming He, and others published the paper "Faster R-CNN: Towards Real-time Object Detection with Region Proposal Networks" (Advances in Neural Information Processing Systems), further improving on Fast R-CNN by introducing a Region Proposal Network (RPN). The RPN generates proposal boxes based on the extracted features. The generated proposal boxes and feature maps are then processed by ROIpooling for cropping and then fed into the Fast R-CNN prediction head for prediction. This approach makes Fast R-CNN an end-to-end object detection network, significantly improving detection speed while maintaining accuracy.

[0005] An et al. published the paper "DRBox-v2: An Improved Detector with Rotatable Boxes for Target Detection in SAR Images," proposing an improved RBox-based target detection framework that employs a multi-layer prior box generation strategy for target detection. They also proposed a combination of focal loss (FL) and hard negative mining to alleviate the imbalance between positive and negative samples.

[0006] Tang et al. published a paper titled "Large-difference-scale target detection using arevised Bhattacharyya distance in SAR images." They improved the Bhattacharyya distance to improve small target detection, thereby better measuring the deviation of bounding boxes caused by large differences in target scales. This can be embedded in anchor-based detection models.

[0007] Zhou et al. published the paper "HRLE-SARDet: A lightweight SAR target detection algorithm based on hybrid representation learning enhancement." They proposed HRLE-SARDet, a learning-enhanced SAR image detection algorithm based on lightweight SAR image features. The network includes a lightweight, scattering feature extraction backbone and a multi-scale feature fusion neck. A hybrid representation learning enhancement module is also designed to extract small target information from SAR images.

[0008] However, due to the existence of coherent speckle noise and obvious edge features in SAR images, the general detection model based on convolutional neural networks does not extract perfect features in the feature extraction stage, resulting in unsatisfactory detection accuracy. Summary of the Invention

[0009] The purpose of the present invention is to address the shortcomings of the existing technology and propose an optimization method for SAR image target detection based on an edge enhancement network, which solves the problems of incomplete feature extraction and low detection accuracy caused by a series of reasons such as the presence of coherent speckle noise and incomplete edge feature extraction.

[0010] The implementation strategy of this invention is as follows: First, an edge enhancement network module is constructed and introduced into a general convolutional neural network-based object detection model. Then, an object detection dataset is divided, and prediction processing is performed on the images in the dataset. The object detection model and decoder are collaboratively trained using the training data. Finally, the verification data is fed into the detection model for detection, resulting in detection results.

[0011] The specific steps of the present invention are as follows:

[0012] Step 1: Build the edge enhancement network module:

[0013] Construct an edge enhancement network module consisting of an edge enhancement module and a context fusion module. The structure of the edge enhancement module is as follows: grouped convolution block → edge enhancement convolution block → relu function → 1×1 convolution; the structure of the grouped convolution block is to set the output channel to 1 / 4 of the input channel through a normal convolution layer and set a unique grouping number for each convolution block; the edge enhancement convolution block is to integrate the edge enhancement operator into the normal convolution;

[0014] The structure of the context fusion module is as follows: dilated DW convolution block → context attention module; the structure of the dilated DW convolution block is as follows: relu function → 3×3 DW convolution → parallel DW convolution → feature splicing; parallel DW convolution is composed of a series of large-kernel DW convolutions; feature splicing splices the features obtained by a series of parallel DW convolutions; the structure of the context attention module is as follows: average pooling layer → 1×1 convolution → 3×3 DW convolution → 1×1 convolution → sigmoid function;

[0015] Step 2: Introduce the edge enhancement network module into the general convolutional neural network-based object detection model:

[0016] The constructed edge enhancement network module is introduced into the backbone network of the object detection model. After placing it in four different stages in the backbone network, the obtained features are fused with the original features.

[0017] Step 3: Divide the object detection dataset:

[0018] Select some annotated optical remote sensing images with no restrictions on image length and width (each batch will be preprocessed, the images in the batch will be scaled, and then padded to a uniform size). If the dataset is more difficult to train, more images will be required. For simple datasets, 1000+ images can be used. Then 80% of the images will form the training set, and 20% of the images will form the validation set and test set;

[0019] Step 4: Perform prediction processing on the images in the dataset:

[0020] First, the data in each mini-batch is horizontally flipped with a probability of 50%. Then, the length and width of the images in each mini-batch are fixed, and the length is scaled to an integer multiple of 32 between 400 and 600 pixels. The width is then padded to an integer multiple of 32 between 400 and 600 pixels.

[0021] Step 5: Use the training data to co-train the object detection model and decoder:

[0022] The first step is to input the preprocessed data into the backbone network (resnet50 or cspnext, etc.) for feature extraction to generate feature maps C2, C3, C4, and C5;

[0023] In the second step, the feature map is fed into the object detection head to predict the category and bounding box, and then the cross entropy loss and bounding box loss are calculated with the true label;

[0024] In the third step, the feature maps C2, C3, C4, and C5 generated in the first step are fed into the edge enhancement network module for edge enhancement module, context fusion module, and feature fusion. Then, a feature map of the same size as the image preprocessed is output, and the mean square error loss function between the preprocessed image and the feature map is calculated.

[0025] The fourth step is to pass the loss functions of the second and third steps back to update the entire network structure at the same time;

[0026] Step 6: Send the verification data to the detection model for detection:

[0027] In the first step, the preprocessed data is input into the backbone network for feature extraction to generate feature maps C2, C3, C4, and C5;

[0028] In the second step, the feature maps C2, C3, C4, and C5 are fed into the object detection head for category and bounding box prediction, and then processed by non-maximum suppression (NMS) to generate the detection results.

[0029] Step 7: Get the test results:

[0030] Map the detection results obtained in step 6 to the corresponding positions of the original image and mark the predicted category information.

[0031] Compared with the existing technology, the present invention has the following advantages:

[0032] First, the structure has strong embedding properties. It can be embedded in some mainstream object detection models, such as RtmDet, YOLO, and Faster R-CNN.

[0033] Second, although it takes longer to train than the detection model without the edge enhancement network module, the prediction time is comparable and is not inferior to the prediction results of a deeper backbone network. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] Figure 1 This is a flow chart of the SAR image target detection optimization method based on edge enhancement network.

[0035] Figure 2 It is a test result diagram of the present invention. DETAILED DESCRIPTION

[0036] The present invention will be described in detail below with reference to the accompanying drawings.

[0037] Reference Figure 1 , the steps of the present invention are further elaborated.

[0038] Step 1: Build the edge enhancement network module:

[0039] Construct an edge enhancement network module consisting of an edge enhancement module and a context fusion module. The structure of the edge enhancement module is as follows: grouped convolution block → edge enhancement convolution block → relu function → 1×1 convolution; the structure of the grouped convolution block is to set the output channel to 1 / 4 of the input channel through a normal convolution layer and set a unique grouping number for each convolution block; the edge enhancement convolution block is to integrate the edge enhancement operator into the normal convolution;

[0040] The structure of the context fusion module is as follows: dilated DW convolution block → context attention module; the structure of the dilated DW convolution block is as follows: relu function → 3×3 DW convolution → parallel DW convolution → feature splicing; parallel DW convolution is composed of a series of large-kernel DW convolutions; feature splicing splices the features obtained by a series of parallel DW convolutions; the structure of the context attention module is as follows: average pooling layer → 1×1 convolution → 3×3 DW convolution → 1×1 convolution → sigmoid function;

[0041] Step 2: Introduce the edge enhancement network module into the general convolutional neural network-based object detection model:

[0042] The constructed edge enhancement network module is introduced into the backbone network of the object detection model. After placing it in four different stages in the backbone network, the obtained features are fused with the original features.

[0043] Step 3: Divide the object detection dataset:

[0044] Select some annotated optical remote sensing images with no restrictions on image length and width (each batch will be preprocessed, the images in the batch will be scaled, and then padded to a uniform size). If the dataset is more difficult to train, more images will be required. For simple datasets, 1000+ images can be used. Then 80% of the images will form the training set, and 20% of the images will form the validation set and test set;

[0045] Step 4: Perform prediction processing on the images in the dataset:

[0046] First, the data in each mini-batch is horizontally flipped with a probability of 50%. Then, the length and width of the images in each mini-batch are fixed, and the length is scaled to an integer multiple of 32 between 400 and 600 pixels. The width is then padded to an integer multiple of 32 between 400 and 600 pixels.

[0047] Step 5: Use the training data to co-train the object detection model and decoder:

[0048] The first step is to input the preprocessed data into the backbone network (resnet50 or cspnext, etc.) for feature extraction to generate feature maps C2, C3, C4, and C5;

[0049] In the second step, the feature map is fed into the object detection head to predict the category and bounding box, and then the cross entropy loss and bounding box loss are calculated with the true label;

[0050] In the third step, the feature maps C2, C3, C4, and C5 generated in the first step are fed into the edge enhancement network module for edge enhancement module, context fusion module, and feature fusion. Then, a feature map of the same size as the image preprocessed is output, and the mean square error loss function between the preprocessed image and the feature map is calculated.

[0051] The fourth step is to pass the loss functions of the second and third steps back to update the entire network structure at the same time;

[0052] Step 6: Send the verification data to the detection model for detection:

[0053] In the first step, the preprocessed data is input into the backbone network for feature extraction to generate feature maps C2, C3, C4, and C5;

[0054] In the second step, the feature maps C2, C3, C4, and C5 are fed into the object detection head for category and bounding box prediction, and then processed by non-maximum suppression (NMS) to generate the detection results.

[0055] Step 7: Get the test results:

[0056] Map the detection results obtained in step 6 to the corresponding positions of the original image and mark the predicted category information.

[0057] The effects of the present invention are further described below in conjunction with simulation.

[0058] 1. Simulation experiment conditions:

[0059] The simulation experiment conditions of the present invention are as follows: server GPU: GeForce RTX3090, video memory 24G;

[0060] The software platform for the simulation experiment of the present invention is: Ubuntu 18.04 system, Python 3.10, and PyTorch-GPU 2.0.0.

[0061] 2. Analysis of simulation content and experimental results:

[0062] This paper simulates the existing technology (RtmDet method) and its introduction of an edge enhancement network module to detect SAR images, and obtains detection results. The simulation uses the following datasets: SSDD, RSDD-SAR, and SAR-AIRcraft.

[0063] Several existing technologies used in the simulation experiments are:

[0064] The Faster R-CNN detection method is an object detection method proposed by Shaoqing Ren, Kaiming He, and others in "Ren, S., He, K., Girshick, R., Sun, J. (2015). Faster r-cnn: Towards real-time object detection with region proposal networks. Advances in neural informationprocessing systems, 28, 91-99.", referred to as the Faster R-CNN detection method;

[0065] The Cascade R-CNN detection method is an object detection method proposed by Cai Z et al. in "Cai Z, Vasconcelos N. Cascade r-cnn: Delving into high quality object detection[C] / / Proceedings of the IEEE conference on computer vision and pattern recognition. 2018: 6154-6162."

[0066] Dynamic R-CNN is an object detection algorithm proposed by Zhang et al. in “Zhang H, Chang H, Ma B, et al. Dynamic R-CNN: Towards high quality object detection via dynamic training[C] / / Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XV 16. Springer International Publishing, 2020: 260-275.”

[0067] The Libra R-CNN detection method is an object detection method proposed by Pang et al. in "Pang J, Chen K, Shi J, et al. Librar-cnn: Towards balanced learning for object detection[C] / / Proceedings of theIEEE / CVF conference on computer vision and pattern recognition. 2019: 821-830."

[0068] The Grid R-CNN detection method is an object detection method proposed by Lu et al. in "Lu X, Li B, Yue Y, et al. Grid r-cnn[C] / / Proceedings of the IEEE / CVF conference on computer vision and pattern recognition. 2019: 7363-7372."

[0069] The YOLOX detection method is an object detection method proposed by Ge et al. in "Ge Z, Liu S, Wang F, et al. Yolox: Exceedingyolo series in 2021[J]. arXiv preprint arXiv:2107.08430, 2021."

[0070] The RtmDet detection method is an object detection method proposed by Lyu C et al. in "Lyu C, Zhang W, Huang H, et al. Rtmdet:An empirical study of designing real-time object detectors[J]. arXiv preprintarXiv:2212.07784, 2022."

[0071] The input images used in the simulation experiments of the present invention are the publicly available SSDD dataset and SAR-AIRcraft dataset. The SSDD dataset was released by Wuhan University in 2017. Its images were extracted from RadarSat-2, TerraSAR-X, and Sentinel-1. It contains 1,160 SAR ship images with an image size of 500×500×3. We use 1,000 ship images as the training set and 160 images as the validation set. The SAR-AIRcraft dataset is constructed based on Gaofen-3 satellite images with a resolution of 1 meter. This dataset contains four aircraft of different sizes, 16,434 target instances, and consists of 4,368 images.

[0072] Simulation experiment 1 introduces the edge enhancement network module into RtmDet and compares the detection results on SSDD.

[0073] Simulation experiment 2 introduces the edge enhancement network module into RtmDet and compares the detection results on SAR-AIRcraft.

[0074] In order to verify the effectiveness of the present invention for SAR image detection, the detection results were evaluated using the Coco evaluation index, and the calculation results were plotted in the following table.

[0075] Table 1. Comparison of simulation results of the present invention on the SSDD dataset

[0076] Method #Params(M) GFLOPs mAP(%) AP50(%) AP75(%) Faster R-CNN 41.4 137.2 44.7 80.6 44.5 Cascade R-CNN 69.2 165.9 46.0 81.8 48.0 Libra R-CNN 41.7 141.3 48.1 83.0 52.3 Dynamic R-CNN 41.4 137.2 40.0 82.1 31.1 Grid R-CNN 64.5 253.9 47.4 83.6 48.3 YOLOX 8.9 12.9 47.7 87.3 48.8 RtmDet 8.9 13.9 52.3 88.0 56.7 EEN 11.4 18.3 52.6 89.2 56.3

[0077] Table 2. Comparison of simulation results of the present invention on the SAR-AIRcraft dataset

[0078] Method mAP(%) AP50(%) AP75(%) Faster R-CNN 53.1 91.6 56.8 Cascade R-CNN 55.5 93.7 61.8 Libra R-CNN 54.3 91.3 59.8 Dynamic R-CNN 52.6 90.7 56.2 Grid R-CNN 52.5 92.0 55.0 YOLOX 58.4 95.2 65.9 RtmDet 59.1 95.2 67.9 EEN 65.7 97.0 74.3

[0079] Combined with Table 1, it's clear that the detection algorithm trained with the edge enhancement network module significantly improves detection accuracy compared to RtmDet without it. In our experiments on the SSDD dataset, the EEN model, after embedding the edge enhancement network module, achieved the best AP50, reaching 89.2% and mAP of 52.6%. Table 2 shows the experimental results of mainstream detection models on the SAR-AIRcraft dataset. We can observe that the performance is significantly improved after embedding the edge enhancement network. On the EEN, mAP, AP50, and AP75 reach 65.7%, 97.0%, and 74.3%, respectively.

[0080] The following combination Figure 2 The effects of this simulation experiment are further explained.

[0081] Figure 2 This is the visualization result on the SSDD dataset, which includes the true label and the results predicted by RtmDet and EEN. Figure 2 As can be seen from the figure, EEN has significantly improved both false positives and missed positives on the SSDD dataset compared to RtmDet. This demonstrates that the present invention is superior to the original detection model without the present invention in terms of accuracy, missed positives, and false positives.

Claims

1. A SAR image target detection optimization method based on edge enhancement network, characterized in that: The method comprises: The edge enhancement network module is used to perform collaborative training with the object detection model to enhance the feature extraction network of the detection model and improve the detection accuracy. The specific steps of this method include the following: Step 1: Build the edge enhancement network module: Construct an edge enhancement network module consisting of an edge enhancement module and a context fusion module. The structure of the edge enhancement module is as follows: grouped convolution block → edge enhancement convolution block → relu function → 1×1 convolution. The structure of the grouped convolution block is to set the output channel to 1 / 4 of the input channel through the ordinary convolution layer and set the number of groups for each convolution block. The edge enhancement convolution block integrates the edge enhancement operator into the ordinary convolution. The structure of the context fusion module is as follows: dilated DW convolution block → context attention module; the structure of the dilated DW convolution block is as follows: relu function → 3×3 DW convolution → parallel DW convolution → feature splicing; parallel DW convolution is composed of a series of large-kernel DW convolutions; feature splicing splices the features obtained by a series of parallel DW convolutions; the structure of the context attention module is as follows: average pooling layer → 1×1 convolution → 3×3 DW convolution → 1×1 convolution → sigmoid function; Step 2: Introduce the edge enhancement network module into the general convolutional neural network-based object detection model: The constructed edge enhancement network module is introduced into the backbone network of the object detection model. After placing it in four different stages in the backbone network, the obtained features are fused with the original features. Step 3: Divide the object detection dataset: Select some annotated optical remote sensing images with unlimited length and width. Each batch will be preprocessed by scaling the images in the batch and then padding them into images of uniform size. If the dataset is difficult to train, more images will be required. Simple datasets use 1000+ images. 80% of the images will be used as a training set, and 20% of the images will be used as a validation set and a test set. Step 4: Perform prediction processing on the images in the dataset: First, the data in each mini-batch is horizontally flipped with a probability of 50%. Then, the length and width of the images in each mini-batch are fixed, and the length is scaled to an integer multiple of 32 between 400 and 600 pixels. The width is then padded to an integer multiple of 32 between 400 and 600 pixels. Step 5: Use the training data to co-train the object detection model and decoder: In the first step, the preprocessed data is input into the backbone network for feature extraction to generate feature maps C2, C3, C4, and C5. The backbone network is resnet50 or cspnext. In the second step, the feature map is fed into the object detection head to predict the category and bounding box, and then the cross entropy loss and bounding box loss are calculated with the true label; In the third step, the feature maps C2, C3, C4, and C5 generated in the first step are fed into the edge enhancement network module for edge enhancement module, context fusion module, and feature fusion. Then, a feature map of the same size as the image preprocessed is output, and the mean square error loss function between the preprocessed image and the feature map is calculated. The fourth step is to pass the loss functions of the second and third steps back to update the entire network structure at the same time; Step 6: Feed the verification data into the detection model for detection: In the first step, the preprocessed data is input into the backbone network for feature extraction to generate feature maps C2, C3, C4, and C5; In the second step, the feature maps C2, C3, C4, and C5 are fed into the object detection head for category and bounding box prediction, and then processed by non-maximum suppression (NMS) to generate the detection results. Step 7: Get the test results: Map the detection results obtained in step 6 to the corresponding positions of the original image and mark the predicted category information.

2. The SAR image target detection optimization method based on edge enhancement network according to claim 1 is characterized in that: The construction of the edge enhancement network module and the embedding training of the edge enhancement network module described in steps 1, 2 and 5 design a corresponding edge enhancement network for the backbone network of the detection model, and use the constructed edge enhancement network module for collaborative training with the object detection model to enhance the feature extraction capability in the backbone network and make the detection stronger.

Images