SAR image target detection optimization method based on edge enhancement network
By introducing edge enhancement network module into the SAR image detection model, the problem of incomplete feature extraction in SAR images is solved, the detection accuracy and generalization ability are significantly improved, and the performance of the detection model is improved.
Patent Information
- Application Number
- CN202411269884.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-11
- Publication Date
- 2024-12-03
- Estimated Expiration
- 2044-09-11
AI Technical Summary
Due to the coherent spot noise and obvious edge features in SAR images, the detection model based on convolutional neural networks is incomplete in feature extraction and low detection accuracy.
Build an edge enhancement network module and introduce it into an object detection model based on convolutional neural networks. Through collaborative training, feature extraction capabilities are improved, and feature extraction and detection accuracy are enhanced by combining edge enhancement modules and context fusion modules.
It significantly improves the detection accuracy and generalization capabilities of SAR images, and improves the performance of the detection model, especially in processing edge features and noise environments.
Smart Images

Figure CN119068341A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of remote sensing image building detection technology, and more particularly to a SAR image target detection optimization method and system based on edge enhancement network. The present invention can be used to enhance the detection accuracy in SAR images. Background Art
[0002] Target detection is an important direction in the field of intelligent interpretation of remote sensing images, and can provide basic data for many remote sensing applications. For example, buildings, ships and other man-made objects in remote sensing images are important sources of information for intelligent analysis and decision-making. At present, optical remote sensing images occupy a large proportion due to their high clarity and intuitive effects. However, weather and lighting factors have a great impact on optical remote sensing images, while images obtained by synthetic aperture radar (SAR) are not affected by these factors. Synthetic aperture radar (SAR) imaging has become a key technology in the field of remote sensing and has many advantages over traditional optical sensors. SAR has the ability to obtain geographic images under any conditions, is not affected by weather conditions, and can penetrate clouds, fog, rain and snow. SAR has been widely used in many fields, including national defense, geographic survey, disaster risk monitoring, etc.
[0003] In recent years, due to the advantages of SAR images, they have attracted more and more attention in the field of target detection, and a large number of excellent SAR image target detection have emerged. At present, SAR image target detection mainly includes traditional detection algorithms and deep learning-based detection algorithms. Traditional SAR image target detection algorithms mainly include constant false-alarm rate (CFAR) detection methods and detection algorithms based on manually extracted image texture features. The target detection methods based on deep learning have made significant progress in SAR image target detection because of their powerful feature learning capabilities. These methods are usually based on convolutional neural networks and transformers. The deep learning method can actively extract deep features to avoid manual selection and greatly improve the detection efficiency; in the case of reliable data sets, deep learning also has good robustness and generalization ability, and can adapt to many complex situations. With the development of artificial intelligence technology, many excellent deep learning target detection methods are applied to SAR image detection, but SAR images have the characteristics of coherent speckle noise and obvious edge features, so the model needs further research in terms of detection accuracy and generalization ability.
[0004] Shaoqing Ren, Kaiming He and others published the paper "Faster r-cnn: Towards real-time object detection with region proposal networks" (Advances in neural information processing systems), which continued to improve on the basis of Fast R-CNN. They introduced a region proposal network (RPN) based on Fast R-CNN, and generated proposal boxes for the extracted features through the RPN network. Then the generated proposal boxes and feature maps were sent to ROIpooling for cropping, and then sent to the Fast R-CNN prediction head for prediction. This method makes Fast R-CNN an end-to-end object detection network, which significantly improves the detection speed while maintaining the detection accuracy.
[0005] An et al. published a paper titled "DRBox-v2: An improved detector with rotatable boxes for target detection in SAR images", proposing an improved RBox-based target detection framework that designs a multi-layer prior box generation strategy for target detection. They also proposed a combination of focal loss (FL) and hard negative mining technology to alleviate the imbalance problem between positive and negative samples.
[0006] Tang et al. published a paper titled “Large-difference-scale target detection using avised Bhattacharyya distance in SAR images”. In order to improve the ability of small target detection, they improved the Bhattacharyya distance, which can better measure the deviation of the bounding box caused by large differences in target scales. It can be embedded in the anchor-based detection model.
[0007] Zhou et al. published a paper titled "HRLE-SARDet: A lightweight SAR target detection algorithm based on hybrid representation learning enhancement". Based on the lightweight perspective of SAR image features, HRLE-SARDet is proposed, which is a learning enhanced SAR image detection algorithm. This network includes a lightweight, scattering feature extraction backbone, and a multi-scale feature fusion neck. A hybrid representation learning enhancement module is also designed to extract small target information in SAR images.
[0008] However, due to the existence of coherent speckle noise and obvious edge features in SAR images, the features extracted by general detection models based on convolutional neural networks are imperfect in the feature extraction stage, resulting in unsatisfactory subsequent detection accuracy. Summary of the invention
[0009] The purpose of the present invention is to propose a SAR image target detection optimization method based on edge enhancement network to address the shortcomings of the existing technology, which solves the problems of incomplete feature extraction and low detection accuracy caused by a series of reasons such as the presence of coherent speckle noise and incomplete edge feature extraction.
[0010] The implementation idea of the present invention is: first, construct an edge enhancement network module, and introduce the constructed edge enhancement network module into a general object detection model based on a convolutional neural network. Then, divide the object detection data set, perform prediction processing on the images in the data set, and use the training data to perform collaborative training with the object detection model and the decoder. Finally, send the verification data to the detection model for detection to obtain the detection result.
[0011] The specific steps of the present invention are as follows: Step 1: Build edge enhancement network module: Construct an edge enhancement network module consisting of an edge enhancement module and a context fusion module. The structure of the edge enhancement module is: grouped convolution block → edge enhancement convolution block → relu function → 1×1 convolution; the structure of the grouped convolution block is to set the output channel to 1 / 4 of the input channel through a normal convolution layer and set a separate grouping number for each convolution block; the edge enhancement convolution block is to integrate the edge enhancement operator into the normal convolution; The structure of the context fusion module is: dilated DW convolution block → context attention module; the structure of the dilated DW convolution block is: relu function → 3×3 DW convolution → parallel DW convolution → feature splicing; parallel DW convolution is composed of a series of large-core DW convolutions; feature splicing splices the features obtained by a series of parallel DW convolutions; the structure of the context attention module is: average pooling layer → 1×1 convolution → 3×3 DW convolution → 1×1 convolution → sigmoid function; Step 2: Introduce the edge enhancement network module into the general convolutional neural network-based object detection model: The constructed edge enhancement network module is introduced into the backbone network of the object detection model. After placing it in four different stages in the backbone network, the obtained features are fused with the original features. Step 3: Divide the object detection dataset: Select some annotated optical remote sensing images with unlimited length and width (each batch will preprocess the images, scale the images in the batch, and then fill them into images of uniform size). If the dataset category is difficult to train, more images are required. A simple dataset can use 1000+ images. Then 80% of the images form the training set, and 20% of the images form the validation set and test set. Step 4: Perform prediction processing on the images in the dataset: First, the data in each mini-batch is horizontally flipped with a probability of 50%. Then, the length and width of the images in each mini-batch are fixed, the length is scaled to an integer multiple of 32 between 400 and 600, and the width is padded to an integer multiple of 32 between 400 and 600. Step 5: Use the training data to co-train the object detection model and decoder: The first step is to input the preprocessed data into the backbone network (resnet50 or cspnext, etc.) for feature extraction to generate feature maps C2, C3, C4, and C5; In the second step, the feature map is sent to the object detection head for category and bounding box prediction, and then the cross entropy loss and bounding box loss are calculated with the true label; In the third step, the feature maps C2, C3, C4, and C5 generated in the first step are sent to the edge enhancement network module for edge enhancement module, context fusion module, and feature fusion, and then the feature map of the same size as the image after preprocessing is output, and then the mean square error loss function between the preprocessed image and the feature map is calculated; The fourth step is to simultaneously pass back the loss functions of the second and third steps to update the entire network structure; Step 6: Send the verification data to the detection model for detection: In the first step, the preprocessed data is input into the backbone network for feature extraction to generate feature maps C2, C3, C4, and C5; In the second step, the feature maps C2, C3, C4, and C5 are sent to the object detection head for category and bounding box prediction, and then processed by non-maximum suppression (NMS) to generate detection results; Step 7, get the test results: Map the detection results obtained in step 6 to the corresponding positions of the original image and mark the predicted category information.
[0012] Compared with the prior art, the present invention has the following advantages: First, the structure has strong embedding properties. It can be embedded in some mainstream object detection models, such as RtmDet, YOLO, and Faster R-CNN. Second, although it takes longer to train than the detection model without the edge enhancement network module, the prediction time is comparable and is not inferior to the prediction results of a deeper backbone network. BRIEF DESCRIPTION OF THE DRAWINGS
[0013] Figure 1 This is a flow chart of the SAR image target detection optimization method based on edge enhancement network.
[0014] Figure 2 It is a test result diagram of the present invention. DETAILED DESCRIPTION
[0015] The present invention will be described in detail below in conjunction with the accompanying drawings.
[0016] Reference Figure 1 , the steps of the present invention are further described.
[0017] Step 1: Build edge enhancement network module: Construct an edge enhancement network module consisting of an edge enhancement module and a context fusion module. The structure of the edge enhancement module is: grouped convolution block → edge enhancement convolution block → relu function → 1×1 convolution; the structure of the grouped convolution block is to set the output channel to 1 / 4 of the input channel through a normal convolution layer and set a separate grouping number for each convolution block; the edge enhancement convolution block is to integrate the edge enhancement operator into the normal convolution; The structure of the context fusion module is: dilated DW convolution block → context attention module; the structure of the dilated DW convolution block is: relu function → 3×3 DW convolution → parallel DW convolution → feature splicing; parallel DW convolution is composed of a series of large-core DW convolutions; feature splicing splices the features obtained by a series of parallel DW convolutions; the structure of the context attention module is: average pooling layer → 1×1 convolution → 3×3 DW convolution → 1×1 convolution → sigmoid function; Step 2: Introduce the edge enhancement network module into the general convolutional neural network-based object detection model: The constructed edge enhancement network module is introduced into the backbone network of the object detection model. After placing it in four different stages in the backbone network, the obtained features are fused with the original features. Step 3: Divide the object detection dataset: Select some annotated optical remote sensing images with unlimited length and width (each batch will preprocess the images, scale the images in the batch, and then fill them into images of uniform size). If the dataset category is difficult to train, more images are required. A simple dataset can use 1000+ images. Then 80% of the images form the training set, and 20% of the images form the validation set and test set. Step 4: Perform prediction processing on the images in the dataset: First, the data in each mini-batch is horizontally flipped with a probability of 50%. Then, the length and width of the images in each mini-batch are fixed, the length is scaled to an integer multiple of 32 between 400 and 600, and the width is padded to an integer multiple of 32 between 400 and 600. Step 5: Use the training data to co-train the object detection model and decoder: The first step is to input the preprocessed data into the backbone network (resnet50 or cspnext, etc.) for feature extraction to generate feature maps C2, C3, C4, and C5; In the second step, the feature map is sent to the object detection head for category and bounding box prediction, and then the cross entropy loss and bounding box loss are calculated with the true label; In the third step, the feature maps C2, C3, C4, and C5 generated in the first step are sent to the edge enhancement network module for edge enhancement module, context fusion module, and feature fusion, and then the feature map of the same size as the image after preprocessing is output, and then the mean square error loss function between the preprocessed image and the feature map is calculated; The fourth step is to simultaneously pass back the loss functions of the second and third steps to update the entire network structure; Step 6: Send the verification data to the detection model for detection: In the first step, the preprocessed data is input into the backbone network for feature extraction to generate feature maps C2, C3, C4, and C5; In the second step, the feature maps C2, C3, C4, and C5 are sent to the object detection head for category and bounding box prediction, and then processed by non-maximum suppression (NMS) to generate detection results; Step 7, get the test results: Map the detection results obtained in step 6 to the corresponding positions of the original image and mark the predicted category information.
[0018] The effects of the present invention are further described below in conjunction with simulation.
[0019] 1. Simulation experiment conditions: The simulation experiment conditions of the present invention are as follows: server GPU: GeForce RTX3090, video memory 24G; The software platform of the simulation experiment of the present invention: ubuntu18.04 system, python 3.10, pytorch-gpu2.0.0.
[0020] 2. Analysis of simulation content and experimental results: The simulation of the present invention is to detect SAR images by introducing the edge enhancement network module of the existing technology (RtmDet method) to obtain the detection results. The data sets used in the simulation are: SSDD, RSDD-SAR and SAR-AIRcraft.
[0021] Several existing technologies used in the simulation experiment are: The Faster R-CNN detection method is an object detection method proposed by Shaoqing Ren, Kaiming He and others in "Ren, S., He, K., Girshick, R., Sun, J. (2015). Faster r-cnn: Towards real-time object detection with region proposal networks. Advances in neural information processing systems, 28, 91-99.", referred to as the Faster R-CNN detection method; The Cascade R-CNN detection method is an object detection method proposed by Cai Z et al. in "Cai Z, Vasconcelos N. Cascade r-cnn: Delving into high quality object detection[C] / / Proceedings of the IEEE conference on computer vision and pattern recognition. 2018: 6154-6162." The Dynamic R-CNN detection method is an object detection algorithm proposed by Zhang et al. in "Zhang H, Chang H, Ma B, et al. Dynamic R-CNN: Towards high quality object detection via dynamic training [C] / / Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XV 16. Springer International Publishing, 2020: 260-275." The Libra R-CNN detection method is an object detection method proposed by Pang et al. in "Pang J, Chen K, Shi J, et al. Librar-cnn: Towards balanced learning for object detection[C] / / Proceedings of theIEEE / CVF conference on computer vision and pattern recognition. 2019: 821-830." The Grid R-CNN detection method is an object detection method proposed by Lu et al. in "Lu X, Li B, Yue Y, et al. Grid r-cnn[C] / / Proceedings of the IEEE / CVF conference on computer vision and pattern recognition. 2019: 7363-7372." The YOLOX detection method is an object detection method proposed by Ge et al. in "Ge Z, Liu S, Wang F, et al. Yolox: Exceedingyolo series in 2021[J]. arXiv preprint arXiv:2107.08430, 2021." The RtmDet detection method is an object detection method proposed by Lyu C et al. in “Lyu C, Zhang W, Huang H, et al. Rtmdet:An empirical study of designing real-time object detectors[J]. arXiv preprintarXiv:2212.07784, 2022.”
[0022] The input images used in the simulation experiment of the present invention are the public SSDD dataset and SAR-AIRcraft dataset. The SSDD dataset was released by Wuhan University in 2017. Its images were extracted from RadarSat-2, TerraSAR-X, and Sentinel-1, including 1,160 SAR ship images with an image size of 500×500×3. We use 1,000 ship images as training sets and 160 images as validation sets. The SAR-AIRcraft dataset is constructed based on the Gaofen-3 satellite images with a resolution of 1 meter. The dataset contains four aircraft of different sizes, 16,434 target instances, and consists of 4,368 images.
[0023] Simulation experiment 1 introduces the edge enhancement network module into RtmDet and compares the detection results on SSDD.
[0024] Simulation experiment 2 introduces the edge enhancement network module into RtmDet and compares the detection results on SAR-AIRcraft.
[0025] In order to verify the effect of the present invention on SAR image detection, the detection results are evaluated using the coco evaluation index, and the calculation results are plotted in the following table.
[0026] Table 1. Comparison table of simulation experimental results of the present invention on SSDD dataset Method #Params(M) GFLOPs mAP(%) AP50(%) AP75(%) Faster R-CNN 41.4 137.2 44.7 80.6 44.5 Cascade R-CNN 69.2 165.9 46.0 81.8 48.0 Libra R-CNN 41.7 141.3 48.1 83.0 52.3 Dynamic R-CNN 41.4 137.2 40.0 82.1 31.1 Grid R-CNN 64.5 253.9 47.4 83.6 48.3 YOLOX 8.9 12.9 47.7 87.3 48.8 RtmDet 8.9 13.9 52.3 88.0 56.7 EEN 11.4 18.3 52.6 89.2 56.3 Table 2. Comparison of simulation results of the present invention on the SAR-AIRcraft dataset Method mAP(%) AP50(%) AP75(%) Faster R-CNN 53.1 91.6 56.8 Cascade R-CNN 55.5 93.7 61.8 Libra R-CNN 54.3 91.3 59.8 Dynamic R-CNN 52.6 90.7 56.2 Grid R-CNN 52.5 92.0 55.0 YOLOX 58.4 95.2 65.9 RtmDet 59.1 95.2 67.9 EEN 65.7 97.0 74.3 Combined with Table 1 above, it is not difficult to see that the detection algorithm that introduces the edge enhancement network module for training has significantly improved the detection accuracy compared to RtmDet without the introduction. On the SSDD dataset, the EEN model with the edge enhancement network module embedded in our experiment obtained the best AP50, with AP50 reaching 89.2% and mAP reaching 52.6%. Combined with Table 2, the table shows the experimental results of the mainstream detection models on the SAR-AIRcraft dataset. We can observe that the performance has been significantly improved after the edge enhancement network is embedded. On EEN, mAP, AP50 and AP75 can reach 65.7%, 97.0% and 74.3% respectively.
[0027] Combine the following Figure 2 The effect of this simulation experiment is further explained.
[0028] Figure 2 This is a visualization result on the SSDD dataset, which includes the true labels and the results predicted by RtmDet and EEN. Figure 2 We can see that EEN has much better performance in terms of false detection and missed detection on the SSDD dataset than RtmDet, which means that the present invention is superior to the original detection model without the present invention in terms of accuracy, missed detection and false detection.
Claims
1. An optimization method for SAR image target detection based on edge enhancement network, It is characterized in that The method comprises: The edge enhancement network module and the object detection model are used for collaborative training to enhance the feature extraction network of the detection model and improve the detection accuracy. The specific steps of this method include the following: Step 1: Build edge enhancement network module: Construct an edge enhancement network module consisting of an edge enhancement module and a context fusion module. The structure of the edge enhancement module is: grouped convolution block → edge enhancement convolution block → relu function → 1×1 convolution; the structure of the grouped convolution block is to set the output channel to 1 / 4 of the input channel through a normal convolution layer and set a separate grouping number for each convolution block; the edge enhancement convolution block is to integrate the edge enhancement operator into the normal convolution; The structure of the context fusion module is: dilated DW convolution block → context attention module; the structure of the dilated DW convolution block is: relu function → 3×3 DW convolution → parallel DW convolution → feature splicing; parallel DW convolution is composed of a series of large-core DW convolutions; feature splicing splices the features obtained by a series of parallel DW convolutions; the structure of the context attention module is: average pooling layer → 1×1 convolution → 3×3 DW convolution → 1×1 convolution → sigmoid function; Step 2: Introduce the edge enhancement network module into the general convolutional neural network-based object detection model: The constructed edge enhancement network module is introduced into the backbone network of the object detection model. After placing it in four different stages in the backbone network, the obtained features are fused with the original features. Step 3: Divide the object detection dataset: Select some annotated optical remote sensing images with unlimited length and width (each batch will preprocess the images, scale the images in the batch, and then fill them into images of uniform size). If the dataset category is difficult to train, more images are required. A simple dataset can use 1000+ images. Then 80% of the images form the training set, and 20% of the images form the validation set and test set. Step 4: Perform prediction processing on the images in the dataset: First, the data in each mini-batch is horizontally flipped with a probability of 50%. Then, the length and width of the images in each mini-batch are fixed, the length is scaled to an integer multiple of 32 between 400 and 600, and the width is padded to an integer multiple of 32 between 400 and 600. Step 5: Use the training data to co-train the object detection model and decoder: The first step is to input the preprocessed data into the backbone network (resnet50 or cspnext, etc.) for feature extraction to generate feature graphs C2, C3, C4, and C5; In the second step, the feature map is sent to the object detection head for category and bounding box prediction, and then the cross entropy loss and bounding box loss are calculated with the true label; In the third step, the feature maps C2, C3, C4, and C5 generated in the first step are sent to the edge enhancement network module for edge enhancement module, context fusion module, and feature fusion, and then the feature map of the same size as the image after preprocessing is output, and then the mean square error loss function between the preprocessed image and the feature map is calculated; The fourth step is to simultaneously pass back the loss functions of the second and third steps to update the entire network structure; Step 6: Send the verification data to the detection model for detection: In the first step, the preprocessed data is input into the backbone network for feature extraction to generate feature maps C2, C3, C4, and C5; In the second step, the feature maps C2, C3, C4, and C5 are sent to the object detection head for category and bounding box prediction, and then processed by non-maximum suppression (NMS) to generate detection results; Step 7, get the test results: Map the detection results obtained in step 6 to the corresponding positions of the original image and mark the predicted category information.
2. According to the SAR image target detection optimization method based on edge enhancement network described in claim 1, It is characterized in that The construction of the edge enhancement network module and the embedding training of the edge enhancement network module described in steps 1, 2 and 5 design a corresponding edge enhancement network for the backbone network of the detection model, and use the constructed edge enhancement network module for collaborative training with the object detection model to enhance the feature extraction capability in the backbone network and make the detection stronger.
Citation Information
Patent Citations
Multilevel semantic fusion cloud and cloud shadow detection method and device, and storage medium
CN114943876A
Animal identification method based on dual-channel feature fusion
CN115273131A
Camouflage target detection method based on attention mechanism and convolutional neural network
CN116228702A
Infrared small target detection method and device based on deep fusion of edge details and deep features
CN116468980A
Battery pole piece defect detection method, computing equipment and storage medium
CN117058105A
