SAR image fusion detection method based on style transfer

Abstract

The present invention discloses a SAR image target detection method based on style transfer, which mainly solves the problem of low detection performance caused by insufficient number of image samples and low quality in the prior art when performing SAR image target detection. The scheme is: using a style transfer network to generate a visible light image corresponding to the SAR sample; sending both the SAR original image and the visible light image to the target detection network to obtain two-way detection results, and pre-processing them to divide training data and test data; constructing a classification network formed by a convolutional neural network and a long-short time memory network in parallel; sending the training data to the classification network for training; inputting the test data into the trained classification network for decision-level fusion to obtain the final detection result. The present invention has high detection accuracy, especially in the case of insufficient data samples and complex sample conditions, it can still obtain good detection results, and can be used for SAR image processing.

Description

Technical Field

[0001] The invention belongs to the technical field of radar image processing, and in particular relates to a method for detecting targets in SAR images, which can be used for image processing. Background Art

[0002] Synthetic aperture radar SAR is an advanced active microwave earth observation equipment. It can be installed on a variety of different flight platforms such as aircraft, satellites, and spacecraft to conduct all-day and all-weather ground observations and obtain target images similar to optical photos. Due to its certain surface penetration ability, SAR is widely used in surveying and mapping, military, navigation, agriculture and forestry, geological survey, disaster warning and other fields, and plays a very important role in the national economy and military applications. Therefore, using SAR data for target detection is also one of the research directions of image interpretation and has important research value.

[0003] Existing SAR image target detection methods can be divided into two categories: non-learning target detection algorithms and machine learning-based target detection algorithms. The constant false alarm rate CFAR algorithm is a traditional non-learning target detection algorithm. The dual-parameter CFAR detection algorithm is an improved CFAR detection method and a pixel-level target detection algorithm. This method assumes that the statistical distribution model of background clutter is a Gaussian distribution. First, the detection threshold is calculated according to the given false alarm probability, and then the statistical detection amount of the pixel to be detected is calculated. Finally, the statistical detection amount is compared with the detection threshold. If it is greater than the threshold, it is judged as a target pixel, otherwise it is judged as a clutter pixel, and finally the pixel-level target detection result is obtained. The dual-parameter CFAR algorithm performs the same operation on each pixel in the image, so the detection time is relatively long. At the same time, the algorithm needs to set a fixed-size window according to the prior information of the target in the image. When the size of the same target or different targets in the image is greatly different, the detection performance of the algorithm will be reduced.

[0004] The continuous development of society has led to the continuous expansion of a large number of easy-to-use cloud computing infrastructures. Numerous open source machine learning frameworks and an expanding service ecosystem have paved the way for the successful application of neural networks in different fields. Since Ross Girshick proposed the regional convolutional neural network R-CNN model in 2013, the target detection and recognition has transitioned from traditional manual feature extraction to feature extraction based on deep learning, and target detection has since stepped onto a new level. The target detection algorithm is mainly divided into two schools: the "two-step" algorithm represented by R-CNN and the "one-step" algorithm represented by SSD and YOLO series. The R-CNN model first generates candidate regions and then puts them into a CNN for feature extraction. At the output end, on the one hand, it is connected to a classifier to determine the category of the target, and on the other hand, it is connected to a regressor to further determine the appropriate position of the border, that is, to find the border that "just" frames the target. One year after R-CNN was proposed, a team led by He Kaiming and Ren Shaoqing proposed the spatial pyramid pooling network SPP Net, which optimized R-CNN. That is, instead of generating candidate regions first and then convolving, it became convolution first and then generating regions. By changing the order, the storage volume was reduced and the training speed was accelerated. Later, Ren Shaoqing and others proposed the concept of regional generation network RPN. RPN predicts whether there is a target at each position in the image to be detected and gives the most likely candidate region. Since RPN and fast-R-CNN share the feature extraction part of the convolutional neural network, only the features of the image to be detected need to be extracted once, which speeds up the target detection. This algorithm is called Faster-R-CNN. The above target detection algorithm has achieved excellent detection results in the field of visible light image target detection, but SAR images have complex background noise, lack of image data, and difficulty in target feature extraction compared to visible light images. Therefore, the above algorithm is not ideal when directly applied to SAR image target detection.

[0005] To solve this problem, the patent document with application number CN 201610541067 proposes a SAR image target detection method based on CFAR detection and deep learning. The method first constructs a target detection model Faster R-CNN containing two modules, RPN and Fast R-CNN, and uses labeled training SAR images to iteratively train RPN and Fast-R-CNN in stages, and finally obtains a trained target detection model; when testing, the CFAR detection algorithm is first used to extract the detection slice, and then the trained model is used to perform target detection on the test detection slice, and finally the target detection results of all the detection slices are merged at the relative position of the original SAR image to obtain the detection result of the test SAR image. This method utilizes the powerful feature extraction ability of the convolutional neural network and achieves a high target detection accuracy. However, since this method relies on a large amount of labeled training data, when the SAR image training data is small, it is easy to cause the deep model to overfit, thereby reducing the target detection accuracy. Summary of the invention

[0006] The present invention aims to address the deficiencies of the above-mentioned existing methods and propose a SAR image fusion detection method based on style transfer to enhance sample data and improve the accuracy of SAR image target detection.

[0007] The technical solution of the present invention is: to solve the problem of small number of samples in existing SAR image datasets, a style transfer network is used to enhance the data of SAR samples, different samples are detected respectively, and the detection accuracy of SAR image targets is improved by performing decision-level fusion on the detection results. The specific implementation includes the following:

[0008] (1) Obtain the visible light image corresponding to the original SAR image data:

[0009] (1a) Use the SEN1_2 dataset to train the Pix2Pix network using the gradient descent method to obtain the trained Pix2Pix network;

[0010] (1b) Input the SAR image ship target detection dataset SSDD into the trained Pix2Pix network to obtain its corresponding visible light dataset;

[0011] (2) The two data sets, the SAR image raw data and the visible light image corresponding to the SAR image raw data, are divided into a detection image training set and a detection image verification set in a ratio of 7:3;

[0012] (3) Using the two-way detection image training set and the adaptive momentum estimation optimization algorithm Adam to train the YOLO-v3 target detection network, a trained YOLO-v3 target detection network is obtained;

[0013] (4) Input the two detection image datasets into the trained YOLO-v3 network respectively to obtain the two target detection results;

[0014] (5) The two-way target detection results obtained in (4) are preprocessed by decision-level fusion to obtain a data set of detection results, which is then divided into a detection result training set and a test set in a ratio of 8:2 as the input of the decision fusion network;

[0015] (6) Construct a classification neural network composed of a convolutional neural network (CNN) and a long short-term memory (LSTM) network in parallel;

[0016] (7) Sending the test result training set into the constructed classification neural network, using the batch gradient descent method to iteratively train its network parameters to obtain a trained decision-level fusion network model;

[0017] (8) The detection result test set is input into the trained decision-level fusion network to obtain the SAR image intelligent fusion target detection result.

[0018] Compared with the prior art, the present invention has the following advantages:

[0019] 1) Since the present invention uses the Pix2Pix style transfer network, the original SAR image can be style transferred to obtain its corresponding visible light image. In addition to the features of the SAR image itself, the features of the corresponding visible light image can be introduced. In view of the problems of small sample size and low sample quality of current SAR image data, data enhancement processing is performed, which can display the features of the target from different angles, help to better detect the target, and provide richer feature information for subsequent intelligent decision fusion.

[0020] 2) The present invention builds a CNN-LSTM intelligent decision fusion network, which can be used to intelligently fuse the two detection results of the previous level, and intelligently select decision rules and decision conditions through the neural network to improve the detection performance of the overall network, avoiding the problem that conventional decision-level fusion strategies require manual selection of decision thresholds and design of decision rules, and cannot effectively utilize the detection results. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1 It is a flow chart for implementing the present invention.

[0022] Figure 2 This is a decision-level fusion structure diagram constructed in the present invention. DETAILED DESCRIPTION

[0023] The embodiments and effects of the present invention are further described in detail below with reference to the accompanying drawings.

[0024] Reference Figure 1 The SAR image target detection method based on style migration of the present invention is implemented in the following steps:

[0025] Step 1: Obtain the visible light image corresponding to the SAR image raw data.

[0026] (1.1) Download the SEN1_2 dataset from the public website and use it to train the Pix2Pix network using the gradient descent method:

[0027] (1.1.1) Set the learning rate to 0.0002, and the momentum parameters to β1 = 0.5, β2 = 0.999;

[0028] (1.1.2) The training batch size is set to 1, that is, each iteration trains with one image as a group, and the network optimizes the parameters once each iteration;

[0029] (1.1.3) Select the adaptive moment estimation algorithm Adam as the network optimization algorithm, and use this algorithm to calculate and correct the first-order moment and second-order moment of each round of training gradient to optimize the network parameters in a single iteration;

[0030] (1.1.4) Set the maximum number of network iterations to 100, perform iterative training for 100 times, and obtain the trained Pix2Pix network model;

[0031] (1.2) Download the SAR image ship target detection dataset SSDD from the public website and input the dataset into the trained Pix2Pix network to obtain its corresponding visible light dataset.

[0032] Step 2: Divide the data set.

[0033] The two data sets, the SAR image raw data and the visible light image corresponding to the SAR image raw data, are divided into a detection image training set and a detection image verification set in a ratio of 7:3.

[0034] Step 3: Use the two-way detection image training set to train the existing YOLO-v3 target detection network using the adaptive momentum estimation optimization algorithm Adam to obtain a trained YOLO-v3 target detection network.

[0035] (3.1) The longest and widest side of the detection image is transformed to 416 in size, and then the short side is compressed in the same proportion. For the short side that does not reach 416 in length, gray pixels are filled at both ends of the short side to make its length reach 416, so as to achieve proportional scaling of the detection image;

[0036] (3.2) In order to enable the YOLO-v3 network model to learn the characteristics of the target in various forms, the training set images are randomly flipped left and right and cropped at each iteration;

[0037] (3.3) Assume that the training batch size is 2, that is, each iteration is trained with 2 images as a group, and the model optimizes the parameters once every iteration;

[0038] (3.4) Assume that the initial learning rate is 0.001, and use the Adam algorithm to calculate and correct the first-order moment and second-order moment of each round of training gradient to optimize the network parameters in a single iteration;

[0039] (3.5) Set the maximum number of network iterations to 150, repeat (3.1)-(3.4) 150 times in total, and obtain the trained YOLO-v3 network model.

[0040] Step 4: Obtain the target detection results and preprocess them.

[0041] (4.1) Input the two detection image datasets into the trained YOLO-v3 network respectively to obtain the two-way target detection results;

[0042] (4.2) Preprocess the two-way target detection results:

[0043] Reference Figure 2 , the specific implementation of this step is as follows:

[0044] (4.2.1) Each ship target in the two detection results is framed in the image with a rectangular target, whose coordinate value is (x1, y1, x2, y2). The intersection-over-union ratio (IOU) between each target in the two results is calculated and compared with the set first threshold ω1 = 0.5:

[0045] If IOU ≥ ω1, it is considered that both paths have detected the target, and the two-path detection results of this target are merged and saved, and step (4.2.2) is executed;

[0046] If IOU < ω1, it is considered that the target is detected in only one path, the coordinate value of the other detection result is 0, and the result is saved;

[0047] (4.2.2) Calculate the IOU of the ship target detected in the merged result and the ship target corresponding to the label in the test set of the detection result, and compare it with the set second threshold ω2 = 0.5:

[0048] If IOU ≥ ω2, the correct target is considered to be detected, and a marker 1 representing the correct value is added after the merged detection result;

[0049] If IOU < ω2, it is considered that the correct target is not detected, and a marker 0 representing the error value is added after the merged detection result.

[0050] Step 5: Build a CNN-LSTM classification neural network.

[0051] (5.1) Construct a convolutional neural network (CNN) model:

[0052] Assume a convolutional neural network CNN model, including 2 convolutional layers, 2 pooling layers, 1 fully connected layer and 1 dropout layer, where:

[0053] The first convolutional layer is the input layer, with 60,000 input nodes, 64 convolution kernels, and a convolution kernel size of 1×5;

[0054] The second convolutional layer is the output layer, with 50 output nodes, 128 convolution kernels, and a convolution kernel size of 1×3;

[0055] Both pooling layers use downsampling with a sampling size of 1×2;

[0056] The dropout rate of the dropout layer is 0.7;

[0057] Set the learning rate of the convolutional neural network CNN to 0.01, and use the [0,1] normal distribution to initialize the weights W of the network nodes. l and bias value b l , the activation functions of the convolutional layer, pooling layer and fully connected layer all use linear rectification functions, expressed as f(x)=max(0,x), where x is the independent variable and the x value is equal to the input value of the node in the layer where the activation function is located;

[0058] The first convolutional layer, the first pooling layer, the second convolutional layer, the second pooling layer, the fully connected layer, and the dropout layer are cascaded in sequence to obtain a convolutional neural network CNN model.

[0059] (5.2) Constructing the long short-term memory unit network LSTM:

[0060] Assume that the number of layers of the long short-term memory unit is 1, the number of input nodes is 60,000, the number of output nodes is 50, and the weight W of the memory unit node is initialized using the [0,1] normal distribution l and bias value b l .

[0061] (5.3) The networks constructed in steps (5.1) and (5.2) are connected in parallel to obtain a CNN-LSTM classification neural network.

[0062] Step 6: Use batch gradient descent method to iteratively train the classification neural network.

[0063] (6.1) Set the data batch size to 64, divide the test result training set into multiple batches of data according to the batch size, and send these data batch by batch into the classification neural network for feature extraction;

[0064] (6.2) Adaptive moment estimation algorithm Adam is selected to calculate and correct the first-order and second-order moments of each round of training gradients using the features extracted by the network, so as to optimize the network parameters in a single iteration;

[0065] (6.3) Set the maximum number of network iterations to 150, repeat (6.1) and (6.2) 150 times in total, and obtain a trained classification neural network model.

[0066] Step 7: Obtain the SAR image intelligent fusion target detection result.

[0067] The detection result test set is input into the trained classification neural network model to obtain the final SAR image intelligent fusion target detection result.

[0068] The technical effects of the present invention are further illustrated below through simulation experiments.

[0069] 1. Simulation experiment conditions:

[0070] Windows 10 system.

[0071] Build the YOLO-v3 network under Pytorch of Python 3.8, and the development language is Python.

[0072] Build a CNN-LSTM classification neural network under Keras in Python 3.6, and the development language is Python.

[0073] 2. Simulation content:

[0074] Under the above conditions, the SSDD dataset was used to conduct target detection simulation experiments using the present invention and the existing target detection algorithm YOLO-v3, and the accuracy and recall of target detection of the two methods were calculated, and the detection results were compared, as shown in Table 1:

[0075] Table 1 Comparison of target detection results of two methods

[0076] The present invention Existing YOLO-v3 algorithm Accuracy 88.73 81.72 Recall 88.22 91.5

[0077] As shown in Table 1, the accuracy of target detection of the present invention is 88.73%, which is significantly higher than the accuracy of the existing target detection algorithm YOLO-v3 (81.72%). The recall rate is 88.22%, which is slightly lower than the 91.5% of the YOLO-v3 algorithm. However, it is at the expense of a small part of the recall rate, which greatly improves the accuracy of target detection and achieves a better detection effect overall.

[0078] In summary, the SAR image target detection algorithm based on style transfer proposed in the present invention can perform more accurate target detection on SAR images.

Claims

1. A SAR image target detection method based on style transfer, characterized in that: These include: (1) Obtain the visible light image corresponding to the original SAR image data: (1a) Use the SEN1_2 dataset to train the Pix2Pix network using the gradient descent method to obtain the trained Pix2Pix network; (1b) Input the SAR image ship target detection dataset SSDD into the trained Pix2Pix network to obtain its corresponding visible light dataset; (2) The two data sets, the SAR image raw data and the visible light image corresponding to the SAR image raw data, are divided into a detection image training set and a detection image verification set in a ratio of 7:3; (3) Using the two-way detection image training set and the adaptive momentum estimation optimization algorithm Adam to train the YOLO-v3 target detection network, a trained YOLO-v3 target detection network is obtained; (4) Input the two detection image verification sets into the trained YOLO-v3 network respectively to obtain the two-way target detection results; (5) The two-way target detection results obtained in (4) are preprocessed by decision-level fusion to obtain a data set of detection results, which is then divided into a detection result training set and a test set in a ratio of 8:2 as the input of the decision fusion network; (6) Construct a classification neural network composed of a convolutional neural network (CNN) and a long short-term memory (LSTM) network in parallel; (7) Sending the test result training set into the constructed classification neural network, using the batch gradient descent method to iteratively train its network parameters to obtain a trained decision-level fusion network model; (8) The detection result test set is input into the trained decision-level fusion network to obtain the SAR image intelligent fusion target detection result.

2. The method according to claim 1, characterized in that: In (1a), the Pix2Pix network is trained using the SEN1_2 dataset by gradient descent, as follows: (1a1) Set the learning rate to 0.0002, and the momentum parameters to β1 = 0.5, β2 = 0.999; (1a2) The training batch size is set to 1, that is, each iteration is trained with one image as a group, and the network optimizes the parameters once every iteration; (1a3) Select the adaptive moment estimation algorithm Adam as the network optimization algorithm, and use this algorithm to calculate and correct the first-order moment and second-order moment of each round of training gradient to optimize the network parameters in a single iteration; (1a4) Set the maximum number of network iterations to 100, perform iterative training for 100 times, and obtain a trained network model.

3. The method according to claim 1, characterized in that: In (3), the YOLO-v3 target detection network is trained using the detection image training set using the adaptive moment estimation algorithm Adam, which is implemented as follows: (3a) The longest and widest side of the detection image is transformed to a size of 416, and then the short side is compressed at the same ratio. For the short side that does not reach the length of 416, gray pixels are filled at both ends of the short side to make its length reach 416, so as to achieve equal-proportion scaling of the detection image; (3b) Randomly flip and crop the scaled image. Set the training batch size to 2, that is, each iteration trains with 2 images as a group, and optimizes the parameters once per iteration; (3c) Set the initial learning rate to 0.001, and use the Adam algorithm to calculate and correct the first-order moment and second-order moment of the gradient in each round of training to optimize the network parameters in a single iteration; (3d) Set the maximum number of network iterations to 150, repeat (3a)-(3c) 150 times in total, and obtain the trained network model.

4. The method according to claim 1, characterized in that: In (5), the two-way YOLO target detection results are preprocessed as follows: (5a) Merge the two detection results of the target image: Each ship target in the two detection results is framed in the image with a rectangular target, whose coordinate value is (x1, y1, x2, y2). The intersection-over-union ratio (IOU) between the targets in the two results is calculated and compared with the set first threshold ω1=0.5: If IOU ≥ ω1, it is considered that both paths have detected the target, and the two-path detection results of the target are merged and saved, and (5b) is executed; If IOU < ω1, it is considered that the target is detected in only one path, the coordinate value of the other detection result is 0, and the result is saved; (5b) Generate target image classification label: Calculate the IOU of the ship target detected in the merged result and the ship target corresponding to the label in the test set of the detection result, and compare it with the set second threshold ω2 = 0.5: If IOU ≥ ω2, the correct target is considered to be detected, and a marker 1 representing the correct value is added after the merged detection result; If IOU < ω2, it is considered that the correct target is not detected, and a marker 0 representing the error value is added after the merged detection result.

5. The method according to claim 1, characterized in that: The convolutional neural network (CNN) model established in (6) includes 2 convolutional layers, 2 pooling layers, 1 fully connected layer and 1 dropout layer, and its structure is: First convolution layer -> first pooling layer -> second convolution layer -> second pooling layer -> fully connected layer -> dropout layer; The first convolution layer is the input layer, the number of input nodes is 60,000, the number of convolution kernels is 64, and the convolution kernel size is 1×5; The second convolution layer is the output layer, the number of output nodes is 50, the number of convolution kernels is 128, and the size of the convolution kernel is 1×3; Both pooling layers use downsampling, with a sampling size of 1×2; The dropout rate of this dropout layer is 0.7; The learning rate in each layer of the network is 0.01, and the weights W of the nodes in each layer are initialized using the [0,1] normal distribution. l and bias value b l The activation functions of the convolutional layer, pooling layer, and fully connected layer all use linear rectification functions, expressed as f(x)=max(0,x), where x is the independent variable and the x value is equal to the input value of the node in the layer where the activation function is located.

6. The method according to claim 1, characterized in that: The LSTM model constructed in (6) is composed of a layer of LSTM units, with 60,000 input nodes, 50 output nodes, and a weight W of the memory unit node. l and bias value b l Initialized using a [0,1] normal distribution.

7. The method according to claim 1, characterized in that: In (7), the batch gradient descent method is used to iteratively train the classification neural network parameters, which is implemented as follows: (7a) Set the data batch size to 64, divide the test result training set into multiple batches of data according to the batch size, and send these batches to the neural network for feature extraction; (7b) Adaptive moment estimation algorithm Adam is selected to calculate and correct the first-order moment and second-order moment of each round of training gradient using the features extracted by the network, so as to optimize the network parameters in a single iteration; (7c) Set the maximum number of network iterations to 150, repeat (7a) and (7b) 150 times in total, and obtain the trained network model.

Images