SAR Target Recognition Method Based on Sub-Aperture Guidance and Complex Convolutional Neural Network

Abstract

The present invention is a SAR target recognition method based on sub-aperture guidance for a complex convolutional neural network. First, the SAR image is decomposed into sub-aperture images. Secondly, a target recognition model is constructed based on the complex convolutional neural network, and the target recognition model is used for target recognition to obtain a preliminary classification result and calculate the recognition loss. Then, a reconstruction network is constructed, and each reconstruction layer of the reconstruction network includes two complex transposed convolutional operations. The deep features extracted by the target recognition model are input into the reconstruction network to obtain a reconstructed image. The sub-aperture image is used as a label to calculate the reconstruction loss. Finally, the feature extraction network of the target recognition model is trained, and the feature extraction network is optimized according to the recognition loss and the reconstruction loss to obtain an optimized target recognition model. This method uses image reconstruction as an auxiliary task, jointly guides network training with the auxiliary task and the classification task, and guides the network to learn through the sub-aperture image in the training stage, effectively improving the model's recognition ability for target separability features.

Description

Technical Field

[0001] The present invention belongs to the technical field of radar target recognition, and particularly relates to a SAR target recognition method based on a sub-aperture guided complex convolutional neural network. Background Art

[0002] In recent years, radar imaging technology has made rapid progress and has been widely used in many aspects such as military, agriculture, forestry, geology, ocean, disaster, surveying, etc. Synthetic Aperture Radar (SAR) is an active sensor that uses microwaves for sensing. Compared with other types of sensors such as infrared and optical sensors, SAR imaging is not restricted by conditions such as illumination and weather, and can observe targets of interest all-weather and all-day. Therefore, SAR has become an important means of earth observation and military reconnaissance at present, and target recognition based on SAR images has received more and more extensive attention.

[0003] In traditional SAR target recognition methods, methods based on template matching and methods based on models are widely used and the technology is relatively mature. The template matching method is a most basic pattern recognition method. It generates a large number of templates from targets in different images, and classifies the targets by matching the templates with the ROI (Region Of Interesting) region. However, the template matching method has a large computational redundancy and a slow running speed, and it is difficult to be applied to actual target recognition tasks. The model-based method extracts data features from the physical model or conceptual model of the target, and predicts the attributes of the target under different postures and configurations. The difficulty lies in the complexity of the physical model and the difficulty of implementation, which severely restricts the application of the model-based method in the field of SAR target recognition.

[0004] Compared with traditional algorithms that manually design features, deep learning methods can automatically extract features. Thanks to the increasing maturity of deep learning theory and technology, convolutional neural networks have become the dominant machine learning method in the field of SAR target recognition. Compared with traditional SAR target recognition methods, deep learning-based algorithms have the advantages of higher efficiency, higher accuracy, and stronger robustness. Although deep learning methods have achieved satisfactory results in SAR target recognition tasks, existing deep learning methods generally treat SAR images as grayscale images for processing, and directly use deep models to perform target recognition on SAR images. Since SAR images are complex data themselves and contain phase information compared with natural images, treating them as grayscale images will lose a large amount of target information and affect the improvement of the performance of the recognition task. Summary of the Invention

[0005] Aiming at the deficiencies of the prior art, the technical problem to be solved by the present invention is to propose a SAR target recognition method based on a sub-aperture guided complex convolutional neural network.

[0006] The technical solution adopted by the present invention to solve the above technical problems is as follows:

[0007] A method for SAR target recognition based on sub-aperture guidance using a complex convolutional neural network, comprising the following steps:

[0008] Step 1, decompose the SAR image to obtain sub-aperture images;

[0009] Step 2, construct a target recognition model based on a complex convolutional neural network. The target recognition model includes a feature extraction network and a complex fully connected layer located after the feature extraction network. The feature extraction network includes a pooling layer and four feature extraction layers connected in sequence. The pooling layer includes one complex convolution operation and one pooling operation. Each feature extraction layer includes two complex convolution operations. The results after the two complex convolution operations are added to the input to obtain the output of the feature extraction layer;

[0010] Input the SAR image into the feature extraction network to extract deep features from the SAR image; input the deep features into the complex fully connected layer to perform target recognition on the SAR image to obtain a preliminary classification result; calculate the recognition loss according to the actual class label of the SAR image and the preliminary classification result;

[0011] Step 3, construct a reconstruction network. The reconstruction network includes five reconstruction layers, and each reconstruction layer includes two complex transposed convolution operations; input the deep features extracted in Step 2 into the reconstruction network for image reconstruction to obtain a reconstructed image; use the sub-aperture image as a label and calculate the reconstruction loss according to the reconstructed image and the sub-aperture image;

[0012] Step 4, train the feature extraction network, and optimize and update the parameters of the feature extraction network according to the recognition loss and the reconstruction loss to obtain an optimized target recognition model; input the SAR image to be recognized into the optimized target recognition model for target recognition.

[0013] Further, the first step includes: First, perform a fast Fourier transform on the SAR image along the azimuth dimension to obtain the Doppler spectrum in the azimuth dimension; then, perform a windowing removal process on the Doppler spectrum and evenly divide the windowing-removed Doppler spectrum into multiple parts; finally, perform an inverse fast Fourier transform on each part of the Doppler spectrum to obtain sub-aperture images.

[0014] Further, the calculation formula for the recognition loss is:

[0015]

[0016]

[0017] where f j (x r+ix i ) represents the confidence that the preliminary classification result belongs to category j, x r 、x i Represent the real and imaginary parts of the preliminary classification results, y j is the actual category label of the SAR image, and N is the total number of categories;

[0018] The calculation formula of reconstruction loss is:

[0019]

[0020] Where m represents the number of sub-aperture images obtained by decomposing the SAR image, n represents the number of pixels in each image, and x represents the number of sub-aperture images obtained by decomposing the SAR image. k =x kR +ix kI represents the kth complex pixel value of the reconstructed image, x kR 、x kI Respectively represent the real and imaginary parts of the k-th complex pixel value of the reconstructed image, y k =y kR +iy kI is the kth complex pixel value of the subaperture image, y kR ,y kI represent the real and imaginary parts of the kth complex pixel value of the sub-aperture image respectively.

[0021] Compared with the prior art, the present invention has the following beneficial effects:

[0022] 1. High detection accuracy

[0023] Compared with the conventional deep learning SAR target recognition method, the present invention effectively utilizes the physical prior knowledge and complex information of SAR images. On the one hand, the feature extraction network is constructed by using complex convolution, which reasonably utilizes the complex information of SAR images to extract the target features of complex SAR images. On the other hand, the multi-task learning strategy is used to use image reconstruction as an auxiliary task, and the auxiliary tasks and classification tasks are used to jointly guide the feature extraction network for training. The sub-aperture images are used to guide the feature extraction network learning in the training stage, which effectively improves the model's ability to recognize the separable features of the target.

[0024] 2. Fully explore image information

[0025] Different from natural images, due to the special imaging characteristics of the SAR system, the physical essence of SAR image formation is the coherent superposition of electromagnetic vectors after the interaction between electromagnetic waves and the scene or target. It is composed of multiple sub-aperture images. The sub-aperture images contain rich ground object features and target electromagnetic scattering information, and reflect the scattering characteristics of the target from different angles. The specific target features of the same target may be different from different perspectives. When the type of the target cannot be recognized from a certain angle, there may be separable features of the target from other angles. Therefore, compared with the original SAR image, the sub-aperture image provides multi-angle target information and increases the possibility of distinguishing different types of targets. Different from conventional deep learning methods, the method of the present invention utilizes the complex components in the SAR image and the sub-aperture images obtained by decomposing the SAR image, and fully excavates the separable features of different targets from limited SAR data. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] Figure 1 is the overall flowchart of the method of the present invention;

[0027] Figure 2 is the schematic diagram of SAR image decomposition of the present invention;

[0028] Figure 3 is the structural diagram of the feature extraction network of the present invention;

[0029] Figure 4 is the schematic diagram of complex convolution operation of the present invention;

[0030] Figure 5 is the structural diagram of the reconstruction network of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0031] The technical solution of the present invention will be described in detail below in conjunction with the drawings and specific embodiments, and the protection scope of the present application is not limited thereto.

[0032] The present invention is a SAR target recognition method based on sub-aperture-guided complex convolutional neural network (referred to as the method, see Figures 1 to 5 ), which includes the following steps:

[0033] Step 1: Decompose the SAR image into multiple sub-aperture images; as Figure 2 shown, first perform a fast Fourier transform (FFT) on the SAR image along the azimuth dimension to obtain the Doppler spectrum in the azimuth dimension; then, perform windowing removal processing on the Doppler spectrum to eliminate the windowing effect during the imaging process, and evenly divide the windowing-removed Doppler spectrum into multiple parts; finally, perform an inverse fast Fourier transform (IFFT) on each part of the Doppler spectrum to obtain multiple sub-aperture images, and the sub-aperture images are single-channel images;

[0034] Step 2: Construct a target recognition model based on a complex convolutional neural network. The target recognition model includes a feature extraction network and a complex fully connected layer located after the feature extraction network. The feature extraction network is used to extract deep features of the SAR image, and the complex fully connected layer is used for classification; as Figure 3 shown, the feature extraction network includes a pooling layer and four feature extraction layers connected in sequence. The pooling layer includes one complex convolution operation and one pooling operation. Each feature extraction layer includes two complex convolution operations, and the results after the two complex convolution operations are added to the input to obtain the output of the feature extraction layer;

[0035] Figure 4 is a schematic diagram of the complex convolution operation, and the expression of the complex convolution operation is:

[0036]

[0037] In the formula, M = M R +iM I represents the complex feature, K = K R +iK I represents the complex convolution kernel, M R , K R respectively represent the real parts of the complex feature and the complex convolution kernel, M I , K I respectively represent the imaginary parts of the complex feature and the complex convolution kernel, i represents the imaginary number, and * represents the convolution operation;

[0038] Input the SAR image into the feature extraction network to extract deep features from the SAR image;

[0039] Input the deep features into the complex fully connected layer to perform target recognition on the SAR image to obtain a preliminary classification result; calculate the recognition loss according to the actual class label of the SAR image and the preliminary classification result. The recognition loss function is:

[0040]

[0041]

[0042] In the formula, f j (x r +ix i ) represents the confidence that the preliminary classification result belongs to class j, x r , x i respectively represent the real part and the imaginary part of the preliminary classification result, y j is the actual class label of the SAR image, and N is the total number of classes;

[0043] Step 3: Construct a reconstruction network, as Figure 5As shown in the figure, the reconstruction network includes five reconstruction layers, and each reconstruction layer includes two complex transposed convolutional operations; the deep features extracted in Step 2 are input into the reconstruction network for image reconstruction to obtain a reconstructed image; the sub-aperture image is used as a label, and the reconstruction loss is calculated based on the reconstructed image and the sub-aperture image. The reconstruction loss function is as follows:

[0044]

[0045] where m represents the number of sub-aperture images obtained by decomposing the SAR image. In this embodiment, m = 3; n represents the number of pixel points in each image, and x k = x kR + ix kI represents the k-th complex pixel value of the reconstructed image. x kR , x kI respectively represent the real part and the imaginary part of the k-th complex pixel value of the reconstructed image. y k = y kR + iy kI is the k-th complex pixel value of the sub-aperture image. y kR , y kI respectively represent the real part and the imaginary part of the k-th complex pixel value of the sub-aperture image;

[0046] Step 4: Use the gradient descent method to train the feature extraction network, and optimize and update the parameters of the feature extraction network according to the recognition loss function in Step 2 and the reconstruction loss function in Step 3 to obtain an optimized target recognition model. The optimized target recognition model is used for target recognition of SAR images.

[0047] Experimental verification: The hardware environment for the experiment is as follows: the graphics card is NVIDIA GeForce RTX 2080 Ti, the software environment is the Ubuntu 18.04 system, CUDA 10.2, and the Pytorch deep learning framework; the experimental dataset used is the MSTAR dataset, which includes ground vehicle targets of ten different target types, aspect angles, depression angles, and serial numbers, covering all directions in the range of 0 - 360°. The original data used in this experiment is the MSTAR data with complex components.

[0048] To verify the effectiveness of the method of the present invention, the method of the present invention is compared with common real convolutional neural networks, including ResNet18, ResNet10, VGG16, and Net4. Among them, Net4 is a lightweight network, including two convolutional layers and two fully connected layers; ResNet10 has a feature extraction network with the same architecture as the method of the present invention, but both are real convolutional operations and there is no reconstruction network. The specific experimental results are shown in Table 1.

[0049] Table 1 Comparison Experiment Results between the Method of the Present Invention and Real-Valued Convolutional Neural Networks

[0050]

[0051] As can be seen from Table 1, compared with the four real-valued convolutional neural networks, the method of the present invention has the highest accuracy. On the one hand, for SAR images, the present invention applies the SAR target recognition method based on the sub-aperture-guided complex convolutional neural network for target recognition. The experimental results show that the recognition method of the present invention has good performance and significantly higher accuracy compared with the existing deep learning methods. On the one hand, SAR images are originally complex data. The deep learning method based on real-valued convolutional neural networks directly takes the modulus value of SAR images, converts them into real-valued images for recognition, losing the available effective information and resulting in a decrease in accuracy. On the other hand, the present invention makes full use of the target information contained in sub-aperture images, increases the discrimination between different categories of targets, and improves the classification accuracy.

[0052] Table 2 Comparison Experiment Results between the Method of the Present Invention and Deep Learning Methods under the Condition of Limited Samples

[0053]

[0054] Table 2 shows the comparison results between the method of the present invention and deep learning methods under the condition of limited samples. It can be seen from Table 2 that the accuracy of the method of the present invention is significantly higher than that of the other methods. This is because the method of the present invention uses the multi-task learning strategy, takes image reconstruction as an auxiliary task, jointly guides the feature extraction network for training through the classification task and the auxiliary task, and guides the learning of the network through sub-aperture images in the training stage, effectively improving the model's recognition ability for target discriminative features. The experimental results under the condition of limited samples can better show that the method of the present invention is applicable to the situation where SAR lacks a large number of labeled samples, more in line with the original intention of real-time SAR target recognition, and has good application prospects.

[0055] Table 3 Comparison Experiment Results between the Method of the Present Invention and the Recognition Method Based on Complex Convolutional Neural Networks

[0056]

[0057] The method of the present invention is compared with the recognition method based on complex convolutional neural networks. The experimental results are shown in Table 3. The method of the present invention has the highest recognition accuracy, further verifying that the present invention makes full use of the target information contained in sub-aperture images, increases the discrimination between different categories of targets, and improves the classification accuracy.

[0058] Matters not described in the present invention are applicable to the prior art.

Claims

1. A SAR target recognition method based on sub-aperture guidance for a complex convolutional neural network, characterized in that, The method includes the following steps: Step 1: Decompose the SAR image to obtain sub-aperture images; Step 2: Construct a target recognition model based on a complex convolutional neural network. The target recognition model includes a feature extraction network and a complex fully-connected layer located after the feature extraction network. The feature extraction network includes a pooling layer and four feature extraction layers connected in sequence. The pooling layer includes one complex convolution operation and one pooling operation. Each feature extraction layer includes two complex convolution operations. The results after the two complex convolution operations are added to the input to obtain the output of the feature extraction layer; Input the SAR image into the feature extraction network to extract deep features from the SAR image; input the deep features into the complex fully-connected layer to perform target recognition on the SAR image to obtain a preliminary classification result; calculate the recognition loss according to the actual class label and the preliminary classification result of the SAR image; Step 3: Construct a reconstruction network. The reconstruction network includes five reconstruction layers. Each reconstruction layer includes two complex transposed convolution operations; input the deep features extracted in Step 2 into the reconstruction network for image reconstruction to obtain a reconstructed image; use the sub-aperture image as the label and calculate the reconstruction loss according to the reconstructed image and the sub-aperture image; Step 4: Train the feature extraction network and optimize and update the parameters of the feature extraction network according to the recognition loss and the reconstruction loss to obtain an optimized target recognition model; input the SAR image to be recognized into the optimized target recognition model for target recognition.

2. The SAR target recognition method based on sub-aperture guidance of a complex convolutional neural network according to claim 1, wherein Step 1 includes: First, perform a fast Fourier transform on the SAR image along the azimuth dimension to obtain the Doppler spectrum in the azimuth dimension; then, perform windowing processing on the Doppler spectrum and divide the windowed Doppler spectrum into multiple parts; finally, perform an inverse fast Fourier transform on each part of the Doppler spectrum to obtain sub-aperture images.

3. The method for SAR target recognition based on sub-aperture guidance of a complex convolutional neural network according to claim 1 or 2, characterized in that, The calculation formula for the recognition loss is: where f j (x r +ix i ) represents the confidence that the preliminary classification result belongs to class j, x r , x i represent the real part and the imaginary part of the preliminary classification result respectively, y j is the actual class label of the SAR image, and N is the total number of classes; The calculation formula for the reconstruction loss is: Among them, m represents the number of sub-aperture images obtained by SAR image decomposition, n represents the number of pixel points in each image, and x k = x kR + ix kI represents the k-th complex pixel value of the reconstructed image, and x kR , x kI represent the real part and the imaginary part of the k-th complex pixel value of the reconstructed image respectively. y k = y kR + iy kI is the k-th complex pixel value of the sub-aperture image, and y kR , y kI represent the real part and the imaginary part of the k-th complex pixel value of the sub-aperture image respectively.

Images