A SAR image target recognition method based on fully convolutional lightweight neural network

Abstract

The present invention relates to the field of radar image processing, and more specifically to a method for SAR image target recognition based on a fully convolutional lightweight neural network. This method constructs a fully convolutional lightweight neural network model based on an attention mechanism and performs fine-grained model saturation quantization compression on the network model. This allows the network model to be embedded in edge devices such as FPGAs, meeting real-time and low-storage requirements while maintaining high recognition accuracy for SAR images.

Description

Technical Field

[0001] The present invention relates to the field of radar image processing, and in particular to a SAR image target recognition method based on a fully convolutional lightweight neural network. Background Art

[0002] Synthetic aperture radar (SAR), a high-resolution imaging radar, can operate continuously over a wide range under various weather conditions, producing high-resolution radar images similar to those obtained with optical photography. In the military, SAR is widely used due to its exceptional ability to detect targets such as aircraft, tanks, and ships. As a key branch of SAR image interpretation, rapid, real-time automatic recognition of targets in SAR images has garnered increasing attention.

[0003] Since the advent of AlexNet in the field of deep learning, an increasing number of neural networks have achieved extremely high accuracy in image classification tasks, leading to rapid progress in the field of automatic target recognition in SAR images. However, the higher the performance of a neural network, the greater the computational workload and storage requirements. Many edge devices lack these resources, leading to an increasing emphasis on building lightweight networks or model compression. Typical lightweight networks, such as the MobileNet series and the ShuffleNet series, have achieved impressive lightweighting results. Model compression techniques can be broadly categorized into pruning, quantization, and knowledge distillation.

[0004] Model compression technology can be used to further reduce the size of the model based on the constructed lightweight network, so that the model can meet the needs of small edge devices with limited storage and computing power. Therefore, the present invention proposes a SAR image target recognition method based on a fully convolutional lightweight neural network. Summary of the Invention

[0005] In view of the problems existing in the prior art, the object of the present invention is to provide a SAR image target recognition method based on a fully convolutional lightweight neural network.

[0006] In order to achieve the above objectives, the present invention adopts the following technical solutions to achieve them.

[0007] A SAR image target recognition method based on a fully convolutional lightweight neural network includes the following steps:

[0008] Step 1: Obtain a SAR image set, treat each SAR image and its corresponding category as a set of data, and obtain a sample set containing multiple sets of data; randomly divide the sample set into a training set and a test set, and perform normalization on all samples;

[0009] Step 2: Build a fully convolutional lightweight neural network model ψ based on the attention mechanism;

[0010] Step 3: Train the full convolution lightweight neural network model ψ based on the attention mechanism to obtain the trained full convolution lightweight neural network model ψ' based on the attention mechanism;

[0011] Step 4: Perform fine-grained model saturation quantization compression processing on the trained attention-based fully convolutional lightweight neural network model ψ';

[0012] Step 5: Use the processed full convolution lightweight neural network model based on the attention mechanism to identify the SAR image target and obtain the target type of the SAR image target.

[0013] Compared with the existing technology, the beneficial effects of the present invention are: constructing a fully convolutional lightweight neural network model based on the attention mechanism, and performing fine-grained model saturation quantization compression processing on the network model, so that the network model can be embedded in edge devices such as FPGA, meeting the real-time and low storage requirements while maintaining a high recognition accuracy for SAR images. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] The present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0015] Figure 1 is a flow chart of the method of the present invention;

[0016] Figure 2 This is a framework diagram of a fully convolutional lightweight neural network based on an attention mechanism according to the method of the present invention;

[0017] Figure 3 Schematic diagram of the structure of the depth-separable convolution module of the method of the present invention;

[0018] Figure 4 Schematic diagram of the structure of the SE attention mechanism module of the method of the present invention;

[0019] Figure 5 This is a heat map of the recognition and classification of test samples by a trained network model without model saturation quantization compression processing;

[0020] Figure 6 This is a heat map of the recognition and classification of test samples after the network model has been trained once and has been subjected to model saturation quantization compression processing. DETAILED DESCRIPTION

[0021] The embodiments of the present invention will be described in detail below with reference to examples. However, those skilled in the art will understand that the following examples are only used to illustrate the present invention and should not be construed as limiting the scope of the present invention.

[0022] refer to Figure 1 , a SAR image target recognition method based on a fully convolutional lightweight neural network, comprising the following steps:

[0023] Step 1: Obtain a SAR image set, treat each SAR image and its corresponding category as a set of data, and obtain a sample set containing multiple sets of data; randomly divide the sample set into a training set and a test set, and perform normalization on all samples;

[0024] Step 2: Build a fully convolutional lightweight neural network model ψ based on the attention mechanism;

[0025] Specifically, refer to Figure 2 The fully convolutional lightweight neural network model ψ based on the attention mechanism includes a first depth-wise separable convolution module, a second depth-wise separable convolution module, a first maximum pooling layer, a third depth-wise separable convolution module, a first SE attention mechanism module, a fourth depth-wise separable convolution module, a second SE attention mechanism module, a fifth depth-wise separable convolution module, a second maximum pooling layer, a sixth depth-wise separable convolution module, a seventh depth-wise separable convolution module, an eighth depth-wise separable convolution module, a third maximum pooling layer, a ninth depth-wise separable convolution module, a fourth maximum pooling layer, a tenth depth-wise separable convolution module, an eleventh depth-wise separable convolution module, and a global average pooling layer, which are cascaded in sequence.

[0026] refer to Figure 3 , all depth-wise separable convolution modules contain sequentially cascaded depth-wise separable convolution layers, BN layers, and ReLU activation function layers;

[0027] The depthwise separable convolution layer mainly consists of two parts: channel-by-channel convolution and point-by-point convolution. In the channel-by-channel convolution, one convolution kernel is responsible for one channel, and one channel is convolved by only one convolution kernel. The number of channels of the feature map generated by this process is exactly the same as the number of channels of the input. Point-by-point convolution is very similar to conventional convolution. Its convolution kernel size is 1×1×M, where M is the number of channels of the previous layer. The point-by-point convolution operation will perform a weighted combination of the feature maps generated by the channel-by-channel convolution in the depth direction to generate a new feature map. The number of channels of the new feature map is the number of convolution kernels of the point-by-point convolution. Compared with conventional convolution, the computational complexity of depthwise separable convolution is only 1 / 3 of that of conventional convolution.

[0028] refer to Figure 4The input of the SE attention mechanism module is a feature map of H×W×C, where H and W are the height and width of the feature map respectively, and C is the channel of the feature map; the H×W×C feature map is first subjected to a global pooling operation to obtain a 1×1×C feature map, and then the 1×1×C feature map is reduced to 1 / 4 of the original number of channels C by a 1×1 convolution kernel. After the Relu activation function, the feature map is further increased to the original number of channels C by a 1×1 convolution kernel, and then activated by the h_sigmoid activation function to obtain a 1×1×C feature map. This 1×1×C feature map is matrix-multiplied with the input H×W×C feature map to obtain a new feature map of different channel importance and then input it into the next layer.

[0029] The input of the fully convolutional lightweight neural network model ψ based on the attention mechanism is the SAR image, and the output is the category of the target in the SAR image.

[0030] Step 3: Train the full convolution lightweight neural network model ψ based on the attention mechanism to obtain the trained full convolution lightweight neural network model ψ' based on the attention mechanism;

[0031] The sub-steps of step 3 are as follows:

[0032] In sub-step 3.1, input the training set data into the fully convolutional lightweight neural network model ψ based on the attention mechanism and calculate the loss function of the network output layer. The calculation formula is as follows:

[0033]

[0034] Where yi is the true label of the sample, is the predicted label of the sample;

[0035] In sub-step 3.2, the network is trained using the back-propagation algorithm and the Adam optimizer, and the cosine learning rate decay strategy is used to update the parameters of each layer in the network.

[0036] In sub-step 3.3, repeat sub-step 3.2 and iterate to continuously update the parameters until the loss function converges, and obtain the trained full convolutional lightweight neural network model ψ' based on the attention mechanism.

[0037] Step 4: Perform fine-grained model saturation quantization compression processing on the trained attention-based fully convolutional lightweight neural network model ψ';

[0038] Based on the fine-grained model saturation quantization compression method, floating-point calculations are converted to fixed-point calculations to accelerate forward reasoning, and floating-point type storage of weights is converted to fixed-point type storage to compress model storage space;

[0039] The sub-steps of step 4 are as follows:

[0040] In sub-step 4.1, randomly select a portion of data from the test set as the calibration set. Run forward inference on each SAR image in the calibration set and calculate the distribution histogram of each channel data in each layer, where each distribution histogram has 2048 bins.

[0041] In sub-step 4.2, the original distribution histogram is truncated from the 128th bin to the last bin, and the KL divergence between the original distribution histogram and the truncated distribution histogram is calculated. The calculation formula of KL divergence is as follows:

[0042]

[0043] In the formula, p is the original distribution, q is the truncated distribution, and the information entropy is Cross Entropy

[0044] Let the bin with the smallest KL divergence be the optimal threshold T; each channel data in each layer needs to find an optimal threshold T, and the process of quantizing with the optimal threshold T is called saturation quantization;

[0045] In sub-step 4.3, use the optimal threshold T to calculate the quantization factor scale_X of the activation value and store it in advance as a quantization factor table for direct call in subsequent forward reasoning;

[0046] In sub-step 4.4, the channel weight data is quantized in advance using the absolute maximum value in each channel of each layer in the weight data, i.e., non-saturated quantization, and the quantized fixed-point weight data is stored. The size of the weight file after storage is 1 / 4 of the original size. The quantization formula is:

[0047]

[0048] Where W is a floating-point weight and QW is a fixed-point weight with an int8 bit width.

[0049] In sub-step 4.5, use the quantization factor scale_X to perform saturation quantization on the activation value data in real time during forward inference. The quantization formula is:

[0050] X=scake_X×QX

[0051] Where X is a floating-point activation value and QX is a fixed-point activation value;

[0052] In substep 4.6, during forward reasoning, the pre-stored fixed-point weights and their quantization factors are read, and the activation values ​​are saturated quantized in real time. After each layer of quantization is completed, the fixed-point weights are multiplied by the fixed-point activation values, and then dequantized into floating-point data using the quantization factor before entering the next layer. This is repeated for each layer. The calculations during forward reasoning are converted from floating-point calculations to 8-bit fixed-point calculations, achieving a computational acceleration effect.

[0053] In sub-step 4.7, to further reduce the amount of computation during forward reasoning and improve the speed of forward reasoning, we merge the BN layer with the convolutional layer preceding it during forward reasoning.

[0054] Specifically, the calculation formula of the ordinary convolution layer is: conv =ω·x+b;

[0055] In ordinary BN layer calculations, it is necessary to calculate the mean variance of the elements in a minibatch, then for x, it is necessary to subtract the mean and divide it by the standard deviation, and finally use the two parameters γ and β to perform an affine transformation. The specific formula is as follows:

[0056]

[0057] Among them, μ β is the mean of the elements in a minibatch;

[0058]

[0059] in, is the variance of elements in a minibatch;

[0060]

[0061] Among them, ∈ is a very small constant to prevent the denominator from being 0;

[0062]

[0063] The BN fusion process is as follows:

[0064]

[0065] make Then we have:

[0066]

[0067] This completes the computational fusion of the BN layer and the convolutional layer in forward reasoning, achieving the effect of reducing the amount of calculation and accelerating reasoning.

[0068] Step 5: Use the processed full convolution lightweight neural network model based on the attention mechanism to identify the SAR image target and obtain the target type of the SAR image target.

[0069] Simulation test

[0070] The simulation experiment was run on an NVIDIA GTX 1080 GPU and was performed using the Tensorflow framework in the Pycharm compiler.

[0071] The SAR image set is the MSTAR End of Observation (EOC) dataset, which includes three categories: BTR70, BMP2, and T70. The training set contains 698 SAR images and corresponding category labels, and the test set contains 1,365 SAR images and corresponding category labels. The target area to be identified in each SAR image is located at the exact center of the image. A fully convolutional lightweight neural network model based on the attention mechanism was trained seven times using the training set. The following simulations were performed using each of these seven trained models.

[0072] Simulation 1: The fully convolutional lightweight neural network model based on the attention mechanism is not subjected to model saturation quantization compression processing and directly recognizes the test set.

[0073] Simulation 2: The fully convolutional lightweight neural network model based on the attention mechanism undergoes model saturation quantization compression processing and then recognizes the test set.

[0074] refer to Figure 5 、 Figure 6 ,It can be seen that although the model saturation quantization compression processing will bring a certain loss of recognition accuracy to the network model, the loss of recognition accuracy is very small.

[0075] The network model weight storage size and average recognition accuracy of simulation 1 and simulation 2 under 7 training cycles are shown in the following table.

[0076] Network model for simulation 1 Network model for Simulation 2 Weight storage size 802KB 201KB Average recognition accuracy 97.75％ 97.30％

[0077] As can be seen from the table above, the weight file storage size of the fully convolutional lightweight network model based on the attention mechanism is further compressed to 1 / 4 after model saturation quantization compression processing. At the same time, the average recognition accuracy loss is less than 0.5%, remaining above 97%. This shows that the method of the present invention can maintain high recognition accuracy for SAR images while meeting the storage space requirements of edge devices such as FPGAs.

[0078] Although this specification has provided a detailed description of the present invention using general descriptions and specific embodiments, it will be apparent to those skilled in the art that modifications and improvements may be made based on the present invention. Therefore, such modifications and improvements, which do not depart from the spirit of the present invention, are intended to be within the scope of protection claimed herein.

Claims

1. A SAR image target recognition method based on a fully convolutional lightweight neural network, characterized in that: The following steps are involved: Step 1: Obtain a SAR image set, treat each SAR image and its corresponding category as a set of data, and obtain a sample set containing multiple sets of data; randomly divide the sample set into a training set and a test set, and perform normalization on all samples; Step 2: Build a fully convolutional lightweight neural network model ψ based on the attention mechanism; Step 3: Train the full convolution lightweight neural network model ψ based on the attention mechanism to obtain the trained full convolution lightweight neural network model ψ' based on the attention mechanism; Step 4: Perform fine-grained model saturation quantization compression processing on the trained attention-based fully convolutional lightweight neural network model ψ'; The sub-steps of step 4 are as follows: In sub-step 4.1, randomly select a portion of data from the test set as the calibration set. Run forward inference on each SAR image in the calibration set and calculate the distribution histogram of each channel data in each layer, where each distribution histogram has 2048 bins. In sub-step 4.2, the original distribution histogram is truncated from the 128th bin to the last bin, and the KL divergence between the original distribution histogram and the truncated distribution histogram is calculated. The calculation formula of KL divergence is as follows: In the formula, p is the original distribution, q is the truncated distribution, and the information entropy is Cross Entropy Let the Bin with the smallest KL divergence be the optimal threshold T; an optimal threshold T needs to be found for each channel data in each layer; Sub-step 4.3: Calculate the quantization factor scale_X of the activation value using the optimal threshold T and store it in advance as a quantization factor table; In sub-step 4.4, the channel weight data is quantized in advance using the absolute maximum value of each channel in each layer of the weight data, i.e., non-saturated quantization, and the quantized fixed-point weight data is stored; the quantization formula is: Where W is a floating-point weight and QW is a fixed-point weight with an int8 bit width. In sub-step 4.5, use the quantization factor scale_X to perform saturation quantization on the activation value data in real time during forward inference. The quantization formula is: X=scale_X×QX Where X is a floating-point activation value and QX is a fixed-point activation value; Sub-step 4.6: During forward inference, read the pre-stored fixed-point weights and their quantization factors, and perform saturation quantization on the activation values ​​in real time. After each layer of quantization is completed, the fixed-point weights are multiplied by the fixed-point activation values, and then dequantized into floating-point data using the quantization factors before entering the next layer. In sub-step 4.7, the BN layer is combined with the convolution layer preceding the BN layer during forward reasoning. In step 5, the processed attention-based fully convolutional lightweight neural network model is used to identify the SAR image target and obtain the target type of the SAR image target.

2. The SAR image target recognition method based on a fully convolutional lightweight neural network according to claim 1 is characterized in that: The fully convolutional lightweight neural network model ψ based on the attention mechanism includes a first depth-wise separable convolution module, a second depth-wise separable convolution module, a first maximum pooling layer, a third depth-wise separable convolution module, a first SE attention mechanism module, a fourth depth-wise separable convolution module, a second SE attention mechanism module, a fifth depth-wise separable convolution module, a second maximum pooling layer, a sixth depth-wise separable convolution module, a seventh depth-wise separable convolution module, an eighth depth-wise separable convolution module, a third maximum pooling layer, a ninth depth-wise separable convolution module, a fourth maximum pooling layer, a tenth depth-wise separable convolution module, an eleventh depth-wise separable convolution module, and a global average pooling layer, wherein all depth-wise separable convolution modules include a depth-wise separable convolution layer, a BN layer, and a Relu activation function layer, which are cascaded in sequence. The input of the fully convolutional lightweight neural network model ψ based on the attention mechanism is the SAR image, and the output is the category of the target in the SAR image.

3. The SAR image target recognition method based on a fully convolutional lightweight neural network according to claim 1 is characterized in that: The sub-steps of step 3 are as follows: In sub-step 3.1, input the training set data into the fully convolutional lightweight neural network model ψ based on the attention mechanism and calculate the loss function of the network output layer. The calculation formula is as follows: Where y i is the true label of the sample, is the predicted label of the sample; In sub-step 3.2, the network is trained using the back-propagation algorithm and the Adam optimizer, and the cosine learning rate decay strategy is used to update the parameters of each layer in the network. Sub-step 3.3, repeat sub-step 3.2 until the loss function converges, and obtain the trained full convolutional lightweight neural network model ψ' based on the attention mechanism.

4. The SAR image target recognition method based on a fully convolutional lightweight neural network according to claim 1 is characterized in that: Specifically, the BN fusion process is as follows: make Then we have:

Images