Remote Sensing SAR Target Detection Method Based on Network Pruning and Parameter Quantization

Abstract

The present invention discloses a remote sensing SAR target detection method based on the combination of network pruning and parameter quantization, which mainly solves the defects of high model complexity and slow reasoning speed of existing remote sensing SAR target detection methods. Its implementation scheme is: obtain the divided training set and test set from the public remote sensing SAR target detection data set, and perform data expansion on the training set; perform data enhancement on the expanded training set; adjust the existing lightweight network to build a benchmark model; train the benchmark model and calculate its performance index; evaluate the importance of each feature extraction module in the adjusted network, prune unimportant modules, and then perform filter pruning; design a quantization search space, search for a mixed precision quantization scheme of the pruned model, obtain the final model, and use the model for SAR target detection. The present invention saves training costs while improving detection accuracy, and can be used in target recognition scenarios with limited computing resources and high real-time requirements.

Description

Technical Field

[0001] The present invention belongs to the technical field of computer vision, and in particular is a remote sensing SAR target detection method, which can be used for SAR image recognition scenarios with limited computing resources and high real-time requirements. Background Art

[0002] Synthetic aperture radar (SAR) is one of the important means of earth observation. It is widely used in civil fields such as marine rescue and marine law enforcement, as well as military fields such as real-time marine monitoring and detection. SAR image data is usually a single-channel grayscale image, which has the characteristics of many small targets, unclear target features, uneven target distribution, and high similarity between targets and backgrounds. In recent years, convolutional neural networks (CNNs) have been widely used in SAR image target detection. Compared with traditional detection algorithms that require cumbersome feature design and modeling, CNN-based methods have shown better performance than traditional methods due to their ability to autonomously learn parameters and automatically extract features.

[0003] Currently, CNN-based target detection algorithms are mainly divided into two categories. The first category is the two-stage target detection algorithm represented by the RCNN (Region-CNN) series, which first generates candidate regions on the bounding box containing the target and then performs target detection. This type of method has high detection accuracy but slow reasoning speed, and it is difficult to cope with scenes with high real-time requirements. The second category is the single-stage target detection algorithm represented by SSD (Single Shot MultiBox Detector) and YOLOv3 (You Only Look Once). This type of algorithm does not generate candidate regions, but directly performs target detection through regression. It has a small amount of calculation and fast reasoning speed. It performs well in resource-constrained environments, but the detection accuracy of this type of method is usually not as good as the two-stage method.

[0004] Convolutional neural network (CNN) is the core part of the target detection algorithm and is used to extract image features. In order to pursue higher accuracy, researchers usually design complex networks with a large number of redundant parameters. Such a network structure consumes a lot of computing resources and storage resources both in the training stage and in the inference stage. Therefore, in practical applications, it is necessary to balance accuracy and resource consumption. The main method of balancing is to compress the designed CNN network. Although this compression will cause some accuracy loss, it can greatly reduce the redundant parameters of the model, and while improving the inference speed, it can reduce the complexity and resource consumption of the model, thereby achieving efficient SAR target detection.

[0005] The most commonly used method for model compression is model pruning. According to the rule of pruning granularity from fine to coarse, pruning can be divided into weight pruning, channel / filter pruning and layer pruning. Among them:

[0006] Weight pruning, also known as unstructured pruning, is based on the idea of ​​sparseening the original weight matrix and reducing the amount of computation during inference by reducing the connections between neurons. This method can reduce the amount of computation without modifying the network structure, but improving the inference speed requires dedicated hardware that implements sparse convolution.

[0007] Channel / filter pruning and layer pruning, also known as structured pruning, change the original network structure and can easily improve the inference speed, but are usually accompanied by a loss of accuracy. Zhou Zhou et al. proposed using Lasso regression and least squares method to perform channel pruning on the two-stage model Faster-RCNN in the paper "Zhou Zhou, Wang Haipeng, Xu Feng, Zhang Zhijun, Wang Zhicheng, SAR image ship detection optimization algorithm based on channel pruning. AEROSPACESHANGHAI (CHINESE & ENGLISH) Vol. 37, No. 4, 2020", which reduced 56% of the model parameters. However, due to the complex structure of the original network, the inference speed is still not ideal.

[0008] Traditional pruning schemes are mostly carried out along the following lines: 1) Train a large, over-parameterized network to obtain a baseline model; 2) Prune the baseline model based on a certain strategy; 3) Fine-tune the pruned model to obtain a pruned CNN model. Lin et al. found in the paper "Mingbao Lin, Rongrong Ji, Yan Wang, Yichen Zhang, Baochang Zhang, Yonghong Tian, ​​Ling Shao. HRank: Filter Pruning using High-Rank Feature Map. In CVPR 2020." that the average rank of multiple feature maps generated by a single filter is always the same, so they proposed a method for pruning filters according to the size of the feature map rank. Although this method does not require fine-tuning training, the model compression rate cannot reach a high level because only filter pruning is performed.

[0009] Most current pruning schemes have absorbed the ideas of neural network architecture search (NAS) and knowledge distillation, and use complex iterative training to find better structured pruning schemes. In the patent document "SAR ship target detection method based on network pruning and knowledge distillation" (patent application number: CN202011308276.8, publication number: CN112308019A) applied by the National University of Defense Technology of the People's Liberation Army of China, a single-stage YOLOv3 detector was used as the reference detection framework. After channel pruning of the network, knowledge distillation guided by the relationship between feature maps was used to restore performance. However, the use of NAS methods requires a lot of GPU support and is usually only suitable for research use.

[0010] Another common method of model compression is parameter quantization. Han Song and others used shared weights to quantize network parameters for the first time in deep compression, compressing the model by 2 to 3 times. Parameter quantization is a technology that replaces the default 32-bit floating-point calculation with low-bit fixed-point calculation, and is mostly used to save storage resources when the model is deployed on hardware. Although quantization can effectively reduce the model size, number of parameters and computing intensity, it often brings huge precision loss, especially when it is less than 4 bits or even 1 bit, the precision challenge is even greater.

[0011] Traditional quantization methods usually store model parameters uniformly with fixed bits. However, due to the different feature extraction capabilities of different depth layers of the network, fixed-bit quantization cannot adapt to the characteristics of the network structure, which may cause important information loss and make it difficult to achieve the optimal compression ratio. Using mixed precision quantization instead of fixed-bit quantization is a commonly used quantization method. It uses different quantization bits to represent parameters at different depth layers of the network. Due to the complex and changeable network structure, in order to find a better mixed precision quantization strategy, complex methods such as grid search or reinforcement learning are usually used, but these methods require a lot of computing resources and time costs.

[0012] In summary, the existing pruning and quantization methods have three shortcomings: 1) Both methods only compress one aspect of the model and do not combine the advantages of multiple compression methods; 2) The implementation project is too complicated and requires a lot of computing resources; 3) There is no special design for remote sensing SAR image target detection tasks. These shortcomings make it difficult to achieve a high level of network model compression rate, and high-speed SAR target detection cannot be achieved in a resource-constrained environment. Summary of the invention

[0013] The purpose of the present invention is to address the deficiencies of the above-mentioned prior art and propose a remote sensing SAR target detection method based on the combination of network pruning and parameter quantization to reduce the consumption of computing and storage resources, improve the compression rate of the network model, and improve the detection speed of SAR targets in a resource-constrained environment.

[0014] To achieve the above object, the technical solution of the present invention includes the following:

[0015] (1) Obtain the divided training set D from the public remote sensing SAR target detection dataset train and the test set D test , and use geometric transformation to expand the images in the training set to obtain the expanded training set D exp ;

[0016] (2) After the expansion, the training set D expThe proportions of target images of different sizes and images with a large amount of background are counted in the training set, and the images with smaller proportions are selected as difficult samples in the expanded training set. The samples are then subjected to offline data enhancement and online data enhancement in turn to obtain the enhanced training set D aug ;

[0017] (3) Building a SAR target detection benchmark model:

[0018] (3a) The existing lightweight network structure is adjusted, that is, the original classification layer is changed into a detection layer, and some downsampling layers in the deep layer of the network are modified into non-downsampling layers, and the adjusted lightweight network is used as a feature extraction backbone network N, which includes an input layer, a hidden layer and a detection layer. The hidden layer is composed of multiple feature extraction modules with the same structure, each feature extraction module includes a possible downsampling layer and multiple non-downsampling layers, the downsampling layer is composed of a convolutional layer, a pooling layer or a rearrangement layer, and each non-downsampling layer is composed of a convolutional layer, a BN layer, an activation layer or a connection layer;

[0019] (3b) Use K-Means clustering algorithm to cluster the expanded training set D exp The labeled target boxes are clustered to obtain the prior anchor box for this data set, and this anchor box is used as the anchor box of the existing YOLO series single-stage detector to obtain the SAR target detector. The detector is connected to the backbone network N to form a benchmark model for SAR target detection.

[0020] (4) Update the network weight parameters of the baseline model:

[0021] (4a) The YOLO loss function is used as the loss function of the baseline model, and the weight parameters of the backbone network N are initialized using the random number seed S;

[0022] (4b) The enhanced training set D aug Input into the initialized benchmark model to start training, use the momentum stochastic gradient descent algorithm SGD to optimize the YOLO loss function to update the network weight parameters, save the network weight parameters every 10 cycles, and stop training when the set maximum number of iterations is reached to obtain multiple updated network weight parameters;

[0023] (5) Performance indicators for evaluating the baseline model:

[0024] (5a) Use the saved multiple network weight parameters to update the baseline model on the test set D test Calculate the F1 score of each updated benchmark model, take the maximum value of the F1 score as F10, and the network weight parameter corresponding to F10 is represented by W0;

[0025] (5b) Calculate the number of parameters P and floating point operations FLOPs of W0, and record the calculation results as P0 and FLOPs0 respectively;

[0026] (6) Rough pruning of the backbone network N:

[0027] (6a) Setting a module mask m for the n feature extraction modules in the backbone network N, and using the module mask m to perform One Hot encoding on all feature extraction modules of the network to obtain n masked subnetworks;

[0028] (6b) All n mask subnetworks share the weight parameter W0, fine-tune each mask subnetwork and calculate its performance index F1 score F1 i , parameter quantity P i and FLOPs i ;

[0029] (6c) According to the performance index of each mask subnet, the importance index I of each feature extraction module in the backbone network N is calculated, and these importance indexes I are compared with the set importance index threshold I thr For comparison, when I is less than I thr When , the feature extraction module corresponding to the importance index I in the backbone network N is pruned to obtain the backbone network N after rough pruning m ;

[0030] (7) After rough pruning, the backbone network N m To perform fine pruning:

[0031] (7a) Evaluation of the backbone network N after rough pruning based on the Hrank method m The importance of filters in the convolutional layer, and the important filters and unimportant filters are divided. In the backbone network N after rough pruning m Based on this, unimportant filters are cut off to obtain a finely pruned network N P ;

[0032] (7b) Use the same random number seed S to initialize the finely pruned network N P The network weight parameters are then trained using the same training method as in (4) to fine-tune the pruned network N P Perform training and save the network weight parameter with the highest F1 score, denoted as W P ;

[0033] (8) After fine pruning, the network N P Perform parameter quantization:

[0034] (8a) The finely pruned network N P The feature extraction module in is changed to a quantization module;

[0035] (8b) Selecting quantization strategies of different precisions for weight parameters and activation outputs, and designing a quantization search space based on the importance index I of the feature extraction module;

[0036] (8c) In the enhanced training set D aug Iterate and search for the best quantization scheme, and apply the best quantization scheme to the finely pruned network N P In the final network N Q ;

[0037] (8d) Use W P As the final network N Q The pre-trained weight parameters are fine-tuned to obtain the final network weight parameters W Q ;

[0038] (9) Use the final network N Q Replace the backbone network N of the baseline model and use the final network weight parameter W Q Update the network weight parameters to obtain the final SAR target detection model;

[0039] (10) The test set D test Input it into the final SAR target detection model to obtain accurate SAR target detection results.

[0040] Compared with the prior art, the present invention has the following advantages:

[0041] 1. The present invention selects a faster single-stage target detection algorithm and further implements a compression scheme of module pruning, filter pruning and mixed precision quantization. Compared with the existing compression method using a two-stage detection algorithm plus channel pruning and the method of directly using a single-stage detection algorithm, the compression rate of the model is greatly improved, and SAR target detection can be realized on more marginal hardware devices;

[0042] 2. Aiming at the data characteristics of remote sensing SAR targets, the present invention uses a variety of data enhancement methods to enhance the original training set data, thereby improving the richness of the original data, and constructs a high-performance benchmark model by reducing the sampling rate of the backbone network, thereby improving the accuracy of SAR target recognition;

[0043] 3. The present invention performs coarse-grained module pruning by analyzing the importance of modules, and performs fine-grained filter pruning based on the HRank method, which reduces the training cost compared to the existing model compression methods using reinforcement learning and knowledge distillation;

[0044] 4. The present invention uses a method that combines random initialization training with pre-training plus fine-tuning training, uses different training methods for different models, reduces the difficulty of training, and improves the target detection accuracy of the model. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Figure 1 It is a flow chart of the implementation of the present invention;

[0046] Figure 2 is a sample diagram of a remote sensing SAR image used in the present invention;

[0047] Figure 3 It is a sub-flow chart for implementing network pruning in the present invention;

[0048] Figure 4 It is a schematic diagram of the design parameter quantization search space in the present invention. DETAILED DESCRIPTION

[0049] The embodiments of the present invention are further described below in conjunction with the accompanying drawings.

[0050] Reference Figure 1 ,The present paper proposes a remote sensing SAR target detection method based on ,network pruning and parameter quantization, which includes five stages: data preprocessing, ,building and training of a benchmark model, calculating ,performance indicators, network pruning and parameter quantization.

[0051] The specific implementation is as follows:

[0052] Step 1: Data preprocessing.

[0053] 1.1) Obtain public data sets and expand the data:

[0054] Get the divided training set D from the public remote sensing SAR target detection dataset train and the test set D test , a single image in the dataset can be represented as X 1×H×W , where (1×H×W) represents the single channel, height, and width of the image respectively;

[0055] Since the public remote sensing SAR datasets usually contain less data, it is necessary to expand the acquired datasets. train The image is translated and rotated respectively, and the translated image G' i',j' And the image G' after rotation transformation W',Q' Add them to the training set D train In the expanded training set D exp ;

[0056] The translation transformation formula is as follows:

[0057] G' i',j' =G i+x,j+y

[0058] Among them G i,jrepresents the original image, (i, j) represents the coordinates before translation, (i', j') represents the coordinates after translation, and (x, y) represents the translation direction;

[0059] The rotation transformation formula is as follows:

[0060] G' W',Q' =G Wcosθ+Qsinθ,-Wsinθ+Qcosθ

[0061] Where (W,Q) represents the coordinates before rotation, (W',Q') represents the coordinates after rotation, and θ represents the rotation angle;

[0062] 1.2) For the expanded training set D exp Perform offline data enhancement and online data enhancement:

[0063] 1.2.1) According to the COCO evaluation index, in the expanded training set D exp In the image, calculate the area of ​​the true target box of all targets and divide it into 32 2 and 96 2 As nodes, all targets are divided into small targets, medium targets, and large targets. The number of images containing various types of targets is counted. At the same time, images with a lot of background noise are manually selected as difficult sample images, such as Figure 2 As shown, Figure 2 (a) is an image containing a small object. Figure 2 (b) is an image containing medium-sized objects. Figure 2 (c) is an image containing a large object. Figure 2 (d) is an image containing a lot of background noise, where the white box is the real target box added manually later;

[0064] 1.2.2) Offline data enhancement: Scaling images with small objects in difficult examples and background denoising for some images with large backgrounds;

[0065] 1.2.3) Online data enhancement: The training set images after offline data enhancement are read into the memory, and then these images are randomly flipped, randomly expanded, and randomly erased in turn, so that the same image is different in different iterative training cycles, and the data set D after data enhancement is obtained. aug .

[0066] Step 2: Build a baseline model and train it.

[0067] 2.1) Select a lightweight network structure based on computing resources, and then decide whether to connect the feature pyramid FPN and PAN modules based on the network structure;

[0068] The existing lightweight network structure includes an input layer, a hidden layer, and a detection layer, wherein the hidden layer is usually composed of multiple feature extraction modules with the same structure, each feature extraction module contains a possible downsampling layer and multiple non-downsampling layers, the downsampling layer is composed of a convolutional layer, a pooling layer, or a rearrangement layer, and each non-downsampling layer is composed of a convolutional layer, a BN layer, an activation layer, or a connection layer. Due to limited computing resources, this example does not add feature pyramid FPN and PAN modules;

[0069] 2.2) Adjust the selected lightweight network structure:

[0070] Usually, the lightweight network structure contains 5 downsampling layers with a step size of 2, which will reduce the size of the network output feature map to 1 / 32 of the original image, resulting in the loss of small target information. Therefore, it needs to be adjusted, that is, the classification layer in the original structure is changed to the detection layer, and some downsampling layers in the deep layer of the network are modified to non-downsampling layers. The modification method is to select different modification methods according to the different structures used by the downsampling layer: if the downsampling layer uses a convolutional layer, its step size is changed to 1; if the downsampling layer uses a pooling layer, an empty layer is used instead of the layer. After the modification, the number of network modules that do not change the scale of the feature map will increase, and the number of such modules can be appropriately reduced;

[0071] Using the adjusted lightweight network as the feature extraction backbone network N can enhance the network's ability to recognize small targets. Although the amount of calculation is increased compared to the original network, it can greatly improve the target detection performance.

[0072] 2.3) Use K-Means clustering algorithm to cluster the expanded training set D exp Cluster the target frames of the labels to obtain multiple groups of prior anchor frames for this data set, use these anchor frames as anchor frames of the existing YOLO series single-stage detectors to obtain SAR target detectors, connect the detectors to the backbone network N, and form a SAR target detection benchmark model;

[0073] 2.4) Use the YOLO detector to obtain the loss function of YOLOv2 and use it as the loss function of the baseline model:

[0074] Use the YOLO detector to divide the input image into S 2 grids, each grid has a group A of prior anchor boxes obtained by clustering. Assume that the coordinate of the upper left corner of the grid is c x and c y , the width and height of the prior anchor box are p w and p h , the backbone network regresses each prior anchor box and obtains 5 outputs, namely t x ,t y ,t w ,th and t o , which correspond to the x, y coordinates, width, height and confidence of the regression box respectively;

[0075] Convert the 5 outputs as follows to get b x 、b y 、b w 、b h and P(object), which correspond to the x, y coordinates, width, height and probability of being the target of the predicted box respectively:

[0076] b x =σ(t x )+c x

[0077] b y =σ(t y )+c y

[0078]

[0079]

[0080] P(object)=σ(t o )*IOU(box pred ,box truth )

[0081] Among them, σ() is the sigmoid function, IOU(box pred ,box truth ) is the overlap ratio between the regression frame and the true value frame;

[0082] The loss function of YOLOv2 is obtained from the above conversion parameters:

[0083]

[0084] in, is the number of target categories, is the number of prior anchor boxes, g is the coordinate variable corresponding to the true value frame, b is the coordinate variable corresponding to the predicted frame, i is the grid number, and there are S 2 grids, j is the serial number of the prior anchor box, and each grid has a set of A prior anchor boxes; and It is a flag to determine the overlap between the prior anchor box and the true value box, and is used to determine which set of prior anchor boxes to use to predict the results: if the overlap rate between the jth prior anchor box in the i-th grid and a true value box is greater than the set overlap rate threshold, then otherwise If the overlap rate between the jth prior anchor box in the i-th grid and all the true value boxes is less than the set overlap threshold, then otherwise

[0085] 2.5) Use the random number seed S to initialize the weight parameters of the backbone network N and use the enhanced training set D aug Input into the initialized benchmark model to start training, and use the momentum stochastic gradient descent algorithm SGD to optimize the YOLO loss function to update the network weight parameters:

[0086] 2.5.1) Assume that the initial number of iterations is t = 0 and the maximum number of iterations is t max , initial acceleration v0 = 0, use random number seed S to initialize network weight parameter θ0;

[0087] 2.5.2) At each iteration, from the enhanced training set D aug Get a batch of training samples of size n (X t ,Y t ) n , and then randomly select a training sample from this batch of samples , the training samples and θ t As a parameter of the YOLO loss function, get the current loss function where i∈1,2,...,n;

[0088] 2.5.3) At the tth iteration, the network weight parameter θ is updated using the following expression to obtain the updated network weight parameter θ t+1 :

[0089] θ t+1 =θ t -v t

[0090]

[0091] Among them, v t represents the accumulated acceleration at time t, and v in the first iteration t =v0,θ t is the model weight parameter at time t, and θ is the first iteration t =θ0, α represents the power size, the value is 0.9, η is the learning rate;

[0092] 2.5.4) Repeat 2.5.2) and 2.5.3) and add the enhanced training set D aug One iteration is recorded as one cycle, and the network weight parameters are saved every 10 cycles until the maximum number of iterations t is reached. maxStop training when , and get multiple network weight parameters.

[0093] Step 3: Calculate performance indicators.

[0094] The performance indicators include: F1 score, parameter quantity P and floating point operation number FLOPs, which are specifically implemented as follows:

[0095] 3.1) Update the benchmark model using multiple saved network weight parameters and set the test set D test Input into each updated benchmark model to obtain the target detection result of each updated benchmark model, and obtain the following variable values ​​for each updated benchmark model from the target detection result of each updated benchmark model:

[0096] True positive TP: the number of targets correctly detected;

[0097] False positive FP: the number of targets that are detected incorrectly, i.e. false alarms;

[0098] False negative FN: the number of targets that were not identified, i.e. missed detection;

[0099] Using the above three variable values, calculate the precision Pr, recall Re and F1 score of each updated baseline model:

[0100]

[0101]

[0102]

[0103] The maximum value of the F1 score is recorded as F10, and the network weight parameter corresponding to F10 is represented by W0;

[0104] 3.2) Calculate the number of parameters P and floating point operations FLOPs of W0:

[0105] 3.2.1) Use the following formula to calculate the parameter P of each convolutional layer in the backbone network N corresponding to W0: i：

[0106] P i =K 2 ×C in ×C out

[0107] Among them, K is the convolution kernel size, C in is the number of input channels, C out is the number of output channels;

[0108] 3.2.2) The number of parameters P of all convolutional layers in the backbone network N corresponding to W0 iSum and get the total parameter P0 of W0:

[0109]

[0110] Where n is the number of convolutional layers in the backbone network N;

[0111] 3.2.3) Use the following formula to calculate the floating point operation number FLOPs of each convolutional layer in the backbone network N corresponding to W0: i :

[0112] FLOPs i =2K 2 ×C in ×C out ×H out ×W out

[0113] Among them, K is the convolution kernel size, H out is the output feature map height, W out is the output feature map width;

[0114] 3.2.4) FLOPs of all convolutional layers in the backbone network N corresponding to W0 i Add up and get the total floating point operations FLOPs0 of W0:

[0115]

[0116] Step 4: Prune the network according to the performance indicators calculated in step 3.

[0117] The goal of network pruning is to reduce the number of parameters P and floating point operations FLOPs while ensuring that F1 decreases within reasonable expectations.

[0118] Reference Figure 3 , this step performs coarse-grained module pruning and fine-grained filter pruning on the network in turn as follows:

[0119] 4.1) Perform module pruning on the backbone network N:

[0120] 4.1.1) Set the module mask m = {m1,m2,...,m i ,...,m n}, where m i Corresponding to the i-th feature extraction module in the backbone network N, m i ∈{0,1}, i=1,2,...,n, n represents the number of feature extraction modules in the backbone network N;

[0121] 4.1.2) Let m j|j≠i =1,m i= 0, a mask subnet is obtained that masks the i-th feature extraction module;

[0122] 4.1.3) Repeat 4.1.2) to obtain n masked subnets;

[0123] 4.1.4) With all n mask subnetworks sharing the weight parameter W0, fine-tune each mask subnetwork and calculate their performance index F1 score F1 i , parameter quantity P i and FLOPs i ;

[0124] 4.1.5) Calculate the difference ΔF1 between each mask subnet performance index and the baseline model performance index F10, P0 and FLOPs0 i , ΔP i , ΔFLOPs i :

[0125] ΔF1 i =F10-F1 i

[0126] ΔP i =P0-P i

[0127] ΔFLOPs i =FLOPs0-FLOPs i

[0128] 4.1.6) Define the importance index I of the i-th feature extraction module in the backbone network N i for:

[0129]

[0130] Among them, α, β, and γ are ΔF1 i , ΔP i , ΔFLOPs i The impact factor constant, I i Indicates the importance of each module, which means the impact of the balanced value of each unit parameter amount and floating-point operation number on the change of F1. The larger I is, the more important the module is;

[0131] 4.1.7) Compare the importance index I of these modules with the set importance index threshold I thr For comparison, when I is less than I thr When , the feature extraction module corresponding to the importance index I in the backbone network N is pruned to obtain the backbone network N after module pruning m ;

[0132] 4.2) Based on the Hrank method, the backbone network N after module pruningm Perform filter pruning:

[0133] Since the average rank of multiple feature maps generated by a single filter is always the same, the importance of the filter can be judged by the rank of the feature map, and then the unimportant filters are pruned to obtain the pruned network. m The pruning process of each convolutional layer is as follows:

[0134] 4.2.1) Optional training set D aug Several images in are input to the backbone network N m In the network, the average rank of each convolutional layer output feature map is calculated as R = {r1, r2, ... r i ,...,r n}, where r i is the i-th filter w i The average rank of the output feature map, i∈{1,2,...,n}, where n represents the number of filters in this layer;

[0135] 4.2.2) Sort R in descending order to obtain the average rank of the sorted convolutional layer output feature map in, represents the i-th value in R from high to low;

[0136] 4.2.3) Set the number of filters to be retained n1 and the number of filters to be pruned n2, where n1+n2=n;

[0137] 4.2.4) From Select the filters corresponding to the first n1 values ​​to get the important filter set from Select the filters corresponding to the last n2 values ​​to get an unimportant filter set in, for The filter corresponding to the jth value in , j∈{1,2,...,n1}, for The filter corresponding to the kth value in , k∈{1,2,...,n2};

[0138] 4.2.5) Remove the unimportant filter set U of all convolutional layers w After that, we get the finely pruned network N P , use the same random number seed S to initialize the finely pruned network N P The network weight parameters are then used to prune the network N P Perform training and calculate performance indicators, save the network weight parameter with the highest F1 score, denoted as W P .

[0139] Step 5: Quantify the network parameters to obtain the final SAR target detection model.

[0140] After pruning, the network parameters are further compressed by parameter quantization:

[0141] 5.1) Use the quantization function to quantize the weight parameters in the full-precision network module to obtain the quantized weight parameters w q :

[0142]

[0143] Among them, Quan k is the quantization function, which is expressed as: k is the number of quantization bits;

[0144] w τ is the original network weight parameter, w q is the quantized network weight parameter;

[0145] 5.2) Use the quantization function to quantize the activation output in the full-precision network module to obtain the quantized activation output:

[0146]

[0147]

[0148] Among them, x is the activation function input, y is the activation output, and y q is the quantized activation output, α is a learnable parameter;

[0149] 5.3) After pruning, the network N P The weight parameters and activation outputs of the quantization bit value range are set differently, that is, the quantization bit value range Q of the weight parameters is set w = {4,5,...,8}, set the quantization bit value range Q of the activation output a ={3,4,...,7};

[0150] 5.4) The importance index of all feature extraction modules I = {I1, I2, ..., I i ,...I n} Sort in ascending order to obtain the importance index of the feature extraction module after ascending order Among them, I i represents the importance index of the i-th feature extraction module, represents the i-th value in I from small to large, i∈{1,2,...,n};

[0151] Reference Figure 4 In this example, Io , Q w and Q a Divide into three subsets, so that the three subsets correspond to each other in order, forming a quantitative search space, that is, for I o Each value in the feature extraction module corresponds to the corresponding Q w Subset and Q a Perform quantitative search in subsets;

[0152] 5.5) Each time a quantitative search is performed, the Q corresponding to each feature extraction module is w Subset and Q a Select a value from each subset as the k value of the quantization function in the weight and activation output, respectively, to obtain the quantized network, and then use it in the enhanced training set D aug Fine-tune training and calculate performance indicators;

[0153] 5.6) Repeat 5.5) to search for the best quantization scheme, and apply the best quantization scheme found to the finely pruned network N P In the final network N Q ;

[0154] 5.7) Since the final network N after quantization Q The number of parameters is very small, and it is difficult to train from scratch using random initialization parameters, so the network weight parameters W after fine pruning are used. P As the final network N Q The pre-trained weight parameters are fine-tuned to obtain the final network weight parameters W Q ;

[0155] 5.8) Use the final network N Q Replace the backbone network N of the baseline model and use the final network weight parameter W Q Update the network weight parameters to obtain the final SAR target detection model.

[0156] Step 6: Set the test set D test Input it into the final SAR target detection model to obtain accurate SAR target detection results.

[0157] The above description is only a specific example of the present invention and does not constitute any limitation to the present invention. It is obvious that for professionals in this field, after understanding the content and principles of the present invention, they may make various modifications and changes in form and details without departing from the principles and structures of the present invention. However, these modifications and changes based on the ideas of the present invention are still within the scope of protection of the claims of the present invention.

Claims

1. A remote sensing SAR target detection method based on network pruning and parameter quantization, characterized in that: include: (1) Obtain the divided training set D from the public remote sensing SAR target detection dataset train and the test set D test , and use geometric transformation to expand the images in the training set to obtain the expanded training set D exp ; (2) After the expansion, the training set D exp The proportions of target images of different sizes and images with a large amount of background are counted in the training set, and the images with smaller proportions are selected as difficult samples in the expanded training set. The samples are then subjected to offline data enhancement and online data enhancement in turn to obtain the enhanced training set D aug ; (3) Building a SAR target detection benchmark model: (3a) The existing lightweight network structure is adjusted, that is, the original classification layer is changed into a detection layer, and some downsampling layers in the deep layer of the network are modified into non-downsampling layers, and the adjusted lightweight network is used as a feature extraction backbone network N, which includes an input layer, a hidden layer and a detection layer. The hidden layer is composed of a plurality of feature extraction modules with the same structure, each feature extraction module includes a downsampling layer and a plurality of non-downsampling layers, the downsampling layer is composed of a convolutional layer, a pooling layer or a rearrangement layer, and each non-downsampling layer is composed of a convolutional layer, a BN layer, an activation layer or a connection layer; (3b) Use K-Means clustering algorithm to cluster the expanded training set D exp The labeled target boxes are clustered to obtain the prior anchor box for this data set, and this anchor box is used as the anchor box of the existing YOLO single-stage detector to obtain the SAR target detector. The detector is connected to the backbone network N to form a benchmark model for SAR target detection. (4) Update the network weight parameters of the baseline model: (4a) The YOLO loss function is used as the loss function of the baseline model, and the weight parameters of the backbone network N are initialized using the random number seed S; (4b) The enhanced training set D aug Input into the initialized benchmark model to start training, use the momentum stochastic gradient descent algorithm SGD to optimize the YOLO loss function to update the network weight parameters, save the network weight parameters every 10 cycles, and stop training when the set maximum number of iterations is reached to obtain multiple updated network weight parameters; (5) Performance indicators for evaluating the baseline model: (5a) Use the saved multiple network weight parameters to update the baseline model on the test set D test Calculate the F1 score of each updated benchmark model, take the maximum value of the F1 score as F10, and the network weight parameter corresponding to F10 is represented by W0; (5b) Calculate the number of parameters P and floating point operations FLOPs of W0, and record the calculation results as P0 and FLOPs0 respectively; (6) Rough pruning of the backbone network N: (6a) Setting a module mask m for the n feature extraction modules in the backbone network N, and using the module mask m to perform One Hot encoding on all feature extraction modules of the network to obtain n masked subnetworks; (6b) All n mask subnetworks share the weight parameter W0, fine-tune each mask subnetwork and calculate its performance index F1 score F1 i , parameter quantity P i and FLOPs i ; (6c) According to the performance index of each mask subnet, the importance index I of each feature extraction module in the backbone network N is calculated, and these importance indexes I are compared with the set importance index threshold I thr For comparison, when I is less than I thr When , the feature extraction module corresponding to the importance index I in the backbone network N is pruned to obtain the backbone network N after rough pruning m ; (7) After rough pruning, the backbone network N m To perform fine pruning: (7a) Evaluation of the backbone network N after rough pruning based on the Hrank method m The importance of filters in the convolutional layer, and the important filters and unimportant filters are divided. In the backbone network N after rough pruning m Based on this, unimportant filters are cut off to obtain a finely pruned network N P ; (7b) Use the same random number seed S to initialize the finely pruned network N P The network weight parameters are then trained using the same training method as in (4) to fine-tune the pruned network N P Perform training and save the network weight parameter with the highest F1 score, denoted as W P ; (8) After fine pruning, the network N P Perform parameter quantization: (8a) The finely pruned network N P The feature extraction module in is changed to a quantization module; (8b) Selecting quantization strategies of different precisions for weight parameters and activation outputs, and designing a quantization search space based on the importance index I of the feature extraction module; (8c) In the enhanced training set D aug Iterate and search for the best quantization scheme, and apply the best quantization scheme to the finely pruned network N P In the final network N Q ; (8d) Use W P As the final network N Q Pre-trained weight parameters, fine-tuning training to obtain the final network weight parameters W Q ; (9) Use the final network N Q Replace the backbone network N of the baseline model and use the final network weight parameter W Q Update the network weight parameters to obtain the final SAR target detection model; (10) The test set D test Input it into the final SAR target detection model to obtain accurate SAR target detection results.

2. The method according to claim 1, wherein the image is data expanded in (1), which is implemented as follows: 1a) For the obtained training set D train The image is translated and the translated image G' is obtained. i',j' : G' i',j' =G i+x,j+y , Where G represents the original image, (i, j) represents the coordinates before translation, (i', j') represents the coordinates after translation, and (x, y) represents the translation direction; 1b) Perform a rotation transformation on the acquired training set image to obtain the rotated image G' W',Q' : G' W',Q' =G Wcosθ+Qsinθ,-Wsinθ+Qcosθ in, (W,Q) represents the coordinates before rotation, (W',Q') represents the coordinates after rotation, and θ represents the rotation angle; 1c) The translated image G' i',j' and the rotated image G' W',Q' Add them to the training set D train In the expanded training set D exp .

3. The method according to claim 1, wherein (2) the expanded training set D exp The difficult samples in the above example are enhanced offline and online respectively, which can be realized as follows: 2a) Selecting difficult examples: After the expansion of the training set D exp In the image, calculate the area of ​​the true target box of all targets and divide it into 32 2 and 96 2 As nodes, all targets are divided into small targets, medium targets, and large targets. The number of images containing various types of targets is counted, and images with a lot of background noise are manually selected as difficult samples. 2b) Offline Data Enhancement: The images with small objects in the difficult examples are scaled, and the images with a lot of background are denoised to obtain the training set after offline data enhancement. 2c) Online data enhancement: The training set images after offline data enhancement are read into the memory, and then these images are randomly flipped, randomly expanded, and randomly erased in turn, so that the same image is different in different iterative training cycles, and the data enhancement dataset D is obtained. aug .

4. The method according to claim 1, wherein the detection benchmark model loss function in (4a) is expressed as follows: in, is the number of target categories, is the number of prior anchor boxes, g is the coordinate variable corresponding to the true value frame, b is the coordinate variable corresponding to the predicted frame, i is the grid number, and there are S 2 grids, j is the serial number of the prior anchor box, and each grid has a set of A prior anchor boxes. and It is the flag bit that determines the overlap between the prior anchor box and the true value box. σ() is the sigmoid function. IOU(box pred ,box truth ) is the overlap ratio between the prior anchor box and the true value box, b x =σ(t x )+c x , b y =σ(t y )+c y , t x ,t y ,t w ,t h and t o These are the five outputs obtained by the model for each prior anchor box, which correspond to the x, y coordinates, width, height and confidence of the regression box, respectively. x and c y is the x, y coordinates of the upper left corner of the grid, p w and p h is the width and height of the prior anchor box.

5. The method according to claim 1, wherein (4b) uses the momentum stochastic gradient descent algorithm SGD to optimize the YOLO loss function and update the weight parameters of the network, which is implemented as follows: 4b1) The initial number of iterations is t = 0, and the maximum number of iterations is t max , initial acceleration v0=0, randomly initialize network weight parameter θ0; 4b2) At each iteration, a batch of training samples (X t ,Y t ) n Randomly select a training sample from The training samples and θ t As a parameter of the YOLO loss function, get the current loss function where i∈1,2,...,n; 4b3) At the tth iteration, the network weight parameter θ is updated using the following expression to obtain the updated network weight parameter θ t+1 : i t+1 =θ t -v t in, v t represents the accumulated acceleration at time t, and v in the first iteration t =v0,θ t is the model weight parameter at time t, and θ is the first iteration t =θ0, α represents the power size, the value is 0.9, η is the learning rate; 4b4) Repeat 4b2) and 4b3) until the maximum number of iterations t is reached. max When , stop updating and get the final network weight parameters 6. The method according to claim 1, wherein in (5a) in the test set D test The F1 score of each updated baseline model is calculated in , which is implemented as follows: 5a1) Set the test set D test Input into each updated benchmark model to obtain the target detection result of each updated benchmark model; 5a2) From the target detection results of each updated baseline model, obtain the following variable values ​​for each updated baseline model: True positive TP: the number of targets correctly detected; False positive FP: the number of targets that are detected incorrectly, i.e. false alarms; False negative FN: the number of targets that were not identified, i.e. missed detection; 5a3) Using the above three variable values, calculate the precision Pr and recall Re of each updated baseline model: 5a4) Using the calculated precision Pr and recall Re, calculate the F1 score of each updated baseline model:

7. The method according to claim 1, wherein the calculation of the parameter quantity P and the floating point operation number FLOPs of the network weight parameter W0 that makes the F1 score reach the highest in (5b) is implemented as follows: 5b1) Calculate the parameter P of each convolutional layer in the backbone network N corresponding to W0 i： P i =K 2 ×C in ×C out in, K is the convolution kernel size, C in is the number of input channels, C out is the number of output channels; 5b2) The number of parameters P of all convolutional layers in the backbone network N corresponding to W0 i Sum and get the total parameter P0 of W0: Where n is the number of convolutional layers in the backbone network N; 5b3) Calculate the floating point operations FLOPs of each convolutional layer in the backbone network N corresponding to W0 i : FLOPs i =2K 2 ×C in ×C out ×H out ×W out Among them, K is the convolution kernel size, H out is the output feature map height, W out is the output feature map width; 5b4) FLOPs of all convolutional layers in the backbone network N corresponding to W0 i Add up and get the total floating point operations FLOPs0 of W0:

8. The method according to claim 1, wherein (6b) uses a module mask to perform One Hot encoding on all feature extraction modules of the backbone network N, which is implemented as follows: 6b1) Set module mask m = {m1,m2,...,m i ,...,m n }, where m i Corresponding to the i-th feature extraction module in the backbone network N, m i ∈{0,1}, i=1,2,...,n, n represents the number of feature extraction modules in the backbone network N; 6b2) Let m j|j≠i =1,m i = 0, a mask subnet is obtained that masks the i-th feature extraction module; 6b3) Repeat 6b2) to obtain n masked subnets.

9. The method according to claim 1, wherein in (6c), the importance index I of each feature extraction module in the backbone network N is calculated according to the performance index of each mask subnet, which is implemented as follows: 6c1) Calculate the difference between the performance index of each mask subnet and the performance index of the baseline model: ΔF1 i =F10-F1 i ΔP i =P0-P i ΔFLOPs i =FLOPs0-FLOPs i 6c2) Define the importance index I of the i-th feature extraction module in the backbone network N i for: in, α, β, and γ are ΔF1 i , ΔP i , ΔFLOPs i The impact factor constant.

10. The method according to claim 1, wherein (7a) evaluates the rough pruned backbone network N based on the Hrank method m The importance of filters in the convolutional layer, and the division of important filters and unimportant filters, is implemented as follows: 7a1) Optional training set D aug Several images in are input to the backbone network N m In the above example, the average rank of the output feature map of each convolutional layer of the network is calculated as R = {r1, r2, ... r i ,...,r n },in, r i is the i-th filter w i The average rank of the output feature map, i∈{1,2,...,n}, where n represents the number of filters in this layer; 7a2) Sort R in descending order to obtain the average rank of the sorted convolutional layer output feature map in, represents the i-th value in R from high to low; 7a3) setting the number of filters to be retained n1 and the number of filters to be pruned n2, where n1+n2=n; 7a4) From Select the filters corresponding to the first n1 values ​​to get the important filter set from Select the filters corresponding to the last n2 values ​​to get an unimportant filter set in, for The filter corresponding to the jth value in , j∈{1,2,...,n1}, for The filter corresponding to the kth value in , k∈{1,2,...,n2}.

11. The method according to claim 1, wherein in (8a), the full-precision network module is modified into a quantized network module, which is implemented as follows: 8b1) Use the quantization function to quantize the weight parameters in the full-precision network module to obtain the quantized weight parameters w q : in, Quan k And the quantization function is expressed as: k is the number of quantization bits; w τ is the original network weight parameter, w q is the quantized network weight parameter; 8b2) Use the quantization function to quantize the activation output in the full-precision network module to obtain the quantized activation output: Among them, x is the activation function input, y is the activation output, and y q is the quantized activation output, and α is a learnable parameter.

12. The method according to claim 1, wherein in (8b), quantization strategies of different precisions are selected for weight parameters and activation outputs, and a quantization search space is designed according to an importance index I of a feature extraction module, and is implemented as follows: 8a1) Set the quantization bit value range Q of the weight parameter w = {4,5,...,8}, set the quantization bit value range Q of the activation output a ={3,4,...,7}; 8a2) The importance index of all feature extraction modules I = {I1, I2, ..., I i ,...I n } Sort in ascending order to obtain the importance index of the feature extraction module after ascending order in, I i represents the importance index of the i-th feature extraction module, represents the i-th value in I from small to large, i∈{1,2,...,n}; 8a3) I o , Q w and Q a Divide into x subsets, so that the three subsets correspond one by one in order to form a quantitative search space, that is, for I o Each value in corresponds to a feature extraction module, in which the corresponding Q w Subset and Q a Quantitative search in subsets.

Images