Classification network compression method and device, electronic equipment and storage medium

Abstract

The invention provides a classification network compression method and device, electronic equipment and a computer readable storage medium, and relates to the technical field of deep learning. The compression method comprises the steps that an initial convolution class network and a disturbance input image are acquired, the initial convolution class network comprises at least one convolution layer, and the convolution layer comprises a plurality of convolution kernels; performing forward propagation on the disturbance input image based on the initial convolution class network, and determining a mask image of each convolution kernel; and based on the mask image of each convolution kernel, determining a removed convolution kernel of the initial convolution point class network, and compressing the initial convolution point class network to obtain a target convolution point class network. The problems that the robustness of convolution kernel extraction features cannot be accurately measured, the classification accuracy of a compression model cannot be guaranteed, and the technical difficulty of efficient bidirectional search of a disturbance space and a structure space is large in the prior art are at least solved. The method is suitable for network compression and model compression scenes.

Description

Technical Field

[0001] The present invention relates to the field of deep learning technology, and in particular to a compression method, device, electronic device and computer-readable storage medium for a classification network. Background Art

[0002] Neural network parameter pruning is a technique to reduce the size and computational complexity of neural network models, and plays an important role in reducing computational latency. The principle of neural network parameter pruning is to reduce the number of model parameters and computational complexity by removing unnecessary neurons, connections, or weights.

[0003] At present, the existing technology mainly implements neural network parameter pruning through the following steps:

[0004] 1. Model pre-training: Train a complete neural network model until the neural network model reaches a satisfactory performance level. Model pre-training is the basis of the pruning process, because pruning can only ensure that the pruned model is still effective when the pre-trained model has performed well.

[0005] 2. Parameter importance evaluation: Evaluating the importance of each parameter in the neural network can be achieved through a variety of methods, such as weight amplitude, gradient amplitude, sparse regularization, and identifying parameters that contribute less to the output of the model so as to remove the parameters with less contribution without loss.

[0006] 3. Parameter removal: Remove or prune unimportant parameters based on the parameter importance evaluation results and the specified compression rate, such as setting the target weight to zero or completely removing the target connection or neuron. For convolutional neural networks, the convolution kernels with the specified sequence number on the target layer and the convolution plates with the same sequence number on each convolution kernel in the next layer are directly deleted from the original network.

[0007] 4. Parameter fine-tuning: After removing or pruning parameters, the model is usually fine-tuned using a smaller learning rate and total number of iterations to restore the performance that may have been lost due to parameter removal or pruning. Fine-tuning is an iterative process that may need to be repeated many times until the model performance is stable. The redundant parameters and computational complexity of convolutional neural networks are mainly concentrated in the large number of convolution kernels and convolution operations in each layer. Therefore, it is necessary to evaluate the importance of each layer of convolution kernels and determine the parameters that have less impact on the model output performance.

[0008] However, in order to ensure the availability of the compressed model, the prior art removes redundant parameters in the neural network pruning, which are weights that are insensitive to the distribution of the training set data. It often assumes or assumes that the distribution of the training set data conforms to the distribution characteristics of the test set or the data collected in the real environment, resulting in overfitting of the compression model to the training data, and model drift for input data with a certain distribution offset. In addition, the prior art method of implementing convolutional neural network pruning can only be performed in common application scenarios. With a given compression rate, the target neural network is effectively compressed and accelerated. Under the condition of adversarial perturbation input (usually adversarial perturbation is not observable by the naked eye), the parameter importance evaluation criteria in the prior art cannot accurately measure the robustness of the convolution kernel extracted features, the classification accuracy of the compression model cannot be guaranteed, and the efficient two-way search technology of the perturbation space and the structure space is difficult. Summary of the invention

[0009] The technical problem to be solved by the present invention is to address the above-mentioned deficiencies in the prior art and to provide a compression method, device, electronic device and computer-readable storage medium for a classification network. The method can realize intelligent and effective compression of the classification network, improve the generation efficiency and anti-disturbance of the optimal compression structure of the classification network, achieve a balance between the robustness and real-time performance of the optimal compression structure of the classification network, and improve the deployment availability of the optimal compression structure of the classification network.

[0010] In a first aspect, the present invention provides a compression method for a classification network, comprising: obtaining an initial convolution classification network and a perturbed input image, wherein the initial convolution classification network includes at least one convolution layer, and the convolution layer includes multiple convolution kernels; forward propagating the perturbed input image based on the initial convolution classification network to determine a mask image of each convolution kernel; based on the mask image of each convolution kernel, determining a removed convolution kernel of the initial convolution classification network, and compressing the initial convolution classification network to obtain a target convolution classification network.

[0011] Preferably, the obtaining of the initial convolution classification network and the perturbed input image specifically includes: obtaining the initial convolution classification network, the original image and the corresponding classification label; inputting the original image into the initial convolution classification network for back propagation, and generating a perturbed input on the original image based on an iterative gradient attack algorithm to obtain a perturbed input image.

[0012] Preferably, the method of forward propagating the perturbed input image based on the initial convolution classification network to determine the mask image of each convolution kernel specifically includes: forward propagating the perturbation input image based on the initial convolution classification network to obtain the output feature map of each convolution kernel; performing inter-layer weighted accumulation on the output feature maps of each convolution kernel of different convolution layers to obtain the cumulative feature map attention of each convolution kernel; performing maximum-minimum normalization on the cumulative feature map attention of each convolution kernel of the same convolution layer to obtain the input mask of each convolution kernel; and performing dot product of the input mask of each convolution kernel and the perturbation input image to obtain the mask image of each convolution kernel.

[0013] Preferably, the output feature maps of each convolution kernel of different convolution layers are weightedly accumulated between layers to obtain the cumulative feature map attention of each convolution kernel, specifically including: calculating the cumulative feature map attention of each convolution kernel according to formula (1):

[0014]

[0015] (1), where x represents the perturbed input image, Represents the output feature map of the i-th convolution kernel under the l-th convolution layer, l = 1, 2, ..., N l , i=1,2,…,n l , Up(·) refers to the bilinear upsampling function, represents the weight of the lth convolutional layer, Represents the cumulative feature map attention of the i-th convolution kernel under the current convolution layer.

[0016] Preferably, the cumulative feature map attention of each convolution kernel of the same convolution layer is subjected to the maximum-minimum normalization in the layer to obtain the input mask of each convolution kernel, specifically comprising: calculating the input mask of each convolution kernel according to formula (2):

[0017] in, N l *n l The cumulative feature map attention set of convolution kernels, Represents the input mask of the i-th convolution kernel under the current convolution layer.

[0018] Preferably, the method of determining the removed convolution kernel of the initial convolution classification network based on the mask image of each convolution kernel specifically includes: inputting the mask image of each convolution kernel into the initial convolution classification network to obtain the classification result of the mask image; calculating the classification confidence of the mask image according to the classification result of the mask image and the corresponding classification label; sorting the convolution kernels of the same convolution layer based on the classification confidence of the mask image; and determining the removed convolution kernel of the initial convolution classification network according to the preset compression rate and the sorting result.

[0019] Preferably, the compressed initial convolution classification network obtains the target convolution classification network, specifically including: removing the convolution kernel of the initial convolution classification network to obtain a compressed convolution classification network; inputting the original image into the compressed convolution classification network for back propagation, and generating a perturbation input on the original image based on an iterative gradient attack algorithm to obtain a second perturbation vector; synthesizing the first perturbation vector and the second perturbation vector to obtain a third perturbation vector, wherein the first perturbation vector comes from the original image input into the initial convolution classification network for back propagation, and generating a perturbation input on the original image based on an iterative gradient attack algorithm; based on the third perturbation vector, the initial convolution classification network and the compressed convolution classification network, performing teacher-student joint adversarial training to obtain the target convolution classification network.

[0020] Preferably, the synthesizing the first disturbance vector and the second disturbance vector to obtain the third disturbance vector specifically includes: calculating the third disturbance vector according to formula (3):

[0021]

[0022] Among them, t iter represents the current training iteration number of the compressed convolutional classification network, T iter represents the total number of training iterations of the compressed convolutional classification network, represents the second perturbation vector, represents the first perturbation vector, represents the parameters of the compressed convolutional classification network, represents the parameters of the initial convolutional classification network, and x represents the perturbed input image.

[0023] In a second aspect, the present invention also provides a compression device for a classification network, comprising an acquisition module, a determination module and a compression module, the acquisition module being used to acquire an initial convolutional classification network and a perturbed input image, wherein the initial convolutional classification network comprises at least one convolutional layer, and the convolutional layer comprises multiple convolutional kernels; the determination module being connected to the acquisition module and being used to forward propagate the perturbed input image based on the initial convolutional classification network and determine the mask image of each convolutional kernel; the compression module being connected to the determination module and being used to determine the removed convolution kernel of the initial convolutional classification network based on the mask image of each convolutional kernel, and compressing the initial convolutional classification network to obtain a target convolutional classification network.

[0024] In a third aspect, the present invention further provides an electronic device, comprising a memory and a processor, wherein the memory stores a computer program, and the processor is configured to run the computer program to implement the classification network compression method provided in the first aspect.

[0025] In a fourth aspect, the present invention further provides a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the classification network compression method provided in the first aspect is implemented.

[0026] The present invention provides a compression method, device, electronic device and computer-readable storage medium for a classification network. By using a perturbed input image that conforms to the distribution characteristics of data collected in a real environment and has disturbances as a training set for an initial convolution classification network, the training set is closer to the actual use situation, and the adaptability of the convolution classification network to various disturbances that may be encountered in real scenes can be enhanced, and the robustness of the convolution classification network can be improved. During the forward propagation of the perturbed input image by the initial convolution classification network, the feature focus area of ​​each convolution kernel in the perturbed input image is obtained, and a mask image of each convolution kernel is generated. The mask image helps to visualize the focus of the convolution kernel on specific features, so that the subsequent evaluation of the contribution of the convolution kernel becomes more intuitive. The mask image of each convolution kernel is then classified through the initial convolution classification network, and the importance of each convolution kernel is evaluated according to the classification result, so as to accurately identify the convolution kernels with low importance and can be removed in the initial convolution classification network, and then the initial convolution classification network is compressed in a targeted manner, and the optimal compression structure of the classification network with strong recognition robustness and anti-disturbance is quickly generated. Therefore, the present invention can realize intelligent and effective compression of the classification network, improve the generation efficiency and anti-disturbance of the optimal compression structure of the classification network, achieve a balance between the robustness and real-time performance of the optimal compression structure of the classification network, and improve the deployment availability of the optimal compression structure of the classification network. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Figure 1 This is a flow chart of a classification network compression method according to Embodiment 1 of the present invention;

[0028] Figure 2 This is a flow chart of a classification network compression method according to Embodiment 2 of the present invention;

[0029] Figure 3 This is a flow chart of a classification network compression method according to Embodiment 3 of the present invention;

[0030] Figure 4 This is a schematic diagram of the structure of a compression device for a classification network according to Embodiment 4 of the present invention;

[0031] Figure 5 This is a schematic diagram of the structure of a compression device of a classification network in Embodiment 5 of the present invention;

[0032] Figure 6 This is a structural diagram of a compression device for a classification network according to Example 6 of the present invention. DETAILED DESCRIPTION

[0033] In order to enable those skilled in the art to better understand the technical solution of the present invention, the embodiments of the present invention will be further described in detail below with reference to the accompanying drawings.

[0034] It should be understood that the specific embodiments and drawings described herein are only used to explain the present invention rather than to limit the present invention.

[0035] It can be understood that, in the absence of conflict, the various embodiments of the present invention and the various features in the embodiments can be combined with each other.

[0036] It can be understood that, for the convenience of description, the drawings of the present invention only show the parts related to the present invention, while the parts irrelevant to the present invention are not shown in the drawings.

[0037] It can be understood that each unit and module involved in the embodiments of the present invention may correspond to only one physical structure, or may be composed of multiple physical structures, or multiple units and modules may be integrated into one physical structure.

[0038] It can be understood that, in the absence of conflict, the functions and steps marked in the flowcharts and block diagrams of the present invention may occur in an order different from that marked in the drawings.

[0039] It is understood that the flowcharts and block diagrams of the present invention illustrate the possible architectures, functions, and operations of the systems, devices, equipment, and methods according to the various embodiments of the present invention. Each box in the flowchart or block diagram may represent a unit, module, program segment, or code, which contains executable instructions for implementing the specified functions. Moreover, each box or combination of boxes in the block diagram and flowchart may be implemented by a hardware-based system that implements the specified functions, or may be implemented by a combination of hardware and computer instructions.

[0040] It can be understood that the units and modules involved in the embodiments of the present invention can be implemented by software or hardware. For example, the units and modules can be located in a processor.

[0041] Embodiment 1:

[0042] like Figure 1 As shown, this embodiment provides a compression method for a classification network.

[0043] In this embodiment, the classification network mainly includes the following types: ① Convolutional Neural Networks (CNN), which is particularly suitable for processing image data, extracts features through convolutional layers, and is applied to image classification and object detection tasks. ② Recurrent Neural Network (RNN), which is mainly used to process sequence data and is suitable for text classification and time series analysis. Among them, LSTM (Long Short-Term Memory) and GRU (gated recurrent unit) are common RNN variants. ③ Fully connected network (Deep-Learning Neural Network, DNN), composed of multiple fully connected layers, is suitable for the classification of structured data, but usually has poor results in image and time series tasks. ④ Self-attention network (Transformer), which has been widely used in the field of natural language processing in recent years, is suitable for processing text classification tasks, and captures long-distance dependencies through self-attention mechanism. ⑤ Graph Neural Network (GNN), which is used to process graph structured data and is suitable for classification tasks in the field of social networks and recommendation systems. This embodiment takes the classification network as a convolutional neural network (i.e., a convolutional classification network) as an example.

[0044] It should be noted that convolutional neural network is a deep learning model widely used in the field of image classification, which mainly includes convolution operation and pooling operation. In the forward propagation process, the convolution operation first uses the convolution kernel to move on the input image in a sliding manner. Each time it moves, the convolution kernel is element-by-element multiplied by the local area of ​​the input image and the results are summed. The output is then nonlinearly mapped through the activation function to obtain an element in the output feature map to introduce the nonlinear characteristics of the network. The pooling operation is used to reduce the spatial resolution of the feature map and to extract the position invariance of the features to a certain extent, and to reduce the dimension of the feature map by taking the maximum or average value in a specific area. Repeating the convolution operation and pooling operation multiple times forms the depth of the network, and the convolutional neural network is a multi-layer structure. Each layer contains multiple convolution kernels and activation functions to form the width of the network. After multiple convolution and pooling operations, the obtained features are Figure 1 The network is flattened and connected to one or more fully connected layers, which can be used for the final classification task.

[0045] Compression methods for classification networks, including:

[0046] S101, obtaining an initial convolution classification network and a perturbed input image, wherein the initial convolution classification network includes at least one convolution layer, and the convolution layer includes multiple convolution kernels.

[0047] In this embodiment, the initial convolution classification network refers to a pre-built convolution neural network that meets the classification requirements. The perturbed input image refers to an image that meets the distribution characteristics of the real environment collected data and has perturbations.

[0048] VGG (Visual Geometry Group Network) and ResNet (Residual Network) are both large-scale convolutional neural network models, which respectively construct deep networks by stacking multiple small convolutional layers and introducing residual connections, and are widely used in various computer vision tasks such as image classification, target detection, and image segmentation. In this embodiment, the initial convolution classification network includes but is not limited to: VGG-16, ResNet-50, ResNet-56, and ResNet-110 sequential and cross-layer connection networks.

[0049] Specifically, S101: obtaining an initial convolution classification network and a perturbed input image, including steps S1011-S1012:

[0050] S1011, obtaining an initial convolution classification network, an original image, and corresponding classification labels.

[0051] In this embodiment, the original image includes but is not limited to: a grayscale image, a black and white binary image, and a color RGB image (an image composed of three basic color channels of red, green, and blue). In this embodiment, the original image is a color RGB image as an example. The image acquisition unit acquires images in a real environment to obtain an original image set, wherein the original image set includes at least one original image. The original images in the original image set are classified and identified to obtain classification labels corresponding to the original images, wherein the classification labels are not limited to: One-hot vectors.

[0052] It should be noted that one-hot vector is an encoding method for representing categorical data, which is commonly used in machine learning and deep learning. One-hot vector converts each category into a unique binary vector with the following characteristics: ① Single activation: In each one-hot vector, only one element is 1 (indicating the category), and the rest are 0, which ensures that each category is unique and has no overlap. For example, for three categories (A, B, C): category A is represented as [1,0,0], category B is represented as [0,1,0], and category C is represented as [0,0,1]. ② Dimension: If there are N categories, then the length of the one-hot vector for each category is N.

[0053] In classification tasks, one-hot encoding is often used to convert labels into a format suitable for model training. For example, when using neural networks for multi-classification, category labels are usually converted to one-hot encoding to facilitate the calculation of loss functions. The one-hot vector representation avoids the order relationship that may be introduced by digital encoding. For example, if integers (such as 0, 1, 2) are used to represent categories, the model may mistakenly believe that these numbers have a size relationship. Therefore, using one-hot encoding can prevent ambiguity.

[0054] S1012, input the original image into the initial convolution classification network for back propagation, and generate a perturbation input on the original image based on an iterative gradient attack algorithm to obtain a perturbation input image.

[0055] In this embodiment, back propagation is an optimization method for training a neural network, which mainly adjusts the network parameters by calculating the gradient of the loss function with respect to the weights of the neural network to minimize the loss function. Back propagation first calculates the difference between the output of the neural network and the true classification label, that is, the loss value, through forward propagation. Then, starting from the output layer of the neural network, the error signal is back propagated, the local gradient of each neuron is calculated layer by layer, and the weights are updated according to the chain rule. The weight updating process involves calculating the partial derivative of each weight, that is, the sensitivity of the loss function to each weight. Through multiple iterations, back propagation can effectively adjust the weights of the neural network, so that the neural network can better fit the training data and improve the prediction accuracy.

[0056] The Projected Gradient Descent (PGD) algorithm is essentially a white-box attack method that obtains the perturbation input that maximizes the loss function by accessing the training gradient of the victim model. Since the PGD algorithm considers both the direction and amplitude changes of the perturbation vector and continuously iterates to search for the optimal perturbation input, the PGD algorithm can explore the perturbation input space more efficiently and reduce the time consumption of traversing the perturbation input space.

[0057] Therefore, on the premise of obtaining the model parameters of the initial convolution classification network, combined with the cross-entropy loss function, back propagation is performed and the gradient derivative is transferred to the image input layer of the initial convolution classification network. Based on the iterative gradient attack algorithm, a perturbation input is generated on the original image to obtain a perturbation input image. Specifically: ① Input the original image X into the initial convolution classification network for forward propagation, that is, according to p = model (X), calculate the output probability distribution of the initial convolution classification network for the original image. ② Use the cross-entropy loss function, that is, L = -log(p(ytarget|X)) to calculate the loss L, where ytarget refers to the target class that is misclassified by the initial convolution classification network. ③ Through back propagation, that is, according to Calculate the gradient of loss L with respect to the original image X ④Get reverse gradient Take the reverse gradient The opposite number of and multiply it by the learning rate (step size), that is, according to Generate a perturbation input, where sign(·) refers to the function used to take the reverse gradient, and sign(·) only retains the positive and negative direction information. ⑤ Apply the perturbation input to the original image to generate the perturbation input image X ′ =X+ΔX. Of course, in order to keep the generated perturbed input image within the valid range, it is usually necessary to crop the perturbed input image to the allowed value range, such as: X ′ = clip(X′,0,1), or for an image with pixel values ​​in the range [0.255], X ′ =clip(X′,0,255). ⑥ Repeat ①-⑤, that is, the original image is used as input to calculate the reverse gradient once, and after the reverse gradient is superimposed on the original image, the generated perturbed input image can still be used as the input of the initial convolution classification network, and the reverse gradient is continued to be obtained through back propagation, and it is continued to be superimposed on the perturbed input image, and the cycle is repeated until the pre-set gradient amplitude or the upper limit of the iteration round is met to obtain the final perturbed input image. This embodiment combines the PGD algorithm and back propagation, and uses gradient information to maximize the loss function of the classifier to generate a perturbed input image that is visually very similar to the original image but can deceive the classifier, so that the perturbed input image is closer to the real attack and defense scene, and the generation efficiency of the perturbed input image is improved, thereby improving the robustness of the target convolution classification network generated according to the perturbed input image in response to attacks.

[0058] S102, forward propagating the perturbed input image based on the initial convolution classification network to determine the mask image of each convolution kernel.

[0059] Specifically, S102: forward propagating the perturbed input image based on the initial convolution classification network to determine the mask image of each convolution kernel, including steps S1021 to S1024:

[0060] S1021, forward propagating the perturbed input image based on the initial convolution classification network to obtain the output feature map of each convolution kernel.

[0061] In this embodiment, during the forward propagation of the perturbed input image by the initial convolution classification network, each convolution kernel slides (convolve) in different areas of the perturbed input image, and the area of ​​action of each convolution kernel (i.e., the local receptive field) causes it to obtain different responses at different positions. Therefore, even in areas with similar input image content, the output feature maps of each convolution kernel will be different due to the differences in the parameters of each convolution kernel and the different positions of the convolution operation. The model parameters of the initial convolution classification network are essentially the weights and biases of the convolution kernels. Therefore, during the forward propagation of the perturbed input image by the initial convolution classification network, this embodiment can clearly and systematically obtain the output feature maps of each convolution kernel based on the model parameters of the initial convolution classification network.

[0062] S1022, perform inter-layer weighted accumulation on the output feature maps of each convolution kernel of different convolution layers to obtain the accumulated feature map attention of each convolution kernel.

[0063] Specifically, S1022: performing inter-layer weighted accumulation on the output feature maps of each convolution kernel of different convolution layers to obtain the cumulative feature map attention of each convolution kernel, including: calculating the cumulative feature map attention of each convolution kernel according to formula (1):

[0064]

[0065] (1), where x represents the perturbed input image, Represents the output feature map of the i-th convolution kernel under the l-th convolution layer, l = 1, 2, ..., N l , i=1,2,…,n l , Up(·) refers to the bilinear upsampling function, represents the weight of the lth convolutional layer, Represents the cumulative feature map attention of the i-th convolution kernel under the current convolution layer.

[0066] In this embodiment, due to the multiple downsampling operations of the initial convolution classification network, the resolution of the output feature map of the convolution kernel is usually inconsistent with the perturbed input image. Therefore, the prior art usually adopts a bilinear interpolation transformation to ensure the spatial scale alignment of the initial convolution classification network. However, as the target scale ratio to be transformed increases, the output feature map of the deep layer of the initial convolution classification network loses attention to the details after conversion. Therefore, when evaluating parameters, the importance scores of the deep parameters of the initial convolution classification network are extremely low, almost 0, and difficult to compare.

[0067] In order to avoid the problem of feature map attention disappearance caused by layer depth, this embodiment obtains the lost detail attention from the output feature map of the shallow layer, that is, the output feature maps of each convolution kernel of different convolution layers are weightedly accumulated between layers, specifically: ① Obtain the output feature map of the target convolution kernel of the target convolution layer, and the output feature map of the convolution kernel of all convolution layers before the target convolution layer, where the target convolution layer and the target convolution kernel are predefined. For example: Given a perturbed input image with a classification label of y It is the output feature map of all convolution kernels of the lth convolution layer. ② The output feature maps of all convolution kernels of the same convolution layer before the target convolution layer are averaged to obtain the mean feature map of all convolution layers before the target convolution layer, that is, the shallow feature map. ③ Taking e as the base and the number of convolution layers as the exponent, the weight values ​​of the target convolution layer and all convolution layers before the target convolution layer are calculated, and the weight values ​​of the target convolution layer and all convolution layers before the target convolution layer are normalized. ④ Multiply the mean feature map of all convolution layers before the target convolution layer with the corresponding weight value, multiply the output feature map of the target convolution layer with the corresponding weight value, sum the two multiplication results, and use the sum result to correct the output feature map of the target convolution kernel of the target convolution layer. Compared with the prior art that directly adopts The cumulative feature map attention of each convolution kernel is calculated. This embodiment uses the output feature maps of all convolution kernels before the current convolution layer for inter-layer weighted accumulation, obtains the lost detail attention from the output feature maps of the shallow layer, solves the problem of feature map attention disappearance caused by layer depth, and thus better evaluates the importance of each convolution kernel parameter.

[0068] It should be noted that the bilinear upsampling function Up(·) is a two-dimensional interpolation method used to estimate the value of an unknown point. The principle is to use the values ​​of four adjacent vertices in a known grid to estimate through linear interpolation. First, on the two horizontal lines in the small grid where the target point is located, linear interpolation is performed on the two known values ​​on the two horizontal lines to obtain two intermediate values; then linear interpolation is performed again between the two intermediate values ​​to obtain the final estimated value of the target point. The bilinear upsampling function Up(·) gradually narrows the interpolation range and comprehensively considers the influence of the four known vertices. It can effectively smooth the transition and is often used for image scaling and smoothing.

[0069] S1023, performing maximum-minimum normalization on the accumulated feature map attention of each convolution kernel of the same convolution layer to obtain an input mask of each convolution kernel.

[0070] Specifically, S1023: performing maximum-minimum normalization on the cumulative feature map attention of each convolution kernel of the same convolution layer to obtain the input mask of each convolution kernel, including: calculating the input mask of each convolution kernel according to formula (2):

[0071]

[0072] in, N k *n k The cumulative feature map attention set of convolution kernels, Represents the input mask of the i-th convolution kernel under the current convolution layer.

[0073] In this embodiment, the cumulative feature map of the convolution kernel is focused becomes the input mask of the convolution kernel, usually the accumulated feature map attention The linear mapping becomes a value in the range of 0-1. However, directly using the maximum-minimum normalization method to pay attention to the cumulative feature maps of each convolution kernel will change the amount of information carried by different convolution kernels in the same convolution layer. For example: a convolution kernel whose output feature map value distribution range is near 0 carries very little information, but after direct mapping with maximum-minimum normalization, some of the original values ​​will be mapped to an interval close to 1, and the high values ​​mapped to the interval close to 1 will form a high attention area in the perturbed input image. Therefore, the attention score of the convolution kernel with lower importance will be mistakenly increased. In order to avoid mistakenly increasing the attention score of the convolution kernel with lower importance, this embodiment takes into account that the parameter importance comparison is usually only compared on different convolution kernels of the target convolution layer, and the cumulative feature map attention of all convolution kernels of the same convolution layer, such as: the cumulative feature map attention of all convolution kernels of the lth convolution layer. By performing maximum-minimum normalization, the input mask of each convolution kernel is obtained, which improves the recognition accuracy of the response strength of the output feature map of the convolution kernel, and thus improves the effectiveness of using the response strength as a guide for the subsequent compression of the initial convolution classification network.

[0074] S1024, perform dot product of the input mask of each convolution kernel and the perturbed input image to obtain a mask image of each convolution kernel.

[0075] In this embodiment, the cumulative feature map of the convolution kernel is focused After becoming the input mask of the convolution kernel, the input mask of the convolution kernel is multiplied by the perturbed input image to generate a mask image that conforms to the distribution of the convolution kernel feature focus area.

[0076] S103, based on the mask images of each convolution kernel, determine the removed convolution kernel of the initial convolution classification network, and compress the initial convolution classification network to obtain the target convolution classification network.

[0077] Specifically, based on the mask images of the respective convolution kernels, the removal convolution kernel of the initial convolution classification network is determined, including steps S1031 to S1034:

[0078] S1031, input the mask images of each convolution kernel into the initial convolution classification network to obtain the classification result of the mask image.

[0079] S1032, calculating the classification confidence of the mask image according to the classification result of the mask image and the corresponding classification label.

[0080] In this embodiment, after generating a mask image that conforms to the distribution of the convolution kernel feature focus area, the mask image of each convolution kernel is input into the initial convolution classification network, and the classification result of the mask image of each convolution kernel is output. y (.), the classification result of the mask image and the corresponding classification label y, directly calculate the classification confidence Softmax of the mask image. If there are more than one original image and there are K original images, then according to the formula Calculate the classification confidence Softmax of the mask image, where x k represents the perturbed input image generated based on the kth original image, y k Represents the classification label corresponding to the kth original image.

[0081] S1033, sorting the convolution kernels of the same convolution layer based on the classification confidence of the mask image.

[0082] S1034, determining the removed convolution kernel of the initial convolution classification network according to the preset compression rate and sorting result.

[0083] In this embodiment, the convolution kernels are sorted according to the classification confidence, and the sorting result is used as the convolution kernel importance evaluation result, that is, the convolution kernel with high classification confidence is the convolution kernel with high importance, and the convolution kernel with low classification confidence is the redundant convolution kernel. The redundant convolution kernel is determined to be removed.

[0084] Specifically, compressing the initial convolution classification network to obtain the target convolution classification network includes steps S1035 to S1038:

[0085] S1035, remove the convolution kernel of the initial convolution classification network to obtain a compressed convolution classification network.

[0086] In this embodiment, neural network parameter pruning removes redundant and low-information parameters from the pre-trained neural network model, thereby fine-tuning the structure of the neural network model, accelerating model calculation and reducing storage space. There are currently two representative neural network pruning methods: weight pruning and convolution kernel pruning. Weight pruning is to prune some weights within the convolution kernel to obtain a sparse weight matrix, while convolution kernel pruning is to remove the parameters contained in the entire convolution kernel from the neural network.

[0087] It should be noted that weight pruning usually introduces unstructured sparse connections, which will cause irregular hardware memory access in the actual calculation process, and will affect the computational efficiency of the network. Therefore, the neural network compression method based on structured pruning is more effective. For example, the pruning index based on the one-norm of the convolution kernel weight takes into account that the smaller the one-norm, the closer the output feature map of the convolution kernel is to zero, the less information the output feature map has, and the smaller the impact on network performance; the pruning method based on Taylor expansion links the importance of the convolution kernel with the network accuracy, and Taylor expands the change in accuracy loss after the convolution kernel is removed from the neural network to approximate the contribution value of each convolution kernel to the convergence of the loss function, which can be used to evaluate the importance of the convolution kernel; the pruning method based on reconstruction error takes the difference between the output of the lower layer of the target convolution layer after the parameter is removed and the original output. The smaller the difference, the lower the importance of the convolution kernel; the pruning algorithm based on the amount of feature information establishes a connection between the output feature map of the convolution kernel and the importance of the parameters, and judges the importance by the amount of information contained in the feature map.

[0088] S1036, input the original image into the compressed convolution classification network for back propagation, and generate a perturbation input on the original image based on an iterative gradient attack algorithm to obtain a second perturbation vector.

[0089] In this embodiment, the perturbation vector refers to a reverse gradient set used to be superimposed on the original image to generate a perturbed input image. For example, the perturbation input is generated on the original image for three iterations, and the reverse gradient includes The perturbation vector is

[0090] S1037, synthesizing the first perturbation vector and the second perturbation vector to obtain a third perturbation vector, wherein the first perturbation vector comes from the original image input to the initial convolution classification network for back propagation, and is obtained when a perturbation input is generated on the original image based on an iterative gradient attack algorithm.

[0091] In this embodiment, in the process of pruning the convolution kernel parameters, that is, removing the convolution kernel of the initial convolution classification network to obtain the compressed convolution classification network, the convolution kernel parameters of the initial convolution classification network, that is, the model parameters, are always retained, and then the original image can be input into the compressed convolution classification network for back propagation, and based on the iterative gradient attack algorithm, a perturbation input is generated on the original image to obtain the second perturbation vector. In this embodiment, the original image is input into the initial convolution classification network for back propagation, and based on the iterative gradient attack algorithm, a perturbation input is generated on the original image. After obtaining the perturbation input image, the first perturbation vector can also be calculated.

[0092] Specifically, S1037: synthesizing the first disturbance vector and the second disturbance vector to obtain a third disturbance vector, including: calculating the third disturbance vector according to formula (3):

[0093]

[0094] Among them, t iter represents the current training iteration number of the compressed convolutional classification network, T iter represents the total number of training iterations of the compressed convolutional classification network, represents the second perturbation vector, represents the first perturbation vector, represents the parameters of the compressed convolutional classification network, represents the parameters of the initial convolutional classification network, and x represents the perturbed input image.

[0095] In this embodiment, for the neural network pruning task with a given compression rate, as the neural network model gets closer and closer to the expected compression structure, the convolution kernel parameters for robustly extracting target features will inevitably be removed, and the anti-disturbance stability of the neural network model will also weaken. In addition, the disturbance vector generated by the PGD algorithm is only related to the corresponding convolution classification network structure parameters, and is generated by the number of iterations and the disturbance amplitude control. Therefore, it is impossible to ensure that the second disturbance vector generated based on the compressed convolution classification network covers the anti-disturbance space of the initial convolution classification network. In order to restore the robustness of the neural network model, this embodiment performs a weighted summation of the first disturbance vector and the second disturbance vector to obtain a third disturbance vector. The third disturbance vector is superimposed on the original image to generate a perturbed input image for subsequent teacher-student joint adversarial training.

[0096] The third perturbation vector is obtained by weighted summing the first perturbation vector and the second perturbation vector. Specifically, ① obtain the current training iteration number and the preset total iteration number of the compressed convolution classification network; ② take the ratio of the current training iteration number to the total iteration number and input The weight value of the second perturbation vector is obtained from the function, that is, ③ As the weight value of the first disturbance vector; ④ The third disturbance vector = the weight value of the first disturbance vector * the first disturbance vector + the weight value of the second disturbance vector * the second disturbance vector.

[0097] It should be noted that and It can reflect the progressive synthesis method of the third perturbation vector. In the subsequent teacher-student joint adversarial training, the perturbation vector is mainly based on the convolution kernel parameters of the compressed convolution classification network. As the number of training iterations increases, the third perturbation vector gradually shifts toward the perturbation space of the original convolution classification network.

[0098] S1038, based on the third perturbation vector, the initial convolution classification network and the compressed convolution classification network, perform teacher-student joint adversarial training to obtain a target convolution classification network.

[0099] In this embodiment, after the third perturbation vector is superimposed on the original image to generate a perturbed input image, the compressed convolution classification network is used as the student model and the initial convolution classification network is used as the teacher model to perform teacher-student joint adversarial training. In this embodiment, the third perturbation vector synthesized by the first perturbation vector and the second perturbation vector is used for the teacher-student joint adversarial training data of the initial convolution classification network and the compressed convolution classification network, the perturbation space of the original convolution classification network is retained, and multiple perturbation vectors are generated and combined. The trained target convolution classification network can more effectively identify and resist adversarial attacks, so that the target convolution classification network can achieve higher computational efficiency and smaller model volume without losing performance.

[0100] It should be noted that in order to reduce the time consumption of compression, a one-time pruning scheme is adopted, that is, the process of determining the removal of convolution kernels of the initial convolution classification network, compressing the initial convolution classification network, pruning and performing teacher-student joint adversarial training to obtain the target convolution classification network is performed only once.

[0101] The present embodiment provides a compression method for a classification network. By using a perturbed input image that conforms to the distribution characteristics of data collected in a real environment and has disturbances as a training set for an initial convolution classification network, the training set is made closer to the actual use situation, and the adaptability of the convolution classification network to various disturbances that may be encountered in real scenes is enhanced, and the robustness of the convolution classification network is improved. During the forward propagation of the perturbed input image by the initial convolution classification network, the feature focus area of ​​each convolution kernel in the perturbed input image is obtained, and a mask image of each convolution kernel is generated. The mask image helps to visualize the focus of the convolution kernel on a specific feature, so that subsequent It becomes more intuitive to evaluate the contribution of convolution kernels. Then, the mask images of each convolution kernel are classified through the initial convolution classification network. The importance of each convolution kernel is evaluated according to the classification results, so as to accurately identify the convolution kernels with low importance and can be removed in the initial convolution classification network, and then the initial convolution classification network is compressed in a targeted manner, and the optimal compression structure of the classification network with strong recognition robustness and anti-disturbance is quickly generated, so as to realize intelligent and effective compression of the classification network, improve the generation efficiency and anti-disturbance of the optimal compression structure of the classification network, achieve a balance between robustness and real-time performance of the optimal compression structure of the classification network, and improve the deployment availability of the optimal compression structure of the classification network. In addition, the PGD algorithm and back propagation are combined, and the gradient information is used to maximize the loss function of the classifier to generate a perturbed input image that is visually very similar to the original image but can deceive the classifier, so that the perturbed input image is closer to the real attack and defense scene, and the generation efficiency of the perturbed input image is improved, thereby improving the robustness of the target convolution classification network generated according to the perturbed input image in response to attacks. By using the output feature maps of all convolution kernels before the current convolution layer for inter-layer weighted accumulation, the lost detail attention is obtained from the output feature maps of the shallow layer, solving the problem of feature map attention disappearance caused by the depth of the layer, and thus better evaluating the importance of each convolution kernel parameter. Considering that the comparison of parameter importance is usually only performed on different convolution kernels of the target convolution layer, the cumulative feature map attention of all convolution kernels in the same convolution layer, such as the cumulative feature map attention of all convolution kernels in the lth convolution layer Perform maximum-minimum normalization to obtain the input mask of each convolution kernel, improve the recognition accuracy of the response strength of the output feature map of the convolution kernel, and then improve the effectiveness of using the response strength as the basis for subsequent compression of the initial convolution classification network. The third perturbation vector synthesized by the first perturbation vector and the second perturbation vector is used for the teacher-student joint adversarial training data of the initial convolution classification network and the compressed convolution classification network, retaining the perturbation space of the original convolution classification network, generating and combining multiple perturbation vectors, and the trained target convolution classification network can more effectively identify and resist adversarial attacks, so that the target convolution classification network can achieve higher computational efficiency and smaller model size without losing performance.

[0102] Embodiment 2:

[0103] like Figure 2 As shown, this embodiment provides a classification network compression method, and the classification network compression method includes:

[0104] S201, obtaining an initial convolution classification network, an original image and a corresponding classification label, inputting the original image into the initial convolution classification network for back propagation, and generating a perturbation input on the original image based on an iterative gradient attack algorithm to obtain a perturbation input image.

[0105] In this embodiment, the original image and the corresponding classification label are Figure 2 The original image training set and the original training dataset in .

[0106] S202, based on the initial convolution classification network, forward propagation is performed on the perturbed input image to obtain the output feature map of each convolution kernel, inter-layer weighted accumulation is performed on the output feature maps of each convolution kernel of different convolution layers to obtain the cumulative feature map attention of each convolution kernel, and the cumulative feature map attention of each convolution kernel of the same convolution layer is normalized to the maximum-minimum in the layer to obtain the input mask of each convolution kernel.

[0107] In this embodiment, the output feature map is Figure 2 The feature map in .

[0108] S203, perform dot product of the input mask of each convolution kernel and the perturbed input image to obtain the mask image of each convolution kernel, input the mask image of each convolution kernel into the initial convolution classification network to obtain the classification result of the mask image, and calculate the classification confidence of the mask image according to the classification result of the mask image and the corresponding classification label.

[0109] In this embodiment, the mask image of each convolution kernel is Figure 2 Mapped image in .

[0110] S204, based on the classification confidence of the mask image, sort the convolution kernels of the same convolution layer, determine the removal convolution kernel of the initial convolution classification network according to the preset compression rate and sorting result, remove the removal convolution kernel of the initial convolution classification network, obtain the compressed convolution classification network, input the original image into the compressed convolution classification network for back propagation, and based on the iterative gradient attack algorithm, generate a perturbation input on the original image to obtain a second perturbation vector, synthesize the first perturbation vector and the second perturbation vector to obtain a third perturbation vector, wherein the first perturbation vector comes from the original image input into the initial convolution classification network for back propagation, and based on the iterative gradient attack algorithm, generates a perturbation input on the original image, and obtains it when the third perturbation vector, the initial convolution classification network and the compressed convolution classification network are used for teacher-student joint adversarial training to obtain the target convolution classification network.

[0111] In this embodiment, the convolution kernel is removed. Figure 2 The parameters with low importance in the compressed convolutional classification network are Figure 2 The compression model in .

[0112] The present embodiment provides a compression method for a classification network. By using a perturbed input image that conforms to the distribution characteristics of data collected in a real environment and has disturbances as a training set for an initial convolution classification network, the training set is made closer to the actual use situation, and the adaptability of the convolution classification network to various disturbances that may be encountered in real scenes is enhanced, and the robustness of the convolution classification network is improved. During the forward propagation of the perturbed input image by the initial convolution classification network, the feature focus area of ​​each convolution kernel in the perturbed input image is obtained, and a mask image of each convolution kernel is generated. The mask image helps to visualize the focus of the convolution kernel on a specific feature, so that subsequent It becomes more intuitive to evaluate the contribution of the convolution kernel. The mask image of each convolution kernel is then classified through the initial convolution classification network. The importance of each convolution kernel is evaluated according to the classification results, so as to accurately identify the convolution kernels with low importance and can be removed in the initial convolution classification network. Then, the initial convolution classification network is compressed in a targeted manner, and the optimal compression structure of the classification network with strong recognition robustness and anti-disturbance is quickly generated, so as to realize intelligent and effective compression of the classification network, improve the generation efficiency and anti-disturbance of the optimal compression structure of the classification network, achieve a balance between the robustness and real-time performance of the optimal compression structure of the classification network, and improve the deployment availability of the optimal compression structure of the classification network.

[0113] Embodiment 3:

[0114] like Figure 3 As shown, this embodiment also provides a classification network compression method. The classification network compression method includes:

[0115] Step 1: obtain the initial convolution classification network, the original image and the corresponding classification label, input the original image into the initial convolution classification network for back propagation, and generate a perturbation input on the original image based on the iterative gradient attack algorithm to obtain a perturbation input image.

[0116] In this embodiment, the initial convolution classification network is Figure 3 The pre-trained network in , the classification label is Figure 3 One-hot tags in .

[0117] Step 2: Based on the initial convolution classification network, the perturbed input image is forward propagated to obtain the output feature map of each convolution kernel, and the output feature map of each convolution kernel of different convolution layers is weightedly accumulated between layers to obtain the cumulative feature map attention of each convolution kernel. The cumulative feature map attention of each convolution kernel of the same convolution layer is normalized to the maximum and minimum in the layer to obtain the input mask of each convolution kernel.

[0118] In this embodiment, the output feature map is Figure 3 The convolution kernel feature map in .

[0119] Step 3: Multiply the input mask of each convolution kernel and the perturbed input image to obtain the mask image of each convolution kernel, input the mask image of each convolution kernel into the initial convolution classification network, obtain the classification result of the mask image, and calculate the classification confidence of the mask image according to the classification result of the mask image and the corresponding classification label.

[0120] In this embodiment, the classification confidence is Figure 3 The confidence score at the label in .

[0121] Step 4. Based on the classification confidence of the mask image, sort the convolution kernels of the same convolution layer, determine the removed convolution kernel of the initial convolution classification network according to the preset compression rate and sorting result, remove the removed convolution kernel of the initial convolution classification network, and obtain the compressed convolution classification network. Input the original image into the compressed convolution classification network for back propagation, and generate a perturbation input on the original image based on the iterative gradient attack algorithm to obtain a second perturbation vector, synthesize the first perturbation vector and the second perturbation vector to obtain a third perturbation vector, wherein the first perturbation vector comes from the original image input into the initial convolution classification network for back propagation, and based on the iterative gradient attack algorithm, it is obtained when the perturbation input is generated on the original image. Based on the third perturbation vector, the initial convolution classification network and the compressed convolution classification network, perform teacher-student joint adversarial training to obtain the target convolution classification network.

[0122] In this embodiment, the compressed convolution classification network is Figure 3 In the compressed network, the first perturbation vector is Figure 3 The perturbation vector 1 in the second perturbation vector is Figure 3 The perturbation vector 2 in the third perturbation vector is Figure 3 The perturbation vector 3 in the teacher-student joint adversarial training is Figure 3 Combined resistance weight training in .

[0123] The present embodiment provides a compression method for a classification network. By using a perturbed input image that conforms to the distribution characteristics of data collected in a real environment and has disturbances as a training set for an initial convolution classification network, the training set is made closer to the actual use situation, and the adaptability of the convolution classification network to various disturbances that may be encountered in real scenes is enhanced, and the robustness of the convolution classification network is improved. During the forward propagation of the perturbed input image by the initial convolution classification network, the feature focus area of ​​each convolution kernel in the perturbed input image is obtained, and a mask image of each convolution kernel is generated. The mask image helps to visualize the focus of the convolution kernel on a specific feature, so that subsequent It becomes more intuitive to evaluate the contribution of the convolution kernel. The mask image of each convolution kernel is then classified through the initial convolution classification network. The importance of each convolution kernel is evaluated according to the classification results, so as to accurately identify the convolution kernels with low importance and can be removed in the initial convolution classification network. Then, the initial convolution classification network is compressed in a targeted manner, and the optimal compression structure of the classification network with strong recognition robustness and anti-disturbance is quickly generated, so as to realize intelligent and effective compression of the classification network, improve the generation efficiency and anti-disturbance of the optimal compression structure of the classification network, achieve a balance between the robustness and real-time performance of the optimal compression structure of the classification network, and improve the deployment availability of the optimal compression structure of the classification network.

[0124] Embodiment 4:

[0125] like Figure 4 As shown, this embodiment also provides a compression device for a classified network. The compression device for a classified network includes:

[0126] The synthesis module is used to obtain the initial convolution classification network, the original image and the corresponding classification label, input the original image into the initial convolution classification network for back propagation, and generate a perturbation input on the original image based on the iterative gradient attack algorithm to obtain a perturbation input image.

[0127] The alignment module is used to forward propagate the perturbed input image based on the initial convolution classification network to obtain the output feature map of each convolution kernel, perform inter-layer weighted accumulation on the output feature maps of each convolution kernel of different convolution layers to obtain the cumulative feature map attention of each convolution kernel, perform maximum-minimum normalization on the cumulative feature map attention of each convolution kernel of the same convolution layer, and obtain the input mask of each convolution kernel.

[0128] The evaluation module is used to perform the dot product of the input mask of each convolution kernel and the perturbed input image to obtain the mask image of each convolution kernel, input the mask image of each convolution kernel into the initial convolution classification network to obtain the classification result of the mask image, and calculate the classification confidence of the mask image according to the classification result of the mask image and the corresponding classification label.

[0129] A fine-tuning module is used to sort the convolution kernels of the same convolution layer based on the classification confidence of the mask image, determine the removed convolution kernel of the initial convolution classification network according to the preset compression rate and sorting result, remove the removed convolution kernel of the initial convolution classification network, and obtain a compressed convolution classification network, input the original image into the compressed convolution classification network for back propagation, and generate a perturbation input on the original image based on an iterative gradient attack algorithm to obtain a second perturbation vector, synthesize the first perturbation vector and the second perturbation vector to obtain a third perturbation vector, wherein the first perturbation vector comes from the original image input into the initial convolution classification network for back propagation, and based on the iterative gradient attack algorithm, it is obtained when the perturbation input is generated on the original image, and based on the third perturbation vector, the initial convolution classification network and the compressed convolution classification network, perform teacher-student joint adversarial training to obtain the target convolution classification network.

[0130] The present embodiment provides a compression device for a classification network. By using a perturbed input image that conforms to the distribution characteristics of data collected in a real environment and has disturbances as a training set for an initial convolution classification network, the training set is closer to the actual use situation, and the adaptability of the convolution classification network to various disturbances that may be encountered in real scenes is enhanced, and the robustness of the convolution classification network is improved. During the forward propagation of the perturbed input image by the initial convolution classification network, the feature focus area of ​​each convolution kernel in the perturbed input image is obtained, and a mask image of each convolution kernel is generated. The mask image helps to visualize the focus of the convolution kernel on specific features, making the subsequent evaluation of the contribution of the convolution kernel more intuitive. The mask image of each convolution kernel is then classified through the initial convolution classification network, and the importance of each convolution kernel is evaluated according to the classification result, so as to accurately identify the convolution kernels with low importance and can be removed in the initial convolution classification network, and then the initial convolution classification network is compressed in a targeted manner, so as to realize intelligent and effective compression of the classification network, improve the generation efficiency and anti-disturbance of the optimal compression structure of the classification network, achieve a balance between the robustness and real-time performance of the optimal compression structure of the classification network, and improve the deployment availability of the optimal compression structure of the classification network.

[0131] Embodiment 5:

[0132] like Figure 5 As shown, this embodiment also provides a compression device for a classified network. The compression device 300 for a classified network includes: a receiver 301 , a transmitter 302 , a processor 303 , a graphics processor 304 and a memory 305 .

[0133] The receiver 301 is used to receive instructions and data.

[0134] The transmitter 302 is used to send instructions and data.

[0135] The memory 305 is used to store computer-executable instructions.

[0136] The processor 303 and the graphics processor 304 are used to execute the computer execution instructions stored in the memory 305 to implement the various steps performed by the compression method of the classification network in Example 1, Example 2 and Example 3.

[0137] Optionally, the graphics processor 304 may be independent or integrated with the processor 303 .

[0138] Embodiment 6:

[0139] like Figure 6 As shown, this embodiment also provides a compression device for a classification network, including an acquisition module 61, a determination module 62 and a compression module 63, the acquisition module 61 is used to acquire an initial convolution classification network and a perturbed input image, wherein the initial convolution classification network includes at least one convolution layer, and the convolution layer includes multiple convolution kernels, the determination module 62 is connected to the acquisition module 61, and is used to forward propagate the perturbed input image based on the initial convolution classification network, and determine the mask image of each convolution kernel, the compression module 63 is connected to the determination module 62, and is used to determine the removed convolution kernel of the initial convolution classification network based on the mask image of each convolution kernel, and compress the initial convolution classification network to obtain the target convolution classification network.

[0140] Specifically, the acquisition module 61 includes: an acquisition unit 611 and a generation unit 612. The acquisition unit 611 is used to acquire the initial convolution classification network, the original image and the corresponding classification label. The generation unit 612 is used to input the original image into the initial convolution classification network for back propagation, and based on the iterative gradient attack algorithm, generate a perturbation input on the original image to obtain a perturbation input image.

[0141] Specifically, the determination module 62 includes: a first forward propagation unit 621, a weighted accumulation unit 622, a normalization unit 623 and a dot multiplication unit 624. The first forward propagation unit 621 is used to perform forward propagation on the perturbed input image based on the initial convolution classification network to obtain the output feature map of each convolution kernel. The weighted accumulation unit 622 is used to perform inter-layer weighted accumulation on the output feature maps of each convolution kernel of different convolution layers to obtain the cumulative feature map attention of each convolution kernel. The normalization unit 623 is used to perform the maximum-minimum normalization in the layer on the cumulative feature map attention of each convolution kernel of the same convolution layer to obtain the input mask of each convolution kernel. The dot multiplication unit 624 is used to dot-multiply the input mask of each convolution kernel and the perturbation input image to obtain the mask image of each convolution kernel.

[0142] Specifically, the weighted accumulation unit 622 includes: a first calculation subunit, configured to calculate the cumulative feature map attention of each convolution kernel according to formula (1):

[0143]

[0144] (1), where x represents the perturbed input image, Represents the output feature map of the i-th convolution kernel under the l-th convolution layer, l = 1, 2, ..., N l , i=1,2,…,n l , Up(·) refers to the bilinear upsampling function, represents the weight of the lth convolutional layer, Represents the cumulative feature map attention of the i-th convolution kernel under the current convolution layer.

[0145] Specifically, the normalization unit 623 includes: a second calculation subunit, configured to calculate the input mask of each convolution kernel according to formula (2):

[0146] in, N l *n l The cumulative feature map attention set of convolution kernels, Represents the input mask of the i-th convolution kernel under the current convolution layer.

[0147] Specifically, the compression module 63 includes: a classification unit 631, a calculation unit 632, a sorting unit 633 and a determination unit 634. The classification unit 631 is used to input the mask images of each convolution kernel into the initial convolution classification network to obtain the classification result of the mask image. The calculation unit 632 is used to calculate the classification confidence of the mask image according to the classification result of the mask image and the corresponding classification label. The sorting unit 633 is used to sort the convolution kernels of the same convolution layer based on the classification confidence of the mask image. The determination unit 634 is used to determine the removed convolution kernel of the initial convolution classification network according to the preset compression rate and the sorting result.

[0148] Optionally, the compression module 63 also includes: a removal unit 635, a second back propagation unit 636, a synthesis unit 637 and a training unit 638, the removal unit 635 is used to remove the convolution kernel of the initial convolution classification network to obtain a compressed convolution classification network, the second back propagation unit 636 is used to input the original image into the compressed convolution classification network for back propagation, and based on the iterative gradient attack algorithm, generate a perturbation input on the original image to obtain a second perturbation vector, the synthesis unit 637 is used to synthesize the first perturbation vector and the second perturbation vector to obtain a third perturbation vector, wherein the first perturbation vector comes from the original image input into the initial convolution classification network for back propagation, and based on the iterative gradient attack algorithm, the perturbation input is generated on the original image, and the training unit 638 is used to perform teacher-student joint adversarial training based on the third perturbation vector, the initial convolution classification network and the compressed convolution classification network to obtain a target convolution classification network.

[0149] Specifically, the synthesis unit 637 includes: a third calculation subunit, configured to calculate a third disturbance vector according to formula (3):

[0150]

[0151] Among them, t iter represents the current training iteration number of the compressed convolutional classification network, T iter represents the total number of training iterations of the compressed convolutional classification network, represents the second perturbation vector, represents the first perturbation vector, represents the parameters of the compressed convolutional classification network, represents the parameters of the initial convolutional classification network, and x represents the perturbed input image.

[0152] It can be understood that the compression device of a classification network provided above executes the compression method of the classification network corresponding to the embodiment 1 provided above. Therefore, the beneficial effects that can be achieved can refer to the beneficial effects of the scheme corresponding to the compression method of the classification network in the embodiment 1 above, and will not be repeated here.

[0153] Embodiment 7:

[0154] This embodiment also provides an electronic device, including a memory and a processor, wherein the memory stores a computer program, and the processor is configured to run the computer program to implement the classification network compression method in the above-mentioned embodiments 1, 2 and 3.

[0155] Embodiment 8:

[0156] This embodiment also provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the compression method of the classification network in the above-mentioned embodiments 1, 2 and 3 is implemented.

[0157] It is to be understood that the above embodiments are merely exemplary embodiments used to illustrate the principles of the present invention, but the present invention is not limited thereto. For those of ordinary skill in the art, various modifications and improvements can be made without departing from the spirit and essence of the present invention, and these modifications and improvements are also considered to be within the scope of protection of the present invention.

Claims

1. A compression method for a classification network, characterized in that: include: Acquire an initial convolution classification network and a perturbed input image, wherein the initial convolution classification network includes at least one convolution layer, and the convolution layer includes a plurality of convolution kernels; Based on the initial convolution classification network, the perturbed input image is forward propagated to determine the mask image of each convolution kernel; Based on the mask images of each convolution kernel, the removed convolution kernel of the initial convolution classification network is determined, and the initial convolution classification network is compressed to obtain the target convolution classification network.

2. The compression method of the classification network according to claim 1, characterized in that: The obtaining of the initial convolution classification network and the perturbed input image specifically includes: Get the initial convolution classification network, original image and corresponding classification label; The original image is input into the initial convolution classification network for back propagation, and based on the iterative gradient attack algorithm, a perturbation input is generated on the original image to obtain a perturbation input image.

3. The compression method of the classification network according to claim 1, characterized in that: The method of forward propagating the perturbed input image based on the initial convolution classification network to determine the mask image of each convolution kernel specifically includes: Based on the initial convolution classification network, the perturbed input image is forward propagated to obtain the output feature map of each convolution kernel; The output feature maps of each convolution kernel of different convolution layers are weightedly accumulated between layers to obtain the cumulative feature map attention of each convolution kernel; The accumulated feature map attention of each convolution kernel in the same convolution layer is normalized to the maximum and minimum in the layer to obtain the input mask of each convolution kernel; Multiply the input mask of each convolution kernel and the perturbed input image to obtain the mask image of each convolution kernel.

4. The compression method of the classification network according to claim 3, characterized in that: The output feature maps of each convolution kernel of different convolution layers are weightedly accumulated between layers to obtain the accumulated feature map attention of each convolution kernel, specifically including: According to formula (1), the cumulative feature map attention of each convolution kernel is calculated: Where x represents the perturbed input image, Represents the output feature map of the i-th convolution kernel under the l-th convolution layer, l = 1, 2, ..., N l , i=1,2,…,n l , Up(·) refers to the bilinear upsampling function, represents the weight of the lth convolutional layer, Represents the cumulative feature map attention of the i-th convolution kernel under the current convolution layer.

5. The compression method of the classification network according to claim 4, characterized in that: The cumulative feature map attention of each convolution kernel of the same convolution layer is subjected to the maximum-minimum normalization in the layer to obtain the input mask of each convolution kernel, specifically including: According to formula (2), the input mask of each convolution kernel is calculated: in, N l *n l The cumulative feature map attention set of convolution kernels, Represents the input mask of the i-th convolution kernel under the current convolution layer.

6. The compression method of the classification network according to claim 2, characterized in that: The step of determining the removal convolution kernel of the initial convolution classification network based on the mask image of each convolution kernel specifically includes: Inputting the mask images of each convolution kernel into the initial convolution classification network to obtain the classification result of the mask image; Calculating the classification confidence of the mask image according to the classification result of the mask image and the corresponding classification label; Based on the classification confidence of the mask image, sorting the convolution kernels of the same convolution layer; According to the preset compression rate and sorting results, the removal convolution kernel of the initial convolution classification network is determined.

7. The compression method of the classification network according to claim 2, characterized in that: The compressing the initial convolution classification network to obtain the target convolution classification network specifically includes: Remove the convolution kernel of the initial convolution classification network to obtain a compressed convolution classification network; The original image is input into the compressed convolution classification network for back propagation, and based on the iterative gradient attack algorithm, a perturbation input is generated on the original image to obtain a second perturbation vector; The first perturbation vector and the second perturbation vector are synthesized to obtain a third perturbation vector, wherein the first perturbation vector is obtained by inputting the original image into the initial convolution classification network for back propagation and generating a perturbation input on the original image based on an iterative gradient attack algorithm; Based on the third perturbation vector, the initial convolution classification network and the compressed convolution classification network, teacher-student joint adversarial training is performed to obtain the target convolution classification network.

8. The compression method of the classification network according to claim 7, characterized in that: The synthesizing the first disturbance vector and the second disturbance vector to obtain a third disturbance vector specifically includes: According to formula (3), calculate the third disturbance vector: Among them, t iter represents the current training iteration number of the compressed convolutional classification network, T iter represents the total number of training iterations of the compressed convolutional classification network, represents the second perturbation vector, represents the first perturbation vector, represents the parameters of the compressed convolutional classification network, represents the parameters of the initial convolutional classification network, and x represents the perturbed input image.

9. A compression device for a classification network, characterized in that: Including acquisition module, determination module and compression module, An acquisition module is used to acquire an initial convolution classification network and a perturbed input image, wherein the initial convolution classification network includes at least one convolution layer, and the convolution layer includes a plurality of convolution kernels. The determination module is connected to the acquisition module and is used to forward propagate the perturbed input image based on the initial convolution classification network to determine the mask image of each convolution kernel. The compression module is connected to the determination module and is used to determine the removed convolution kernels of the initial convolution classification network based on the mask images of each convolution kernel, and compress the initial convolution classification network to obtain the target convolution classification network.

10. An electronic device, characterized in that: The method comprises a memory and a processor, wherein the memory stores a computer program, and the processor is configured to run the computer program to implement a classification network compression method as described in any one of claims 1 to 8.

11. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, a compression method for a classification network as described in any one of claims 1 to 8 is implemented.

Images