A network layer pruning method for image classification based on Taylor expansion

By analyzing the importance of the BN layer based on the Taylor expansion method in the image classification network, layer pruning is solved, and the problem of insufficient channel pruning speed returns in the existing technology is achieved, and network compression and acceleration effects are achieved while ensuring accuracy , suitable for equipment with limited resources.

CN114037858BActive Publication Date: 2024-11-08NORTHWESTERN POLYTECHNICAL UNIV
View PDF 4 Cites 0 Cited by

Patent Information

Application Number
CN202111237007.1
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2021-10-24
Publication Date
2024-11-08
Estimated Expiration
2041-10-24

AI Technical Summary

Technical Problem

The existing network channel pruning method based on Taylor expansion only analyzes the convolution layer filter parameters or the scaling factor in the BN layer, resulting in the speed gain of channel pruning is not as good as that of layer pruning, and for networks with more layers. Not effective.

Method used

By calculating the importance of each BasicBlock in the pre-trained model, the Taylor expansion method is used to analyze the impact of the scaling factor and bias of the BN layer on the network loss function, and layer pruning is performed to achieve network compression and acceleration. The specific steps include iterative training to obtain gradient information, calculate the importance of the BN layer, normalize the global sorting, remove the lowest-important BasicBlock, form the network model after layer pruning, and retrain it to restore classification accuracy.

Benefits of technology

On the premise of ensuring classification accuracy, a high compression rate and computing speed improvement is achieved. It is suitable for terminal devices with limited resources, such as mobile phones. For example, the number of layers of ResNet-56 is reduced from 56 to 34 layers, the parameter quantity and The calculation volume has been significantly reduced, and the accuracy rate has increased from 93.21% to 93.35%.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN114037858B_ABST
    Figure CN114037858B_ABST
Patent Text Reader

Abstract

The present invention relates to a Taylor expansion-based image classification network layer pruning method, which belongs to the field of image processing and recognition technology. The scaling factor and bias factor of the BN layer are analyzed simultaneously, and the network layer with less influence on the network loss function is identified and eliminated based on the Taylor expansion method, thereby achieving the purpose of network layer pruning. It is applied to image classification tasks, and a higher compression rate can be achieved under the premise of ensuring the classification accuracy, while improving the operation speed of the model, which is convenient for its application in terminal mobile devices such as mobile phones.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The invention belongs to the technical field of image processing and recognition, and in particular relates to an image classification network layer pruning method based on Taylor expansion. Background Art

[0002] Deep convolutional neural networks have achieved great success in the field of machine vision, achieving recognition capabilities that surpass human levels. Thanks to the huge training data set and network scale, the neural network model can extract robust features that represent the target image through complex calculations, and then use them for classification and recognition. However, with the improvement of recognition accuracy, the amount of computation and calculation of the network model is also increasing, requiring huge storage space and computing resources, and the requirements for computing devices are getting higher and higher. The real-time performance of the calculation is difficult to guarantee, which limits its application in terminal devices such as mobile phones, cars, and satellites.

[0003] Parameter pruning is an effective model compression method that can achieve the effect of network slimming by removing redundant or unimportant parameters in the network. Zhang Jianming, Wang Wei, Lu Chaoquan, et al. (Traffic Sign Classification Algorithm Based on Compressed Convolutional Neural Network, Journal of Huazhong University of Science and Technology (Natural Science Edition), 2019, 47(01): 108-113) proposed a channel pruning method based on Taylor expansion. The change in the loss function caused by removing the feature map channel is obtained according to the Taylor expansion to determine the filter channel with less impact on network performance, and then perform channel pruning. This method performs Taylor expansion at the filter parameter of 0. Due to the large number of parameters in the filter, the result is easily affected by individual parameters. Considering that the convolutional layer in the network is generally followed by a batch normalization (BN) layer, and there are only two parameters in the BN layer to control the size of the output feature map, it is more suitable for Taylor expansion. On the other hand, the channel pruning method is not as effective as layer pruning in accelerating the network. For networks with more layers, layer pruning can achieve greater speed gains. Summary of the invention

[0004] Technical issues to be solved

[0005] The existing network channel pruning methods based on Taylor expansion only analyze the convolutional layer filter parameters or the scaling factors in the BN layer, and the speed gain of channel pruning is not as good as that of layer pruning. In order to avoid the shortcomings of the prior art, the present invention proposes an image classification network layer pruning method based on Taylor expansion.

[0006] Technical Solution

[0007] A Taylor expansion-based image classification network layer pruning method, characterized by the following steps:

[0008] Step 1: Perform an iterative training on the pre-trained model on the training set, and obtain the gradient information of each parameter in the network through error back propagation;

[0009] Step 2: Calculate the importance p of each BasicBlock in the network according to the following formula l :

[0010]

[0011]

[0012] In the formula, Represents the importance of the jth BN layer in the lth BasicBlock. and They are respectively i and β i When set to 0 The amount of change, and They are γ i and β i about The gradient of , l∈(1,2,…,L), L is the total number of BasicBlocks in the network; and They are the trainable scaling factor and bias in the BN layer respectively;

[0013] The p in the same resolution stage l Normalized to [0,1], that is:

[0014]

[0015] Where P l is the importance of the normalized BasicBlock, p max and p min In this stage, p l The maximum and minimum values ​​of ;

[0016] For BasicBlocks with downsampling layers, since they cannot be pruned due to their particularity, the importance of these BasicBlocks is set to 1;

[0017] Step 3: According to the compression rate r to be achieved, determine the number of BasicBlocks to be retained as l0 = rL, and the number of BasicBlocks to be removed as l1 = L-l0; the importance P of all BasicBlocks l Perform global sorting and remove the least important BasicBlocks to obtain the network model after layer pruning.

[0018] Step 4: Train the network model after layer pruning to restore the classification accuracy of the model and obtain the final network model.

[0019] A computer system, characterized in that it comprises: one or more processors, and a computer-readable storage medium for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement the above-mentioned method.

[0020] A computer-readable storage medium, characterized in that it stores computer-executable instructions, which are used to implement the above method when executed.

[0021] A computer program, characterized by comprising computer executable instructions, which are used to implement the above method when executed.

[0022] Beneficial Effects

[0023] The present invention proposes a Taylor expansion-based image classification network layer pruning method, which simultaneously analyzes the scaling factor and bias factor of the BN layer, identifies and removes the network layer with less influence on the network loss function based on the Taylor expansion method, thereby achieving the purpose of network layer pruning. Applying it to image classification tasks can achieve a higher compression rate while ensuring the classification accuracy, while improving the model's computing speed, making it easier to use in terminal mobile devices such as mobile phones.

[0024] The Taylor expansion-based image classification network layer pruning method designed by the present invention can effectively sort the importance of each network layer, and then remove the network layer with less importance, so as to achieve the effect of model compression and acceleration. For example, the number of layers of the compressed ResNet-56 network is reduced from 56 to 34, the number of parameters is reduced from 0.85M to 0.41M, the amount of calculation is reduced from 127M to 74.6M, the operation time is reduced from 6.614ms to 4.362ms, and the accuracy is increased from 93.21% to 93.35%. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] The drawings are only for the purpose of illustrating particular embodiments and are not to be considered limiting of the present invention. Like reference symbols denote like components throughout the drawings.

[0026] Figure 1 This is a flowchart of image classification network layer pruning based on Taylor expansion.

[0027] Figure 2 This is the normalized importance result graph of each BasicBlock in the ResNet-56 network. DETAILED DESCRIPTION

[0028] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention. In addition, the technical features involved in the various embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other.

[0029] Residual structures often exist in modern neural networks. Take ResNet as an example. It consists of multiple BasicBlocks. Each BasicBlock includes a main branch and a bypass branch, and its output is equal to the sum of the two branches. The main branch includes multiple convolutional layers and BN layers, and the bypass branch is an identity mapping, that is, the input is directly output. If you want to perform layer pruning, treat each BasicBlock as a whole. If the output of the main branch is very small, then you can remove the main branch and directly use the input as the output, that is, remove all the network layers in the BasicBlock. Therefore, the layer pruning problem can be converted into a problem of measuring the importance of the main branch output in the BasicBlock.

[0030] In the main branch, each convolutional layer is often followed by a BN layer, and the calculation process is:

[0031]

[0032] In the formula, is the input of the BN layer, that is, the output of the previous convolutional layer, is the output of the BN layer, n is the number of channels, h′ and w′ are the height and width of the feature map, respectively. is the mean of x in each training batch, is the standard deviation of x in each training batch, and are the trainable scaling factor and bias in the BN layer, respectively, and ε is a positive number that prevents the denominator from being zero.

[0033] In order to obtain the importance of a single BN layer, a method based on Taylor expansion is used to obtain each parameter γ=[γ1,γ2,…,γ n ] T and β=[β1,β2,…,β n ] T When they are set to 0, the network loss function The sum of the influences of , we get:

[0034]

[0035] In the formula, Represents the importance of the jth BN layer in the lth BasicBlock. and They are respectively i and β i When set to 0 The amount of change, and They are γ i and β i about The gradient can be obtained by back propagation during network training.

[0036] For a BasicBlock with m BN layers, since the layers are in series, the overall importance of the BasicBlock can be obtained by multiplication:

[0037]

[0038] Where l∈(1,2,…,L), L is the total number of BasicBlocks in the network.

[0039] Based on this, the present invention proposes a Taylor expansion-based image classification network layer pruning method, the process of which is as follows: Figure 1 The specific steps are as follows:

[0040] (1) Perform an iterative training on the pre-trained model on the training set and obtain the gradient information of each parameter in the network through error back propagation.

[0041] (2) According to formula (2) and formula (3), calculate the importance p of each BasicBlock in the network l .

[0042] As the network deepens, the image resolution decreases in stages, resulting in p l The sizes are not comparable, so a normalization method is needed to convert the p l Normalized to [0,1], that is:

[0043]

[0044] Where P l is the importance of the normalized BasicBlock, p max and p min In this stage, p l The maximum and minimum values ​​of .

[0045] For BasicBlocks with downsampling layers, since they cannot be pruned due to their particularity, the importance of these BasicBlocks is set to 1.

[0046] (3) According to the compression rate r to be achieved, the number of BasicBlocks to be retained is determined to be l0 = rL, and the number of BasicBlocks to be removed is determined to be l1 = L-l0. l Perform global sorting and remove the least important BasicBlocks to obtain the network model after layer pruning.

[0047] (4) The network model after layer pruning is trained to restore the classification accuracy of the model and obtain the final network model. This model has fewer layers than the original model, so it can achieve the purpose of network compression, and the reduction in the number of layers can bring about a significant speed improvement.

[0048] The specific implementation of the present invention is described below in conjunction with an example of image classification, but the technical content of the present invention is not limited to the scope described. The specific implementation includes the following steps:

[0049] (1) Build a convolutional neural network for image classification and construct an image dataset with a large number of training samples and labels.

[0050] (2) Perform an iterative training on the pre-trained model on the training set and obtain the gradient information of each parameter in the network through error back propagation.

[0051] (3) The importance p of each BasicBlock in the network is calculated using the Taylor expansion method of the present invention and the scaling factor and bias factor of the BN layer. l , after normalization, we get P l .

[0052] (4) Set the importance of all BasicBlocks to P l Perform global sorting and remove the 11 least important BasicBlocks according to the compression rate to be achieved to obtain the network model after layer pruning.

[0053] (5) The network model after layer pruning is trained on the data set to restore the classification accuracy of the model and obtain the final network model. Using this model for image classification can achieve the effect of network compression and acceleration while ensuring the classification accuracy.

[0054] The method of the present invention is used to perform layer pruning on the ResNet-56 network. Figure 2 is the normalized importance P of each BasicBlock in the network lResult graph. The compression rate is set to 0.6, that is, 60% of the network layers are retained, and the remaining 40% of the network layers are pruned. Table 1 shows the compression results of the ResNet-56 network layer pruning using the present invention. The original ResNet-56 model before compression contains 56 layers, 0.85M parameters and 127M calculations. The reasoning time for a single picture is 6.614ms, and the accuracy on CIFAR-10 is 93.21%. The model obtained by the compression method of the present invention has 34 layers, 0.41M parameters, 74.6M required calculations, 4.362ms for a single picture reasoning, and the accuracy is improved to 93.35%. This shows that the image classification network layer pruning method based on Taylor expansion proposed in the present invention can effectively remove unimportant layers in the network, while ensuring the accuracy of the network to achieve the effect of network compression and acceleration.

[0055] Table 1 Network compression results of the present invention

[0056] Evaluation indicators Number of layers Parameter quantity (M) Calculation Amount(M) Single picture time consumption (ms) Accuracy Before compression 56 0.85 127 6.614 93.21% After compression 34 0.41 74.6 4.362 93.35%

[0057] The above description is only a specific implementation mode of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can easily think of various equivalent modifications or substitutions within the technical scope disclosed in the present invention, and these modifications or substitutions should be included in the protection scope of the present invention.

Claims

1. A Taylor expansion-based image classification network layer pruning method, characterized in that Here are the steps: Step 1: Perform an iterative training on the pre-trained model on the training set, and obtain the gradient information of each parameter in the network through error back propagation; Step 2: Calculate the importance p of each BasicBlock in the network according to the following formula l : In the formula, Represents the importance of the jth BN layer in the lth BasicBlock. and They are respectively i and β i When set to 0 The amount of change, and They are γ i and β i about The gradient of , l∈(1,2,…,L), L is the total number of BasicBlocks in the network; and They are the trainable scaling factor and bias in the BN layer respectively; The p in the same resolution stage l Normalized to [0,1], that is: Where P l is the importance of the normalized BasicBlock, p max and p min In this stage, p l The maximum and minimum values ​​of ; For BasicBlocks with downsampling layers, since they cannot be pruned due to their particularity, the importance of these BasicBlocks is set to 1; Step 3: According to the compression rate r to be achieved, determine the number of BasicBlocks to be retained as l0 = rL, and the number of BasicBlocks to be removed as l1 = L-l0; the importance P of all BasicBlocks l Perform global sorting and remove the least important BasicBlocks to obtain the network model after layer pruning. Step 4: Train the network model after layer pruning to restore the classification accuracy of the model and obtain the final network model.

2. A computer system, characterized in that include: One or more processors, and a computer-readable storage medium for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors are enabled to implement the method of claim 1.

3. A computer-readable storage medium, characterized in that Computer executable instructions are stored, and when the instructions are executed, they are used to implement the method of claim 1.

4. A computer program product, characterized in that The invention comprises computer executable instructions, which are used to implement the method of claim 1 when being executed.

Citation Information

Patent Citations

  • Flexible deep learning network model compression method based on channel gradient pruning

    CN112396179A

  • Teacher blackboard writing action recognition method based on multi-granularity convolutional neural network pruning

    CN112800977A

  • Model compression device and program

    JP2021108039A

  • Efficient image classification method based on structured pruning

    US20210049423A1