Image classification method based on adaptive decaying flexible pruning algorithm

Abstract

The application discloses an image classification method based on an adaptive attenuation flexible pruning algorithm. The main disadvantage of flexible pruning is that the zero operation is performed on the model weight in the initial stage of image classification network training, which causes more information loss, and the information may have a positive effect on the accuracy of the image classification network. Unlike flexible pruning, the application uses the filter attenuation operation to replace the original zero operation, which can accelerate the inference speed of the image classification network and the model performance is not greatly affected, so that the image classification network retains more effective information in the initial stage of training.

Description

Technical Field

[0001] This invention relates to the field of computer vision, and in particular to an image classification method based on an adaptive decay flexible pruning algorithm. Background Technology

[0002] In image classification tasks, training the image classification network is crucial for achieving good classification performance. However, the complex model structure and large number of model parameters in image classification networks are obstacles to network training. Network pruning techniques compress the network by removing some unimportant parameters and connections from the deep neural network. This limits its practical application. Filter pruning directly removes entire filters in convolutional layers or entire neurons in fully connected layers to achieve structured pruning. The pruned network has the same underlying structural units as the original network and can be directly deployed on existing hardware and software platforms to achieve compression and acceleration, but the pruning rate is limited compared to weight pruning. With the development of research, the pruning approach has been continuously refined and can be divided into two main categories: importance-based pruning and sparse learning pruning.

[0003] Network pruning is one of the most common methods for reducing the complexity of image classification models and has been extensively studied in the field of model compression. Network pruning can be divided into hard pruning and flexible pruning. Hard pruning directly removes filters and all filters connected to them in the model, which leads to a reduction in feature maps and further degrades model performance. This method usually relies more on a pre-trained model for pruning and fine-tuning. The main drawback of flexible pruning is that it sets the model weights to zero in the initial stage of image classification network training, which causes the loss of more information that could potentially have a positive impact on the accuracy of the image classification network.

[0004] Therefore, in image classification tasks, how to prune image classification networks to retain more effective information in the initial training stage and thus improve image classification accuracy is a technical problem that urgently needs to be solved. Summary of the Invention

[0005] The purpose of this invention is to solve the problem that excessive information loss is easily caused when pruning image classification networks in the prior art, and to provide an image classification method based on an adaptive decay flexible pruning algorithm.

[0006] The specific technical solution adopted in this invention is as follows:

[0007] An image classification method based on an adaptive decay flexible pruning algorithm, comprising:

[0008] S1: Obtain the initialized image classification network W and the training dataset X used to train the image classification network;

[0009] S2: Train the state-of-the-art image classification network W using the training dataset X;

[0010] S3: Using the L2 paradigm of each filter in each convolutional layer as an importance index, evaluate the importance of each filter in the image classification network, and sort the filters in each convolutional layer from high to low importance.

[0011] S4: Based on the preset pruning rate p and the importance determined in S3, select the filters with the lowest importance from the filters contained in each convolutional layer of the image classification network as the filters to be pruned. Then, calculate the corresponding current attenuation ratio according to the initial attenuation ratio α0, the current training round of the image classification network, and the value of the L2 norm of the filter itself. The larger the current training round, the smaller the current attenuation ratio. Use the current attenuation ratio to attenuate the weights of the filters to be pruned, thereby performing a flexible pruning operation on the image classification network.

[0012] S5: Iterate through steps S2 to S4 until the image classification network converges, obtaining a sparse network model;

[0013] S6: Remove all filters with a value of zero from the image classification network to obtain the final compact model, and use the compact model for image classification.

[0014] Preferably, the image classification network is a ResNet network.

[0015] Preferably, step S3 specifically includes the following steps:

[0016] S31: Calculate the L2 norm of each filter in each convolutional layer of the image classification network W ||W i，j ||2:

[0017]

[0018] Among them, W i，j W represents the j-th filter in the i-th layer of the image classification network W. i，j (n, k1, k2) represents the weight at position (k1, k2) on the nth channel of the jth filter in the i-th layer, where n i K represents the number of filters in the (i-1)th layer of the image classification network W, where K×K is the filter size.

[0019] S32: Use the L2 norm of the filter as the criterion for importance. Sort the filters in each convolutional layer of the image classification network W from high to low according to the L2 norm. The larger the L2 norm, the more important the filter.

[0020] Preferably, step S4 specifically includes the following steps:

[0021] S41: Obtain the preset pruning rate p and initial attenuation ratio α0;

[0022] S42: Based on the pruning rate p, the filters with the lowest importance in each convolutional layer of the image classification network W are selected as the filters to be pruned. Specifically, for the i-th layer of the image classification network W, the filters with the lowest importance n are selected. i+1 ×p filters are used as filters to be pruned;

[0023] S43: Calculate the n values ​​in the i-th layer of the image classification network W after the t-th training iteration. i+1 The attenuation ratio α of ×p filters to be pruned i，j (t):

[0024]

[0025] Where, α i，j λ represents the attenuation coefficient of the j-th filter in the i-th layer of the image classification network, λ is the hyperparameter coefficient, t0 is the initial training epoch, and t max This is the maximum number of training rounds;

[0026] S44: Using α i，j For W i，j Perform a decay operation to obtain the decayed network weights:

[0027]

[0028] S45: Replace the network weights of each filter to be pruned in the image classification network W with the attenuated network weights, and proceed with the next round of training.

[0029] Preferably, during the training process of the image classification network, a zeroing threshold close to 0 needs to be set. When the threshold of each filter decays to below the zeroing threshold, the weight of the filter is directly set to 0.

[0030] Preferably, the zeroing threshold is set to 10. -4 10 -6 .

[0031] Compared with the prior art, the present invention has the following advantages:

[0032] The main drawback of existing flexible pruning techniques is that it zeros out the model weights at the initial stage of image classification network training, resulting in the loss of more information that could potentially positively impact the accuracy of the image classification network. Unlike flexible pruning, this invention uses filter attenuation operations instead of the original zeroing operation. This allows the image classification network to retain more effective information in the initial training stage, thereby improving image classification accuracy. Attached Figure Description

[0033] Figure 1 This is a flowchart of an image classification method based on an adaptive decay flexible pruning algorithm.

[0034] Figure 2 This is a schematic diagram of the flexible pruning process based on adaptive decay in this invention. Detailed Implementation

[0035] To make the above-mentioned objects, features, and advantages of the present invention more apparent and understandable, the specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings. Many specific details are set forth in the following description to provide a thorough understanding of the present invention. However, the present invention can be practiced in many other ways different from those described herein, and those skilled in the art can make similar modifications without departing from the spirit of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed below. Technical features in the various embodiments of the present invention can be combined accordingly without mutual conflict.

[0036] To avoid the drawbacks of unstructured pruning, this invention employs a filter pruning algorithm. Removing the entire filter offers several advantages: 1) The pruned model retains its structural integrity, allowing for perfect adaptation to existing deep learning libraries; 2) Memory usage is significantly reduced. This reduction stems not only from the model parameters themselves but, more importantly, from the reduction in intermediate network outputs, which is impossible in unstructured pruning algorithms; 3) Since the network structure remains fundamentally unchanged, the pruned model can be combined with other compression algorithms to further reduce computational load; 4) Using the pruned model can also accelerate other computer vision tasks (such as object detection and semantic segmentation). The specific implementation of this invention is described in detail below.

[0037] like Figure 1 As shown, in a preferred embodiment of the present invention, an image classification method based on an adaptive decay flexible pruning algorithm is provided, comprising the following steps:

[0038] S1: Obtain the initialized image classification network and the training dataset for training the image classification network.

[0039] In the embodiments of the present invention, the specific image classification network trained is not limited, such as the ResNet network, and the specific training dataset used can also be adjusted according to the actual situation.

[0040] S2: Train the current state-of-the-art image classification network using the training dataset.

[0041] S3: Using the L2 paradigm of each filter in each convolutional layer as an importance index, evaluate the importance of each filter in the image classification network, and sort the filters in each convolutional layer from high to low importance.

[0042] In an embodiment of the present invention, step S3 specifically includes the following steps:

[0043] S31: Calculate the L2 norm of each filter in each convolutional layer of the image classification network W ||W i，j ||2:

[0044]

[0045] Among them, W i，j W represents the j-th filter in the i-th layer of the image classification network W. i，j (n, k1, k2) represents the weight at position (k1, k2) on the nth channel of the jth filter in the i-th layer, where n i K represents the number of filters in the (i-1)th layer of the image classification network W, where K×K is the filter size.

[0046] S32: Use the L2 norm of the filter as the criterion for importance. Sort the filters in each convolutional layer of the image classification network W from high to low according to the L2 norm. The larger the L2 norm, the more important the filter.

[0047] S4: Based on the preset pruning rate and the importance determined in S3, select the filters with the lowest importance from the filters contained in each convolutional layer of the image classification network as the filters to be pruned. Then, calculate the corresponding current attenuation ratio according to the initial attenuation ratio, the current training epoch of the image classification network, and the value of the L2 norm of the filter itself. The larger the current training epoch, the smaller the current attenuation ratio. Use the current attenuation ratio to attenuate the weights of the filters to be pruned, thereby performing a flexible pruning operation on the image classification network.

[0048] In an embodiment of the present invention, step S4 specifically includes the following steps:

[0049] S41: Obtain the preset pruning rate p and initial attenuation ratio α0;

[0050] S42: Based on the pruning rate p, the filters with the lowest importance in each convolutional layer of the image classification network W are selected as the filters to be pruned. Specifically, for the i-th layer of the image classification network W, the filters with the lowest importance n are selected. i+1 ×p filters are used as filters to be pruned;

[0051] S43: Calculate the n values ​​in the i-th layer of the image classification network W after the t-th training iteration. i+1 The attenuation ratio α of ×p filters to be pruned i，j (t). It is worth noting that the determination of the filter weight attenuation ratio in this invention should not only be related to the training phase, but also to the filter itself. Therefore, this invention incorporates the L2 paradigm of the filter into the calculation of the attenuation ratio, and the calculation formula is as follows:

[0052]

[0053] Where, α i，j λ represents the attenuation coefficient of the j-th filter in the i-th layer of the image classification network, λ is the hyperparameter coefficient (set to 0.058 in the example), t0 is the initial training epoch, and t max This is the maximum number of training rounds;

[0054] Although the L2 norm of filters requiring attenuation weights is ranked late in their convolutional layers, some filters may still have a large L2 norm. The above formula can slow down the attenuation rate of such filters. Similarly, this formula can accelerate the attenuation rate of filters with a small L2 norm.

[0055] It's important to note that because the filter weights in an image classification network decay continuously, they will approach 0, but it's difficult for them to directly reach 0. Therefore, during the training of the image classification network, a zeroing threshold close to 0 can be set. When α... i，j When the filter weights decay below the zero-reset threshold (t), they are directly set to 0. This allows for a truly compact model to be obtained during the filter removal phase. The zero-reset threshold can be set to 10. -4 10 -6 .

[0056] S44: Using α i，j For W i，j Perform a decay operation to obtain the decayed network weights:

[0057]

[0058] S45: Replace the network weights of each filter to be pruned in the image classification network W with the attenuated network weights, and proceed with the next round of training.

[0059] S5: Iterate through steps S2 to S4 until the image classification network converges, obtaining a sparse network model.

[0060] The pruning process of this invention can be found in [reference needed]. Figure 2 As shown.

[0061] S6: Remove all filters with a value of zero from the image classification network to obtain the final compact model, and use the compact model for image classification.

[0062] In the above process, after the filter weights are decayed, these weights need to be updated through a cycle of model training. In contrast, hard pruning directly removes the filter and all filters connected to it in the model, which leads to a reduction in feature maps and further degrades model performance. This method usually relies more on a pre-trained model for pruning and fine-tuning. Similar to flexible pruning methods, this invention maintains the same capacity as the original model during training, reducing reliance on the pre-trained model and integrating the decay process into normal model training, which does not significantly impact the performance of the compact model. Unlike flexible pruning methods, the decay operation of this invention can retain more information, thereby further improving the model's accuracy. This is why the results of this invention are superior to those of flexible pruning methods.

[0063] The original model undergoes multiple iterations of filter selection, filter attenuation, and filter restoration before finally converging. However, the size of this converged network model remains the same as the original model. The difference lies in that each convolutional layer in the converged model contains numerous all-zero filters, whose output feature maps are also zero, meaning the input feature maps of the filters connected to them in the next convolutional layer are zero. Therefore, removing these all-zero filters and all their connected filters in this invention has no impact on the model's performance. Thus, this invention can reconstruct each layer into a smaller-capacity convolutional layer, resulting in a final compact model. Where i∈[1,L].

[0064] In an image classification network model, the pruning ratio of the i-th convolutional layer is p. i To obtain the final compact model, it is necessary to remove n from each layer. i+1 ×p i There are i ∈ [1, L] filters, and the filters in the remaining layers connected to these filters also need to be removed. Therefore, the dimension of the output feature map of the i-th layer is reduced from n. i+1 ×h i+1 ×w i+1 Reduce to n i+1 ×(1-p i )×h i+1 ×w i+1Similarly, the output feature map of layer i-1 will be adjusted according to the pruning ratio p of this layer. i-1 From n i ×h i ×w i Reduce to n i ×(1-p i-1 )×h i ×w i This will serve as the input feature map for the i-th layer, thus reducing the computational cost of the i-th layer from... Reduce to Therefore, theoretically, the computational cost of the compact model will be reduced to (1-p) times that of the original model. i )×(1-p i-1 This ratio can accelerate the inference speed of image classification network models.

[0065] To verify the performance of the methods described in S1-S6 of this invention, the pruning algorithms proposed in S1-S6 were applied to the mainstream image classification network ResNet on the standard image classification dataset CIFAR-10 and the well-known ILSVRC-2012 dataset, and extensive experiments were conducted on various ResNet structures. Experiments show that, in most cases, the pruning algorithm of this invention outperforms some previous network pruning algorithms. It performs exceptionally well on the CIFAR-10 dataset, reducing floating-point computations by 41.5% and 28.4% for ResNet-32 and ResNet-56 respectively, with only a loss of 0.13% and 0.12% in model accuracy. Notably, on ResNet-110, the 40.8% reduction in floating-point operations even improved accuracy by 0.26%, and the 52.3% reduction still improved accuracy by 0.12% compared to the original network. On ResNet-32, the 28.8% reduction in floating-point operations improved accuracy by 0.47%. Although the pruning algorithm did not improve the accuracy of the original network on the ILSVRC-2012 dataset, it improved the top-1 and top-5 accuracy of the pruned model compared to other algorithms.

[0066] The embodiments described above are merely preferred embodiments of the present invention and are not intended to limit the invention. Those skilled in the art can make various changes and modifications without departing from the spirit and scope of the invention. Therefore, all technical solutions obtained through equivalent substitution or transformation fall within the protection scope of the present invention.

Claims

1. An image classification method based on an adaptive decay flexible pruning algorithm, characterized in that, include: S1: Obtain the initialized image classification network and the training dataset for training the image classification network; S2: Train the state-of-the-art image classification network using the training dataset in one round; S3: Using the L2 paradigm of each filter in each convolutional layer as an importance index, evaluate the importance of each filter in the image classification network, and sort the filters in each convolutional layer from high to low importance. S4: Based on the preset pruning rate and the importance determined in S3, select the filters with the lowest importance from the filters contained in each convolutional layer of the image classification network as the filters to be pruned. Then, calculate the corresponding current attenuation ratio according to the initial attenuation ratio, the current training epoch of the image classification network, and the value of the L2 norm of the filter itself. The larger the current training epoch, the smaller the current attenuation ratio. Use the current attenuation ratio to attenuate the weights of the filters to be pruned, thereby performing a flexible pruning operation on the image classification network. S5: Iterate through steps S2 to S4 until the image classification network converges, obtaining a sparse network model; S6: Remove all filters with a value of zero from the image classification network to obtain the final compact model, and use the compact model for image classification.

2. The flexible pruning method for image classification networks based on adaptive decay as described in claim 1, characterized in that, The image classification network is a ResNet network.

3. The flexible pruning method for image classification networks based on adaptive decay as described in claim 1, characterized in that, Step S3 specifically includes the following steps: S31: Computational Image Classification Network L2 norm of each filter in each convolutional layer : ； in, Represents image classification networks Middle The first in the layer One filter, Representing the The first in the layer The filter in the nth filter On the passage Weights at position Represents image classification networks Middle The number of filters in the layer, where K×K is the filter size; S32: Use the L2 paradigm of the filters as the importance metric, and rank the image classification networks from high to low L2 paradigm. The filters in each convolutional layer are sorted, and the larger the L2 norm, the more important the filter.

4. The flexible pruning method for image classification networks based on adaptive decay as described in claim 1, characterized in that, Step S4 specifically includes the following steps: S41: Obtain the preset pruning rate and initial attenuation ratio ; S42: Based on pruning rate Image classification networks The filters with the lowest importance ranking in each convolutional layer are selected as the filters to be pruned, where for image classification networks... The Choose the layer with the lowest importance. One filter is used as the filter to be pruned; S43: Calculate the image classification network separately. Current number The first round of training In the layer Attenuation ratio of each filter to be pruned : ； in, The first in the image classification network The first in the layer The attenuation coefficient of each filter, These are hyperparameter coefficients. For the initial training round, This is the maximum number of training rounds; S44: Use right Perform a decay operation to obtain the decayed network weights: ； S45: Replace the image classification network with attenuated network weights. The network weights of each filter to be pruned are used for the next round of training.

5. The flexible pruning method for image classification networks based on adaptive decay as described in claim 1, characterized in that, During the training of the image classification network, the zeroing threshold needs to be set to 10. -4 ~10 -6 When the threshold of each filter decays to below the zero threshold, the weight of the filter is directly set to 0.

Images