Model pruning method, device, equipment and storage medium

Abstract

The present invention discloses a model pruning method, device, equipment and storage medium, which can be widely used in the field of artificial intelligence technology. The method of the present invention comprises: obtaining a plurality of convolutional layer combinations in a model to be pruned, the model to be pruned comprises a plurality of convolutional layers connected in sequence, each convolutional layer combination comprises a first convolutional layer and a second convolutional layer, the second convolutional layer is the next convolutional layer of the first convolutional layer; determining a filter corresponding to the first convolutional layer and a channel corresponding to the second convolutional layer, the filter in the first convolutional layer corresponds one-to-one to the channel in the second convolutional layer, and the first convolutional layer comprises at least one filter; obtaining first matrix data of the filter and second matrix data of the channel, and calculating a pruning index corresponding to the filter according to the first matrix data and the second matrix data; performing pruning processing on the model to be pruned according to the pruning index to obtain a pruned model; the present invention can effectively improve the accuracy of the pruned model.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to a model pruning method, device, equipment and storage medium. Background Art

[0002] The ImageNet challenge has made significant progress in exploring various architectural choices for CNNs. The general trend over the past few years has been that networks are getting deeper and deeper, with an overall increase in the number of parameters and convolution operations. These high-capacity networks have a large inference cost, especially when used in embedded sensors or mobile devices, which generally have limited computational and power resources. For these applications, computational efficiency and network size are key enabling factors in addition to accuracy. In addition, for web services that provide image search and image classification APIs, the operating time budget is typically hundreds of thousands of images per second, which requires low inference time. Currently, model compression / pruning methods are often used to reduce storage and computational costs.

[0003] Model compression / pruning methods in related technologies usually remove redundant feature maps from the trained network. This can be achieved by introducing a three-layer pruning weight method and locating pruning candidates using particle filtering, which selects the best combination from some randomly generated masks to achieve model compression. It is also possible to detect infrequently activated feature maps through facial sample input data, and then compress the model. However, the related modules removed by the above method do not have a specific physical meaning and are less interpretable, which can easily lead to low accuracy of the compressed model. Summary of the invention

[0004] The following is a summary of the subject matter described in detail herein. This summary is not intended to limit the scope of the claims.

[0005] The embodiments of the present invention provide a model pruning method, apparatus, device and storage medium, which can effectively improve the accuracy of the pruning model.

[0006] In a first aspect, an embodiment of the present invention provides a model pruning method, comprising:

[0007] Acquire multiple convolutional layer combinations in the model to be pruned, wherein the model to be pruned includes multiple convolutional layers connected in sequence, each of the convolutional layer combinations includes a first convolutional layer and a second convolutional layer, and the second convolutional layer is a next convolutional layer of the first convolutional layer;

[0008] Determine a filter corresponding to the first convolutional layer and a channel corresponding to the second convolutional layer in the convolutional layer combination, wherein the filter in the first convolutional layer corresponds to the channel in the second convolutional layer one-to-one, and the first convolutional layer includes at least one of the filters;

[0009] Acquire the first matrix data of the filter and the second matrix data of the channel, and calculate the pruning index corresponding to the filter according to the first matrix data and the second matrix data;

[0010] The model to be pruned is pruned according to the pruning index to obtain a pruned model.

[0011] In a second aspect, an embodiment of the present invention provides a model pruning device, including:

[0012] A data acquisition module, used to acquire multiple convolutional layer combinations in a model to be pruned, wherein the model to be pruned includes multiple convolutional layers connected in sequence, each of the convolutional layer combinations includes a first convolutional layer and a second convolutional layer, and the second convolutional layer is a next convolutional layer of the first convolutional layer;

[0013] a data determination module, configured to determine a filter corresponding to the first convolutional layer and a channel corresponding to the second convolutional layer in the convolutional layer combination, wherein the filter in the first convolutional layer corresponds to the channel in the second convolutional layer one-to-one, and the first convolutional layer includes at least one of the filters;

[0014] An index calculation module, used to obtain the first matrix data of the filter and the second matrix data of the channel, and calculate the pruning index corresponding to the filter according to the first matrix data and the second matrix data;

[0015] A pruning processing module is used to perform pruning processing on the model to be pruned according to the pruning index to obtain a pruned model.

[0016] In a third aspect, an embodiment of the present invention provides a model pruning device, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the model pruning method of the previous embodiment when executing the computer program.

[0017] In a fourth aspect, an embodiment of the present invention provides a computer-readable storage medium storing computer-executable instructions, wherein the computer-executable instructions are used to execute the model pruning method of the previous embodiment.

[0018] The beneficial effects of the embodiments of the present invention include: the embodiments of the present invention first obtain multiple convolutional layer combinations in the model to be pruned, and the model to be pruned includes multiple convolutional layers connected in sequence, each convolutional layer combination includes a first convolutional layer and a second convolutional layer, and the second convolutional layer is the next convolutional layer of the first convolutional layer; determine the filter corresponding to the first convolutional layer and the channel corresponding to the second convolutional layer in the convolutional layer combination, and the filter in the first convolutional layer corresponds one-to-one to the channel in the second convolutional layer, the first convolutional layer includes at least one filter, and then obtain the first matrix data of the filter and the second matrix data of the channel, calculate the pruning index corresponding to the filter according to the first matrix data and the second matrix data, and finally prune the model to be pruned according to the pruning index to obtain a pruned model; compared with the related art, the embodiments of the present invention can effectively improve the accuracy of the pruned model.

[0019] Other features and advantages of the present invention will be described in the following description, and partly become apparent from the description, or understood by practicing the present invention. The purpose and other advantages of the present invention can be realized and obtained by the structures particularly pointed out in the description, claims and drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] The accompanying drawings are used to provide a further understanding of the technical solution of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the technical solution of the present invention and do not constitute a limitation on the technical solution of the present invention.

[0021] Figure 1 A schematic diagram of a flow chart of a model pruning method according to an embodiment of the present invention;

[0022] Figure 2 Schematic diagram of the structure of consecutive convolutional layers in the model to be pruned according to an embodiment of the present invention;

[0023] Figure 3 A schematic diagram of a process for obtaining a pruning index corresponding to a filter according to an embodiment of the present invention;

[0024] Figure 4 A schematic diagram of a process for obtaining product matrix data according to an embodiment of the present invention;

[0025] Figure 5 A schematic diagram of the pruning process according to an embodiment of the present invention;

[0026] Figure 6 A schematic diagram of a process of a pruning filter according to an embodiment of the present invention;

[0027] Figure 7 This is a schematic diagram of the process after obtaining the pruning model according to an embodiment of the present invention;

[0028] Figure 8The figure is a schematic diagram of the structure of a model pruning device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0029] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0030] It should be noted that, although the functional modules are divided in the device schematic diagram and the logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in a different order than the module division in the device or the order in the flowchart. The terms "first", "second", etc. in the specification, claims and the above drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence.

[0031] In the related art, AlexNet and VGGNet can be pruned by pruning small amounts of weights and then retraining without compromising overall accuracy. However, pruning parameters does not necessarily reduce computing time because most of the deleted parameters come from the fully connected layers. For example, the fully connected layers of VGG-16 account for 90% of the total parameters but less than 1% of the entire floating-point operations.

[0032] It is understandable that the convolutional layers in the model can be compressed and accelerated. Large-capacity CNNs (convolutional neural networks) usually have a lot of redundancy between different filters and features. Therefore, the computational cost of CNNs (convolutional neural networks) can be reduced by pruning filters. Compared with pruning the weights of the entire network in the related art, filter pruning is a structured pruning method that does not require the introduction of sparsity, so there is no need to use a sparse library or any specialized hardware. The number of pruned filters is directly related to the acceleration because it reduces the number of matrix multiplications and can be adjusted for the target acceleration.

[0033] In order to reduce the computational cost of the convolution layer, the related art proposes an approximate convolution operation, such as representing the weight matrix as the product of two smaller low-rank matrices without changing the original number of filters. Other methods to reduce the computational cost of convolution include using a convolution method based on FFT (Fast Fourier Transform) and a convolution method using the Winograd (fast convolution) algorithm. In addition, quantization and binarization can be used to reduce the size of the model, which can reduce the computational cost by reducing the size of the model.

[0034] It is understandable that in order to reduce computational costs and not generate additional overhead, in related technologies, redundant feature maps are usually removed from the trained network, for example, by introducing a three-layer pruning weight method and locating pruning candidates using particle filtering, which selects the best combination from some randomly generated masks. Feature maps that are not frequently activated can also be detected through facial sample input data. However, the above methods do not deeply consider the feature extraction capabilities of the modules in the model, and the related modules removed do not have a specific physical meaning and are less interpretable, which can easily lead to low accuracy of the compressed model.

[0035] Based on this, the embodiments of the present invention provide a model pruning method, device, equipment and storage medium. The embodiments of the present invention can effectively improve the accuracy of the pruning model; in addition, the embodiments of the present invention can also reduce the computing cost without generating additional overhead.

[0036] Specifically, refer to Figure 1 , an embodiment of the present invention provides a model pruning method, including but not limited to the following steps:

[0037] Step S100, obtaining multiple convolutional layer combinations in the model to be pruned, wherein the model to be pruned includes multiple convolutional layers connected in sequence, each convolutional layer combination includes a first convolutional layer and a second convolutional layer, and the second convolutional layer is the next convolutional layer of the first convolutional layer;

[0038] It should be noted that the model to be pruned is the model that actually needs to be pruned. For example, for CNN (convolutional neural network), in order to reduce the computational cost of the CNN (convolutional neural network) model, the embodiment of the present invention can be implemented by pruning filters. It is understandable that the model to be pruned usually includes multiple convolutional layers connected in sequence.

[0039] The model to be pruned in an embodiment of the present invention has multiple convolutional layer combinations. For example, in some embodiments, two consecutive and adjacent convolutional layers in the model to be pruned are regarded as a convolutional layer combination, and it is defined that each convolutional layer combination includes a first convolutional layer and a second convolutional layer, and the second convolutional layer is the next convolutional layer of the first convolutional layer. For example, assuming that the model to be pruned includes N (N>1) convolutional layers connected in sequence, the first convolutional layer and the second convolutional layer in the model to be pruned can be a convolutional layer combination, in which case the first convolutional layer serves as the first convolutional layer, and the second convolutional layer serves as the second convolutional layer; the second convolutional layer and the third convolutional layer can be a convolutional layer combination, in which case the second convolutional layer serves as the first convolutional layer, and the third convolutional layer serves as the second convolutional layer; the third convolutional layer and the fourth convolutional layer serve as a convolutional layer combination; ...; the N-1th convolutional layer and the Nth convolutional layer serve as a convolutional layer combination, in which case the N-1th convolutional layer serves as the first convolutional layer, and the Nth convolutional layer serves as the second convolutional layer. Therefore, in an embodiment of the present invention, the first convolutional layer can be the first convolutional layer, and the second convolutional layer, the third convolutional layer, the fourth convolutional layer, ..., the N-1th convolutional layer, the Nth convolutional layer can be the second convolutional layer in the current convolutional layer combination, and can also be used as the first convolutional layer in the next convolutional layer combination, which will not be repeated here.

[0040] It should be noted that the number of convolutional layers corresponding to different types of models to be pruned is usually different. For example, the ResNet-56 (Residual Neural Network-56, a 56-layer residual neural network) model has 55 convolutional layers.

[0041] Step S200, determining a filter corresponding to a first convolutional layer and a channel corresponding to a second convolutional layer in a convolutional layer combination, wherein the filter in the first convolutional layer corresponds one-to-one to the channel in the second convolutional layer, and the first convolutional layer includes at least one filter;

[0042] It can be understood that in the model to be pruned, for multiple sequentially connected convolutional layers, the filters corresponding to the current convolutional layer can correspond one-to-one to the channels in the next convolutional layer.

[0043] In some embodiments, in each convolutional layer combination, the filter corresponding to the first convolutional layer and the channel corresponding to the second convolutional layer may be determined.

[0044] In addition, for the last convolutional layer in the model to be pruned, that is, the Nth convolutional layer, it can also be used as the first convolutional layer in the next convolutional layer combination. At this time, since it is the last convolutional layer, and for the model architecture, the convolutional layer is usually connected to the fully connected layer, that is, the last convolutional layer is connected to the fully connected layer, and for the combination of the Nth convolutional layer and the fully connected layer, the Nth convolutional layer is the first convolutional layer, the fully connected layer is the second convolutional layer, and the filter in the Nth convolutional layer corresponds to the channel in the fully connected layer one by one. Therefore, the pruning index of the filter corresponding to the Nth convolutional layer can be calculated through step S300 of the embodiment of the present invention. That is, the last convolutional layer can obtain the pruning index of the filter corresponding to the last convolutional layer with the help of the fully connected layer behind it.

[0045] For specific reference, Figure 2 A schematic diagram showing the structure of consecutive convolutional layers in the model to be pruned, where bs=1.

[0046] Figure 2 In the figure, A, B and C all represent three consecutive feature maps in the convolutional neural network. The part between A and B is represented as a convolution layer, and the part between B and C is represented as a convolution layer. That is, feature map A is input into the convolution layer between A and B, and feature map B is obtained after convolution processing; feature map B is input into the convolution layer between B and C, and feature map C is obtained after convolution processing.

[0047] Specifically analyze a convolutional layer between A and B, where a convolutional layer between B and C is similar to the convolutional layer between A and B.

[0048] In the dotted box in the first column on the right side of feature map A, four rectangular blocks represent four filters from top to bottom, and in the dotted box in the second column on the right side of feature map A, four rectangles represent BN (Batch Normalization) corresponding to the four filters one by one; in the dotted box in the third column on the right side of feature map A, four rectangles represent ReLu (Rectified Linear Unit) corresponding to the four filters one by one. It can be understood that the color of the box diagram in the figure is only used for better distinction and has no specific reference meaning, and the embodiment of the present invention does not make specific limitations on this.

[0049] Therefore, for the convolution layer between B and C, the dotted boxes from left to right represent five filters, BN (Batch Normalization) corresponding to the five filters one by one, and ReLu (Rectified Linear Unit) corresponding to the five filters one by one.

[0050] It is understandable that different filters may correspond to different BN and ReLu. A convolutional layer may include multiple filters, for example Figure 2 In the convolutional layer in , each cuboid represents a filter. For each filter, starting from the first rectangle at the front of the filter, calculations are performed from front to back to obtain a total of several channels. For example, in the dotted box in the first column on the right side of feature map A, each filter includes three rectangles, and the three rectangles are arranged in sequence from front to back. Different rectangles represent different channels, that is, each filter includes three channels.

[0051] It is understandable that after inputting feature map A into the convolution layer between A and B, feature map B is obtained, and the number of layers corresponding to feature map B is 4, which is the same as the number of filters corresponding to the convolution layer between A and B (i.e., four filters); and after inputting feature map B into the convolution layer between B and C, feature map C is obtained, and the number of layers corresponding to feature map C is 5, which is the same as the number of filters corresponding to the convolution layer between B and C (i.e., five filters). It is understandable that the number of layers corresponding to the feature map, i.e., each rectangle is spliced ​​in sequence from front to back and finally output to obtain the feature map.

[0052] In some embodiments, the number of filters in the convolutional layers gradually increases, that is, the number of filters in the second convolutional layer is greater than the number of filters in the first convolutional layer, and so on.

[0053] Step S300, obtaining the first matrix data of the filter and the second matrix data of the channel, and calculating the pruning index corresponding to the filter according to the first matrix data and the second matrix data;

[0054] Through the first matrix data and the second matrix data, the pruning index of the filter corresponding to the first convolutional layer in the two convolutional layers in each convolutional layer combination can be calculated.

[0055] It should be noted that, since the first matrix data of the filter is usually three-dimensional, such as a 3×3×3 matrix, and the second matrix data of the channel is usually two-dimensional, such as a 3×3 matrix, and in order to calculate the pruning index corresponding to the filter according to the first matrix data and the second matrix data, the channel can be repeatedly copied, for example, three times, so as to obtain a 3×3×3 matrix of the same dimension as the first matrix data, that is, the second matrix data and the first matrix data are matrix data of the same dimension. Therefore, when the first matrix data and the second matrix data are matrix data of different dimensions, the channel in the second convolution layer corresponding to the filter in the first convolution layer can be processed so that the second matrix data of the channel and the first matrix data are matrix data of the same dimension.

[0056] Step S400: Pruning the model to be pruned according to the pruning index to obtain a pruned model.

[0057] It can be understood that since the model to be pruned has multiple convolutional layer combinations, and the first convolutional layer in the convolutional layer combination includes at least one filter, through steps S100 to S300, pruning indicators corresponding to multiple filters can be obtained. For example, in some embodiments, pruning indicators corresponding to the filters of all convolutional layers can be obtained.

[0058] In the embodiment of the present invention, since the second convolutional layer, the third convolutional layer, the fourth convolutional layer, ..., the N-1th convolutional layer, and the Nth convolutional layer can all be used as the first convolutional layer in the next convolutional layer combination, the pruning index of the filter corresponding to the first convolutional layer can be obtained through the first convolutional layer and the second convolutional layer in each convolutional layer combination. Thus, through steps S100 to S300, the pruning process of the model to be pruned according to the pruning index can be implemented in step S400 to obtain the pruned model.

[0059] The model pruning method of the embodiment of the present invention sequentially obtains two convolutional layers in the convolutional layer combination in the model to be pruned. Since the two convolutional layers in the convolutional layer combination are continuous and adjacent, and the filter corresponding to the first convolutional layer is in a one-to-one correspondence with the channel in the second convolutional layer, the pruning index corresponding to the filter of the first convolutional layer can be obtained by calculating the first matrix data of the filter of the first convolutional layer and the second matrix data of the channel of the second convolutional layer. For example, the first filter in the first convolutional layer corresponds to the first channel of all filters in the second convolutional layer; the second filter in the first convolutional layer corresponds to the second channel of all filters in the second convolutional layer; the third filter in the first convolutional layer corresponds to the third channel of all filters in the second convolutional layer..., that is, the number of filters in the first convolutional layer is the same as the number of channels corresponding to all filters in the second convolutional layer. It can be understood that the number of layers corresponding to the feature map output by the current convolutional layer (that is, each convolutional layer in the model to be pruned) is the same as the number of filters corresponding to the current convolutional layer.

[0060] Compared with the related art, the embodiment of the present invention prunes the model to be pruned according to the pruning indicators of the filters corresponding to the convolutional layers in the model to be pruned to obtain a pruned model, which can effectively improve the accuracy of the pruned model and reduce the computational cost without incurring additional overhead.

[0061] In the related art, scanning calculations are required when performing convolution operations, but the embodiments of the present invention do not require scanning calculations, which is more efficient and concise, and reduces computational complexity.

[0062] Reference Figure 3It can be understood that, in step S300, the pruning index corresponding to the filter is calculated according to the first matrix data and the second matrix data, including but not limited to the following steps:

[0063] Step S310, performing product calculation on the first matrix data and the second matrix data to obtain product matrix data;

[0064] Step S320: taking absolute values ​​of the matrix elements in the product matrix data respectively and performing cumulative summation calculation to obtain a pruning index corresponding to the filter.

[0065] It should be noted that, since the model to be pruned includes multiple convolutional layers, each convolutional layer usually includes multiple filters. Therefore, through steps S310 and S320, pruning indicators corresponding to multiple filters in each convolutional layer can be obtained.

[0066] After obtaining the product matrix data, it is necessary to take the absolute values ​​of all matrix elements in the product matrix data. This is because in the matrix data, there are matrix elements whose corresponding values ​​are negative numbers. By taking the absolute values, it can be ensured that the corresponding values ​​of all matrix elements in the product matrix data are positive or zero. Then, all matrix elements after taking the absolute values ​​are added and summed to obtain the accumulated sum data, which is the pruning index corresponding to the current filter.

[0067] For example, the first matrix data of the second filter in the first convolutional layer corresponds to the second matrix data of the second channel in all filters in the second convolutional layer, and the first matrix data and the second matrix data are multiplied to obtain the product matrix data, and then the pruning index corresponding to the second filter in the first convolutional layer is obtained through step S320. In this way, the pruning index corresponding to each filter in each convolutional layer of the model to be pruned can be obtained.

[0068] Reference Figure 4 It can be understood that in step S310, the first matrix data and the second matrix data are multiplied to obtain the product matrix data, including but not limited to the following steps:

[0069] Step S311, performing product calculations on the first matrix elements in the first matrix data and the second matrix elements in the second matrix data respectively, to obtain product matrix data.

[0070] It can be understood that the product calculation of the first matrix data and the second matrix data is specifically to perform product calculations on the first matrix elements in the first matrix data and the second matrix elements in the second matrix data respectively on the corresponding elements to obtain product matrix data.

[0071] For example, the first first matrix element corresponding to the first row and first column in the first matrix data is multiplied with the first second matrix element corresponding to the first row and first column in the second matrix data; or, the second first matrix element in the horizontal direction corresponding to the second row and first column in the first matrix data is multiplied with the second second matrix element in the horizontal direction corresponding to the second row and first column in the second matrix data; or, the second first matrix element in the horizontal direction and second vertical direction corresponding to the second row and second column in the first matrix data is multiplied with the second second matrix element in the horizontal direction and second vertical direction corresponding to the second row and second column in the second matrix data, thereby realizing the product calculation of corresponding elements.

[0072] It can be understood that the above embodiment is directed to the situation where the first matrix elements in the first matrix data correspond one to one with the second matrix elements in the second matrix data.

[0073] For the case where the first matrix elements in the first matrix data and the second matrix elements in the second matrix data cannot correspond one to one, for example, for 3×3×3 first matrix data and 5×5×5 second matrix data, the values ​​corresponding to the first matrix elements in the first matrix data that cannot correspond to the second matrix data are set to zero. In this way, it is also possible to perform product calculations on the corresponding elements of the first matrix elements in the first matrix data and the second matrix elements in the second matrix data to obtain product matrix data.

[0074] Reference Figure 5 It can be understood that, in step S400, pruning the model to be pruned is performed according to the pruning index to obtain the pruned model, including but not limited to the following steps:

[0075] Step S410: screening the filter according to the pruning index to obtain a pruning filter that meets the preset pruning rate;

[0076] Step S420: Prune the model to be pruned according to the pruning filter to obtain a pruned model.

[0077] The embodiment of the present invention performs pruning processing on the model to be pruned based on pruning indicators corresponding to filters in multiple convolutional layers in the model to be pruned.

[0078] For example, in some embodiments, according to all convolutional layers in the model to be pruned, the filter corresponding to each pruning indicator is screened to obtain a pruned filter that meets a preset pruning rate.

[0079] It can be understood that after obtaining the pruning index (i.e., importance) corresponding to the filter of the convolutional layer in the model to be pruned, it is possible to determine which filters meet the preset pruning rate by setting the preset pruning rate and the numerical value corresponding to the pruning index of the filter, and use the filters that meet the preset pruning rate as pruned filters.

[0080] After obtaining the pruning filter, the model to be pruned can be pruned according to the pruning filter to obtain the pruning model. That is, the embodiment of the present invention prunes the redundant filters that do not meet the preset pruning rate based on the pruning filter, thereby pruning the model to be pruned. The pruning model obtained by the embodiment of the present invention has higher accuracy.

[0081] Reference Figure 6 It can be understood that, in step S410, the filter is screened according to the pruning index to obtain a pruning filter that meets the preset pruning rate, including but not limited to the following steps:

[0082] Step S411, sorting the pruning indicators to obtain sorted data;

[0083] Step S412: According to a preset pruning rate, filter processing is performed on the filter corresponding to the sorted data to obtain a pruning filter.

[0084] Specifically, since the pruning index of the filter is the accumulated sum data, the corresponding filters can be sorted according to the obtained accumulated sum data to obtain the sorted data corresponding to the filter.

[0085] For example, in some embodiments, the pruning indicators corresponding to multiple filters may be sorted from small to large to obtain sorted data; or, the pruning indicators corresponding to multiple filters may be sorted from large to small to obtain sorted data.

[0086] In some embodiments, the pruning indicators corresponding to the filters in all convolutional layers may be sorted, which is not specifically limited in the embodiments of the present invention.

[0087] By setting a preset pruning rate in advance, the filters corresponding to the sorted data can be screened, for example, a certain number of redundant filters with relatively small pruning indicators and their related weights are removed to obtain a pruned model, namely a pruned model.

[0088] Reference Figure 7 It is understandable that, in step S400, after obtaining the pruning model, the following steps are included but not limited to:

[0089] Step S500: retrain the pruning model to update the pruning index.

[0090] It can be understood that the embodiment of the present invention saves the retraining time of pruning filters in multi-layer convolutional layers by adopting a one-time pruning and retraining approach rather than layer-by-layer iterative fine-tuning, which is crucial for pruning very deep networks. The embodiment of the present invention can effectively improve efficiency.

[0091] In some embodiments, the pruning model is retrained to update the pruning indicators of the filters corresponding to all convolutional layers.

[0092] For example, after obtaining the pruned model, the pruned model can be retrained to fine-tune the pruned model, thereby restoring the accuracy of the pruned model. The embodiment of the present invention can maintain a high accuracy while reducing the size of the pruned model.

[0093] The model pruning method according to an embodiment of the present invention is described below with a specific embodiment.

[0094] The embodiment of the present invention takes the ResNet-56 model as an example to verify the model pruning method of the embodiment of the present invention. Obtain a data set, for example, by using cifar10 (a data set for universal object recognition), which has a total of multiple, for example, 60,000 color images, which are 32*32 and divided into 10 classes, with 6,000 images in each class. Specifically, 50,000 images can be set for training to form 5 training sets, each with 10,000 images, and another 10,000 images for testing, which constitute a separate test set.

[0095] For example, the test data in the test set comes from each of the 10 categories mentioned above, and 1,000 images are randomly selected from each category. The remaining data is randomly arranged to form the training set. It is understandable that the number of images in each category in a training set is not necessarily the same. In general, there are 5,000 images in each category in the training set.

[0096] Each experiment performed model compression training for 300 epochs.

[0097] It can be understood that when a complete dataset passes through the ResNet-56 model once and returns once, this process is called an epoch. That is, all training sets undergo one forward propagation and one backward propagation in the ResNet-56 model, which means that one epoch is the process of training all training sets once.

[0098] The embodiments of the present invention are implemented based on hardware, for example, a GPU processor is used, and the hardware adopts the PyTorch framework.

[0099] As shown in Table 1, the effects of compression / pruning methods of various models are compared.

[0100] Table 1. Comparison results of compression methods

[0101]

[0102]

[0103] Among them, the model compression / pruning methods corresponding to GAL-0.6, HRankPlus, and NPPM are all compression / pruning methods of related technologies. Specifically, GAL-0.6 represents the pruning of CNN (convolutional neural network) with optimal structure achieved by generative adversarial learning; HRankPlus represents the model pruning method based on the rank of feature graphs; NPPM represents the method based on model pruning by maximizing performance. Ours in Table 1 represents the model pruning method of the embodiment of the present invention.

[0104] In addition, in Table 1, RawModel Top-1 (%) represents the accuracy data of the model before pruning; Pruned Top-1 (%) represents the accuracy data of the model after pruning (i.e., the pruned model); Top-1↓ (%) represents the decrease in model accuracy before and after pruning; FLOPs↓ (%) represents the decrease in model calculation amount before and after pruning; Params↓ (%) represents the decrease in model parameter amount before and after pruning.

[0105] It can be understood that, for FLOPs↓(%) and Params↓(%), the more corresponding decrease data there are, the better the model performance is; and for Top-1↓(%), on the contrary, that is, the fewer corresponding decrease data there are (such as -0.55 in the embodiment of the present invention, compared with -0.12, -0.31, and -0.36 in the related technologies, the embodiment of the present invention has the least decrease), the better the model performance is.

[0106] It can be seen from Table 1 that the model pruning method of the embodiment of the present invention performs the most pruning on the model, while the Top-1↓(%) of the model is improved the most. Therefore, the model pruning method of the embodiment of the present invention can not only reduce the size of the pruned model, but also has a regularization effect. The pruned model obtained by the model pruning method of the embodiment of the present invention has good performance, and can prune redundant model parameters by discovering redundant model parameters, thereby obtaining a pruned model with higher accuracy.

[0107] It can be understood that for multiple convolutional layer combinations in the model to be pruned, the filter in the first convolutional layer in each convolutional layer combination can correspond to the channel in the second convolutional layer one by one. In addition, the second convolutional layer in the current convolutional layer combination can be used as the first convolutional layer in the next convolutional layer combination. Therefore, the first matrix element in the first matrix data of the filter and the second matrix element in the second matrix data of the corresponding channel are respectively multiplied by the corresponding elements to obtain the pruning index corresponding to the filter. It can be understood that the embodiment of the present invention is equivalent to feature extraction in image processing, and the filter in the first convolutional layer and the channel in the second convolutional layer are regarded as filters for feature extraction in image processing. When the filter and the channel are more similar, the features to be extracted are more similar, and the accumulated sum data obtained, that is, the pruning index corresponding to the filter, is larger. The larger the pruning index here, the greater the similarity of the features to be extracted in the two convolutional layers in each convolutional layer combination. In other words, the features extracted by the first convolutional layer have a greater effect on the subsequent convolutional layers, and also have a greater impact on the final results of the model.

[0108] Through the model pruning method of the embodiment of the present invention, a pruned filter with feature extraction capability can be obtained, and redundant filters without feature extraction capability are pruned to ensure the accuracy of the pruned model.

[0109] It can be understood that by sequentially multiplying the matrix data corresponding to the filter corresponding to the first convolution layer and the channel corresponding to the second convolution layer in two consecutive and adjacent convolution layers, that is, the two convolution layers in each convolution layer combination, the information in the two consecutive and adjacent convolution layers is fused to achieve efficient pruning.

[0110] Therefore, by pruning some redundant filters with low feature extraction capabilities or redundant filters without feature extraction capabilities, since the contribution of these redundant filters to the network model is usually very small, after using the model pruning method of the embodiment of the present invention, relatively good model performance can be obtained to achieve higher accuracy.

[0111] It is understandable that the model pruning method of the embodiment of the invention can acquire and process relevant data based on artificial intelligence technology. Among them, artificial intelligence (AI) is the theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results.

[0112] It is understandable that the basic technologies of artificial intelligence generally include technologies such as sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, big data processing technology, operation / interaction systems, mechatronics, etc. Artificial intelligence software technology mainly includes computer vision technology, robotics technology, biometrics technology, speech processing technology, natural language processing technology, and machine learning / deep learning.

[0113] For example, the present application may be particularly applicable to neural networks.

[0114] Reference Figure 8 One embodiment of the present invention further provides a model pruning device, comprising:

[0115] The data acquisition module 100 is used to acquire multiple convolutional layer combinations in the model to be pruned, wherein the model to be pruned includes multiple convolutional layers connected in sequence, each convolutional layer combination includes a first convolutional layer and a second convolutional layer, and the second convolutional layer is the next convolutional layer of the first convolutional layer;

[0116] A data determination module 200, configured to determine a filter corresponding to a first convolutional layer and a channel corresponding to a second convolutional layer in a convolutional layer combination, wherein the filter in the first convolutional layer corresponds one-to-one to the channel in the second convolutional layer, and the first convolutional layer includes at least one filter;

[0117] An index calculation module 300 is used to obtain the first matrix data of the filter and the second matrix data of the channel, and calculate the pruning index corresponding to the filter according to the first matrix data and the second matrix data;

[0118] The pruning processing module 400 is used to perform pruning processing on the model to be pruned according to the pruning index to obtain a pruned model.

[0119] It should be noted that the contents of the method embodiments of the present invention are all applicable to the system embodiments. The functions specifically implemented by the system embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method, which will not be repeated here.

[0120] In addition, an embodiment of the present invention further provides a model pruning device, which includes: a memory, a processor, and a computer program stored in the memory and executable on the processor.

[0121] The processor and the memory may be connected via a bus or other means.

[0122] The memory, as a non-transient computer-readable storage medium, can be used to store non-transient software programs and non-transient computer executable programs. In addition, the memory may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some embodiments, the memory may optionally include a memory remotely disposed relative to the processor, and these remote memories may be connected to the processor via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0123] It should be noted that the model pruning device in the embodiment of the present invention can be applied as a model pruning method as in the above embodiment. The model pruning device in the embodiment of the present invention and the model pruning method as in the above embodiment have the same inventive concept, so these embodiments have the same implementation principles and technical effects, which will not be described in detail here.

[0124] The non-transient software program and instructions required to implement the model pruning method of the above embodiment are stored in the memory. When executed by the processor, the model pruning method of the above embodiment is executed, for example, the above described Figure 1 The method steps S100 to S400, Figure 3 Steps S310 to S320 of the method, Figure 4 Step S311 of the method, Figure 5 Steps S410 to S420 of the method, Figure 6 Steps S411 to S412 of the method, Figure 7 Method step S500 in .

[0125] The model pruning device embodiment described above is merely illustrative, and the units described as separate components may or may not be physically separated, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the embodiment of the present invention.

[0126] In addition, an embodiment of the present invention further provides a computer-readable storage medium, which stores computer-executable instructions. The computer-executable instructions are executed by a processor or a controller, for example, by a processor in the above-mentioned model pruning device embodiment, so that the above-mentioned processor can execute the model pruning method in the above-mentioned embodiment, for example, execute the above-mentioned Figure 1 The method steps S100 to S400, Figure 3 Steps S310 to S320 of the method, Figure 4 Step S311 of the method, Figure 5Steps S410 to S420 of the method, Figure 6 Steps S411 to S412 of the method, Figure 7 Method step S500 in .

[0127] It will be appreciated by those skilled in the art that all or some of the steps and systems in the disclosed method above may be implemented as software, firmware, hardware and appropriate combinations thereof. Some physical components or all physical components may be implemented as software executed by a processor, such as a central processing unit, a digital signal processor or a microprocessor, or may be implemented as hardware, or may be implemented as an integrated circuit, such as an application specific integrated circuit. Such software may be distributed on a computer-readable medium, which may include a computer storage medium (or a non-transitory medium) and a communication medium (or a temporary medium). As known to those skilled in the art, the term computer storage medium includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information (such as computer-readable instructions, data structures, program modules or other data). Computer storage media include, but are not limited to, RAM, ROM, EEPROM, flash memory or other memory technologies, CD-ROM, digital versatile disks (DVD) or other optical disk storage, magnetic cassettes, magnetic tapes, disk storage or other magnetic storage devices, or any other medium that may be used to store desired information and may be accessed by a computer. Furthermore, it is well known to those skilled in the art that communication media typically embodies computer readable instructions, data structures, program modules, or other data in a modulated data signal such as a carrier wave or other transport mechanism, and may include any information delivery media.

[0128] The above is a specific description of the preferred implementation of the present invention, but the present invention is not limited to the above-mentioned implementation mode. Technical personnel familiar with the field can also make various equivalent deformations or substitutions without violating the spirit of the present invention. These equivalent deformations or substitutions are all included in the scope defined by the claims of the present invention.

Claims

1. A model pruning method, characterized in that: include: Acquire multiple convolutional layer combinations in the model to be pruned, wherein the model to be pruned includes multiple convolutional layers connected in sequence, each of the convolutional layer combinations includes a first convolutional layer and a second convolutional layer, and the second convolutional layer is a next convolutional layer of the first convolutional layer; Determine a filter corresponding to the first convolutional layer and a channel corresponding to the second convolutional layer in the convolutional layer combination, wherein the filter in the first convolutional layer corresponds to the channel in the second convolutional layer one-to-one, and the first convolutional layer includes at least one of the filters; Acquire first matrix data of the filter and second matrix data of the channel, wherein the second matrix data and the first matrix data are matrix data of the same dimension, and the second matrix data is data obtained by processing the image; Performing product calculations on the first matrix elements in the first matrix data and the second matrix elements in the second matrix data to obtain product matrix data, taking absolute values ​​of the matrix elements in the product matrix data and performing cumulative sum calculations to obtain a pruning index corresponding to the filter; wherein the first matrix elements in the first matrix data correspond to the second matrix elements in the second matrix data in a one-to-one manner; The model to be pruned is pruned according to the pruning index to obtain a pruned model.

2. The model pruning method according to claim 1, characterized in that: The step of pruning the model to be pruned according to the pruning index to obtain a pruned model includes: The filter is screened according to the pruning index to obtain a pruning filter that meets a preset pruning rate; The model to be pruned is pruned according to the pruning filter to obtain a pruned model.

3. The model pruning method according to claim 2, characterized in that: The filtering process of the filter according to the pruning index to obtain a pruning filter that meets a preset pruning rate includes: Sorting the pruning indicators to obtain sorted data; According to the preset pruning rate, the filter corresponding to the sorted data is screened to obtain the pruning filter.

4. The model pruning method according to claim 2 or 3, characterized in that: After obtaining the pruning model, the following steps are included: The pruning model is retrained to update the pruning index.

5. A model pruning device, characterized in that: include: A data acquisition module, used to acquire multiple convolutional layer combinations in a model to be pruned, wherein the model to be pruned includes multiple convolutional layers connected in sequence, each of the convolutional layer combinations includes a first convolutional layer and a second convolutional layer, and the second convolutional layer is a next convolutional layer of the first convolutional layer; a data determination module, configured to determine a filter corresponding to the first convolutional layer and a channel corresponding to the second convolutional layer in the convolutional layer combination, wherein the filter in the first convolutional layer corresponds to the channel in the second convolutional layer one-to-one, and the first convolutional layer includes at least one of the filters; An index calculation module is used to obtain the first matrix data of the filter and the second matrix data of the channel, perform product calculations on the first matrix elements in the first matrix data and the second matrix elements in the second matrix data to obtain product matrix data, and perform cumulative sum calculations on the matrix elements in the product matrix data to obtain the pruning index corresponding to the filter; wherein the first matrix elements in the first matrix data and the second matrix elements in the second matrix data correspond one to one, the second matrix data and the first matrix data are matrix data of the same dimension, and the second matrix data is data obtained by processing the image; A pruning processing module is used to perform pruning processing on the model to be pruned according to the pruning index to obtain a pruned model.

6. A model pruning device, characterized in that: include: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the model pruning method according to any one of claims 1 to 4 when executing the computer program.

7. A computer-readable storage medium, characterized in that: Computer executable instructions are stored, and the computer executable instructions are used to execute the model pruning method described in any one of claims 1 to 4.

Images