Deep convolutional neural network model compression method, device and equipment
By evaluating the efficiency of convolution kernel information transfer and filter importance of the deep convolution neural network model, combined with the adaptive reference pruning rate, effective compression and acceleration of the deep convolution neural network model is achieved, and edge equipment is solved The problem of excessive parameter quantity and operation quantity in deployment has maintained model performance.
Patent Information
- Application Number
- CN202410928772.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-11
- Publication Date
- 2024-11-26
Smart Images

Figure CN119026661A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of deep learning technology, and in particular to a deep convolutional neural network model compression method, device and equipment. Background Art
[0002] In recent years, deep convolutional neural networks (DCNNs) and their development have received widespread attention and have made great achievements in many sub-fields of artificial intelligence, such as image classification, object detection, and image segmentation. With the continuous development of fields such as the Internet of Things and autonomous driving, many edge devices also need to deploy deep network models to improve their intelligence level. High-performance deep network models are often accompanied by huge parameters and computational complexity. How to deploy these network models on resource-constrained edge devices has become a difficult problem. To solve this problem, people have proposed model compression technology to reduce the number of parameters and computational complexity of the model while ensuring the performance of the original model, and finally obtain a compact and lightweight model.
[0003] At present, the main methods of model compression include parameter quantization, knowledge distillation, low-rank decomposition, network pruning, etc. Network pruning is one of the common means of model compression. It removes unimportant network parameters by setting evaluation criteria to achieve the effect of model compression. According to the different pruning granularity, network pruning can be roughly divided into four categories: weight pruning, convolution kernel pruning, filter / channel pruning, and layer pruning. Among them, weight pruning is called unstructured pruning, and the other three categories are called structured pruning. Weight pruning reduces the size of the model by setting the parameters in the model to zero. It can be pruned at the finest granularity to obtain a model with a highly sparse parameter matrix. However, weight pruning has a fatal disadvantage. Most frameworks and hardware cannot accelerate sparse matrix calculations, so weight pruning does not actually reduce the amount of model calculations. Convolution kernel pruning takes the two-dimensional convolution kernel as the basic unit. By setting evaluation criteria for the convolution kernel, unimportant convolution kernel parameters are set to zero, but the zeroed convolution kernel parameters will still remain in the model. Since layer pruning can cause significant damage to the network structure (especially shallow networks), it is rarely used directly. Filter pruning directly removes unimportant filters in the convolutional layer. The pruned network is easy to accelerate using the existing computing architecture, so it has become one of the main directions of network pruning.
[0004] In most previous filter pruning methods that use convolution kernel parameters as a metric, all convolution kernel parameters of the filter are usually directly taken as a whole, and then the importance or similarity of this whole parameter is measured as the importance of this filter. This evaluation idea ignores the fact that the filter is composed of convolution kernels one by one, and the difference in the efficiency of convolution kernels in the filter to transmit input channel information will also affect the correct evaluation of the filter, resulting in poor pruning effect. Summary of the invention
[0005] In view of this, the present invention provides a deep convolutional neural network model compression method, device and equipment to solve the problem of effective pruning of deep convolutional network models in the prior art.
[0006] In a first aspect, the present invention provides a deep convolutional neural network model compression method, the method comprising:
[0007] Pre-training the preset deep convolutional neural network to obtain the network model to be pruned;
[0008] The information transfer efficiency of the convolution kernel of each convolution layer of the network model to be pruned is measured, and the convolution kernel whose information transfer efficiency measurement value meets the preset conditions is regarded as a significant convolution kernel;
[0009] The importance of the filter is measured using the significant convolution kernel to obtain the importance measurement value;
[0010] Construct an adaptive baseline pruning rate for each convolutional layer, and prune the filter of each convolutional layer based on the filter importance metric and the adaptive baseline pruning rate;
[0011] The pruned network model is fine-tuned to obtain a lightweight network model with performance similar to the original network model.
[0012] The deep convolutional neural network model compression method provided in this embodiment first pre-trains the model, and then uses the parameters of these significant convolution kernels to evaluate the importance of the filter by retaining the convolution kernels with higher information transmission efficiency for the same input channel, thereby reducing the interference of the convolution kernel parameters with lower information transmission efficiency on the convolution kernel parameters with higher information transmission efficiency, and can better evaluate the importance of the filter. In order to reduce the subjectivity caused by manually setting each layer of pruning, the present invention adaptively determines the baseline pruning rate of each convolution layer by constructing a layer sensitivity sparse coefficient. Finally, pruning and fine-tuning are performed to obtain a lightweight network with performance similar to the original model. The present invention can effectively compress and accelerate the network model, greatly reduce the number of parameters and floating-point operations of the model, and the accuracy is almost unchanged, and is applicable to a variety of common convolutional neural networks.
[0013] In an optional implementation, the information transfer efficiency of the convolution kernel of each convolution layer of the network model to be pruned is measured, and the convolution kernel whose information transfer efficiency measurement value meets the preset conditions is used as a significant convolution kernel, including:
[0014] Use the p-norm to measure the information transfer efficiency of the convolution kernel of each group in each convolution layer;
[0015] Based on the intra-group sensitivity sparsity index and information transfer efficiency metric, the benchmark number of significant convolution kernels retained by each group in each convolution layer is obtained;
[0016] The significant convolution kernel sparse factor is set to adjust the sparseness of the baseline number of significant convolution kernels to obtain the number of significant convolution kernels;
[0017] Based on the number of salient convolution kernels, a binary labeling matrix is used in each convolutional layer to label the salient convolution kernels.
[0018] The embodiment of the present invention shields the convolution kernels with low information transmission efficiency in each group of input channel convolution kernels, so that the quality of the output channel will not be reduced while utilizing the information of each input channel.
[0019] In an optional implementation, the number of significant convolution kernels is calculated by the following formula:
[0020]
[0021]
[0022] In the formula, is the rounding symbol; It is the benchmark number of significant convolution kernels retained by the mth group in the lth layer, representing the maximum number of equivalent transmission channels of input information. This value is obtained by dividing the sum of the group convolution kernels assigned to the output features by the single maximum number of group convolution kernels assigned to the output features. α is the significant convolution kernel sparse factor, and its value range is is the number of significant convolution kernels in the mth group of the lth layer.
[0023] In the embodiment of the present invention, the value distribution of each group convolution kernel in each convolution layer is different, so the benchmark number of significant convolution kernels retained by each group is adaptively obtained, which reduces the subjectivity of taking a fixed ratio. The sparsity of the significant convolution kernel is controlled by setting the significant convolution kernel sparse factor, thereby improving the pruning flexibility of different networks.
[0024] In an optional implementation, the using a significant convolution kernel to measure the importance of the filter to obtain an importance measurement value includes:
[0025] For the lth convolutional layer, the importance measure of the nth filter is calculated as follows:
[0026]
[0027] In the formula, is the p-norm of the m-th convolution kernel parameter of the n-th filter in the l-th layer, is the mth mask of the nth filter in the lth layer, is the importance measure of the n-th filter in the l-th layer.
[0028] The significant convolution kernel parameters retained in the embodiment of the present invention are used to evaluate the filter, which can suppress the convolution kernel with low information transmission efficiency for the same input channel from interfering with the information of the output channel, thereby obtaining a more accurate filter evaluation.
[0029] In an optional implementation, constructing an adaptive benchmark pruning rate for each convolutional layer, and pruning the filter of each convolutional layer based on the filter importance metric and the adaptive benchmark pruning rate, includes:
[0030] For each convolutional layer to be pruned, a layer sensitivity sparse coefficient is constructed, and based on the layer sensitivity sparse coefficient, a benchmark number of retained filters for each convolutional layer is adaptively obtained;
[0031] By setting the layer sensitivity sparse factor, the base number of filters is adjusted to achieve the actual number of pruned branches z for each convolutional layer.
[0032] The filters whose filter importance metric value of each layer is greater than the z-th largest filter importance metric value in the layer are retained.
[0033] In the embodiment of the present invention, different convolutional layers in a convolutional neural network have different emphases on image extraction features and adopt different sparsity levels. Therefore, the baseline number of filters is thinned by setting the layer sensitivity sparsity factor to adjust the pruning intensity.
[0034] In an optional implementation, retaining filters whose filter importance metric value of each layer is greater than the z-th largest filter importance metric value of the layer includes:
[0035] Will In order of importance from highest to lowest:
[0036]
[0037] in:
[0038] z=P l
[0039] Get the pruning mask I of each layer of filters:
[0040]
[0041] Where P l is the actual number of pruning branches at each layer, Indicates whether the nth filter of the lth layer needs to be pruned. When it is 1, the corresponding filter is retained; when When it is 0, the corresponding filter is clipped.
[0042] The embodiment of the present invention can effectively retain each layer of filters by sorting the importance metrics of the filters and then generating a pruning mask for the filters.
[0043] In a second aspect, the present invention provides a deep convolutional neural network model compression device, the device comprising:
[0044] A pre-training module is used to pre-train a preset deep convolutional neural network to obtain a network model to be pruned;
[0045] The significant convolution kernel selection module is used to measure the information transfer efficiency of the convolution kernel of each convolution layer of the network model to be pruned, and the convolution kernel whose information transfer efficiency measurement value meets the preset conditions is regarded as a significant convolution kernel;
[0046] A filter importance measurement module is used to measure the importance of the filter using a significant convolution kernel to obtain an importance measurement value;
[0047] A pruning module is used to construct an adaptive benchmark pruning rate for each convolutional layer and prune the filter of each convolutional layer based on the filter importance metric and the adaptive benchmark pruning rate;
[0048] The model fine-tuning module is used to fine-tune the pruned network model to obtain a lightweight network model with performance similar to the original network model.
[0049] In a third aspect, the present invention provides a computer device, comprising: a memory and a processor, the memory and the processor being communicatively connected to each other, the memory storing computer instructions, and the processor executing the deep convolutional neural network model compression method of the first aspect or any corresponding embodiment thereof by executing the computer instructions.
[0050] In a fourth aspect, the present invention provides a computer-readable storage medium having computer instructions stored thereon, the computer instructions being used to enable a computer to execute the deep convolutional neural network model compression method of the first aspect or any corresponding embodiment thereof.
[0051] In a fifth aspect, the present invention provides a computer program product, comprising computer instructions, which are used to enable a computer to execute the deep convolutional neural network model compression method of the above-mentioned first aspect or any corresponding embodiment thereof. BRIEF DESCRIPTION OF THE DRAWINGS
[0052] In order to more clearly illustrate the specific implementation methods of the present invention or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.
[0053] Figure 1 is a schematic diagram of a process of compressing a deep convolutional neural network model according to an embodiment of the present invention;
[0054] Figure 2 is a flowchart of another deep convolutional neural network model compression method according to an embodiment of the present invention;
[0055] Figure 3 is a schematic diagram of a convolutional layer structure of a deep convolutional neural network according to an embodiment of the present invention;
[0056] Figure 4 2 is a schematic diagram of the distribution of significant convolution kernels of the second layer of the VGG16 network under different intra-group sensitivity sparsity factors according to an embodiment of the present invention;
[0057] Figure 5 is a structural block diagram of a deep convolutional neural network model compression device according to an embodiment of the present invention;
[0058] Figure 6 It is a schematic diagram of the hardware structure of a computer device according to an embodiment of the present invention. DETAILED DESCRIPTION
[0059] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.
[0060] In order to significantly improve the intelligence level of terminals in many applications, the support of deep network models is indispensable. However, since deep models contain a huge number of parameters, directly transplanting them to terminal devices often faces huge challenges. The present invention provides a filter importance measurement method based on significant convolution kernels, and combines a controllable pruning strategy with an adaptive benchmark pruning rate to achieve effective compression of the DCNN model. It can effectively compress the parameters of the model without almost losing the performance of the model, thereby reducing the computational complexity of the model, making deployment on the terminal easier and more efficient.
[0061] According to an embodiment of the present invention, an embodiment of a deep convolutional neural network model compression method is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer device such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.
[0062] In this embodiment, a deep convolutional neural network model compression method is provided, which can be used in the above-mentioned computer device. Figure 1 is a flowchart of a deep convolutional neural network model compression method according to an embodiment of the present invention. Figure 1 As shown, the process includes the following steps:
[0063] Step S101, pre-training a preset deep convolutional neural network to obtain a network model to be pruned.
[0064] In this embodiment, the purpose of pre-training the preset deep convolutional neural network model is to obtain a network model to be pruned. The network models include a single-branch network VGGNet, a residual network ResNet, and a densely connected network DenseNet. These networks can be used for computer vision tasks such as image classification. Specifically, the existing deep learning neural network training framework, such as Pytorch, Tensorflow, etc., can be used to train the selected network to obtain the network model to be pruned.
[0065] Step S102, measuring the information transfer efficiency of the convolution kernel of each convolution layer of the network model to be pruned, and taking the convolution kernel whose information transfer efficiency measurement value meets the preset conditions as the significant convolution kernel.
[0066] Specifically, for a single convolution kernel, the more feature information the convolution kernel extracts from the input channel, the more information is transmitted to the output channel, and the more significant the convolution kernel is. The filter is composed of convolution kernels one by one, and the filter contains some equipotential convolution kernels with low information transmission efficiency for the same input channel. Such convolution kernels will interfere with the information transmitted by the convolution kernels with higher information transmission efficiency, affecting the correct evaluation of the overall importance of the filter. Therefore, the embodiment of the present invention shields the convolution kernels with lower information transmission efficiency for the same input channel, which can not only ensure that the information of the input channel is not completely lost, but also reduce the impact of non-significant convolution kernels on the overall evaluation of the filter. Specifically, the p-norm is used to measure the feature extraction ability of the convolution kernel. The larger the norm of the convolution kernel, the more significant the extracted features are, and the higher the information transmission efficiency of the convolution kernel is. The convolution kernel with high information transmission efficiency is selected from the same group of input channel convolution kernels and recorded as a significant convolution kernel.
[0067] Step S103: Use the significant convolution kernel to measure the importance of the filter to obtain an importance measurement value.
[0068] Specifically, one of the core links of filter pruning is to better evaluate the importance of filters. After obtaining the significant convolution kernel of each convolution layer, the importance of each filter is measured using the significant convolution kernel. Specifically, for the lth convolution layer, the importance measure of the nth filter is calculated by the following formula:
[0069]
[0070] In the formula, is the p-norm of the m-th convolution kernel parameter of the n-th filter in the l-th layer, is the mth mask of the nth filter in the lth layer, is the importance metric of the nth filter in the lth layer. Then the importance metric of each filter in the lth layer can be comprehensively expressed as a metric vector
[0071] Step S104, constructing an adaptive benchmark pruning rate for each convolutional layer, and pruning the filter of each convolutional layer based on the filter importance metric value and the adaptive benchmark pruning rate.
[0072] Specifically, in addition to the importance measurement of the filter, the design of the pruning rate of each convolutional layer is also an important link in filter pruning. Many studies have shown that in DCNN, different convolutional layers have different sensitivities to pruning. Setting a higher pruning rate in a layer with higher sensitivity will cause serious damage to network performance. Therefore, it is necessary to set a suitable pruning rate for each layer. At present, most pruning methods only set the pruning rate of each layer through experience, and there is no adaptive benchmark pruning rate to provide guidance. In order to reduce the subjectivity of manually setting the pruning rate of each layer, the present invention adopts an adaptive pruning rate method based on the importance measurement of the significant convolution kernel filter and performs pruning.
[0073] Step S105, fine-tuning the pruned network model to obtain a lightweight network model with performance similar to that of the original network model.
[0074] The embodiment of the present invention fine-tunes the pruned network model so that the performance of the pruned model reaches a performance close to that of the original model, thereby improving the accuracy of the model and finally obtaining a pruned compact model.
[0075] In this embodiment, a deep convolutional neural network model compression method is provided, which is applied to the above-mentioned computer device, such as Figure 2 As shown, the process includes the following steps:
[0076] Step S201: pre-train the preset deep convolutional neural network to obtain a network model to be pruned. Figure 1 Step S101 of the illustrated embodiment will not be described in detail here.
[0077] Step S202, measure the information transfer efficiency of the convolution kernel of each convolution layer of the network model to be pruned, and use the convolution kernel whose information transfer efficiency measurement value meets the preset conditions as a significant convolution kernel. Specifically, step S201 includes the following process:
[0078] Step S2021, using the p-norm to measure the information transfer efficiency of the convolution kernel of each group in each convolution layer.
[0079] Specifically, the embodiment of the present invention uses the p-norm to measure the feature extraction capability of the convolution kernel. The larger the norm of the convolution kernel, the more significant the corresponding extracted features, and the higher the information transmission efficiency of the convolution kernel. The present invention uses the 1-norm to measure each convolution kernel for example.
[0080] Step S2022, based on the intra-group sensitivity sparsity index and the information transfer efficiency metric, obtain the benchmark number of significant convolution kernels retained by each group in each convolution layer.
[0081] Specifically, the group convolution kernel extracts features from the same input channel (the structure of the convolution layer is as follows Figure 3 As shown, the relationship between group, filter and convolution kernel can be reflected. Therefore, in the embodiment of the present invention, a convolution kernel with higher information transmission efficiency for the same input channel is selected in the group, which is recorded as a significant convolution kernel (SCK). The 1,q norm of the group is defined as:
[0082]
[0083] When q=1, q=∞, we can get:
[0084]
[0085] In formula (2), It is the input feature of the mth group of the lth layer The accumulation of the extracted feature information strength represents The input feature information is passed through group N l The convolution kernel is passed to the lth layer N l The sum of different output feature information.
[0086] In formula (3), It is the input feature of the mth group of the lth layer Extract the maximum value of the feature information intensity, representing the lth layer N l Different output feature information can obtain the maximum amount of single information extracted by the mth group.
[0087] Step S2023, setting the significant convolution kernel sparse factor to adjust the sparseness of the benchmark number of significant convolution kernels to obtain the number of significant convolution kernels.
[0088] Specifically, after using the 1-norm to measure the information transfer efficiency of each convolution kernel of the convolution layer, the embodiment of the present invention needs to select several convolution kernels with the highest information transfer efficiency in each group as significant convolution kernels. You can directly set a fixed ratio and then select the corresponding convolution kernel in each group, but this method is highly subjective. Therefore, the embodiment of the present invention constructs an intra-group sensitivity sparsity coefficient (ISSC). According to the ISSC, the baseline number of significant convolution kernels retained in each group in each convolution layer can be obtained, and then a significant convolution kernel sparsity factor α is set to adjust the sparsity. The formula of ISSC is as follows:
[0089]
[0090] In formula (5), is the rounding symbol; It is the benchmark number of significant convolution kernels retained by the mth group in the lth layer, representing the maximum number of equivalent transmission channels of input information. This value is obtained by dividing the sum of the group convolution kernels assigned to the output features by the single maximum amount of the group convolution kernels assigned to the output features. It can be understood as the number of convolution kernels required for the single maximum amount of convolution kernels.
[0091] Since the value distribution of each convolution layer and each group convolution kernel is different, the adaptively obtained reference number of SCKs retained for each group is also different, reducing the subjectivity of taking a fixed ratio. After obtaining the reference number of retained, in order to improve the pruning flexibility of different networks, the embodiment of the present invention sets a significant convolution kernel sparsity factor α to control the sparsity of the significant convolution kernel, and obtains the final number of retained significant convolution kernels of the mth group in the lth layer: Among them, the value range of the significant convolution kernel sparse factor α is Specifically, the value of α is determined based on the deep convolutional network model and the specific application scenario.
[0092] Step S2024: Based on the number of significant convolution kernels, the significant convolution kernels are labeled using a binary labeling matrix in each convolution layer.
[0093] Specifically, after obtaining the number of significant convolution kernels in each convolution layer, the embodiment of the present invention uses a binary labeling matrix D to label the significant convolution kernels in each convolution layer. Taking the mth group in the lth convolution layer as an example, first The convolution kernels are sorted from high to low by 1-norm value:
[0094]
[0095] in, is the convolution kernel at the rth position in the above sorting The 1-norm of :
[0096]
[0097] Then the value of the nth row element of the significant convolution kernel binary label matrix D for the mth group is:
[0098]
[0099] Among them, D l is the binary label matrix of the significant convolution kernels of the lth convolution layer, which is obtained by comparing the norm of each convolution kernel with the norm of the rth largest convolution kernel in the group where the convolution kernel belongs. When α takes different values, taking the second convolution layer of VGG16 as an example, the distribution of significant convolution kernels is as follows: Figure 4 shown.
[0100] Step S203: Use the significant convolution kernel to measure the importance of the filter to obtain an importance measurement value.
[0101] Specifically, for the lth convolutional layer, the importance measure of the nth filter is shown as follows:
[0102]
[0103] In formula (9), is the 1-norm of the m-th convolution kernel parameter of the n-th filter in the l-th layer, is the mth mask of the nth filter in the lth layer, is the importance measure of the n-th filter in the l-th layer.
[0104] Formula (9) is the weighted 1-norm of the significant convolution kernel binary label matrix D to measure the importance of the filter. Then the importance measure value of each filter in the lth layer can be comprehensively expressed as a metric vector
[0105] Step S204, constructing an adaptive benchmark pruning rate for each convolutional layer, and pruning the filter of each convolutional layer based on the filter importance metric and the adaptive benchmark pruning rate. Specifically, step S204 includes the following process:
[0106] Step S2041, constructing a layer sensitivity sparse coefficient for each convolutional layer to be pruned, and adaptively obtaining a benchmark number of retained filters for each convolutional layer based on the layer sensitivity sparse coefficient.
[0107] For each convolutional layer to be pruned, construct the layer sensitivity sparse coefficient C l . Specifically:
[0108]
[0109]
[0110] In formula (10), ||S l ||1 is the sum of the importance scores of all filters in the lth layer, representing the sum of the feature extraction capabilities of the filters in the lth layer.
[0111] In formula (11), ||S l || ∞ is the single highest importance score for the l-th layer filter, representing the single maximum amount of feature extraction capability of the l-th layer filter.
[0112] In formula (12), It can be understood that the maximum amount of feature extraction of a single filter is used to allocate the number of filters required to achieve the feature extraction capability of the current layer, that is, the convolution layer should retain the baseline number of filters. Since the value distribution of the filter importance score of each convolution layer is different, The baseline number of filters that need to be retained in each convolutional layer can be adaptively obtained. l Subtract the baseline number of filters that need to be retained to obtain the baseline number C of filters to be pruned at each layer. l .
[0113] Step S2042, adjusting the sparsity of the baseline number of filters by setting the layer sensitivity sparsity factor to obtain the actual number z of pruning of each convolutional layer.
[0114] Specifically, the number of benchmarks C for pruning each layer is obtained adaptively l After that, by setting the layer sensitivity sparse factor β l The number of benchmarks is sparsely adjusted, and finally the actual number of pruning branches P of each layer is obtained. l .
[0115] P l =[βl C l ](13)
[0116] In formula (13), is the rounding symbol, and the value range of β is When β l When both are 1, it means that the benchmark pruning number is used directly as the actual pruning number. In convolutional neural networks, different convolutional layers have different focuses on image feature extraction, so the sparsity levels used are also different. The lower layers generally extract the overall image and texture, and then pass the information to the next layer. It is not suitable to prune too much, so β is generally set to a small value. Other convolutional layers can generally directly use the benchmark pruning number.
[0117] Step S2043, retain the filters whose filter importance metric value of each layer is greater than the z-th largest filter importance metric value of the layer.
[0118] Specifically, after obtaining the number of pruning branches P l After that, by comparing the importance score of the filter of each layer with the score of the filter with the largest zth largest value in the layer, the filter to be retained in each layer can be obtained. In order of importance from highest to lowest:
[0119]
[0120] in:
[0121] z=P l (15)
[0122] Finally, the pruning mask I of each layer of filters is obtained:
[0123]
[0124] In formula (16), Indicates whether the nth filter in layer l needs to be pruned. When it is 1, the corresponding filter needs to be retained; when When it is 0, the corresponding filter needs to be cut off.
[0125] Step S205, fine-tuning the pruned network model to obtain a lightweight network model with performance similar to that of the original network model.
[0126] Specifically, after the network model is pruned, the accuracy will generally decrease due to the loss of some weights. The decreased accuracy can be compensated by fine-tuning a certain number of times, so that the performance of the pruned compressed model is similar to that of the original network model. The specific fine-tuning process is the retraining process of the pruned model, which will not be repeated here.
[0127] The method provided by the embodiment of the present invention can effectively compress and accelerate the network model, greatly reduce the number of parameters and the amount of calculation of the model, and the accuracy is almost not reduced. It is applicable to a variety of common convolutional neural networks, such as VGG16, ResNet56, DenseNet40, etc.; it is also applicable to data sets of different scales, such as small-scale data sets CIFAR-10, CIFAR-100, large-scale data sets ImageNet, etc.
[0128] In this embodiment, a deep convolutional neural network model compression device is also provided, which is used to implement the above-mentioned embodiments and preferred implementation modes, and will not be repeated here. As used below, the term "module" can implement a combination of software and / or hardware for a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceivable.
[0129] This embodiment provides a deep convolutional neural network model compression device, such as Figure 5 As shown, including:
[0130] A pre-training module 501 is used to pre-train a preset deep convolutional neural network to obtain a network model to be pruned;
[0131] A significant convolution kernel selection module 502 is used to measure the information transfer efficiency of the convolution kernel of each convolution layer of the network model to be pruned, and use the convolution kernel whose information transfer efficiency measurement value meets the preset conditions as a significant convolution kernel;
[0132] The filter importance measurement module 503 is used to measure the importance of the filter using the significant convolution kernel to obtain an importance measurement value;
[0133] A pruning module 504, configured to construct an adaptive benchmark pruning rate for each convolutional layer, and prune the filter of each convolutional layer based on the filter importance metric and the adaptive benchmark pruning rate;
[0134] The model fine-tuning module 505 is used to fine-tune the pruned network model to obtain a lightweight network model with performance similar to that of the original network model.
[0135] In some optional implementations, the significant convolution kernel selection module 502 includes:
[0136] The information transfer efficiency measurement unit is used to measure the information transfer efficiency of the convolution kernel of each group in each convolution layer using the p-norm;
[0137] A unit for determining the benchmark number of significant convolution kernels, used to obtain the benchmark number of significant convolution kernels retained by each group in each convolution layer based on the intra-group sensitivity sparsity index and the information transfer efficiency metric value;
[0138] A significant convolution kernel number determination unit is used to set a significant convolution kernel sparse factor to adjust the sparseness of the benchmark number of significant convolution kernels to obtain the number of significant convolution kernels;
[0139] The significant convolution kernel labeling unit is used to label the significant convolution kernels using a binary labeling matrix in each convolution layer based on the number of significant convolution kernels.
[0140] In some optional embodiments, the significant convolution kernel marking unit includes:
[0141] In the mth group in the lth convolutional layer, the convolution kernel norm value
[0142] Sort by p-norm value from high to low:
[0143]
[0144] in, The convolution kernel at the rth position in the above sorting The p-norm of :
[0145]
[0146] Then the value of the nth row element of the significant convolution kernel binary label matrix D for the mth group is:
[0147]
[0148] D l is the binary label matrix of the lth convolutional layer, which is obtained by comparing the norm of each convolution kernel with the norm of the rth largest convolution kernel in the group where the convolution kernel belongs.
[0149] In some optional implementations, in the filter importance measurement module 503, for the lth convolution layer, the importance measurement value of the nth filter is calculated by the following formula:
[0150]
[0151] In the formula, is the p-norm of the m-th convolution kernel parameter of the n-th filter in the l-th layer, is the mth mask of the nth filter in the lth layer, is the importance measure of the nth filter in the lth layer.
[0152] In some optional implementations, the pruning module 504 includes:
[0153] A sparse adjustment unit is used to adjust the sparseness of the base number of filters by setting the layer sensitivity sparse factor to obtain the actual pruning number z of each convolutional layer;
[0154] The filter pruning unit is used to retain the filters whose filter importance metric value of each layer is greater than the z-th largest filter importance metric value of the layer.
[0155] In some optional implementations, the filter pruning unit includes:
[0156] Will In order of importance from highest to lowest:
[0157]
[0158] in:
[0159] z=P l
[0160] Get the pruning mask I of each layer of filters:
[0161]
[0162] In the formula, Indicates whether the nth filter of the lth layer needs to be pruned. When it is 1, the corresponding filter is retained; when When it is 0, the corresponding filter is clipped.
[0163] The further functional description of each of the above modules and units is the same as that of the above corresponding embodiments and will not be repeated here.
[0164] The deep convolutional neural network model compression device in this embodiment is presented in the form of a functional unit, where the unit refers to an ASIC (Application Specific Integrated Circuit) circuit, a processor and memory that executes one or more software or fixed programs, and / or other devices that can provide the above-mentioned functions.
[0165] The embodiment of the present invention also provides a computer device having the above Figure 5 The deep convolutional neural network model compression device shown.
[0166] See also Figure 6 , Figure 6 is a schematic diagram of the structure of a computer device provided by an optional embodiment of the present invention, such as Figure 6As shown, the computer device includes: one or more processors 10, a memory 20, and interfaces for connecting various components, including high-speed interfaces and low-speed interfaces. Various components are connected to each other using different buses for communication, and can be installed on a common mainboard or installed in other ways as needed. The processor can process the instructions executed in the computer device, including instructions stored in or on the memory to display the graphical information of the GUI on an external input / output device (such as, a display device coupled to the interface). In some optional embodiments, if necessary, multiple processors and / or multiple buses can be used together with multiple memories and multiple memories. Similarly, multiple computer devices can be connected, and each device provides some necessary operations (for example, as a server array, a group of blade servers, or a multi-processor system). Figure 6 A processor 10 is taken as an example.
[0167] The processor 10 may be a central processing unit, a network processor or a combination thereof. The processor 10 may further include a hardware chip. The hardware chip may be a dedicated integrated circuit, a programmable logic device or a combination thereof. The programmable logic device may be a complex programmable logic device, a field programmable gate array, a general purpose array logic or any combination thereof.
[0168] The memory 20 stores instructions executable by at least one processor 10, so that at least one processor 10 executes the method shown in the above embodiment.
[0169] The memory 20 may include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function; the data storage area may store data created according to the use of the computer device, etc. In addition, the memory 20 may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some optional embodiments, the memory 20 may optionally include a memory remotely arranged relative to the processor 10, and these remote memories may be connected to the computer device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.
[0170] The memory 20 may include a volatile memory, such as a random access memory; the memory may also include a non-volatile memory, such as a flash memory, a hard disk or a solid state drive; the memory 20 may also include a combination of the above types of memory.
[0171] The computer device further comprises a communication interface 30 for the computer device to communicate with other devices or a communication network.
[0172] The embodiment of the present invention also provides a computer-readable storage medium. The method according to the embodiment of the present invention can be implemented in hardware, firmware, or can be implemented as a computer code that can be recorded in a storage medium, or can be implemented as a computer code that is originally stored in a remote storage medium or a non-temporary machine-readable storage medium and will be stored in a local storage medium through a network download, so that the method described herein can be stored in such software processing on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. Among them, the storage medium can be a magnetic disk, an optical disk, a read-only storage memory, a random access memory, a flash memory, a hard disk or a solid-state hard disk, etc.; further, the storage medium can also include a combination of the above types of memories. It can be understood that a computer, a processor, a microprocessor controller, or programmable hardware includes a storage component that can store or receive software or computer code. When the software or computer code is accessed and executed by a computer, a processor, or hardware, the method shown in the above embodiment is implemented.
[0173] A part of the present invention may be applied as a computer program product, such as a computer program instruction, which, when executed by a computer, can call or provide the method and / or technical solution according to the present invention through the operation of the computer. Those skilled in the art should understand that the existence of the computer program instruction in a computer-readable medium includes, but is not limited to, a source file, an executable file, an installation package file, etc., and accordingly, the way in which the computer program instruction is executed by the computer includes, but is not limited to: the computer directly executes the instruction, or the computer compiles the instruction and then executes the corresponding compiled program, or the computer reads and executes the instruction, or the computer reads and installs the instruction and then executes the corresponding installed program. Here, the computer-readable medium may be any available computer-readable storage medium or communication medium accessible to the computer.
[0174] Although the embodiments of the present invention have been described in conjunction with the accompanying drawings, those skilled in the art may make various modifications and variations without departing from the spirit and scope of the present invention, and such modifications and variations are all within the scope defined by the appended claims.
Claims
1. A deep convolutional neural network model compression method, characterized in that: The method comprises: Pre-training the preset deep convolutional neural network to obtain the network model to be pruned; The information transfer efficiency of the convolution kernel of each convolution layer of the network model to be pruned is measured, and the convolution kernel whose information transfer efficiency measurement value meets the preset conditions is regarded as a significant convolution kernel; The importance of the filter is measured using the significant convolution kernel to obtain the importance measurement value; Construct an adaptive baseline pruning rate for each convolutional layer, and prune the filter of each convolutional layer based on the filter importance metric and the adaptive baseline pruning rate; The pruned network model is fine-tuned to obtain a lightweight network model with performance similar to the original network model.
2. The method according to claim 1, characterized in that The method measures the information transfer efficiency of the convolution kernel of each convolution layer of the network model to be pruned, and uses the convolution kernel whose information transfer efficiency measurement value meets the preset conditions as the significant convolution kernel, including: Use the p-norm to measure the information transfer efficiency of the convolution kernel of each group in each convolution layer; Based on the intra-group sensitivity sparsity index and information transfer efficiency metric, the benchmark number of significant convolution kernels retained by each group in each convolution layer is obtained; The significant convolution kernel sparse factor is set to adjust the sparseness of the baseline number of significant convolution kernels to obtain the number of significant convolution kernels; Based on the number of salient convolution kernels, a binary labeling matrix is used in each convolutional layer to label the salient convolution kernels.
3. The method according to claim 2, characterized in that The number of significant convolution kernels is calculated by the following formula: In the formula, is the rounding symbol; It is the benchmark number of significant convolution kernels retained by the mth group in the lth layer, representing the maximum number of equivalent transmission channels of input information. This value is obtained by dividing the sum of the group convolution kernels assigned to the output features by the single maximum number of group convolution kernels assigned to the output features. α is the significant convolution kernel sparse factor, and its value range is is the number of significant convolution kernels in the mth group of the lth layer.
4. The method according to claim 3, characterized in that The use of the significant convolution kernel to measure the importance of the filter to obtain the importance measurement value includes: For the lth convolutional layer, the importance metric of the nth filter is calculated as follows: In the formula, is the p-norm of the m-th convolution kernel parameter of the n-th filter in the l-th layer, is the mth mask of the nth filter in the lth layer, is the importance measure of the n-th filter in the l-th layer.
5. The method according to claim 1 or 4, characterized in that: The step of constructing an adaptive benchmark pruning rate for each convolutional layer and pruning the filter of each convolutional layer based on the filter importance metric value and the adaptive benchmark pruning rate includes: For each convolutional layer to be pruned, a layer sensitivity sparse coefficient is constructed, and based on the layer sensitivity sparse coefficient, a benchmark number of retained filters for each convolutional layer is adaptively obtained; By setting the layer sensitivity sparse factor, the base number of filters is adjusted to achieve the actual number of pruned branches z for each convolutional layer. The filters whose filter importance metric value of each layer is greater than the z-th largest filter importance metric value in the layer are retained.
6. The method according to claim 5, characterized in that The step of retaining filters whose filter importance metric value of each layer is greater than the filter importance metric value of the zth largest filter in the layer includes: Will In order of importance from highest to lowest: in: z=P l Get the pruning mask I of each layer of filters: Where P l is the actual number of pruning branches at each layer, Indicates whether the nth filter of the lth layer needs to be pruned. When it is 1, the corresponding filter is retained; when When it is 0, the corresponding filter is clipped.
7. A deep convolutional neural network model compression device, characterized in that: The device comprises: A pre-training module is used to pre-train a preset deep convolutional neural network to obtain a network model to be pruned; The significant convolution kernel selection module is used to measure the information transfer efficiency of the convolution kernel of each convolution layer of the network model to be pruned, and the convolution kernel whose information transfer efficiency measurement value meets the preset conditions is regarded as a significant convolution kernel; A filter importance measurement module is used to measure the importance of the filter using a significant convolution kernel to obtain an importance measurement value; A pruning module is used to construct an adaptive benchmark pruning rate for each convolutional layer and prune the filter of each convolutional layer based on the filter importance metric and the adaptive benchmark pruning rate; The model fine-tuning module is used to fine-tune the pruned network model to obtain a lightweight network model with performance similar to the original network model.
8. A computer device, characterized in that: include: A memory and a processor, wherein the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the deep convolutional neural network model compression method described in any one of claims 1 to 6 by executing the computer instructions.
9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a computer to execute the deep convolutional neural network model compression method described in any one of claims 1 to 6.
10. A computer program product, characterized in that It includes computer instructions, which are used to cause a computer to execute the deep convolutional neural network model compression method described in any one of claims 1 to 6.
Citation Information
Cited By
Text reasoning method based on online structure pruning and related equipment
CN119647605A
