Channel pruning method and device, computer device and storage medium

Abstract

The application relates to a channel pruning method and device, a computer device and a storage medium, and is applied to a neural network comprising a convolution layer, the neural network is applied to image processing, the convolution layer comprises a channel, the method comprises the following steps: acquiring the rank of the convolution layer according to image data; acquiring the entropy of the convolution layer according to the image data; acquiring the overall index of the convolution layer according to the rank and the entropy of the convolution layer; obtaining the pruning ratio of each convolution layer according to the overall index; acquiring the Shapley value of the channel in the convolution layer; and pruning the channel in the convolution layer according to the Shapley value of the channel and the pruning ratio of the convolution layer. The channel pruning method can improve the performance of the neural network through channel pruning, thereby improving the efficiency of image processing.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of image processing, in particular to a channel pruning method and device, a computer device and a storage medium. BACKGROUND

[0002] A neural network is a network that can realize nonlinear operation while having high redundancy and high information content, and the neural network can include a convolutional layer. The convolutional neural network achieves excellent performance in computer vision task processing, such as image processing. However, the neural network including the convolutional layer requires a large number of parameters and a large amount of calculation cost, which makes it difficult to deploy on mobile devices and embedded devices. Even if the architecture of the neural network is adjusted, excessive parameterization and redundancy still exist.

[0003] In order to solve the above problems, it is necessary to compress the neural network including the convolutional layer, and pruning the channels included in the convolutional layer is an effective method for accelerating and compressing the neural network. In the prior art, the specific pruning ratio of different layers is generally defined according to experience, and the pruning standard is manually formulated, which results in low pruning efficiency and poor pruning effect, and further affects the performance of the neural network. SUMMARY

[0004] The technical problem to be solved by the present application is that the pruning standard needs to be manually specified during channel pruning, which is inefficient and ineffective.

[0005] To solve the above problems, the present application provides a channel pruning method, which is applied to a neural network including a convolutional layer, the neural network is applied to image processing, and the convolutional layer includes channels. The method comprises:

[0006] According to the image data, the rank of the convolutional layer is obtained;

[0007] According to the image data, the entropy of the convolutional layer is obtained;

[0008] According to the rank of the convolutional layer and the entropy of the convolutional layer, the overall index of the convolutional layer is obtained;

[0009] According to the overall index, the pruning ratio of each convolutional layer is obtained;

[0010] The Shapley value of the channel in the convolutional layer is obtained;

[0011] According to the Shapley value of the channel and the pruning ratio of the convolutional layer, the channel in the convolutional layer is pruned.

[0012] Optionally, the image data includes the number of layers, and before the rank of the convolutional layer is obtained according to the image data, the method further comprises:

[0013] obtaining a total number of channels of the convolution layer;

[0014] the obtaining the rank of the convolution layer comprises:

[0015] obtaining a feature level corresponding to each of the channels in the convolution layer according to the image data;

[0016] obtaining a total value of the feature levels of the convolution layer according to all the feature levels;

[0017] obtaining a rank of each of the convolution layers according to the total value of the feature levels, the total number of channels of the convolution layer and a rank formula, the rank formula being:

[0018]

[0019] wherein R(C i ) is the rank of the convolution layer, n i is the total number of channels of the convolution layer C i , j is the jth channel in C i , B is the number of the layers, b is the bth layer in B, is the feature level.

[0020] Optionally, the obtaining the entropy of the convolution layer comprises:

[0021] obtaining a probability distribution of the output of the channel according to the total number of channels, the total value of the feature levels and a normalization function formula, the normalization function formula being:

[0022]

[0023] wherein n i is the number of channels of the convolution layer C i , j is the jth channel in C i , B is the number of the layers, b is the bth layer in B, as is the probability distribution of the output of the convolution C i .

[0024] obtaining the entropy of the convolution layer according to the probability distribution, the total number of channels and an entropy formula, the entropy formula being:

[0025]

[0026] wherein R(C i ) is the rank of the convolution layer, n i is the number of channels of the convolution layer C i , j is the jth channel in C iB is the number of the layers, b is the bth layer in B, is the rank of the feature.

[0027] Optionally, the total index of the convolution layer is obtained according to the rank of the convolution layer and the entropy of the convolution layer, comprising:

[0028] A normalized range is obtained according to the rank of the convolution layer and the entropy of the convolution layer;

[0029] A fusion function is obtained according to the normalized range;

[0030] The total index is obtained according to the fusion function.

[0031] Optionally, the total number of channels includes a number of deleted channels, and the pruning ratio of each convolution layer is obtained according to the total index, comprising:

[0032] A fusion value is obtained according to the total index;

[0033] The number of deleted channels is obtained according to the fusion value;

[0034] The pruning ratio of each convolution layer is obtained according to the number of deleted channels and the total number of channels.

[0035] Optionally, the Shapley value of the channel in the convolution layer is obtained, comprising:

[0036] Any two or more channels in the convolution layer are selected as an alliance to obtain a plurality of alliances;

[0037] A characteristic equation is obtained according to the alliance, and it is judged whether the characteristic equation meets a preset condition, if the preset condition is met, then:

[0038] The marginal contribution value of the channel pair to all alliances including the channel is obtained according to the characteristic equation and a marginal contribution formula,

[0039] The Shapley value of the channel is obtained according to the marginal contribution value, the alliance, the total number of channels and an average contribution formula.

[0040] Optionally, the channel includes a deleted channel and a remaining channel;

[0041] Before the channel in the convolution layer is pruned according to the Shapley value of the channel and the pruning ratio of the convolution layer, the method further comprises:

[0042] The number of convolution layers is obtained,

[0043] According to the deleted channel and the remaining channel, an indicator function of the channel is obtained;

[0044] According to the channel Shapley value and the pruning ratio of the convolutional layer, the channel in the convolutional layer is pruned.

[0045] According to the Shapley value, the indicator function, the total number of channels, the number of convolutional layers, and a pruning function, a pruning parameter is obtained.

[0046] According to the pruning parameter and the pruning ratio, the channel is pruned.

[0047] The channel pruning method provided by the application, for a neural network used for image processing, defines redundancy and information amount indicators in a convolutional layer included in the neural network as a rank of the convolutional layer and an entropy of the convolutional layer, obtains a corresponding pruning ratio through an information set according to a fusion function, uses a Shapley value of each channel to represent a contribution size of the channel to optimization, and prunes according to the contribution size and the pruning ratio, thereby reducing harm to the neural network, improving performance of the neural network, and improving efficiency of image processing.

[0048] The application further provides a channel pruning device applied to a neural network including a convolutional layer, the neural network being applied to image processing, and the convolutional layer including channels.

[0049] An obtaining unit is configured to obtain a rank of the convolutional layer according to image data, and obtain an entropy of the convolutional layer according to the image data.

[0050] An analyzing unit is configured to obtain an overall indicator of the convolutional layer according to the rank of the convolutional layer and the entropy of the convolutional layer, and obtain a pruning ratio of each convolutional layer according to the overall indicator.

[0051] The obtaining unit is further configured to obtain a Shapley value of a channel in the convolutional layer.

[0052] A pruning unit is configured to prune the channel in the convolutional layer according to the Shapley value of the channel and the pruning ratio of the convolutional layer.

[0053] The channel pruning device provided by the application, for a neural network used for image processing, defines redundancy and information amount indicators in a convolutional layer included in the neural network as a rank of the convolutional layer and an entropy of the convolutional layer, obtains a corresponding pruning ratio through an information set according to a fusion function, uses a Shapley value of each channel to represent a contribution size of the channel to optimization, and prunes according to the contribution size and the pruning ratio, thereby reducing harm to the neural network, improving performance of the neural network, and improving efficiency of image processing.

[0054] The application further provides a computer device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the processor implements any one of the channel pruning methods.

[0055] The computer device of the application, for a neural network used for image processing, defines redundancy and information amount indexes in a convolution layer included in the neural network as a rank of the convolution layer and an entropy of the convolution layer, and according to a fusion function, the indexes are taken as an information set of the convolution layer, a corresponding pruning ratio is obtained from the information set, and at the same time, a Shapley value of each channel is used to represent a contribution size of the channel to optimization, pruning is performed according to the contribution size and the pruning ratio, harm to the neural network is reduced, performance of the neural network is improved, and thus efficiency of image processing is improved.

[0056] The application further provides a computer readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements any one of the channel pruning methods.

[0057] The computer readable storage medium of the application, for a neural network used for image processing, defines redundancy and information amount indexes in a convolution layer included in the neural network as a rank of the convolution layer and an entropy of the convolution layer, and according to a fusion function, the indexes are taken as an information set of the convolution layer, a corresponding pruning ratio is obtained from the information set, and at the same time, a Shapley value of each channel is used to represent a contribution size of the channel to optimization, pruning is performed according to the contribution size and the pruning ratio, harm to the neural network is reduced, performance of the neural network is improved, and thus efficiency of image processing is improved. BRIEF DESCRIPTION OF DRAWINGS

[0058] Figure 1 A channel pruning method flowchart in an embodiment of the application;

[0059] Figure 2 A channel pruning method flowchart in an embodiment of the application;

[0060] Figure 3 A channel pruning method flowchart in an embodiment of the application;

[0061] Figure 4 A channel pruning method flowchart in an embodiment of the application;

[0062] Figure 5 A channel pruning method flowchart in an embodiment of the application;

[0063] Figure 6 A channel pruning method flowchart in an embodiment of the application;

[0064] Figure 7 Performance comparison chart for verifying the performance of the channel pruning method in the embodiment of the application;

[0065] Figure 8 Performance comparison chart for verifying the performance of the channel pruning method in the embodiment of the application;

[0066] Figure 9 Performance comparison chart for verifying the performance of the channel pruning method in the embodiment of the application;

[0067] Figure 10 Performance comparison chart for verifying the performance of the channel pruning method in the embodiment of the application;

[0068] Figure 11 Performance comparison chart for verifying the performance of the channel pruning method in the embodiment of the application;

[0069] Figure 12 Performance comparison chart for verifying the performance of the channel pruning method in the embodiment of the application;

[0070] Figure 13 Pruning ratio and fusion value fitting curve schematic diagram of the channel pruning method in the embodiment of the application;

[0071] Figure 14 Pruning ratio and fusion value fitting curve schematic diagram of the channel pruning method in the embodiment of the application;

[0072] Figure 15 Channel pruning device schematic diagram in the embodiment of the application;

[0073] Figure 16 Computer device schematic diagram in the embodiment of the application. DETAILED DESCRIPTION

[0074] To make the objectives, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be described below in connection with the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative work fall within the scope of protection of the present application.

[0075] The embodiment provides a channel pruning method, which is applied to a neural network comprising a convolution layer, the neural network is applied to image processing, the convolution layer comprises a channel, and the method comprises the following steps:

[0076] S1: acquiring the rank of the convolution layer according to image data;

[0077] S2: acquiring the entropy of the convolution layer according to the image data;

[0078] S3: obtaining an overall index of the convolutional layer according to the rank of the convolutional layer and the entropy of the convolutional layer;

[0079] S4: obtaining a pruning ratio of each of the convolutional layers according to the overall index;

[0080] S5: obtaining a shapley value of a channel in the convolutional layer;

[0081] S6: pruning the channel in the convolutional layer according to the shapley value of the channel and the pruning ratio of the convolutional layer.

[0082] Wherein, in processing the image, the convolutional layer with low rank contains a large amount of redundant information, and needs to be compressed more compactly. Since each feature map under different image processing remains almost unchanged, the rank of the convolutional layer can be estimated by the image;

[0083] The convolutional layer with low entropy includes a low amount of channel information, so the entropy of the convolutional layer is estimated by the given input image;

[0084] The rank of the convolutional layer corresponds to the redundancy of the convolutional layer output, and the entropy of the convolutional layer corresponds to the information amount of the convolutional layer output;

[0085] The overall index of the convolutional layer is then obtained by the rank of the convolutional layer and the entropy of the convolutional layer, as a concentration of layer information, and the pruning ratio of the convolutional layer is obtained according to the concentration of layer information;

[0086] When pruning the channel according to the pruning ratio, the channel with the smallest contribution to loss optimization needs to be pruned, therefore, the shapley value is used to evaluate the contribution of the channel, and the channel is pruned according to the shapley value.

[0087] In combination with Figure 2 and Figure 3 As shown, when processing the image batch, it is divided into two parts, including a preprocessing stage and a pruning stage;

[0088] In the preprocessing stage, a sample image batch is input to obtain the rank and entropy of each channel (the channel is represented by "C") in the convolutional layer (the rank is represented by "R" and the entropy is represented by "E"), the corresponding fusion value (the fusion value is represented by "F") and the shapley value (the shapley value is represented by "S");

[0089] In the pruning stage, the channels in a layer are regarded as players, and a negative shapley value indicates that the player has made an adverse contribution to cooperation. In each layer, the channel with the smallest shapley value is discarded.

[0090] The channel pruning method of the application, for a neural network for image processing, defines redundancy and information amount indicators in a convolution layer included in the neural network as a rank of the convolution layer and an entropy of the convolution layer, obtains an overall indicator according to the rank of the convolution layer and the entropy of the convolution layer, obtains a corresponding pruning ratio through the overall indicator, and uses a Shapley value of each channel to represent the contribution size of the channel to optimization, and prunes according to the contribution size and the pruning ratio, reduces the harm to the neural network, improves the performance of the neural network, and thus improves the efficiency of image processing.

[0091] In the embodiment of the application, the image data includes a number of layers, and before the rank of the convolution layer is obtained according to the image data, the method further includes:

[0092] obtaining a total number of channels of the convolution layer;

[0093] The rank of the convolution layer is obtained by:

[0094] obtaining a feature level corresponding to each channel in the convolution layer according to the image data;

[0095] obtaining a total value of feature levels of the convolution layer according to all the feature levels;

[0096] obtaining a rank of each convolution layer according to the total value of feature levels, the total number of channels of the convolution layer, and a rank formula, the rank formula being:

[0097]

[0098] wherein R(C i ) is the rank of the convolution layer, n i is the total number of channels of the convolution layer C i , j is the jth channel in C i , B is the number of layers, b is the bth layer in B, is the feature level.

[0099] In the embodiment, before the rank of the convolution layer is calculated, image data is obtained, a small number of image batches are used, fixed images B are selected in the images, each layer is mapped through the fixed images B, a feature level corresponding to each channel is obtained, the feature levels of all channels are summed to obtain a total value of feature levels, and the rank of the convolution layer is obtained through a rank formula according to the number of channels and the total value of feature levels.

[0100] wherein the rank formula is:

[0101]

[0102] R(C iis the rank of the convolution layer, n i is the total number of channels of the convolution layer C i , j is the jth channel in C i , B is the number of graph layers, b is the bth graph layer in B, is the level of the feature.

[0103] The channel pruning method of the application represents the information concentration of the convolution layer through the rank of the convolution layer, which facilitates the subsequent acquisition of the pruning ratio and provides a reasonable pruning standard, so that the image processing precision is higher.

[0104] In the embodiment of the application, the entropy of the convolution layer is obtained by:

[0105] According to the total number of channels, the total value of the feature level and the normalization function formula, the probability distribution of the output of the channel is obtained, and the normalization function formula is:

[0106]

[0107] wherein n i is the number of channels of the convolution layer C i , j is the jth channel in C i , B is the number of graph layers, b is the bth graph layer in B, as the output of the convolution measurement C i probability distribution;

[0108] According to the probability distribution, the total number of channels and the entropy formula, the entropy of the convolution layer is obtained, and the entropy formula is:

[0109]

[0110] wherein R(C i ) is the rank of the convolution layer, n i is the number of channels of the convolution layer C i , j is the jth channel in C i , B is the number of graph layers, b is the bth graph layer in B, is the level of the feature.

[0111] In the embodiment, since the entropy of the convolution layer is similar to the level of the graph layer, the softmax function is used to map the channel between 0 and 1, the output of the convolution measurement C i is displayed as a probability distribution, and the softmax function is:

[0112]

[0113] wherein n i is the total number of channels of the convolution layer C i , j is the jth channel in C i , B is the number of graph layers, b is the bth graph layer in B, as the probability distribution output by the convolution layer C i ;

[0114] After obtaining H(C ), the average entropy of the convolution layer is obtained according to an entropy formula:

[0115]

[0116] wherein H(C i ) is the entropy of the convolution layer, is the probability distribution output by the convolution layer C i , j is the jth channel in C i , and n i is the total number of channels of the convolution layer C i .

[0117] The channel pruning method provided by the application can represent the information of the convolution layer through the rank of the convolution layer, so that the pruning ratio can be obtained easily, a reasonable pruning standard is provided, the harm to the neural network is reduced, the performance of the neural network is improved, the accuracy of image processing is higher, and the efficiency of image processing is improved.

[0118] In the embodiment of the application, the total index of the convolution layer is obtained according to the rank of the convolution layer and the entropy of the convolution layer, and the method comprises the following steps:

[0119] A normalized range is obtained according to the rank of the convolution layer and the entropy of the convolution layer.

[0120] A fusion function is obtained according to the normalized range.

[0121] The total index is obtained according to the fusion function.

[0122] In the embodiment, when the rank and the entropy of the convolution layer are obtained, it can be found that the rank and the entropy of the output of the convolution layer remain almost unchanged, but the values of the rank and the entropy fluctuate slightly under different image batches, and the internal changes between the ranks and the entropies are not completely consistent. Therefore, in order to eliminate the above inconsistencies, a fusion function is defined, and the rank and the entropy are normalized to the range [a, b], the fusion function includes the two indexes, and a total index is obtained as the information of the convolution layer, wherein the fusion function is:

[0123]

[0124] wherein, O(C i ) is the fusion value of the convolution layer C i , Y represents R(C i ) and H(C i ), and Z represents {R(C i ) and H(C i )}, L is the number of convolution layers; O(C i ) is normalized to the range [a, b]; the smaller the fusion value of the convolution layer is, the less information the convolution layer has, and thus the pruning ratio of the convolution layer is reduced.

[0125] The channel pruning method provided by the application obtains an overall index through a fusion function, and the overall index is used as an information set of the whole layer, the fusion value of the information set is used to allocate the pruning ratio of each convolution layer, the harm to the neural network is reduced, the performance of the neural network is improved, and the efficiency of image processing is improved by processing the image according to the pruning rule.

[0126] In the embodiment of the application, the total number of channels includes the number of deleted channels, and the pruning ratio of each convolution layer is obtained according to the overall index, including:

[0127] The fusion value is obtained according to the overall index;

[0128] The number of deleted channels is obtained according to the fusion value;

[0129] The pruning ratio of each convolution layer is obtained according to the number of deleted channels and the total number of channels.

[0130] In the embodiment, the channels in the convolution layer C i can be divided into two groups, including deleted channels and remaining channels, the deleted channels U i , and the remaining channels Q i . The pruning ratio of each convolution layer is obtained according to the number of deleted channels and the total number of channels.

[0131] The channel pruning method provided by the application allocates the pruning ratio of the convolution layer through the fusion value, that is, the pruning ratio of the convolution layer is allocated according to the redundancy and information amount of each convolution layer, which replaces the technical solution of manually setting the pruning ratio in the prior art, and the sensitivity of compression is referred to, so that the pruning of the convolution layer is more accurate, the accuracy of image processing is higher, and the harm to the neural network is reduced.

[0132] In the embodiment of the application, the Shapley value of the channel in the convolution layer is obtained, including: ​

[0133] Select any two or more channels from all channels in the convolution layer as a coalition, and obtain multiple coalitions;

[0134] Obtain a characteristic equation according to the coalition, and determine whether the characteristic equation satisfies a preset condition, if the preset condition is satisfied, then:

[0135] According to the characteristic equation and a marginal contribution formula, obtain a marginal contribution value of the channel pair to all coalitions including the channel,

[0136] According to the marginal contribution value, the coalition, the total number of channels and an average contribution formula, obtain a Shapley value of the channel.

[0137] In the embodiment, value is produced from an environment in which players participate in cooperation, and the reward is the personal contribution of the players, which is called the Shapley value, and a channel pruning method is introduced on this basis, the convolution layer is regarded as a game in which multiple players cooperate to produce output, and the n i channels in the convolution layer C i are regarded as a collection, that is, Select any two or more channels in the P set as a coalition m, and obtain multiple coalitions, wherein Obtain a characteristic equation v(m) according to the coalition m, if the following conditions are satisfied: and Non-intersecting subsets m1, v(m1∪m2)≥v(m1)+v(m2), calculate the marginal contribution of the channel pair to all coalitions including the channel according to a marginal contribution formula, and the marginal contribution formula is:

[0138]

[0139] If the function f^ maps each subset of channels from the activation output to a real number, obtain the Shapley value through an average contribution formula, and the average contribution formula is:

[0140]

[0141] Wherein, Indicates the average contribution of the channel pair to the convolution layer, that is, the Shapley value.

[0142] The channel pruning method of the application uses the Shapley value of each channel to represent the contribution size of the channel to optimization, prunes according to the contribution size and the pruning ratio, reduces the harm to the neural network, improves the performance of the neural network, makes the image processing more accurate, and improves the efficiency of image processing.

[0143] In the embodiments of the present application, the channels include deleted channels and remaining channels;

[0144] Before pruning the channels in the convolutional layer according to the Shapley values of the channels and the pruning ratio of the convolutional layer, the method further comprises:

[0145] obtaining the number of convolutional layers,

[0146] obtaining an indicator function of the channels according to the deleted channels and the remaining channels;

[0147] The pruning of the channels in the convolutional layer according to the Shapley values of the channels and the pruning ratio of the convolutional layer comprises:

[0148] obtaining a pruning parameter according to the Shapley values, the indicator function, the total number of channels, the number of convolutional layers and a pruning function,

[0149] pruning the channels according to the pruning parameter and the pruning ratio.

[0150] In the embodiments, the number of convolutional layers is obtained and set as L, and the pruning of the channels is expressed as an optimization problem through a pruning function, wherein the pruning function is:

[0151]

[0152] wherein, is the indicator function, when is greater than 0, then when is greater than 0, then Through the pruning function, the deleted channel information is reduced as much as possible, that is, the least important channel is identified.

[0153] The channel pruning method of the present application uses the Shapley value of each channel to represent the contribution size of the channel to optimization, and prunes according to the contribution size and the pruning ratio, thereby providing a reasonable pruning standard for channel pruning, improving the accuracy of image processing, reducing the harm to the neural network, and improving the performance of the neural network.

[0154] For verification of the channel pruning method, different architectures can be pruned on CIFAR-10 and ImageNet, including VGGNet, ResNet and DenseNet. The experiments are performed on two NVIDIA RTX 3090 cpus, 1024 and 128 images are randomly selected for estimating the convolutional layer information of CIFAR-10 and ImageNet, respectively; and the range is set to [1, 10] for scaling the rank, entropy and aggregation value of the convolutional layer.

[0155] The 200 models are trained on CIFAR-10 using SGD, and the 90 models are trained on ImageNet; the batch size is 256, and the initial learning rate is 0.1; the Top-1 accuracy on CIFAR-10 and the Top-1 and Top-5 accuracy on ImageNet are used to evaluate the classification ability; the FLOP and the parameter are used to reflect the acceleration and the compression ratio; on CIFAR-10, the Shapley value of each layer is calculated according to the pre-trained model;

[0156] The rank, the entropy and the corresponding fusion value of the convolution layer output are obtained by Eqn, and the "stage" with the same feature map size is used to represent a stack of layers; in each stage, the fusion value is summed and divided by the number of layers, and the fusion value of the stage composed of the shallower layers is greater than the fusion value of the stage composed of the deeper layers, which indicates that the information concentration of the shallower layers is higher than that of the deeper layers; therefore, the pruning ratio of the shallower layers can be set to a smaller value, and the pruning ratio of the deeper layers should be set to a larger pruning ratio; in the experiment, the network is pruned in the manner of each stage, that is, the pruning ratio of a stage represents the total number of channels to be deleted in the layers within the stage;

[0157] In combination with Figures 4 to 12 As shown in FIG. 1, the performance of the VGG-16, the ResNet-56 / 110 and the DenseNet-40 pruned by the method on CIFAR-10;

[0158] The accuracy change data are obtained by experiments on different network structures;

[0159] Among them, compared with SSS and CP, the accuracy of CICC decreases by a lower rate (0.74% v.s. SSS 0.94%, CP 2.46%) for VGG-16; for ResNet-56, the accuracy of CICC decreases by 0.28% at a larger acceleration ratio (FPGM 58.1% v.s. 52.6%, DBP 52.0%) and a compression ratio (DBP 43.9% v.s. 40.0%), which is lower than that of FPGM (0.33%) and DBP (0.42%); for ResNet-110, the accuracy of CICC increases by 0.88%, which is higher than that of SFP (0.18%) and HRank (0.73%); in addition, the accuracy of CICC increases by (0.48%), which is higher than that of FPGM (0.16%); for DenseNet-40, compared with HRank, CICC has the potential to compress the model, specifically, although the accuracy of CICC decreases by 0.14%, a compression ratio of 51.9% is achieved;

[0160] Compared with different accelerations, when the acceleration ratio is not more than 51.8%, the channel pruning method of the application has higher accuracy than the baseline model (93.68%), the channel pruning method of the application injects more effective intervals, which helps to standardize the neural network, provides reasonable pruning criteria for channel pruning, reduces the harm to the neural network, improves the performance of the neural network, and thus improves the efficiency of image processing.

[0161] In combination Figure 13 and Figure 14 As shown in the fitting curve between the pruning ratio and the fusion value, first, an exponential function is obtained to fit the relationship between the total number of channels to be deleted and the fusion value of the convolution layer: f(x) = a*x b +b, where a = 44.58, b = -3.56, and c = 11.85, in combination Figures 15 to 16 As shown in the fitting curve between the pruning ratio and the fusion value, three curves for estimating the relationship are shown.

[0162] In combination Figure 15 As shown in the fitting curve between the pruning ratio and the fusion value, three curves for estimating the relationship are shown.

[0163] The acquisition unit 110 is configured to acquire the rank of the convolution layer according to the image data, and acquire the entropy of the convolution layer according to the image data;

[0164] The analysis unit 120 is configured to acquire the overall index of the convolution layer according to the rank of the convolution layer and the entropy of the convolution layer, and obtain the pruning ratio of each convolution layer according to the overall index;

[0165] The acquisition unit 110 is further configured to acquire the Shapley value of the channel in the convolution layer;

[0166] The pruning unit 130 is configured to prune the channel in the convolution layer according to the Shapley value of the channel and the pruning ratio of the convolution layer.

[0167] In an embodiment of the present application,

[0168] The acquisition unit 110 is further configured to acquire the total number of channels of the convolution layer, and acquire the feature level corresponding to each channel in the convolution layer according to the image data;

[0169] According to all the feature levels, the total feature level value of the convolution layer is acquired;

[0170] According to the total feature level value, the total number of channels of the convolution layer, and the rank formula, the rank of each convolution layer is acquired;

[0171] According to the total number of channels, the total value of feature levels, and a normalization function formula, a probability distribution of an output of the channel is obtained.

[0172] According to the probability distribution, the total number of channels, and an entropy formula, an entropy of the convolution layer is obtained.

[0173] The analysis unit 120 is further configured to obtain a normalization range according to the rank of the convolution layer and the entropy of the convolution layer.

[0174] According to the normalization range, a fusion function is obtained.

[0175] According to the fusion function, the overall index is obtained.

[0176] According to the overall index, a fusion value is obtained.

[0177] According to the fusion value, the number of deleted channels is obtained.

[0178] According to the number of deleted channels and the total number of channels, a pruning ratio of each convolution layer is obtained.

[0179] The obtaining unit 110 is further configured to select any two or more channels from all channels in the convolution layer as an alliance, and obtain a plurality of alliances.

[0180] According to the alliance, a feature equation is obtained, and it is determined whether the feature equation satisfies a preset condition, if the preset condition is satisfied, then:

[0181] According to the feature equation and a marginal contribution formula, a marginal contribution value of the channel to all alliances including the channel is obtained.

[0182] According to the marginal contribution value, the alliance, the total number of channels, and an average contribution formula, a Shapley value of the channel is obtained.

[0183] The obtaining unit 110 is further configured to obtain a number of convolution layers, and obtain an indicator function of the channel according to the deleted channel and the remaining channel.

[0184] The pruning unit 130 is further configured to obtain a pruning parameter according to the Shapley value, the indicator function, the total number of channels, the number of convolution layers, and a pruning function, and prune the channel according to the pruning parameter and the pruning ratio.

[0185] The channel pruning device of the application, for a neural network used for image processing, defines redundancy and information amount indexes in a convolution layer included in the neural network as a rank of the convolution layer and an entropy of the convolution layer, obtains a corresponding pruning ratio through information concentration according to a fusion function, and uses a Shapley value of each channel to represent a contribution size of the channel to optimization, and prunes according to the contribution size and the pruning ratio, reduces harm to the neural network, improves performance of the neural network, and thus improves efficiency of image processing.

[0186] In combination Figure 16 The application further provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, and the processor implements the following steps when executing the computer program:

[0187] S1: obtaining a rank of the convolution layer according to image data;

[0188] S2: obtaining an entropy of the convolution layer according to the image data;

[0189] S3: obtaining an overall index of the convolution layer according to the rank of the convolution layer and the entropy of the convolution layer;

[0190] S4: obtaining a pruning ratio of each convolution layer according to the overall index;

[0191] S5: obtaining a Shapley value of a channel in the convolution layer;

[0192] S6: pruning the channel in the convolution layer according to the Shapley value of the channel and the pruning ratio of the convolution layer.

[0193] The computer device of the application, for a neural network used for image processing, defines redundancy and information amount indexes in a convolution layer included in the neural network as a rank of the convolution layer and an entropy of the convolution layer, obtains a corresponding pruning ratio through information concentration according to a fusion function, and uses a Shapley value of each channel to represent a contribution size of the channel to optimization, and prunes according to the contribution size and the pruning ratio, reduces harm to the neural network, improves performance of the neural network, and thus improves efficiency of image processing.

[0194] The application further provides a computer readable storage medium, in which a computer program is stored, and the computer program is executed by a processor to implement the following steps:

[0195] S1: obtaining a rank of the convolution layer according to image data;

[0196] S2: obtaining an entropy of the convolution layer according to the image data;

[0197] S3: obtaining an overall index of the convolutional layer according to the rank of the convolutional layer and the entropy of the convolutional layer;

[0198] S4: obtaining a pruning ratio of each of the convolutional layers according to the overall index;

[0199] S5: obtaining a Shapley value of a channel in the convolutional layer;

[0200] S6: pruning the channel in the convolutional layer according to the Shapley value of the channel and the pruning ratio of the convolutional layer.

[0201] The computer readable storage medium of the present application, for a neural network for image processing, defines the redundancy and information amount index in the convolutional layer included in the neural network as the rank of the convolutional layer and the entropy of the convolutional layer, and obtains the corresponding pruning ratio through the information set according to the fusion function, and at the same time, the Shapley value of each channel is used to represent the contribution size of the channel to optimization, and pruning is performed according to the contribution size and the pruning ratio, which reduces the harm to the neural network and improves the performance of the neural network, thereby improving the efficiency of image processing.

[0202] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be completed by a computer program instructing related hardware, and the program can be stored in a non-volatile computer readable storage medium. When the program is executed, it can include the processes of the above-mentioned embodiments. Any reference to memory, storage, database or other medium used in the embodiments provided by the present application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration but not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0203] It has to be noted that, in the present document, relational terms are intended only to convey a possible relationship between elements or

[0204] The foregoing is considered as illustrative only of the principles of the application. Numerous modifications and changes will readily occur to those skilled in the art, which modifications and changes are to be understood as intended to be encompassed by the general scope of the application. Accordingly, the application is not to be limited to the above described or illustrated embodiments, but is intended to encompass all embodiments consistent with the principles of the application.

Claims

1. A method of trimming a passageway, characterized by, The method is applied to a neural network comprising convolutional layers, the neural network being applied to image processing, the convolutional layers comprising channels, and the method comprising: obtaining ranks of the convolutional layers according to image data; obtaining entropies of the convolutional layers according to the image data; obtaining an overall index of the convolutional layers according to the ranks of the convolutional layers and the entropies of the convolutional layers, specifically comprising: obtaining a normalization range according to the ranks of the convolutional layers and the entropies of the convolutional layers; obtaining a fusion function according to the normalization range; and obtaining the overall index according to the fusion function; the fusion function being: wherein, is the fusion value of the convolution layer , Y represents and , Z represents and , , L is the number of convolution layers; and is normalized to the range [a, b]. obtaining pruning ratios of each of the convolutional layers according to the overall index, specifically comprising: obtaining a fusion value according to the overall index; obtaining a number of deleted channels according to the fusion value; and obtaining the pruning ratio of each of the convolutional layers according to the number of deleted channels and a total number of channels; obtaining Shapley values of the channels in the convolutional layers; pruning the channels in the convolutional layers according to the Shapley values of the channels and the pruning ratios of the convolutional layers.

2. The method of claim 1, wherein, The image data comprises a number of layers, and before the step of obtaining the ranks of the convolutional layers according to the image data, the method further comprises: obtaining the total number of channels of the convolutional layers; the step of obtaining the ranks of the convolutional layers according to the image data comprises: obtaining feature levels corresponding to each of the channels in the convolutional layers according to the image data; obtaining a total value of the feature levels of the convolutional layers according to all the feature levels; obtaining the rank of each of the convolutional layers according to the total value of the feature levels, the total number of channels of the convolutional layers, and a rank formula, the rank formula being: in, It is the rank of the convolutional layer. It is a convolutional layer The total number of channels, j is In B, the j-th channel is defined, B is the number of layers, and b is the b-th layer in B. It is the feature level.

3. The method of claim 2, wherein, the step of obtaining the entropies of the convolutional layers according to the image data comprises: obtaining a probability distribution of outputs of the channels according to the total number of channels, the total value of the feature levels, and a normalization function formula, the normalization function formula being: wherein, is the number of channels of the convolution layer j is the jth channel in is the number of graphs, b is the bth graph in B, as is the probability distribution output by the convolution measurement ​ obtaining the entropy of the convolutional layers according to the probability distribution, the total number of channels, and an entropy formula, the entropy formula being: where, is the entropy of the convolutional layer, is the number of channels of the convolutional layer the output probability distribution, j is the jth channel in, is the number of channels of the convolutional layer the total number of channels of the convolutional layer.

4. The method of claim 1, wherein, the step of obtaining the Shapley values of the channels in the convolutional layers comprises: selecting any two or more channels from all the channels in the convolutional layers as a coalition to obtain a plurality of coalitions; obtaining a feature equation according to the coalition and determining whether the feature equation satisfies a preset condition, if the feature equation satisfies the preset condition, then: obtaining a marginal contribution value of the channels to all the coalitions comprising the channels according to the feature equation and a marginal contribution formula, obtaining the Shapley value of the channels according to the marginal contribution value, the coalition, the total number of channels, and an average contribution formula.

5. The method of claim 1, wherein, The channels comprise deleted channels and remaining channels; before the step of pruning the channels in the convolutional layers according to the Shapley values of the channels and the pruning ratios of the convolutional layers, the method further comprises: obtaining a number of convolutional layers, obtaining an indicator function of the channels according to the deleted channels and the remaining channels; the step of pruning the channels in the convolutional layers according to the Shapley values of the channels and the pruning ratios of the convolutional layers comprises: According to the Shapley value, the indicator function, the total number of channels, the number of convolution layers, and a pruning function, a pruning parameter is obtained, According to the pruning parameter and the pruning ratio, the channels are pruned.

6. A passageway trimming device characterized by, The application is applied to a neural network comprising convolution layers, the neural network is applied to image processing, the convolution layers comprise channels, and the device comprises: An acquisition unit is configured to acquire a rank of the convolution layers according to image data, and to acquire an entropy of the convolution layers according to the image data; An analysis unit is configured to acquire an overall index of the convolution layers according to the rank of the convolution layers and the entropy of the convolution layers, specifically comprising: obtaining a normalized range according to the rank of the convolution layers and the entropy of the convolution layers; obtaining a fusion function according to the normalized range; and obtaining the overall index according to the fusion function; The fusion function is: wherein, is the fusion value of the convolution layer , Y represents and , Z represents and , , L is the number of convolution layers; and is normalized to the range [a, b]; According to the overall index, a pruning ratio of each convolution layer is obtained, specifically comprising: obtaining a fusion value according to the overall index; obtaining a number of deleted channels according to the fusion value; and obtaining the pruning ratio of each convolution layer according to the number of deleted channels and a total number of channels; The acquisition unit is further configured to acquire a Shapley value of a channel in the convolution layers; A pruning unit is configured to prune the channels in the convolution layers according to the Shapley value of the channels and the pruning ratio of the convolution layers.

7. A computer device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, The processor executes the computer program to implement the steps of the method in any one of claims 1 to 5.

8. A computer-readable storage medium having stored thereon a computer program, characterized in that, The computer program is executed by the processor to implement the steps of the method in any one of claims 1 to 5.

Images