Pruning method of deep learning model, electronic equipment and storage medium

Abstract

The invention discloses a pruning method of a deep learning model, electronic equipment and a storage medium, and belongs to the field of computers. The method comprises the steps of obtaining a first channel importance factor of a target channel in a to-be-pruned deep learning model, the first channel importance factor being used for evaluating the importance of the target channel in the to-be-pruned deep learning model, the first channel importance factor is obtained by evaluating a feature map change condition of the to-be-pruned deep learning model before and after pruning; the size of the first channel importance factor is in positive correlation with a feature map change condition before and after pruning of the to-be-pruned deep learning model; and based on the first channel importance factor, pruning the to-be-pruned deep learning model to obtain a target deep learning model, the size of the first channel importance factor being negatively correlated with the pruning probability of the target channel. The method is used for model pruning and is used for solving the problem that the performance of a deep learning model is greatly influenced.

Description

Technical Field

[0001] The present application relates to the field of computers, and specifically to a pruning method, electronic device, and storage medium for a deep learning model. Background Art

[0002] Model pruning technology is an effective deep learning model compression technology that can significantly reduce the computing and storage requirements of the model. A reasonable pruning strategy can reduce memory consumption and greatly improve the operating efficiency of the model while maintaining model performance, so that the model can adapt to resource-constrained environments. It is a very important part of model pruning technology.

[0003] Related technologies usually evaluate the channel importance of deep learning models through the model parameters of the deep learning models, thereby removing unimportant channels in the deep learning models. However, this method easily leads to the loss of effective feature information. This model pruning method has the problem of a significant impact on the performance of the deep learning model. Summary of the invention

[0004] The embodiments of the present application provide a deep learning model pruning method, electronic device and storage medium, which can solve the problem of the related technology that the performance of the deep learning model is greatly affected.

[0005] In a first aspect, an embodiment of the present application provides a method for pruning a deep learning model, the method comprising: Obtaining a first channel importance factor of a target channel in the deep learning model to be pruned, where the first channel importance factor is used to evaluate the importance of the target channel in the deep learning model to be pruned, and the first channel importance factor is obtained by evaluating the change of the feature graph of the deep learning model to be pruned before and after pruning; the size of the first channel importance factor is positively correlated with the change of the feature graph of the deep learning model to be pruned before and after pruning; Based on the first channel importance factor, the deep learning model to be pruned is pruned to obtain a target deep learning model, and the size of the first channel importance factor is negatively correlated with the pruning probability of the target channel.

[0006] In a second aspect, an embodiment of the present application provides an electronic device, which includes a processor and a memory, wherein the memory stores programs or instructions that can be run on the processor, and when the program or instructions are executed by the processor, the steps of the method described in the first aspect are implemented.

[0007] In a third aspect, an embodiment of the present application provides a computer-readable storage medium, on which a program or instruction is stored, and when the program or instruction is executed, the steps of the method described in the first aspect are implemented.

[0008] In a fourth aspect, an embodiment of the present application provides a computer program product, which includes a computer program, and when the computer program is executed by a processor, it implements the steps of the method described in the first aspect.

[0009] At least one of the above technical solutions provided in the embodiments of the present application can achieve the following technical effects: In an embodiment of the present application, a first channel importance factor of a target channel in a deep learning model to be pruned is obtained, the first channel importance factor is used to evaluate the importance of the target channel in the deep learning model to be pruned, and the first channel importance factor is obtained by evaluating the change in feature graph of the deep learning model to be pruned before and after pruning; the size of the first channel importance factor is positively correlated with the change in feature graph of the deep learning model to be pruned before and after pruning; based on the first channel importance factor, the deep learning model to be pruned is pruned to obtain a target deep learning model, and the size of the first channel importance factor is negatively correlated with the pruning probability of the target channel. In this way, the first channel importance factor is obtained by evaluating the change of the feature graph before and after pruning the deep learning model to be pruned. Compared with the method of evaluating the channel importance of the deep learning model by the model parameters of the deep learning model to be pruned, the method of evaluating the importance of the target channel of the deep learning model to be pruned by the change of the feature graph before and after pruning the deep learning model to be pruned is more objective. In the process of pruning the target channel of the deep learning model to be pruned, the size of the first channel importance factor is negatively correlated with the pruning probability of the target channel, and the size of the first channel importance factor is positively correlated with the change of the feature graph before and after pruning the deep learning model to be pruned. That is to say, the larger the first channel importance factor of the target channel, the greater the change of the feature graph before and after pruning the deep learning model to be pruned, and the smaller the pruning probability of the target channel. In this way, the important features extracted by the model can be effectively retained, the risk of model performance degradation can be reduced, and the problem of a large impact on the performance of the deep learning model in the related technology is solved. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for use in the embodiments are briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present invention and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without creative work.

[0011] Figure 1 It is a flowchart of a deep learning model pruning method provided in an embodiment of the present application; Figure 2It is a flowchart of another deep learning model pruning method provided in an embodiment of the present application; Figure 3 It is an overall conceptual diagram of a deep learning model pruning method provided in an embodiment of the present application; Figure 4 It is a flowchart of another deep learning model pruning method provided in an embodiment of the present application; Figure 5 is a schematic diagram of a model pruning process provided in an embodiment of the present application; Figure 6 It is a specific flow chart of a method for pruning a deep learning model provided in an embodiment of the present application; Figure 7 It is a structural block diagram of a pruning device for a deep learning model provided in an embodiment of the present application; Figure 8 It is a structural block diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0012] The following will be combined with the drawings in the embodiments of the present application to clearly describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0013] The terms "first", "second", etc. in the specification and claims of the present application are used to distinguish similar objects, and are not used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable under appropriate circumstances, so that the embodiments of the present application can be implemented in an order other than those illustrated or described here, and the objects distinguished by "first", "second", etc. are generally of one type, and the number of objects is not limited. For example, the first object can be one or more. In addition, "and / or" in the specification and claims represents at least one of the connected objects, and the character " / " generally indicates that the objects associated with each other are in an "or" relationship.

[0014] Therefore, the following detailed description of the embodiments of the present invention provided in the accompanying drawings is not intended to limit the scope of the invention claimed for protection, but merely represents selected embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0015] The pruning method of the deep learning model provided in the embodiment of the present application is applied to the model pruning technology, and can be used to prune the deep learning model with a large number of parameters. Specifically, the first channel importance factor of the channel in the deep learning model to be pruned can be obtained first, and the first channel importance factor can be used to evaluate the change of the feature graph of the deep learning model to be pruned before and after pruning, and then the deep learning model to be pruned can be pruned by the first channel importance factor. The larger the first channel importance factor, the less likely the corresponding channel is to be deleted, so as to retain the important features extracted by the model, thereby ensuring the performance of the model.

[0016] The pruning method of the deep learning model provided in the embodiment of the present application can be executed by a target device, wherein the target device can be one electronic device or multiple electronic devices. That is to say, the pruning method of the deep learning model provided in the embodiment of the present application can be executed by an electronic device, wherein the electronic device can be a terminal device such as a desktop computer, a laptop computer, a mobile phone, a tablet, etc., or a server, such as an independent physical server, a server cluster composed of multiple servers, and a cloud server capable of cloud computing. In the case where the pruning method of the deep learning model provided in the embodiment of the present application is executed by multiple electronic devices, these multiple electronic devices can form a service cluster, which cooperate with each other to complete each step.

[0017] The following is a detailed description of the deep learning model pruning method provided in the embodiment of the present application through specific embodiments and their application scenarios in conjunction with the accompanying drawings.

[0018] See also Figure 1 , Figure 1 1 is a flow chart of a pruning method for a deep learning model provided in an embodiment of the present application. Figure 1 As shown, the method comprises the following steps: Step 110: Obtain a first channel importance factor of a target channel in the deep learning model to be pruned, wherein the first channel importance factor is used to evaluate the importance of the target channel in the deep learning model to be pruned, and the first channel importance factor is obtained by evaluating the change of the feature graph of the deep learning model to be pruned before and after pruning; the size of the first channel importance factor is positively correlated with the change of the feature graph of the deep learning model to be pruned before and after pruning.

[0019] In an embodiment of the present application, the target channel may include all channels (feature map channels) in the deep learning model to be pruned. The deep learning model to be pruned may include a trained deep learning model to be pruned, that is, after obtaining the network model to be pruned, the network model to be pruned may be trained by the training set data to obtain a trained deep learning model to be pruned, and the complete model weights in the deep learning model to be pruned may be saved. Among them, the deep learning model to be pruned may be a convolutional neural network model with a large number of parameters that meets business performance. In order to reduce the model parameter amount of the deep learning model to be pruned, the deep learning model to be pruned may be pruned.

[0020] In the process of pruning the deep learning model to be pruned, unimportant channels in the deep learning model to be pruned can be deleted. In this way, even if some channels in the deep learning model to be pruned are deleted, the performance of the deep learning model to be pruned will not be greatly affected. In order to evaluate the importance of the target channel in the deep learning model to be pruned, the evaluation of the importance of the target channel in the deep learning model to be pruned can be converted into evaluating the changes in the feature graph before and after pruning the deep learning model to be pruned. Since the feature graph can directly reflect the effect of the deep learning model to be pruned, the changes in the feature graph can more directly show the changes in the deep learning model to be pruned before and after pruning. This evaluation method is more objective and accurate.

[0021] Specifically, the magnitude of the first channel importance factor is positively correlated with the change in the feature graph before and after pruning the deep learning model to be pruned. In other words, the greater the change in the feature graph before and after pruning the deep learning model to be pruned, the greater the first channel importance factor corresponding to the channel.

[0022] Step 120: Based on the first channel importance factor, prune the deep learning model to be pruned to obtain a target deep learning model, and the size of the first channel importance factor is negatively correlated with the pruning probability of the target channel.

[0023] In an embodiment of the present application, each channel in the deep learning model to be pruned may correspond to a first channel importance factor. For a channel in the deep learning model to be pruned, the larger the first channel importance factor corresponding to the channel, the smaller the pruning probability of the channel, and the less likely the channel is to be deleted. In other words, the channel corresponding to the smaller first channel importance factor (for example, the first channel importance factor is less than a threshold) in the deep learning model to be pruned can be deleted from the deep learning model to be pruned. After pruning the deep learning model to be pruned, a pruned deep learning model is obtained.

[0024] After obtaining the pruned deep learning model, the training set data used when training the network model to be pruned can be used to obtain the complete weights obtained by training the network model to be pruned, and the pruned deep learning model can be initialized. Then, the initialized model can be retrained to obtain a trained target deep learning model. In this process, the pruned deep learning model is trained for a certain number of rounds (i.e., the parameters of the model are fine-tuned) to gradually adjust its parameters until the overall performance of the model stabilizes, indicating that the model is back in a relatively satisfactory convergence state. The purpose of retraining the initialized model here is to compensate for the performance degradation that may be caused by pruning, and to ensure that the model can still provide accurate and reliable prediction results.

[0025] In an embodiment of the present application, a first channel importance factor of a target channel in a deep learning model to be pruned is obtained, the first channel importance factor is used to evaluate the importance of the target channel in the deep learning model to be pruned, and the first channel importance factor is obtained by evaluating the change in feature graph of the deep learning model to be pruned before and after pruning; the size of the first channel importance factor is positively correlated with the change in feature graph of the deep learning model to be pruned before and after pruning; based on the first channel importance factor, the deep learning model to be pruned is pruned to obtain a target deep learning model, and the size of the first channel importance factor is negatively correlated with the pruning probability of the target channel. In this way, the first channel importance factor is obtained by evaluating the change of the feature graph before and after pruning the deep learning model to be pruned. Compared with the method of evaluating the channel importance of the deep learning model by the model parameters of the deep learning model to be pruned, the method of evaluating the importance of the target channel of the deep learning model to be pruned by the change of the feature graph before and after pruning the deep learning model to be pruned is more objective. In the process of pruning the target channel of the deep learning model to be pruned, the size of the first channel importance factor is negatively correlated with the pruning probability of the target channel, and the size of the first channel importance factor is positively correlated with the change of the feature graph before and after pruning the deep learning model to be pruned. That is to say, the larger the first channel importance factor of the target channel, the greater the change of the feature graph before and after pruning the deep learning model to be pruned, and the smaller the pruning probability of the target channel. In this way, the important features extracted by the model can be effectively retained, the risk of model performance degradation can be reduced, and the problem of a large impact on the performance of the deep learning model in the related technology is solved.

[0026] See also Figure 2 , Figure 2 is a flow chart of another deep learning model pruning method provided in an embodiment of the present application. Figure 2 As shown, the method comprises the following steps: Step 210: Input the validation set data into the deep learning model to be pruned to obtain a first feature map.

[0027] In the embodiments of this application, please refer to Figure 3 , Figure 3 This is a general concept diagram of a deep learning model pruning method provided in an embodiment of the present application. Figure 3 As shown, by inputting the validation set data into the deep learning model to be pruned, the feature map obtained by each layer of the deep learning model to be pruned, that is, the first feature map, can be obtained. Since the deep learning model to be pruned has been optimized based on the training set data, if the training set data is continued to be used to evaluate the importance of the channels of the deep learning model to be pruned, it may affect the credibility of the feature mutual information index. Therefore, in order to solve this problem, the embodiment of the present application uses the validation set data to measure the changes in the feature maps of the deep learning model to be pruned before and after pruning. Among them, the validation set data can be expressed as .

[0028] Step 220: Obtain a second channel importance factor, apply the second channel importance factor to the deep learning model to be pruned, and obtain a channel importance factor optimization model.

[0029] In an embodiment of the present application, in order to evaluate the channel importance of the deep learning model to be pruned, after obtaining the deep learning model to be pruned, the weight of the deep learning model to be pruned (the model parameters obtained during the training process) can be fixed, and for each layer of the network in the deep learning model to be pruned, the second channel importance factor can be introduced into the weight of each layer of the network. The second channel importance factor can be a preset importance factor, which can be set based on experience.

[0030] by Figure 3 For example, the first layer of the network includes 4 channels, the second layer of the network includes 4 channels..., the Nth layer of the network includes 4 channels, and the importance factor of the second channel can be applied to the first channel of the first layer of the network , the second channel importance factor can be applied to the second channel of the first layer network , and so on, the second channel importance factor can be applied to the fourth channel of the Nth layer network , and obtain the channel importance factor optimization model.

[0031] Step 230: Input the validation set data into the channel importance factor optimization model to obtain a second feature map.

[0032] In the embodiment of the present application, after obtaining the channel importance factor optimization model, the validation set data can be input into the channel importance factor optimization model to obtain the feature map of each layer network output of the channel importance factor optimization model, that is, the second feature map. Figure 3 As shown, the feature map obtained by the first layer network of the channel importance factor optimization model is , the feature map obtained by the second layer network is , the feature map obtained by the Nth layer network is .

[0033] In one embodiment of the present application, the deep learning model to be pruned and the channel importance factor optimization model both include an N-layer network, each channel in the N-layer network of the deep learning model to be pruned corresponds to a first channel importance factor, and each channel in the N-layer network of the channel importance factor optimization model corresponds to a second channel importance factor, where N is a positive integer; the second feature map is obtained by the following formula: ; in, is the second feature graph of the i-th layer network in the N-layer network in the channel importance factor optimization model, is the second channel importance factor of the jth channel of the i-th layer network in the N-layer network in the channel importance factor optimization model, is the first feature map of the jth channel of the i-th network layer in the deep learning model to be pruned, K is the number of channels of the i-th network layer in the N-layer network in the channel importance factor optimization model, i is a positive integer less than or equal to N, and j is a positive integer less than or equal to K.

[0034] In an embodiment of the present application, the second feature map of the deep learning model to be pruned can be predicted by introducing the second channel importance factor into the deep learning model to be pruned. The second feature map of the deep learning model to be pruned can be predicted by the above formula based on the first feature map and the second channel importance factor. Wherein, softmax is an activation function. Specifically, the specific expression of the activation function softmax is as follows: ; Among them, e is a natural constant, is the channel importance factor of the jth channel of the i-th layer network of the deep learning model to be pruned, K is the number of channels of the i-th layer network of the deep learning model to be pruned, and k is a positive integer less than or equal to K.

[0035] Step 240: Obtain a target loss value based on the first feature map, the second feature map and the target loss function.

[0036] In an embodiment of the present application, in order to obtain the first channel importance factor, a gradient descent optimization algorithm may be used to optimize the second channel importance factor. Specifically, the target loss value of the target loss function may be calculated by using a gradient descent optimization algorithm, and the second channel importance factor may be iteratively updated continuously until the target loss value of the target loss function is at a stable value, thereby obtaining the first channel importance factor. The target loss function may be used to evaluate the changes in the feature graphs of the deep learning model to be pruned before and after pruning.

[0037] In order to determine the changes in the feature graphs of the deep learning model to be pruned before and after pruning, the embodiment of the present application proposes a feature mutual information index, which is used to measure the degree of mutual dependence between two variables. The feature mutual information index is used to measure the changes in the feature graphs of the entire network. Specifically, assuming , ..., It is the possible feature map after pruning each layer in the N-layer network model. Represents the unpruned feature graph in the i-th layer of the network. The following formula can be used to measure the changes in the feature graph of the entire network model, and the changes in the feature graph of the entire network model are recorded as Diff. Among them, I (*) is mainly used to evaluate the changes in features in a single-layer network. Because in neural networks, high-dimensional feature representations require a large amount of calculation when calculating mutual information, in order to reduce the amount of calculation, the embodiment of the present application proposes to use the Hilbert-Schmidt Independence Criterion (HSIC) to simplify the operation, and the simplified indicator is named feature mutual information I(X,Y). The specific calculation formula for feature mutual information is shown below.

[0038] ; ; Among them, Diff is used to indicate the change of the feature graph of the deep learning model to be pruned. is the first feature graph of the i-th layer network in the deep learning model to be pruned, is the second feature graph of the i-th layer network in the deep learning model to be pruned, It is used to indicate the change of the feature graph of the i-th layer network in the deep learning model to be pruned. I(X,Y) is a measure of the degree of mutual dependence between variables X and Y. is the Frobenius norm. X is the validation set data The feature map of the network model before pruning, Y is the feature map of the network model after pruning.

[0039] In one embodiment of the present application, the target loss function includes a first loss function and a second loss function, the first loss function is used to evaluate the change between the first feature map and the second feature map, and the second loss function is the loss function used in the training process of the deep learning model to be pruned.

[0040] In the embodiment of the present application, the target loss function can be combined with the loss function of feature mutual information and the loss function of validation set data. Figure 3 As shown, the second feature map (the feature map obtained by the first layer network of the channel importance factor optimization model) can be , the feature map obtained by the second layer network , the feature map obtained by the Nth layer network ) to calculate feature mutual information. The target loss function may include a first loss function for evaluating the change between the first feature map and the second feature map, and a second loss function used in the training process of the deep learning model to be pruned. Figure 3 As shown, the loss of the validation set data can be determined by the output result of the channel importance factor and the true label. For example, if the deep learning model to be pruned is an image classification model, the second loss function of the deep learning model to be pruned can be a cross entropy loss function.

[0041] The objective loss function is as follows: ; ; in, is the objective loss function, is the first loss function, is the second loss function, is the preset weight coefficient, is the first feature graph of the i-th layer network of the deep learning model to be pruned, is the second feature map of the i-th layer network of the channel importance factor optimization model, N is the number of network layers of the deep learning model to be pruned, is the transpose of the first feature map of the i-th layer network of the deep learning model to be pruned, The transpose of the second feature map of the i-th layer network of the channel importance factor optimization model.

[0042] Through the above formula, the above formula can be combined to obtain the formula of the target loss function as follows: ; in, is the objective loss function, is the second loss function, is the preset weight coefficient, is the first feature graph of the i-th layer network of the deep learning model to be pruned, is the second feature map of the i-th layer network of the channel importance factor optimization model, N is the number of network layers of the deep learning model to be pruned, is the transpose of the first feature map of the i-th layer network of the deep learning model to be pruned, The transpose of the second feature map of the i-th layer network of the channel importance factor optimization model.

[0043] Step 250: Based on the target loss value, update the second channel importance factor to obtain the first channel importance factor.

[0044] In an embodiment of the present application, by using a gradient descent optimization algorithm for the target loss function, the second channel importance factor can be iteratively updated by an optimizer (such as an Adam optimizer), and the latest target loss value can be continuously calculated until the obtained target loss value is at a stable value. The iterative update of the second channel importance factor can be stopped, and the latest second channel importance factor can be determined as the first channel importance factor.

[0045] Step 260: Based on the first channel importance factor, prune the deep learning model to be pruned to obtain a target deep learning model, and the size of the first channel importance factor is negatively correlated with the pruning probability of the target channel.

[0046] In an embodiment of the present application, a feature mutual information method is introduced to quantify the changes in feature maps in the neural network model before and after pruning. This method can more accurately capture the impact of pruning operations on the internal feature maps of the model; the feature mutual information method can be used to predict the performance degradation of the model after pruning in advance, which helps to avoid performance loss caused by excessive pruning; during the model pruning process, the feature mutual information method can help identify which feature maps are more critical to the final output, thereby retaining the more important parts.

[0047] See also Figure 4 , Figure 4 is a flow chart of another deep learning model pruning method provided in an embodiment of the present application. Figure 4 As shown, the method comprises the following steps: Step 410: Obtain a first channel importance factor of a target channel in a deep learning model to be pruned, wherein the deep learning model to be pruned includes an N-layer network, each channel in the N-layer network corresponds to a first channel importance factor, and N is a positive integer.

[0048] Step 420: Obtain N target pruning numbers corresponding to the N-layer network, where each layer of the N-layer network corresponds to a target pruning number among the N target pruning numbers.

[0049] In the embodiment of the present application, a compression ratio can be set, and for the i-th layer network in the N-layer network, the product of the number of channels of the i-th layer network and the compression ratio can be determined as the target pruning number corresponding to the i-th layer network. However, according to the actual model structure, the correlation between layers in the structure of the deep learning model to be pruned is relatively large, and the influence of different layers of networks on the deep learning model to be pruned is quite different. Setting the same compression ratio for different layers of networks has a certain impact on the performance of the deep learning model to be pruned.

[0050] The following is an explanation of the strong correlation between layers in the deep learning model to be pruned. Figure 5 , Figure 5 Schematic diagram of a model pruning process provided in an embodiment of the present application. Figure 5 As shown in FIG. 1 , a picture with a size of 24×24×3 can be input to the deep learning model to be pruned. For the first layer network in the deep learning model to be pruned, the first layer network has channels, the target pruning number of the first layer network is , in the feature map output by the first layer network, it can be seen from the Select from channels channels, the channels are deleted from the first layer of the network. Since there is a corresponding association between layers in the convolutional neural network, that is, when deleting the first layer of the network After channels, the feature map output of the first layer will also be reduced channels. Figure 5 As shown, the feature map of the first layer network is composed of Transformed into .

[0051] At the same time, the convolution kernel size of the next layer of the first layer network (the second layer network) will also be affected by the output feature map of the first layer. The convolution kernel size of the second layer network is determined by Transformed into If the channel importance factor in the second layer network is the smallest Channels are deleted, and the convolution kernel size of the second layer network becomes .

[0052] Through the above channel pruning process, it can be seen that there is a certain correlation between the pruning of the i-th layer network and the pruning of the i+1-th layer network in the N-layer network in the deep learning model to be pruned. Therefore, in the process of determining the number of pruning of each layer of the network in the deep learning model to be pruned, how to ensure the performance of the pruned model while meeting the expected model size is also an urgent problem to be solved.

[0053] To solve this problem, the embodiment of the present application proposes a solution algorithm for pruning channels, so as to determine the number of pruning channels required in each layer of the deep learning model to be pruned, and to minimize the loss of model accuracy while satisfying the model compression size. Specifically, the embodiment of the present application establishes a non-convex optimization problem, so that the target loss function value of the pruned deep learning model on the validation set data is minimized while satisfying the target constraints. This also shows that the pruning strategy has the least impact on the entire deep learning model to be pruned while satisfying the expected model size. In order to solve this non-convex optimization problem, the solution algorithm for the number of pruning channels used in the embodiment of the present application is a heuristic search algorithm, an algorithm based on intuition or experience, which can give a feasible solution with good results at an acceptable cost (computing time and space).

[0054] In one embodiment of the present application, obtaining N target pruning numbers corresponding to the N-layer network includes: obtaining N first pruning numbers corresponding to the N-layer network; updating the N first pruning numbers through a target loss function and a target constraint to obtain N target pruning numbers, wherein the target constraint makes the total number of channels of the N-layer network after pruning the deep learning model to be pruned less than or equal to a preset target number of channels.

[0055] In an embodiment of the present application, N first pruning numbers can be first obtained as a set of feasible solutions, and the N first pruning numbers can be fine-tuned. The N first pruning numbers after fine-tuning can satisfy the target constraint condition, and the N first pruning numbers after fine-tuning obtained in this way can be used as a set of new solutions. The quantitative relationship between the target loss function value obtained based on the N first pruning numbers after fine-tuning and the target loss function value obtained based on the N first pruning numbers before fine-tuning can be determined through the target loss function. In the case where the target loss function value obtained based on the N first pruning numbers after fine-tuning is less than the target loss function value obtained based on the N first pruning numbers before fine-tuning, the N first pruning numbers can be updated and the N first pruning numbers can be updated to the N first pruning numbers after fine-tuning.

[0056] The N first pruning numbers can be continuously updated in the above manner, and in the process of updating, the target loss function value can be continuously reduced while satisfying the target constraint condition. The number of updates can be preset, and when the number of updates to the N first pruning numbers reaches the preset number of updates, the latest N first pruning numbers can be determined as the N target pruning numbers. The target constraint condition can be shown as follows: ; ; in, is the objective loss function, is the first loss function, is the second loss function, is the preset weight coefficient, is the number of remaining channels of the i-th layer network of the model after pruning, and Size is the total number of preset channels of the deep learning model to be pruned. For example, if the total number of channels of the deep learning model to be pruned is 100 and the pruning ratio is 70%, then the total number of preset channels can be determined to be 70. In other words, the total number of channels of the target deep learning model obtained after pruning is less than or equal to 70.

[0057] Exemplarily, in one embodiment of the present application, the updating of the N first pruning numbers through the target loss function and the target constraint condition to obtain the N target pruning numbers includes: determining a first loss value based on the N first pruning numbers and the target loss function, and the N first pruning numbers satisfy the target constraint condition; obtaining N second pruning numbers; when the total number of channels of the N-layer network in the first pruning model does not satisfy the target constraint condition, adjusting the N second pruning numbers in the following manner until the total number of channels of the N-layer network in the second pruning model satisfies the target constraint condition: for the i-th layer of the N second pruning numbers, the first loss value is determined, and the N first pruning numbers satisfy the target constraint condition; obtaining N second pruning numbers; when the total number of channels of the N-layer network in the first pruning model does not satisfy the target constraint condition, adjusting the N second pruning numbers in the following manner until the total number of channels of the N-layer network in the second pruning model satisfies the target constraint condition: The second pruning number corresponding to the i-th network is determined according to the N first pruning numbers, when the first hyperparameter of the deep learning model to be pruned meets the first target condition, the first pruning number corresponding to the i-th network among the N first pruning numbers is determined as the second pruning number corresponding to the i-th network; when the first hyperparameter meets the first target condition and the second hyperparameter of the deep learning model to be pruned meets the second target condition, the second pruning number corresponding to the i-th network is adjusted, and the adjusted second pruning number corresponding to the i-th network is within the target value range; when the first hyperparameter does not meet the first target condition, the second pruning number corresponding to the i-th network is re-obtained.

[0058] When the total number of channels of the N-layer network in the first pruning model satisfies the target constraint, a second loss value is determined based on the N second pruning numbers and the target loss function; when the second loss value is less than the first loss value, the N second pruning numbers are determined as the N target pruning numbers; when the second loss value is greater than or equal to the first loss value, the N first pruning numbers are determined as the N target pruning numbers.

[0059] Among them, the first pruning model is a model obtained after pruning the deep learning model to be pruned based on the N second pruning numbers, and the second pruning model is a model obtained after pruning the deep learning model to be pruned based on the adjusted N second pruning numbers.

[0060] In the embodiment of the present application, a set of feasible solutions can be initialized, namely, the N first pruning numbers. Alternatively, multiple sets of feasible solutions can be initialized, namely, multiple sets of N third pruning numbers that satisfy the target constraint conditions, and the target loss function value obtained based on any one of the multiple sets of feasible solutions is calculated, and the N first pruning numbers can be a set of feasible solutions in the multiple sets of feasible solutions. The multiple sets of feasible solutions can be denoted as HM, as shown below: ; Wherein, HMS is the number of feasible solutions, for example, HMS can be 5. A row in the above HM is used to represent a group of feasible solutions, and each group of feasible solutions includes N third pruning numbers, Used to indicate the third pruning number corresponding to the i-th layer network in the m-th group of feasible solutions solved for the k-th time.

[0061] N second pruning numbers can be obtained, and the N second pruning numbers can be randomly generated. The deep learning model to be pruned can be pruned based on the N second pruning numbers to obtain a first pruning model. When the total number of channels of the N-layer network in the first pruning model does not meet the target constraint condition, for the second pruning number corresponding to the i-th layer network in the N second pruning numbers, when the first hyperparameter meets the first target condition, the first pruning number corresponding to the i-th layer network in the N first pruning numbers is determined as the second pruning number corresponding to the i-th layer network. The first target condition may include that the first hyperparameter is greater than or equal to a first random number, the first hyperparameter is a hyperparameter of the deep learning model to be pruned, which can be recorded as HMCR, for example, HMCR=0.9, and the first random number is a randomly generated random number with a value range between 0 and 1, which can be recorded as .

[0062] When the first hyperparameter satisfies the first target condition, it can be further determined whether the second hyperparameter satisfies the second target condition. When the first hyperparameter satisfies the first target condition and the second hyperparameter satisfies the second target condition, the second pruning number corresponding to the i-th layer network is adjusted, and the adjusted second pruning number corresponding to the i-th layer network is within the target value range.

[0063] The second target condition may include that the second hyperparameter is greater than or equal to a second random number, the second hyperparameter is another hyperparameter of the deep learning model to be pruned, which can be recorded as PAR, for example, PAR=0.5, and the second random number is a randomly generated random number with a value range between 0 and 1, which can be recorded as In this case, the second pruning number corresponding to the i-th network layer is adjusted, that is, the first pruning number corresponding to the i-th network layer among the N first pruning numbers is adjusted.

[0064] In the process of adjusting the second pruning number corresponding to the i-th layer network, the second pruning number corresponding to the i-th layer network can be fine-tuned based on a preset adjustment step and a third random number. The specific process of fine-tuning can be referred to the following formula: ;in, is the second pruning number corresponding to the i-th layer network, BW is the preset adjustment step, and the preset adjustment step can be set to 20, is the third random number.

[0065] After adjusting the second pruning number corresponding to the i-th layer network, it can be determined whether the adjusted second pruning number corresponding to the i-th layer network is within a target numerical range, and the target numerical range is a numerical range between a preset upper limit value of the pruning number of the i-th layer network and a preset lower limit value of the pruning number of the i-th layer network. In the case that the adjusted second pruning number corresponding to the i-th layer network is greater than the preset upper limit value of the pruning number of the i-th layer network, the adjusted second pruning number corresponding to the i-th layer network can be re-determined as the preset upper limit value of the pruning number of the i-th layer network. In the case that the adjusted second pruning number corresponding to the i-th layer network is less than the preset lower limit value of the pruning number of the i-th layer network, the adjusted second pruning number corresponding to the i-th layer network can be re-determined as the preset lower limit value of the pruning number of the i-th layer network. In this case, it can be determined that the adjusted second pruning number corresponding to the i-th layer network is within the target numerical range.

[0066] In order to jump out of the local optimum, when the first hyperparameter does not meet the first target condition, the second pruning number corresponding to the i-th layer network can be re-acquired. In the process of re-acquiring the second pruning number corresponding to the i-th layer network, the N first pruning numbers can be jumped out, and the second pruning number corresponding to the i-th layer network can be randomly determined. However, it should be noted that the randomly determined second pruning number corresponding to the i-th layer network is within the target value range.

[0067] For example, in one embodiment of the present application, the second pruning number corresponding to the i-th layer network can be re-determined based on the preset upper limit of the pruning number of the i-th layer network, the preset lower limit of the pruning number of the i-th layer network and the fourth random number. The fourth random number is a randomly generated random number with a numerical range between 0 and 1. In the process of re-determining the second pruning number corresponding to the i-th layer network, the following formula can be referred to: ; in, is the second pruning number corresponding to the i-th layer network, is the fourth random number, is the preset lower limit of the number of pruning branches in the i-th layer network, is the preset upper limit of the number of pruning branches of the i-th layer network.

[0068] In an embodiment of the present application, when the total number of channels of the N-layer network in the first pruning model does not meet the target constraint condition, the above operation can be repeated to adjust the N second pruning numbers until the total number of channels of the N-layer network after pruning the deep learning model to be pruned based on the N second pruning numbers meets the target constraint condition. In other words, the deep learning model to be pruned can be pruned based on the adjusted N second pruning numbers to obtain a second pruning model. When the total number of channels of the N-layer network in the first pruning model meets the target constraint condition, the adjustment of the N second pruning numbers can be stopped.

[0069] When the total number of channels of the N-layer network in the first pruning model or the second pruning model satisfies the target constraint, the second loss value can be determined based on the N second pruning numbers and the target loss function; when the second loss value is less than the first loss value, the N second pruning numbers are determined as the N target pruning numbers; when the second loss value is greater than or equal to the first loss value, the N first pruning numbers are determined as the N target pruning numbers.

[0070] In an embodiment of the present application, when the total number of channels of the N-layer network after pruning the deep learning model to be pruned based on the N second pruning numbers satisfies the target constraint condition, the N second pruning numbers can be used as a set of feasible solutions, and the second loss values ​​of the N second pruning numbers can be determined. When the second loss value of the N second pruning numbers is less than the first loss value of the N first pruning numbers, it can be considered that the pruning scheme of the N second pruning numbers is better than the pruning scheme of the N first pruning numbers, and the N second pruning numbers can be determined as the N target pruning numbers. In addition, the optimization of the pruning scheme can also be continued in the above manner, the number of pruning scheme optimizations can be preset, and the number of pruning scheme optimizations can be recorded as K, for example, K can be 20. When the number of optimizations reaches the number of pruning scheme optimizations, the optimization of the pruning scheme is stopped, and the N target pruning numbers can be determined based on the current optimal pruning scheme.

[0071] Step 430: For the i-th network in the N-layer network, perform the following operations to obtain a target deep learning model: sort the K channels in the i-th network in ascending order according to the first channel importance factor, starting from the target channel corresponding to the smallest first channel importance factor among the K channels, select Q channels from the K channels in ascending order of the first channel importance factor, and delete the Q channels from the i-th network, where Q is equal to the target pruning number corresponding to the i-th network, K is a positive integer, and Q is a positive integer less than or equal to K.

[0072] In an embodiment of the present application, the number of channels to be pruned in each layer of the network in the deep learning model to be pruned can be obtained through the algorithm for solving the number of pruned channels. For the i-th layer of the network in the deep learning model to be pruned, the K channels in the i-th layer of the network can be sorted in order of channel importance factors from small to large, starting from the channel with the smallest channel importance factor among the K channels, and selecting Q channels from the K channels in order of channel importance factors from small to large, Q is equal to the number of channels to be pruned in the i-th layer of the network, and the Q channels are deleted from the i-th layer of the network. After pruning the N-layer network, the model obtained after pruning can be determined as the target deep learning model.

[0073] In an embodiment of the present application, the pruning number of each layer in the deep learning model to be pruned can be set individually. Compared with the method of using the same pruning ratio in the related art, the pruning scheme obtained by this method is more flexible and reduces the interference caused by manually setting the pruning ratio.

[0074] The following will further introduce the pruning method of the deep learning model in the embodiment of the present application through a specific example. The embodiment of the present application can be used to solve a tree falling task, and the goal is to determine whether there are fallen trees in the image. Tree falling refers to the phenomenon that the growth of upright trees becomes crooked, or even the whole plant falls to the ground. In order to solve this task, a classification network model can be trained based on the existing tree falling data. For example, the classification network model can adopt ResNet50, ResNet18, MobileNet and other network models. At this time, the trained classification network model can be used as the deep learning model to be pruned, and this deep learning model to be pruned is retained. The following takes the selection of ResNet18 model as the deep learning model to be pruned as an example for introduction.

[0075] First, a channel importance factor optimization model for optimizing channel importance factors can be constructed. Specifically, the channel importance factor is set on the feature map channel in the ResNet18 model. , construct as Figure 1 The channel importance factor optimization model shown in FIG. The data can be validation set data, and the weights can be the weights trained by the deep learning model to be pruned. In the network optimization process, only the channel importance factor can be optimized. The loss function of the channel importance factor optimization model is described in the above formula. In the process of iteratively updating the channel importance factor, the optimizer (Adam optimizer) can be used based on the validation set data to optimize the channel importance factor. Iterate and update until the channel importance factor optimization model is stable, and the channel importance factor at this time can be selected To represent the importance of the feature channels of the ResNet18 model.

[0076] Then, the number of channels that need to be pruned for each layer of the ResNet18 model can be calculated, and the number of feature map channels that need to be deleted for each layer of the ResNet18 model can be recorded as , according to the above algorithm 1 The specific process can be referred to above and will not be repeated here.

[0077] The number of feature map channels that need to be deleted in each layer of the ResNet18 model is obtained by the above method. And the channel importance factor of the feature map channel of each layer of the network in the ResNet18 model After that, for the i-th layer network in the ResNet18 model, the feature map channels of the i-th layer network can be sorted in descending order according to the importance factor, and the ones ranked later are sorted. For the pruned ResNet18 model, you can load the training set data and the weights obtained during the training of the ResNet18 model before pruning, and fine-tune the pruned ResNet18 model until the entire network model converges again, and verify the classification effect of the obtained network model.

[0078] See also Figure 6 , Figure 6 Detailed flowchart of a method for pruning a deep learning model provided in an embodiment of the present application. Figure 6 As shown, the method comprises the following steps: Step 610: Input the validation set data into the deep learning model to be pruned to obtain a first feature map.

[0079] Step 620: Obtain a second channel importance factor, apply the second channel importance factor to the deep learning model to be pruned, and obtain a channel importance factor optimization model.

[0080] Step 630: Input the validation set data into the channel importance factor optimization model to obtain a second feature map.

[0081] In an embodiment of the present application, the deep learning model to be pruned and the channel importance factor optimization model both include an N-layer network, each channel in the N-layer network of the deep learning model to be pruned corresponds to a first channel importance factor, and each channel in the N-layer network of the channel importance factor optimization model corresponds to a second channel importance factor, where N is a positive integer; the second feature map is obtained by the following formula: ; in, is the second feature graph of the i-th layer network in the N-layer network in the channel importance factor optimization model, is the second channel importance factor of the jth channel of the i-th layer network in the N-layer network in the channel importance factor optimization model, is the first feature map of the jth channel of the i-th network layer in the deep learning model to be pruned, K is the number of channels of the i-th network layer in the N-layer network in the channel importance factor optimization model, i is a positive integer less than or equal to N, and j is a positive integer less than or equal to K.

[0082] Step 640: Obtain a target loss value based on the first feature map, the second feature map and the target loss function.

[0083] In an embodiment of the present application, the target loss function includes a first loss function and a second loss function, the first loss function is used to evaluate the change between the first feature map and the second feature map, and the second loss function is the loss function used in the training process of the deep learning model to be pruned; The objective loss function is as follows: ;

[0084] in, is the objective loss function, is the first loss function, is the second loss function, is the preset weight coefficient, is the first feature graph of the i-th layer network of the deep learning model to be pruned, is the second feature map of the i-th layer network of the channel importance factor optimization model, N is the number of network layers of the deep learning model to be pruned, is the transpose of the first feature map of the i-th layer network of the deep learning model to be pruned, The transpose of the second feature map of the i-th layer network of the channel importance factor optimization model.

[0085] Step 650: Based on the target loss value, update the second channel importance factor to obtain the first channel importance factor.

[0086] Step 660: Obtain N target pruning numbers corresponding to the N-layer network, where each layer of the N-layer network corresponds to a target pruning number among the N target pruning numbers.

[0087] In an embodiment of the present application, N first pruning numbers corresponding to the N-layer network can be obtained; the N first pruning numbers are updated through the target loss function and the target constraint conditions to obtain N target pruning numbers, and the target constraint conditions make the total number of channels of the N-layer network after pruning the deep learning model to be pruned less than or equal to the preset target channel number.

[0088] In the process of updating the N first pruning numbers through the target loss function and the target constraint condition to obtain the N target pruning numbers, a first loss value can be determined based on the N first pruning numbers and the target loss function, and the N first pruning numbers satisfy the target constraint condition. Then, N second pruning numbers are obtained.

[0089] When the total number of channels of the N-layer network in the first pruning model does not meet the target constraint condition, the N second pruning numbers are adjusted in the following manner until the total number of channels of the N-layer network in the second pruning model meets the target constraint condition: for the second pruning number corresponding to the i-th layer network among the N second pruning numbers, when the first hyperparameter of the deep learning model to be pruned meets the first target condition, the first pruning number corresponding to the i-th layer network among the N first pruning numbers is determined as the second pruning number corresponding to the i-th layer network; when the first hyperparameter meets the first target condition and the second hyperparameter of the deep learning model to be pruned meets the second target condition, the second pruning number corresponding to the i-th layer network is adjusted, and the adjusted second pruning number corresponding to the i-th layer network is within the target value range; when the first hyperparameter does not meet the first target condition, the second pruning number corresponding to the i-th layer network is re-obtained.

[0090] When the total number of channels of the N-layer network in the first pruning model satisfies the target constraint condition, a second loss value is determined based on the N second pruning numbers and the target loss function.

[0091] When the second loss value is less than the first loss value, the N second pruning numbers are determined as the N target pruning numbers. When the second loss value is greater than or equal to the first loss value, the N first pruning numbers are determined as the N target pruning numbers.

[0092] Among them, the first pruning model is a model obtained after pruning the deep learning model to be pruned based on the N second pruning numbers, and the second pruning model is a model obtained after pruning the deep learning model to be pruned based on the adjusted N second pruning numbers.

[0093] Step 670: For the i-th layer network in the N-layer network, perform the following operations to obtain a target deep learning model: sort the K channels in the i-th layer network in ascending order of the first channel importance factor, starting from the target channel corresponding to the smallest first channel importance factor among the K channels, select Q channels from the K channels in ascending order of the first channel importance factor, and delete the Q channels from the i-th layer network, where Q is equal to the target pruning number corresponding to the i-th layer network.

[0094] In an embodiment of the present application, a first channel importance factor of a target channel in a deep learning model to be pruned is obtained, the first channel importance factor is used to evaluate the importance of the target channel in the deep learning model to be pruned, and the first channel importance factor is obtained by evaluating the change in feature graph of the deep learning model to be pruned before and after pruning; the size of the first channel importance factor is positively correlated with the change in feature graph of the deep learning model to be pruned before and after pruning; based on the first channel importance factor, the deep learning model to be pruned is pruned to obtain a target deep learning model, and the size of the first channel importance factor is negatively correlated with the pruning probability of the target channel. In this way, the first channel importance factor is obtained by evaluating the change of the feature graph before and after pruning the deep learning model to be pruned. Compared with the method of evaluating the channel importance of the deep learning model by the model parameters of the deep learning model to be pruned, the method of evaluating the importance of the target channel of the deep learning model to be pruned by the change of the feature graph before and after pruning the deep learning model to be pruned is more objective. In the process of pruning the target channel of the deep learning model to be pruned, the size of the first channel importance factor is negatively correlated with the pruning probability of the target channel, and the size of the first channel importance factor is positively correlated with the change of the feature graph before and after pruning the deep learning model to be pruned. That is to say, the larger the first channel importance factor of the target channel, the greater the change of the feature graph before and after pruning the deep learning model to be pruned, and the smaller the pruning probability of the target channel. In this way, the important features extracted by the model can be effectively retained, the risk of model performance degradation can be reduced, and the problem of a large impact on the performance of the deep learning model in the related technology is solved.

[0095] It is important to understand that Figures 1 to 6 The explanations of the same or corresponding steps in the above may refer to each other. For example, Figure 2 The explanation of step 210 and step 220 in Figure 6 Step 610 and step 620 in .

[0096] At the same time, it should be understood that a pruning method for a deep learning model provided in an embodiment of the present application may have the following beneficial effects: by comprehensively considering the changes in the feature graph before and after single-layer pruning and its disturbance effect on the entire network model, the network model is pruned, which effectively solves the problem of serious performance loss in the model pruning method used in the related art during the reasoning acceleration process. First, the related art often only focuses on the changes in model parameters and ignores the importance of feature graphs. By introducing the mutual information method based on feature graphs, the embodiment of the present application can accurately measure the changes in feature graphs before and after pruning, which not only reduces redundant parameters during the model pruning process, but also effectively retains the important features extracted by the model, thereby reducing the risk of model performance degradation. Second, in the embodiment of the present application, a channel importance factor is introduced into the network model, and the importance of each channel can be solved by combining the mutual information loss of the feature graph and the verification set loss, using gradient descent. This method can dynamically adjust and optimize the pruning strategy of each channel, making pruning more accurate, thereby ensuring that important features are retained and further improving the performance of the pruned network model. Third, the model pruning method used in related technologies usually gives the same treatment to each layer of the deep learning model to be pruned, while the present application takes into account the difference in importance of each layer of the network weight to the entire network. By establishing an optimization model to solve the pruning ratio scheme for each layer of the network, the pruning effect of each layer of the network can be optimized in a targeted manner to avoid excessive pruning of the key layers of the model, thereby maintaining the overall performance of the network. In other words, the embodiment of the present application can effectively reduce the performance loss of the model by comprehensively considering the changes in feature maps and the importance of channels, and can perform accurate channel importance evaluation and inter-layer differentiation processing, making the pruning strategy more reasonable and efficient. At the same time, it can successfully accelerate model reasoning while ensuring performance to meet actual application needs.

[0097] In addition, this application measures the change of feature graphs based on the feature mutual information method, and introduces the feature mutual information method to quantify the change of feature graphs in the neural network model before and after pruning. This method can more finely capture the impact of pruning operations on the internal feature graphs of the model; the feature mutual information method can be used to predict the performance degradation of the model after pruning, which helps to avoid performance loss caused by excessive pruning; in the process of model pruning, the feature mutual information method can help identify which feature graphs are more critical to the final output, thereby retaining the more important parts; this method has good versatility and can be applied to different neural network architectures and task scenarios, increasing the practicality and portability of the technical solution. This application introduces a feature channel importance factor to characterize the importance of each channel in the model; when evaluating the importance of the channel, the feature mutual information loss and the performance loss on the validation set are comprehensively considered, providing a multi-dimensional evaluation standard; the contribution of each channel to the overall model performance can be accurately evaluated, providing a basis for subsequent model pruning; by introducing the channel importance factor, the transparency of the internal mechanism of the model can also be improved, making the model pruning process more interpretable. This application solves the optimal solution for the number of pruning for each layer of the network through modeling, and establishes a mathematical model to determine the number of pruning for each layer of the network based on the channel importance factor, which can ensure that the pruning operation is more accurate; by solving the optimization model, the optimal pruning ratio for each layer is obtained, so that the performance of the model after pruning reaches the optimal state; through the optimization method, the performance of the model after pruning is maximized, and a high prediction accuracy is maintained; this modeling method can automatically generate pruning strategies, reducing the interference caused by manually setting the pruning ratio; this modeling method is applicable to a variety of neural network models, and enhances its adaptability in different application scenarios. This application comprehensively considers the disturbance effect of single-layer pruning on the overall model. It not only analyzes the impact of single-layer pruning on local performance, but also evaluates the disturbance effect of pruning on the entire model to ensure model optimization from a global perspective. By comprehensively considering the interaction between multiple layers, a multi-layer collaborative pruning strategy is proposed to avoid the negative impact of a single pruning operation on the overall performance. The pruning strategy is dynamically adjusted according to the evaluation results of the disturbance effect to ensure the stability of the model performance. The robustness of the model is enhanced by considering the global disturbance effect. By comprehensively considering the disturbance effect of single-layer pruning on the overall model, the impact of pruning on the overall network performance is minimized. This application uses a gradient descent optimization algorithm to solve the channel importance factor, which provides a solution for efficiently solving the channel importance factor. In the solution process, both the feature mutual information loss and the performance loss on the validation set are considered to ensure a comprehensive evaluation of the channel importance. The gradient descent optimization algorithm can realize automatic optimization of the channel importance factor without manual adjustment of parameters, which improves the automation level of model optimization.

[0098] See also Figure 7 , Figure 7is a structural block diagram of a pruning device for a deep learning model provided in an embodiment of the present application. Figure 7 As shown, an embodiment of the present application provides a deep learning model pruning device 700, and the deep learning model pruning device 700 includes: an acquisition module 710 and a pruning module 720.

[0099] The acquisition module 710 is used to obtain a first channel importance factor of a target channel in the deep learning model to be pruned, where the first channel importance factor is used to evaluate the importance of the target channel in the deep learning model to be pruned, and the first channel importance factor is obtained by evaluating the change of the feature graph of the deep learning model to be pruned before and after pruning; the size of the first channel importance factor is positively correlated with the change of the feature graph of the deep learning model to be pruned before and after pruning; The pruning module 720 is used to prune the deep learning model to be pruned based on the first channel importance factor to obtain a target deep learning model, and the size of the first channel importance factor is negatively correlated with the pruning probability of the target channel.

[0100] In an embodiment of the present application, a first channel importance factor of a target channel in a deep learning model to be pruned is obtained, the first channel importance factor is used to evaluate the importance of the target channel in the deep learning model to be pruned, and the first channel importance factor is obtained by evaluating the change in feature graph of the deep learning model to be pruned before and after pruning; the size of the first channel importance factor is positively correlated with the change in feature graph of the deep learning model to be pruned before and after pruning; based on the first channel importance factor, the deep learning model to be pruned is pruned to obtain a target deep learning model, and the size of the first channel importance factor is negatively correlated with the pruning probability of the target channel. In this way, the first channel importance factor is obtained by evaluating the change of the feature graph before and after pruning the deep learning model to be pruned. Compared with the method of evaluating the channel importance of the deep learning model by the model parameters of the deep learning model to be pruned, the method of evaluating the importance of the target channel of the deep learning model to be pruned by the change of the feature graph before and after pruning the deep learning model to be pruned is more objective. In the process of pruning the target channel of the deep learning model to be pruned, the size of the first channel importance factor is negatively correlated with the pruning probability of the target channel, and the size of the first channel importance factor is positively correlated with the change of the feature graph before and after pruning the deep learning model to be pruned. That is to say, the larger the first channel importance factor of the target channel, the greater the change of the feature graph before and after pruning the deep learning model to be pruned, and the smaller the pruning probability of the target channel. In this way, the important features extracted by the model can be effectively retained, the risk of model performance degradation can be reduced, and the problem of a large impact on the performance of the deep learning model in the related technology is solved.

[0101] The deep learning model pruning device provided in the embodiment of the present application can implement each process implemented in the above method embodiment. To avoid repetition, it will not be repeated here.

[0102] like Figure 8 As shown, an embodiment of the present application also provides an electronic device 800. The electronic device 800 includes: a processor 810 and a memory 820, and the memory 820 stores a program or instruction, and the program or instruction is executed by the processor 810 to implement the steps of any of the methods described above. For example, when the program is executed by the processor 810, the following process is implemented: obtaining a first channel importance factor of a target channel in the deep learning model to be pruned, the first channel importance factor is used to evaluate the importance of the target channel in the deep learning model to be pruned, and the first channel importance factor is obtained by evaluating the change of the feature graph of the deep learning model to be pruned before and after pruning; the size of the first channel importance factor is positively correlated with the change of the feature graph of the deep learning model to be pruned before and after pruning; based on the first channel importance factor, the deep learning model to be pruned is pruned to obtain a target deep learning model, and the size of the first channel importance factor is negatively correlated with the pruning probability of the target channel. In this way, the first channel importance factor is obtained by evaluating the change of the feature graph before and after pruning the deep learning model to be pruned. Compared with the method of evaluating the channel importance of the deep learning model by the model parameters of the deep learning model to be pruned, the method of evaluating the importance of the target channel of the deep learning model to be pruned by the change of the feature graph before and after pruning the deep learning model to be pruned is more objective. In the process of pruning the target channel of the deep learning model to be pruned, the size of the first channel importance factor is negatively correlated with the pruning probability of the target channel, and the size of the first channel importance factor is positively correlated with the change of the feature graph before and after pruning the deep learning model to be pruned. That is to say, the larger the first channel importance factor of the target channel, the greater the change of the feature graph before and after pruning the deep learning model to be pruned, and the smaller the pruning probability of the target channel. In this way, the important features extracted by the model can be effectively retained, the risk of model performance degradation can be reduced, and the problem of a large impact on the performance of the deep learning model in the related technology is solved.

[0103] An embodiment of the present application also provides a readable storage medium, on which a program or instruction is stored. When the program or instruction is executed by a processor, the steps of each embodiment of the pruning method of a deep learning model are implemented, and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.

[0104] The processor is the processor in the electronic device described in the above embodiment. The readable storage medium includes a computer readable storage medium, such as a computer read-only memory ROM, a random access memory RAM, a magnetic disk or an optical disk.

[0105] An embodiment of the present application further provides a chip, which includes a processor and a communication interface, wherein the communication interface is coupled to the processor, and the processor is used to run programs or instructions to implement the various processes of the above-mentioned method embodiment, and can achieve the same technical effect. To avoid repetition, it will not be repeated here.

[0106] An embodiment of the present application provides a computer program product, which is stored in a storage medium. The program product is executed by at least one processor to implement the various processes of the above-mentioned method embodiment and can achieve the same technical effect. To avoid repetition, it will not be repeated here.

[0107] It should be noted that, in this article, the terms "comprise", "include" or any other variant thereof are intended to cover non-exclusive inclusion, so that the process, method, article or device including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise one..." do not exclude the presence of other identical elements in the process, method, article or device including the element. In addition, it should be pointed out that the scope of the methods and devices in the embodiments of the present application is not limited to performing functions in the order shown or discussed, and may also include performing functions in a substantially simultaneous manner or in reverse order according to the functions involved, for example, the described method may be performed in an order different from that described, and various steps may also be added, omitted, or combined. In addition, the features described with reference to certain examples may be combined in other examples.

[0108] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a computer software product, which is stored in a storage medium (such as ROM / RAM, a magnetic disk, or an optical disk), and includes a number of instructions for enabling a terminal (which can be a mobile phone, a computer, a server, or a network device, etc.) to execute the methods described in each embodiment of the present application.

[0109] The embodiments of the present application are described above in conjunction with the accompanying drawings, but the present application is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of the present application, ordinary technicians in this field can also make many forms without departing from the purpose of the present application and the scope of protection of the claims, all of which are within the protection of the present application.

Claims

1. A pruning method for a deep learning model, characterized in that: include: Obtaining a first channel importance factor of a target channel in the deep learning model to be pruned, where the first channel importance factor is used to evaluate the importance of the target channel in the deep learning model to be pruned, and the first channel importance factor is obtained by evaluating changes in feature graphs of the deep learning model to be pruned before and after pruning; The magnitude of the first channel importance factor is positively correlated with the change of the feature graph of the deep learning model to be pruned before and after pruning; Based on the first channel importance factor, the deep learning model to be pruned is pruned to obtain a target deep learning model, and the size of the first channel importance factor is negatively correlated with the pruning probability of the target channel.

2. The method according to claim 1, characterized in that The obtaining of the first channel importance factor of the target channel in the deep learning model to be pruned includes: Input the validation set data into the deep learning model to be pruned to obtain the first feature map; Obtaining a second channel importance factor, applying the second channel importance factor to the deep learning model to be pruned, and obtaining a channel importance factor optimization model; Inputting the validation set data into the channel importance factor optimization model to obtain a second feature map; Obtaining a target loss value based on the first feature map, the second feature map, and a target loss function; Based on the target loss value, the second channel importance factor is updated to obtain the first channel importance factor.

3. The method according to claim 2, characterized in that The deep learning model to be pruned and the channel importance factor optimization model both include an N-layer network, each channel in the N-layer network of the deep learning model to be pruned corresponds to a first channel importance factor, and each channel in the N-layer network of the channel importance factor optimization model corresponds to a second channel importance factor, where N is a positive integer; the second feature map is obtained by the following formula: ； in, is the second feature graph of the i-th layer network in the N-layer network in the channel importance factor optimization model, is the second channel importance factor of the jth channel of the i-th layer network in the N-layer network in the channel importance factor optimization model, is the first feature map of the jth channel of the i-th network layer in the deep learning model to be pruned, K is the number of channels of the i-th network layer in the N-layer network in the channel importance factor optimization model, i is a positive integer less than or equal to N, and j is a positive integer less than or equal to K.

4. The method according to claim 2, characterized in that: The target loss function includes a first loss function and a second loss function, the first loss function is used to evaluate the change between the first feature map and the second feature map, and the second loss function is a loss function used in the training process of the deep learning model to be pruned; The objective loss function is as follows: ； ； in, is the objective loss function, is the first loss function, is the second loss function, is the preset weight coefficient, is the first feature graph of the i-th layer network of the deep learning model to be pruned, is the second feature map of the i-th layer network of the channel importance factor optimization model, N is the number of network layers of the deep learning model to be pruned, is the transpose of the first feature map of the i-th layer network of the deep learning model to be pruned, The transpose of the second feature map of the i-th layer network of the channel importance factor optimization model.

5. The method according to claim 1, characterized in that The deep learning model to be pruned includes an N-layer network, each channel in the N-layer network corresponds to a first channel importance factor, and N is a positive integer; The step of pruning the deep learning model to be pruned based on the first channel importance factor includes: Obtaining N target pruning numbers corresponding to the N-layer network, where each layer of the N-layer network corresponds to one target pruning number among the N target pruning numbers; For the i-th network in the N-layer network, the following operations are performed: the K channels in the i-th network are sorted in ascending order according to the first channel importance factor, starting from the target channel corresponding to the smallest first channel importance factor among the K channels, Q channels are selected from the K channels in ascending order according to the first channel importance factor, and the Q channels are deleted from the i-th network, where Q is equal to the target pruning number corresponding to the i-th network, K is a positive integer, and Q is a positive integer less than or equal to K.

6. The method according to claim 5, characterized in that The obtaining of N target pruning numbers corresponding to the N-layer network includes: Obtaining N first pruning numbers corresponding to the N-layer network; The N first pruning numbers are updated through the target loss function and the target constraint conditions to obtain N target pruning numbers, and the target constraint conditions make the total number of channels of the N-layer network after pruning the deep learning model to be pruned less than or equal to the preset target channel number.

7. The method according to claim 6, characterized in that The updating of the N first pruning numbers through the target loss function and the target constraint condition to obtain the N target pruning numbers includes: Determine a first loss value based on the N first pruning numbers and the target loss function, wherein the N first pruning numbers satisfy the target constraint condition; Get N second pruning numbers; When the total number of channels of the N-layer network in the first pruning model does not meet the target constraint condition, the N second pruning numbers are adjusted in the following manner until the total number of channels of the N-layer network in the second pruning model meets the target constraint condition: for the second pruning number corresponding to the i-th layer network in the N second pruning numbers, when the first hyperparameter of the deep learning model to be pruned meets the first target condition, the first pruning number corresponding to the i-th layer network in the N first pruning numbers is determined as the second pruning number corresponding to the i-th layer network; when the first hyperparameter meets the first target condition and the second hyperparameter of the deep learning model to be pruned meets the second target condition, the second pruning number corresponding to the i-th layer network is adjusted, and the adjusted second pruning number corresponding to the i-th layer network is within the target value range; when the first hyperparameter does not meet the first target condition, the second pruning number corresponding to the i-th layer network is re-obtained; When the total number of channels of the N-layer network in the first pruning model satisfies the target constraint condition, determining a second loss value based on the N second pruning numbers and the target loss function; When the second loss value is less than the first loss value, determining the N second pruning numbers as N target pruning numbers; When the second loss value is greater than or equal to the first loss value, determining the N first pruning numbers as N target pruning numbers; Among them, the first pruning model is a model obtained after pruning the deep learning model to be pruned based on the N second pruning numbers, and the second pruning model is a model obtained after pruning the deep learning model to be pruned based on the adjusted N second pruning numbers.

8. An electronic device, characterized in that: The method comprises a processor and a memory, wherein the memory stores a program or instruction running on the processor, and when the program or instruction is executed by the processor, the steps of the method according to any one of claims 1 to 7 are implemented.

9. A computer-readable storage medium, characterized in that: The medium stores a program or an instruction, and when the program or the instruction is executed, the steps of the method according to any one of claims 1 to 7 are implemented.

10. A computer program product, characterized in that The invention comprises a computer program, which implements the method according to any one of claims 1 to 7 when being executed by a processor.

Images