Pruning methods for deep learning models, electronic devices and storage media

Abstract

This application discloses a pruning method, electronic device, and storage medium for deep learning models, belonging to the field of computer science. The method includes: obtaining a first channel importance factor for a target channel in a deep learning model to be pruned; the first channel importance factor is used to evaluate the importance of the target channel in the deep learning model to be pruned; the first channel importance factor is obtained by evaluating the changes in the feature map of the deep learning model before and after pruning; the magnitude of the first channel importance factor is positively correlated with the changes in the feature map of the deep learning model before and after pruning; and based on the first channel importance factor, pruning the deep learning model to be pruned to obtain a target deep learning model; the magnitude of the first channel importance factor is negatively correlated with the pruning probability of the target channel. This method is used for model pruning to address problems that significantly impact the performance of deep learning models.

Description

TECHNICAL FIELD

[0001] The present application belongs to the field of computer, and particularly relates to a deep learning model pruning method, an electronic device and a storage medium. BACKGROUND

[0002] The model pruning technology is an effective deep learning model compression technology, which can significantly reduce the calculation requirement and storage requirement of the model. A reasonable pruning strategy can reduce the memory consumption and greatly improve the running efficiency of the model while maintaining the performance of the model, so that the model can adapt to the resource-limited environment, and is a very important part of the model pruning technology.

[0003] The related technology usually evaluates the channel importance of the deep learning model through the model parameters of the deep learning model, so as to remove the unimportant channels in the deep learning model. However, this method is prone to loss of effective feature information, and the model pruning method has a large influence on the performance of the deep learning model. SUMMARY

[0004] The embodiments of the present application provide a deep learning model pruning method, an electronic device and a storage medium, which can solve the problem of large influence on the performance of the deep learning model in the related technology.

[0005] In a first aspect, the embodiments of the present application provide a deep learning model pruning method, which comprises the following steps:

[0006] obtaining a first channel importance factor of a target channel in a deep learning model to be pruned, the first channel importance factor being used to evaluate the importance of the target channel in the deep learning model to be pruned, and the first channel importance factor being obtained by evaluating the feature map change before and after pruning of the deep learning model to be pruned; the size of the first channel importance factor is positively correlated with the feature map change before and after pruning of the deep learning model to be pruned;

[0007] pruning the deep learning model to be pruned based on the first channel importance factor to obtain a target deep learning model, and the size of the first channel importance factor being negatively correlated with the pruning probability of the target channel.

[0008] In a second aspect, the embodiments of the present application provide an electronic device, which comprises a processor and a memory, the memory stores programs or instructions executable on the processor, and the programs or instructions are executed by the processor to implement the steps of the method according to the first aspect.

[0009] In a third aspect, an embodiment of the present application provides a computer readable storage medium, which stores a program or instructions, and the program or instructions, when executed, implement the steps of the method according to the first aspect.

[0010] In a fourth aspect, an embodiment of the present application provides a computer program product, which comprises a computer program, and the computer program, when executed by a processor, implements the steps of the method according to the first aspect.

[0011] The above at least one technical solution provided by the embodiments of the present application can achieve the following technical effects:

[0012] In the embodiments of the present application, a first channel importance factor of a target channel in a to-be-pruned deep learning model is obtained, the first channel importance factor is used to evaluate the importance of the target channel in the to-be-pruned deep learning model, and the first channel importance factor is obtained by evaluating the feature map change before and after pruning of the to-be-pruned deep learning model; the size of the first channel importance factor is positively correlated with the feature map change before and after pruning of the to-be-pruned deep learning model; and based on the first channel importance factor, the to-be-pruned deep learning model is pruned to obtain a target deep learning model, and the size of the first channel importance factor is negatively correlated with the pruning probability of the target channel. In this way, the first channel importance factor is obtained by evaluating the feature map change before and after pruning of the to-be-pruned deep learning model, and compared with the way of evaluating the importance of the channel of the deep learning model by using the model parameters of the to-be-pruned deep learning model, the way of evaluating the importance of the target channel of the to-be-pruned deep learning model by using the feature map change before and after pruning of the to-be-pruned deep learning model is more objective. In the process of pruning the target channel of the to-be-pruned deep learning model, the size of the first channel importance factor is negatively correlated with the pruning probability of the target channel, the size of the first channel importance factor is positively correlated with the feature map change before and after pruning of the to-be-pruned deep learning model, that is, the larger the first channel importance factor of the target channel, the larger the feature map change before and after pruning of the to-be-pruned deep learning model, and the smaller the pruning probability of the target channel. This can effectively preserve important features extracted by the model and reduce the risk of performance decline of the model, and solves the problem of the related art that the performance of the deep learning model is greatly affected. BRIEF DESCRIPTION OF DRAWINGS

[0013] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following will briefly introduce the drawings needed to be used in the embodiments. It should be understood that the following drawings only show some embodiments of the present application, and therefore should not be regarded as a limitation on the scope, and for those skilled in the art, other related drawings can also be obtained without creative labor.

[0014] Figure 1 This is a flowchart of a pruning method for a deep learning model provided in an embodiment of this application;

[0015] Figure 2 This is a flowchart of another pruning method for a deep learning model provided in an embodiment of this application;

[0016] Figure 3 This is a general conceptual diagram of a pruning method for a deep learning model provided in an embodiment of this application;

[0017] Figure 4 This is a flowchart of another pruning method for a deep learning model provided in an embodiment of this application;

[0018] Figure 5 This is a schematic diagram of a model pruning process provided in an embodiment of this application;

[0019] Figure 6 This is a flowchart illustrating a method for pruning a deep learning model according to an embodiment of this application.

[0020] Figure 7 This is a structural block diagram of a pruning device for a deep learning model provided in an embodiment of this application;

[0021] Figure 8 This is a structural block diagram of an electronic device provided in an embodiment of this application. Detailed Implementation

[0022] The technical solutions of the embodiments of this application will be clearly described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this application. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0023] The terms "first," "second," etc., used in the specification and claims of this application are used to distinguish similar objects and not to describe a specific order or sequence. It should be understood that such use of data can be interchanged where appropriate so that embodiments of this application can be implemented in orders other than those illustrated or described herein, and the objects distinguished by "first," "second," etc., are generally of the same class and the number of objects is not limited; for example, a first object can be one or more. Furthermore, in the specification and claims, "and / or" indicates at least one of the connected objects, and the character " / " generally indicates that the preceding and following objects are in an "or" relationship.

[0024] Therefore, the following detailed description of the embodiments of the invention provided in the accompanying drawings is not intended to limit the scope of the claimed invention, but merely to illustrate selected embodiments of the invention. All other embodiments obtained by those skilled in the art based on the embodiments of the invention without inventive effort are within the scope of protection of the invention.

[0025] The deep learning model pruning method provided in this application is applied in model pruning techniques and can be used to prune deep learning models with a large number of parameters. Specifically, the first channel importance factor of each channel in the deep learning model to be pruned can be obtained first. The first channel importance factor is used to evaluate the changes in the feature map of the deep learning model before and after pruning. Then, the deep learning model to be pruned is pruned based on the first channel importance factor. The larger the first channel importance factor, the less likely the channel is to be deleted, so as to retain the important features extracted by the model and thus ensure the performance of the model.

[0026] The deep learning model pruning method provided in this application embodiment can be executed by a target device, which can be a single electronic device or multiple electronic devices. That is, the deep learning model pruning method provided in this application embodiment can be executed by a single electronic device, which can be a terminal device such as a desktop computer, laptop, mobile phone, or tablet, or a server, such as a standalone physical server, a server cluster composed of multiple servers, or a cloud server capable of cloud computing. When the deep learning model pruning method provided in this application embodiment is executed by multiple electronic devices, these multiple electronic devices can form a service cluster, and they cooperate to complete each step.

[0027] The pruning method for deep learning models provided in this application will be described in detail below with reference to the accompanying drawings, through specific embodiments and application scenarios.

[0028] Please see Figure 1 , Figure 1 This is a flowchart illustrating a pruning method for a deep learning model provided in an embodiment of this application. For example... Figure 1 As shown, the method includes the following steps:

[0029] Step 110: Obtain the first channel importance factor of the target channel in the deep learning model to be pruned. The first channel importance factor is used to evaluate the importance of the target channel in the deep learning model to be pruned. The first channel importance factor is obtained by evaluating the changes in the feature map of the deep learning model to be pruned before and after pruning. The magnitude of the first channel importance factor is positively correlated with the changes in the feature map of the deep learning model to be pruned before and after pruning.

[0030] In this embodiment, the target channel may include all channels (feature map channels) in the deep learning model to be pruned. The deep learning model to be pruned may include a pre-trained deep learning model; that is, after obtaining the network model to be pruned, it can be trained using training set data to obtain a pre-trained deep learning model, and the complete model weights in the pre-pruned deep learning model can be preserved. The deep learning model to be pruned may be a convolutional neural network model with a large number of parameters that meets business performance requirements. To reduce the number of model parameters in the deep learning model to be pruned, pruning can be performed.

[0031] During the pruning process of the deep learning model to be pruned, unimportant channels can be deleted. This way, even if some channels are deleted, the performance impact on the model is minimal. To evaluate the importance of target channels in the model, this can be transformed into evaluating the changes in the feature maps before and after pruning. Since feature maps directly reflect the performance of the model, changes in feature maps more directly demonstrate the changes before and after pruning, making this evaluation method more objective and accurate.

[0032] Specifically, the magnitude of the first channel importance factor is positively correlated with the changes in the feature map of the deep learning model before and after pruning. In other words, the greater the change in the feature map of the deep learning model before and after pruning, the greater the importance factor of the first channel.

[0033] Step 120: Based on the first channel importance factor, prune the deep learning model to be pruned to obtain the target deep learning model. The magnitude of the first channel importance factor is negatively correlated with the pruning probability of the target channel.

[0034] In this embodiment, each channel in the deep learning model to be pruned can correspond to a first channel importance factor. For a channel in the deep learning model to be pruned, the larger the first channel importance factor, the lower the probability of pruning that channel, and the less likely that channel is to be deleted. That is, channels with smaller first channel importance factors (e.g., first channel importance factors less than a threshold) can be deleted from the deep learning model to be pruned. After pruning the deep learning model to be pruned, a pruned deep learning model is obtained.

[0035] After obtaining the pruned deep learning model, the training set data used to train the network model to be pruned can be used to obtain the complete weights obtained from training the network model to be pruned, and the pruned deep learning model can be initialized. Then, the initialized model can be retrained to obtain the trained target deep learning model. In this process, by performing a certain number of training rounds on the pruned deep learning model (i.e., fine-tuning the model's parameters), its parameters are gradually adjusted until the overall performance of the model stabilizes, indicating that the model has returned to a relatively satisfactory convergence state. The purpose of retraining the initialized model here is to compensate for the performance degradation that may be caused by pruning, ensuring that the model can still provide accurate and reliable prediction results.

[0036] In this embodiment, a first channel importance factor of the target channel in the deep learning model to be pruned is obtained. The first channel importance factor is used to evaluate the importance of the target channel in the deep learning model to be pruned. The first channel importance factor is obtained by evaluating the changes in the feature map of the deep learning model to be pruned before and after pruning. The magnitude of the first channel importance factor is positively correlated with the changes in the feature map of the deep learning model to be pruned before and after pruning. Based on the first channel importance factor, the deep learning model to be pruned is pruned to obtain the target deep learning model. The magnitude of the first channel importance factor is negatively correlated with the pruning probability of the target channel. Thus, the first channel importance factor is obtained by evaluating the changes in the feature map of the deep learning model before and after pruning. Compared to evaluating the channel importance of the deep learning model through its model parameters, evaluating the importance of the target channel of the deep learning model through the changes in the feature map before and after pruning is more objective. During the pruning process of the target channel of the deep learning model, the magnitude of the first channel importance factor is negatively correlated with the pruning probability of the target channel, and positively correlated with the changes in the feature map of the deep learning model before and after pruning. In other words, the larger the first channel importance factor of the target channel, the greater the changes in the feature map of the deep learning model before and after pruning, and the smaller the pruning probability of the target channel. This can effectively preserve the important features extracted by the model, reduce the risk of model performance degradation, and solve the problem of related technologies having a significant impact on the performance of deep learning models.

[0037] Please see Figure 2 , Figure 2 This is a flowchart of another deep learning model pruning method provided in an embodiment of this application. For example... Figure 2 As shown, the method includes the following steps:

[0038] Step 210: Input the validation set data into the deep learning model to be pruned to obtain the first feature map.

[0039] In the embodiments of this application, reference can be made to Figure 3 , Figure 3 This is a general conceptual diagram of a pruning method for a deep learning model provided in an embodiment of this application. For example... Figure 3 As shown, by inputting validation set data into the deep learning model to be pruned, feature maps, i.e., the first feature maps, can be obtained for each layer of the deep learning model to be pruned. Since the deep learning model to be pruned has already been optimized based on the training set data, continuing to use the training set data to evaluate the importance of the channels in the deep learning model to be pruned may affect the reliability of the feature mutual information metric. Therefore, to address this issue, this embodiment uses validation set data to measure the changes in the feature maps of the deep learning model before and after pruning. The validation set data can be represented as... .

[0040] Step 220: Obtain the second channel importance factor, apply the second channel importance factor to the deep learning model to be pruned, and obtain the channel importance factor optimization model.

[0041] In this embodiment, to evaluate the channel importance of the deep learning model to be pruned, after obtaining the deep learning model to be pruned, the weights of the deep learning model to be pruned (model parameters obtained during training) can be fixed. For each layer of the deep learning model to be pruned, a second channel importance factor can be introduced into the weights of each layer. The second channel importance factor can be a preset importance factor, which can be set based on experience.

[0042] by Figure 3 Taking an example, the first layer of the network has 4 channels, the second layer has 4 channels, and so on, up to the Nth layer, which also has 4 channels. An importance factor for the second channel can be applied to the first channel of the first layer. A second-channel importance factor can be applied to the second channel of the first-layer network. Similarly, a second channel importance factor can be applied to the fourth channel of the Nth layer network. Thus, the channel importance factor optimization model was obtained.

[0043] Step 230: Input the validation set data into the channel importance factor optimization model to obtain the second feature map.

[0044] In this embodiment of the application, after obtaining the channel importance factor optimization model, validation set data can be input into the channel importance factor optimization model to obtain the feature map output by each layer of the channel importance factor optimization model, i.e., the second feature map. For example... Figure 3 As shown, the feature map obtained by the first layer of the channel importance factor optimization model is... The feature map obtained from the second layer of the network is The feature map obtained from the Nth layer of the network is .

[0045] In one embodiment of this application, both the deep learning model to be pruned and the channel importance factor optimization model include N-layer networks. Each channel in the N-layer network of the deep learning model to be pruned corresponds to a first channel importance factor, and each channel in the N-layer network of the channel importance factor optimization model corresponds to a second channel importance factor, where N is a positive integer. The second feature map is obtained by the following formula:

[0046] ;

[0047] in, This is the second feature map of the i-th layer in the N-layer network of the channel importance factor optimization model. This refers to the second channel importance factor of the j-th channel in the i-th layer of the N-layer network in the channel importance factor optimization model. Let K be the first feature map of the j-th channel of the i-th layer network in the deep learning model to be pruned, and K be the number of channels of the i-th layer network in the N-layer network of the channel importance factor optimization model, where i is a positive integer less than or equal to N and j is a positive integer less than or equal to K.

[0048] In this embodiment, the second feature map of the pruned deep learning model can be predicted by introducing a second channel importance factor into the model to be pruned. The second feature map of the pruned deep learning model can be predicted based on the first feature map and the second channel importance factor using the above formula. Here, softmax is an activation function; specifically, the expression for the softmax activation function is as follows:

[0049] ;

[0050] Where e is the natural constant, Let K be the channel importance factor of the j-th channel of the i-th layer of the deep learning model to be pruned, K be the number of channels of the i-th layer of the deep learning model to be pruned, and k be a positive integer less than or equal to K.

[0051] Step 240: Based on the first feature map, the second feature map, and the target loss function, obtain the target loss value.

[0052] In this embodiment, to obtain the first channel importance factor, a gradient descent optimization algorithm can be used to optimize the second channel importance factor. Specifically, the gradient descent optimization algorithm can be applied to the target loss function to calculate the target loss value, and the second channel importance factor can be iteratively updated until the target loss value of the target loss function reaches a stable value, thus obtaining the first channel importance factor. The target loss function can be used to evaluate the changes in the feature map of the deep learning model before and after pruning.

[0053] To determine the changes in the feature maps of the deep learning model before and after pruning, this application proposes a feature mutual information metric. Feature mutual information is used to measure the degree of interdependence between two variables, and the feature mutual information metric is used to measure the changes in the feature maps of the entire network. Specifically, assuming... ,..., This represents the possible feature maps after pruning each layer in an N-layer network model. This represents the unpruned feature map in the i-th layer of the network. The following formula can be used to measure the change in the feature map of the entire network model, denoted as Diff. I(*) is primarily used to evaluate the feature changes in a single layer. Since calculating mutual information for high-dimensional feature representations in neural networks involves significant computation, this application proposes using the Hilbert-Schmidt Independence Criterion (HSIC) to simplify the calculation. The simplified index is named Feature Mutual Information I(X,Y), and the specific formula for calculating feature mutual information is shown below.

[0054] ;

[0055] ;

[0056] Here, Diff is used to indicate the changes in the feature map of the deep learning model to be pruned. Let be the first feature map of the i-th layer of the deep learning model to be pruned. This refers to the second feature map of the i-th layer of the deep learning model to be pruned. I(X,Y) is used to indicate the changes in the feature map of the i-th layer of the deep learning model to be pruned, and measures the degree of interdependence between variables X and Y. Let X be the Frobenius norm. X represents the validation set data. Y is the feature map of the network model before pruning, and Y is the feature map of the network model after pruning.

[0057] In one embodiment of this application, the target loss function includes a first loss function and a second loss function. The first loss function is used to evaluate the changes between the first feature map and the second feature map, and the second loss function is the loss function used during the training process of the deep learning model to be pruned.

[0058] In this embodiment, the target loss function can combine a loss function based on feature mutual information and a loss function based on validation set data. For example... Figure 3 As shown, the second feature map (the feature map obtained from the first layer of the channel importance factor optimization model) can be processed. The feature map obtained from the second layer network The feature map obtained from the Nth layer of the network The mutual information of features is calculated. The target loss function may include a first loss function for evaluating the changes between the first feature map and the second feature map, and a second loss function used during the training of the deep learning model to be pruned. Figure 3 As shown, the loss of the validation set data can be determined by the output of the channel importance factor and the true labels. For example, if the deep learning model to be pruned is an image classification model, then the second loss function of the deep learning model to be pruned can be the cross-entropy loss function.

[0059] The target loss function is as follows:

[0060] ;

[0061] ;

[0062] in, Let the target loss function be... Let the first loss function be... For the second loss function, The preset weighting coefficients, Let be the first feature map of the i-th layer of the deep learning model to be pruned. Here, N represents the second feature map of the i-th layer of the channel importance factor optimization model, and N is the number of network layers in the deep learning model to be pruned. This is the transpose of the first feature map of the i-th layer of the deep learning model to be pruned. This is the transpose of the second feature map of the i-th layer network of the channel importance factor optimization model.

[0063] The above formulas can be combined to obtain the formula for the target loss function as follows:

[0064] ;

[0065] in, Let the target loss function be... For the second loss function, The preset weighting coefficients, Let be the first feature map of the i-th layer of the deep learning model to be pruned. Here, N represents the second feature map of the i-th layer of the channel importance factor optimization model, and N is the number of network layers in the deep learning model to be pruned. This is the transpose of the first feature map of the i-th layer of the deep learning model to be pruned. This is the transpose of the second feature map of the i-th layer network of the channel importance factor optimization model.

[0066] Step 250: Based on the target loss value, update the importance factor of the second channel to obtain the importance factor of the first channel.

[0067] In this embodiment of the application, by using the gradient descent optimization algorithm on the target loss function, the second channel importance factor can be iteratively updated by an optimizer (e.g., the Adam optimizer), and the latest target loss value can be continuously calculated until the target loss value is stable. Then, the iterative update of the second channel importance factor can be stopped, and the latest second channel importance factor can be determined as the first channel importance factor.

[0068] Step 260: Based on the first channel importance factor, prune the deep learning model to be pruned to obtain the target deep learning model. The magnitude of the first channel importance factor is negatively correlated with the pruning probability of the target channel.

[0069] In this embodiment, the feature mutual information method is introduced to quantify the changes in feature maps in the neural network model before and after pruning. This method can capture the impact of pruning operations on the feature maps inside the model more precisely. The feature mutual information method can be used to predict the performance degradation of the model after pruning in advance, which helps to avoid performance loss caused by over-pruning. During the model pruning process, the feature mutual information method can help identify which feature maps are more critical to the final output, thereby retaining the more important parts.

[0070] Please see Figure 4 , Figure 4 This is a flowchart of another deep learning model pruning method provided in an embodiment of this application. For example... Figure 4 As shown, the method includes the following steps:

[0071] Step 410: Obtain the first channel importance factor of the target channel in the deep learning model to be pruned. The deep learning model to be pruned includes an N-layer network, and each channel in the N-layer network corresponds to a first channel importance factor, where N is a positive integer.

[0072] Step 420: Obtain the N target pruning numbers corresponding to the N-layer network, where each layer of the N-layer network corresponds to one of the N target pruning numbers.

[0073] In this embodiment, a compression ratio can be set. For the i-th layer in the N-layer network, the product of the number of channels in the i-th layer and the compression ratio can be determined as the target pruning number for the i-th layer. However, depending on the actual model structure, the layers in the deep learning model to be pruned are highly correlated, and the impact of different layers on the deep learning model to be pruned varies greatly. Setting the same compression ratio for different layers will have a certain impact on the performance of the deep learning model to be pruned.

[0074] The following explanation addresses the issue of high inter-layer correlation in the deep learning model to be pruned. (See also...) Figure 5 , Figure 5 This is a schematic diagram of a model pruning process provided in an embodiment of this application. For example... Figure 5 As shown, an image of size 24×24×3 can be input into the deep learning model to be pruned. For the first layer of the deep learning model to be pruned, the first layer has... There are channels, and the target pruning number for the first layer network is . In the feature map output by the first layer network, it can be seen from the... Select from the channels Each channel will... Each channel is removed from the first layer of the network. Because there are relationships between layers in a convolutional neural network, that is, when removing the channel from the first layer of the network… After each channel, the output of the feature map of the first layer will also decrease. One channel. For example... Figure 5 As shown, the feature map of the first layer network is composed of Transform into .

[0075] At the same time, the kernel size of the next layer (the second layer) is also affected by the output feature map of the first layer. The kernel size of the second layer is determined by... Transform into If the channel importance factor in the second layer network is minimized... With one channel removed, the kernel size of the second layer network can be changed to... .

[0076] As can be seen from the above channel pruning process, there is a certain correlation between the pruning of the i-th layer and the pruning of the (i+1)-th layer in the N-layer network of the deep learning model to be pruned. Therefore, in determining the number of pruning layers in each layer of the deep learning model to be pruned, ensuring the performance of the pruned model while meeting the expected model size is also an urgent problem to be solved.

[0077] To address this issue, this application proposes an algorithm for determining the number of pruning channels. This algorithm aims to determine the number of layers to be pruned in each layer of the deep learning model to be pruned, while minimizing model accuracy loss while meeting the model compression size requirements. Specifically, this application establishes a non-convex optimization problem to minimize the objective loss function value of the pruned deep learning model on the validation set data, under the condition of satisfying the objective constraints. This also indicates that the pruning strategy has the least impact on the entire deep learning model to be pruned while meeting the expected model size. To solve this non-convex optimization problem, the algorithm used in this application to determine the number of pruning channels is a heuristic search algorithm, an algorithm constructed based on intuition or experience, which can provide a feasible solution with good performance within an acceptable cost (computation time and space).

[0078] In one embodiment of this application, obtaining the N target pruning numbers corresponding to the N-layer network includes: obtaining the N first pruning numbers corresponding to the N-layer network; updating the N first pruning numbers through a target loss function and target constraints to obtain the N target pruning numbers, wherein the target constraints make the total number of channels of the N-layer network after pruning the deep learning model to be pruned less than or equal to a preset target number of channels.

[0079] In this embodiment, N first pruning numbers can be obtained as a set of feasible solutions. These N first pruning numbers can be fine-tuned, and the fine-tuned N first pruning numbers can satisfy the target constraint condition. The resulting fine-tuned N first pruning numbers can then be used as a new set of solutions. The quantitative relationship between the target loss function value obtained based on the fine-tuned N first pruning numbers and the target loss function value obtained based on the un-fine-tuned N first pruning numbers can be determined using the target loss function. If the target loss function value obtained based on the fine-tuned N first pruning numbers is less than the target loss function value obtained based on the un-fine-tuned N first pruning numbers, the N first pruning numbers can be updated to the fine-tuned N first pruning numbers.

[0080] The above method allows for continuous updating of the N first pruning numbers, and during the updating process, the target loss function value can be continuously reduced while satisfying the target constraints. A preset number of updates can be set; when the preset number of updates to the N first pruning numbers is reached, the latest N first pruning numbers can be determined as the N target pruning numbers. The target constraints can be expressed as follows:

[0081] ;

[0082] ;

[0083] in, Let the target loss function be... Let the first loss function be... For the second loss function, The preset weighting coefficients, Here, represents the number of remaining channels in the i-th layer of the pruned model, and Size represents the preset total number of channels in the deep learning model to be pruned. For example, if the total number of channels in the deep learning model to be pruned is 100 and the pruning ratio is 70%, then the preset total number of channels can be determined to be 70. That is, the total number of channels in the target deep learning model obtained after pruning is less than or equal to 70.

[0084] For example, in one embodiment of this application, updating the N first pruning numbers to obtain N target pruning numbers through a target loss function and target constraints includes: determining a first loss value based on the N first pruning numbers and the target loss function, wherein the N first pruning numbers satisfy the target constraints; obtaining N second pruning numbers; and adjusting the N second pruning numbers in the following manner if the total number of channels in the N-layer network of the first pruning model does not satisfy the target constraints, until the total number of channels in the N-layer network of the second pruning model satisfies the target constraints: for the i-th layer of the N second pruning numbers... The second pruning number corresponding to the i-th layer network is determined as the second pruning number corresponding to the i-th layer network among the N first pruning numbers, provided that the first hyperparameters of the deep learning model to be pruned meet the first objective condition; if the first hyperparameters meet the first objective condition and the second hyperparameters of the deep learning model to be pruned meet the second objective condition, the second pruning number corresponding to the i-th layer network is adjusted, and the adjusted second pruning number corresponding to the i-th layer network is within the target value range; if the first hyperparameters do not meet the first objective condition, the second pruning number corresponding to the i-th layer network is re-obtained.

[0085] If the total number of channels in the N-layer network in the first pruning model satisfies the target constraint, a second loss value is determined based on the N second pruning numbers and the target loss function; if the second loss value is less than the first loss value, the N second pruning numbers are determined as N target pruning numbers; if the second loss value is greater than or equal to the first loss value, the N first pruning numbers are determined as N target pruning numbers.

[0086] Wherein, the first pruning model is the model obtained by pruning the deep learning model to be pruned based on the N second pruning numbers, and the second pruning model is the model obtained by pruning the deep learning model to be pruned based on the adjusted N second pruning numbers.

[0087] In this embodiment, a set of feasible solutions can be initialized, namely the N first pruning numbers. Alternatively, multiple sets of feasible solutions can be initialized, namely multiple sets of N third pruning numbers that satisfy the target constraint, and the target loss function value is calculated based on any one of the multiple sets of feasible solutions, wherein the N first pruning numbers can be one set of feasible solutions among the multiple sets of feasible solutions. The multiple sets of feasible solutions can be denoted as HM, as shown below:

[0088] ;

[0089] Here, HMS represents the number of feasible solutions; for example, HMS can be 5. A single line in HM above represents a set of feasible solutions, and each set includes N third-pruned branches. This is used to represent the third pruning number of the i-th layer network in the m-th feasible solution of the k-th solution.

[0090] N second pruning numbers can be obtained, and these N second pruning numbers can be randomly generated. The deep learning model to be pruned can be pruned based on these N second pruning numbers to obtain a first pruned model. If the total number of channels in the N layers of the first pruned model does not meet the target constraint, for the second pruning number corresponding to the i-th layer among the N second pruning numbers, if the first hyperparameter meets the first target condition, the first pruning number corresponding to the i-th layer among the N first pruning numbers is determined as the second pruning number corresponding to the i-th layer.

[0091] The first target condition may include the first hyperparameter being greater than or equal to a first random number. The first hyperparameter is a hyperparameter of the deep learning model to be pruned, which can be denoted as HMCR, for example, HMCR=0.9. The first random number is a randomly generated number with a value between 0 and 1, which can be denoted as HMCR=0.9. .

[0092] If the first hyperparameter satisfies the first objective condition, it can be further determined whether the second hyperparameter satisfies the second objective condition. If the first hyperparameter satisfies the first objective condition and the second hyperparameter satisfies the second objective condition, the second pruning number corresponding to the i-th layer network is adjusted, and the adjusted second pruning number corresponding to the i-th layer network is within the target value range.

[0093] The second objective condition may include the second hyperparameter being greater than or equal to the second random number. The second hyperparameter is another hyperparameter of the deep learning model to be pruned, which can be denoted as PAR, for example, PAR = 0.5. The second random number is a randomly generated number with a value between 0 and 1, which can be denoted as... In this case, the second pruning number corresponding to the i-th layer network is adjusted, which means adjusting the first pruning number corresponding to the i-th layer network among the N first pruning numbers.

[0094] During the adjustment of the second pruning number corresponding to the i-th layer network, the second pruning number corresponding to the i-th layer network can be fine-tuned based on a preset adjustment step size and a third random number. The specific fine-tuning process can be found in the following formula: ;in, Here, BW represents the second pruning number corresponding to the i-th layer of the network, and BW represents the preset adjustment step size, which can be set to 20. The third random number.

[0095] After adjusting the second pruning number corresponding to the i-th layer network, it can be determined whether the adjusted second pruning number is within a target value range. The target value range is the range between the upper and lower limits of the preset pruning number for the i-th layer network. If the adjusted second pruning number is greater than the upper limit, the adjusted second pruning number can be redefined as the upper limit. If the adjusted second pruning number is less than the lower limit, the adjusted second pruning number can be redefined as the lower limit. In this way, it can be determined that the adjusted second pruning number is within the target value range.

[0096] To escape local optima, if the first hyperparameter does not meet the first objective condition, the second pruning number corresponding to the i-th layer network can be re-obtained. During the process of re-obtaining the second pruning number corresponding to the i-th layer network, the N first pruning numbers can be skipped, and the second pruning number corresponding to the i-th layer network can be randomly determined. However, it should be noted that the randomly determined second pruning number corresponding to the i-th layer network is within the target value range.

[0097] For example, in one embodiment of this application, the second pruning number corresponding to the i-th layer network can be re-determined based on the preset upper limit of the pruning number of the i-th layer network, the preset lower limit of the pruning number of the i-th layer network, and a fourth random number. The fourth random number is a randomly generated number with a value between 0 and 1. The following formula can be used as a reference in the process of re-determining the second pruning number corresponding to the i-th layer network:

[0098] ;

[0099] in, This is the second pruning number corresponding to the i-th layer network. The fourth random number, This is the lower limit of the preset pruning number for the i-th layer network. This is the upper limit of the preset pruning number for the i-th layer network.

[0100] In this embodiment, if the total number of channels in the N-layer network of the first pruning model does not meet the target constraint, the above operation can be repeated to adjust the N second pruning numbers until the total number of channels in the N-layer network after pruning the deep learning model to be pruned based on the N second pruning numbers meets the target constraint. That is, the deep learning model to be pruned can be pruned based on the adjusted N second pruning numbers to obtain the second pruning model. If the total number of channels in the N-layer network of the first pruning model meets the target constraint, the adjustment of the N second pruning numbers can be stopped.

[0101] If the total number of channels in the N-layer network in the first pruning model or the second pruning model satisfies the target constraint, a second loss value can be determined based on the N second pruning numbers and the target loss function; if the second loss value is less than the first loss value, the N second pruning numbers are determined as N target pruning numbers; if the second loss value is greater than or equal to the first loss value, the N first pruning numbers are determined as N target pruning numbers.

[0102] In this embodiment, if the total number of channels in the N-layer network after pruning the deep learning model to be pruned based on the N second pruning numbers satisfies the target constraint, the N second pruning numbers can be considered as a set of feasible solutions, and the second loss values ​​of the N second pruning numbers can be determined. If the second loss values ​​of the N second pruning numbers are less than the first loss values ​​of the N first pruning numbers, the pruning scheme of the N second pruning numbers can be considered superior to the pruning scheme of the N first pruning numbers, and the N second pruning numbers can be determined as N target pruning numbers. Furthermore, the pruning scheme can be further optimized in the above manner. The number of optimization attempts can be preset, denoted as K, for example, K can be 20. When the number of optimization attempts reaches the target number, the optimization of the pruning scheme is stopped, and N target pruning numbers can be determined based on the current optimal pruning scheme.

[0103] Step 430: For the i-th layer of the N-layer network, perform the following operations to obtain the target deep learning model: Sort the K channels in the i-th layer network in ascending order of the importance factor of the first channel. Starting from the target channel with the smallest first channel importance factor among the K channels, select Q channels from the K channels in ascending order of the first channel importance factor, and delete the Q channels from the i-th layer network. Q is equal to the target pruning number corresponding to the i-th layer network, K is a positive integer, and Q is a positive integer less than or equal to K.

[0104] In this embodiment, the number of channels to be pruned in each layer of the deep learning model to be pruned can be obtained through the algorithm for calculating the number of pruned channels. For the i-th layer of the deep learning model to be pruned, the K channels in the i-th layer can be sorted in ascending order of channel importance factor. Starting with the channel with the smallest channel importance factor among the K channels, Q channels are selected from the K channels in ascending order of channel importance factor, where Q equals the number of channels to be pruned in the i-th layer. These Q channels are then removed from the i-th layer. After pruning the N-layer network, the pruned model can be determined as the target deep learning model.

[0105] In this embodiment, the number of pruning points for each layer in the deep learning model to be pruned can be set individually. Compared with the method of using the same pruning ratio in related technologies, the pruning scheme obtained by this method is more flexible and reduces the interference caused by manually setting the pruning ratio.

[0106] The following specific embodiment will further illustrate the pruning method of the deep learning model in this application. This application embodiment can be used to solve a tree-falling task, the goal of which is to determine whether fallen trees appear in an image. Tree falling refers to the phenomenon where upright trees grow crookedly, or even completely collapse to the ground. To solve this task, a classification network model can be trained based on existing tree-falling data. For example, the classification network model can be ResNet50, ResNet18, MobileNet, etc. In this case, the trained classification network model can be used as the deep learning model to be pruned, and this pruned deep learning model is retained. The following description uses the selection of the ResNet18 model as the deep learning model to be pruned as an example.

[0107] First, a channel importance factor optimization model can be constructed to optimize the channel importance factor. Specifically, channel importance factors are set on the feature map channels in the ResNet18 model. , construct as Figure 1 The channel importance factor optimization model is shown. The data can be validation set data, and the weights can be the weights trained in the deep learning model to be pruned. During network optimization, only the channel importance factors need to be optimized. The channel importance factor optimization model undergoes iterative updates, and the loss function is as described in the formula above. During the iterative update of the channel importance factor, an optimizer (Adam optimizer) can be used based on the validation set data to optimize the channel importance factor. The process continues iteratively until the channel importance factor optimization model stabilizes, at which point the channel importance factor can be selected. This represents the importance of the feature channels in the ResNet18 model.

[0108] Then, the number of channels that need to be pruned for each layer of the ResNet18 model can be calculated. The number of feature map channels that need to be removed for each layer of the ResNet18 model can be denoted as... Algorithm 1 can be used to pair The solution process can be found in the above-mentioned manner, and will not be repeated here.

[0109] The above method is used to obtain the number of feature map channels that need to be deleted in each layer of the ResNet18 model. Channel importance factor of feature map channels in each layer of the ResNet18 model Next, for the i-th layer of the ResNet18 model, the feature map channels of the i-th layer can be sorted in descending order of importance factor, and the channels with lower importance factors can be sorted. One channel is deleted. For the pruned ResNet18 model, the training set data and the weights obtained during the training of the ResNet18 model before pruning can be loaded to fine-tune the pruned ResNet18 model until the entire network model converges again, and the classification effect of the obtained network model can be verified.

[0110] Please see Figure 6 , Figure 6 This is a flowchart illustrating a method for pruning a deep learning model according to an embodiment of this application. Figure 6 As shown, the method includes the following steps:

[0111] Step 610: Input the validation set data into the deep learning model to be pruned to obtain the first feature map.

[0112] Step 620: Obtain the second channel importance factor, apply the second channel importance factor to the deep learning model to be pruned, and obtain the channel importance factor optimization model.

[0113] Step 630: Input the validation set data into the channel importance factor optimization model to obtain the second feature map.

[0114] In this embodiment, both the deep learning model to be pruned and the channel importance factor optimization model include N-layer networks. Each channel in the N-layer network of the deep learning model to be pruned corresponds to a first channel importance factor, and each channel in the N-layer network of the channel importance factor optimization model corresponds to a second channel importance factor, where N is a positive integer. The second feature map is obtained using the following formula:

[0115] ;

[0116] in, This is the second feature map of the i-th layer in the N-layer network of the channel importance factor optimization model. This refers to the second channel importance factor of the j-th channel in the i-th layer of the N-layer network in the channel importance factor optimization model. Let K be the first feature map of the j-th channel of the i-th layer network in the deep learning model to be pruned, and K be the number of channels of the i-th layer network in the N-layer network of the channel importance factor optimization model, where i is a positive integer less than or equal to N and j is a positive integer less than or equal to K.

[0117] Step 640: Based on the first feature map, the second feature map, and the target loss function, obtain the target loss value.

[0118] In this embodiment of the application, the target loss function includes a first loss function and a second loss function. The first loss function is used to evaluate the change between the first feature map and the second feature map, and the second loss function is the loss function used in the training process of the deep learning model to be pruned.

[0119] The target loss function is as follows:

[0120] ;

[0121]

[0122] in, Let the target loss function be... Let the first loss function be... For the second loss function, The preset weighting coefficients, Let be the first feature map of the i-th layer of the deep learning model to be pruned. Here, N represents the second feature map of the i-th layer of the channel importance factor optimization model, and N is the number of network layers in the deep learning model to be pruned. This is the transpose of the first feature map of the i-th layer of the deep learning model to be pruned. This is the transpose of the second feature map of the i-th layer network of the channel importance factor optimization model.

[0123] Step 650: Based on the target loss value, update the importance factor of the second channel to obtain the importance factor of the first channel.

[0124] Step 660: Obtain the N target pruning numbers corresponding to the N-layer network, where each layer of the N-layer network corresponds to one of the N target pruning numbers.

[0125] In this embodiment of the application, N first pruning numbers corresponding to the N-layer network can be obtained; the N first pruning numbers are updated by the target loss function and the target constraint to obtain N target pruning numbers, wherein the target constraint makes the total number of channels of the N-layer network after pruning of the deep learning model to be pruned less than or equal to the preset target number of channels.

[0126] In the process of updating the N first pruning numbers to obtain N target pruning numbers through the target loss function and target constraints, a first loss value can be determined based on the N first pruning numbers and the target loss function, and the N first pruning numbers satisfy the target constraints. Then, N second pruning numbers are obtained.

[0127] If the total number of channels in the N-layer network of the first pruning model does not meet the target constraint, the N second pruning numbers are adjusted in the following way until the total number of channels in the N-layer network of the second pruning model meets the target constraint: For the second pruning number corresponding to the i-th layer of the N second pruning numbers, if the first hyperparameter of the deep learning model to be pruned meets the first target condition, the first pruning number corresponding to the i-th layer of the N first pruning numbers is determined as the second pruning number corresponding to the i-th layer; if the first hyperparameter meets the first target condition and the second hyperparameter of the deep learning model to be pruned meets the second target condition, the second pruning number corresponding to the i-th layer is adjusted, and the adjusted second pruning number corresponding to the i-th layer is within the target value range; if the first hyperparameter does not meet the first target condition, the second pruning number corresponding to the i-th layer is re-obtained.

[0128] If the total number of channels in the N-layer network in the first pruning model satisfies the target constraint, the second loss value is determined based on the N second pruning numbers and the target loss function.

[0129] If the second loss value is less than the first loss value, the N second pruning numbers are determined as N target pruning numbers. If the second loss value is greater than or equal to the first loss value, the N first pruning numbers are determined as N target pruning numbers.

[0130] Wherein, the first pruning model is the model obtained by pruning the deep learning model to be pruned based on the N second pruning numbers, and the second pruning model is the model obtained by pruning the deep learning model to be pruned based on the adjusted N second pruning numbers.

[0131] Step 670: For the i-th layer of the N-layer network, perform the following operations to obtain the target deep learning model: Sort the K channels in the i-th layer network in ascending order of the first channel importance factor. Starting from the target channel with the smallest first channel importance factor among the K channels, select Q channels from the K channels in ascending order of the first channel importance factor, and delete the Q channels from the i-th layer network. Q is equal to the target pruning number corresponding to the i-th layer network.

[0132] In this embodiment, a first channel importance factor of the target channel in the deep learning model to be pruned is obtained. The first channel importance factor is used to evaluate the importance of the target channel in the deep learning model to be pruned. The first channel importance factor is obtained by evaluating the changes in the feature map of the deep learning model to be pruned before and after pruning. The magnitude of the first channel importance factor is positively correlated with the changes in the feature map of the deep learning model to be pruned before and after pruning. Based on the first channel importance factor, the deep learning model to be pruned is pruned to obtain the target deep learning model. The magnitude of the first channel importance factor is negatively correlated with the pruning probability of the target channel. Thus, the first channel importance factor is obtained by evaluating the changes in the feature map of the deep learning model before and after pruning. Compared to evaluating the channel importance of the deep learning model through its model parameters, evaluating the importance of the target channel of the deep learning model through the changes in the feature map before and after pruning is more objective. During the pruning process of the target channel of the deep learning model, the magnitude of the first channel importance factor is negatively correlated with the pruning probability of the target channel, and positively correlated with the changes in the feature map of the deep learning model before and after pruning. In other words, the larger the first channel importance factor of the target channel, the greater the changes in the feature map of the deep learning model before and after pruning, and the smaller the pruning probability of the target channel. This can effectively preserve the important features extracted by the model, reduce the risk of model performance degradation, and solve the problem of related technologies having a significant impact on the performance of deep learning models.

[0133] It is important to understand that Figures 1 to 6 The explanations of the same or corresponding steps can be cross-referenced. For example, Figure 2 The explanations of steps 210 and 220 are applicable to Figure 6 Steps 610 and 620 in the process.

[0134] Meanwhile, it should be understood that the deep learning model pruning method provided in this application embodiment has the following beneficial effects: By comprehensively considering the changes in feature maps before and after single-layer pruning and their perturbation effect on the entire network model, the network model is pruned, which effectively solves the problem of severe performance loss in the inference acceleration process of the model pruning methods used in related technologies. First, related technologies often only focus on the changes in model parameters, while ignoring the importance of feature maps. By introducing a feature map-based mutual information method, this application embodiment can accurately measure the changes in feature maps before and after pruning. This makes it possible not only to reduce redundant parameters during model pruning, but also to effectively retain the important features extracted by the model, thereby reducing the risk of model performance degradation. Second, this application embodiment introduces a channel importance factor into the network model. By combining feature map mutual information loss and validation set loss, the importance of each channel can be solved using gradient descent. This method can dynamically adjust and optimize the pruning strategy of each channel, making pruning more accurate, thereby ensuring that important features are retained and further improving the performance of the pruned network model. Third, the model pruning methods used in related technologies typically apply the same treatment to each layer of the deep learning model to be pruned. However, this application considers the different importance of each layer's weights to the entire network. An optimization model can be established to solve for the pruning ratio scheme of each layer, enabling targeted optimization of the pruning effect of each layer and avoiding over-pruning of key layers, thereby maintaining the overall network performance. In other words, the embodiments of this application can effectively reduce model performance loss by comprehensively considering feature map changes and channel importance, enabling accurate channel importance assessment and inter-layer differentiation processing, making the pruning strategy more reasonable and efficient. Simultaneously, it can successfully accelerate model inference while ensuring performance, meeting the needs of practical applications.

[0135] Furthermore, this application measures feature map changes based on the feature mutual information method, introducing the feature mutual information method to quantify the changes in feature maps in the neural network model before and after pruning. This method can more precisely capture the impact of pruning operations on the internal feature maps of the model. The feature mutual information method can be used to predict performance degradation after model pruning in advance, helping to avoid performance loss caused by over-pruning. During model pruning, the feature mutual information method can help identify which feature maps are more critical to the final output, thus retaining the more important parts. This approach has good versatility and can be applied to different neural network architectures and task scenarios, increasing the practicality and portability of the technical solution. This application introduces a feature channel importance factor to characterize the importance of each channel in the model. When evaluating channel importance, it comprehensively considers the feature mutual information loss and the performance loss on the validation set, providing a multi-dimensional evaluation standard. It can accurately assess the contribution of each channel to the overall model performance, providing a basis for subsequent model pruning. By introducing the channel importance factor, the transparency of the model's internal mechanisms can also be improved, making the model pruning process more interpretable. This application solves for the optimal pruning scheme for each layer of the network by modeling. Based on the channel importance factor, a mathematical model is established to determine the pruning amount for each layer, ensuring more precise pruning operations. By solving the optimization model, the optimal pruning ratio for each layer is obtained, maximizing the performance of the pruned model. Through optimization methods, the performance of the pruned model is maximized, maintaining a high prediction accuracy. This modeling method can automatically generate pruning strategies, reducing interference from manually setting pruning ratios. This modeling method is applicable to various neural network models, enhancing its adaptability in different application scenarios. This application comprehensively considers the perturbation effect of single-layer pruning on the overall model. It analyzes not only the impact of single-layer pruning on local performance but also evaluates the perturbation effect of pruning on the entire model, ensuring model optimization from a global perspective. By comprehensively considering the interactions between multiple layers, a multi-layer collaborative pruning strategy is proposed to avoid the negative impact of a single pruning operation on overall performance. The pruning strategy is dynamically adjusted based on the evaluation results of the perturbation effect to ensure the stability of model performance. The consideration of global perturbation effects enhances the robustness of the model. By comprehensively considering the perturbation effect of single-layer pruning on the overall model, the impact of pruning on the overall network performance is minimized. This application employs the gradient descent optimization algorithm to solve for the channel importance factor, providing an efficient solution. During the solution process, both feature mutual information loss and performance loss on the validation set are considered simultaneously, ensuring a comprehensive evaluation of channel importance. The gradient descent optimization algorithm enables automatic optimization of the channel importance factor without manual parameter adjustment, improving the automation level of model optimization.

[0136] Please see Figure 7 , Figure 7This is a structural block diagram of a pruning device for a deep learning model provided in an embodiment of this application. Figure 7 As shown in the figure, this application embodiment provides a deep learning model pruning device 700, which includes an acquisition module 710 and a pruning module 720.

[0137] The acquisition module 710 is used to acquire the first channel importance factor of the target channel in the deep learning model to be pruned. The first channel importance factor is used to evaluate the importance of the target channel in the deep learning model to be pruned. The first channel importance factor is obtained by evaluating the changes in the feature map of the deep learning model to be pruned before and after pruning. The magnitude of the first channel importance factor is positively correlated with the changes in the feature map of the deep learning model to be pruned before and after pruning.

[0138] The pruning module 720 is used to prune the deep learning model to be pruned based on the first channel importance factor to obtain the target deep learning model. The magnitude of the first channel importance factor is negatively correlated with the pruning probability of the target channel.

[0139] In this embodiment, a first channel importance factor of the target channel in the deep learning model to be pruned is obtained. The first channel importance factor is used to evaluate the importance of the target channel in the deep learning model to be pruned. The first channel importance factor is obtained by evaluating the changes in the feature map of the deep learning model to be pruned before and after pruning. The magnitude of the first channel importance factor is positively correlated with the changes in the feature map of the deep learning model to be pruned before and after pruning. Based on the first channel importance factor, the deep learning model to be pruned is pruned to obtain the target deep learning model. The magnitude of the first channel importance factor is negatively correlated with the pruning probability of the target channel. Thus, the first channel importance factor is obtained by evaluating the changes in the feature map of the deep learning model before and after pruning. Compared to evaluating the channel importance of the deep learning model through its model parameters, evaluating the importance of the target channel of the deep learning model through the changes in the feature map before and after pruning is more objective. During the pruning process of the target channel of the deep learning model, the magnitude of the first channel importance factor is negatively correlated with the pruning probability of the target channel, and positively correlated with the changes in the feature map of the deep learning model before and after pruning. In other words, the larger the first channel importance factor of the target channel, the greater the changes in the feature map of the deep learning model before and after pruning, and the smaller the pruning probability of the target channel. This can effectively preserve the important features extracted by the model, reduce the risk of model performance degradation, and solve the problem of related technologies having a significant impact on the performance of deep learning models.

[0140] The deep learning model pruning device provided in this application embodiment can implement all the processes implemented in the above method embodiments, and will not be described again here to avoid repetition.

[0141] like Figure 8As shown in the illustration, this application also provides an electronic device 800. The electronic device 800 includes a processor 810 and a memory 820. The memory 820 stores programs or instructions, which, when executed by the processor 810, implement the steps of any of the methods described above. For example, when the program is executed by the processor 810, it implements the following process: obtaining a first channel importance factor of the target channel in the deep learning model to be pruned; the first channel importance factor is used to evaluate the importance of the target channel in the deep learning model to be pruned; the first channel importance factor is obtained by evaluating the changes in the feature map of the deep learning model before and after pruning; the magnitude of the first channel importance factor is positively correlated with the changes in the feature map of the deep learning model before and after pruning; based on the first channel importance factor, pruning is performed on the deep learning model to be pruned to obtain a target deep learning model; the magnitude of the first channel importance factor is negatively correlated with the pruning probability of the target channel. Thus, the first channel importance factor is obtained by evaluating the changes in the feature map of the deep learning model before and after pruning. Compared to evaluating the channel importance of the deep learning model through its model parameters, evaluating the importance of the target channel of the deep learning model through the changes in the feature map before and after pruning is more objective. During the pruning process of the target channel of the deep learning model, the magnitude of the first channel importance factor is negatively correlated with the pruning probability of the target channel, and positively correlated with the changes in the feature map of the deep learning model before and after pruning. In other words, the larger the first channel importance factor of the target channel, the greater the changes in the feature map of the deep learning model before and after pruning, and the smaller the pruning probability of the target channel. This can effectively preserve the important features extracted by the model, reduce the risk of model performance degradation, and solve the problem of related technologies having a significant impact on the performance of deep learning models.

[0142] This application also provides a readable storage medium storing a program or instructions that, when executed by a processor, implement the steps of various embodiments of the deep learning model pruning method and achieve the same technical effect. To avoid repetition, these steps will not be repeated here.

[0143] The processor is the processor in the electronic device described in the above embodiments. The readable storage medium includes computer-readable storage media, such as computer read-only memory (ROM), random access memory (RAM), magnetic disk, or optical disk.

[0144] This application embodiment also provides a chip, which includes a processor and a communication interface. The communication interface is coupled to the processor. The processor is used to run programs or instructions to implement the various processes of the above method embodiments and achieve the same technical effect. To avoid repetition, it will not be described again here.

[0145] This application provides a computer program product, which is stored in a storage medium and executed by at least one processor to implement the various processes of the above method embodiments and achieve the same technical effects. To avoid repetition, it will not be described again here.

[0146] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element. Furthermore, it should be noted that the scope of the methods and apparatuses in the embodiments of this application is not limited to performing functions in the order shown or discussed, but may also include performing functions substantially simultaneously or in the reverse order, depending on the functions involved. For example, the described methods may be performed in a different order than described, and various steps may be added, omitted, or combined. Additionally, features described with reference to certain examples may be combined in other examples.

[0147] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a computer software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0148] The embodiments of this application have been described above with reference to the accompanying drawings. However, this application is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of this application without departing from the spirit and scope of the claims, and all of these forms are within the protection scope of this application.

Claims

1. A method for pruning a deep learning model, the method comprising: The method is performed by a target device, and comprises: Based on the tree lodging data, a classification network model is trained, and the trained classification network model is used as a to-be-pruned deep learning model; Obtain a first channel importance factor of a target channel in the to-be-pruned deep learning model, the first channel importance factor is used to evaluate the importance of the target channel in the to-be-pruned deep learning model, the first channel importance factor is obtained by evaluating the feature map change before and after pruning the to-be-pruned deep learning model; the size of the first channel importance factor is positively correlated with the feature map change before and after pruning the to-be-pruned deep learning model; Based on the first channel importance factor, the to-be-pruned deep learning model is pruned to obtain a target deep learning model, and the size of the first channel importance factor is negatively correlated with the pruning probability of the target channel; the target deep learning model is used to determine whether there is a tree with lodging in the image; Wherein, the first channel importance factor of the target channel in the to-be-pruned deep learning model is obtained by: Input the verification set data into the to-be-pruned deep learning model to obtain a first feature map; Obtain a second channel importance factor, and apply the second channel importance factor to the to-be-pruned deep learning model to obtain a channel importance factor optimization model; Input the verification set data into the channel importance factor optimization model to obtain a second feature map; Based on the first feature map, the second feature map and a target loss function, a target loss value is obtained; Based on the target loss value, the second channel importance factor is updated to obtain a first channel importance factor; Wherein, the target loss function includes a first loss function and a second loss function, the first loss function is used to evaluate the change between the first feature map and the second feature map, and the second loss function is a loss function used in the training process of the to-be-pruned deep learning model.

2. The method of claim 1, wherein, The to-be-pruned deep learning model and the channel importance factor optimization model both include N-layer networks, each channel in the N-layer network of the to-be-pruned deep learning model corresponds to a first channel importance factor, each channel in the N-layer network of the channel importance factor optimization model corresponds to a second channel importance factor, and N is a positive integer; the second feature map is obtained by the following formula: ； wherein, is a second feature map of an i-th network in the N-layer network in the channel importance factor optimization model, is a second channel importance factor of a j-th channel of the i-th network in the N-layer network in the channel importance factor optimization model, is a first feature map of a j-th channel of an i-th network in the to-be-pruned deep learning model, K is a channel number of the i-th network in the N-layer network in the channel importance factor optimization model, i is a positive integer less than or equal to N, and j is a positive integer less than or equal to K.

3. The method of claim 1, wherein, The target loss function is as follows: ； ； wherein, is the target loss function, is the first loss function, is the second loss function, is a preset weight coefficient, is the first feature map of the i-th network layer of the to-be-pruned deep learning model, is the second feature map of the i-th network layer of the channel importance factor optimization model, and N is the number of network layers of the to-be-pruned deep learning model, is the transpose of the first feature map of the i-th network layer of the to-be-pruned deep learning model, is the transpose of the second feature map of the i-th network layer of the channel importance factor optimization model.

4. The method of claim 1, wherein, The to-be-pruned deep learning model includes N-layer networks, each channel in the N-layer network corresponds to a first channel importance factor, and N is a positive integer; Based on the first channel importance factor, the to-be-pruned deep learning model is pruned, comprising: Obtain N target pruning numbers corresponding to the N-layer network, each layer in the N-layer network corresponds to one of the N target pruning numbers; For an i-th layer network in the N-layer network, the following operations are performed: K channels in the i-th layer network are sorted in ascending order of first channel importance factors, starting from a target channel corresponding to a smallest first channel importance factor among the K channels, Q channels are selected from the K channels in ascending order of the first channel importance factors, the Q channels are deleted from the i-th layer network, Q is equal to a target pruning number corresponding to the i-th layer network, K is a positive integer, and Q is a positive integer less than or equal to K.

5. The method of claim 4, wherein, The N target pruning numbers corresponding to the N-layer network are obtained by: obtaining N first pruning numbers corresponding to the N-layer network; updating the N first pruning numbers by using a target loss function and a target constraint condition to obtain N target pruning numbers, the target constraint condition being that a total number of channels of the N-layer network after pruning of the to-be-pruned deep learning model is less than or equal to a preset target number of channels.

6. The method of claim 5, wherein, The N target pruning numbers are obtained by updating the N first pruning numbers by using a target loss function and a target constraint condition, and the target constraint condition is that a total number of channels of the N-layer network after pruning of the to-be-pruned deep learning model is less than or equal to a preset target number of channels. The N target pruning numbers are obtained by updating the N first pruning numbers by using a target loss function and a target constraint condition, and the target constraint condition is that a total number of channels of the N-layer network after pruning of the to-be-pruned deep learning model is less than or equal to a preset target number of channels. a first loss value is determined based on the N first pruning numbers and the target loss function, the N first pruning numbers satisfying the target constraint condition; obtaining N second pruning numbers; in a case where the total number of channels of the N-layer network in the first pruning model does not satisfy the target constraint condition, the N second pruning numbers are adjusted by the following manner until the total number of channels of the N-layer network in the second pruning model satisfies the target constraint condition: for an i-th layer network corresponding to a second pruning number in the N second pruning numbers, in a case where a first hyperparameter of the to-be-pruned deep learning model satisfies a first target condition, a first pruning number corresponding to the i-th layer network in the N first pruning numbers is determined as the second pruning number corresponding to the i-th layer network; in a case where the first hyperparameter satisfies the first target condition and a second hyperparameter of the to-be-pruned deep learning model satisfies a second target condition, the second pruning number corresponding to the i-th layer network is adjusted, and an adjusted second pruning number corresponding to the i-th layer network is within a target value range; in a case where the first hyperparameter does not satisfy the first target condition, the second pruning number corresponding to the i-th layer network is re-obtained; in a case where the total number of channels of the N-layer network in the first pruning model satisfies the target constraint condition, a second loss value is determined based on the N second pruning numbers and the target loss function; in a case where the second loss value is less than the first loss value, the N second pruning numbers are determined as the N target pruning numbers; in a case where the second loss value is greater than or equal to the first loss value, the N first pruning numbers are determined as the N target pruning numbers; wherein the first pruning model is a model obtained by pruning the to-be-pruned deep learning model based on the N second pruning numbers, and the second pruning model is a model obtained by pruning the to-be-pruned deep learning model based on the adjusted N second pruning numbers.

7. An electronic device, comprising: A computer program product comprising a computer readable medium having stored thereon a computer program or instructions that, when executed by a processor, implement the steps of the method of any of claims 1-6.

8. A computer-readable storage medium, characterized in that, A medium having stored thereon a program or instructions that, when executed, implement the steps of the method of any of claims 1-6.

9. A computer program product, characterised in that, A computer program that, when executed by a processor, implements the method of any of claims 1-6.

Images