A model pruning method and apparatus
By obtaining the gradient values and computational cost reduction values of network layers, the channels to be pruned are determined, which solves the problem of poor versatility of existing methods and achieves more efficient model pruning and is applicable to terminal devices with limited computing resources.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- JILUO TECH (SHANGHAI) CO LTD
- Filing Date
- 2022-11-16
- Publication Date
- 2026-07-17
Smart Images
Figure CN115688911B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of model compression technology, and in particular to a model pruning method and apparatus. Background Technology
[0002] With the development of artificial intelligence technology, the application of neural network models is becoming increasingly widespread. Considering the large number of parameters and computational demands of network models, pruning is necessary to improve their processing speed. Pruning, or compressing the network model, aims to reduce model size, lower resource consumption, and improve response time. In existing model pruning processes, it is necessary to determine the channels to be pruned; this determination process is called sensitivity analysis. Existing sensitivity analysis methods are all heuristic, such as determining the channels to be pruned based on the L1 Norm values of the weights. Heuristic methods are based on weight characteristics, which are easier to implement, but they cannot reflect the changes in the model before and after pruning, resulting in poor generality. Summary of the Invention
[0003] This invention provides a model pruning method and apparatus to solve the above-mentioned problems.
[0004] This invention provides a model pruning method, comprising:
[0005] Obtain the first parameter information of each network layer in the model to be pruned, where the first parameter information is the original parameter information;
[0006] The gradient value of the output channel of each network layer is obtained based on the first parameter information;
[0007] Based on the sample image information and the first parameter information, the computational cost reduction value of each network layer output channel is obtained. The computational cost reduction value is the computational cost reduction value after channel pruning of the network layer.
[0008] The channels to be pruned are determined based on the gradient value and the decrease in computational cost, and the model to be pruned is pruned according to the channels to be pruned to obtain the target model.
[0009] According to a model pruning method provided by the present invention, after obtaining the first parameter information of each network layer in the model to be pruned, the method further includes:
[0010] Obtain the mask information corresponding to the first parameter information;
[0011] Accordingly, obtaining the gradient value of the output channel of each network layer based on the first parameter information includes:
[0012] The second parameter information is obtained based on the mask information and the first parameter information, wherein the second parameter information is the parameter information pruned based on the mask information;
[0013] The gradient is calculated based on the second parameter information and the first parameter information to obtain the gradient value.
[0014] According to a model pruning method provided by the present invention, the first parameter information includes a first weight and a first bias, and the second parameter information includes a second weight and a second bias;
[0015] Accordingly, obtaining the second parameter information based on the mask information and the first parameter information includes:
[0016] Calculate the product between the first weight and the mask value in the mask information to obtain the second weight;
[0017] Calculate the product between the first bias and the mask value in the mask information to obtain the second bias;
[0018] The step of calculating the gradient based on the second parameter information and the first parameter information to obtain the gradient value includes:
[0019] The gradient of the second weight is calculated to obtain the weight gradient, and the gradient of the second bias is calculated to obtain the bias gradient.
[0020] Calculate the product of the weight gradient and the first weight, and the product of the bias gradient and the first bias. Then sum the product of the weight gradient and the first weight and the product of the bias gradient and the first bias, and use the summation result as the gradient value.
[0021] According to a model pruning method provided by the present invention, the sample image information includes the number of sample images, the width and height of the sample images, and the first parameter information includes the number of input channels of the network layer and the size of the convolution kernel;
[0022] The step of obtaining the computational cost reduction value of each network layer output channel based on the sample image information and the first parameter information includes:
[0023] The computational cost reduction is calculated by multiplying the number of sample images, the width and height of the sample images, the number of input channels of the network layer, and the size of the convolution kernel. This computational cost reduction represents the decrease in computational cost of the model to be pruned after pruning one channel in the network layer.
[0024] According to a model pruning method provided by the present invention, the step of determining the channel to be pruned based on the gradient value and the computational cost reduction value includes:
[0025] Obtain the ratio between the gradient value and the squared result of the decrease in computational cost;
[0026] Sort the ratios corresponding to all network layers in ascending order, determine the top p ratios from the sorting results, and identify the channels corresponding to the top p ratios as channels to be pruned.
[0027] According to the model pruning method provided by the present invention, after obtaining the gradient value and the decrease in computational cost, the method further includes:
[0028] The gradient value and the decrease in computational cost are normalized to obtain the normalized gradient value and the normalized decrease in computational cost.
[0029] Accordingly, the step of multiplying the gradient value and the computational decrease value includes:
[0030] The normalized gradient value is multiplied by the normalized decrease in computational cost.
[0031] According to a model pruning method provided by the present invention, the step of pruning the model to be pruned according to the pruning channel to obtain a target model includes:
[0032] The model to be pruned is pruned according to the pruning channel to obtain the pruned model;
[0033] If the computational cost of the pruned model exceeds the preset target computational cost, the pruned model will be output as the target model.
[0034] The present invention also provides a model pruning device, comprising:
[0035] The parameter information acquisition module is used to acquire the first parameter information of each network layer in the model to be pruned, where the first parameter information is the original parameter information.
[0036] The channel gradient acquisition module is used to acquire the gradient value of the output channel of each network layer based on the first parameter information.
[0037] The computational cost reduction acquisition module is used to acquire the computational cost reduction value of each network layer output channel based on the sample image information and the first parameter information. The computational cost reduction value is the computational cost reduction value after the network layer has undergone channel pruning.
[0038] The pruning channel determination module is used to determine the pruning channel based on the gradient value and the computational cost reduction value, and to prune the model to be pruned based on the pruning channel to obtain the target model.
[0039] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement any of the model pruning methods described above.
[0040] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements any of the model pruning methods described above.
[0041] The model pruning method and apparatus provided by this invention determine the channels to be pruned based on the gradient value and the decrease in computational cost. The gradient value corresponding to the number of channels reflects the sensitivity of channel pruning. Compared to heuristic sensitivity analysis methods, the gradient-based method of this invention is more versatile. Furthermore, in addition to the gradient value, it also incorporates the decrease in computational cost, thereby prioritizing the pruning of channels with low sensitivity and high computational cost, thus accelerating the pruning efficiency. Attached Figure Description
[0042] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.
[0043] Figure 1 This is a schematic flowchart of the model pruning method provided in an embodiment of the present invention;
[0044] Figure 2 This is a schematic diagram of the model pruning device provided in an embodiment of the present invention;
[0045] Figure 3 This is a schematic diagram of the physical structure of an electronic device provided in an embodiment of the present invention. Detailed Implementation
[0046] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.
[0047] Figure 1 This is a flowchart illustrating the model pruning method provided in an embodiment of the present invention; as shown below. Figure 1 As shown, the pruning method of this model includes:
[0048] S101, Obtain the first parameter information of each network layer in the model to be pruned. The first parameter information is the original parameter information.
[0049] In this step, we first obtain the original neural network model (which is a conventional neural network model) that has been pre-trained using training image data. We then use the original neural network model as the model to be pruned and obtain all network parameters (i.e., the first parameter information) in the pruned model. The network parameters include: the weights and biases in each network layer, the number of input channels, the number of output channels, the size of the convolution kernel, and other information.
[0050] S102, obtain the gradient value of the output channel of each network layer according to the first parameter information.
[0051] In this step, gradient calculations are performed on each output channel of each network layer based on the first parameter information to obtain the gradient value corresponding to each output channel in each network layer. The larger the gradient value, the higher the sensitivity and the more important the output channel; the smaller the gradient value, the lower the sensitivity and the less important the output channel, which can be used as a candidate channel to be pruned.
[0052] S103, based on the sample image information and the first parameter information, obtain the computational cost reduction value for each network layer's output channel. The computational cost reduction value is the reduction value after channel pruning of the network layer.
[0053] In this step, we specify which channel in the network layer will be pruned, and then calculate the reduction in computational cost of the model to be pruned after pruning that channel. The specific channel to be pruned is not limited; regardless of which channel in the same network layer is pruned, the reduction in computational cost for that network layer will be the same.
[0054] S104, determine the pruning channel based on the gradient value and the decrease in computational cost, and prune the model to be pruned based on the pruning channel to obtain the target model.
[0055] In this step, the gradient value reflects the sensitivity of channel pruning and shows the changes in the model before and after pruning. The decrease in computational cost reflects the pruning cost. By combining the above two dimensions of information, the channels to be pruned are determined. This allows for priority pruning of channels with lower sensitivity and higher computational cost, thereby obtaining a target model with better pruning effect.
[0056] After obtaining the target model, it can be ported to a terminal with high computational requirements for forward inference. For example, the pruned point cloud 3D detection model can be applied to the vehicle for real-time detection of external targets.
[0057] The model pruning method provided in this invention determines the channels to be pruned based on the gradient value and the decrease in computational cost. It uses the gradient value corresponding to the number of channels to reflect the sensitivity of channel pruning. Compared to heuristic sensitivity analysis methods, the gradient-based method of this invention is more versatile. Furthermore, in addition to the gradient value, it also incorporates the decrease in computational cost, thus prioritizing the pruning of channels with low sensitivity and high computational cost, thereby accelerating the pruning efficiency.
[0058] In some embodiments of the present invention, after obtaining the first parameter information of each network layer in the model to be pruned, the method further includes:
[0059] Obtain the mask information corresponding to the first parameter information.
[0060] Accordingly, obtaining the gradient value of the output channel of each network layer based on the first parameter information includes:
[0061] The second parameter information is obtained based on the mask information and the first parameter information, wherein the second parameter information is the parameter information pruned based on the mask information.
[0062] The gradient is calculated based on the second parameter information and the first parameter information to obtain the gradient value.
[0063] In this embodiment, the first parameter information includes the weights and biases of each network layer. Each weight or bias corresponds to a mask, which has a value of 1 or 0. When the mask value is 0, the corresponding weight or bias is reset to 0; when the mask value is 1, the corresponding weight or bias remains unchanged. During the first pruning of the model to be pruned, all masks are initialized to 1. In subsequent pruning processes, the mask information is determined based on the pruning status of the model's network parameters after pruning. That is, the mask corresponding to the weights and biases of the determined channels to be pruned is set to 0, and the mask corresponding to the remaining weights and biases is set to 1.
[0064] After obtaining the mask information, the second parameter information is determined by the product of the mask value and the corresponding weight or bias, which is the weight and bias value of each network layer in the pruned model.
[0065] The gradient value of each output channel in the network layer is calculated based on the network parameters after pruning (i.e., the first parameter information) and the network parameters before pruning (i.e., the second parameter information). This gradient value can reflect the pruning sensitivity of the channel.
[0066] The model pruning method provided in this invention determines the pruning sensitivity of a channel by calculating the gradient value of the output channel, which is more versatile than the existing method of determining pruning sensitivity based on weight features.
[0067] In some embodiments of the present invention, the first parameter information includes a first weight and a first bias, and the second parameter information includes a second weight and a second bias.
[0068] Accordingly, obtaining the second parameter information based on the mask information and the first parameter information includes:
[0069] Calculate the product between the first weight and the mask value in the mask information to obtain the second weight, i.e., weight' = weight * mask.
[0070] Calculate the product between the first bias and the mask value in the mask information to obtain the second bias, i.e., bias' = bias * mask.
[0071] The step of calculating the gradient based on the second parameter information and the first parameter information to obtain the gradient value includes:
[0072] The gradient of the second weight is calculated to obtain the weight gradient, and the gradient of the second bias is calculated to obtain the bias gradient.
[0073] Calculate the product of the weight gradient and the first weight, and the product of the bias gradient and the first bias. Then sum the product of the weight gradient and the first weight with the product of the bias gradient and the first bias, and use the sum as the gradient value. That is, in, The dimensional information is (c o The dimension information of the weight is (c o c i ,k,k),c o c is the number of output channels. i is the number of input channels, and k*k is the kernel size. For the weight gradient, For the bias gradient, The gradient of the mask.
[0074] The model pruning method provided in this embodiment of the invention analyzes the pruning sensitivity by performing gradient calculation on the output channel, thereby improving the versatility of the pruning method.
[0075] In some embodiments of the present invention, the sample image information includes the number of sample images n, the width w of the sample images, and the height h, and the first parameter information includes the number of input channels c of the network layer. i And the size of the convolution kernel, k*k. Here, the number of sample images refers to the batch size of the training data during the training process.
[0076] The step of obtaining the computational cost reduction value of each network layer output channel based on the sample image information and the first parameter information includes:
[0077] Based on the number of sample images n, the width w and height h of the sample images, and the number of input channels c of the network layer. i The computational cost is reduced by multiplying the kernel size k*k by the kernel size, i.e., ΔFLOPs = n*h*w*c. i *k*k. The above computational cost reduction values represent the reduction in computational cost of the model to be pruned after pruning one channel in a network layer. Since the final determination is the channel to be pruned, and the computational cost reduction after pruning any channel in each network layer is consistent, we assume that any output channel of each network layer is pruned, and then calculate the computational cost reduction of all network layers after output channel pruning under this assumption.
[0078] The model pruning method provided in this invention determines the pruning cost by obtaining the reduction in computational cost after channel pruning, thereby helping to improve pruning efficiency.
[0079] In some embodiments of the present invention, determining the pruning channel based on the gradient value and the decrease in computational cost includes:
[0080] The ratio between the gradient value and the square of the decrease in computational cost is obtained, specifically:
[0081] Sort the ratios corresponding to all network layers in ascending order, determine the top p ratios from the sorting results, and identify the channels corresponding to the top p ratios as channels to be pruned.
[0082] As can be seen from the above, The dimension is c o That is, each output channel of each network layer corresponds to an application. Value, for all Value multiplied by ΔFLOPs -2 , that is Value divided by ΔFLOPs 2 Then, obtain the output channels corresponding to each output channel in each network layer of the entire model to be pruned. The values are then sorted from smallest to largest, and the p smallest ratios (p is set based on experience) are selected. The corresponding network layer and channel index are determined based on the selected p ratios. The relevant channels are then pruned based on the determined network layer and channel index, that is, the mask corresponding to each weight and bias in the channel to be pruned is set to 0.
[0083] The model pruning method provided in this embodiment of the invention obtains p channels to be pruned based on the product between the gradient value and the decrease in computational cost, so as to prune channels with lower sensitivity and higher computational cost, thereby improving pruning efficiency.
[0084] In some embodiments of the present invention, after obtaining the gradient value and the decrease in computational cost, the method further includes:
[0085] For the gradient value The computational cost reduction value ΔFLOPs is normalized to obtain the normalized gradient value and the normalized computational cost reduction value.
[0086] Accordingly, the step of multiplying the gradient value and the computational decrease value includes:
[0087] The normalized gradient value is multiplied by the normalized decrease in computational cost.
[0088] In this embodiment, for Both ΔFLOPs and ΔFLOPs are normalized to their maximum and minimum values, so that the normalized values are... Both the normalized ΔFLOPs and the normalized ΔFLOPs are between 0 and 1, thus reducing the amount of subsequent calculations.
[0089] In some embodiments of the present invention, the step of pruning the model to be pruned according to the pruning channel to obtain the target model includes:
[0090] The model to be pruned is pruned according to the pruning channel to obtain the pruned model.
[0091] If the computational cost of the pruned model exceeds the preset target computational cost, the pruned model will be output as the target model.
[0092] In this embodiment, since a single channel pruning operation cannot yield a target model that can be directly ported to a terminal with limited computing power, a target computational cost (set according to requirements) needs to be set as the cutoff condition for the pruning process. The specific implementation process is as follows: after determining the channels to be pruned, the model is pruned, and the computational cost of the pruned model is calculated. If the calculated result exceeds the target computational cost, pruning stops; if the calculated result does not exceed the target computational cost, a new pruning sensitivity analysis is performed based on the channel gradient to determine the channels to be pruned, thus achieving another round of model pruning.
[0093] The model pruning method provided in this embodiment of the invention obtains the target model by setting a target computational load, so as to meet the application requirements of the model.
[0094] In some embodiments of the present invention, the model pruning method includes the following steps:
[0095] S201, Obtain the neural network model, and multiply the weights and biases of each layer by a mask. Here, the mask is initialized to 1.
[0096] S202, at preset intervals, analyze the pruning sensitivity of each channel in the neural network model, that is, calculate the gradient value corresponding to the output channel of each layer of the network. And calculate the computational cost reduction ΔFLOPs after pruning for each layer's output channel; fusion The ΔFLOPs are used to determine the channels to be pruned. It should be noted that the preset number of training cycles refers to the condition that the original neural network model's training cycles (iter) are divisible by the preset cycle (iter_interval) (i.e., iter % iter_interval == 0). When the training cycles (iter) are not divisible by the preset cycle (iter_interval), the parameters of the pruned model are fine-tuned using sample image data from the neural network model's training process.
[0097] S203: Calculate the computational cost of the pruned model. If the computational cost exceeds the target computational cost, stop pruning and output the pruned model as the target model. If the computational cost does not exceed the target computational cost, continue fine-tuning and pruning, repeating the process until the target computational cost is met, and obtain the final target model.
[0098] The model pruning method provided in this embodiment of the invention combines gradient value and computational cost reduction value, thereby prioritizing the pruning of channels with low sensitivity and high computational cost, accelerating pruning efficiency, and improving the versatility of model pruning.
[0099] The model pruning device provided by the present invention is described below. The model pruning device described below can be referred to in correspondence with the model pruning method described above.
[0100] Figure 2 This is a schematic diagram of the model pruning device provided in an embodiment of the present invention, as shown below. Figure 2 As shown, the model pruning device includes a parameter information acquisition module 201, a channel gradient acquisition module 202, a computational cost reduction acquisition module 203, and a pruning channel determination module 204.
[0101] The parameter information acquisition module 201 is used to acquire the first parameter information of each network layer in the model to be pruned. The first parameter information is the original parameter information.
[0102] In this module, we first obtain the original neural network model (which is a conventional neural network model) that has been pre-trained using training data. We then use this original neural network model as the model to be pruned and obtain all network parameters (i.e., the first parameter information) in the pruned model. The network parameters include: the weights and biases in each network layer, the number of input channels, the number of output channels, the size of the convolution kernel, and other information.
[0103] The channel gradient acquisition module 202 is used to acquire the gradient value of the output channel of each network layer based on the first parameter information.
[0104] In this module, gradient calculations are performed on each output channel of each network layer based on the first parameter information to obtain the gradient value corresponding to each output channel in each network layer. The larger the gradient value, the higher the sensitivity and the more important the output channel; the smaller the gradient value, the lower the sensitivity and the less important the output channel, which can be used as a candidate channel to be pruned.
[0105] The computational cost reduction acquisition module 203 is used to acquire the computational cost reduction value of each network layer's output channel based on the sample image information and the first parameter information. The computational cost reduction value is the computational cost reduction value after channel pruning of the network layer.
[0106] In this module, if a specific channel in a network layer is pruned, the computational cost reduction of the model to be pruned after pruning that channel is calculated. The specific channel to be pruned is not limited; regardless of which channel within the same network layer is pruned, the computational cost reduction for that network layer will be the same.
[0107] The pruning channel determination module 204 is used to determine the pruning channel based on the gradient value and the computational cost reduction value, and to prune the model to be pruned based on the pruning channel to obtain the target model.
[0108] In this module, gradient values reflect the sensitivity of channel pruning and show the changes in the model before and after pruning. The decrease in computational cost reflects the pruning cost. By combining the above two dimensions of information, the channels to be pruned are determined, so that channels with lower sensitivity and higher computational cost can be pruned first, thereby obtaining a target model with better pruning effect.
[0109] After obtaining the target model, it can be ported to a terminal with high computational requirements for forward inference. For example, the pruned point cloud 3D detection model can be applied to the vehicle for real-time detection of external targets.
[0110] The model pruning device provided in this invention determines the channels to be pruned based on the gradient value and the decrease in computational cost. It uses the gradient value corresponding to the number of channels to reflect the sensitivity of channel pruning. Compared to heuristic sensitivity analysis methods, the gradient-based method of this invention is more versatile. Furthermore, in addition to the gradient value, it also incorporates the decrease in computational cost, thereby prioritizing the pruning of channels with low sensitivity and high computational cost, thus accelerating the pruning efficiency.
[0111] Figure 3 This is a schematic diagram of the physical structure of an electronic device provided in an embodiment of the present invention, such as... Figure 3 As shown, the electronic device may include a processor 310, a communication interface 320, a memory 330, and a communication bus 340, wherein the processor 310, the communication interface 320, and the memory 330 communicate with each other through the communication bus 340. The processor 310 can call logical instructions in the memory 330 to execute a model pruning method. The model pruning method includes: obtaining first parameter information of each network layer in the model to be pruned, wherein the first parameter information is the original parameter information; obtaining the gradient value of the output channel of each network layer according to the first parameter information; obtaining the computational cost reduction value of the output channel of each network layer according to the sample image information and the first parameter information, wherein the computational cost reduction value is the computational cost reduction value after channel pruning of the network layer; determining the channel to be pruned according to the gradient value and the computational cost reduction value, and pruning the model to be pruned according to the channel to be pruned to obtain the target model.
[0112] Furthermore, the logical instructions in the aforementioned memory 330 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.
[0113] In another aspect, the present invention also provides a non-transitory computer-readable storage medium storing a computer program thereon. When executed by a processor, the computer program implements a model pruning method, which includes: acquiring first parameter information of each network layer in the model to be pruned, wherein the first parameter information is original parameter information; acquiring gradient values of the output channels of each network layer based on the first parameter information; acquiring a computational cost reduction value of the output channels of each network layer based on sample image information and the first parameter information, wherein the computational cost reduction value is the computational cost reduction value after channel pruning of the network layer; determining the channels to be pruned based on the gradient values and the computational cost reduction value; and pruning the model to be pruned based on the channels to be pruned to obtain a target model.
[0114] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.
[0115] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., including several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods of various embodiments or some parts of embodiments.
[0116] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.
Claims
1. A model pruning method, characterized in that, include: Obtain the first parameter information of each network layer in the model to be pruned, where the first parameter information is the original parameter information; The gradient value of the output channel of each network layer is obtained based on the first parameter information; Based on the sample image information and the first parameter information, the computational cost reduction value of each network layer output channel is obtained. The computational cost reduction value is the computational cost reduction value after channel pruning of the network layer. Obtain the ratio between the gradient value and the squared result of the decrease in computational cost; Sort the ratios corresponding to all network layers in ascending order, determine the first p ratios from the sorting results, and identify the channels corresponding to the first p ratios as channels to be pruned. The model to be pruned is pruned according to the pruning channel to obtain the target model.
2. The model pruning method according to claim 1, characterized in that, After obtaining the first parameter information of each network layer in the model to be pruned, the method further includes: Obtain the mask information corresponding to the first parameter information; Accordingly, obtaining the gradient value of the output channel of each network layer based on the first parameter information includes: The second parameter information is obtained based on the mask information and the first parameter information, wherein the second parameter information is the parameter information pruned based on the mask information; The gradient is calculated based on the second parameter information and the first parameter information to obtain the gradient value.
3. The model pruning method according to claim 2, characterized in that, The first parameter information includes a first weight and a first bias, and the second parameter information includes a second weight and a second bias; Accordingly, obtaining the second parameter information based on the mask information and the first parameter information includes: Calculate the product between the first weight and the mask value in the mask information to obtain the second weight; Calculate the product between the first bias and the mask value in the mask information to obtain the second bias; The step of calculating the gradient based on the second parameter information and the first parameter information to obtain the gradient value includes: The gradient of the second weight is calculated to obtain the weight gradient, and the gradient of the second bias is calculated to obtain the bias gradient. Calculate the product of the weight gradient and the first weight, and the product of the bias gradient and the first bias. Then sum the product of the weight gradient and the first weight and the product of the bias gradient and the first bias, and use the summation result as the gradient value.
4. The model pruning method according to claim 1, characterized in that, The sample image information includes the number of sample images, the width and height of the sample images, and the first parameter information includes the number of input channels of the network layer and the size of the convolution kernel; The step of obtaining the computational cost reduction value of each network layer output channel based on the sample image information and the first parameter information includes: The computational cost reduction is calculated by multiplying the number of sample images, the width and height of the sample images, the number of input channels of the network layer, and the size of the convolution kernel. This computational cost reduction represents the decrease in computational cost of the model to be pruned after pruning one channel in the network layer.
5. The model pruning method according to claim 1, characterized in that, After obtaining the gradient value and the decrease in computational cost, the method also includes: The gradient value and the decrease in computational cost are normalized to obtain the normalized gradient value and the normalized decrease in computational cost. The normalized gradient value is multiplied by the normalized decrease in computational cost.
6. The model pruning method according to any one of claims 1-5, characterized in that, The step of pruning the model to be pruned according to the pruning channel to obtain the target model includes: The model to be pruned is pruned according to the pruning channel to obtain the pruned model; If the computational cost of the pruned model exceeds the preset target computational cost, the pruned model will be output as the target model.
7. A model pruning device, characterized in that, include: The parameter information acquisition module is used to acquire the first parameter information of each network layer in the model to be pruned, where the first parameter information is the original parameter information. The channel gradient acquisition module is used to acquire the gradient value of the output channel of each network layer based on the first parameter information. The computational cost reduction acquisition module is used to acquire the computational cost reduction value of each network layer output channel based on the sample image information and the first parameter information. The computational cost reduction value is the computational cost reduction value after the network layer has undergone channel pruning. The pruning channel determination module is used for: Obtain the ratio between the gradient value and the squared result of the decrease in computational cost; Sort the ratios corresponding to all network layers in ascending order, determine the first p ratios from the sorting results, and identify the channels corresponding to the first p ratios as channels to be pruned. The model to be pruned is pruned according to the pruning channel to obtain the target model.
8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the model pruning method as described in any one of claims 1 to 6.
9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the model pruning method as described in any one of claims 1 to 6.