Model pruning method and device, electronic equipment and storage medium
By acquiring and utilizing the importance of network layers in the model to be pruned for widening and pruning, the problem of low accuracy in existing structured pruning methods is solved, and the accuracy of models deployed efficiently on edge devices is improved.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- 伟光有限公司(CN)
- Filing Date
- 2022-12-20
- Publication Date
- 2026-05-01
AI Technical Summary
Existing structured pruning methods do not produce high accuracy in the pruned model, which means that some accuracy needs to be sacrificed when deploying small models in practical applications.
By obtaining the importance of each network layer in the model to be pruned, the network layers to be widened and the reference network layers are determined. The network layers to be widened are then widened, and pruning is performed based on a preset sparsity to obtain a target model with higher accuracy.
The accuracy of the pruned model has been improved, enabling it to maintain high performance when deployed on resource-constrained edge devices.
Smart Images
Figure CN115983367B_ABST
Abstract
Description
Model pruning methods, devices, electronic equipment and storage media Technical Field
[0001] This application belongs to the field of electronic technology, specifically relating to a model pruning method, apparatus, electronic device, and storage medium. Background Technology
[0002] Pruning algorithms can be divided into structured pruning and unstructured pruning. Structured pruning targets a specific layer of the network, removing redundant output channels to reduce the number of parameters. Unstructured pruning, on the other hand, does not completely delete an output channel but sets some elements to zero. While unstructured pruning can more accurately locate redundant parameters, structured pruning enjoys better hardware support and is easier to implement on hardware devices, thus finding wider application in practical tasks. However, the accuracy of the pruned model still needs improvement. Summary of the Invention
[0003] In view of the above problems, this application proposes a model pruning method, apparatus, electronic device, and storage medium to improve the above problems.
[0004] In a first aspect, embodiments of this application provide a model pruning method, the method comprising: obtaining the importance of each of multiple network layers of a model to be pruned; determining, based on the importance of each of the multiple network layers, a network layer to be widened and a reference network layer of the model to be pruned, wherein the reference network layer is a network layer other than the network layer to be widened among the multiple network layers; widening the network layer to be widened in the model to be pruned to obtain a widened model to be pruned including the widened network layer and the reference network layer; and pruning the widened model to be pruned based on a preset sparsity to obtain a target model.
[0005] Secondly, embodiments of this application provide a model pruning apparatus, the apparatus comprising: an importance acquisition unit, configured to acquire the importance of each of multiple network layers of a model to be pruned; a network layer determination unit, configured to determine, based on the importance of each of the multiple network layers, a network layer to be widened and a reference network layer of the model to be pruned, wherein the reference network layer is a network layer other than the network layer to be widened among the multiple network layers; a widening unit, configured to widen the network layer to be widened in the model to be pruned, to obtain a widened model to be pruned including the widened network layer and the reference network layer; and a pruning unit, configured to prune the widened model to be pruned based on a preset sparsity, to obtain a target model.
[0006] Thirdly, embodiments of this application provide an electronic device, including one or more processors and a memory; one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, and the one or more programs are configured to perform the methods described above.
[0007] Fourthly, embodiments of this application provide a computer-readable storage medium storing program code, wherein the above-described method is executed when the program code is run.
[0008] This application provides a model pruning method, apparatus, electronic device, and storage medium. First, the importance of each network layer in the model to be pruned is obtained. Then, based on the importance of each network layer, the network layers to be widened and reference network layers are determined. The reference network layers are all network layers except the ones to be widened. The network layers to be widened are then widened to obtain a widened pruning model including the widened network layers and the reference network layers. Finally, the widened pruning model is pruned based on a preset sparsity to obtain the target model. By using this method, the network layers to be widened are determined according to the importance of each network layer in the model. By widening the determined high-importance network layers to obtain a widened model, and then pruning the widened model, a more accurate target model can be obtained. Attached Figure Description
[0009] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0010] Figure 1 shows a schematic diagram of an application scenario of a model pruning method proposed in an embodiment of this application;
[0011] Figure 2 shows a schematic diagram of an application scenario of a model pruning method proposed in an embodiment of this application;
[0012] Figure 3 shows a flowchart of a model pruning method proposed in an embodiment of this application;
[0013] Figure 4 shows a flowchart of a model pruning method proposed in another embodiment of this application;
[0014] Figure 5 shows a flowchart of a model pruning method according to another embodiment of this application;
[0015] Figure 6 shows a structural block diagram of a model pruning device proposed in an embodiment of this application;
[0016] Figure 7 shows a structural block diagram of an electronic device or server for performing a model pruning method according to an embodiment of the present application.
[0017] Figure 8 shows a storage unit in an embodiment of this application for storing or carrying program code that implements the model pruning method according to an embodiment of this application. Detailed Implementation
[0018] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.
[0019] Deep learning technology based on convolutional neural networks has flourished in recent years, achieving remarkable results in many vision and language-related tasks, some even surpassing human performance. However, these outstanding achievements largely depend on the complexity of the network structure and the large number of parameters, which in turn places higher demands on computing power and memory. For some edge devices (such as mobile phones and cameras), computing power and memory are severely limited, which restricts the deployment and application of deep convolutional neural networks on these devices.
[0020] To address these issues, academia and industry have proposed a series of model compression methods, such as quantization, pruning, and distillation. Among these, pruning algorithms can significantly reduce the number of parameters in a network by removing redundant parameters, thus "slimming down" deep neural networks and enabling their deployment on resource-constrained edge devices.
[0021] Pruning algorithms can be divided into structured pruning and unstructured pruning. Structured pruning targets a specific layer of the network, removing redundant output channels to reduce the number of parameters. Unstructured pruning, on the other hand, does not completely delete an output channel, but rather sets some of its elements to zero. While unstructured pruning can more precisely locate redundant parameters, structured pruning enjoys better hardware support and is easier to implement on hardware devices, thus finding wider application in practical tasks.
[0022] However, the inventors, in their research on related model pruning methods, discovered that existing structured pruning methods typically set a metric—an evaluation index—to determine the importance of channels in the network. Generally, a higher metric indicates a more important channel, while a lower metric indicates a less important channel. Then, the channel importance is ranked, and a threshold for deleting channels is calculated based on the user's desired sparsity. Channels with a metric below this threshold are deleted, thus completing the pruning process. Existing metric-based structured pruning methods generally result in smaller, less accurate models after pruning. Even with a certain amount of fine-tuning, the accuracy still lags behind the larger, unpruned model. Therefore, in practical applications, deploying smaller models often comes at the cost of sacrificing some accuracy.
[0023] Therefore, the inventors have proposed the model pruning method, apparatus, electronic device, and storage medium of this application. First, the importance of each network layer in the model to be pruned is obtained. Then, based on the importance of each network layer, the network layers to be widened and reference network layers of the model to be pruned are determined. The reference network layers are all network layers except the network layers to be widened. The network layers to be widened are then widened to obtain a widened model to be pruned, including the widened network layers and the reference network layers. Finally, the widened model to be pruned is pruned based on a preset sparsity to obtain the target model. By using the above method, the network layers to be widened are determined according to the importance of each network layer in the model. By widening the determined high-importance network layers to be widened to obtain a widened model, and then pruning the widened model, a more accurate target model can be obtained.
[0024] In this embodiment of the application, the provided model pruning method can be executed by an electronic device. In this mode of execution by an electronic device, all steps in the model pruning method provided in this embodiment of the application can be performed by the electronic device. For example, as shown in FIG1, the processor of the electronic device 100 can obtain the importance of each of the multiple network layers of the model to be pruned in real time; based on the importance of each of the multiple network layers, the network layer to be widened and the reference network layer of the model to be pruned are determined, wherein the reference network layer is the network layer other than the network layer to be widened among the multiple network layers; the network layer to be widened of the model to be pruned is widened to obtain a widened model to be pruned including the widened network layer and the reference network layer; the widened model to be pruned is pruned based on a preset sparsity to obtain the target model.
[0025] Furthermore, the model pruning method provided in this application embodiment can also be executed by a server (cloud). Correspondingly, in this server-executed method, the server can obtain the importance of each of the multiple network layers of the model to be pruned in real time; based on the importance of each of the multiple network layers, determine the network layer to be widened and the reference network layer of the model to be pruned, where the reference network layer is the network layer other than the network layer to be widened; widen the network layer to be widened in the model to be pruned to obtain a widened model to be pruned including the widened network layer and the reference network layer; prune the widened model to be pruned based on a preset sparsity to obtain the target model.
[0026] Alternatively, the process can be performed collaboratively by electronic devices and a server. In this collaborative approach, some steps of the model pruning method provided in this application embodiment are performed by electronic devices, while other steps are performed by the server.
[0027] For example, as shown in Figure 2, the electronic device 100 can perform a model pruning method including: obtaining the importance of each of the multiple network layers of the model to be pruned; determining the network layer to be widened and the reference network layer of the model to be pruned based on the importance of each of the multiple network layers, wherein the reference network layer is a network layer other than the network layer to be widened among the multiple network layers; then, the server 200 performs widening processing on the network layer to be widened of the model to be pruned to obtain a widened model to be pruned including the widened network layer and the reference network layer; and pruning the widened model to be pruned based on a preset sparsity to obtain the target model.
[0028] It should be noted that in this method where electronic devices and servers work together, the steps performed by the electronic devices and servers are not limited to those described in the examples above. In practical applications, the steps performed by the electronic devices and servers can be dynamically adjusted according to the actual situation.
[0029] It should be noted that the electronic device 100 can be, in addition to the smartphones shown in Figures 1 and 2, in-vehicle infotainment systems, wearable devices, tablets, laptops, smart speakers, etc. The server 200 can be a standalone physical server, a server cluster consisting of multiple physical servers, or a distributed system.
[0030] The embodiments of this application will now be described in detail with reference to the accompanying drawings.
[0031] Please refer to Figure 3. An embodiment of this application provides a model pruning method, applied to an electronic device or server as shown in Figure 1 or Figure 2. The method includes:
[0032] Step S110: Obtain the importance of each of the multiple network layers in the model to be pruned.
[0033] In this embodiment, the model to be pruned can be a pre-trained neural network model based on training data, such as a convolutional neural network model, a fully connected neural network model, a residual neural network model, and a generative adversarial neural network model, etc., without specific limitations. The model to be pruned may include multiple network layers, and the types of network layers may include input layers, convolutional layers, pooling layers, recurrent layers, fully connected layers, and output layers, etc., without specific limitations. For example, the multiple network layers of the model to be pruned may include an input layer, multiple convolutional layers, pooling layers, multiple recurrent layers, fully connected layers, and an output layer, etc.
[0034] As one approach, the process of training a model to be pruned based on training data includes: acquiring training data; iteratively training the model to be trained based on the training data until the training termination condition is met, thereby obtaining the model to be pruned.
[0035] The training data can be defined as D = {x = {x0, x1, ..., x...} N}, y={y0,y1,...,y N}}, where x and y represent the input data and the corresponding output data, respectively. In the embodiments of this application, the training data can be determined according to the function of the model. For example, if the model is used to identify objects in an image, then the training data can be images including the target objects; if the model is used to predict human key points, then the training data can be images including human key points, etc.
[0036] The training termination condition can be a pre-set condition representing the end of training. For example, the training termination condition can be set to the number of training iterations meeting a preset number; or the training termination condition can be the loss function value meeting a preset loss function value; or the training termination condition can also be the network parameters of the model to be trained meeting a preset network parameter. The network parameters of the model to be trained can include the weights, biases, etc., corresponding to each channel in each network layer, and are not specifically limited here.
[0037] In this embodiment of the application, if the model to be trained may include L network layers, the network parameters of the model to be trained can be defined as follows: Where L represents the number of network layers in the model to be trained, and C l This represents the number of output channels of the l-th network layer, 1≤l≤L, C L This indicates the number of output channels in the Lth layer of the network. This represents the Cth layer of the first network layer. l Parameters of each output channel Represents the Cth layer of the Lth network layer. L The parameters of each output channel. Generally, the number of channels C can also be considered as... l Defined as the width of the l-th network layer, C l The larger the value, the wider the l-th layer of the network.
[0038] Furthermore, the loss function can be defined as C(D|W), used to calculate the loss function values for the input and output data. The batch size of the network input to the model to be trained is defined as 1, and the feature map output by the l-th network layer is . Among them, H l The height of the characteristic map, W l The width of the feature map.
[0039] Then, for the k-th channel of the l-th network layer, the weights will be... The convolution (*) is applied to the output z of the previous layer. l-1 The feature map of the k-th channel can be obtained. Right now in, p is the kernel size, and R(*) is the activation function, such as a linear rectified function. This represents the parameter of the k-th output channel of the l-th network layer.
[0040] When iteratively training the model to be trained using training data, training can be terminated when the network parameters of the model to be trained meet the preset network parameters, or when the loss function value calculated by the loss function C(D|W) meets the preset loss function value. No specific limitation is made here.
[0041] In the embodiments of this application, the multiple network layers of the model to be pruned can be some network layers selected from all network layers of the model to be pruned, or they can be all network layers of the model to be pruned.
[0042] The importance of each network layer characterizes its significance to the model to be pruned. Optionally, the importance of each network layer can be calculated based on a measure of each output channel within that layer, where the measure characterizes the significance of each output channel to that layer. One approach is to pre-calculate and store the importance of each network layer, retrieving it from storage when the importance of multiple network layers in the model to be pruned is needed. Alternatively, the importance of each network layer in the model to be pruned can be calculated in real-time.
[0043] Step S120: Based on the importance of each of the multiple network layers, determine the network layer to be widened and the reference network layer of the pruning model. The reference network layer is the network layer other than the network layer to be widened among the multiple network layers.
[0044] In this embodiment, the network layer to be widened is the network layer that needs widening from multiple network layers in the model to be pruned; the reference network layer is the network layer that does not need widening from multiple network layers in the model to be pruned. The network layer to be widened and the reference network layer constitute multiple network layers in the model to be pruned. Widening can be understood as increasing the width of certain network layers in the model to be pruned, i.e., increasing the number of output channels in certain network layers, or it can be understood as modifying the weights of certain network layers in the original model.
[0045] One approach is to, after obtaining the importance of each network layer in the model to be pruned, sort the multiple network layers of the model according to their importance, resulting in a sorted set of network layers. The sorting order can be from highest to lowest importance or from lowest to highest importance; no specific limitation is made here.
[0046] After obtaining multiple sorted network layers, a preset number of network layers or network layers at a specified sorting position can be selected as the network layers that need to be widened, i.e., the network layers to be widened are determined. The network layers in the pruning model other than the determined network layers to be widened are used as reference network layers.
[0047] Step S130: Widen the network layer to be widened in the pruning model to obtain a widened pruning model including a widened network layer and a reference network layer.
[0048] In this embodiment of the application, after determining the network layer to be widened in the above manner, the determined network layer to be widened is widened to obtain the widened network layer, that is, the widened network layer.
[0049] When widening the network layers to be widened in a pruning model, different levels of widening can be applied to different layers, i.e., adding different numbers of output channels to each layer. One approach is to determine the number of output channels added to each layer based on its importance. Specifically, the higher the importance of the layer, the more output channels are added; conversely, the lower the importance, the fewer output channels are added. Alternatively, the widening can be performed according to a preset widening ratio, where the preset widening ratio represents the number of output channels added to each layer.
[0050] Step S140: Prune the widened model to be pruned based on a preset sparsity to obtain the target model.
[0051] In this embodiment, the preset sparsity is a pre-set proportion of output channels to be deleted from the network layers of the widened pruning model. For example, the preset sparsity can be set to 0.2, which means that 20% of the output channels in each network layer are deleted.
[0052] As one approach, the preset sparsity can include more than one sparsity. Specifically, in order to more accurately prune the widened model to be pruned, different sparsities can be set for the widening network layer and the reference network layer in the widened model to be pruned, and then different degrees of pruning can be performed on the widened network layer and the reference network layer based on different sparsities.
[0053] When pruning a widened, pruned model according to a preset sparsity, pruning can be performed on each network layer based on the metric values corresponding to the channels within that layer. Specifically, when pruning each network layer based on the metric values corresponding to the channels within that layer, a preset metric threshold can be obtained. Then, a number of output channels with metric values less than the threshold, corresponding to a preset sparsity, can be deleted. Alternatively, each output channel can be sorted according to its metric value, and the final number of channels after sorting and deleting according to the preset sparsity is not specifically limited here. Deleting output channels can also be understood as setting the weights corresponding to those output channels to 0.
[0054] This application provides a model pruning method. First, it obtains the importance of each network layer in the model to be pruned. Then, based on the importance of each network layer, it determines the network layers to be broadened and reference network layers. The reference network layers are all network layers except the ones to be broadened. Next, it broadens the network layers to be broadened, resulting in a broadened pruned model including the broadened and reference network layers. Finally, it prunes the broadened pruned model based on a preset sparsity to obtain the target model. By using this method, the network layers to be broadened are determined according to the importance of each network layer in the model. By broadening the determined, high-importance network layers to be broadened to obtain a broadened model, and then pruning the broadened model, a more accurate target model can be obtained.
[0055] Please refer to Figure 4. An embodiment of this application provides a model pruning method, applied to an electronic device or server as shown in Figure 1 or Figure 2. The method includes:
[0056] Step S210: Obtain the importance of each of the multiple network layers in the model to be pruned.
[0057] Step S220: Based on the importance of each of the multiple network layers, determine the network layer to be widened and the reference network layer of the pruning model.
[0058] Step S230: Obtain the preset number of widening layers and the preset widening ratio.
[0059] In this embodiment, the preset number of widening layers represents how many network layers in the pruning model need to be widened, and the preset widening ratio represents the proportion of increased output channels for each network layer that needs to be widened. For example, the preset number of widening layers can be defined as N. w (number of layers to be widened), the preset widening ratio is defined as r w (ratio of widening). Among them, Characterizing the nth w A preset widening ratio for each widened network layer, n w This indicates the index for widening the network layer.
[0060] Step S240: Based on the preset number of widening layers and the preset widening ratio, widen the network layers of the model to be widened to obtain the widened pruning model including the widened network layers and the reference network layers.
[0061] In this embodiment, the network layers in the pruning model that need to be widened are determined based on a preset number of widening layers, and the number of output channels that need to be added to each network layer that needs to be widened is determined based on a preset widening ratio.
[0062] Specifically, when determining the network layers in the pruning model that need widening based on a preset number of widening layers, the importance of each network layer and the preset number of widening layers can be used to determine the network layers that need widening. Then, based on the measure of each output channel of each network layer and the preset widening ratio, the number of output channels that need to be added to each network layer that needs widening can be determined. For example, if the multiple network layers of the pruning model are sorted in descending order of importance, the top N... w layer It needs to be widened (i.e., the number of output channels is increased). After widening, the result is... in, The parameters characterizing each output channel in the network layer to be broadened. The parameters characterize each output channel in the widened network layer. Characterizes the number of output channels for each widened network layer. This indicates the number of output channels for the network layer to be widened.
[0063] LN without widening w layer and widened Together, they form a new widened model (i.e., a widened model to be pruned).
[0064] Step S250: Retrain the widened pruning model to obtain the trained widened pruning model.
[0065] In this embodiment, retraining refers to retraining the widened pruning model using the aforementioned training data until the training termination condition is met, resulting in a trained widened pruning model. The N of the widened pruning model obtained after retraining... w New output channel added in the layer The metric may be greater than the original output channel, so further pruning can preserve the more important output channels.
[0066] In the embodiments of this application, after the widened model to be pruned is retrained, the newly added output channels may be more important than the original output channels, so more important channels can be preserved during pruning.
[0067] The process may further include the following steps before step S250:
[0068] Step S251: Obtain the total number of channels for each network layer of the model to be pruned.
[0069] In this embodiment, the total number of channels for each network layer of the model to be pruned refers to the original number of output channels of each network layer before widening.
[0070] Step S252: Based on the first sparsity, determine the number of first channels corresponding to each network layer to be widened and each reference network layer of the model to be pruned, wherein the number of first channels is the number of channels to be deleted corresponding to each network layer to be widened and each reference network layer.
[0071] In this embodiment of the application, before pruning, the user sets a target sparsity s for each network layer of the model to be pruned. l ∈[0,1), and generally, the same sparsity is set for all network layers of the model to be pruned, i.e., {s l =s,l∈[1,L]}. However, after widening, the model to be pruned contains N w The layer increases the number of output channels, therefore, to achieve the original compression level, the number of channels to be deleted needs to be recalculated. For network layers that have not undergone widening... (Referring to the network layer), the number of output channels that need to be deleted remains unchanged. Similarly, the number of channels that originally needed to be deleted for each network layer to be widened was...
[0072] Step S253: Based on the total number of channels and the first number of channels corresponding to each network layer of the model to be pruned, determine the second number of channels corresponding to each network layer, where the second number of channels is the remaining number of channels corresponding to each network layer.
[0073] In this embodiment, the network layer of the model to be pruned includes a network layer to be widened and a reference network layer. Specifically, based on the total number of channels and the first number of channels corresponding to each network layer of the model to be pruned, the second number of channels corresponding to each network layer is determined; that is, based on the total number of channels and the first number of channels corresponding to each network layer to be widened, the second number of channels corresponding to each network layer to be widened is determined. Similarly, based on the total number of channels and the second number of channels corresponding to each reference network layer, the second number of channels corresponding to each reference network layer is determined.
[0074] The number of second channels corresponding to each reference network layer can be calculated using the following formula: This indicates the number of second channels corresponding to the l-th reference network layer.
[0075] The number of second channels corresponding to each network layer to be widened can be calculated using the following formula: Indicates the nth w The number of second channels corresponding to the network layer to be widened.
[0076] Step S254: Obtain the total number of channels for each widened network layer of the widened pruning model.
[0077] In the embodiments of this application, the total number of channels corresponding to each widened network layer can be understood as the number of output channels corresponding to each network layer after widening processing.
[0078] Step S255: Based on the total number of channels corresponding to each widened network layer and the number of second channels corresponding to each network layer to be widened, determine the number of first channels corresponding to each widened network layer of the widened pruning model.
[0079] In this embodiment of the application, for the widened network layer The number of output channels that need to be deleted should be:
[0080] Step S256: Determine the second sparsity based on the total number of channels and the second number of channels corresponding to each broadened network layer of the broadened pruning model.
[0081] In this embodiment, based on the total number of channels and the first number of channels corresponding to each broadened network layer of the broadened pruning model, the second sparsity corresponding to each broadened network layer is determined. Wherein, each broadened network layer... The corresponding second sparsity is Different widened network layers can correspond to different second sparsity.
[0082] Step S260: Prune the reference network layer of the trained widened pruning model based on the first sparsity, and prune the widened network layer of the trained widened pruning model based on the second sparsity to obtain the target model. The first sparsity is a preset sparsity, and the second sparsity is determined based on the number of channels in the widened network layer.
[0083] In this embodiment, the reference network layer in the trained widened pruning model is pruned using a first sparsity, and the widened network layer in the trained widened pruning model is pruned using a calculated second sparsity.
[0084] This application provides a model pruning method that determines the network layers to be broadened according to the importance of each network layer in the model, broadens the network layers with high importance to obtain a broadened model, and then prunes the broadened model to obtain a target model with higher accuracy.
[0085] Please refer to Figure 5. An embodiment of this application provides a model pruning method, applied to an electronic device or server as shown in Figure 1 or Figure 2. The method includes:
[0086] Step S310: Obtain the measure corresponding to each channel in each network layer of the model to be pruned, wherein the measure is used to characterize the importance of the corresponding channel.
[0087] In this embodiment, the measure corresponding to each output channel in each network layer of the model to be pruned can be obtained through a preset calculation method and based on the weights corresponding to each output channel. The preset calculation method may include the connection sensitivity-based method SNIP or the Taylor first-order expansion method TaylorFO.
[0088] The measure corresponding to each output channel of each network layer can be expressed as: in, This is represented as the measure corresponding to the k-th output channel of the l-th network layer.
[0089] Step S320: Based on the measures corresponding to the channels included in each network layer, determine the importance of each network layer to obtain the importance of each network layer of the model to be pruned.
[0090] One approach is to determine the importance of each network layer based on the measure corresponding to each channel included in each network layer, so as to obtain the importance of each network layer of the model to be pruned. This includes: determining the average measure corresponding to each network layer based on the measure corresponding to each channel included in each network layer, and using the average measure as the importance of each network layer, so as to obtain the importance of each network layer of the model to be pruned.
[0091] In this embodiment, based on the metric of each output channel of each network layer, the metric of each network layer can be defined, that is, the importance of each network layer can be defined. Specifically, for the l-th network layer, its corresponding importance I... l Defined as: That is, the importance of the l-th network layer is determined by its C l Each output channel metric The average value determines the outcome.
[0092] Step S330: Based on the importance of each of the multiple network layers, sort the multiple network layers of the model to be pruned in descending order of importance to obtain the sorted multiple network layers.
[0093] In this embodiment of the application, after obtaining the importance of each of the multiple network layers, the multiple network layers are arranged in descending order of importance so as to more quickly determine the network layer to be widened from the multiple network layers.
[0094] Step S340: From the sorted multiple network layers, obtain the network layer at the specified sorting position as the network layer to be widened in the pruning model.
[0095] In this embodiment of the application, the specified sorting position can be the network layer that is arranged at the first specified sorting position. For example, the top 5 network layers can be selected from the sorted network layers as the network layers to be widened.
[0096] Step S350: Select the network layers other than the network layer to be widened from the sorted network layers as the reference network layers of the model to be pruned.
[0097] Step S360: Widen the network layer to be widened in the pruning model to obtain a widened pruning model including a widened network layer and a reference network layer.
[0098] Step S370: Prune the widened model to be pruned based on a preset sparsity to obtain the target model.
[0099] In this embodiment of the application, after obtaining the target model, the pruned model can be optimized and trained a certain number of times as needed, so that the target model obtained after training has higher accuracy.
[0100] This application provides a model pruning method that determines the network layers to be broadened according to the importance of each network layer in the model, broadens the network layers with high importance to obtain a broadened model, and then prunes the broadened model to obtain a target model with higher accuracy.
[0101] Please refer to Figure 6. An embodiment of this application provides a model pruning device 400, the device 400 comprising:
[0102] Importance acquisition unit 410 is used to acquire the importance of each of the multiple network layers of the model to be pruned.
[0103] In one approach, the importance acquisition unit 410 is specifically used to acquire the measure corresponding to each channel included in each network layer of the model to be pruned, the measure being used to characterize the importance of the corresponding channel; based on the measure corresponding to each channel included in each network layer, the importance corresponding to each network layer is determined, so as to obtain the importance corresponding to each of the multiple network layers of the model to be pruned.
[0104] Specifically, the importance acquisition unit 410 is used to determine the average measure corresponding to each network layer based on the measure corresponding to each channel included in each network layer, and to use the average measure as the importance corresponding to each network layer, so as to obtain the importance corresponding to each of the multiple network layers of the model to be pruned.
[0105] The network layer determination unit 420 is used to determine the network layer to be widened and the reference network layer of the pruning model based on the importance of each of the plurality of network layers. The reference network layer is the network layer other than the network layer to be widened among the plurality of network layers.
[0106] In one approach, the network layer determination unit 420 is specifically used to sort the multiple network layers of the model to be pruned in descending order of importance based on the importance of each of the multiple network layers, to obtain a sorted set of multiple network layers; from the sorted multiple network layers, the network layer at a specified sorting position is selected as the network layer to be widened in the model to be pruned; and the network layers other than the network layer to be widened in the sorted multiple network layers are selected as the reference network layers in the model to be pruned.
[0107] The widening unit 430 is used to widen the network layer to be widened in the pruning model to obtain a widened pruning model including a widened network layer and a reference network layer.
[0108] In one approach, the widening unit 430 is specifically used to obtain a preset number of widening layers and a preset widening ratio; based on the preset number of widening layers and the preset widening ratio, the widening network layer of the model to be widened is widened to obtain the widened pruning model including the widened network layer and the reference network layer.
[0109] The pruning unit 440 is used to prune the widened model to be pruned based on a preset sparsity to obtain the target model.
[0110] In one approach, the pruning unit 440 is specifically used to retrain the widened model to be pruned to obtain a trained widened model to be pruned; to prune the reference network layer of the trained widened model to be pruned based on a first sparsity, and to prune the widened network layer of the trained widened model to be pruned based on a second sparsity to obtain the target model. The first sparsity is a preset sparsity, and the second sparsity is determined based on the number of channels in the widened network layer.
[0111] Furthermore, the pruning unit 440 is also used to obtain the total number of channels corresponding to each network layer of the model to be pruned; determine the first number of channels corresponding to each network layer to be widened and each reference network layer of the model to be pruned based on the first sparsity, wherein the first number of channels is the number of channels to be deleted for each network layer to be widened and each reference network layer; determine the second number of channels corresponding to each network layer based on the total number of channels and the first number of channels for each network layer of the model to be pruned, wherein the second number of channels is the number of remaining channels for each network layer; obtain the total number of channels corresponding to each widened network layer of the model to be pruned; determine the first number of channels corresponding to each widened network layer of the model to be pruned based on the total number of channels and the second number of channels corresponding to each network layer to be widened; and determine the second sparsity based on the total number of channels and the first number of channels for each widened network layer of the model to be pruned.
[0112] It should be noted that the device embodiments in this application correspond to the aforementioned method embodiments. The specific principles in the device embodiments can be found in the content of the aforementioned method embodiments, and will not be repeated here.
[0113] The following description, with reference to Figure 7, will illustrate an electronic device or server provided in this application.
[0114] Referring to Figure 7, based on the model pruning method and apparatus described above, this application embodiment also provides another electronic device or server 800 capable of executing the aforementioned model pruning method. The electronic device or server 800 includes one or more (only one shown in the figure) processors 802, a memory 804, and a network module 806 coupled together. The memory 804 stores programs capable of executing the contents of the aforementioned embodiments, and the processor 802 can execute the programs stored in the memory 804.
[0115] The processor 802 may include one or more processing cores. The processor 802 connects to various parts of the electronic device or server 800 using various interfaces and lines. It executes various functions and processes data by running or executing instructions, programs, code sets, or instruction sets stored in the memory 804, and by calling data stored in the memory 804. Optionally, the processor 802 may be implemented using at least one hardware form of Digital Signal Processing (DSP), Field-Programmable Gate Array (FPGA), or Programmable Logic Array (PLA). The processor 802 may integrate one or more of the following: Central Processing Unit (CPU), Graphics Processing Unit (GPU), and modem. The CPU primarily handles the operating system, user interface, and applications; the GPU is responsible for rendering and drawing the displayed content; and the modem handles wireless communication. It is understood that the modem may also not be integrated into the processor 802 and may be implemented separately using a communication chip.
[0116] The memory 804 may include random access memory (RAM) or read-only memory (ROM). The memory 804 can be used to store instructions, programs, code, code sets, or instruction sets. The memory 804 may include a program storage area and a data storage area. The program storage area may store instructions for implementing an operating system, instructions for implementing at least one function (such as touch functionality, sound playback functionality, image playback functionality, etc.), and instructions for implementing the various method embodiments described below. The data storage area may also store data created during the use of the electronic device or server 800 (such as phonebooks, audio and video data, chat log data, etc.).
[0117] The network module 806 is used to receive and transmit electromagnetic waves, realizing the mutual conversion between electromagnetic waves and electrical signals, thereby communicating with communication networks or other devices, such as audio playback devices. The network module 806 may include various existing circuit elements for performing these functions, such as antennas, radio frequency transceivers, digital signal processors, encryption / decryption chips, user identity modules (SIM cards), memory, etc. The network module 806 can communicate with various networks such as the Internet, corporate intranets, and wireless networks, or communicate with other devices through wireless networks. The aforementioned wireless networks may include cellular telephone networks, wireless local area networks (WLANs), or metropolitan area networks (MANs). For example, the network module 806 can interact with base stations.
[0118] Please refer to Figure 8, which shows a structural block diagram of a computer-readable storage medium provided in an embodiment of this application. The computer-readable storage medium 900 stores program code that can be called by a processor to execute the methods described in the above method embodiments.
[0119] The computer-readable storage medium 900 may be an electronic memory such as flash memory, EEPROM (Electrically Erasable Programmable Read-Only Memory), EPROM, hard disk, or ROM. Optionally, the computer-readable storage medium 900 includes a non-transitory computer-readable storage medium. The computer-readable storage medium 900 has storage space for program code 910 that performs any of the method steps described above. This program code can be read from or written to one or more computer program products. The program code 910 may be compressed, for example, in a suitable form.
[0120] This application provides a model pruning method, apparatus, electronic device, and storage medium. First, the importance of each network layer in the model to be pruned is obtained. Then, based on the importance of each network layer, the network layers to be widened and reference network layers are determined. The reference network layers are all network layers except the network layer to be widened. The network layer to be widened is then widened to obtain a widened pruning model including the widened network layer and the reference network layer. Finally, the widened pruning model is pruned based on a preset sparsity to obtain the target model. By using this method, network layers to be widened are determined according to the importance of each network layer in the model. Widening the high-importance network layers to obtain a widened model, followed by pruning, can yield a more accurate target model.
[0121] The embodiments of the present invention have been described above with reference to the accompanying drawings. However, the present invention is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of the present invention without departing from the spirit and scope of the claims, and all of these forms are within the protection scope of the present invention.
Claims
1. A model pruning method, characterized in that, The method includes: obtaining the importance of each of the multiple network layers of the model to be pruned; determining the network layer to be widened and the reference network layer of the model to be pruned based on the importance of each of the multiple network layers, wherein the reference network layer is a network layer other than the network layer to be widened among the multiple network layers; widening the network layer to be widened in the model to be pruned to obtain a widened model to be pruned including the widened network layer and the reference network layer; and pruning the widened model to be pruned based on a preset sparsity to obtain a target model, wherein the target model is used to deploy on an edge device so that the edge device can identify objects in an image or predict human key points by running the target model.
2. The method according to claim 1, characterized in that, The step of widening the network layers of the model to be pruned to obtain a widened pruning model including a widened network layer and a reference network layer includes: obtaining a preset number of widening layers and a preset widening ratio; and based on the preset number of widening layers and the preset widening ratio, widening the network layers of the model to be pruned to obtain the widened pruning model including a widened network layer and a reference network layer.
3. The method according to claim 2, characterized in that, The step of pruning the widened model to be pruned based on a preset sparsity to obtain a target model includes: retraining the widened model to be pruned to obtain a trained widened model to be pruned; pruning the reference network layer of the trained widened model to be pruned based on a first sparsity; and pruning the widened network layer of the trained widened model to be pruned based on a second sparsity to obtain the target model. The first sparsity is a preset sparsity, and the second sparsity is determined based on the number of channels in the widened network layer.
4. The method according to claim 3, characterized in that, Before retraining the widened pruning model to obtain the trained widened pruning model, the method further includes: obtaining the total number of channels corresponding to each network layer of the pruning model; determining the first number of channels corresponding to each network layer to be widened and each reference network layer of the pruning model based on the first sparsity, wherein the first number of channels is the number of channels to be deleted for each network layer to be widened and each reference network layer; determining the second number of channels corresponding to each network layer based on the total number of channels and the first number of channels, wherein the second number of channels is the number of remaining channels for each network layer; obtaining the total number of channels corresponding to each widened network layer of the widened pruning model; determining the first number of channels corresponding to each widened network layer of the widened pruning model based on the total number of channels and the second number of channels corresponding to each network layer to be widened; and determining the second sparsity based on the total number of channels and the first number of channels corresponding to each widened network layer of the widened pruning model.
5. The method according to claim 1, characterized in that, The step of obtaining the importance of each of the multiple network layers of the model to be pruned includes: obtaining the measure corresponding to each channel included in each network layer of the model to be pruned, wherein the measure is used to characterize the importance of the corresponding channel; and determining the importance of each network layer based on the measure corresponding to each channel included in each network layer, so as to obtain the importance of each of the multiple network layers of the model to be pruned.
6. The method according to claim 5, characterized in that, The step of determining the importance of each network layer based on the measure corresponding to each channel included in each network layer, so as to obtain the importance of each network layer of the model to be pruned, includes: determining the average measure corresponding to each network layer based on the measure corresponding to each channel included in each network layer, and using the average measure as the importance of each network layer, so as to obtain the importance of each network layer of the model to be pruned.
7. The method according to claim 1, characterized in that, Based on the importance of each of the multiple network layers, the determination of the network layers to be widened and the reference network layers of the pruning model includes: sorting the multiple network layers of the pruning model in descending order of importance based on the importance of each of the multiple network layers, obtaining sorted multiple network layers; selecting the network layer at a specified sorting position from the sorted multiple network layers as the network layer to be widened in the pruning model; and selecting the network layers other than the network layer to be widened from the sorted multiple network layers as the reference network layers of the pruning model.
8. A model pruning device, characterized in that, The apparatus includes: an importance acquisition unit, configured to acquire the importance of each of the multiple network layers of the model to be pruned; a network layer determination unit, configured to determine, based on the importance of each of the multiple network layers, a network layer to be widened and a reference network layer of the model to be pruned, wherein the reference network layer is a network layer other than the network layer to be widened among the multiple network layers; a widening unit, configured to widen the network layer to be widened in the model to be pruned, to obtain a widened model to be pruned including the widened network layer and the reference network layer; and a pruning unit, configured to prune the widened model to be pruned based on a preset sparsity, to obtain a target model, wherein the target model is used to deploy on an edge device so that the edge device can identify objects in an image or predict human key points by running the target model.
9. An electronic device, characterized in that, It includes one or more processors; one or more programs are stored in memory and configured to be executed by the one or more processors according to any one of claims 1-7.
10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores program code, wherein the program code, when executed by a processor, performs the method according to any one of claims 1-7.
Citation Information
Patent Citations
Pruning method for convolutional neural network model
CN112990420A
Data processing method, system and device, and readable storage medium
WO2022021868A1