Pruning method and device based on target model, electronic device and storage medium

Abstract

The application discloses a pruning method and device based on a target model, electronic equipment and a storage medium, and relates to the field of artificial intelligence, and the pruning method comprises the following steps: determining a to-be-pruned layer in a target model, adding a preset pruning layer after each to-be-pruned layer to obtain a target network structure, under the condition that the gradient update of the parameters of each network layer in the target network structure is completed, screening a first redundant channel set in the preset pruning layer, determining a second redundant channel set in the to-be-pruned layer based on the first redundant channel set, and pruning each second redundant channel in the second redundant channel set to obtain a pruned target model. The application solves the technical problem that the accuracy of pruning in the prior art is poor, and the pruned model cannot accurately process image files.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence, and more specifically, to a pruning method, apparatus, electronic device, and storage medium based on a target model. Background Technology

[0002] Currently, convolutional neural networks have achieved high accuracy and strong generalization capabilities through deep structures and over-parameterization. However, the increasing depth and multi-layer architecture are making the networks increasingly large. For example, VGG16 (Visual Geometry Group Network) has more than 15G of floating-point operations (FLOPs), more than 140 million network parameters, and requires more than 500MB of storage space to store these parameters. The increasing complexity of convolutional neural networks (CNNs) limits their deployment and commercialization in edge devices, primarily in the following aspects: (1) The sheer number of parameters consumes considerable storage resources; millions or even tens of millions of trainable parameters make it difficult to deploy deep CNNs in resource-sensitive platforms such as embedded or IoT devices; (2) Floating-point computation also imposes a significant computational burden; convolution operations are computationally intensive on high-resolution images and cannot be applied to devices with low computing power; (3) Overparameterized models result in high energy costs; the batteries of edge devices deplete rapidly under the influence of complex computations, posing a challenge to their battery life. Therefore, it is necessary to reduce the storage, computation, and battery life costs of CNNs, i.e., to compress neural networks.

[0003] For a pre-trained convolutional neural network, one of the main goals of model compression is to remove redundant parameters while maintaining the original training accuracy. Model pruning is a model compression technique that can be divided into structured (channel) pruning and unstructured (fine-grained) pruning.

[0004] In related technologies, model pruning can be performed using the following methods: (1) A channel pruning algorithm based on input and output layers. By studying the correlation between the preceding and following layers in the forward propagation of the convolutional neural network, the input and output channels of two consecutive layers can be used as structural sparsity regularization terms for pre-training before pruning, and a channel-level pruning metric based on the input and output layers is proposed. However, the sparsity terms during pre-training of this algorithm are applied to all convolutional layer channels, and the sparsity terms are coupled with the loss function of the original network for training. This is an algorithm that couples training and pruning, making it difficult to achieve a balance between accuracy loss and sparsity rate, resulting in the pruned model being unable to accurately process image files; (2) A decoupled ResRep (lossless channel pruning) pruning algorithm, which can evaluate the channel pruning algorithm from the aspects of resistance and pruningability, and reparameterize the CNN into memory and forgetting parts. However, the redundant channels of each layer in this algorithm are judged using a metric based on the size of the norm of this layer, ignoring the correlation between the preceding and following layers of the convolutional neural network. Meanwhile, the algorithm only considers the sparsity regularization of the current layer to divide the pruning parameters, which makes the lower layer input channels not have the same sparsity as the upper layer output filters. It cannot accurately identify redundant channel parameters, resulting in the pruned model being unable to accurately process image files.

[0005] There is currently no effective solution to the above problems. Summary of the Invention

[0006] This invention provides a pruning method, apparatus, electronic device, and storage medium based on a target model, to at least solve the technical problem in related technologies where the pruning accuracy is poor, resulting in the pruned model being unable to accurately process image files.

[0007] According to one aspect of the present invention, a pruning method based on a target model is provided, comprising: determining layers to be pruned in a target model, wherein the target model is a trained neural network model; adding a preset pruning layer after each layer to be pruned to obtain a target network structure, wherein the target network structure includes: multiple network layers, wherein the network layers are original network layers or the preset pruning layers; after gradient updates of the parameters of each network layer in the target network structure are completed, filtering a first set of redundant channels in the preset pruning layers; based on the first set of redundant channels, determining a second set of redundant channels in the layers to be pruned, and pruning each second redundant channel in the second set of redundant channels to obtain the pruned target model, wherein the pruned target model is used to process a preset file.

[0008] Optionally, before adding a preset pruning layer after each of the layers to be pruned, the method further includes: constructing a preset convolution kernel with a preset width and a preset height; constructing the preset pruning layer based on the preset convolution kernel; initializing the convolution kernel parameters in the preset pruning layer to an identity matrix; and setting the bias parameters in the preset pruning layer to a preset value.

[0009] Optionally, before filtering the first set of redundant channels in the preset pruning layer, the method further includes: performing a preset sparse loss operation on the preset pruning layer to update the gradient of the parameters in the preset pruning layer; determining the gradient value of the first weight tensor with respect to the preset sparse loss based on the first weight tensor of the preset pruning layer and the second weight tensor of the layer to be pruned located below the preset pruning layer, wherein the preset sparse loss is obtained by performing the preset sparse loss operation on the preset pruning layer; determining a target loss value using a target loss function based on a preset penalty factor and the preset sparse loss; and determining that the gradient update of the parameters of the preset pruning layer is complete if the gradient value is less than the target loss value.

[0010] Optionally, before determining the target loss value using the target loss function based on the preset penalty factor and the preset sparse loss, the method further includes: constructing the target loss function based on the initial loss function, the product of the initial sparse loss and the initial penalty factor, and the product of the preset sparse loss and the preset penalty factor, wherein the preset sparse loss is obtained by accumulating the sum of squares of the first weight tensor and the sum of squares of the second weight tensor.

[0011] Optionally, before filtering the first set of redundant channels in the preset pruning layer, the method further includes: updating the gradient of the parameters of the original network layer using a preset gradient descent strategy; and determining that the gradient update of the parameters of the original network layer is complete if the gradient value of the parameters of the original network layer is less than the loss value obtained through the initial loss function.

[0012] Optionally, the step of filtering the first set of redundant channels in the preset pruning layer includes: determining a metric value based on a preset pruning rate; determining a weight value for each channel in the preset pruning layer based on a first weight tensor of the preset pruning layer and a second weight tensor of the layer to be pruned located below the preset pruning layer; sorting all the weight values ​​to obtain a sorting result; and determining the first set of redundant channels based on the sorting result and the metric value.

[0013] Optionally, the step of determining the second redundant channel set in the layer to be pruned based on the first redundant channel set includes: determining the pruning result of the preset pruning layer based on the first redundant channel set; merging the pruning result into the layer to be pruned to obtain the output result of the layer to be pruned; and determining the second redundant channel set based on the output result.

[0014] According to another aspect of the present invention, a pruning device based on a target model is also provided, comprising: a determining unit, configured to determine layers to be pruned in a target model, wherein the target model is a trained neural network model; an adding unit, configured to add a preset pruning layer after each layer to be pruned to obtain a target network structure, wherein the target network structure includes: a plurality of network layers, wherein the network layers are original network layers or the preset pruning layers; a filtering unit, configured to filter a first set of redundant channels in the preset pruning layers after gradient updates of the parameters of each network layer in the target network structure are completed; and a pruning unit, configured to determine a second set of redundant channels in the layers to be pruned based on the first set of redundant channels, and prune each second redundant channel in the second set of redundant channels to obtain the pruned target model, wherein the pruned target model is used to process a preset file.

[0015] Optionally, the pruning device further includes: a first construction module, configured to construct a preset convolution kernel with a preset width and a preset height before adding a preset pruning layer after each layer to be pruned; a second construction module, configured to construct the preset pruning layer based on the preset convolution kernel; and a first initialization module, configured to initialize the convolution kernel parameters in the preset pruning layer to an identity matrix and set the bias parameters in the preset pruning layer to a preset value.

[0016] Optionally, the pruning device further includes: a first update module, configured to perform a preset sparse loss operation on the preset pruning layer before filtering the first redundant channel set in the preset pruning layer, thereby updating the gradient of the parameters in the preset pruning layer; a first determination module, configured to determine the gradient value of the first weight tensor with respect to the preset sparse loss based on the first weight tensor of the preset pruning layer and the second weight tensor of the layer to be pruned located below the preset pruning layer, wherein the preset sparse loss is obtained by performing the preset sparse loss operation on the preset pruning layer; a second determination module, configured to determine a target loss value using a target loss function based on a preset penalty factor and the preset sparse loss; and a third determination module, configured to determine that the gradient update of the parameters of the preset pruning layer is complete if the gradient value is less than the target loss value.

[0017] Optionally, the pruning device further includes: a third construction module, configured to construct the target loss function based on the initial loss function, the product of the initial sparse loss and the initial penalty factor, and the product of the preset sparse loss and the preset penalty factor before determining the target loss value using the target loss function based on the preset penalty factor and the preset sparse loss, wherein the preset sparse loss is obtained by accumulating the sum of squares of the first weight tensor and the sum of squares of the second weight tensor.

[0018] Optionally, the pruning device further includes: a second update module, used to update the gradient of the parameters of the original network layer using a preset gradient descent strategy before filtering the first redundant channel set in the preset pruning layer; and a fourth determination module, used to determine that the gradient update of the parameters of the original network layer is complete when the gradient value of the parameters of the original network layer is less than the loss value obtained by the initial loss function.

[0019] Optionally, the filtering unit includes: a fifth determining module, used to determine a metric value based on a preset pruning rate; a sixth determining module, used to determine the weight value of each channel in the preset pruning layer based on the first weight tensor of the preset pruning layer and the second weight tensor of the layer to be pruned located below the preset pruning layer; a first sorting module, used to sort all the weight values ​​to obtain a sorting result; and a seventh determining module, used to determine the first redundant channel set based on the sorting result and the metric value.

[0020] Optionally, the pruning unit includes: an eighth determining module, used to determine the pruning result of the preset pruning layer based on the first redundant channel set; a first merging module, used to merge the pruning result into the layer to be pruned, to obtain the output result of the layer to be pruned; and a ninth determining module, used to determine the second redundant channel set based on the output result.

[0021] According to another aspect of the present invention, a computer-readable storage medium is also provided, the computer-readable storage medium including a stored computer program, wherein, when the computer program is executed, it controls the device where the computer-readable storage medium is located to perform the above-described pruning method based on the target model.

[0022] According to another aspect of the present invention, an electronic device is also provided, including one or more processors and a memory, the memory being used to store one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors cause the one or more processors to implement the above-described pruning method based on the target model.

[0023] In this disclosure, the layers to be pruned in the target model are determined, and a preset pruning layer is added after each layer to be pruned to obtain the target network structure. After the gradient update of the parameters of each network layer in the target network structure is completed, the first set of redundant channels in the preset pruning layer is selected. Based on the first set of redundant channels, the second set of redundant channels in the layers to be pruned is determined, and each second redundant channel in the second set of redundant channels is pruned to obtain the pruned target model. In this disclosure, a preset pruning layer can be added after the pruning layer of the target model to decouple the gradient update process and the pruning process. After the gradient update of the parameters of each network layer in the target network structure is completed, the first redundant channel set in the preset pruning layer is selected, and then the second redundant channel set in the pruning layer is determined according to the first redundant channel set. Then, all the second redundant channels in the pruning layer are pruned to obtain the pruned target model. This not only achieves the purpose of accurately identifying redundant channels and improving pruning efficiency, but also reduces storage, computing and battery life costs of the pruned target model without losing accuracy. This solves the technical problem in related technologies where the pruning accuracy is poor, resulting in the pruned model being unable to accurately process image files. Attached Figure Description

[0024] The accompanying drawings, which are included to provide a further understanding of the invention and form part of this application, illustrate exemplary embodiments of the invention and, together with their description, serve to explain the invention and do not constitute an undue limitation thereof. In the drawings:

[0025] Figure 1 This is a flowchart of an optional target model-based pruning method according to an embodiment of the present invention;

[0026] Figure 2 This is a schematic diagram of an optional network structure with a pruning layer according to an embodiment of the present invention;

[0027] Figure 3 This is a schematic diagram of an optional decoupling-based front-to-back layer pruning process according to an embodiment of the present invention;

[0028] Figure 4 This is a schematic diagram of an optional target model-based pruning device according to an embodiment of the present invention;

[0029] Figure 5 This is a hardware structure block diagram of an electronic device (or mobile device) for a pruning method based on a target model according to an embodiment of the present invention. Detailed Implementation

[0030] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0031] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0032] It should be noted that the target model-based pruning method and apparatus disclosed herein can be used in the field of artificial intelligence for pruning based on a target model, and can also be used in any field other than artificial intelligence for pruning based on a target model. This disclosure does not limit the application field of the target model-based pruning method and apparatus.

[0033] It should be noted that all information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, and displayed data) disclosed herein are information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, use, and processing of such data must comply with the relevant laws, regulations, and standards of the relevant countries and regions, and corresponding access points are provided for users to choose to authorize or refuse. For example, this system has interfaces with relevant users or organizations. Before obtaining relevant information, a request to obtain the information must be sent to the aforementioned user or organization through the interface, and the relevant information will be obtained only after receiving consent from the aforementioned user or organization.

[0034] The following embodiments of the present invention can be applied to various systems / applications / devices that perform pruning based on target models.

[0035] For trained models, channel pruning algorithms in related technologies couple training and pruning by using sparse weights and ignore the relationships between preceding and following layers, resulting in suboptimal training and pruning performance and inaccurate identification of redundant channels. To address these issues, this invention proposes a pruning algorithm based on decoupling training and pruning, using preceding and following layer metrics and sparsity terms to prune trained models, achieving efficient channel pruning for convolutional networks.

[0036] This invention utilizes a dedicated pruning layer for decoupling, proposing a structured sparsity based on the correlation between consecutive layers for pruning layer channels only. It also employs a pruning metric to determine redundancy within this layer for global pruning. This addresses the issues of coupled training and pruning processes and poor pruning performance, accurately identifying channel redundancy information while simultaneously dividing training and pruning operations.

[0037] The decoupling-based pre- and post-layer pruning algorithm proposed in this invention can be used for pruning tasks of pre-trained models. First, a pruning layer module for decoupling is added after the convolutional layer requiring pruning, with the same initialization parameters as the original network output. This improves the accuracy of pruning judgment and facilitates the use of pre- and post-layer metrics. Second, addressing the drawback of existing metrics that only consider sparsity regularization within the current layer, a pre- and post-layer sparsity regularization term is proposed, applicable only to the pruning layer. The metric applied to the pruning layer incorporates the correlation between pre- and post-layer layers into the pruning judgment, establishing a correspondence between the output of the previous layer and the input channels of the next layer, thus removing redundant channels. Finally, non-essential channels with a pre-set pruning ratio are removed according to the pre- and post-layer metrics, and the pruning layer module is equivalently merged into the original structure, resulting in a network model with the same number of layers but sparse channels. Compared to channel pruning algorithms in related technologies, the algorithm in this invention achieves high accuracy after pruning, solving the problems of maintaining the original training accuracy and inaccurate identification of redundant channels.

[0038] The present invention will now be described in detail with reference to various embodiments.

[0039] Example 1

[0040] According to an embodiment of the present invention, an embodiment of a pruning method based on a target model is provided. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Furthermore, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.

[0041] Figure 1 This is a flowchart of an optional target model-based pruning method according to an embodiment of the present invention, such as... Figure 1 As shown, the method includes the following steps:

[0042] Step S101: Determine the layers to be pruned in the target model, where the target model is a trained neural network model.

[0043] Step S102: Add a preset pruning layer after each layer to be pruned to obtain the target network structure, wherein the target network structure includes: multiple network layers, and the network layers are the original network layers or preset pruning layers.

[0044] Step S103: After the gradient update of the parameters of each network layer in the target network structure is completed, the first set of redundant channels in the preset pruning layer is selected.

[0045] Step S104: Based on the first set of redundant channels, determine the second set of redundant channels in the layer to be pruned, and prune each of the second redundant channels in the second set of redundant channels to obtain the pruned target model, wherein the pruned target model is used to process the preset file.

[0046] Through the above steps, the layers to be pruned in the target model can be determined. A preset pruning layer is added after each layer to be pruned to obtain the target network structure. After the gradient update of the parameters of each network layer in the target network structure is completed, the first redundant channel set in the preset pruning layer is selected. Based on the first redundant channel set, the second redundant channel set in the layers to be pruned is determined, and each second redundant channel in the second redundant channel set is pruned to obtain the pruned target model. In this embodiment of the invention, a preset pruning layer can be added after the pruning layer of the target model to decouple the gradient update process and the pruning process. After the gradient update of the parameters of each network layer in the target network structure is completed, the first redundant channel set in the preset pruning layer is selected, and then the second redundant channel set in the pruning layer is determined according to the first redundant channel set. Then, all the second redundant channels in the pruning layer are pruned to obtain the pruned target model. This not only achieves the purpose of accurately identifying redundant channels and improving pruning efficiency, but also reduces storage, computing and battery life costs of the pruned target model without losing accuracy. This solves the technical problem in related technologies where the pruning accuracy is poor, resulting in the pruned model being unable to accurately process image files.

[0047] The embodiments of the present invention will now be described in detail with reference to the steps described above.

[0048] Step S101: Determine the layers to be pruned in the target model, where the target model is a trained neural network model.

[0049] In this embodiment of the invention, in order to decouple the training process (which can be performed through gradient updates) from the pruning process, the main steps include: setting the pruning layer and updating the gradient of the target model. First, the pruning layer needs to be set. This requires determining the layers to be pruned in the target model (which is a trained neural network model to be pruned). These layers can be convolutional layers, fully connected layers, or convolutional layers followed by a BN (Batch Normalization) layer.

[0050] Optionally, before adding a preset pruning layer after each layer to be pruned, the method further includes: constructing a preset convolution kernel with a preset width and a preset height; constructing a preset pruning layer based on the preset convolution kernel; initializing the convolution kernel parameters in the preset pruning layer to an identity matrix; and setting the bias parameters in the preset pruning layer to preset values.

[0051] In this embodiment of the invention, adding a pruning layer is a change to the original network structure of the target model. To avoid altering the output of the layer to be pruned (such as a convolutional layer) and to ensure that it can be merged back into the original network structure after pruning, a 1×1 convolution can be used. Since the width and height of the convolutional kernel are both only 1, the output feature map is the result of a linear combination of all input channels. Therefore, in this embodiment, a 1×1 convolutional pruning layer can be added after the layer to be pruned in the network structure. To preserve the output of the original convolutional layer conv1, the kernel parameters of the pruning layer are initialized to an identity matrix, and the bias parameters are set to 0, thereby separating the channels to be pruned from the original network and preserving all their output information. For fully connected layers, they can be equivalently represented as 1×1 convolutional layers, so pruning layers can also be used for pruning fully connected layers. Some networks have a BN layer after the convolutional layer; in this case, the pruning layer follows the output of the BN layer without affecting the pruning effect. In this way, the pruning layer is specifically responsible for identifying redundant channels, while the original convolutional layer is only responsible for training to improve accuracy. After gradient updates, the two are merged to obtain a compact model with sparse channels.

[0052] In this embodiment of the invention, in order not to change the output of the layer to be pruned, a preset convolution kernel (i.e., a 1×1 convolution kernel) with a preset width and a preset height can be constructed first. Then, based on the preset convolution kernel, a preset pruning layer is constructed. The convolution kernel parameters in the preset pruning layer need to be initialized to the identity matrix, and the bias parameters in the preset pruning layer need to be set to a preset value (i.e., 0).

[0053] Step S102: Add a preset pruning layer after each layer to be pruned to obtain the target network structure, wherein the target network structure includes: multiple network layers, and the network layers are the original network layers or preset pruning layers.

[0054] In this embodiment of the invention, a pre-constructed preset pruning layer is added after each layer to be pruned to obtain a target network structure. The target network structure includes multiple network layers, which are either the original network layer (i.e., the layer to be pruned) or the preset pruning layer.

[0055] Figure 2 This is a schematic diagram of an optional network structure with a pruning layer according to an embodiment of the present invention, such as... Figure 2 As shown, a pruning layer (with a structure of 16x16x3x3) can be added after the original convolutional layer conv1 (with a structure of 16x16x1x1), and the original convolutional layer conv2 (with a structure of 16x16x3x3) can be connected after the pruning layer.

[0056] Optionally, before filtering the first set of redundant channels in the preset pruning layer, the method further includes: performing a preset sparse loss operation on the preset pruning layer to update the gradient of the parameters in the preset pruning layer; determining the gradient value of the first weight tensor with respect to the preset sparse loss based on the first weight tensor of the preset pruning layer and the second weight tensor of the layer to be pruned located below the preset pruning layer, wherein the preset sparse loss is obtained by performing a preset sparse loss operation on the preset pruning layer; determining the target loss value using a target loss function based on a preset penalty factor and the preset sparse loss; and determining that the gradient update of the parameters of the preset pruning layer is complete if the gradient value is less than the target loss value.

[0057] In another optional embodiment, before determining the target loss value using the target loss function based on the preset penalty factor and the preset sparse loss, the method further includes: constructing a target loss function based on the initial loss function, the product of the initial sparse loss and the initial penalty factor, and the product of the preset sparse loss and the preset penalty factor, wherein the preset sparse loss is obtained by accumulating the sum of squares of the first weight tensor and the sum of squares of the second weight tensor.

[0058] In this embodiment of the invention, after setting the pruning layer, the network parameters can be updated and pruned using the pruning layer. To resolve the contradiction between network sparsity and training loss, the gradient update method used in this embodiment is divided into two parts. First, the parameter update of the pruning layer: to avoid affecting the normal optimization and update of parameters in other layers, this embodiment only applies sparsity regularization to the weights of this layer. Furthermore, since this layer is only responsible for filtering unimportant channels and removing redundancy, the training gradients of redundant channels can be ignored. This embodiment sets a binary mask to set the gradients of redundant channels to 0, retaining only the gradients that are sparsely correlated with their channel characteristics. Assume the neural network loss function (i.e., the initial loss function) is J. ori (X,Y,W), where X represents the input, Y represents the output, and W represents the weight matrix. K represents the kernel K of the trimmed layer convolution.p If one channel is involved, then the gradient update method of F (the convolution kernel parameter) is as follows. (That is, to perform a preset sparse loss operation on the preset pruning layer and update the gradient of the parameters in the preset pruning layer):

[0059]

[0060] Among them, the four-dimensional weight tensor W (l) ∈R O×I×K×K Represented in two-dimensional form O represents the output, I represents the input, K represents the kernel width and height respectively, and P represents the output. pn λ represents the regularization penalty term based on information from the preceding and following layers. pn Let m ∈ {0,1} be the penalty factor, representing a binary mask. The higher the redundancy of the channels, the better. The smaller the impact on the loss function, the better. After determining which channels are unimportant using metrics, the m-value of these channels is set to 0, making them less important. Under the influence of , it approaches 0.

[0061] Second, the gradient update of the remaining parameters. Applying a penalty term only to the parameters of the pruning layer will not affect the normal gradient update of the original layer parameters. Therefore, it is only necessary to use stochastic gradient descent to update these parameters (that is, a preset gradient descent strategy (i.e., stochastic gradient descent) can be used to update the gradient of the parameters of the original network layer).

[0062] In this embodiment, the structural regularization term can generate structural sparsity in the convolutional weight matrix of the current layer, but the convolutional inputs of the lower layers corresponding to the output of the current layer are often ignored. This results in the lower layer input channels not having the same sparsity as the upper layer output filters. The combined effect of the sparse front-layer output and the dense back-layer input causes some important weights in the back layer to be incorrectly deleted. In coupled sparse training methods, since structural sparsity is applied to each convolutional layer, the negative impact of ignoring the lower layer inputs can be compensated to some extent by sparsifying the output channels. However, in networks with added pruning layers, applying sparsity only to the weight tensor of that layer will reveal the problem. To improve this situation, this embodiment proposes a front-to-back layer sparsity regularization term for pruning layers, which allows the interconnected front-to-back convolutional channels to obtain the same importance, and the pruning layer parameters take into account the sparsity information of the front and back layers during gradient updates. The loss function J(X,Y,W) with sparse layers (i.e., the target loss function) can be expressed in the following form (i.e., the target loss function is constructed based on the initial loss function, the product of the initial sparse loss and the initial penalty factor, and the product of the preset sparse loss and the preset penalty factor):

[0063] J(X,Y,W)=J ori (X,Y,W)+λP(W)+λpn P pn (K p (2);

[0064] Among them, J ori (X,Y,W) represents the original loss function (i.e., the initial loss function) without a loss term, λP(W) represents the original sparse training, λ represents the initial penalty factor, P(W) represents the initial sparse loss, and λ pn P represents the preset penalty factor. pn (K p ) indicates that only the trimming layer K is affected. p Apply a customized sparse loss (i.e., a preset sparse loss). Thus, K p This allows some less important weights to tend towards 0 under the influence of the loss term; specifically, P... pn (K p This can be expressed as (i.e., the pre-defined sparse loss is obtained by summing the squares of the first weight tensor and the squares of the second weight tensor):

[0065]

[0066] Among them, W (l) ∈R O×I×K×K W represents the first weight tensor of the pruning layer. (l+1) ∈R O'×O×T×T Let O' represent the second weight tensor of the convolutional layer below the pruning layer, O' represent the input, O represent the output, and T represent the width and height of the convolutional kernel, respectively. Equation (1) That is Regarding P pn (K p The gradient of the first weight tensor with respect to the preset sparse loss can be calculated (i.e., based on the first weight tensor of the preset pruning layer and the second weight tensor of the layer to be pruned located below the preset pruning layer, the gradient value of the first weight tensor with respect to the preset sparse loss is determined, which is obtained by performing a preset sparse loss operation on the preset pruning layer):

[0067]

[0068] Then, based on the preset penalty factor and the preset sparse loss, the target loss function (i.e., formula (2)) can be used to determine the target loss value. If the gradient value is less than the target loss value, it can be determined that the gradient update of the parameters of the preset pruning layer is complete.

[0069] In this embodiment, if the gradient of formula (4) is directly applied to the unpruned convolutional layer, it will affect the gradient used to maintain network performance. Therefore, during backpropagation, only the pruned layer K is applied. pThe gradient is penalized by formula (4). According to formula (1), the parameters of this layer still carry the gradients related to the original network loss. Redundant parameters need to be selected based on certain metrics, and the first term of these parameters should be set to 0.

[0070] Optionally, before filtering the first set of redundant channels in the preset pruning layer, the method further includes: using a preset gradient descent strategy to update the gradient of the parameters of the original network layer; and determining that the gradient update of the parameters of the original network layer is complete if the gradient value of the parameters of the original network layer is less than the loss value obtained through the initial loss function.

[0071] In this embodiment of the invention, the parameters of the original network layer can be updated using stochastic gradient descent (i.e., a preset gradient descent strategy (i.e., stochastic gradient descent) can be used to update the gradient of the parameters of the original network layer). If the gradient value of the parameters of the original network layer is less than the loss value obtained through the initial loss function, it can be determined that the gradient update of the parameters of the original network layer is complete.

[0072] Step S103: After the gradient update of the parameters of each network layer in the target network structure is completed, the first set of redundant channels in the preset pruning layer is selected.

[0073] Optionally, the step of filtering the first set of redundant channels in the preset pruning layer includes: determining a metric value based on a preset pruning rate; determining the weight value of each channel in the preset pruning layer based on the first weight tensor of the preset pruning layer and the second weight tensor of the layer to be pruned located below the preset pruning layer; sorting all weight values ​​to obtain a sorting result; and determining the first set of redundant channels based on the sorting result and the metric value.

[0074] In this embodiment of the invention, the metric for selecting redundant channels in the pruning layer should also reflect the correlation between preceding and following layers, and the weight value of each channel in the preset pruning layer should be calculated. The formula is as follows (that is, based on the first weight tensor of the preset pruning layer and the second weight tensor of the layer to be pruned located below the preset pruning layer, the weight value of each channel in the preset pruning layer is determined):

[0075]

[0076] Where j represents the number of channels, and i represents the index of a specific channel. In this embodiment, it serves as a metric for measuring the importance of channels in the l-th trimmed layer. The larger the value, the more important this channel is compared to other channels. Similar to formula (4), It also includes parameter information for both layer l (the pruning layer) and layer l+1 (the layer after the pruning layer). Squaring the corresponding weights is to avoid incorrect judgments caused by the cancellation of positive and negative values. Based on the metric of the preceding and following layers, the correct redundant channels can be selected effectively while maintaining a certain level of model accuracy.

[0077] In this embodiment of the invention, after the gradients of the parameters of each network layer in the target network structure have been updated, the first redundant channel set in the preset pruning layer can be screened. Specifically, the metric standard value can be determined according to the preset pruning rate (for example, the number of channels to be pruned can be calculated according to the preset pruning rate and the number of channels in the pruning layer, and the number of channels to be pruned is the metric standard value). Then, the weight value of each channel in the preset pruning layer is calculated according to formula (5), and then all weight values ​​are sorted to obtain the sorting result. Then, the first redundant channel set is determined according to the sorting result and the metric standard value (that is, if it is sorted from smallest to largest, the first redundant channel set (i.e., the channels to be pruned) includes the channels with the previous metric standard value (such as the first 30 channels); if it is sorted from largest to smallest, the first redundant channel set includes the channels with the next metric standard value (such as the last 30 channels)).

[0078] Step S104: Based on the first set of redundant channels, determine the second set of redundant channels in the layer to be pruned, and prune each of the second redundant channels in the second set of redundant channels to obtain the pruned target model, wherein the pruned target model is used to process the preset file.

[0079] Optionally, the step of determining the second redundant channel set in the layer to be pruned based on the first redundant channel set includes: determining the pruning result of the preset pruning layer based on the first redundant channel set; merging the pruning result into the layer to be pruned to obtain the output result of the layer to be pruned; and determining the second redundant channel set based on the output result.

[0080] In this embodiment of the invention, to address the phenomenon of diverse model structures, after pruning, the convolutional layers and Batch Normalization (BN) layers of the network are merged to facilitate the subsequent merging of multiple convolutional layers and to obtain a unified "convolutional layer + pruning layer + convolutional layer" structure. After removing redundant channels from the pruning layer according to the front-to-back layer metric, the channels of the original network's convolutional layers are not affected by the pruning. Next, the pruning results of the pruning layer are integrated into the original convolutional layer, that is, a 1×1 convolutional kernel with an output channel O less than or equal to an input channel I is merged with a convolutional kernel with I output and input channels. This is achieved by performing a convolution operation on the weights and biases of the original convolutional kernel using a 1×1 convolutional kernel, and then using the resulting new convolutional kernel to calculate the output channels, thus obtaining an equivalent output result (i.e., based on the first set of redundant channels, the pruning result of the preset pruning layer is determined, and the pruning result is merged into the layer to be pruned to obtain the output result of the layer to be pruned). Assuming P is the new convolutional kernel after merging the BN layers, Q is the 1×1 convolutional kernel of the trimmed layer, and B is the bias after merging, then the output of the trimmed layer can be expressed as follows:

[0081]

[0082] Where F represents the kernel size, This represents the convolution operation. Since Q is simply a linear recombination of channels, it is additive. Therefore, convolving with the kernel weights and biases respectively yields:

[0083]

[0084] Among them, Q pairs The convolution results are linearly combined between channels. Transposing P merges Q into P to obtain P', and merging Q into B to obtain B'. Assume P T If we want to transpose P, then...

[0085]

[0086] Then, the second set of redundant channels can be determined based on the output of the merged pruning layer.

[0087] In this embodiment, after determining the second set of redundant channels in the layer to be pruned based on the first set of redundant channels, each of the second redundant channels in the second set of redundant channels can be pruned to obtain the pruned target model (i.e., the pruned compact model). Then, the pruned target model can be used to process a preset file (e.g., a high-resolution image file). In this way, while maintaining training accuracy, the storage, computing, and battery life costs of the device can be reduced.

[0088] The following describes in detail another optional implementation method.

[0089] In this embodiment, coupled training and pruning negatively impact the final pruning result. To improve the pruning rate and ensure model accuracy, the two should be decoupled. Furthermore, the pruning evaluation criteria should consider the sparsity of both the preceding and following layers to determine the importance of channels. Therefore, this embodiment proposes a decoupled preceding-following layer pruning algorithm.

[0090] Figure 3 This is a schematic diagram of an optional decoupling-based front-to-back layer pruning process according to an embodiment of the present invention, such as... Figure 3 As shown, network training and channel pruning are first decoupled. A 1×1 convolutional pruning module is added after the original convolutional layer (i.e., Conv1 conv 3x3 convolution, which is the front layer). The original convolutional layer (i.e., Conv2 conv 3x3 convolution, which is the back layer) is connected after the pruning module. The gradient of the pruning module is updated based on the sparsity terms of the front and back layers. Then, the redundant channels of the pruning module are obtained. Based on the redundant channels of the pruning module, the convolutional layers Conv1 conv and Conv2 conv are pruned to obtain the pruned convolutional layers.

[0091] In this embodiment of the invention, the specific process of decoupled front-to-back layer pruning is as follows: First, decoupling is performed, separating network training and channel pruning. A 1×1 convolutional pruning module is added after the original convolutional layer, and only pruning-related gradient updates are performed. Second, regarding the sparsity terms and metrics of the front-to-back layers, a sparsification method for pruning layer parameters is proposed, setting the customized regularization term as the sum of the structural sparsity terms of the front and back layers, solving the problem of not considering the sparsity of the back layer in decoupled pruning. Furthermore, a criterion for measuring the importance of channels in the pruned layer is proposed using the idea of ​​front-to-back layer correlation, making the judgment of redundant channels more accurate. Finally, the pruned model is obtained. After training, the channels of the pruned layer become sparse under the action of the front-to-back layer sparsity terms. These channels are equivalently merged with the front convolutional layer, resulting in a pruned model with the same structure as the original model but a narrower width, which improves the pruning rate while maintaining network accuracy.

[0092] In this embodiment of the invention, a pruning algorithm based on decoupled training and pruning, using front-to-back layer metrics and sparsity terms, is proposed. This algorithm achieves efficient convolutional network channel pruning, solves the problem of missing inter-layer information in related technologies, and takes into account the gradient information of front-to-back layers while ensuring network accuracy. It can achieve the effects of compressing the network model, reducing the number of network parameters and computational cost.

[0093] The following is a detailed description with reference to another embodiment.

[0094] Example 2

[0095] The pruning device based on the target model provided in this embodiment includes multiple implementation units, each of which corresponds to a specific implementation step in Embodiment 1 above.

[0096] Figure 4 This is a schematic diagram of an optional target model-based pruning device according to an embodiment of the present invention, such as... Figure 4 As shown, the pruning device may include: a determining unit 40, an adding unit 41, a filtering unit 42, and a pruning unit 43, wherein...

[0097] The determining unit 40 is used to determine the layers to be pruned in the target model, wherein the target model is a trained neural network model;

[0098] Adding unit 41 is used to add a preset pruning layer after each layer to be pruned to obtain the target network structure, wherein the target network structure includes: multiple network layers, and the network layers are the original network layers or preset pruning layers;

[0099] The filtering unit 42 is used to filter the first set of redundant channels in the preset pruning layer after the gradient update of the parameters of each network layer in the target network structure is completed.

[0100] The pruning unit 43 is used to determine the second redundant channel set in the layer to be pruned based on the first redundant channel set, and to prune each second redundant channel in the second redundant channel set to obtain the pruned target model, wherein the pruned target model is used to process the preset file.

[0101] The aforementioned pruning device can determine the layers to be pruned in the target model through the determining unit 40, add a preset pruning layer after each layer to be pruned through the adding unit 41 to obtain the target network structure, filter the first set of redundant channels in the preset pruning layer through the filtering unit 42 after the gradient update of the parameters of each network layer in the target network structure is completed, and determine the second set of redundant channels in the layers to be pruned through the pruning unit 43 based on the first set of redundant channels, and prune each second redundant channel in the second set of redundant channels to obtain the pruned target model. In this embodiment of the invention, a preset pruning layer can be added after the pruning layer of the target model to decouple the gradient update process and the pruning process. After the gradient update of the parameters of each network layer in the target network structure is completed, the first redundant channel set in the preset pruning layer is selected, and then the second redundant channel set in the pruning layer is determined according to the first redundant channel set. Then, all the second redundant channels in the pruning layer are pruned to obtain the pruned target model. This not only achieves the purpose of accurately identifying redundant channels and improving pruning efficiency, but also reduces storage, computing and battery life costs of the pruned target model without losing accuracy. This solves the technical problem in related technologies where the pruning accuracy is poor, resulting in the pruned model being unable to accurately process image files.

[0102] Optionally, the pruning device further includes: a first construction module for constructing a preset convolution kernel with a preset width and a preset height before adding a preset pruning layer after each layer to be pruned; a second construction module for constructing a preset pruning layer based on the preset convolution kernel; and a first initialization module for initializing the convolution kernel parameters in the preset pruning layer to an identity matrix and setting the bias parameters in the preset pruning layer to a preset value.

[0103] Optionally, the pruning device further includes: a first update module, configured to perform a preset sparse loss operation on the preset pruning layer before filtering the first redundant channel set in the preset pruning layer, thereby updating the gradient of the parameters in the preset pruning layer; a first determination module, configured to determine the gradient value of the first weight tensor with respect to the preset sparse loss based on the first weight tensor of the preset pruning layer and the second weight tensor of the layer to be pruned located below the preset pruning layer, wherein the preset sparse loss is obtained by performing a preset sparse loss operation on the preset pruning layer; a second determination module, configured to determine a target loss value using a target loss function based on a preset penalty factor and the preset sparse loss; and a third determination module, configured to determine that the gradient update of the parameters of the preset pruning layer is complete when the gradient value is less than the target loss value.

[0104] Optionally, the pruning device further includes: a third construction module, used to construct a target loss function based on an initial loss function, the product of the initial sparse loss and the initial penalty factor, and the product of the preset sparse loss and the preset penalty factor before determining the target loss value using the target loss function based on a preset penalty factor and a preset sparse loss, wherein the preset sparse loss is obtained by accumulating the sum of squares of the first weight tensor and the sum of squares of the second weight tensor.

[0105] Optionally, the pruning device further includes: a second update module, used to update the gradient of the parameters of the original network layer using a preset gradient descent strategy before filtering the first redundant channel set in the preset pruning layer; and a fourth determination module, used to determine that the gradient update of the parameters of the original network layer is complete when the gradient value of the parameters of the original network layer is less than the loss value obtained by the initial loss function.

[0106] Optionally, the filtering unit includes: a fifth determining module for determining a metric value based on a preset pruning rate; a sixth determining module for determining the weight value of each channel in the preset pruning layer based on the first weight tensor of the preset pruning layer and the second weight tensor of the layer to be pruned located below the preset pruning layer; a first sorting module for sorting all weight values ​​to obtain a sorting result; and a seventh determining module for determining a first set of redundant channels based on the sorting result and the metric value.

[0107] Optionally, the pruning unit includes: an eighth determining module, used to determine the pruning result of the preset pruning layer based on the first redundant channel set; a first merging module, used to merge the pruning result into the layer to be pruned, to obtain the output result of the layer to be pruned; and a ninth determining module, used to determine the second redundant channel set based on the output result.

[0108] The pruning device described above may also include a processor and a memory. The determination unit 40, the addition unit 41, the filtering unit 42, the pruning unit 43, etc., are all stored in the memory as program units, and the processor executes the program units stored in the memory to realize the corresponding functions.

[0109] The aforementioned processor contains a kernel, which retrieves the corresponding program units from memory. One or more kernels can be configured. By adjusting kernel parameters, a second set of redundant channels in the layer to be pruned is determined based on a first set of redundant channels. Each redundant channel in the second set is then pruned to obtain the pruned target model.

[0110] The aforementioned memory may include non-permanent memory in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM, and the memory includes at least one memory chip.

[0111] This application also provides a computer program product, which, when executed on a data processing device, is suitable for executing an initialization program with the following method steps: determining the layers to be pruned in the target model, adding a preset pruning layer after each layer to be pruned to obtain the target network structure, and after the gradient update of the parameters of each network layer in the target network structure is completed, filtering a first set of redundant channels in the preset pruning layer, determining a second set of redundant channels in the layers to be pruned based on the first set of redundant channels, and pruning each second redundant channel in the second set of redundant channels to obtain the pruned target model.

[0112] According to another aspect of the present invention, a computer-readable storage medium is also provided, the computer-readable storage medium including a stored computer program, wherein, when the computer program is running, it controls the device where the computer-readable storage medium is located to execute the above-described pruning method based on the target model.

[0113] According to another aspect of the present invention, an electronic device is also provided, including one or more processors and a memory, the memory being used to store one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors cause the one or more processors to implement the target model-based pruning method described above.

[0114] Figure 5 This is a hardware structure block diagram of an electronic device (or mobile device) for a pruning method based on a target model according to an embodiment of the present invention. Figure 5 As shown, an electronic device may include one or more ( Figure 5 The processor 502 (which may include, but is not limited to, a microprocessor MCU or a programmable logic device FPGA, etc.) and a memory 504 for storing data may also be included. In addition, it may include: a display, an input / output interface (I / O interface), a universal serial bus (USB) port (which may be included as one of the ports of the I / O interface), a network interface, a keyboard, a power supply, and / or a camera. Those skilled in the art will understand that... Figure 5 The structure shown is for illustrative purposes only and does not limit the structure of the electronic device described above. For example, the electronic device may also include components that are more... Figure 5 The more or fewer components shown, or having the same Figure 5 The different configurations shown.

[0115] The sequence numbers of the above embodiments of the present invention are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0116] In the above embodiments of the present invention, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0117] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. The device embodiments described above are merely illustrative; for example, the division of units can be a logical functional division, and in actual implementation, there may be other division methods. For instance, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the displayed or discussed mutual coupling, direct coupling, or communication connection may be through some interfaces; the indirect coupling or communication connection between units or modules may be electrical or other forms.

[0118] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0119] Furthermore, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0120] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, read-only memory (ROM), random access memory (RAM), portable hard drives, magnetic disks, or optical disks.

[0121] The above description is only a preferred embodiment of the present invention. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A method of pruning based on a target model, characterized by, Comprising: The following steps are performed by an electronic device: determining a to-be-pruned layer in a target model, wherein the target model is a trained neural network model, the target model is applied to the field of image processing, and is a neural network model for processing image files; adding a preset pruning layer after each to-be-pruned layer to obtain a target network structure, wherein the target network structure comprises a plurality of network layers, and the network layers are original network layers or the preset pruning layer; in a case where the gradient update of the parameters of each network layer in the target network structure is completed, screening a first redundant channel set in the preset pruning layer; wherein screening the first redundant channel set in the preset pruning layer comprises: determining a metric standard value based on a preset pruning rate; determining a weight value of each channel in the preset pruning layer based on a first weight tensor of the preset pruning layer and a second weight tensor of the to-be-pruned layer located at a next layer of the preset pruning layer; sorting all the weight values to obtain a sorting result; and determining the first redundant channel set based on the sorting result and the metric standard value; based on the first redundant channel set, determining a second redundant channel set in the to-be-pruned layer, and pruning each second redundant channel in the second redundant channel set to obtain a pruned target model, wherein the pruned target model is deployed on an embedded device, an Internet of Things device, a computing capability limited device, or an edge device, and the pruned target model is used to process a preset file, which is an image file; wherein, based on the first redundant channel set, determining the second redundant channel set in the to-be-pruned layer comprises: determining a pruning result of the preset pruning layer based on the first redundant channel set; merging the pruning result to the to-be-pruned layer to obtain an output result of the to-be-pruned layer; and determining the second redundant channel set based on the output result.

2. The pruning method of claim 1, wherein, Before adding a preset pruning layer after each to-be-pruned layer, further comprising: constructing a preset convolution kernel with a preset width and a preset height; constructing the preset pruning layer based on the preset convolution kernel; initializing the convolution kernel parameters in the preset pruning layer as an identity matrix, and setting the bias parameters in the preset pruning layer to a preset numerical value.

3. The pruning method of claim 1, wherein, Before screening the first redundant channel set in the preset pruning layer, further comprising: performing a preset sparse loss operation on the preset pruning layer to update the gradient of the parameters in the preset pruning layer; determining a gradient value of the first weight tensor with respect to a preset sparse loss based on the first weight tensor of the preset pruning layer and the second weight tensor of the to-be-pruned layer located at a next layer of the preset pruning layer, wherein the preset sparse loss is obtained by performing the preset sparse loss operation on the preset pruning layer; determining a target loss value using a target loss function based on a preset penalty factor and the preset sparse loss; in a case where the gradient value is less than the target loss value, determining that the gradient update of the parameters of the preset pruning layer is completed.

4. The pruning method of claim 3, wherein, Before the target loss value is determined by using the target loss function based on the preset penalty factor and the preset sparse loss, the method further comprises: The target loss function is constructed based on the initial loss function, the product of the initial sparse loss and the initial penalty factor, and the product of the preset sparse loss and the preset penalty factor, wherein the preset sparse loss is obtained by accumulating the sum of squares of the first weight tensor and the sum of squares of the second weight tensor.

5. The pruning method of claim 4, wherein, Before the first redundant channel set in the preset pruning layer is screened, the method further comprises: The gradient of the parameter of the original network layer is updated by using a preset gradient descent strategy. In a case where the gradient value of the parameter of the original network layer is less than the loss value obtained by the initial loss function, it is determined that the gradient update of the parameter of the original network layer is completed.

6. A target model-based pruning apparatus characterized by comprising: Comprise: A determination unit is configured to determine a to-be-pruned layer in a target model, wherein the target model is a trained neural network model, the target model is applied to the field of image processing, and is a neural network model for processing image files. An adding unit is configured to add a preset pruning layer after each to-be-pruned layer to obtain a target network structure, wherein the target network structure comprises a plurality of network layers, and each network layer is an original network layer or the preset pruning layer. A screening unit is configured to screen a first redundant channel set in the preset pruning layer in a case where the gradient update of the parameter of each network layer in the target network structure is completed. A pruning unit is configured to determine a second redundant channel set in the to-be-pruned layer based on the first redundant channel set, and prune each second redundant channel in the second redundant channel set to obtain a pruned target model, wherein the pruned target model is deployed on an embedded device, an Internet of Things device, a computing capability limited device, or an edge device, and the pruned target model is used to process a preset file, which is an image file. The screening unit comprises a fifth determination module configured to determine a metric standard value based on a preset pruning rate, a sixth determination module configured to determine a weight value of each channel in the preset pruning layer based on a first weight tensor of the preset pruning layer and a second weight tensor of the to-be-pruned layer located at a next layer of the preset pruning layer, a first sorting module configured to sort all the weight values to obtain a sorting result, and a seventh determination module configured to determine the first redundant channel set based on the sorting result and the metric standard value. The pruning unit comprises an eighth determination module configured to determine a pruning result of the preset pruning layer based on the first redundant channel set, a first merging module configured to merge the pruning result to the to-be-pruned layer to obtain an output result of the to-be-pruned layer, and a ninth determination module configured to determine the second redundant channel set based on the output result.

7. A computer readable storage medium characterized by The computer readable storage medium comprises a stored computer program, wherein the computer readable storage medium controls a device where the computer readable storage medium is located to perform the target model based pruning method in any one of claims 1 to 5 when the computer program is running.

8. An electronic device, comprising: The computer readable storage medium comprises a stored computer program, wherein the computer readable storage medium controls a device where the computer readable storage medium is located to perform the target model based pruning method in any one of claims 1 to 5 when the computer program is running.

Images