Cross-layer measurement deep convolutional neural network channel pruning method

Abstract

The invention relates to the technical field of deep learning, and discloses a cross-layer measurement deep convolutional neural network channel pruning method, and the method comprises the steps: carrying out the pre-training of a deep convolutional neural network model, and obtaining a to-be-pruned network model; selecting a significant convolution kernel according to statistical distribution characteristics of convolution kernel norms; performing cross-layer measurement on the channel based on the selected significant convolution kernel; constructing the layer pruning sensitivity by using the BN layer full parameters, and partitioning the pruning rate of each layer by using the overall pruning rate; performing channel pruning; according to the method, through selection of a significant convolution kernel, reduction of interference caused by a non-significant convolution kernel, cross-layer measurement of channel importance, comprehensive measurement of the importance degree of a channel in the whole model, evaluation of pruning sensitivity of each layer by using BN layer full parameters, and partitioning of the pruning rate, subjectivity of manual setting of the pruning rate is avoided; and the channel is pruned, and knowledge is applied to distillation and fine adjustment, so that a high-performance lightweight model is obtained.

Description

Technical Field

[0001] The present invention relates to the field of deep learning technology, and in particular to a cross-layer metric deep convolutional neural network channel pruning method. Background Art

[0002] In recent years, deep convolutional neural networks (DCNNs) and their derivative technologies have continued to trigger a research boom in the field of artificial intelligence, and have continuously refreshed the performance ceiling in core visual tasks such as image classification, object detection, and image segmentation. With the vigorous development of technologies such as the Internet of Things (IoT), autonomous driving, and smart terminals, the intelligent upgrade of edge devices has put forward an urgent need for lightweight deployment of deep learning models. However, high-performance network models are often accompanied by a large number of parameters and computational complexity, making them difficult to run efficiently on edge devices with limited computing power and storage capacity (such as embedded chips and mobile terminals). How to break through the hardware resource bottleneck and achieve efficient deployment of the model while maintaining model performance has become a key challenge in the current application of artificial intelligence.

[0003] Model compression technology is a core solution to this contradiction. This technology can significantly reduce the number of model parameters and computational complexity while maintaining model prediction accuracy, ultimately resulting in a lightweight model suitable for edge computing. Currently, mainstream model compression methods include parameter quantization, knowledge distillation, low-rank decomposition, and network pruning. Network pruning, a commonly used method in model compression, removes redundant parameters from the network by constructing specific evaluation criteria, thereby reducing the model size. Therefore, how to prune deep convolutional neural network structures has become a pressing issue. Summary of the Invention

[0004] In view of this, the present invention provides a cross-layer metric deep convolutional neural network channel pruning method to solve the problem of how to prune the deep convolutional neural network structure.

[0005] In a first aspect, the present invention provides a cross-layer metric deep convolutional neural network channel pruning method, the method comprising:

[0006] Pre-train the deep convolutional neural network model to obtain the network model to be pruned;

[0007] In each convolutional layer of the network model to be pruned, a significant convolution kernel is selected based on the statistical distribution characteristics of the convolution kernel norm;

[0008] Based on the selected significant convolution kernels, the channels are measured across layers and the importance metrics are calculated.

[0009] Use all parameters of the BN layer to construct the layer pruning sensitivity, and use the overall pruning rate to allocate the pruning rate of each layer to confirm the pruning rate of each layer;

[0010] Channel pruning is performed based on the channel importance metric of each convolutional layer and the pruning rate of each layer;

[0011] Combine knowledge distillation to fine-tune the pruned network model.

[0012] The present invention selects significant convolution kernels for the convolutional layer of the network model to be pruned obtained by pre-training to reduce the interference caused by non-significant convolution kernels when measuring importance, performs cross-layer measurement on the channels to comprehensively measure the importance of the channels in the entire model, and uses the full parameters of the BN layer to evaluate the pruning sensitivity of each layer when setting the pruning rate. The overall pruning rate is used to allocate the pruning rate of each layer, avoiding the subjectivity of manually setting the pruning rate of each layer. The channel is pruned and fine-tuned using knowledge distillation to reduce the impact of performance degradation caused by model pruning, thereby obtaining a high-performance lightweight model and realizing pruning of the deep convolutional neural network structure.

[0013] In an optional implementation, performing cross-layer measurement on the channel and calculating the importance measurement value includes:

[0014] Calculate the importance metric of each channel generation side;

[0015] Calculate the importance metric of each channel receiving side;

[0016] A comprehensive channel importance metric value is calculated based on the importance metric value of each channel generation side and the importance metric value of each channel reception side, or based on the importance metric value of each channel generation side.

[0017] The present invention realizes cross-layer channel importance measurement by calculating the importance measurement values ​​of each channel generation side and receiving side respectively, overcoming the problem of perspective limitation of traditional single-layer evaluation system and achieving more reasonable evaluation of channels.

[0018] In an optional implementation, the generation-side importance metric value is calculated according to the following formula:

[0019]

[0020] in, is the generation-side importance metric of the n-th output channel of the l-th convolutional layer, is the L1 norm of the mth convolution kernel of the nth filter in the lth convolution layer, M l is the number of input channels of the lth convolutional layer, Used to characterize whether the mth convolution kernel of the nth filter in the lth convolution layer is a significant convolution kernel.

[0021] The present invention reflects the channel generation quality of the current layer by calculating the generation side importance metric value.

[0022] In an optional implementation, the receiving side importance metric value is calculated according to the following formula:

[0023]

[0024] in, is the receiving side importance metric of the n-th output channel of the l-th convolutional layer, is the L1 norm of the i-th convolution kernel of the n-th group in the l-th convolution layer, N l+1 is the number of output channels of the l+1th convolutional layer, Used to indicate whether the i-th convolution kernel of the n-th group in the l+1-th convolution layer is a significant convolution kernel.

[0025] The present invention reflects the channel reception quality of the current layer by calculating the receiving side importance metric value.

[0026] In an optional embodiment, calculating a comprehensive channel importance metric value based on the importance metric value of each channel generation side and the importance metric value of each channel reception side, or based on the importance metric value of each channel generation side, includes:

[0027] For the last layer output channel, the generated side importance metric is determined as the comprehensive importance metric of the channel;

[0028] For non-last layer output channels, the product of the importance metric value on the generation side and the importance metric value on the reception side is determined as the comprehensive importance metric value of the channel.

[0029] The present invention comprehensively measures the importance of the channel in the entire model from the two perspectives of feature generation and transmission by jointly generating the importance metric value on the side and receiving the importance metric value on the side.

[0030] In an optional implementation, the channel comprehensive importance metric is calculated according to the following formula:

[0031]

[0032] in, is the comprehensive importance measure of the n-th output channel of the l-th convolutional layer, To generate the side importance measure, is the importance metric of the receiving side, and L represents the total number of layers in the network.

[0033] The present invention calculates a comprehensive importance metric value instead of the traditional measurement based on the generating side or the receiving side alone to comprehensively evaluate the importance of the channel.

[0034] In an optional embodiment, channel pruning is performed based on the channel importance metric of each convolutional layer and the pruning rate of each layer, including:

[0035] According to the number of channels in each convolutional layer and the pruning rate of each layer, calculate the number of channels that need to be pruned in each convolutional layer;

[0036] Compare the channel importance metric of each layer with the channel importance metric of the preset position in the layer to obtain the pruning mask of each layer channel. The pruning mask is used to indicate whether the channel needs to be retained or pruned.

[0037] Prune the channels that need to be pruned.

[0038] The present invention calculates the number of channels that need to be pruned in each layer and performs pruning according to the calculated number of channels to be pruned. According to the comparison result of the channel importance metric value and the channel metric value of the preset position in the layer, the channels that do not need to be retained are determined to achieve channel pruning.

[0039] In a second aspect, the present invention provides a deep convolutional neural network channel pruning system with cross-layer metrics, the system comprising:

[0040] Model pre-training module, used to pre-train the deep convolutional neural network model to obtain the network model to be pruned;

[0041] The significant convolution kernel selection module is used to select significant convolution kernels in each convolution layer of the network model to be pruned based on the statistical distribution characteristics of the convolution kernel norm;

[0042] Importance measurement module, used to perform cross-layer measurement on channels based on the selected significant convolution kernels and calculate the importance measurement value;

[0043] The pruning rate confirmation module is used to construct the layer pruning sensitivity using all parameters of the BN layer, and use the overall pruning rate to allocate the pruning rate of each layer to confirm the pruning rate of each layer;

[0044] The pruning module is used to perform channel pruning based on the channel importance metric of each convolutional layer and the pruning rate of each layer;

[0045] The fine-tuning module is used to fine-tune the pruned network model in combination with knowledge distillation.

[0046] In a third aspect, the present invention provides a computer device comprising: a memory and a processor, the memory and the processor being communicatively connected to each other, the memory storing computer instructions, and the processor executing the computer instructions to execute the cross-layer metric deep convolutional neural network channel pruning method of the above-mentioned first aspect or any corresponding embodiment thereof.

[0047] In a fourth aspect, the present invention provides a computer-readable storage medium having computer instructions stored thereon, the computer instructions being used to enable a computer to execute the cross-layer metric deep convolutional neural network channel pruning method of the above-mentioned first aspect or any corresponding embodiment thereof.

[0048] In a fifth aspect, the present invention provides a computer program product comprising computer instructions, the computer instructions being used to enable a computer to execute the cross-layer metric deep convolutional neural network channel pruning method of the above-mentioned first aspect or any corresponding embodiment thereof. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] In order to more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the specific embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0050] Figure 1 1 is a flow chart of a cross-layer metric deep convolutional neural network channel pruning method according to an embodiment of the present invention;

[0051] Figure 2 is a signal flow graph of two adjacent convolutional layers according to an embodiment of the present invention;

[0052] Figure 3 : This is a visualization diagram of the L1 norm size of the convolution kernel of the second convolutional layer of VGG16 according to an embodiment of the present invention;

[0053] Figure 4 1 is a structural block diagram of a deep convolutional neural network channel pruning system based on cross-layer metrics according to an embodiment of the present invention;

[0054] Figure 5 Schematic diagram of the hardware structure of a computer device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0055] To make the purpose, technical solutions, and advantages of the embodiments of the present invention more clear, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without making creative efforts shall fall within the scope of protection of the present invention.

[0056] In intelligent applications, deep network models are key to enhancing the intelligence of end devices. However, these models, due to their large number of parameters, pose deployment challenges on resource-constrained devices. They not only require extensive storage and computing resources, but also increase energy consumption, impacting device performance. Model compression technology effectively reduces computational complexity by reducing the number and size of parameters without compromising performance. This makes models easier to deploy on end devices, improves efficiency, and promotes the expanded application of intelligent technologies. Advances in this technology are crucial for promoting the widespread adoption of artificial intelligence.

[0057] Network pruning is a commonly used method for model compression. It removes redundant parameters from the network by establishing specific evaluation criteria, thereby reducing the model size. Based on the granularity of pruning, network pruning can be broadly divided into four categories: weight pruning, convolution kernel pruning, filter / channel pruning, and layer pruning.

[0058] Among them, weight pruning is called unstructured pruning, while the remaining convolution kernel pruning, filter / channel pruning, and layer pruning all fall under the category of structured pruning. Weight pruning first evaluates the importance of the parameters in the model, then sets those that are not important to zero. Pruning is performed at a very fine level to generate a highly sparse parameter matrix. Convolution kernel pruning uses the two-dimensional convolution kernel as the basic unit. According to pre-set evaluation criteria, the parameters of the unimportant convolution kernel are set to zero, but these parameters are still retained in the model. Because layer pruning has a significant impact on the network architecture, it is rarely used directly in practical applications.

[0059] Filter / channel pruning achieves model compression by directly removing redundant filters / channels in the convolutional layer. The pruned network is easy to optimize using the existing computing architecture, which makes it an important direction in the field of network pruning.

[0060] In response to the problem of limited perspective in traditional single-layer evaluation systems, an embodiment of the present invention provides a cross-layer metric channel pruning method for deep convolutional neural networks. In terms of channel importance measurement, the output channel convolution kernel of the current layer and the input channel convolution kernel of the next layer are used to establish a joint measurement model for the generation side and the reception side. In terms of pruning rate design, the present invention constructs layer pruning sensitivity through all parameters of the BN layer, and uses the overall pruning rate to allocate the pruning rate of each layer. In terms of fine-tuning after pruning, the knowledge distillation method is used as a supplement, and the original network model is used as the teacher model, and the pruned network is used as the student model to further reduce the performance degradation caused by model pruning, and finally obtain a high-performance lightweight model.

[0061] According to an embodiment of the present invention, an embodiment of a deep convolutional neural network channel pruning method with cross-layer metrics is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in an order different from that shown here.

[0062] In this embodiment, a cross-layer metric deep convolutional neural network channel pruning method is provided. Figure 1 is a flow chart of a deep convolutional neural network channel pruning method based on cross-layer metrics according to an embodiment of the present invention. Figure 1 As shown, the process includes the following steps:

[0063] Step S101: pre-train the deep convolutional neural network model to obtain a network model to be pruned.

[0064] In this embodiment of the present invention, a deep convolutional neural network model is pre-trained to obtain a network model to be pruned. Network models include single-branch VGGNet networks, multi-branch residual networks (ResNet), and multi-branch densely connected networks (DenseNet). These network models can be used for computer vision tasks such as image classification and object detection. Using existing deep learning neural network training frameworks, such as Pytorch, the selected network model is trained to obtain the original model, which also serves as the teacher model for knowledge distillation.

[0065] Step S102: In each convolutional layer of the network model to be pruned, a significant convolution kernel is selected according to the statistical distribution characteristics of the convolution kernel norm.

[0066] In an embodiment of the present invention, within the convolution layer, there are some convolution kernels with weak information transmission capabilities. These convolution kernels are like "noise" and are referred to as insignificant convolution kernels in this embodiment of the present invention. Regardless of whether these convolution kernels exist, they have little effect on the quality of the corresponding output channels. When evaluating importance, if insignificant convolution kernels are taken into consideration, it will interfere with the accurate judgment of importance. Therefore, this embodiment of the present invention first divides the convolution kernels into significant convolution kernels and insignificant convolution kernels based on their information transmission efficiency, and then only uses the significant convolution kernels to measure the importance of the channel.

[0067] Step S103: Perform cross-layer measurement on the channels based on the selected significant convolution kernels to calculate the importance measurement value.

[0068] In an embodiment of the present invention, in a DCNN, the output channel of the current layer also serves as the input channel of the next layer. The importance of the filter is measured by the convolution kernel of the current layer, which is equivalent to measuring the channel redundancy from the perspective of the output channel. The output channel convolution kernel of the current layer and the input channel convolution kernel of the next layer are used to establish a joint measurement model of the generating side and the receiving side, perform cross-layer measurement, and calculate the importance measurement value.

[0069] In step S104, all parameters of the BN layer are used to construct the layer pruning sensitivity, and the overall pruning rate is used to allocate the pruning rate of each layer to determine the pruning rate of each layer.

[0070] In the present invention, research has shown that different convolutional layers in a DCNN have different sensitivities to pruning. Excessive pruning of highly sensitive layers can lead to a significant drop in model accuracy that cannot be recovered through fine-tuning. Therefore, it is essential to set an appropriate pruning rate for each convolutional layer.

[0071] Most current pruning methods manually set the pruning rate for each layer, which is highly subjective. To establish a relationship between the sensitivity of a convolutional layer and its pruning rate, this embodiment of the present invention uses the scaling factor and offset factor of the Batch Normalization (BN) layer to construct a layer pruning sensitivity evaluation. Finally, by setting the overall pruning rate, the pruning rate of each convolutional layer is assigned to determine the pruning rate for each layer.

[0072] Step S105: Perform channel pruning based on the channel importance metric of each convolutional layer and the pruning rate of each layer.

[0073] In an embodiment of the present invention, after obtaining the channel importance metric value vector of each convolutional layer and the pruning rate of each layer, the number of channels that need to be pruned in each convolutional layer is calculated, and the channels of each convolutional layer are pruned.

[0074] Step S106: Fine-tune the pruned network model in combination with knowledge distillation.

[0075] In an embodiment of the present invention, during the fine-tuning stage of the network model after pruning, a strategy combining knowledge distillation and fine-tuning is adopted to organically combine the advantages of the two compression methods, ultimately achieving better model performance and compression effect.

[0076] Knowledge distillation, a model compression method in deep learning, aims to transfer the knowledge contained in a large, complex model (the teacher model) to a small, lightweight model (the student model). The key to knowledge distillation is minimizing the difference in the probability distributions output by the teacher and student models.

[0077] The original model is set as the teacher model, and the pruned model is used as the student model. Through fine-tuning, a lightweight network with excellent performance is finally obtained.

[0078] The cross-layer metric deep convolutional neural network channel pruning method provided in this embodiment selects significant convolution kernels for the convolution layer of the pre-trained network model to be pruned to reduce the interference caused by non-significant convolution kernels when measuring importance, performs cross-layer measurement on the channel to comprehensively measure the importance of the channel in the entire model, and uses the full parameters of the BN layer to evaluate the pruning sensitivity of each layer when setting the pruning rate. The overall pruning rate is used to allocate the pruning rate of each layer, avoiding the subjectivity of manually setting the pruning rate of each layer. The channel is pruned and fine-tuned using knowledge distillation to reduce the impact of performance degradation caused by model pruning, thereby obtaining a high-performance lightweight model and realizing pruning of the deep convolutional neural network structure.

[0079] This embodiment provides a cross-layer metric deep convolutional neural network channel pruning method, which includes the following steps:

[0080] Step S201: pre-train the deep convolutional neural network model to obtain a network model to be pruned.

[0081] For details, please see Figure 1 Step S101 of the illustrated embodiment will not be described in detail here.

[0082] Step S202: In each convolutional layer of the network model to be pruned, a significant convolution kernel is selected according to the statistical distribution characteristics of the convolution kernel norm.

[0083] Specifically, the above step S202 includes:

[0084] In step S2021, the convolution kernel of each convolution layer of the network model to be pruned is measured using the L1 norm.

[0085] In step S2022, the statistical distribution of the L1 norm of the convolution kernel of each layer is fitted, the corresponding mean and variance are calculated, and modeling is performed to obtain the threshold for selecting significant convolution kernels in each convolution layer.

[0086] In step S2023, a threshold of a significant convolution kernel is selected based on each convolution layer, and a binary labeling matrix is ​​used to label the significant convolution kernel in each convolution layer.

[0087] In this embodiment of the present invention, the convolution kernels of each convolutional layer of the network model to be pruned are measured using the L1 norm. After calculating the convolution kernel norm of each layer, significant convolution kernels are selected within the convolution layer. By fitting the statistical distribution of the L1 norm of the convolution kernels of each layer, the corresponding mean and variance are obtained, based on which the significant convolution kernel norm threshold for each layer is calculated.

[0088] After obtaining the mean and variance of the convolution kernel of each convolution layer, we model it and obtain the threshold for selecting significant convolution kernels for each convolution layer:

[0089]

[0090] Among them, T l is the selection threshold of the significant convolution kernel of the lth convolution layer, and c is an adjustable parameter used to adjust the judgment strength of the non-significant convolution kernel, so that it can more flexibly select the most appropriate threshold in different tasks and different networks.

[0091] After obtaining the threshold of each layer, a binary labeling matrix D is used in each convolutional layer to label the significant convolution kernels.

[0092] The significant convolution kernels are labeled using a binary labeling matrix according to the following formula:

[0093]

[0094] Among them, D l is the label matrix of the significant convolution kernel of the lth convolution layer, Represents the L1 norm of the m-th convolution kernel in the n-th filter of the l-th convolution layer.

[0095] When the L1 norm of the convolution kernel in the convolution layer is greater than or equal to the threshold of the layer, the position is marked as 1. When the L1 norm of the convolution kernel in the convolution layer is less than the threshold of the layer, the position is marked as 0. Finally, the binary label matrix of each convolution layer is obtained.

[0096] Step S203: Perform cross-layer measurement on the channels based on the selected significant convolution kernels to calculate the importance measurement value.

[0097] Specifically, the above step S203 includes:

[0098] Step S2031 , calculating the importance metric value of each channel generation side.

[0099] Step S2032: Calculate the receiving side importance metric value of each channel.

[0100] Step S2033 : Calculate a comprehensive channel importance metric value based on the importance metric value on the generating side of each channel and the importance metric value on the receiving side of each channel, or based on the importance metric value on the generating side of each channel.

[0101] In an embodiment of the present invention, in a DCNN, the output channel of the current layer also serves as the input channel of the next layer. The importance of the filter is measured by the convolution kernel of the current layer, which is equivalent to measuring the channel redundancy from the perspective of the output channel. In fact, all channels other than the last layer correspond to two sets of convolution kernels, one set is the output channel convolution kernel of the current layer (i.e., filter), and the other set is the input channel convolution kernel of the next layer (i.e., group). The combined use of these two sets of convolution kernels can overcome the problem of perspective limitation of the traditional single-layer evaluation system and make a more reasonable evaluation of the channel.

[0102] In DCNN, the role of convolution kernel is to extract information from the input channel and transfer it to the output channel. In the same convolution layer, a group of convolution kernels that transfer information from different input channels to the same output channel is usually called a filter, and a group of convolution kernels that transfer information from the same input channel to different output channels is usually called a group. The essence of feature transfer is achieved through channel connections between layers. The output channel Y of layer l is l It also serves as the input X of the l+1 layer l+1 ,This design makes each intermediate channel take on two key roles at the same time: generation role and transmission role. When serving as a generation role, the output channel Y l By X l Generated by the convolution kernel of the lth layer. When used as a transmission role, X l+1 It provides information carrier for subsequent feature extraction, and its signal flow diagram is as follows Figure 2 shown.

[0103] like Figure 2 As shown, for an intermediate channel, it can be divided into the generating side and the receiving side. The generating side process is that the current layer filter convolution kernel extracts the information of the current layer input channel and converges to obtain the output channel Y l ,like Figure 2 The process on the receiving side is X l+1 (Y l) is used as the input channel of the next layer and is extracted by the convolution kernel of the next layer and distributed to different output channels of the next layer, such as Figure 2 The importance of cross-layer metrics can be analyzed from both the generation side and the reception side perspectives.

[0104] From a generation perspective, the convolution kernel's parameter settings and complex computational processes during the generation of output channel information play a decisive role in determining the type and richness of the extracted features. The kernel's norm is a key metric, providing a direct reflection of the kernel's ability to extract features from the input channel.

[0105] In many previous filter pruning methods, the L1 or L2 norm is often used to measure the feature extraction ability of the convolution kernel. The importance of the filter is then measured by calculating the sum of the norms of all the convolution kernels of the filter. Generally speaking, the larger the filter norm, the more critical the feature channels extracted and generated by the filter, and the more important they should be retained during model optimization. This evaluation method mainly focuses on the set of output channel convolution kernels (filters) of the current layer, essentially evaluating the input signal of the output channel of the current layer. Therefore, this method has limitations. It only focuses on the generation ability of the current layer and ignores the transmission of features throughout the network.

[0106] From the receiving side, the ability of the downstream layer to receive and process input information is a key link in the performance of the entire model. When each layer receives information from the previous layer, it will screen, integrate, and further extract deep features based on its own task requirements and unique structural characteristics. For example, in a deep neural network, deeper networks often require more abstract and advanced features to complete complex classification or detection tasks. The output channel of the current layer also serves as the input channel of the next layer. The information it carries will be extracted by a group of input channel convolution kernels (group) of the next layer and distributed to different output channels of the next layer. If the feature extraction capability of the group of the next layer is poor, such as Figure 3 The boxed image indicates that the corresponding input channel in the next layer has weak signal outflow capabilities, which in turn reflects the insufficient signal outflow capabilities of the output channel in the current layer, and conversely, it can be inferred that the output channel in the current layer is not important. This clearly demonstrates that it is impossible to accurately determine the importance of intermediate channels if evaluation is based solely on the generation side.

[0107] The importance of cross-layer metrics lies in their ability to comprehensively consider both the generation and reception sides, transcending the limitations of traditional single-perspective approaches. Cross-layer metrics forge close connections between layers, accurately assessing how effectively generated features propagate throughout the network and how effectively they are utilized by the reception side. This in turn improves the accuracy of assessing the importance of intermediate channels.

[0108] After obtaining the significant convolution kernels for each layer, we establish a generation-side importance metric and a reception-side importance metric for each channel in each layer. First, we construct the channel generation-side importance metric. This metric is performed on each set of output channel convolution kernels (filters) in the current layer. Each output channel in the current layer corresponds to a set of output channel convolution kernels. Therefore, after using the significant convolution kernels, the generation-side importance metric for each channel is as follows:

[0109]

[0110] in, is the generation-side importance metric of the n-th output channel of the l-th convolutional layer, is the L1 norm of the mth convolution kernel of the nth filter in the lth convolution layer, M l is the number of input channels of the lth convolutional layer, Used to characterize whether the mth convolution kernel of the nth filter in the lth convolution layer is a significant convolution kernel.

[0111] Among them, if Indicates that the convolution kernel is significant and contributes significantly to feature extraction. This means that the convolution kernel is not significant.

[0112] It can reflect the channel generation quality of the current layer. The larger the value, the more important the channel is from the perspective of the output channel. When the value of reaches its maximum, it means that in the current convolutional layer, the generation capability of the corresponding n-th output channel is the strongest. This indicates that the filter convolution kernel involved in this channel performs best in extracting information from the input channel and aggregating it to generate output features, and can more effectively capture key features.

[0113] In addition to measuring the importance of the channel generation side, it is also necessary to establish an importance measure for the channel reception side. The reception side importance measure is performed on each group of input channel convolution kernels in the next layer. One output channel of the current layer corresponds to a group of input channel convolution kernels in the next layer. Therefore, after using the significant convolution kernel, the reception side importance measure of each channel is as follows:

[0114]

[0115] in, is the receiving side importance metric of the n-th output channel of the l-th convolutional layer, is the L1 norm of the i-th convolution kernel of the n-th group in the l-th convolution layer, N l+1 is the number of output channels of the l+1th convolutional layer, Used to indicate whether the i-th convolution kernel of the n-th group in the l+1-th convolution layer is a significant convolution kernel.

[0116] Among them, if Indicates that the convolution kernel is significant and contributes significantly to feature extraction. This means that the convolution kernel is not significant.

[0117] It can reflect the channel reception quality of the current layer. The larger the value of , the more important the channel is from the perspective of the input channel of the next layer. When the value of reaches its maximum, it means that the signal transmission capability of the nth output channel in the current convolution layer is the strongest. This indicates that the group convolution kernel in the next layer involved in this channel performs best in extracting information from the output channel of the current layer and distributing it to the output features of the next layer, and can transmit more features.

[0118] Specifically, the above step S2033 includes:

[0119] Step S20331: For the last layer of output channels, the generation-side importance metric value is determined as the comprehensive importance metric value of the channel.

[0120] Step S20332: For non-last layer output channels, the product of the generation side importance metric value and the reception side importance metric value is determined as the comprehensive importance metric value of the channel.

[0121] In an embodiment of the present invention, after obtaining the generation-side importance metric value and the receiving-side importance metric value of the channel, the two are combined to evaluate the importance of the channel. For all channels other than the last layer, there are generation-side importance metrics and receiving-side importance metrics. For the output channel of the last layer, since it does not have a group convolution kernel on the receiving side, the generation-side importance metric value is directly used. In summary, the importance metrics of these channels can be finally expressed by the following formula:

[0122]

[0123] Among them, L represents the total number of layers in the network. It is the comprehensive importance measure of the nth output channel of the lth convolutional layer. For non-last layers, this value is generated by the importance measure of the generation side. Importance metric value on the receiving side Multiply them together to get .

[0124] The generation-side importance metric focuses on the ability of the convolution kernel in the current layer to extract information from the input channel and generate the output channel, reflecting the quality of feature generation for that channel at the current layer. The reception-side importance metric, on the other hand, focuses on the ability of the next layer to receive and process the information from the current layer's output channel, reflecting the role of that channel in the information transfer process. Multiplying the two together provides a comprehensive assessment of a channel's importance within the entire model from the two key perspectives of feature generation and transfer. For example, if a channel has a high generation-side importance metric but a low reception-side importance metric, it indicates that while it effectively generates features, it is not effective in transferring them to the next layer. Overall, its importance may not be high. Only when both the generation-side and reception-side evaluations are high does the channel truly play a key role in the model. Compared to traditional approaches that rely solely on generation-side or reception-side metrics, this comprehensive evaluation approach can better measure channels with strong generation capabilities but poor transfer performance, or channels with good transfer performance but weak generation capabilities. This allows for more effective model optimization and efficiency while maintaining performance.

[0125] In step S204, the layer pruning sensitivity is constructed using all the parameters of the BN layer, and the pruning rate of each layer is allocated using the overall pruning rate to determine the pruning rate of each layer.

[0126] Specifically, the above step S204 includes:

[0127] Step S2041: Calculate the pruning sensitivity score of each set of BN parameters corresponding to each convolutional layer.

[0128] Step S2042: Perform maximum and minimum normalization processing on the pruning sensitivity scores.

[0129] Step S2043: sort the global pruning sensitivity scores by size and set a pruning sensitivity flag for each layer.

[0130] Step S2044: Calculate the pruning rate of each layer according to the pruning sensitivity flag of each layer.

[0131] In the embodiment of the present invention, the scaling factor γ and the offset factor β are used to construct the layer pruning sensitivity. First, the pruning sensitivity score of each set of BN parameters corresponding to each convolutional layer is calculated, and the calculation formula is as follows:

[0132]

[0133] in, represents the scaling factor of the i-th group BN parameters in the l-th layer, Represents the offset factor of the BN parameter of the i-th group in the l-th layer, is the pruning sensitivity score of the i-th group of BN parameters in layer l. Since the impact of scaling factor and offset factor on pruning sensitivity may be different in different networks and tasks.

[0134] To better assess pruning sensitivity, the parameter μ is introduced to adjust the relative importance of the scaling factor and the offset factor. When μ is less than 0.5, the offset factor has a greater impact on pruning sensitivity. When μ is greater than 0.5, the scaling factor has a greater impact on pruning sensitivity. When μ is equal to 0.5, the scaling factor and the offset factor have an equal impact on pruning sensitivity. μ defaults to 0.5 and can be adjusted based on the actual pruning situation.

[0135] After obtaining the pruning sensitivity scores for each set of BN parameters, a global pruning rate R is set to allocate the pruning rate for each layer. To facilitate cross-layer comparison of the pruning sensitivity scores for each set of BN parameters and to ensure that at least one filter is retained in each layer, the pruning sensitivity scores are normalized to their maximum and minimum values ​​within each BN layer.

[0136]

[0137] Among them, I l min is the pruning sensitivity score of the smallest set of BN parameters in layer l, I l max is the pruning sensitivity score of the largest set of BN parameters in layer l, is the pruning sensitivity score of the i-th group BN parameters in the l-th layer after maximum and minimum normalization.

[0138] Assume that the lth layer of DCNN has N l filters, and the global pruning rate is R, which means that the network needs to be pruned. filters. When the global After that, the pruning rate of each layer can be distributed by the global pruning rate R. The distribution process is as follows:

[0139] Specifically, first set the global Sort from smallest to largest.

[0140]

[0141] in, is the pruning sensitivity score of the Pth position in the above sorting. A higher score indicates a higher sensitivity to pruning. Set the pruning sensitivity flag of the position before the Pth position to 1, indicating that it is not sensitive to pruning; set the pruning sensitivity flag of the position after the Pth position to 0, indicating that it is sensitive to pruning, as follows:

[0142]

[0143] Finally, the pruning rate of each layer can be obtained:

[0144]

[0145] Among them, p l Represents the pruning rate of layer l.

[0146] Step S205: Perform channel pruning based on the channel importance metric of each convolutional layer and the pruning rate of each layer.

[0147] Specifically, the above step S205 includes:

[0148] Step S2051: Calculate the number of channels to be pruned from each convolutional layer based on the number of channels in each convolutional layer and the pruning rate of each layer.

[0149] In step S2052 , the channel importance metric value of each layer is compared with the channel importance metric value of a preset position in the layer to obtain a pruning mask for each channel of the layer. The pruning mask is used to indicate whether the channel needs to be retained or pruned.

[0150] Step S2053: prune the channels that need to be pruned.

[0151] In the embodiment of the present invention, after obtaining the pruning rate p of each layer l After that, we can get the number of channels that need to be cut off in each layer:

[0152] P l =N l p l (11)

[0153] Among them, N l is the number of channels in the lth convolutional layer, p l is the pruning rate of the lth convolutional layer, P l is the number of channels that need to be cut off in the l-th convolutional layer.

[0154] After obtaining the number of pruning branches, the channels that need to be retained in each layer can be obtained by comparing the channel importance metric value of each layer with the p-th largest channel importance metric value in its layer.

[0155] Specifically, the channel importance metrics of each layer are first sorted from low to high according to importance:

[0156]

[0157] in, is the importance metric of the channel at position p in the above sorting:

[0158] p=P l (13)

[0159] Finally, the pruning mask M of each layer channel is obtained:

[0160]

[0161] in, Indicates whether the n-th channel of the l-th layer needs to be pruned.

[0162] when When it is 1, the corresponding channel needs to be reserved. When it is 0, the corresponding channel needs to be cut off.

[0163] Step S206: Fine-tune the pruned network model in combination with knowledge distillation.

[0164] For details, please see Figure 1 Step S106 of the illustrated embodiment will not be described in detail here.

[0165] This embodiment provides a cross-layer metric deep convolutional neural network channel pruning method that effectively compresses and accelerates network models, significantly reducing the number of model parameters and floating-point operations with little loss of accuracy. This method is applicable to a variety of common convolutional neural networks, such as VGG16, ResNet56, and DenseNet40, as well as datasets of varying sizes, such as the small, simple CIFAR-10 dataset, the small, complex CIFAR-100 dataset, and the large, complex ImageNet dataset.

[0166] In this embodiment, a cross-layer metric deep convolutional neural network channel pruning system is also provided, which is used to implement the above-mentioned embodiments and preferred implementation methods, and will not be repeated here. As used below, the term "module" can implement a combination of software and / or hardware that implements a predetermined function. Although the system described in the following embodiments is preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceived.

[0167] This embodiment provides a cross-layer metric deep convolutional neural network channel pruning system, such as Figure 4 As shown, including:

[0168] The model pre-training module 401 is used to pre-train the deep convolutional neural network model to obtain a network model to be pruned.

[0169] The significant convolution kernel selection module 402 is used to select significant convolution kernels in each convolution layer of the network model to be pruned according to the statistical distribution characteristics of the convolution kernel norm.

[0170] The importance measurement module 403 is used to perform cross-layer measurement on the channels based on the selected significant convolution kernels and calculate the importance measurement value.

[0171] The pruning rate confirmation module 404 is used to construct the layer pruning sensitivity using all parameters of the BN layer, and use the overall pruning rate to allocate the pruning rate of each layer to confirm the pruning rate of each layer.

[0172] The pruning module 405 is used to perform channel pruning based on the channel importance metric of each convolutional layer and the pruning rate of each layer.

[0173] The fine-tuning module 406 is used to fine-tune the pruned network model in combination with knowledge distillation.

[0174] In some optional implementations, the importance measurement module 403 includes:

[0175] The first calculation unit is used to calculate the importance metric value of each channel generation side.

[0176] The second calculation unit is used to calculate the importance metric value of each channel receiving side.

[0177] The third calculation unit is configured to calculate a channel comprehensive importance metric value based on the generation-side importance metric value of each channel and the reception-side importance metric value of each channel, or based on the generation-side importance metric value of each channel.

[0178] In some optional implementations, the third computing unit includes:

[0179] The first determining subunit is configured to determine, for the last layer output channel, the generation-side importance metric value as the comprehensive importance metric value of the channel.

[0180] The second determining subunit is configured to determine, for a non-last layer output channel, a product of a generation side importance metric value and a reception side importance metric value as a comprehensive importance metric value of the channel.

[0181] In some optional implementations, the pruning module 405 includes:

[0182] The third calculation unit is used to calculate the number of channels to be pruned from each convolutional layer according to the number of channels in each convolutional layer and the pruning rate of each layer.

[0183] The comparison unit is used to compare the channel importance metric value of each layer with the channel importance metric value of the preset position in the layer to obtain the pruning mask of each layer channel. The pruning mask is used to indicate whether the channel needs to be retained or pruned.

[0184] The pruning unit is used to prune the channels that need to be pruned.

[0185] The further functional description of each of the above modules and units is the same as that of the above corresponding embodiments and will not be repeated here.

[0186] The cross-layer metric deep convolutional neural network channel pruning system in this embodiment is presented in the form of functional units, where the units refer to ASIC (Application Specific Integrated Circuit) circuits, processors and memories that execute one or more software or fixed programs, and / or other devices that can provide the above functions.

[0187] The embodiment of the present invention also provides a computer device having the above Figure 4 The cross-layer metric deep convolutional neural network channel pruning system shown.

[0188] See also Figure 5 , Figure 5 is a structural diagram of a computer device provided by an optional embodiment of the present invention, such as Figure 5 As shown, the computer device includes: one or more processors 10, memory 20, and interfaces for connecting various components, including high-speed interfaces and low-speed interfaces. Various components utilize different buses to communicate with each other and can be installed on a common mainboard or installed in other ways as needed. The processor can process the instructions executed in the computer device, including instructions stored in the memory or on the memory to display the graphical information of the GUI on an external input / output device (such as, a display device coupled to the interface). In some optional embodiments, if necessary, multiple processors and / or multiple buses can be used together with multiple memories and multiple memories. Equally, multiple computer devices can be connected, and each device provides part of the necessary operations (for example, as a server array, a group of blade servers, or a multi-processor system). Figure 5 A processor 10 is taken as an example.

[0189] The processor 10 may be a central processing unit, a network processor, or a combination thereof. The processor 10 may further include a hardware chip. The hardware chip may be an application-specific integrated circuit, a programmable logic device, or a combination thereof. The programmable logic device may be a complex programmable logic device, a field programmable gate array, a general purpose array logic, or any combination thereof.

[0190] The memory 20 stores instructions that can be executed by at least one processor 10, so as to enable at least one processor 10 to execute the method shown in the above embodiment.

[0191] The memory 20 may include a program storage area and a data storage area, wherein the program storage area may store an operating system and application programs required for at least one function; the data storage area may store data created based on the use of the computer device, etc. In addition, the memory 20 may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some optional embodiments, the memory 20 may optionally include a memory remotely located relative to the processor 10, and these remote memories may be connected to the computer device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0192] The memory 20 may include a volatile memory, such as a random access memory; the memory may also include a non-volatile memory, such as a flash memory, a hard disk or a solid-state drive; the memory 20 may also include a combination of the above types of memory.

[0193] The computer device also includes an input device 30 and an output device 40. The processor 10, the memory 20, the input device 30 and the output device 40 can be connected via a bus or other means. Figure 5 The bus connection is taken as an example.

[0194] The input device 30 can receive input digital or character information and generate key signal input related to user settings and function control of the computer device, such as a touch screen, etc. The output device 40 can include a display device, etc.

[0195] The embodiment of the present invention also provides a computer-readable storage medium. The above-mentioned method according to the embodiment of the present invention can be implemented in hardware, firmware, or implemented as a computer code that can be recorded in a storage medium, or implemented as a computer code that is originally stored in a remote storage medium or a non-temporary machine-readable storage medium and downloaded through a network and will be stored in a local storage medium, so that the method described herein can be stored in such software processing on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. Among them, the storage medium can be a magnetic disk, an optical disk, a read-only storage memory, a random access memory, a flash memory, a hard disk or a solid-state drive, etc.; further, the storage medium can also include a combination of the above-mentioned types of memory. It can be understood that a computer, a processor, a microprocessor controller or programmable hardware includes a storage component that can store or receive software or computer code. When the software or computer code is accessed and executed by a computer, a processor or hardware, the method shown in the above embodiment is implemented.

[0196] A portion of the present invention may be applied as a computer program product, such as a computer program instruction, which, when executed by a computer, can call or provide the method and / or technical solution according to the present invention through the operation of the computer. Those skilled in the art should understand that the form in which the computer program instruction exists in a computer-readable medium includes, but is not limited to, a source file, an executable file, an installation package file, etc. Accordingly, the way in which the computer program instruction is executed by the computer includes, but is not limited to: the computer directly executes the instruction, or the computer compiles the instruction and then executes the corresponding compiled program, or the computer reads and executes the instruction, or the computer reads and installs the instruction and then executes the corresponding installed program. Here, the computer-readable medium may be any available computer-readable storage medium or communication medium that can be accessed by the computer.

[0197] Although the embodiments of the present invention have been described with reference to the accompanying drawings, those skilled in the art may make various modifications and variations without departing from the spirit and scope of the present invention, and such modifications and variations are intended to fall within the scope of this application.

Claims

1. A cross-layer metric deep convolutional neural network channel pruning method, characterized in that: The method comprises: Pre-train the deep convolutional neural network model to obtain the network model to be pruned; In each convolutional layer of the network model to be pruned, a significant convolution kernel is selected based on the statistical distribution characteristics of the convolution kernel norm; Based on the selected significant convolution kernels, the channels are measured across layers and the importance metrics are calculated. Use all parameters of the BN layer to construct the layer pruning sensitivity, and use the overall pruning rate to allocate the pruning rate of each layer to confirm the pruning rate of each layer; Channel pruning is performed based on the channel importance metric of each convolutional layer and the pruning rate of each layer; Combine knowledge distillation to fine-tune the pruned network model.

2. The method according to claim 1, characterized in that The cross-layer measurement of the channel and calculation of the importance measurement value include: Calculate the importance metric of each channel generation side; Calculate the importance metric of each channel receiving side; A comprehensive channel importance metric value is calculated based on the importance metric value of each channel generation side and the importance metric value of each channel reception side, or based on the importance metric value of each channel generation side.

3. The method according to claim 2, characterized in that The importance metric value of the generation side is calculated according to the following formula: in, is the generation-side importance metric of the n-th output channel of the l-th convolutional layer, is the L1 norm of the mth convolution kernel of the nth filter in the lth convolution layer, M l is the number of input channels of the lth convolutional layer, Used to characterize whether the mth convolution kernel of the nth filter in the lth convolution layer is a significant convolution kernel.

4. The method according to claim 2, characterized in that The importance metric value of the receiving side is calculated according to the following formula: in, is the receiving side importance metric of the n-th output channel of the l-th convolutional layer, is the L1 norm of the i-th convolution kernel of the n-th group in the l-th convolution layer, N l+1 is the number of output channels of the l+1th convolutional layer, Used to indicate whether the i-th convolution kernel of the n-th group in the l+1-th convolution layer is a significant convolution kernel.

5. The method according to claim 2, characterized in that The calculating of a comprehensive channel importance metric value based on the importance metric value on the generation side of each channel and the importance metric value on the reception side of each channel, or based on the importance metric value on the generation side of each channel, includes: For the last layer output channel, the generated side importance metric is determined as the comprehensive importance metric of the channel; For non-last layer output channels, the product of the importance metric value on the generation side and the importance metric value on the reception side is determined as the comprehensive importance metric value of the channel.

6. The method according to claim 5, characterized in that The comprehensive channel importance metric is calculated according to the following formula: in, is the comprehensive importance measure of the n-th output channel of the l-th convolutional layer, To generate the side importance measure, is the importance metric of the receiving side, and L represents the total number of layers in the network.

7. The method according to claim 1, characterized in that The channel pruning is performed based on the channel importance metric value of each convolutional layer and the pruning rate of each layer, including: According to the number of channels in each convolutional layer and the pruning rate of each layer, calculate the number of channels that need to be pruned in each convolutional layer; Compare the channel importance metric of each layer with the channel importance metric of the preset position in the layer to obtain the pruning mask of each layer channel, where the pruning mask is used to indicate whether the channel needs to be retained or pruned; Prune the channels that need to be pruned.

8. A cross-layer metric deep convolutional neural network channel pruning system, characterized by: The system comprises: Model pre-training module, used to pre-train the deep convolutional neural network model to obtain the network model to be pruned; The significant convolution kernel selection module is used to select significant convolution kernels in each convolution layer of the network model to be pruned based on the statistical distribution characteristics of the convolution kernel norm; Importance measurement module, used to perform cross-layer measurement on channels based on the selected significant convolution kernels and calculate the importance measurement value; The pruning rate confirmation module is used to construct the layer pruning sensitivity using all parameters of the BN layer, and use the overall pruning rate to allocate the pruning rate of each layer to confirm the pruning rate of each layer; The pruning module is used to perform channel pruning based on the channel importance metric of each convolutional layer and the pruning rate of each layer; The fine-tuning module is used to fine-tune the pruned network model in combination with knowledge distillation.

9. A computer device, characterized in that: include: A memory and a processor, wherein the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the cross-layer metric deep convolutional neural network channel pruning method according to any one of claims 1 to 7 by executing the computer instructions.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium stores computer instructions, which are used to enable a computer to execute the cross-layer metric deep convolutional neural network channel pruning method described in any one of claims 1 to 7.

Images