Neural network pruning method and apparatus, electronic device, and storage medium

Abstract

The application discloses a neural network pruning method and device, electronic equipment and storage medium, and relates to the technical field of computers. The method comprises the following steps: acquiring a network to be pruned, and acquiring a sample data set, wherein the network to be pruned comprises a shallow network, a deep network and a pruning supervision network, and the sample data set comprises a plurality of sample data; inputting the plurality of sample data into the shallow network to obtain initial features output by the shallow network; inputting the initial features into the pruning supervision network to obtain a filter mask for the deep network output by the pruning supervision network; and inputting the filter mask into the deep network to prune the deep network based on the filter mask and obtain a pruned network. According to the application, the filter information of the shallow network in the network to be pruned is transmitted to the deep network in the network to be pruned through the filter mask, and then the deep network is pruned in a self-supervised constraint manner based on the filter mask, so that the performance of the pruned network and the efficiency of network training are improved.

Description

Technical Field

[0001] This application relates to the field of computer technology, and more specifically, to a neural network pruning method, apparatus, electronic device, and storage medium. Background Technology

[0002] With the continuous development of deep learning neural networks, large-scale neural networks have gradually demonstrated their effectiveness and high performance in the field of computer vision, showing excellent performance in various computer vision tasks, including classification, localization, detection, and segmentation. However, most neural networks are currently very complex in construction and have many branches, consuming a lot of computational resources and time, and putting great pressure on practical deployment. Summary of the Invention

[0003] In view of the above problems, this application proposes a neural network pruning method, apparatus, electronic device, and storage medium. By transmitting the filter information of the shallow network in the network to be pruned to the deep network in the network to be pruned through a filter mask, the deep network can be pruned based on the filter mask in a self-supervised constraint manner. This improves the performance of the pruned network and the efficiency of network training, enhances the computational efficiency of the network, and alleviates the deployment pressure of the network.

[0004] In a first aspect, embodiments of this application provide a neural network pruning method, the method comprising: acquiring a network to be pruned and acquiring a sample dataset, wherein the network to be pruned includes a shallow network, a deep network, and a pruning supervision network, and the sample dataset includes multiple sample data; inputting the multiple sample data into the shallow network to obtain initial features output by the shallow network; inputting the initial features into the pruning supervision network to obtain a filter mask output by the pruning supervision network for the deep network; and inputting the filter mask into the deep network to prune the deep network based on the filter mask to obtain a pruned network.

[0005] Secondly, embodiments of this application provide a neural network pruning device, comprising: a network to be pruned acquisition module, an initial feature acquisition module, a filter mask acquisition module, and a pruned network acquisition module. The network to be pruned acquisition module is used to acquire the network to be pruned and to acquire a sample dataset, wherein the network to be pruned includes a shallow network, a deep network, and a pruning supervision network, and the sample dataset includes multiple sample data. The initial feature acquisition module is used to input the multiple sample data into the shallow network to obtain initial features output by the shallow network. The filter mask acquisition module is used to input the initial features into the pruning supervision network to obtain a filter mask output by the pruning supervision network for the deep network. The pruned network acquisition module is used to input the filter mask into the deep network to prune the deep network based on the filter mask, thereby obtaining the pruned network.

[0006] Thirdly, embodiments of this application provide an electronic device, including a memory and a processor, wherein the memory is coupled to the processor, the memory stores instructions, and when the instructions are executed by the processor, the processor performs the above-described method.

[0007] Fourthly, embodiments of this application provide a computer-readable storage medium storing program code, which can be invoked by a processor to execute the above-described method.

[0008] The neural network pruning method, apparatus, electronic device, and storage medium provided in this application obtain a network to be pruned and a sample dataset. The network to be pruned includes a shallow network, a deep network, and a pruning supervision network. The sample dataset includes multiple sample data. The multiple sample data are input into the shallow network to obtain initial features output by the shallow network. The initial features are input into the pruning supervision network to obtain a filter mask for the deep network output by the pruning supervision network. The filter mask is input into the deep network to prune the deep network based on the filter mask, resulting in a pruned network. By transferring the filter information of the shallow network in the network to be pruned to the deep network in the network to be pruned through the filter mask, the deep network is pruned based on the filter mask under self-supervised constraints, thereby improving the performance of the pruned network and the efficiency of network training. Attached Figure Description

[0009] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0010] Figure 1 A flowchart illustrating a neural network pruning method provided in an embodiment of this application is shown;

[0011] Figure 2 This illustration shows a schematic diagram of the structure of a network to be pruned according to an embodiment of this application;

[0012] Figure 3 A flowchart illustrating a neural network pruning method provided in an embodiment of this application is shown;

[0013] Figure 4 A flowchart illustrating a neural network pruning method provided in an embodiment of this application is shown;

[0014] Figure 5 A block diagram of a neural network pruning device according to an embodiment of this application is shown;

[0015] Figure 6 A block diagram of an electronic device for performing a neural network pruning method according to an embodiment of this application is shown.

[0016] Figure 7 A storage unit for storing or carrying program code implementing the neural network pruning method according to an embodiment of the present application is shown. Detailed Implementation

[0017] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings.

[0018] With the rapid development of artificial intelligence, deep learning-based neural network models have been widely applied. Large-scale deep convolutional neural network models have achieved excellent performance in tasks such as image recognition and object detection; among them, neural networks have shown outstanding performance in various computer vision tasks, including classification, localization, detection, and segmentation. However, most current neural networks are very complex in construction, with numerous branches, consuming a large amount of computational resources and time, and placing a great burden on practical deployment. Related technologies have proposed using pruning techniques to remove weights or structures that do not play a role in the neural network, thereby lightweighting the network without affecting its performance, improving computational efficiency, and alleviating the deployment burden.

[0019] In related technologies, pruning techniques can be divided into structured pruning and unstructured pruning; the main difference lies in the granularity of the pruned weights. Structured pruning primarily prunes the channels and filters of convolutional kernels, altering the entire network structure; while unstructured pruning mainly prunes individual weights. Unstructured pruning can achieve higher compression ratios and preserve model performance more completely, but its hardware acceleration effect is relatively poor for sparse structures; while structured pruning can effectively achieve hardware acceleration for sparse structures.

[0020] Pruning techniques can also be divided into static pruning and dynamic pruning based on the training method. Static pruning involves removing unimportant branches or weights based on the evaluation results of the weights after training the entire training set; while dynamic pruning involves ignoring unimportant network branches or weights based on different input samples each time, thus achieving a dynamic pruning process.

[0021] In related technologies, pruning algorithms can be categorized into post-training pruning, pre-training pruning, and in-training pruning. Post-training pruning refers to pruning the model after weight training is complete. However, pruning after training significantly increases training time and incurs additional training costs. Pre-training pruning, on the other hand, can lead to decreased accuracy in judging the importance of branches and weights, removing potentially high-performance branches, resulting in instability, significant changes in model structure, and inaccurate pruning structures, ultimately degrading model performance. In-training pruning determines whether to prune and selects an appropriate pruning strategy based on the model's performance during training, resulting in high computational complexity and consuming substantial computing resources.

[0022] Therefore, in the application of pruning techniques, there are problems such as large model changes during the early training process, decreased model accuracy due to pruning, and huge computational load during pruning.

[0023] To address the aforementioned problems, the inventors, through long-term research, discovered and proposed the neural network pruning method, apparatus, electronic device, and storage medium provided in the embodiments of this application. By transmitting the filter information of the shallow network in the network to be pruned to the deep network in the network to be pruned through a filter mask, the deep network is pruned based on self-supervised constraints using the filter mask, thereby improving the performance of the pruned network and the efficiency of network training. The specific neural network pruning method will be described in detail in the subsequent embodiments.

[0024] Please see Figure 1 , Figure 1A flowchart illustrating a neural network pruning method according to an embodiment of this application is shown. This method transmits filter information from shallow layers of the network to be pruned to deeper layers via filter masks, thereby performing self-supervised pruning on the deeper layers based on the filter masks, thus improving the performance of the pruned network and the efficiency of model training. In specific embodiments, this neural network pruning method can be applied to, for example... Figure 5 The neural network pruning device 200 and the electronic device 100 equipped with the neural network pruning device 200 are shown. Figure 6 The following will use an electronic device as an example to illustrate the specific process of this embodiment. Of course, it is understood that the electronic device used in this embodiment may include smartphones, tablets, wearable electronic devices, etc., and is not limited thereto. The following will focus on... Figure 1 The process shown will be described in detail. The neural network pruning method may specifically include the following steps:

[0025] Step S110: Obtain the network to be pruned and obtain the sample dataset, wherein the network to be pruned includes a shallow network, a deep network, and a pruning supervision network, and the sample dataset includes multiple sample data.

[0026] In one embodiment, the electronic device may have a pre-configured network to be pruned. The electronic device can also obtain the network from a associated cloud or other electronic device via wireless communication technologies (such as WiFi, Bluetooth, Zigbee, etc.), or via a serial communication interface. The network to be pruned can be a convolutional neural network, a recurrent neural network, a MobileNet model, a ResNet model, a RegNet model, an EfficientNet model, etc. This network can be used to perform computer vision classification tasks, such as image classification, object detection, and image segmentation.

[0027] In some implementations, the electronic device can acquire a target task and determine the network to be pruned based on the target task. The target task may be an image processing task, and further, the electronic device may determine the network to be pruned based on the target task by selecting a network within the electronic device capable of image processing. Similarly, the target task may be a text processing task, and further, the electronic device may determine the network to be pruned based on the target task by selecting a network within the electronic device capable of text processing. The target task may also be an audio processing task, and further, the electronic device may determine the network to be pruned based on the target task by selecting a network within the electronic device capable of audio processing.

[0028] In some implementations, the network to be pruned may include shallow networks, deep networks, and pruning supervision networks. Architecturally, the network to be pruned may include intermediate layers in addition to the input and output layers; these intermediate layers can also be understood as hidden layers. The number of intermediate layers in the shallow network to be pruned may be less than a first layer threshold, while the number of intermediate layers in the deep network to be pruned may be greater than a second layer threshold. The first layer threshold is less than or equal to the second layer threshold. Shallow networks are larger than deep networks; shallow networks may include multiple parameters and can perform the target task, while deep networks can be assembled with fewer parameters than shallow networks to perform more functions than shallow networks, including but not limited to performing the target task.

[0029] The pruning supervision network can be composed of convolutional neural networks; specifically, it can be composed of a 1×1 convolutional neural network, a 3×3 convolutional neural network, or a 5×5 convolutional neural network, without any limitation.

[0030] For example, please refer to Figure 2 The diagram illustrates the structure of a network to be pruned according to an embodiment of this application. The network to be pruned may include a shallow network, a deep network, and a pruning supervision network. The pruning supervision network may consist of a 1×1 convolutional neural network.

[0031] In some implementations, the network to be pruned may include multiple pruning modules, wherein each pruning module may include a shallow network, a deep network, and a pruning supervision network; wherein the pruning supervision network included in each pruning module performs pruning supervision constraints on the deep network included in the pruning module.

[0032] In some implementations, the electronic device may have a pre-set sample dataset. The electronic device may also obtain the sample dataset from a associated cloud or electronic device via wireless communication technology, or from a associated electronic device via a serial communication interface. The sample dataset may include multiple sample data points; the sample dataset may be determined based on the target task acquired by the electronic device. For example, if the target task corresponds to an image processing task, the sample dataset acquired by the corresponding electronic device may include multiple images, where each image is also a sample data point; if the target task corresponds to a text processing task, the sample dataset acquired by the corresponding electronic device may include multiple text segments, where each text segment is also a sample data point.

[0033] Step S120: Input the multiple sample data into the shallow network to obtain the initial features output by the shallow network.

[0034] In some implementations, after obtaining the sample dataset, the electronic device can input multiple sample data into the shallow network included in the network to be pruned. Further, the shallow network can extract features from the multiple sample data and output initial features. In some implementations, the sample data is image data. The initial features output by the shallow network after extracting features from the multiple sample data can be Histogram of Oriented Gradient (HOG) features, Local Binary Pattern (LBP) features, Haar-like features, etc., and are not limited here.

[0035] In some implementations, the initial features may include the features corresponding to the sample data output by the shallow network after feature extraction, or the activation state of the shallow network output by the activation function in the shallow network, which can also be understood as the activation information of the shallow network. The activation state of the shallow network may include information about the activated filters in the shallow network.

[0036] Step S130: Input the initial features into the pruning supervision network to obtain the filter mask for the deep network output by the pruning supervision network.

[0037] In some implementations, after the shallow network outputs initial features, the initial features can be input into a pruning supervision network. Furthermore, the pruning supervision network can process the initial features to generate a filter mask for the deep network.

[0038] Among them, the pruning supervision network can perform data augmentation on the initial features, thereby more accurately determining the activated filters in the shallow network, and generating a filter mask based on the activated filters in the shallow network, so as to prune the deep network according to the filter mask.

[0039] Understandably, in computer vision classification tasks, the feature extraction capability of the same network for images of the same category is relatively fixed. Therefore, images of the same category tend to activate a similar set of filters. Furthermore, from a microscopic perspective, for the same set of sample data, a similar set of filters will also be activated when moving from a shallow network to a deep network. Therefore, we can input the activation information from the shallow network into the deep network and constrain the activation of filters in the deep network.

[0040] Step S140: Input the filter mask into the deep network to prune the deep network based on the filter mask, and obtain the pruned network.

[0041] In some implementations, after the pruning supervision network generates a filter mask, the filter mask can be input into the deep network to prune the deep network based on the filter mask, resulting in a pruned network. Pruning the deep network based on the filter mask can involve performing a dot product between the filter mask and the filters in the deep network, thereby selecting the activated filters in the deep network using the filter mask.

[0042] It is understandable that by using filter masks for deep networks, the activation information of shallow networks can be accurately transmitted to deep networks, thereby constraining the pruning of deep networks and obtaining pruned deep networks with structures similar to shallow networks, which can effectively improve the stability of the network structure.

[0043] In some implementations, after pruning a deep network based on a filter mask, the pruned network can be obtained based on the shallow network, the activated filters in the deep network, and the pruning supervision network.

[0044] In some implementations, during the forward propagation of sample data into the network to be pruned, the pruned network can be obtained, as well as the test results of the pruned network performing the target task on the sample data. Furthermore, the pruned network can also perform backpropagation based on the test results to update the weights of the pruning supervision network, thereby ensuring the accuracy of the filter mask generated by the pruning supervision network. It can also update the parameters in the network to be pruned and perform the steps of pruning and training the network to be pruned until the number of times the network to be pruned is trained reaches a first preset number of times.

[0045] Understandably, using a pruning supervision network to constrain pruning can improve the stability of the network structure, effectively preventing drastic changes during training and ensuring the effectiveness of pruning. Furthermore, during pruning, because the pruning supervision network constrains the pruning of deep networks, the structure of deep networks becomes more similar to that of shallow networks, effectively improving training efficiency and shortening training time.

[0046] One embodiment of this application provides a neural network pruning method. This method involves acquiring a network to be pruned and a sample dataset. The network to be pruned includes a shallow network, a deep network, and a pruning supervision network. The sample dataset includes multiple sample data points. Multiple sample data points are input into the shallow network to obtain initial features output by the shallow network. The initial features are then input into the pruning supervision network to obtain a filter mask for the deep network output by the pruning supervision network. The filter mask is then input into the deep network to prune the deep network based on the filter mask, resulting in a pruned network. Furthermore, by transferring the filter information of the shallow network in the network to be pruned to the deep network through the filter mask, the deep network undergoes self-supervised pruning based on the filter mask, thereby improving the performance of the pruned network and the efficiency of model training.

[0047] Please see Figure 3 , Figure 3 A schematic flowchart of a neural network pruning method provided in an embodiment of this application is shown. This method is applied to the aforementioned electronic device, and will be discussed below. Figure 3 The process shown will be described in detail. The neural network pruning method may specifically include the following steps:

[0048] Step S210: Obtain the network to be pruned and obtain the sample dataset, wherein the network to be pruned includes a shallow network, a deep network, and a pruning supervision network, and the sample dataset includes multiple sample data.

[0049] In some implementations, the electronic device can obtain the target task from an associated cloud or electronic device via wireless communication technology, or via a serial communication interface. Optionally, the electronic device may include operation buttons or a screen; the electronic device may also generate a task input interface and display it on the screen. Furthermore, the electronic device can detect the pressed state of the operation buttons or the pressed state of the screen to obtain the target task input by the user based on the task input interface.

[0050] The target task can include computer vision classification tasks such as image classification, object detection, and image segmentation; it can also be image vision processing tasks; or it can be text processing tasks, audio processing tasks, localization tasks, etc., without limitation. Furthermore, after obtaining the target task, the electronic device can obtain the network to be pruned or the sample dataset based on the target task.

[0051] In some implementations, step S210 may include steps S201-S202.

[0052] Step S201: Determine the sample dataset based on the task type.

[0053] In some implementations, after obtaining the target task, the electronic device can determine the task type of the target task and determine the sample dataset based on the task type. The task type of the target task may include image classification, object detection, localization, keyword detection, voiceprint recognition, etc.

[0054] Furthermore, the electronic device determines the sample dataset based on the task type. This can be achieved by pre-setting a correspondence between task types and sample datasets within the electronic device, such as a mapping table. For example, the electronic device may pre-set a sample dataset corresponding to an image classification task, wherein the sample dataset includes multiple images of different categories, i.e., multiple sample data; or the electronic device may pre-set a sample dataset corresponding to a voiceprint recognition task, wherein the sample dataset includes audio recordings of multiple different voiceprints, i.e., multiple sample data.

[0055] Furthermore, electronic devices can obtain sample datasets pre-stored on the electronic device corresponding to the task type of the target task based on the correspondence between task types and sample datasets; electronic devices can also obtain sample datasets corresponding to the task type of the target task from associated cloud or electronic devices through wireless communication technology based on the correspondence between task types and sample datasets; electronic devices can also obtain sample datasets corresponding to the task type of the target task from associated electronic devices through serial communication interfaces based on the correspondence between task types and sample datasets.

[0056] Step S202: Determine the network to be pruned based on the task type.

[0057] In some implementations, after obtaining the target task, the electronic device can determine the network to be pruned based on the task type of the target task. In some embodiments, the electronic device has a pre-set correspondence between task types and networks to be pruned, such as a mapping table. For example, the electronic device may have pre-set networks to be pruned for image processing tasks, such as the MobileNet model; or it may have pre-set networks to be pruned for voiceprint recognition tasks, such as the Transformer model.

[0058] Furthermore, the electronic device can obtain the network to be pruned, which is pre-stored in the electronic device and corresponds to the task type of the target task, based on the correspondence between task type and network to be pruned; the electronic device can also obtain the network to be pruned, which corresponds to the task type of the target task, from the associated cloud or electronic device through wireless communication technology, based on the correspondence between task type and network to be pruned; the electronic device can also obtain the network to be pruned, which corresponds to the task type of the target task, from the associated electronic device through a serial communication interface, based on the correspondence between task type and network to be pruned.

[0059] In some implementations, step S210 may include steps S211-S213.

[0060] Step S211: Determine the shallow network and the deep network based on the task type.

[0061] In some implementations, after obtaining a target task, the electronic device can determine the shallow and deep networks included in the network to be pruned based on the task type of the target task. The shallow and deep networks can execute the target task on the input data. For example, if the target task is an image processing task, the corresponding shallow and deep networks can perform image processing; if the target task is a voiceprint recognition task, the corresponding shallow and deep networks can perform voiceprint recognition.

[0062] Step S212: Determine the pruned supervision network based on the target platform, wherein the target platform is used to deploy the pruned network.

[0063] In some implementations, the electronic device can deploy the pruned network on a target platform, which includes a mobile platform, an edge platform, etc., meaning the target platform is used to deploy the pruned network. Furthermore, the electronic device can determine the pruning supervision network based on the target platform. For example, if the target platform is an edge platform, considering its strong computing power, a convolutional neural network with a size larger than a first size can be selected as the pruning supervision network to improve the accuracy of network pruning; if the target platform is a mobile platform, considering the memory limitations of the mobile platform, a convolutional neural network with a size smaller than a second size can be selected as the pruning supervision network, where the second size can be less than or equal to the first size.

[0064] In some implementations, the electronic device may have a pre-set correspondence between the platform and the pruning monitoring network. Furthermore, the electronic device may obtain the pruning monitoring network pre-stored between the electronic device and the target platform based on the correspondence between the platform and the pruning monitoring network, or obtain the pruning monitoring network corresponding to the target platform from the associated cloud or electronic device through wireless communication technology, or obtain the pruning monitoring network corresponding to the target platform from the associated electronic device through a serial communication interface.

[0065] For example, if the target platform is a mobile phone platform, the electronic device is pre-configured with a pruning supervision network consisting of a mobile phone platform and a 1×1 convolutional network. Further electronic devices can determine the pruning supervision network, i.e., the 1×1 convolutional network, based on the target platform.

[0066] In some implementations, the pruning supervision network may include convolution kernels, and step S212 may include steps S2121-S2122.

[0067] Step S2121: Determine the target size of the convolution kernel based on the target platform.

[0068] In some implementations, the pruning supervision network can be composed of convolutional neural networks, and further, the pruning supervision network can include convolutional kernels. In some implementations, after the electronic device determines the target platform, it can determine the target size of the convolutional kernels included in the pruning supervision network based on the target platform; wherein, the target size of the convolutional kernels included in the pruning supervision network can be 1×1, 3×3, 5×5, etc.

[0069] The electronic device can have a pre-defined mapping relationship between the platform and the size of the convolutional kernels included in the pruning supervision network, such as a mapping table. For example, the size of the convolutional kernels included in the pruning supervision network corresponding to the mobile phone platform is 1×1, and the size of the convolutional kernels included in the pruning supervision network corresponding to the edge platform is 3×3.

[0070] Furthermore, after the electronic device identifies the target platform, it can determine the target size of the convolutional kernels included in the pruning supervision network corresponding to the target platform based on the correspondence between the platform and the size of the convolutional kernels included in the pruning supervision network.

[0071] Step S2122: Determine the pruning supervision network based on the convolution kernel of the target size.

[0072] In some implementations, after determining the target size of the convolutional kernels included in the pruning supervision network, the electronic device can determine the pruning supervision network based on the convolutional kernels of the target size. For example, if the target size is 1×1, then a convolutional network including 1×1 convolutional kernels can be determined as the pruning supervision network.

[0073] Step S213: Based on the shallow network, the deep network, and the pruning supervision network, obtain the network to be pruned.

[0074] In some implementations, after determining that the network to be pruned includes a shallow network, a deep network, and a pruning supervision network, the electronic device can obtain the network to be pruned based on the shallow network, the deep network, and the pruning supervision network.

[0075] In some implementations, the electronic device may include defining multiple task types, with each task type corresponding to different shallow and deep networks. Simultaneously, the electronic device may determine the target platform for deploying the network to be pruned, and determine a pruning supervision network based on the target platform. Furthermore, the electronic device may assemble a pruning module based on the shallow network, deep network, and pruning supervision network corresponding to each task type; it is understood that the network to be pruned may include multiple pruning modules.

[0076] Step S220: Input the multiple sample data into the shallow network to obtain the initial features output by the shallow network.

[0077] Step S230: Input the initial features into the pruning supervision network to obtain the filter mask for the deep network output by the pruning supervision network.

[0078] For a detailed description of steps S220-S230, please refer to the previous description of steps S120-S130, which will not be repeated here.

[0079] Step S240: Multiply the filter mask by the self-supervised coefficients to obtain the filter mask with corresponding weights, wherein the self-supervised coefficients are determined based on the characteristics of the network to be pruned and the characteristics of the multiple sample data.

[0080] In some implementations, before inputting the filter mask into the deep network, the electronic device can multiply the filter mask output by the pruning supervision network by self-supervised coefficients to obtain a filter mask with corresponding weights. The self-supervised coefficients are determined based on the characteristics of the network to be pruned and the characteristics of multiple sample data.

[0081] In this regard, considering that the loss of training the network to be pruned can include the loss of training the shallow network with sample data input, or the loss of training the supervised pruning network, the filter mask output by the supervised pruning network can be multiplied by the self-supervised coefficients to obtain the filter mask with the corresponding weights, thereby adjusting the ratio of the loss of training the shallow network with sample data input and the loss of training the supervised pruning network.

[0082] In some implementations, the self-supervised coefficients are determined based on the characteristics of the network to be pruned and the characteristics of multiple sample data. The characteristics of the network to be pruned may include the size of the network, the number of pruning modules included in the network, etc.; the characteristics of the multiple sample data may include the number of sample data, the size of the sample data, etc. For example, if the size of the network to be pruned is greater than a first size threshold, then the self-supervised coefficients are determined to be a first preset coefficient.

[0083] Step S250: Input the filter mask into the deep network to prune the deep network based on the filter mask, and obtain the pruned network.

[0084] In some implementations, after obtaining a filter mask, the electronic device can input the filter mask into a deep network to prune the deep network based on the filter mask, thereby obtaining a pruned network. Specifically, activated filters in the deep network can be selected based on the filter mask to further prune the deep network and obtain the pruned network.

[0085] In some implementations, step S250 may include steps S251-S252.

[0086] Step S251: Perform a dot product between the filter mask and the filters in the deep network to determine the activated filters in the deep network.

[0087] In some implementations, after obtaining the filter mask, the electronic device can perform a dot product between the filter mask and the filters in the deep network to determine the activated filters in the deep network. It is understood that the filter mask may include the activation states of the shallow network. Furthermore, the deep network can activate filters similar to those in the shallow network, thus making the structure of the deep network similar to that of the shallow network. This allows the network to be pruned to iterate and optimize under similar and stable conditions. Additionally, because the structure of the deep network is similar to that of the shallow network, the training speed of network pruning can be improved, network computational efficiency can be increased, and interference and fluctuations in network pruning training can be reduced, thereby improving the overall accuracy and performance of the network.

[0088] In some implementations, the similarity between the activated filters in the deep network and the activated filters in the shallow network exceeds a similarity threshold. Based on the theory that the same network activates similar filters for the same type of sample data, filter information from the shallow network to be pruned is passed to the deep network via a filter mask, constraining the pruning process. Since the filter mask serves as self-supervised information to constrain the pruning of the deep network, it effectively mitigates drastic changes in network structure during training, improves network stability, and allows the network to iterate and optimize under similar and stable conditions.

[0089] Step S252: Based on the shallow network, the activated filters in the deep network, and the pruning supervision network, obtain the pruned network.

[0090] In some implementations, after obtaining the activated filters in the deep network, the electronic device can obtain a pruned network based on the shallow network, the activated filters in the deep network, and the pruning supervision network. The pruned network can extract initial features from the input sample data using the shallow network, or it can extract deep features from the initial features using the activated filters in the deep network. Furthermore, the electronic device can perform backpropagation on the pruned network based on the deep features to update the parameters of the network to be pruned, and then return to pruning training of the network to be pruned until the pruning training of the network to be pruned reaches a preset number of training iterations, thus obtaining the trained pruned network.

[0091] In some implementations, after step S252, the neural network pruning method provided in an embodiment of this application may further include steps S253-S254.

[0092] Step S253: Input the initial features into the deep network to obtain the deep features output by the activated filters in the deep network.

[0093] In some implementations, the electronic device can input the initial features output by the shallow network in the network to be pruned into the deep network to obtain the deep features output by the activated filters in the deep network.

[0094] In some implementations, the electronic device can calculate the loss of the pruned network after acquiring deep features. For an example, please refer to... Figure 4 The diagram illustrates a flowchart of a neural network pruning method according to an embodiment of this application. After acquiring a sample dataset and the network to be pruned, the electronic device can input multiple sample data points into the shallow network of the network to be pruned to obtain initial features output by the shallow network. Further, the network to be pruned can input the initial features into a pruning supervision network composed of 1×1 convolutional networks to obtain a filter mask for the deep network output by the pruning supervision network. Further still, the network to be pruned can multiply the filter mask by self-supervised coefficients (w) determined based on the characteristics of the network to be pruned and the characteristics of multiple sample data points to obtain a filter mask with corresponding weights, and then input this filter mask into the deep network.

[0095] Furthermore, the filter mask corresponding to the weights can be multiplied with the filters of the deep network to select the activated filters in the deep network. Furthermore, the initial features output from the shallow network can be input into the deep network. The deep network can then extract features from the initial input features, outputting deep features, and calculate the loss based on these deep features. The network to be pruned can include multiple pruning modules.

[0096] In some implementations, after the electronic device obtains the loss of the pruned network, it can backpropagate the weights of the pruning supervision network based on this loss to update the weights of the pruning supervision network. It then returns to the process of inputting multiple sample data into the shallow network to obtain initial features output by the shallow network. These initial features are then input into the pruning supervision network to obtain a filter mask for the deep network output by the pruning supervision network. This filter mask is then input into the deep network to prune the deep network based on the filter mask, resulting in a pruned network. This process continues until the network to be pruned has been trained a preset number of times, thus obtaining a trained pruned network, thereby ensuring the accuracy of the filter mask. Furthermore, the constraint of the pruning supervision network on the deep network pruning effectively improves the stability of the network structure, preventing drastic changes in the network structure during training and ensuring the pruning effect. Simultaneously, during the pruning process, due to the constraint of the pruning supervision network on the deep network pruning, the structure of the deep network is more similar to that of the shallow network, effectively improving the efficiency of network training and reducing the training time.

[0097] Step S254: Backpropagate the pruned network based on the deep features to update the parameters of the pruned network until the number of training iterations of the network to be pruned reaches the preset number of training iterations, and obtain the trained pruned network.

[0098] In some implementations, after the pruned network obtains deep features, it can be backpropagated based on these deep features to update the parameters of the pruned network until the number of training iterations of the network to be pruned reaches a preset number, thus obtaining the trained pruned network.

[0099] Understandably, pruning supervision networks effectively improve the overall performance of pruning networks, enhance network training efficiency, optimize pruning results, and maintain network accuracy and precision while ensuring pruning rate.

[0100] In some implementations, after step S254, the neural network pruning method provided in this application may further include steps S255-S256.

[0101] Step S255: Quantize the trained pruned network to obtain the quantized network.

[0102] In some implementations, after obtaining the trained, pruned network, the electronic device can quantize the trained, pruned network to obtain a quantized network. In some implementations, considering the need to deploy the network on a platform to improve its usability, quantization can reduce the computational load and size of the model while maintaining its accuracy, thereby improving the network's usability.

[0103] In some implementations, the electronic device may have a preset quantization method for the quantization model. Furthermore, after obtaining the trained pruned network, the electronic device can quantize the trained pruned network based on the preset quantization method, resulting in a quantized network. The preset quantization method may include symmetric quantization, asymmetric quantization, channel quantization, layer-by-layer quantization, etc.

[0104] Step S256: Deploy the quantized network on the target platform.

[0105] In some implementations, the electronic device can deploy the quantized network on a target platform. This target platform can be a mobile platform, an edge platform, or the like.

[0106] Optionally, after obtaining the target task, the electronic device can further obtain the model format file corresponding to the target platform, such as .js files, .tflite files, etc. Furthermore, the electronic device can convert the quantized network into the model format file corresponding to the target platform and deploy it on the target platform.

[0107] It is understood that the neural network pruning method provided in one embodiment of this application can remove weights or structures that do not play a role in the deep network included in the network to be pruned, thereby lightweighting the network without affecting its performance, improving its computational efficiency, and alleviating the deployment pressure of the network.

[0108] The neural network pruning method provided in one embodiment of this application is compared to... Figure 1 The neural network pruning method shown in this embodiment can further be applied by multiplying the filter mask by self-supervised coefficients before pruning the deep network based on the filter mask to obtain the pruned network. The self-supervised coefficients are determined based on the characteristics of the network to be pruned and the characteristics of multiple sample data, thereby adjusting the proportion of loss generated by the pruning supervision network, ensuring the accuracy of the filter mask. Furthermore, by supervising and constraining the filters of the deep network based on the filter activation information of the shallow network, the stability of the network structure during pruning can be effectively guaranteed, the accuracy of the pruned network can be improved, the computational load of network pruning can be effectively reduced, and the computational efficiency of the network can be improved.

[0109] Please see Figure 5 , Figure 5 A block diagram of a neural network pruning device according to an embodiment of this application is shown. This neural network pruning device 200 is applied to the aforementioned electronic device. The following will focus on... Figure 5The process is described in detail below. The neural network pruning device 200 includes: a network to be pruned acquisition module 210, an initial feature acquisition module 220, a filter mask acquisition module 230, and a pruned network acquisition module 240, wherein:

[0110] The network to be pruned acquisition module 210 is used to acquire the network to be pruned and to acquire a sample dataset, wherein the network to be pruned includes a shallow network, a deep network, and a pruning supervision network, and the sample dataset includes multiple sample data.

[0111] The initial feature acquisition module 220 is used to input the multiple sample data into the shallow network to obtain the initial features output by the shallow network.

[0112] The filter mask acquisition module 230 is used to input the initial features into the pruning supervision network to obtain the filter mask for the deep network output by the pruning supervision network.

[0113] The pruned network acquisition module 240 is used to input the filter mask into the deep network to prune the deep network based on the filter mask and obtain the pruned network.

[0114] Furthermore, the pruned network acquisition module 240 may include: a filter mask and filter dot product module and a pruned network acquisition submodule, wherein:

[0115] The filter mask and filter dot product module is used to perform dot product between the filter mask and the filters in the deep network to determine the activated filters in the deep network.

[0116] The pruned network acquisition submodule is used to obtain the pruned network based on the shallow network, the activated filters in the deep network, and the pruning supervision network.

[0117] Furthermore, the similarity between the activated filters in the deep network and the activated filters in the shallow network in the filter mask and filter dot product module can be greater than a similarity threshold.

[0118] Furthermore, after obtaining the pruned network based on the filters activated in the shallow network, the deep network, and the pruning supervision network, the neural network pruning device 200 may further include: a deep feature acquisition module and a trained pruned network acquisition module, wherein:

[0119] The deep feature acquisition module is used to input the initial features into the deep network and obtain the deep features output by the activated filters in the deep network.

[0120] The module for obtaining the pruned network is used to backpropagate the pruned network based on the deep features, update the parameters of the pruned network, and train the network to be pruned a preset number of times to obtain the trained pruned network.

[0121] Furthermore, after backpropagating the pruned network based on the deep features to update the parameters of the pruned network until the number of training iterations of the network to be pruned reaches a preset number of training iterations, and obtaining the trained pruned network, the neural network pruning device 200 may further include: a model quantization module and a model deployment module, wherein:

[0122] The model quantization module is used to quantize the trained, pruned network to obtain the quantized network.

[0123] The model deployment module is used to deploy the quantized network on the target platform.

[0124] Furthermore, before inputting the filter mask into the deep network to prune the deep network based on the filter mask and obtain the pruned network, the neural network pruning device 200 may further include: a self-supervised coefficient multiplication module with the filter mask, wherein:

[0125] The self-supervised coefficient and filter mask multiplication module is used to multiply the filter mask by the self-supervised coefficients to obtain the filter mask with corresponding weights, wherein the self-supervised coefficients are determined based on the characteristics of the network to be pruned and the characteristics of the multiple sample data.

[0126] Further, the network to be pruned acquisition module 210 may include: a sample dataset determination unit and a first network to be pruned determination unit, wherein:

[0127] The sample dataset determination unit is used to determine the sample dataset based on the task type.

[0128] The first network to be pruned determination unit is used to determine the network to be pruned based on the task type.

[0129] Furthermore, the network to be pruned acquisition module 210 may further include: a shallow network and deep network determination unit, a pruning supervision network determination unit, and a second network to be pruned determination unit, wherein:

[0130] A shallow network and deep network determination unit is used to determine the shallow network and the deep network based on the task type.

[0131] A pruning supervision network determination unit is used to determine the pruning supervision network based on a target platform, wherein the target platform is used to deploy the pruned network.

[0132] The second network to be pruned determination unit is used to obtain the network to be pruned based on the shallow network, the deep network, and the pruning supervision network.

[0133] Furthermore, the pruning supervision network includes convolutional kernels, and the pruning supervision network determination unit may include: a target size determination unit and a pruning supervision network determination subunit, wherein:

[0134] The target size determination unit is used to determine the target size of the convolution kernel based on the target platform.

[0135] The pruning supervision network determines the sub-unit, which is used to determine the pruning supervision network based on the convolution kernel of the target size.

[0136] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working process of the above-described device and module can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0137] In the several embodiments provided in this application, the coupling between modules can be electrical, mechanical, or other forms of coupling.

[0138] Furthermore, the functional modules in the various embodiments of this application can be integrated into one processing module, or each module can exist physically separately, or two or more modules can be integrated into one module. The integrated modules described above can be implemented in hardware or as software functional modules.

[0139] Please see Figure 6 This document illustrates a structural block diagram of an electronic device according to an embodiment of this application. The electronic device 100 can be a smartphone, tablet computer, e-reader, or other electronic device capable of running applications. The electronic device 100 in this application may include one or more of the following components: a processor 110, a memory 120, and one or more applications, wherein the one or more applications can be stored in the memory 120 and configured to be executed by one or more processors 110, and the one or more applications are configured to perform the methods described in the foregoing method embodiments.

[0140] The processor 110 may include one or more processing cores. The processor 110 connects to various parts within the electronic device 100 using various interfaces and lines, and performs various functions and processes data by running or executing instructions, programs, code sets, or instruction sets stored in the memory 120, and by calling data stored in the memory 120. Optionally, the processor 110 may be implemented using at least one hardware form of Digital Signal Processing (DSP), Field-Programmable Gate Array (FPGA), or Programmable Logic Array (PLA). The processor 110 may integrate one or a combination of several of the following: Central Processing Unit (CPU), Graphics Processing Unit (GPU), and modem. The CPU primarily handles the operating system, user interface, and applications; the GPU is responsible for rendering and drawing the content to be displayed; and the modem handles wireless communication. It is understood that the modem may also not be integrated into the processor 110 and may be implemented separately using a communication chip.

[0141] The memory 120 may include random access memory (RAM) or read-only memory (ROM). The memory 120 can be used to store instructions, programs, code, code sets, or instruction sets. The memory 120 may include a program storage area and a data storage area. The program storage area may store instructions for implementing an operating system, instructions for implementing at least one function (such as touch functionality, sound playback functionality, image playback functionality, etc.), and instructions for implementing the various method embodiments described below. The data storage area may also store data created by the electronic device 100 during use (such as phonebook data, audio and video data, chat log data, etc.).

[0142] Please see Figure 7 This diagram illustrates a structural block diagram of a computer-readable storage medium provided in an embodiment of this application. The computer-readable medium 300 stores program code that can be invoked by a processor to execute the methods described in the above method embodiments.

[0143] The computer-readable storage medium 300 may be an electronic memory such as flash memory, EEPROM (Electrically Erasable Programmable Read-Only Memory), EPROM, hard disk, or ROM. Optionally, the computer-readable storage medium 300 includes a non-transitory computer-readable storage medium. The computer-readable storage medium 300 has storage space for program code 310 that performs any of the method steps described above. This program code can be read from or written to one or more computer program products. The program code 310 may be compressed, for example, in a suitable form.

[0144] In summary, the neural network pruning method, apparatus, electronic device, and storage medium provided in this application obtain a network to be pruned and a sample dataset, wherein the network to be pruned includes a shallow network, a deep network, and a pruning supervision network, and the sample dataset includes multiple sample data; the multiple sample data are input into the shallow network to obtain the initial features output by the shallow network; the initial features are input into the pruning supervision network to obtain the filter mask output by the pruning supervision network for the deep network; the filter mask is input into the deep network to prune the deep network based on the filter mask, thereby obtaining the pruned network. Furthermore, by transferring the filter information of the shallow network in the network to be pruned to the deep network in the network to be pruned through the filter mask, the deep network is pruned based on the filter mask under self-supervised constraints, thereby improving the performance of the pruned network and the efficiency of network training.

[0145] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.

Claims

1. A neural network pruning method, characterized in that, The method includes: Obtain the network to be pruned and obtain the sample dataset, wherein the network to be pruned includes a shallow network, a deep network, and a pruning supervision network, and the sample dataset includes multiple sample data; The multiple sample data are input into the shallow network to obtain the initial features output by the shallow network; The initial features are input into the pruning supervision network to obtain the filter mask for the deep network output by the pruning supervision network. The pruning supervision network is used to determine the activated filters in the shallow network and generate the filter mask based on the activated filters in the shallow network. The filter mask is input into the deep network to prune the deep network based on the filter mask, thereby obtaining the pruned network.

2. The method according to claim 1, characterized in that, The step of inputting the filter mask into the deep network to prune the deep network based on the filter mask and obtain the pruned network includes: The filter mask is multiplied by the filter in the deep network to determine the activated filter in the deep network; The pruned network is obtained based on the shallow network, the activated filters in the deep network, and the pruning supervision network.

3. The method according to claim 2, characterized in that, The similarity between the activated filters in the deep network and the activated filters in the shallow network is greater than a similarity threshold.

4. The method according to claim 2, characterized in that, After obtaining the pruned network based on the filters activated in the shallow network, the deep network, and the pruned supervision network, the method further includes: The initial features are input into the deep network to obtain the deep features output by the activated filters in the deep network; Based on the deep features, backpropagation is performed on the pruned network to update the parameters of the pruned network until the number of training iterations of the network to be pruned reaches the preset number of training iterations, thereby obtaining the trained pruned network.

5. The method according to claim 4, characterized in that, After backpropagating the pruned network based on the deep features to update the parameters of the pruned network until the number of training iterations of the network to be pruned reaches a preset number of training iterations, and obtaining the trained pruned network, the method further includes: The trained, pruned network is quantized to obtain the quantized network. The quantized network is deployed on the target platform.

6. The method according to claim 1, characterized in that, Before inputting the filter mask into the deep network to prune the deep network based on the filter mask and obtain the pruned network, the method further includes: The filter mask is multiplied by the self-supervised coefficients to obtain the filter mask with the corresponding weights, wherein the self-supervised coefficients are determined based on the characteristics of the network to be pruned and the characteristics of the multiple sample data.

7. The method according to claim 1, characterized in that, The process of obtaining the network to be pruned and obtaining the sample dataset includes: The sample dataset is determined based on the task type; The network to be pruned is determined based on the task type.

8. The method according to claim 1, characterized in that, The process of obtaining the network to be pruned includes: The shallow network and the deep network are determined based on the task type; The pruned monitoring network is determined based on the target platform, wherein the target platform is used to deploy the pruned network; The network to be pruned is obtained based on the shallow network, the deep network, and the pruning supervision network.

9. The method according to claim 8, characterized in that, The pruning supervision network includes convolutional kernels, and determining the pruning supervision network based on the target platform includes: The target size of the convolutional kernel is determined based on the target platform. The pruning supervision network is determined based on the convolution kernel of the target size.

10. A neural network pruning device, characterized in that, The device includes: The module for obtaining the network to be pruned is used to obtain the network to be pruned and to obtain a sample dataset. The network to be pruned includes a shallow network, a deep network, and a pruning supervision network. The sample dataset includes multiple sample data. An initial feature acquisition module is used to input the multiple sample data into the shallow network and obtain the initial features output by the shallow network; A filter mask acquisition module is used to input the initial features into the pruning supervision network to obtain the filter mask for the deep network output by the pruning supervision network, wherein the pruning supervision network is used to determine the activated filters in the shallow network and generate the filter mask based on the activated filters in the shallow network. The pruned network acquisition module is used to input the filter mask into the deep network to prune the deep network based on the filter mask and obtain the pruned network.

11. An electronic device, characterized in that, include: One or more processors; Memory; One or more applications, wherein the one or more applications are stored in the memory and configured to be executed by the one or more processors, the one or more applications being configured to perform the method as described in any one of claims 1-9.

12. A computer-readable storage medium, characterized in that, The computer-readable storage medium contains program code that can be invoked by a processor to execute the method as described in any one of claims 1-9.

Images