Channel pruning method and device of model, computer device and storage medium

Abstract

The application relates to a channel pruning method and device of a model, a computer device, a storage medium and a computer program product. The method comprises the following steps: acquiring a calibration sample set, wherein the calibration sample set comprises multiple calibration images; inputting each calibration image into an initial neural network model to be calibrated to obtain an input channel feature map of each calibration image at each network layer of the initial neural network model; determining the rank of each input channel feature map, processing the rank of each input channel feature map, and determining the ranking of the channel index of each network layer; and performing channel pruning processing on the initial neural network model based on the ranking of the channel index of each network layer to obtain a target neural network model. The method can improve the channel pruning efficiency.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to a method, apparatus, computer device, storage medium, and computer program product for channel pruning of a model. Background Technology

[0002] To improve the performance of computer vision tasks, neural networks are constantly being deepened and made more complex, resulting in increasingly large models that not only consume more storage space but also increase computational demands. Therefore, various model compression techniques have emerged, such as model quantization, pruning, and weight sharing, to reduce computational load, shrink model size, and improve inference speed.

[0003] Channel pruning aims to remove redundant channels that have little impact on the results, making the model more efficient and compact. The key to channel pruning is determining which channels can be pruned. Traditional pruning methods typically use a scaling factor as a measure of channel importance to determine which channels to delete. This approach requires training data to obtain the importance information of the weights, and is heavily constrained by the data, resulting in low pruning efficiency. Summary of the Invention

[0004] Therefore, it is necessary to provide a channel pruning method, apparatus, computer equipment, computer-readable storage medium, and computer program product that can improve the efficiency of channel pruning in order to address the above-mentioned technical problems.

[0005] Firstly, this application provides a channel pruning method for a model, the method comprising:

[0006] Obtain a calibration sample set, which includes multiple calibration images;

[0007] Each of the calibration images is input into the initial neural network model to be calibrated to obtain the input channel feature map of each of the calibration images in each network layer of the initial neural network model;

[0008] The rank of each input channel feature map is determined, and the rank of each input channel feature map is processed to determine the sorting of the channel indexes of each network layer;

[0009] Based on the sorting of the channel indexes of each network layer, channel pruning is performed on the initial neural network model to obtain the target neural network model.

[0010] In one embodiment, each network layer includes multiple input channels;

[0011] The step of inputting each of the calibration images into the initial neural network model to be calibrated, and obtaining the input channel feature map of each of the calibration images in each network layer of the initial neural network model, includes:

[0012] Each of the calibration images is input into the initial neural network model to be calibrated, and the input channel feature map output by each input channel of each network layer of the initial neural network model for each calibration image is obtained.

[0013] In one embodiment, each network layer includes multiple input channels; each calibration image in each network layer of the initial neural network model has an input channel feature map corresponding to each input channel.

[0014] The process of determining the rank of each input channel feature map and processing the rank of each input channel feature map to determine the sorting of the channel indices of each network layer includes:

[0015] For any given network layer, the following steps are used to obtain the sorting of the channel indices:

[0016] Elementary transformations are performed on the input channel feature map output from each input channel to determine the rank of each input channel feature map;

[0017] The rank of the same input channel is averaged to obtain the average rank of each input channel.

[0018] The average ranks of each layer are sorted, and the sorting of the channel indices of the network layer is determined based on the sorting results.

[0019] In one embodiment, the step of performing channel pruning on the initial neural network model based on the sorting of the channel indices of each network layer to obtain the target neural network model includes:

[0020] Based on the structural information of each network layer, the channel pruning order is determined;

[0021] Based on the channel pruning order, the neural network model is subjected to channel pruning processing in sequence based on the sorting of the channel indexes of each network layer to obtain the target neural network model.

[0022] In one embodiment, the step of performing channel pruning on the initial neural network model based on the sorting of the channel indices of each network layer to obtain the target neural network model includes:

[0023] Based on the sorting of the channel indices of each network layer, channel pruning is performed sequentially on each network layer of the initial neural network model.

[0024] After channel pruning is performed on all network layers, the target neural network model is obtained.

[0025] In one embodiment, for any network layer in the initial neural network model, the channel pruning process for the targeted network layer includes:

[0026] Determine the current channel index to be processed based on the sorting of the channel indexes of the target network layer;

[0027] The input channels and convolutional kernel weights corresponding to the channel index to be processed are pruned to obtain an intermediate neural network model;

[0028] Load the intermediate neural network model and determine the reconstructed feature map output by the output channel of the intermediate neural network model;

[0029] Based on the original output feature map of the initial neural network model and the reconstructed feature map, the reconstruction error is calculated to obtain the error calculation result;

[0030] If the error calculation result does not meet the preset error condition, the next channel index to be processed is determined according to the sorting of the channel indexes of the target network layer, and the step of pruning the input channel and convolutional kernel weights corresponding to the channel index to be processed is returned to continue execution until the error calculation result meets the preset error condition at any time, and the channel pruning process of the network layer is completed.

[0031] In one embodiment, obtaining the target neural network model after channel pruning is completed in all network layers includes:

[0032] After channel pruning is completed in all network layers, a reconstructed neural network model is obtained, and the reconstructed neural network model is trained to obtain the target neural network model.

[0033] Secondly, this application also provides a channel pruning device for a model, the device comprising:

[0034] The data acquisition module is used to acquire a calibration sample set, which includes multiple calibration images.

[0035] The feature map acquisition module is used to input each of the calibration images into the initial neural network model to be calibrated, and obtain the input channel feature map of each of the calibration images in each network layer of the initial neural network model;

[0036] The processing module is used to determine the rank of each input channel feature map, process the rank of each input channel feature map, and determine the sorting of the channel indexes of each network layer.

[0037] The pruning module is used to perform channel pruning on the initial neural network model based on the sorting of the channel indices of each network layer to obtain the target neural network model.

[0038] Thirdly, this application also provides a computer device. The computer device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the channel pruning steps of the above-described model.

[0039] Fourthly, this application also provides a computer-readable storage medium. The computer-readable storage medium stores a computer program thereon, which, when executed by a processor, implements the channel pruning steps of the above-described model.

[0040] Fifthly, this application also provides a computer program product. The computer program product includes a computer program that, when executed by a processor, implements the channel pruning steps of the above-described model.

[0041] The aforementioned channel pruning method, apparatus, computer equipment, storage medium, and computer program product acquire a calibration sample set, which includes multiple calibration images. Each calibration image is input into an initial neural network model to be calibrated, obtaining the input channel feature map of each calibration image at each network layer of the initial neural network model. The rank of each input channel feature map is determined, and the rank is processed to determine the order of the channel indices of each network layer. Based on the order of the channel indices of each network layer, channel pruning is performed on the initial neural network model to obtain the target neural network model. Specifically, by using the rank of the input channel feature map as the pruning basis to determine the order of the channel indices of each network layer, and then performing channel pruning on the initial neural network model according to the order of the channel indices, the pruning efficiency is high because backpropagation is not required. Furthermore, since channel pruning is performed based on the order of the channel indices, problems of over-pruning or under-pruning can be avoided. Attached Figure Description

[0042] Figure 1 This is a diagram illustrating the application environment of the channel pruning method in one embodiment.

[0043] Figure 2 This is a flowchart illustrating the channel pruning method of a model in one embodiment;

[0044] Figure 3 This is a flowchart illustrating the channel pruning steps of a model in one embodiment;

[0045] Figure 4 This is a flowchart illustrating the channel pruning method for a model in another embodiment;

[0046] Figure 5 This is a flowchart illustrating the channel pruning method for a model in another embodiment;

[0047] Figure 6 This is a flowchart illustrating the channel pruning method of a model in one embodiment;

[0048] Figure 7 This is a flowchart illustrating the channel pruning method of a model in one embodiment;

[0049] Figure 8 This is a structural block diagram of the channel pruning device in one embodiment of the model;

[0050] Figure 9 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0051] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0052] The channel pruning method for the model provided in this application embodiment can be applied to, for example... Figure 1 In the application environment shown, terminal 102 communicates with server 104 via a network. Terminal 102 and server 104 can be used individually to execute the channel pruning method of the model in this application, or they can work together to execute the channel pruning method of the model in this application. Taking the execution of the channel pruning method of the model in this application by terminal 102 or server 104 alone as an example, when performing channel pruning of the model, terminal 102 or server 104 obtains a calibration sample set, which includes multiple calibration images; inputs each calibration image into the initial neural network model to be calibrated, and obtains the input channel feature map of each calibration image in each network layer of the initial neural network model; determines the rank of each input channel feature map, and processes the rank of each input channel feature map to determine the sorting of the channel indexes of each network layer; based on the sorting of the channel indexes of each network layer, performs channel pruning processing on the initial neural network model to obtain the target neural network model.

[0053] The terminal 102 can be, but is not limited to, various personal computers, laptops, smartphones, tablets, IoT devices, and portable wearable devices. IoT devices can include smart speakers, smart TVs, smart air conditioners, and smart in-vehicle systems. Portable wearable devices can include smartwatches, smart bracelets, and head-mounted devices. The server 104 can be implemented using a standalone server or a server cluster consisting of multiple servers.

[0054] In one embodiment, such as Figure 2 As shown, a channel pruning method for a model is provided, which is applied to a computer device (specifically, the computer device may be...). Figure 1 Taking a terminal or server as an example, the following steps are included:

[0055] Step S202: Obtain a calibration sample set, which includes multiple calibration images.

[0056] The calibration sample set can be a calibration dataset consisting of multiple calibration images extracted by computer equipment from a data source (such as a database, validation dataset, etc.) according to actual needs. Specifically, the calibration sample set can consist of 1280 calibration images.

[0057] Step S204: Input each calibration image into the initial neural network model to be calibrated to obtain the input channel feature map of each calibration image in each network layer of the initial neural network model.

[0058] The network layers can be determined by the actual structure of the initial network model. The initial neural network model can be a model that includes multiple network layers, and the input channel feature map can be a two-dimensional feature map output by the input channel of the neural network model.

[0059] In one embodiment, each layer of the initial neural network model includes multiple input channels. Therefore, inputting each calibration image into the initial neural network model to be calibrated to obtain the input channel feature map of each calibration image in each layer of the initial neural network model includes: inputting each calibration image into the initial neural network model to be calibrated to obtain the input channel feature map output by each input channel of each calibration image in each layer of the initial neural network model.

[0060] Specifically, each calibration image is input to each input channel of the network layer, and each input channel outputs the input channel feature map of each calibration image. For example, if the network layer includes four input channels: input channel 1, input channel 2, input channel 3, and input channel 4, and the calibration dataset contains 1280 calibration images, then input channel 1 will output the corresponding input channel feature map for each calibration image, i.e., output 1280 input channel feature maps; correspondingly, input channels 2, 3, and 4 will each output 1280 input channel feature maps.

[0061] In the above embodiments, the computer device performs forward inference by inputting the calibration sample set into the initial neural network model, and determines the input channel feature map output by each input channel of each network layer of the initial neural network model for each calibration image. Then, subsequent calculations can be carried out based on the obtained multiple input channel feature maps. Since the sample size is sufficient, the accuracy of subsequent calculations can be improved to a certain extent.

[0062] Step S206: Determine the rank of each input channel feature map, process the rank of each input channel feature map, and determine the sorting of the channel indices of each network layer.

[0063] The rank of the input channel feature map can be determined by the matrix obtained from the input channel feature map; specifically, the rank can be the number of non-zero rows in the simplified matrix. The channel index can correspond to each input channel of the network layer, and the order of the channel indices can be used to characterize the order in which the computer prunes each input channel.

[0064] Step S208: Based on the sorting of the channel indices of each network layer, perform channel pruning on the initial neural network model to obtain the target neural network model.

[0065] Channel pruning can be a process of trimming the input channels of a network layer. The computer device performs channel pruning on the initial neural network model according to the sorting of the channel indexes of each network layer to obtain the target neural network model. This can eliminate redundant input channels that have little impact on the results, making the model more efficient and compact.

[0066] In the channel pruning method of the above model, a calibration sample set is obtained, which includes multiple calibration images. Each calibration image is input into the initial neural network model to be calibrated to obtain the input channel feature map of each calibration image in each network layer of the initial neural network model. The rank of each input channel feature map is determined, and the rank of each input channel feature map is processed to determine the order of the channel indices of each network layer. Based on the order of the channel indices of each network layer, channel pruning is performed on the initial neural network model to obtain the target neural network model. Specifically, by using the rank of the input channel feature map as the pruning basis to determine the order of the channel indices of each network layer, and then performing channel pruning on the initial neural network model according to the order of the channel indices, the pruning efficiency is high because backpropagation is not required. Furthermore, since channel pruning is performed based on the order of the channel indices, problems of over-pruning or under-pruning can be avoided.

[0067] Among them, high-rank feature maps contain more important information, while low-rank feature maps contain less information and can be prioritized for cropping. Computer devices can determine which channels can be cropped by determining the rank of each input channel.

[0068] Therefore, in one embodiment, the rank of each input channel feature map is determined, and the rank of each input channel feature map is processed to determine the order of the channel indexes of each network layer. This includes: for any network layer, the following steps are used to obtain the order of the channel indexes: performing elementary transformation processing on the input channel feature map output by each input channel to determine the rank of each input channel feature map; averaging the rank of the same input channel to obtain the average rank corresponding to each input channel; sorting the average ranks and determining the order of the channel indexes of the network layer based on the sorting results.

[0069] In this embodiment, each network layer of the initial neural network model includes multiple input channels; each calibration image corresponds to an input channel feature map for each input channel of each network layer in the initial neural network model. The computer device can determine the sorting method of the channel indices for any given network layer in the same way. First, the computer device can determine the size of the input channel feature map (e.g., the channel feature map size is H). l ×W l Then for size H l ×W l The matrix is ​​subjected to elementary transformations to obtain the simplified matrix. The number of non-zero rows in the simplified matrix is ​​the rank of the input channel feature map.

[0070] Furthermore, after obtaining the rank of the input channel feature map, since each calibration image corresponds to an input channel feature map in each input channel, the average rank of the same input channel can be calculated. The following formula is for calculating the average rank:

[0071]

[0072] in, Let represent the average rank of the nth channel in the l-th network layer, and I represent the number of calibration images. This represents the rank of the input channel feature map output from the i-th calibration image in the n-th input channel. Finally, after calculating the average rank of each input channel, the computer can sort the average ranks of the input channels. For example, for the l-th network layer, the calculated average ranks of each input channel of that network layer are sorted to obtain the sorted result. Then, based on the sorting results, the corresponding channel indices are sorted from smallest to largest.

[0073] Specifically, if the channel indices of each input channel are Channel 1, Channel 2, Channel 3, and Channel 4, with Channel 1 corresponding to average rank 1, Channel 2 corresponding to average rank 2, Channel 3 corresponding to average rank 4, and Channel 4 corresponding to average rank 4, after the computer device sorts the average ranks, it determines that average rank 2 > average rank 4 > average rank 1 > average rank 3. Then, the channel indices can be sorted from smallest to largest. The specific sorting result of the channel indices is: Channel 3 (smallest average rank), Channel 1, Channel 4, and Channel 2 (largest average rank).

[0074] In the above embodiments, the computer device sorts the channel indexes according to the sorting results. Since the feature maps with lower ranks carry less information, the computer device can prioritize pruning the channel feature maps and corresponding filter weights of input channels with lower average ranks based on the sorting of the channel indexes. This not only eliminates redundancy but also has virtually no impact on the output of the model, making the model more efficient and compact.

[0075] The initial neural network model consists of multiple network layers. After determining the order of the channel indices of each network layer, the computer device can determine the pruning rate while keeping the structure of other network layers fixed during pruning to reduce the impact of accumulated errors.

[0076] Therefore, in one embodiment, channel pruning is performed on the initial neural network model based on the sorting of the channel indices of each network layer to obtain the target neural network model, including: determining the channel pruning order based on the structural information of each network layer; and performing channel pruning on the neural network model sequentially based on the sorting of the channel indices of each network layer according to the channel pruning order to obtain the target neural network model.

[0077] The structural information of the network layers can be determined by the structure of the initial neural network model itself. The structural information can be the order in which each network layer is located in the initial neural network model. The channel pruning order can be the order in which channel pruning is performed on the network layers. Based on the order in which each network layer is located in the initial neural network model, the computer device can select to process the network layers that come first and process the network layers that come later, and thus process each network layer in sequence to obtain the target neural network model.

[0078] In the above embodiments, after the computer device determines the order of the channel indexes of each network layer, it processes each network layer sequentially while keeping the structure of other network layers fixed when performing channel pruning on a certain network layer, which can effectively reduce the impact of cumulative error.

[0079] In one embodiment, channel pruning is performed on the initial neural network model based on the sorting of the channel indices of each network layer to obtain the target neural network model. This includes: performing channel pruning on each network layer of the initial neural network model sequentially based on the sorting of the channel indices of each network layer; and obtaining the target neural network model after channel pruning has been completed on all network layers.

[0080] In this process, the computer device performs channel pruning on each network layer according to the order of the channel indices of each network layer. This process is then applied sequentially to each network layer of the initial neural network model. This solves the problem of pruning errors (inadequate or excessive pruning) caused by setting a uniform pruning rate when the redundancy of network layers is not the same. Finally, the target neural network model is obtained only after all network layers have undergone channel pruning, which can effectively improve the efficiency of model pruning.

[0081] Specifically, in one embodiment, after channel pruning is completed in all network layers, the target neural network model is obtained, including: after channel pruning is completed in all network layers, a reconstructed neural network model is obtained, and the reconstructed neural network model is trained to obtain the target neural network model.

[0082] The reconstructed neural network model is obtained by pruning each network layer of the initial neural network model using computer equipment. After obtaining the reconstructed neural network model, the computer equipment can use a small number of training samples to train the reconstructed neural network model to fine-tune it, thereby restoring the accuracy lost due to pruning.

[0083] In one embodiment, such as Figure 3 As shown, for any network layer in the initial neural network model, the channel pruning process for that network layer includes:

[0084] Step S302: Determine the current channel index to be processed based on the sorting of the channel indexes of the target network layer.

[0085] The sorting of the channel indexes is determined based on the sorting result of the average rank of each input channel. The current channel index to be processed can be determined according to the sorting. For example, in the above embodiment, the sorting of the channel indexes is: channel 3, channel 1, channel 4 and channel 2. The computer device can select channel 3 (with the smallest average rank) as the current channel index to be processed.

[0086] Step S304: Prune the input channels and convolutional kernel weights corresponding to the channel index to be processed to obtain the intermediate neural network model.

[0087] After determining the channel index to be processed, the computer device can prune the input channels with smaller average rank (i.e., input channel feature maps) based on the channel index, and at the same time prune the convolutional kernel weights (filter weights) corresponding to the channel index to be processed, thus obtaining the pruned intermediate neural network model.

[0088] Step S306: Load the intermediate neural network model and determine the reconstructed feature map output by the output channel of the intermediate neural network model.

[0089] After the computer device completes one cropping operation, it can load the intermediate neural network model to obtain the cropped output feature map (i.e., the reconstructed feature map) output by the output channel.

[0090] Step S308: Based on the original output feature map and reconstructed feature map of the initial neural network model, perform reconstruction error calculation to obtain the error calculation result.

[0091] When calculating reconstruction errors, computer equipment can use the following formula:

[0092]

[0093] Among them, reconstructing feature maps The L2 norm can be used to represent the reconstruction error before and after pruning, where Y is the original output feature map, X′ is the input channel feature map after pruning, and W′ is the convolutional kernel weight after pruning.

[0094] Step S310: If the error calculation result does not meet the preset error condition, then determine the next channel index to be processed according to the sorting of the channel indexes of the network layer to be processed, and return to the step of pruning the input channel and convolutional kernel weights corresponding to the channel index to be processed, and continue to execute until the error calculation result meets the preset error condition at any time, and stop to complete the channel pruning process of the network layer.

[0095] The preset error condition can be determined by a set error threshold. If the error is greater than the threshold, the preset error condition is met; if it is less than the threshold, the preset error condition is not met. Since the reconstruction error gradually increases with cropping, the preset error condition can be used to guide when to stop cropping. In this embodiment, cropping stops when the error calculation result is greater than the set error threshold, indicating that the preset error condition is met.

[0096] If, after pruning, the error calculation result obtained by the computer is less than the set error threshold, it means that the error calculation result does not meet the preset error condition. The computer will then re-determine the next channel index to be processed based on the sorting of the channel indexes of the target network layer, and return to the step of pruning the input channel and convolutional kernel weights corresponding to the channel index to be processed, until the error calculation result meets the preset error condition at any time, thus completing the channel pruning process of the network layer.

[0097] In the above embodiments, the computer device determines the maximum number of pruning steps that the initial neural network model can accept by setting preset error conditions, which can make the pruning more sufficient and effective.

[0098] In one embodiment, such as Figure 4 The diagram shown is a flowchart illustrating the channel pruning method for a model in one embodiment:

[0099] in, Figure 4 The document describes the overall flowchart of channel pruning. Specifically, the computer device inputs the calibration image into the initial neural network model and loads the model. For each network layer and each input channel of that layer, the average rank is calculated to obtain the average rank of each input channel. After obtaining the average rank of each input channel, the average ranks can be sorted. For example, for the l-th network layer, the calculated average ranks of each input channel of that layer are sorted to obtain the sorted result. Then, based on the sorting results, the corresponding channel indices are sorted in ascending order.

[0100] Based on the sorting of the channel index, the computer device can sequentially remove low-rank input feature maps and their corresponding convolutional kernel weights to complete the pruning of each network layer. After pruning, the computer device can fine-tune the reconstructed neural network model obtained after acquisition, that is, use a small number of training samples to train the reconstructed neural network model to restore the accuracy lost due to pruning.

[0101] In one embodiment, such as Figure 5 The diagram shown is a flowchart illustrating the channel pruning method for a model in another embodiment:

[0102] in, Figure 5This involves a flowchart of pruning a certain network layer of an initial neural network model. The computer device can obtain the average rank of each input channel based on the rank of the feature map of each input channel, then set the initial pruning quantity to 0, and determine the order of the channel indices according to the average rank. The input channels with the smaller average rank (i.e., the input channel feature maps) are pruned, and the corresponding convolutional kernel weights are also pruned at the same time.

[0103] After one cropping operation, the cropped reconstructed feature map is output by the output channel. The computer device calculates the reconstruction error based on the original output feature map of the initial neural network model and the reconstructed feature map, and obtains the error value.

[0104] When calculating reconstruction errors, computer equipment can use the following formula:

[0105]

[0106] Among them, reconstructing feature maps The L2 norm can be used to represent the reconstruction error before and after pruning, where Y is the original output feature map, X′ is the input channel feature map after pruning, and W′ is the convolutional kernel weight after pruning.

[0107] If the error value obtained by the computer device is less than the set error threshold, the pruning quantity is incremented by 1. The computer device will then re-sort the channel indexes of the network layer to determine the next channel index to be processed, and return to prune the input channel and convolutional kernel weights corresponding to the channel index until the error value exceeds the error threshold at any time. This completes the channel pruning process of the network layer and saves the maximum pruning quantity.

[0108] In one embodiment, to illustrate the process of channel pruning in a model, the following steps represent the complete flow of the channel pruning method:

[0109] First, before performing model pruning, the computer needs to obtain calibration data. Given the diversity of the validation dataset and the fact that the average rank of the feature maps is unrelated to the number of images, the computer can extract approximately 1280 images from the validation dataset as the calibration dataset. This calibration data is then fed as a batch into the neural network model for forward inference.

[0110] In this process, after the calibration dataset is input into the neural network model, the convolution kernel moves within the calibration image. At each location, the kernel weights are multiplied by the corresponding pixel values ​​in the image, and the results are summed. The two-dimensional convolution formula is as follows, with a kernel size of K. h ×K w Given a 2D image x, output the pixel value y at pixel position (i,j).i,j :

[0111]

[0112] If the input size is H×W×N, N i The kernel depth is the same as the number of input channels, and the kernel size for 3D convolution is k. h ×K w ×N i ,as follows,

[0113]

[0114] Using one convolution kernel yields a two-dimensional output, while using M convolution kernels yields a three-dimensional output.

[0115] Among them, such as Figure 6 The diagram shown is a schematic of the structure of any one network layer. Figure 6 The leftmost one can represent the input channel. Figure 6 The squares formed by lines of different widths can represent the four different input channels of the network layer, and the calibration image can be used to obtain the corresponding input channel feature maps.

[0116] Figure 6 The middle part indicates that this network layer includes 5 convolutional kernels, and the depth of each convolutional kernel is the same as the number of input channels, which is 4. Since this network layer has 5 convolutional kernels, the corresponding output channel feature maps are 5 (i.e., ...). Figure 6 The rightmost square is composed of lines of five different widths.

[0117] For example, if the size of the input channel feature map output by the input channel is H l ×W l H l ×W l After a series of elementary transformations, the matrix can be transformed into the following form:

[0118]

[0119] Where r is the number of non-zero rows in the simplest matrix, i.e., the rank of the matrix.

[0120] Furthermore, the computer device can calculate the rank of the nth input channel of the i-th calibration image in the l-th layer. in, This represents the input channel feature map of the i-th calibration image in the n-th channel of the l-th layer:

[0121]

[0122] Since the average rank of the feature maps generated by a single filter hardly changes, the average rank of each input channel of each network layer can be calculated:

[0123]

[0124] in, Let represent the average rank of the nth channel in the l-th network layer, and I represent the number of calibration images. This represents the rank of the input channel feature map output from the i-th calibration image in the n-th input channel. Finally, after calculating the average rank of each input channel, the computer can sort the average ranks of the input channels. For example, for the l-th network layer, the calculated average ranks of each input channel of that network layer are sorted to obtain the sorted result. Then, based on the sorting results, the corresponding channel indices are sorted from smallest to largest.

[0125] The computer equipment, based on the channel index sorting, can sequentially remove low-rank input channel feature maps and their corresponding convolutional kernel weights, completing the pruning of each network layer. The specific channel pruning process is as follows: Figure 7 As shown, Figure 7 The dashed lines in the text represent channels that need to be cut off.

[0126] After completing one cropping operation, the computer device can obtain the cropped reconstructed feature map output by the output channel. Based on the original output feature map of the initial neural network model and the reconstructed feature map, the computer device calculates the reconstruction error and obtains the error value.

[0127] When calculating reconstruction errors, computer equipment can use the following formula:

[0128]

[0129] Among them, reconstructing feature maps The L2 norm can be used to represent the reconstruction error before and after pruning, where Y is the original output feature map, X′ is the input channel feature map after pruning, and W′ is the convolutional kernel weight after pruning.

[0130] If the error value obtained by the computer device is less than the set error threshold, the pruning quantity is incremented by 1. Furthermore, the computer device will re-sort the channel indexes of the network layer to determine the next channel index to be processed, and return to prune the input channel and convolutional kernel weights corresponding to the channel index until the error value is greater than the error threshold at any time, thus completing the channel pruning process of the network layer and saving the maximum pruning quantity.

[0131] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0132] Based on the same inventive concept, this application also provides a channel pruning device for implementing the channel pruning method of the model described above. The solution provided by this device is similar to the implementation described in the above method; therefore, the specific limitations of the channel pruning device embodiments of one or more models provided below can be found in the limitations of the channel pruning method of the model described above, and will not be repeated here.

[0133] In one embodiment, such as Figure 8 As shown, a channel pruning device 800 for a model is provided, including: a data acquisition module, a feature map acquisition module, a processing module, and a pruning module, wherein:

[0134] The data acquisition module 802 is used to acquire a calibration sample set, which includes multiple calibration images.

[0135] The feature map acquisition module 804 is used to input each calibration image into the initial neural network model to be calibrated, and obtain the input channel feature map of each calibration image in each network layer of the initial neural network model.

[0136] The processing module 806 is used to determine the rank of each input channel feature map, process the rank of each input channel feature map, and determine the sorting of the channel indexes of each network layer.

[0137] The pruning module 808 is used to perform channel pruning on the initial neural network model based on the sorting of the channel indexes of each network layer to obtain the target neural network model.

[0138] In one embodiment, the feature map acquisition module is further configured to input each calibration image into the initial neural network model to be calibrated, and obtain the input channel feature map output by each input channel of each network layer of the initial neural network model for each calibration image, wherein each network layer includes multiple input channels.

[0139] In one embodiment, the processing module is further configured to, for any network layer, perform the following steps to obtain the channel index sorting: perform elementary transformation processing on the input channel feature map output by each input channel to determine the rank of each input channel feature map; perform averaging processing on the ranks of the same input channel to obtain the average rank corresponding to each input channel; sort the average ranks and determine the channel index sorting of the network layer based on the sorting results, wherein each network layer includes multiple input channels; each calibration image has an input channel feature map corresponding to each input channel of each network layer in the initial neural network model.

[0140] In one embodiment, the pruning module is further configured to determine the channel pruning order based on the structural information of each network layer; and based on the channel pruning order, perform channel pruning on the neural network model sequentially based on the sorting of the channel indices of each network layer to obtain the target neural network model.

[0141] In one embodiment, the pruning module is further configured to perform channel pruning on each network layer of the initial neural network model sequentially based on the sorting of the channel indices of each network layer; after channel pruning is completed on all network layers, the target neural network model is obtained.

[0142] In one embodiment, the pruning module is further configured to: determine the current channel index to be processed based on the sorting of the channel indices of the targeted network layer; prune the input channel and convolutional kernel weights corresponding to the channel index to be processed to obtain an intermediate neural network model; load the intermediate neural network model and determine the reconstructed feature map output by the output channel of the intermediate neural network model; perform reconstruction error calculation based on the original output feature map and the reconstructed feature map of the initial neural network model to obtain the error calculation result; if the error calculation result does not meet the preset error condition, determine the next channel index to be processed based on the sorting of the channel indices of the targeted network layer, and return to the step of pruning the input channel and convolutional kernel weights corresponding to the channel index to be processed to continue execution until the error calculation result meets the preset error condition at any time, thereby completing the channel pruning process of the network layer.

[0143] In one embodiment, the pruning module is further configured to obtain a reconstructed neural network model after channel pruning has been completed in all network layers, and to train the reconstructed neural network model to obtain the target neural network model.

[0144] The modules in the channel pruning device of the above model can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in the processor of a computer device in hardware form or independent of it, or stored in the memory of a computer device in software form, so that the processor can call and execute the operations corresponding to each module.

[0145] In one embodiment, a computer device is provided, which may be a terminal or a server, and its internal structure diagram may be as follows: Figure 9 As shown, the computer device includes a processor, memory, input / output interfaces, and a communication interface. The processor, memory, and input / output interfaces are connected via a system bus, and the communication interface, display unit, and input devices are also connected to the system bus via the input / output interfaces. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The input / output interfaces are used for exchanging information between the processor and external devices. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, mobile cellular networks, NFC (Near Field Communication), or other technologies. When the computer program is executed by the processor, it implements a channel pruning method for a model. The display unit of the computer device is used to form a visually visible image and can be a display screen, a projection device, or a virtual reality imaging device. The display screen can be an LCD screen or an e-ink screen. The input device of the computer device can be a touch layer covering the display screen, or buttons, trackballs, or touchpads set on the casing of the computer device, or external keyboards, touchpads, or mice, etc.

[0146] Those skilled in the art will understand that Figure 9 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0147] In one embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps of the channel pruning method of the above model.

[0148] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the steps of the channel pruning method of the above model.

[0149] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps of the channel pruning method of the above model.

[0150] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of the relevant data shall comply with the relevant laws, regulations and standards of the relevant countries and regions.

[0151] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0152] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0153] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A channel pruning method for a model, characterized in that, The method includes: Obtain a calibration sample set, which includes multiple calibration images; Each of the calibration images is input into the initial neural network model to be calibrated to obtain the input channel feature map of each calibration image in each network layer of the initial neural network model; wherein each network layer includes multiple input channels; For any network layer, the rank of each input channel feature map of the network layer is determined by performing elementary transformation on each input channel feature map, and the average rank of each input channel in the network layer is calculated based on the rank of each input channel feature map. Based on the average rank of each input channel of the network layer, the channel indices of the network layer are sorted in ascending order. Based on the sorting of the channel indexes of each network layer, channel pruning is performed on each network layer of the initial neural network model in sequence. After channel pruning is completed in all network layers, a reconstructed neural network model is obtained, and the reconstructed neural network model is trained to obtain the target neural network model. Specifically, for any network layer in the initial neural network model, the channel pruning process for that network layer includes: Based on the sorting of the channel indices of the target network layer, the channel index with the smallest current sorting number is determined as the channel index to be processed; the input channels and convolutional kernel weights corresponding to the channel index to be processed are pruned to obtain an intermediate neural network model; the intermediate neural network model is loaded, and the reconstructed feature map output by the output channels of the intermediate neural network model is determined; based on the original output feature map of the initial neural network model and the reconstructed feature map, the reconstruction error is calculated to obtain the error calculation result; if the error calculation result is less than the error threshold, the next channel index to be processed is determined according to the sorting of the channel indices of the target network layer, and the step of pruning the input channels and convolutional kernel weights corresponding to the channel index to be processed is returned to continue execution until the error calculation result is greater than the error threshold at any time, thus completing the channel pruning process of the network layer; In this process, when performing channel pruning on any one network layer, the structure of other network layers is fixed. The following formula is used to calculate the reconstruction error: Among them, reconstructing feature maps The L2 norm is used to represent the reconstruction error before and after clipping. Y is the original output feature map. This is the cropped input channel feature map. The weights of the cropped convolutional kernels; In this process, the pruning rate is determined while keeping the structure of other network layers fixed.

2. The method according to claim 1, characterized in that, The step of inputting each of the calibration images into the initial neural network model to be calibrated, and obtaining the input channel feature map of each of the calibration images in each network layer of the initial neural network model, includes: Each of the calibration images is input into the initial neural network model to be calibrated, and the input channel feature map output by each input channel of each network layer of the initial neural network model for each calibration image is obtained.

3. The method according to claim 1, characterized in that, Each calibration image corresponds to an input channel feature map for each input channel of each network layer in the initial neural network model; The step of determining the rank of each input channel feature map of the network layer and calculating the average rank of each input channel in the network layer based on the rank of each input channel feature map includes: Elementary transformations are performed on the input channel feature map output from each input channel to determine the rank of each input channel feature map; The rank of each input channel is averaged to obtain the average rank of each input channel.

4. The method according to claim 1, characterized in that, The process of performing channel pruning on each network layer of the initial neural network model based on the sorting of the channel indexes of each network layer includes: Based on the structural information of each network layer, the channel pruning order is determined; Based on the channel pruning order, the neural network model is subjected to channel pruning processing in sequence based on the sorting of the channel indexes of each network layer.

5. The method according to claim 1, characterized in that, After channel pruning is completed in all network layers, the target neural network model is obtained, including: After channel pruning is completed in all network layers, a reconstructed neural network model is obtained, and the reconstructed neural network model is trained to obtain the target neural network model.

6. A channel pruning device for a model, characterized in that, The device includes: The data acquisition module is used to acquire a calibration sample set, which includes multiple calibration images. The feature map acquisition module is used to input each of the calibration images into the initial neural network model to be calibrated, and obtain the input channel feature map of each of the calibration images in each network layer of the initial neural network model; wherein each network layer includes multiple input channels; The processing module is used to determine the rank of each input channel feature map of any network layer by performing elementary transformation processing on each input channel feature map, calculate the average rank of each input channel in the network layer based on the rank of each input channel feature map, and sort the channel index of the network layer in ascending order according to the average rank of each input channel of the network layer. A pruning module is used to sequentially perform channel pruning on each network layer of the initial neural network model based on the sorting of the channel indices of each network layer. After channel pruning is completed on all network layers, a reconstructed neural network model is obtained, and the reconstructed neural network model is trained to obtain a target neural network model. Specifically, for any network layer in the initial neural network model, the channel pruning steps for that network layer include: determining the channel index with the smallest current sorting number as the channel index to be processed based on the sorting of the channel indices of the targeted network layer; pruning the input channels and convolutional kernel weights corresponding to the channel index to be processed to obtain an intermediate neural network model; and loading the intermediate neural network model. The reconstructed feature map output by the output channel of the intermediate neural network model is determined through the network model. Based on the original output feature map of the initial neural network model and the reconstructed feature map, the reconstruction error is calculated to obtain the error calculation result. If the error calculation result is less than the error threshold, the next channel index to be processed is determined according to the sorting of the channel indexes of the targeted network layer, and the step of pruning the input channel and convolutional kernel weights corresponding to the channel index to be processed is returned to continue execution until the error calculation result is greater than the error threshold at any time, thus completing the channel pruning process of the network layer. Among them, when performing channel pruning on any network layer, the structure of other network layers is fixed. The following formula is used to calculate the reconstruction error: Among them, reconstructing feature maps The L2 norm is used to represent the reconstruction error before and after clipping. Y is the original output feature map. This is the cropped input channel feature map. The weights of the cropped convolutional kernels; In this process, the pruning rate is determined while keeping the structure of other network layers fixed.

7. The apparatus according to claim 6, characterized in that, The feature map acquisition module is also used to input each calibration image into the initial neural network model to be calibrated, and obtain the input channel feature map output by each input channel of each network layer of the initial neural network model for each calibration image, wherein each network layer includes multiple input channels.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 5.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 5.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 5.

Images