Classification method based on pruned convolutional neural network and related device

Abstract

The application provides a classification method based on a pruned convolutional neural network and related equipment, including: training a pre-trained classification model based on training data and labels corresponding to the training data; obtaining a pruned classification model by respectively performing structured pruning and unstructured pruning on the pre-trained classification model on the basis of a preset model target performance; obtaining a picture to be classified; inputting the picture to be classified into the pruned classification model to obtain a corresponding classification result. The application can realize hybrid pruning of the classification model on the basis of the preset model target performance, thereby compressing and simplifying the model to the greatest extent.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of model pruning, in particular to a classification method based on pruned convolutional neural network and related equipment. BACKGROUND

[0002] Model pruning is a common method in the field of neural network model compression, which is used to compress the model size and accelerate the model calculation. Generally, it is achieved by pruning the unimportant tensors in the neural network weights to reduce the computational load of the entire neural network. According to the pruning granularity from small to large, the specific pruning granularity includes: fine-grained, vector-level, kernel-level, filter-level and layer-level.

[0003] Among them, fine-grained pruning is to prune the connections or neurons, which is the smallest granularity pruning. Vector-level pruning has a larger granularity than fine-grained pruning, which belongs to intra-kernel pruning. Kernel-level pruning removes a certain convolution kernel, which will discard the response to the corresponding calculation channel in the input channel. Filter-level pruning prunes the entire convolution kernel group, which will change the number of output feature channels in the inference process.

[0004] The above fine-grained pruning, vector-level pruning and kernel-level pruning methods achieve a certain balance between parameter quantity and model performance, but the topology of the network itself has changed, and special algorithm design is needed to support this sparse operation, which is called unstructured pruning.

[0005] Filter-level pruning only changes the filter group and the number of feature channels in the network, and the obtained model can run without special algorithm design, which is called structured pruning. In addition, there is pruning of the entire network layer, which can be regarded as a variant of filter-level pruning, that is, all filters are discarded.

[0006] Hybrid pruning is to prune the entire model from different pruning granularities. The existing hybrid pruning often prunes the model with compression rate as the target, but the performance of the pruned model cannot be controlled, and only 1-2 kinds of pruning granularities are mixed in hybrid pruning, which cannot fully explore the model pruning limit under the given target.

[0007] Therefore, how to compress and simplify the model to the greatest extent by mixing multiple pruning granularities under a given pruning target is a problem to be solved. SUMMARY

[0008] The application provides a classification method based on a pruned convolutional neural network and related equipment to solve the above problems.

[0009] The application provides a classification method based on a pruned convolutional neural network, comprising:

[0010] training a pre-trained classification model based on training data and labels corresponding to the training data;

[0011] obtaining a pruned classification model by performing structural pruning and unstructured pruning on the pre-trained classification model based on a preset model target performance;

[0012] obtaining a picture to be classified;

[0013] inputting the picture to be classified into the pruned classification model to obtain a corresponding classification result.

[0014] According to the classification method based on the pruned convolutional neural network, the structural pruning comprises convolutional layer pruning and filter pruning.

[0015] Correspondingly, the pruned classification model is obtained by performing structural pruning and unstructured pruning on the pre-trained classification model based on a preset model target performance, comprising:

[0016] S1, performing filter pruning sensitivity analysis on each filter of each convolutional layer in the pre-trained classification model to obtain a maximum convolutional layer pruning rate satisfying the preset model target performance;

[0017] S2, determining whether the maximum convolutional layer pruning rate is greater than a preset convolutional layer pruning rate threshold;

[0018] In the case where the maximum convolutional layer pruning rate is greater than the preset convolutional layer pruning rate threshold, performing convolutional layer pruning on the convolutional layer corresponding to the maximum convolutional layer pruning rate;

[0019] In the case where the maximum convolutional layer pruning rate is not greater than the preset convolutional layer pruning rate threshold, performing filter pruning on the filter corresponding to the maximum convolutional layer pruning rate;

[0020] S3, repeating S2 until all maximum convolutional layer pruning rates are exhausted, thereby obtaining a first pruned model;

[0021] S4, performing unstructured pruning on the first pruned model to obtain a second pruned model as the pruned classification model.

[0022] According to the pruning convolutional neural network-based classification method provided by the application, filter pruning sensitivity analysis is performed on each filter of each convolutional layer in a pre-trained classification model, a maximum convolutional layer pruning rate satisfying a preset model target performance is obtained, and the method comprises the following steps:

[0023] A set of to-be-analyzed pruning rates is configured for each convolutional layer, wherein the set of to-be-analyzed pruning rates comprises a plurality of to-be-analyzed pruning rates;

[0024] The model performance of each convolutional layer under each to-be-analyzed pruning rate is calculated in sequence, and it is determined whether the calculated model performance satisfies the preset model target performance;

[0025] In the case where the calculated model performance satisfies the preset model target performance, the to-be-analyzed pruning rate with the maximum value in each convolutional layer is taken as the maximum convolutional layer pruning rate.

[0026] According to the pruning convolutional neural network-based classification method provided by the application, the unstructured pruning is convolution kernel pruning;

[0027] Correspondingly, the S4, the unstructured pruning is performed on the first pruned model to obtain a second pruned model as the pruned classification model, and the method comprises the following steps:

[0028] S41, normalizing all weights in the first pruned model to obtain normalized parameters;

[0029] S42, sorting the normalized parameters according to the numerical value to obtain a sequence of normalized parameters;

[0030] S43, determining to-be-pruned weights from the sequence of normalized parameters by using a predetermined weight pruning positioning method;

[0031] S44, determining whether the kernel pruning rate corresponding to the to-be-pruned weights is greater than a preset kernel pruning rate threshold,

[0032] In the case where the kernel pruning rate corresponding to the to-be-pruned weights is greater than the preset kernel pruning rate threshold, convolution kernel pruning is performed on the convolution kernel corresponding to the kernel pruning rate;

[0033] In the case where the kernel pruning rate corresponding to the to-be-pruned weights is not greater than the preset kernel pruning rate threshold, weight pruning is performed on the to-be-pruned weights;

[0034] S45, repeating the S44 until all to-be-pruned weights are exhausted, so as to obtain a second pruned model, and the second pruned model is taken as the pruned classification model.

[0035] According to the pruning convolutional neural network-based classification method provided in the application, the sorting of the normalized parameters according to numerical values is sorting the normalized parameters from small to large, thereby obtaining a normalized parameter sequence.

[0036] Accordingly, the S43 comprises determining to-be-pruned weights from the normalized parameter sequence by using a predetermined weight pruning positioning method.

[0037] S431, pruning the first i normalized parameters in the normalized parameter sequence to obtain an initial second pruned model, wherein i is initially 1 and is less than the total number of all the normalized parameters in the normalized parameter sequence.

[0038] S432, determining whether the initial second pruned model meets the preset model target performance.

[0039] S433, in the case where the initial second pruned model meets the preset model target performance, taking i+1 as a new i, and repeating the S431 and the S432 until the initial second pruned model does not meet the preset model target performance, and determining the weights corresponding to the first i normalized parameters in the normalized parameter sequence as the to-be-pruned weights according to the new i.

[0040] According to the pruning convolutional neural network-based classification method provided in the application, the preset model target performance is that the performance of the pruned classification model does not decrease by more than a performance threshold.

[0041] The application further provides a pruning convolutional neural network-based classification device, comprising:

[0042] a model training module configured to train a pre-trained classification model based on training data and labels corresponding to the training data;

[0043] a model pruning module configured to obtain a pruned classification model by performing structured pruning and unstructured pruning on the pre-trained classification model based on a preset model target performance;

[0044] a picture acquisition module configured to acquire a picture to be classified;

[0045] a classification module configured to input the picture to be classified into the pruned classification model to obtain a corresponding classification result.

[0046] The application further provides an electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, and the processor implements any of the pruning convolutional neural network-based classification methods as described above when executing the program.

[0047] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements any of the above-mentioned classification methods based on pruned convolutional neural networks.

[0048] The present invention also provides a computer program product, comprising a computer program, which, when executed by a processor, implements any of the above-mentioned classification methods based on pruned convolutional neural networks.

[0049] The classification method and related equipment based on pruned convolutional neural networks provided by the present invention prunes the classification model based on the preset model target performance through two pruning dimensions, structured and unstructured, thereby compressing and simplifying the model to the greatest extent, and using the pruned classification model to classify the images to be classified to quickly obtain the classification results. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] In order to more clearly illustrate the technical solutions in the present invention or the prior art, a brief introduction is given below to the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0051] Figure 1 1 is a flow chart of a classification method based on a pruned convolutional neural network provided by an embodiment of the present invention;

[0052] Figure 2 Schematic diagram of an implementation of the pruning method provided by an embodiment of the present invention;

[0053] Figure 3 A schematic diagram of the structure of a classification device based on a pruned convolutional neural network provided by an embodiment of the present invention;

[0054] Figure 4 A schematic diagram of the physical structure of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0055] To make the objectives, technical solutions, and advantages of the present invention more clear, the technical solutions of the present invention will be clearly and completely described below in conjunction with the accompanying drawings. Obviously, the embodiments described are only some of the embodiments of the present invention, not all of them. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts shall fall within the scope of protection of the present invention.

[0056] Figure 1 : is a flow chart of a classification method based on pruned convolutional neural network provided by an embodiment of the present invention; Figure 1As shown, a classification method based on a pruned convolutional neural network includes the following steps:

[0057] S101, a pre-trained classification model is trained based on training data and labels corresponding to the training data.

[0058] In this embodiment, an autonomous driving scene classification model is pre-trained using autonomous driving scene training data and labels corresponding to the training data. In other embodiments, it can also be a food classification model trained by food training data and corresponding labels, or an animal classification model trained by animal training data and corresponding labels. The present application does not make specific limitations on the classification model.

[0059] S102, a pruned classification model is obtained by respectively performing structured pruning and unstructured pruning on the pre-trained classification model based on a pre-set model target performance.

[0060] In this step, the pre-trained classification model is pruned from two dimensions of structured pruning and unstructured pruning according to the set model target performance (for example, the performance of the pruned model cannot decrease by more than 5%), thereby obtaining a pruned classification model with small model volume and fast calculation rate.

[0061] S103, obtaining a picture to be classified.

[0062] In this step, the picture to be classified can be directly obtained by shooting in the autonomous driving scenario, downloaded from the Internet, or obtained from the database. The present application does not limit the way of obtaining the picture to be classified.

[0063] S104, inputting the picture to be classified into the pruned classification model to obtain a corresponding classification result.

[0064] In this step, the classification result is obtained after inputting the picture to be classified into the pruned classification model. The pruned classification model can quickly classify the picture to be classified and obtain the classification result.

[0065] In this example, the classification result corresponding to the above-mentioned autonomous driving scene classification model is the autonomous driving scene classification result, which includes four kinds of rain night, rain day, sunny night and sunny day. In other embodiments of the present application, it can be rain, sunny, snowy, etc. The present application does not make limitations.

[0066] The classification method based on the pruning convolutional neural network provided by the embodiment of the present application can complete pruning of the classification model on the basis of the preset model target performance through two pruning dimensions of structured and unstructured, thereby compressing and simplifying the model to the greatest extent, and using the pruned classification model to classify the pictures to be classified to quickly obtain the classification result.

[0067] Figure 2 is a schematic diagram of the pruning method provided by the embodiment of the present application; as shown in Figure 2 The structured pruning includes convolution layer pruning and filter pruning.

[0068] Correspondingly, the pruned classification model is obtained by respectively performing structured pruning and unstructured pruning on the pre-trained classification model on the basis of the preset model target performance, and includes:

[0069] S1, performing filter pruning sensitivity analysis on each filter of each convolution layer in the pre-trained classification model to obtain a maximum convolution layer pruning rate satisfying the preset model target performance.

[0070] In this step, at least one filter is included in one convolution layer, and the filter pruning sensitivity analysis is performed on all filters in each convolution layer, so as to determine the influence of each filter on the model performance after pruning, determine the pruning rate configuration of each convolution layer, and the determined pruning rate configuration is the filter pruning sensitivity analysis result.

[0071] Suppose that the model target performance is that the performance of the pruned model does not decrease by more than 5%, and the model performance decrease under different pruning rate configurations of each convolution layer is analyzed in turn based on the filter pruning sensitivity analysis result (i.e. the pruning rate configuration of the convolution layer) of each convolution layer, so as to obtain the upper limit value of the pruning rate of each layer when the performance does not decrease by more than 5%, and the upper limit value is the pruning rate of each convolution layer.

[0072] S2, judging whether the maximum convolution layer pruning rate is greater than a preset convolution layer pruning rate threshold value;

[0073] In the case that the maximum convolution layer pruning rate is greater than the preset convolution layer pruning rate threshold value, the convolution layer corresponding to the maximum convolution layer pruning rate is pruned.

[0074] In the case that the maximum convolution layer pruning rate is not greater than the preset convolution layer pruning rate threshold value, the filter corresponding to the maximum convolution layer pruning rate is pruned.

[0075] In this step, the preset convolution layer pruning rate threshold value is set to 0.9, and if the maximum convolution layer pruning rate obtained through the sensitivity analysis is greater than 0.9, the entire convolution layer is directly pruned.

[0076] If the maximum convolution layer pruning rate is less than or equal to 0.9, pruning is performed according to the L1 value in each filter in the corresponding convolution layer. Specifically, the value of each filter in the convolution layer with the maximum convolution layer pruning rate less than or equal to 0.9 is calculated, and the L1 values are sorted from small to large. The filters with the top ranking are pruned.

[0077] It should be noted that in other embodiments of the present application, the convolution layer pruning rate threshold of 0.9 can be set according to requirements, and the present application does not limit this.

[0078] S3, repeating S2 until all maximum convolution layer pruning rates are exhausted, thereby obtaining a first pruned model.

[0079] In the step, since each convolution layer corresponds to a maximum convolution layer pruning rate, it is necessary to determine whether to prune the convolution layer (4D pruning) or prune a single filter in the convolution layer (3D pruning) for each convolution layer, thereby exhausting all maximum convolution layer pruning rates and completing the structured pruning of the entire pre-trained classification model.

[0080] S4, performing unstructured pruning on the first pruned model to obtain a second pruned model as a pruned classification model.

[0081] In this step, after the above structured pruning (i.e., large-grained pruning), further pruning is performed from the perspective of unstructured pruning, i.e., small-grained pruning. The second pruned model obtained after structured pruning and unstructured pruning is used as the pruned classification model.

[0082] It should be noted that the unstructured pruning can be one or more of fine-grained pruning, vector pruning, and kernel pruning methods.

[0083] The classification method based on the pruned convolution neural network provided by the embodiment of the present application considers two large pruning granularity dimensions of convolution layer pruning and filter pruning, quickly realizes structured pruning, and further performs more fine pruning from the perspective of small pruning granularity on the basis of the completion of structured pruning, thereby being able to quickly and maximally complete model pruning under the performance of the target model.

[0084] Further, the filter pruning sensitivity analysis on each filter of each convolution layer in the pre-trained classification model to obtain the maximum convolution layer pruning rate satisfying the preset model target performance includes:

[0085] A set of to-be-analyzed pruning rates is configured for each convolution layer. The set of to-be-analyzed pruning rates includes a plurality of to-be-analyzed pruning rates.

[0086] calculate the model performance of each convolutional layer under each to-be-analyzed pruning rate in turn, and determine whether the calculated model performance meets the preset model target performance;

[0087] In the case where the calculated model performance meets the preset model target performance, the largest to-be-analyzed pruning rate in each convolutional layer is taken as the maximum convolutional layer pruning rate.

[0088] Specifically, a to-be-analyzed pruning rate set (0.1, 0.15, 0.2, 0.25, …, 0.8, 0.85, 0.9, 0.95) is configured for each convolutional layer.

[0089] The first convolutional layer is taken as a to-be-pruned convolutional layer, and the weights in other convolutional layers remain unchanged, then the model performance (for example, the case of accuracy ACC reduction) is calculated based on each to-be-analyzed pruning rate, and compared with the model target performance (assuming that the accuracy ACC reduction does not exceed 5%), if the calculated model performance meets the target performance, the current to-be-analyzed pruning rate is taken as the to-be-evaluated pruning rate.

[0090] After the model performance is calculated by using each to-be-analyzed pruning rate in the to-be-analyzed pruning rate set and compared with the target performance, a plurality of to-be-evaluated pruning rates are obtained, and the largest to-be-evaluated pruning rate is taken as the maximum convolutional layer pruning rate of the current first convolutional layer.

[0091] By analogy, the maximum convolutional layer pruning rate corresponding to each convolutional layer is obtained.

[0092] It should be noted that the to-be-analyzed pruning rate set can be obtained in the following way:

[0093] The value range and pruning rate step of the to-be-analyzed pruning rate are preset, for example, the value range is set to [0.1, 0.95], and the step is 0.05. Then the to-be-analyzed pruning rate set (0.1, 0.15, 0.2, 0.25, …, 0.8, 0.85, 0.9, 0.95) is obtained by exhaustive method.

[0094] The classification method based on the pruned convolutional neural network provided by the embodiment of the application analyzes the model performance of all possible pruning rates of each convolutional layer, so as to determine the maximum pruning rate under the condition of meeting the target performance.

[0095] Further, the non-structural pruning is convolution kernel pruning;

[0096] Correspondingly, the S4, the non-structural pruning is performed on the first pruned model to obtain a second pruned model as the pruned classification model, comprising:

[0097] S41. Normalize all weights in the first pruned model to obtain normalized parameters.

[0098] In this embodiment, the L1 norm of all parameters of the calculation model is calculated to obtain the normalized weight parameters. In other embodiments of the present application, normalization can also be achieved by L2 norm formula, L3 norm formula, etc., which are not limited in the present application.

[0099] S42. Sort the normalized parameters according to their numerical values to obtain a sequence of normalized parameters.

[0100] In this step, the normalized weight parameters are sorted in ascending order to obtain a sequence of normalized parameters.

[0101] S43. Determine the weight to be pruned from the sequence of normalized parameters using a predetermined weight pruning positioning method.

[0102] In this step, the predetermined weight pruning positioning method refers to pruning analysis of each weight to determine which weight pruning can meet the model target performance, thereby determining the weight to be pruned from the sequence of normalized parameters.

[0103] S44. Determine whether the kernel pruning rate corresponding to the weight to be pruned is greater than a preset kernel pruning rate threshold,

[0104] If the kernel pruning rate corresponding to the weight to be pruned is greater than the preset kernel pruning rate threshold, perform kernel pruning on the convolution kernel corresponding to the kernel pruning rate.

[0105] If the kernel pruning rate corresponding to the weight to be pruned is not greater than the preset kernel pruning rate threshold, perform weight pruning on the weight to be pruned.

[0106] In this step, the preset kernel pruning rate threshold is 0.8. If the kernel pruning rate corresponding to the weight to be pruned is greater than 0.8, the entire convolution kernel is pruned. If the kernel pruning rate corresponding to the weight to be pruned is less than or equal to 0.8, the weights in the convolution kernel are sorted in ascending order, and the weights with low ranking are pruned.

[0107] It should be noted that the kernel pruning rate corresponding to the weight to be pruned is obtained as follows (similar to the maximum convolution layer pruning rate of the convolution layer):

[0108] The model performance after weight pruning of each convolution kernel is analyzed in sequence to obtain an upper limit value of the pruning rate of each layer when the performance degradation is not more than 5%, which is the pruning rate of the convolution kernel.

[0109] S45, repeat S44 until all to-be-pruned weights are pruned, so as to obtain a second pruned model, and the second pruned model is taken as the pruned classification model.

[0110] In this step, after determining whether the to-be-pruned weight is pruned as a convolution kernel or a vector, that is, 2D pruning or 1D pruning, the unstructured pruning of the entire first pruned model is completed, so as to obtain a second pruned model, which is the pruned classification model.

[0111] The classification method based on the pruned convolutional neural network provided in the embodiments of the present application combines convolution kernel pruning and vector pruning with two different pruning granularities to achieve more precise pruning at the unstructured pruning level, and combines convolution layer pruning and filter pruning in structured pruning to adaptively realize mixed granularities, obtain a higher compression ratio of model parameters, realize the pruning limit of the model, and ensure the effect of the model.

[0112] Further, the sorting of the normalized parameters according to the numerical values is sorting the normalized parameters from small to large, so as to obtain a normalized parameter sequence.

[0113] Correspondingly, the S43, determining the to-be-pruned weight from the normalized parameter sequence by using a predetermined weight pruning positioning method, comprises:

[0114] S431, pruning the first i normalized parameters in the normalized parameter sequence to obtain an initial second pruned model, wherein i is initially 1 and is less than the total number of all normalized parameters in the normalized parameter sequence.

[0115] In this step, assuming that there are n normalized parameters in the normalized parameter sequence, in the first round of to-be-pruned weight determination process, i is 1 (i < n), that is, the weight corresponding to the normalized parameter with the smallest numerical value in the normalized parameter sequence is pruned, so as to obtain an initial second pruned model.

[0116] S432, determining whether the initial second pruned model meets the preset model target performance.

[0117] In this step, the performance of the initial second pruned model which has only one weight pruned is evaluated, and it is determined whether the performance evaluation result meets the preset model target performance (assuming that the model performance is reduced by no more than 5%).

[0118] S433. If the initial second pruned model meets the preset model target performance, i+1 is used as the new i, and S431 to S432 are repeated until the initial second pruned model does not meet the preset model target performance, and the weights corresponding to the first i normalized parameters in the normalized parameter sequence are determined as the weights to be pruned according to the new i.

[0119] In this step, the process of determining the weights to be pruned in the first round is continued. If the performance corresponding to the initial second pruned model meets the preset model target performance, it is considered that the initial second pruned model can continue to be pruned, and 1 is added to i to start the second round of determining the weights to be pruned, that is, the weights corresponding to the first two normalized parameters in the normalized parameter sequence are pruned to obtain a new initial second pruned model, and then the model performance analysis and target performance judgment are performed. If the model target performance is still met, the next round of determining the weights to be pruned is performed; if the initial second pruned model cannot meet the model target performance, the loop is terminated, and the weights corresponding to the first i normalized parameters are all used as weights to be pruned.

[0120] The classification method based on pruned convolutional neural network provided by the embodiment of the present invention can adaptively realize the mixing of various granularities, obtain a higher compression ratio of model parameters, achieve the model pruning limit, and ensure the effect of the model.

[0121] Furthermore, the preset model target performance is that the performance degradation of the classification model after pruning does not exceed a performance threshold.

[0122] In this embodiment, the preset model target performance is the target accuracy rate (ACC), and the classification model performance after pruning is set to decrease by no more than 5%. In other embodiments of the present invention, the target performance can also be other model performance evaluation indicators such as false positive rate, precision, recall, etc.

[0123] In addition, it should be noted that the training process of a pre-trained classification model is to first build an initial classification model, then input the training data and corresponding labels into the initial classification model, and continuously update the weights in the initial classification model through forward propagation, loss function calculation, and backpropagation until the initial classification model converges, thus obtaining a trained classification model. Hybrid pruning is performed based on the trained classification model.

[0124] The first pruned model obtained after structured pruning and the second pruned model obtained after unstructured pruning can both be fine-tuned.

[0125] The pruning-based convolutional neural network classification device provided by the present application is described below. The pruning-based convolutional neural network classification device described below can be referred to in correspondence with the pruning-based convolutional neural network classification method described above.

[0126] Figure 3 The pruning-based convolutional neural network classification device provided by the present application is described below. The pruning-based convolutional neural network classification device described below can be referred to in correspondence with the pruning-based convolutional neural network classification method described above. Figure 3 As shown in FIG. 1, a pruning-based convolutional neural network classification device comprises:

[0127] The model training module 301 is configured to train a pre-trained classification model based on training data and labels corresponding to the training data.

[0128] In this embodiment, an autonomous driving scene classification model is pre-trained using autonomous driving scene training data and labels corresponding to the training data. In other embodiments, a food classification model can be trained using food training data and corresponding labels, or an animal classification model can be trained using animal training data and corresponding labels. The present application does not make specific limitations on the classification model.

[0129] The model pruning module 302 is configured to obtain a pruned classification model by performing structured pruning and unstructured pruning on the pre-trained classification model based on a preset model target performance.

[0130] In this module, the pre-trained classification model is pruned according to the set model target performance (for example, the performance of the pruned model cannot decrease by more than 5%) from the two dimensions of structured pruning and unstructured pruning, thereby obtaining a pruned classification model with small model volume and fast calculation rate.

[0131] The picture acquisition module 303 is configured to acquire a picture to be classified.

[0132] In this module, the picture to be classified can be directly captured in an autonomous driving scenario, downloaded from the Internet, or acquired from a database. The present application does not make limitations on the acquisition method of the picture to be classified.

[0133] The classification module 304 is configured to input the picture to be classified into the pruned classification model to obtain a corresponding classification result.

[0134] In this module, the picture to be classified is input into the pruned classification model to obtain a classification result. The pruned classification model can quickly classify the picture to be classified to obtain a classification result.

[0135] In the present example, the classification result corresponding to the above automatic driving scene classification model is an automatic driving scene classification result, which includes four types of rainy night, rainy day, sunny night and sunny day. In other embodiments of the present application, it can be rainy, sunny, snowy, etc., and the present application does not limit this.

[0136] The classification device based on the pruning convolutional neural network provided by the embodiments of the present application can complete pruning of the classification model on the basis of the preset model target performance through two pruning dimensions of structured and unstructured, thereby compressing and simplifying the model to the greatest extent, and using the pruned classification model to classify the pictures to be classified to quickly obtain the classification result.

[0137] The classification device based on the pruning convolutional neural network provided by the embodiments of the present application only needs to provide target contract information by the customer to perform classification based on the pruning convolutional neural network, which is simple and does not involve too much privacy of the customer, and the authentication process based on the blockchain is more reliable and secure, and does not excessively rely on the data stored by the trusted center, thereby protecting the rights and interests of the customer.

[0138] Figure 4 An entity structure schematic diagram of an electronic device provided by the embodiments of the present application is shown in Figure 4 The electronic device can include a processor 410, a communications interface 420, a memory 430 and a communications bus 440, wherein the processor 410, the communications interface 420 and the memory 430 complete mutual communication through the communications bus 440. The processor 410 can call the logical instructions in the memory 430 to execute the classification method based on the pruning convolutional neural network, and the classification method based on the pruning convolutional neural network includes:

[0139] Training a pre-trained classification model based on training data and labels corresponding to the training data;

[0140] Obtaining a pruned classification model by respectively performing structured pruning and unstructured pruning on the pre-trained classification model on the basis of the preset model target performance;

[0141] Obtaining a pruned classification model by respectively performing structured pruning and unstructured pruning on the pre-trained classification model on the basis of the preset model target performance;

[0142] Obtaining a pruned classification model by respectively performing structured pruning and unstructured pruning on the pre-trained classification model on the basis of the preset model target performance;

[0143] Further, the logic instructions in the memory 430 described above can be implemented in the form of software functional units and sold or used as independent products, and can be stored in a computer readable storage medium. Based on such understanding, the technical solutions of the present application essentially or the parts that make contributions to the prior art or parts of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium, and includes several instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods according to the embodiments of the present application. The aforementioned storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM, Read Only Memory), a random access memory (RAM, Random Access Memory), a magnetic disk or an optical disk, and various media that can store program codes.

[0144] In another aspect, the present application also provides a computer program product, the computer program product comprising a computer program, the computer program being stored in a non-transitory computer readable storage medium, and the computer program being executable by a processor to cause a computer to perform a pruning-based convolutional neural network classification method provided by any of the methods described above, the pruning-based convolutional neural network classification method comprising:

[0145] training a pre-trained classification model based on training data and labels corresponding to the training data;

[0146] obtaining a pruned classification model by performing structured pruning and unstructured pruning on the pre-trained classification model based on a preset model target performance;

[0147] obtaining a pruned classification model by performing structured pruning and unstructured pruning on the pre-trained classification model based on a preset model target performance;

[0148] obtaining a pruned classification model by performing structured pruning and unstructured pruning on the pre-trained classification model based on a preset model target performance;

[0149] In another aspect, the present application also provides a computer program product, the computer program product comprising a computer program, the computer program being stored in a non-transitory computer readable storage medium, and the computer program being executable by a processor to cause a computer to perform a pruning-based convolutional neural network classification method provided by any of the methods described above, the pruning-based convolutional neural network classification method comprising:

[0150] training a pre-trained classification model based on training data and labels corresponding to the training data;

[0151] obtaining a pruned classification model by performing structured pruning and unstructured pruning on the pre-trained classification model based on a preset model target performance;

[0152] obtaining a pruned classification model by performing structured pruning and unstructured pruning on the pre-trained classification model based on a preset model target performance;

[0153] Input the picture to be classified into the pruned classification model to obtain a corresponding classification result.

[0154] The device embodiments described above are merely illustrative, wherein the units illustrated as separate components can or can not be physically separated, and the components illustrated as units can or can not be physical units, i.e., can be located in one place, or can be distributed to multiple network units. Part or all of the modules can be selected to achieve the purpose of the embodiment scheme according to actual needs. Those skilled in the art can understand and implement without creative labor.

[0155] Through the description of the above embodiments, those skilled in the art can clearly understand that the embodiments can be realized by means of software and the necessary general hardware platform, and of course can also be realized by hardware. Based on such understanding, the above technical solutions can be embodied in the form of a software product, which can be stored in a computer readable storage medium, such as a ROM / RAM, a magnetic disk, an optical disk, etc., and includes a plurality of instructions to make a computer device (which can be a personal computer, a server, or a network device, etc.) execute the methods of the various embodiments or some parts of the embodiments.

[0156] Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present application, and not to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that: it can still modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacement for part of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A classification method based on pruned convolutional neural network, characterized in that: include: A pre-trained classification model is obtained based on the training data and the labels corresponding to the training data; Based on the preset model target performance, the pre-trained classification model is subjected to structured pruning and unstructured pruning to obtain a pruned classification model; Get the pictures to be classified; Input the image to be classified into the pruned classification model to obtain the corresponding classification result; The structured pruning includes convolutional layer pruning and filter pruning; Accordingly, based on the preset model target performance, the pre-trained classification model is subjected to structured pruning and unstructured pruning respectively to obtain a pruned classification model, including: S1. Perform filter pruning sensitivity analysis on each filter of each convolutional layer in the pre-trained classification model to obtain the maximum convolutional layer pruning rate that meets the preset model target performance; S2. Determine whether the maximum convolutional layer pruning rate is greater than a preset convolutional layer pruning rate threshold; When the maximum convolutional layer pruning rate is greater than a preset convolutional layer pruning rate threshold, performing convolutional layer pruning on the convolutional layer corresponding to the maximum convolutional layer pruning rate; When the maximum convolutional layer pruning rate is not greater than a preset convolutional layer pruning rate threshold, performing filter pruning on the filter corresponding to the maximum convolutional layer pruning rate; S3, repeating S2 until all maximum convolutional layer pruning rates are exhausted, thereby obtaining the first pruned model; S4. Perform unstructured pruning on the first pruned model to obtain a second pruned model as a pruned classification model.

2. The classification method based on pruned convolutional neural network according to claim 1, characterized in that The filter pruning sensitivity analysis is performed on each filter of each convolutional layer in the pre-trained classification model to obtain the maximum convolutional layer pruning rate that meets the preset model target performance, including: Configuring a set of pruning rates to be analyzed for each convolutional layer; wherein the set of pruning rates to be analyzed includes multiple pruning rates to be analyzed; Calculate the model performance of each convolutional layer at each pruning rate to be analyzed in turn, and determine whether the calculated model performance meets the preset model target performance; When the calculated model performance meets the preset model target performance, the pruning rate to be analyzed with the largest value in each convolutional layer is used as the maximum convolutional layer pruning rate.

3. The classification method based on pruned convolutional neural network according to claim 1, characterized in that The non-structural pruning is convolution kernel pruning; Accordingly, the step S4 of performing unstructured pruning on the first pruned model to obtain a second pruned model as a pruned classification model includes: S41, normalizing all weights in the first pruned model to obtain normalized parameters; S42. Sort the normalized parameters according to their numerical values ​​to obtain a normalized parameter sequence; S43, using a predetermined weight pruning positioning method to determine the weight to be pruned from the normalized parameter sequence; S44: Determine whether the core pruning rate corresponding to the weight to be pruned is greater than a preset core pruning rate threshold. When the core pruning rate corresponding to the to-be-pruned weight is greater than a preset core pruning rate threshold, performing convolution kernel pruning on the convolution kernel corresponding to the core pruning rate; When the core pruning rate corresponding to the weight to be pruned is not greater than a preset core pruning rate threshold, performing weight pruning on the weight to be pruned; S45. Repeat S44 until all weights to be pruned are exhausted, thereby obtaining a second pruned model, and using the second pruned model as the pruned classification model.

4. The classification method based on pruned convolutional neural network according to claim 3, characterized in that Sorting the normalized parameters according to numerical values ​​is to sort the normalized parameters from small to large, thereby obtaining a normalized parameter sequence; Accordingly, the step S43 of determining the weights to be pruned from the normalized parameter sequence using a predetermined weight pruning positioning method includes: S431: Prune the first i normalized parameters in the normalized parameter sequence to obtain an initial second pruned model, where the initial value of i is 1 and is less than the total number of all normalized parameters in the normalized parameter sequence; S432: Determine whether the initial second pruned model meets the preset model target performance; S433. If the initial second pruned model meets the preset model target performance, i+1 is used as the new i, and S431 to S432 are repeated until the initial second pruned model does not meet the preset model target performance, and the weights corresponding to the first i normalized parameters in the normalized parameter sequence are determined as the weights to be pruned according to the new i.

5. The classification method based on pruned convolutional neural network according to any one of claims 1 to 4, characterized in that: The preset model target performance is that the performance degradation of the classification model after pruning does not exceed a performance threshold.

6. A classification device based on pruned convolutional neural network, characterized in that include: The model training module is used to obtain a pre-trained classification model based on the training data and the labels corresponding to the training data; A model pruning module is used to obtain a pruned classification model by performing structured pruning and unstructured pruning on the pre-trained classification model based on a preset model target performance; Image acquisition module, used to obtain images to be classified; A classification module is used to input the image to be classified into the pruned classification model to obtain the corresponding classification result; The structured pruning includes convolutional layer pruning and filter pruning; Accordingly, the model pruning module is specifically used to: S1. Perform filter pruning sensitivity analysis on each filter of each convolutional layer in the pre-trained classification model to obtain the maximum convolutional layer pruning rate that meets the preset model target performance; S2. Determine whether the maximum convolutional layer pruning rate is greater than a preset convolutional layer pruning rate threshold; When the maximum convolutional layer pruning rate is greater than a preset convolutional layer pruning rate threshold, performing convolutional layer pruning on the convolutional layer corresponding to the maximum convolutional layer pruning rate; When the maximum convolutional layer pruning rate is not greater than a preset convolutional layer pruning rate threshold, performing filter pruning on the filter corresponding to the maximum convolutional layer pruning rate; S3, repeating S2 until all maximum convolutional layer pruning rates are exhausted, thereby obtaining the first pruned model; S4. Perform unstructured pruning on the first pruned model to obtain a second pruned model as a pruned classification model.

7. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, it implements the classification method based on pruned convolutional neural network as described in any one of claims 1 to 5.

8. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the classification method based on pruned convolutional neural network as described in any one of claims 1 to 5 is implemented.

9. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the classification method based on pruned convolutional neural network as described in any one of claims 1 to 5 is implemented.

Images